23/11/2009

awstats web reporting and Domino extended common log format

Recently I took a look to awstats tool for web access reporting because I needed it for Domino 8.5 on a Linux Suse box .

I read the great post from Julian, but I also had to study the text log files generated from Domino because the log formats available in awstats seems not to match the format from Domino.
I tried awstats LogFormat=1 , and 6 and also an old custom format, but no way... too many dropped records reported by awstats

1) First I read the Domino Administrator 8.5 Infocenter to decode the meaning of a log text line generated from Domino: (extended common format, in my case...)

Depending on the file format you choose, the Access log file records the following Web server request information in the order shown:

Extended Common

  1. Client DNS name or IP address if DNS name is not available
  2. Host header from request, or server IP address if Host header is not available
  3. Remote user if available
  4. Request time stamp
  5. Http request line
  6. Http response status code
  7. Request content length if available, otherwise shows "-"
  8. Referring URL if available, otherwise shows "-"
  9. User agent if available, otherwise shows "-"
  10. Amount of time, in milliseconds, to process the request
  11. Value of the cookie header
  12. Translated URL, (the full path of the actual server resource, if available)

2) Later I arranged my own LogFormat format inside the .conf awstats configuration file:
LogFormat="%host %other %lognamequot %time1 %methodurl %code %bytesd %refererquot %uaquot %other %otherquot %otherquot"


Hope it helps somebody.


1 comment:

Inspi said...

Helpful indeed but I still have two questions that puzzle me.

First, how to retrieve the individual fields. I know that "variables" exist which can be used in the ExtraSectionFirstColumnValuesX field of the config file(URL, host, ...) but how can I retrieve the remote user name for example (for which there is not a variable that I know of)?

And second: the regular expressions seem strange to me. If I write ".*" (without quotes) as the second part in the ExtraSectionFirstColumnValuesX variable I would have expected to be returned the entire field, yet when I run the report I get an error. Any clues?