Google


AWStats logfile analyzer 5.3 Documentation

  


Install, Setup and Use AWStats

AWStats common use is made in 3 steps:
  • Step 1 : The install and setup
  • Step 2 : The build/update of statistics
  • Step 3 : The reading of results


Step 1 : Install and Setup
A) With Apache Server (on Unix/Linux, Windows, MacOS...)

* Step 1-1
Configure your apache web server to have NCSA combined/XLF/ELF log format (you can use your own log format but this predefined logformat is often the best choice and make setup easier). You can do this by changing, in httpd.conf, following directives (See your apache manual for more information):
CustomLog /yourlogpath/yourlogfile common
into
CustomLog /yourlogpath/yourlogfile combined

To be sure the log format change is effective, you can stop Apache, remove all old log files, restart Apache and go to your homepage. This is an example of records you should get then in your new log file:
62.161.78.75 - - [dd/mmm/yyyy:hh:mm:ss +0000] "GET / HTTP/1.1" 200 1234 "http://www.from.com/from.html" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"

* Step 1-2
Copy the contents of the uncompressed cgi-bin folder from your hard drive to your server's cgi-bin directory (this includes awstats.pl, awstats.model.conf, and the lang, lib and plugins sub-directories).

* Step 1-3
If necessary (should not with most Perl interpreter), edit the first (top-most) line of awstats.pl file that is
#!/usr/bin/perl
to reflect the path were your Perl interpreter is installed. Default value works for most of Unix OS, but it also might be
#!/usr/local/bin/perl
With Apache for Windows and ActivePerl interpreter, it might be
#!c:/program files/activeperl/bin/perl

* Step 1-4
Move AWStats icon sub-directories and its content into a directory readable by your web server, for example /yourwwwroot/icon or /yourwwwroot/icons.

* Step 1-5
Copy awstats.model.conf file into a new file named awstats.myvirtualhostname.conf. This new file must be stored in
- /etc/opt/awstats or /etc/awstats or /etc or same directory than awstats.pl (so cgi-bin) for Unix/Linux users.
- same directory than awstats.pl (so cgi-bin) for Windows and other OS.

* Step 1-6
Edit this new config file with your own setup :
- Change LogFile value with full path of your web server log file (You can also use a relative path from your awstats.pl directory).
- Check if LogFormat has the value "1" (it means "NCSA apache combined/ELF/XLF log format").
- Change DirIcons parameter to reflect relative path of icon directory.
- Edit SiteDomain parameter with the main domain name or the intranet web server name used to reach the web site to analyze (Example: www.mydomain.com).
- You can change other parameters if you want.

Step 1 (Install and Setup) is finished. You can jump to the Build/Update Statistics section.



B) With IIS Server

* Step 1-1
Configure IIS to log in "Extended W3C log format" (You can still use your own log format but setup is easier if made like suggested). So, for this, start the IIS Snap-in, select the web site and look at its Properties. Choose W3C Extended Log Format, then Properties, then the Tab Extended Properties and uncheck everything under Extended Properties. Once they are all unchecked, check all following fields:
date
time
c-ip
cs-username
cs-method
cs-uri-stem
sc-status
sc-bytes
cs-version
cs(User-Agent)
cs(Referer)
 To be sure the log format change is effective, you must stop IIS, remove all old log files, restart IIS and go to your homepage. This is an example of records you should get then in the new log file:
2000-07-19 14:14:14 62.161.78.73 - GET / 200 1234 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.htm

* Step 1-2
Copy the contents of the uncompressed cgi-bin folder from your hard drive to your server's cgi-bin directory (this includes awstats.pl, awstats.model.conf, and the lang, lib and plugins sub-directories).

* Step 1-3
Move AWStats icon sub-directories and its content into a directory readable by your web server, for example C:\yourwwwroot\icon.

* Step 1-4
Copy awstats.model.conf file into a new file named awstats.myvirtualhostname.conf. This new file must be stored in
- same directory than awstats.pl (so cgi-bin)

* Step 1-5
Edit this new config file with your own setup :
- Change LogFile value with full path of your web server log file (You can also use a relative path from your awstats.pl directory).
- Change LogFormat to value "2" (it means "IIS Extended W3C log format").
- Change DirIcons parameter to reflect relative path of icon directory.
- Edit SiteDomain parameter with the main domain name or the intranet web server name used to reach the web site to analyze (Example: www.mydomain.com).
- You can change other parameters if you want.

Step 1 (Install and Setup) is finished. You can jump to the Build/Update Statistics section.



C) With other web servers

Setup process is similar to setup for Apache or IIS.
Use LogFormat to value "3" if you have WebStar native log format.



Step 2 : Build/Update Statistics
* Step 2-1
The first analyze/update of statistics can be made the first time manually from the command line (the first time, process may be long) :
awstats.pl -config=myvirtualhostname -update

AWStats will read the config file awstats.myvirtualhostname.conf (or if not found, awstats.conf) and create/update its database with all summary information issued from analyzed log file.
AWStats database files are saved in directory defined by DirData parameter in config file.
When update is finished, you should get a result like this:
Lines in file: 225730
Found 5 dropped records,
Found 124 corrupted records,
Found 0 old records,
Found 225601 new records.

Dropped records are records discarded because they were not user HTTP request or requests were not qualified by AWStats filters (See SkipHosts, SkipUserAgents, SkipFiles, and OnlyFiles parameters). If you want to see which lines were dropped, you can add the -showdropped option on command line.
Corrupted records are records that does not match log format defined by "LogFormat" parameter in AWStats config/domain file. With all webservers you can experience a little bit corrupted records (<5%) even when everythings work correctly. This can be the result of several reasons: Web server internal bugs, bad requests made by buggy browsers, web server brutal stop...
If all your lines are corrupted and LogFormat parameter in AWStats config/domain file is correct, it may be the log format setup in your web server that is wrong. Don't forget that your LogFormat parameter in AWStats config/domain file MUST match the log file format you analyze.
If you want to see which lines are corrupted, you can add the -showcorrupted option on command line.
Old records are simply records that were already processed by a previous update process. So it's not necessary to purge your log file after each update process even if it's highly recommended to do it as often as possible.
New records are records in your log file that were successfully used to build/update statistics.

Note : A log analysis process is slow (one second for each 4500 lines of your logfile with Athlon 1Ghz, plus DNS resolution time for each different IP address in your logfile if DNSLookup is set to 1 and not already done in your log file).
See Benchmark page for more accurate information.

* Step 2-2
Even if AWStats allows "real-time" statistics with its "update from browser feature" (See next section Read Statistics), you should run an update process from a scheduler (command is same than first process) frequently.

You can add instructions in your crontab (Unix/Linux) or your task scheduler (for Windows), to launch frequently this Awstats update process.
For sites with:
- 10,000 visitors a month Launch AWStats once a day
- 50,000 visitors a month Launch AWStats once every 4 hours
- 250,000 visitors a month Launch AWStats once an hour
- 1,000,000 visitors a month Launch AWStats once an hour
This is ABSOLUTELY necessary to keep good performances.
See AWStats Benchmark page for more accurate information.

!!! Warning, if you don't use (or can't use with IIS) the PurgeLogFile parameter, it's very important that you don't forget to purge/rotate your log file yourself (or setup your web server to do it) frequently (You can find help for this on FAQ-SET550). Even if AWStats never analyzes twice the same log record, the more often you clean your log file, the faster AWStats will be.



Step 3 : Read Statistics
To see results of analyze, you have several solutions depending on your security policy.

* First solution is to build the main reports, in a static HTML page, from the command line, like this :
perl awstats.pl -config=virtualhostname -output -staticlinks > awstats.virtualhostname.html

You can also use all other output options (each of them give you another report). This is how to use all other possible output options(1) :
perl awstats.pl -config=virtualhostname -output=allhosts -staticlinks > awstats.virtualhostname.allhosts.html
perl awstats.pl -config=virtualhostname -output=lasthosts -staticlinks > awstats.virtualhostname.lasthosts.html
perl awstats.pl -config=virtualhostname -output=unknownip -staticlinks > awstats.virtualhostname.unknownip.html
perl awstats.pl -config=virtualhostname -output=alllogins -staticlinks > awstats.virtualhostname.alllogins.html
perl awstats.pl -config=virtualhostname -output=lastlogins -staticlinks > awstats.virtualhostname.lastlogins.html
perl awstats.pl -config=virtualhostname -output=allrobots -staticlinks > awstats.virtualhostname.allrobots.html
perl awstats.pl -config=virtualhostname -output=lastrobots -staticlinks > awstats.virtualhostname.lastrobots.html
perl awstats.pl -config=virtualhostname -output=urldetail -staticlinks > awstats.virtualhostname.urldetail.html
perl awstats.pl -config=virtualhostname -output=urlentry -staticlinks > awstats.virtualhostname.urlentry.html
perl awstats.pl -config=virtualhostname -output=urlexit -staticlinks > awstats.virtualhostname.urlexit.html
perl awstats.pl -config=virtualhostname -output=browserdetail -staticlinks > awstats.virtualhostname.browserdetail.html
perl awstats.pl -config=virtualhostname -output=unknownbrowsers -staticlinks > awstats.virtualhostname.unknownbrowsers.html
perl awstats.pl -config=virtualhostname -output=unknownos -staticlinks > awstats.virtualhostname.unknownos.html
perl awstats.pl -config=virtualhostname -output=refererse -staticlinks > awstats.virtualhostname.refererse.html
perl awstats.pl -config=virtualhostname -output=refererpages -staticlinks > awstats.virtualhostname.refererpages.html
perl awstats.pl -config=virtualhostname -output=keyphrases -staticlinks > awstats.virtualhostname.keyphrases.html
perl awstats.pl -config=virtualhostname -output=keywords -staticlinks > awstats.virtualhostname.keywords.html
perl awstats.pl -config=virtualhostname -output=errors404 -staticlinks > awstats.virtualhostname.errors404.html

Note (1): If you prefer, you can use awstats_buildstaticpages tool to build all those pages in one command.

Note (2): You can also add a filter on the following output reports:urldetail, urlentry, urlexit, allhosts, refererpages.
filter can be a regexp on the full key you want awstats to present information about and you must use it after the output parameter separated by a ":".
For example, to output the urldetail report, with a filter on all pages that contains /news, you can use the following command line:
perl awstats.pl -config=virtualhostname -output=urldetail:/news -staticlinks > awstats.virtualhostname.urldetailwithfilter.html

Note (3): If you want to build a report for a particular month, add options -month=MM -year=YYYY.
To build a report for full year (warning: This may use a lot of memory and CPU), add options -month=year -year=YYYY.


* Another solution is to view dynamically your statistics from a browser. For this use URL:
http://www.myserver.mydomain/cgi-bin/awstats.pl?config=virtualhostname
where virtualhostname is used to know which config file to use (AWStats will use awstats.virtualhostname.conf file).

Note (1): All output command line options (except -staticlinks) are still available when using AWStats as a browser.
Just use them as URL parameters like this example http://www.myserver.mydomain/cgi-bin/awstats.pl?month=MM&year=YYYY&output=unknownos

Note (2): If AllowToUpdateStatsFromBrowser parameter is set to 1 in AWStats config/domain file, you will also be able to run the update process from your browser. Just click on link "Update now".