Producing and Using Web Page Statistics

Ever wonder how may times the web pages on the water server are requested? Wonder who requested them or what words were used in a search engine to generate the water server web pages as a result? All this information and more can be found on the web page usage statistics for the water server presented at http://water.usgs.gov/server_stats/. These statistics are prepared using the Analog program. Analog is very flexible in the statistics it can produce for web sites. It reads access logs, computes statistics, and prepares an html file ready to view on the web. This html file can also be customized for a presentation consistent with USGS Visual Identity specifications. To produce statistics for a site, the following may be of some assistance:

• Acquire and load the Analog software provided by DIS if your Web server is on a Sun system: http://wwwqvarsa.er.usgs.gov/unix/solaris/analog.html. This page provides links to documentation and other information about Analog.
• To produce statistics for your web pages similar to those produced for the water server, see the attached configuration file (water.cfg). Many of the lines in the configuration file are self-explanatory, but some will need to be customized for your server and specific web site as described below:
• Specify the path to and name of your access logs (LOGFILE). Analog assumes a default log format. If your format is different, you can tell Analog the exact format to read. See the documentation for more details.
• Specify your desired output file name and location (OUTFILE). Two sets of statistics pages are maintained for our pages, public and internal. The internal pages have more information and are stored in a restricted-access directory for usgs internal use (http://water.usgs.gov/usgs/server_stats).
• Enter the name of your host (HOSTNAME).
• Instead of the default Analog icon at the top of the page, we add the water banner using the LOGO entry. The path to your icon directory needs to be specified.
• A STYLESHEET is used for the water server statistics pages for consistency between pages on the server.
• Analog will read a HEADER and/or FOOTER file to add a navigation bar or additional information you may want to display.
• As explained in the Analog documentation, you can display statistics in many different formats. The water.cfg file example is for one month, and concentrates on the statistics for the weeks and days of that month (MONTHLY OFF, WEEKLY ON, FULLDAILY ON). Another option is for a longer time period, such as water-annual.cfg, which produces statistics for an entire year of log files. Notice that the MONTHLY report is turned ON in the annual configuration file so that each month's usage is displayed while FULLDAILY is turned OFF so that a report for each day in the year is not displayed.
• The public statistics pages have some options turned OFF, such as the GENERAL summary, the HOST report, and the FAILURE report. These can be set ON for internal pages.
• Specify which directories on your website to analyze using the FILEINCLUDE line. If you want all directories assessed, use /*, but to look at only one particular directory on your server, or to separate out the statistics for one directory, specify that directory on the FILEINCLUDE line.
• Similarly, you can exclude directories that are restricted or not of interest by using the FILEEXCLUDE line. As you can see in the water server configuration file, statistics for the water server are not assessed for the /usgs/ or the /cgi-bin/ directory.
• In a standard report, Analog provides request and referrer urls, and any subdirectories for the same url. The configuration for the water server includes commands to report only the main url, no subdirectories unless that subdirectory had more that 50,000 (user specified) entries (REFARGSFLOOR 50000r, REQARGSFLOOR 50000r).
• Specify a minimum or maximum number of lines to report for things such as hosts, pages requested, or referrers to your sites. The setting used on the water server configuration file is for the top 50 of each of these options. These lines are at the bottom of the configuration file (HOSTFLOOR -50r, REQFLOOR -50r, REFFLOOR -50r ).
• Output files are produced daily for the water server. Analog can be run on your access logs as often as needed and a new html file is created each time.
• If you customize your page appearance, it may be helpful to automate the process.