Internal USGS Access Only

Site Maintenance

Insure Link Integrity

• Use a link checker, such as Linklint (Windows, Unix, Solaris rpm, free), LinkScan, (Windows, Unix $300), or Xenu (Windows, free). The perl program webxref can be used to check links in a single html document
  webxref -one file.html
  webxref docmentation

  On the water.usgs.gov web server a batch job runs linklint once a week on the entire site.

• Customize the error documents; With the apache web server it is possible to customize the web pages that will be displayed for given error conditions (ex: 404 Not Found). By default, Apache as distributed by DIS in WRD web servers uses this feature, along with a server-side-include program which sends mail to the file owner about broken links.

• When local pages move, there are two ways to handle moved content on your site:

  • Add a Redirect entry in the web server configuration file. Users may never know that they were redirected. Link checking programs and search engines will see the return code of 300 and know that the old page no longer exists at the linked location. This can be done for entire directories.

  • Leave a referral document with a link to the new material and place and add a line like this in the HEAD section of the html:
    <META HTTP-EQUIV="Refresh"
    Content="6;
    URL=http://wwwqvarsa.er.usgs.gov/www/webmaster/">

    The code above will take you to http://wwwqvarsa.er.usgs.gov/www/webmaster/ after six seconds; It is a good idea to pick a time long enough to allow a user to use a back button once they have arrived, six seconds would be a bare minimum. This method encourages users to change links, but returns a normal return code and so won't be picked up by link checking programs as a redirection.

Insure Content Integrity

• "No abandoned pages!" Somebody should be responsible for every page. They should care that the content is current and relevant, fix it when it is not, and remove the page when obsolete. Write-and-forget is not acceptable.
• Directory and filenames should reflect content organization
• Link to existing information, don't copy it. Possible exception: If you are using a document as an important source you might consider copying it to your server as an archive only to use should the reference disappear in the future.

Web Server Log Files

• These are a hidden resource which is under-utilized. Regular log analysis will show what pages are popular and which ones not. See who your customers are and where they come from. The free program analog (Windows, Unix, Solaris rpm) can create an incredible variety of reports from web server access logs. Sample Web statistics.
• USGS WWW Guidelines state that log files should be maintained for a minimum of 6 months.
• NatWeb is providing usage statistics for each virtual host and working on cumulative statistics for all sites being hosted. We'd like to be able to say, for example: "60 million pages were served by USGS last month."
• Scan error logs on a regular basis to find broken links, CGI script failures, and access problems.

There are Internet services that specialize in helping sites track down problems, such as Net Mechanic and Doctor HTML. Keynote Performance Monitoring is being used to monitor performance and availability.

slide 50

"Mastering a Web Site" online course Created and maintained by Lorna Schmid and David Boldt. http://water.usgs.gov/usgs/training/webmaster/maintenance.html     Last modified: Tue Sep 20 12:19:12 EDT 2005