In Reply Refer To: June 3,1999
Mail Stop 415
OFFICE OF SURFACE WATER TECHNICAL MEMORANDUM NO. 99.07
SUBJECT: PUBLICATIONS--Addendum to Open-File Report "A Workbook for Preparing Surface Water Quality-Assurance Plans for Districts of the U.S. Geological Survey, Water Resources Division"
With the public's increasing access to the Internet, dissemination of rea l-time streamflow data has become one of WRD's most visible, sought after and critical products. Release of these data to the public within a few hours of their collection has presented the division with a host of new q uality assurance concerns. These concerns are not so much with the way th e issues are addressed as with the allowable timeframe in which they are accomplished. In the past, the required elements of surface-water record' s review were commonly performed within weeks or even months from the tim e the data were collected, but always prior to their public release, whic h was typically in the form of the District annual data report. A few co operators required, and were provided, provisional data prior to a full r eview; however, these agencies were well aware of the limitations of, and the caveats that were imposed on, our provisional records. With the diss emination of real-time streamflow data on the Internet, WRD's responsibil ity for ensuring data quality and alerting users to its limitations has i ncreased significantly. The following sections are offered as a template for use in the District's surface-water quality assurance plan to address the issue of quality assurance for real-time data. These sections are also available as part of the template " A Workbook for Preparing Surface W ater Quality-Assurance Plans for Districts of the U.S. Geological Survey, Water Resources Division"
Processing of Real-Time Streamflow Data
A necessary and critical element in maintaining accurate streamflow recor ds on a real-time basis is the need for rating analysis and shift applica tion as soon as practicable after a discharge measurement has been made. The [state the name] District's policy is that rating analyses and shift applications will be performed using the following procedures for data di sseminated on the District's public Web page [state Web page's URL].
[State the District's policy, which should include the maximum allowable time from when a discharge measurement is made at a real-time site to whe n the measurement notes are reviewed and the current shift applied in ADA PS. Depending on the site and situation, some Districts may require that shifts be called in immediately from the field and input by office staff. This may be required during floods if shifts are likely to have a signi ficant affect on peak flows and with special consideration given to sites co-located with National Weather Service (NWS) flood forecast points. Da ta from sites that are critical to water management agencies for their da ily operational requirements may also require more stringent measurement review and shift application procedures. During normal flow conditions an d for sites determined to be non-critical from a water-management standpo int, the District policy may require that measurement notes be reviewed a nd shift applications made within a specified number of workdays followin g completion of the field trip.
A related issue that should be addressed in the District's surface water quality assurance plan is the priorities assigned to the allocation of re sources during a flood. A District may find itself in a position where di scharge measurements are needed at specific locations to help adequately define the upper end of the rating. At the same time, a real-time equipme nt malfunction at a NWS forecast point site might make it necessary to di vert personnel from making the high-flow measurements to the site needing the equipment repairs. These types of priorities need to be carefully co nsidered and documented in the District's quality-assurance plan and floo d plan.]
Web Page Presentation Format
[Name of District] real-time data are served from computers located in [l ocation] maintained by the District. The National Water Information Syste m -Web (NWIS-W) software is used to conform to national USGS standards. [ Describe your District's Web pages (URL, etc.) that provide access to rea l-time data on the Internet and other pertinent information, including We b page links. District policy related to Web page development, review, ap proval and mechanisms for modification should be described.
Review of Real-Time Streamflow Data
Real-time streamflow data that are disseminated on the public Web page mu st be reviewed frequently to ensure their quality and to prevent the dist ribution of erroneous information. The [state the name] District utilizes both automated and manual review procedures to meet this objective.
Automated procedures that have been implemented by the [state the name] D istrict include the setting of minimum and maximum threshold values for s tage and discharge and their rates of change. If exceeded, these settings will initiate warnings of potential errors that will be delivered [by em ail or other system] to the appropriate District personnel. [Name or titl e of person] is the [state the name] District person designated to receiv e and act upon these messages. [Districts might also want to designate a backup person or persons to receive and respond to these messages in the event that the primary designee is away from the office or on leave. If t he District uses additional data screening techniques or processes, they should also be described.]
In addition to the automated procedures, WRD Technical Memorandum 97.17 r equires frequent and on-going screening and review of Web data, including the at least daily review of hydrographs during normal hours of operatio n. The [state the name] District also requires that all Web pages contain ing real- time streamflow data are reviewed regularly for accuracy and/or missing data. [State the District's policy on the required frequency of t his review. This will usually be done on a daily basis but the frequency might vary depending upon current flow conditions or for sites critical t o water management agencies or the NWS. Also state any policy the Distric t may have for monitoring the Web pages in other than normal working hour s. Identify the person or persons responsible for these reviews. In some cases it might be more efficient to have a single designee, especially if the number of real-time sites is small. For Districts with a large numbe r of sites it might be more practical to delegate this duty to more than one person. One possibility would be to have someone from each field offi ce responsible for reviewing the sites within their operational jurisdict ion.]
There are two general types of errors associated with streamflow data tha t are delivered by the real-time system and disseminated on the Internet. The first are persistent-type problems usually associated with some type of equipment failure whether in data collection or transmission, but cou ld also be related to ice effects. Because of the nature of the problem t hey generally occur on a continuing basis for more than a single recordin g interval. The second are the intermittent-type problems, which are oft en the result of a data transmission error. These often show up as either a zero or an unreasonably large value. [State the District policy on dea ling with these types of errors and missing data. In many cases, a single error resulting in a zero or large value of stage, and hence discharge, can be overlooked. However, often when plotted as a hydrograph, these err oneous values are accommodated by forcing the y-axis of the plot to be ex panded, rendering the other data indiscernible. If this occurs, state how it will be addressed. Who is responsible for making the decision to pull a station from the Web page? State the specific criteria used for making the decision.]
Data Qualification Statements
WRD Technical Memorandum 95.19 requires that streamflow data made availab le on the Web should be considered provisional until the formal review pr ocess has been completed. To ensure that everyone who accesses data from the Web are aware of this, data qualification statements must be included at key locations with a clickable heading "Provisional Data Subject to R evision" on all real-time data pages. [Describe the qualification stateme nts that are used by your District and identify the URL's where they are located. Districts may choose to identify various levels of record revisi on and review as they are completed. For instance, it may be desirable to indicate the date, time, and flow conditions (stage and discharge) of th e most recent discharge measurement. An indication of whether the measure ment had been checked and whether a shift was applied as a result of the measurement could also be included.]
Thomas H. Yorke
Chief, Office of Surface Water
Distribution: A, B, FO, PO