In Reply Refer To:
Mail Stop 412
June 26, 2007
Office of Water Quality Technical Memorandum 2007.03
Subject: Changes to the NWISWeb Aggregation and Display of Water-Quality Data
Major changes are forthcoming to the National Water Information System (NWIS) processes for compilation and display of water-quality data on the NWISWeb system. These changes will be implemented by stages during 2007. This memorandum describes and explains policy changes that underpin the changes to NWISWeb and describes possible changes to workflows in the U.S. Geological Survey (USGS) Water-Science Centers (WSCs).
Previously, water-quality data were aggregated annually for NWISWeb using a manual process that retrieved a limited set of sample- and result-level attributes from the NWIS QWDATA system. Annual aggregations occurred after the USGS annual Water Data Reports were published and only data from the prior and previous years were displayed on NWISWeb. The previous annual updates to NWISWeb allowed WSCs the opportunity to review data before transfer to NWISWeb. This review was needed because data quality could not be adequately documented on NWISWeb due to the limited water-quality data attributes available.
The new process will aggregate water-quality data automatically, on a daily or more frequent basis, and will include a more complete set of data attributes. This process will display reviewed as well as provisional water-quality results. New NWISWeb enhancements include display of data-quality attributes such as the data-quality indicator code, value qualifiers, reporting limits, and other important supporting sample and result information.
Sites, samples, and results designated as proprietary (using site-level codes, sample-level analysis status codes, or result-level data-quality indicators (DQI) codes); samples that are marked for internal-use only (using the sample-level processing status code); and results from unapproved and research methods (as designated by DQI code) will not be displayed by NWISWeb. This continues the previous policy for release of data to the public. WSCs must have approval from their Regional Hydrologist to designate data as proprietary, and the designation should generally be removed after a predetermined interval (WRD Policy Memo 99.07). Other guidance on the release of sensitive water resources information can be found at http://wwwnwis.er.usgs.gov/communications/2005news/050712sensitive_data_guidelines.html.
During the first stage of implementation, all appropriate (as described above) NWIS water-quality data, as of the retrieval date, will be compiled and displayed on NWISWeb. During the second stage, software will be deployed that will transfer incremental changes for NWIS water-quality data to NWISWeb.
These changes in data processing are motivated by the needs of our federal, State, and local customers. The changes are part of continuing improvements in the processes for providing relevant scientific information to the Nation and for collaborating with others in the scientific community. These changes will support the USGS’ on-going effort to share data with the U.S. Environmental Protection Agency (USEPA) and provide timely information to decision makers during emergencies, such as natural disasters and pollution incidents. (Data custodians are reminded to consult Office of Water Quality Technical Memorandum 2002.15 [http://phoenix.cr.usgs.gov/policy/OWQ_02.15_dqi.html] for options for using DQIs to flag data-quality issues in the QWDATA database.)
A review of the data-management workflow in WSCs may be needed as a result of these policy and process changes: (1) Data-entry personnel should be careful to use the correct qualifiers during sample login, especially for proprietary and internal-use-only samples (see NWIS guidance for information on when data should be flagged as proprietary-- http://wwwnwis.er.usgs.gov/SECURITY/Privacy/Proprietary.data.Policy.html); (2) More timely review of analytical results may be needed, since provisional data will be supplied to internet users; (3) Project personnel must use DQI codes in their local NWIS-QWDATA database to document the review and web transfer status of results. Guidance and training will be provided to WSCs to assist in understanding potential workflow changes caused by the new NWISWeb water-quality aggregation and web displays.
WSCs may experience changes in the types of questions received from users of NWISWeb. Previously, requests have predominantly been for data; however, questions in the future may regard method and data-quality. USGS WSC directors may want to notify their cooperators about these changes after they are implemented on NWISWeb. The following paragraph is suggested for this purpose:
The USGS is expediting the delivery of new water-quality data to our NWISWeb system to provide data in the timeliest manner possible. As a result, some provisional, unreviewed water-quality results will become available. There are several aspects to this change in data delivery that may help you understand and use these data. Analytical results from any particular sampling activity are commonly returned by the laboratory in groups of results that are associated with particular laboratory processes. For example, field measurements and nutrient analyses typically are delivered first and organic analyses are usually last. About 60 percent of samples collected by the USGS have additional analytes added subsequent to the initial data release. The review and approval of laboratory results by the USGS may not be final until all analytical results for a particular sample have been supplied by the laboratory. The USGS is attempting to meet your needs for rapid data disclosure by providing data when still provisional. The USGS places a very high value on data quality. If you find anomalous or questionable data on our NWISWeb system, we will attempt to resolve the data issue. We advise caution regarding the potential for use of unreviewed data in policy decisions and we welcome your questions. Please contact the appropriate USGS Water-Science Center with any questions about data collected in their area.
Walton H. Low /s/ for Timothy L. Miller
Chief, Office of Water Quality
This memorandum does not supersede any other Office of Water Quality Technical Memorandum.
Distribution: All WRD Employees