This report describes the policy and procedures used by the Oregon District to manage and store data collected during hydrologic investigations. It is the policy of the Oregon District that data collected to meet the objectives of projects for hydrologic investigations be documented, organized, and archived in a manner that (1) facilitates retrieval, evaluation, and use by District personnel and others; and (2) enables verification of data contained in all reports and computer data bases.
Procedures for the management of project data for hydrologic investigations are the following:
Scientific data collected by the U.S. Geo-logical Survey (USGS) is a valuable national resource that must be managed properly (Biesecker, 1991). Data-collection activities in the Water Resources Division (WRD) of the USGS commonly are divided into two principal programs: basic-data collection and hydrologic investigations. Basic-data programs provide routine, ongoing data from a nationwide system of stream-gaging stations, ground-water observation wells, and surface- and ground-water-quality sampling sites. These data usually are obtained to assess long-term trends and to establish base-line conditions for the Nation's water resources (USGS, 1984a). Basic-data programs of the USGS are recognized widely for their excellent data management. This excellence has resulted from the use of long-established procedures, training, and careful review. Most hydrologic investigations are in response to a recognized problem or an anticipated potential problem (USGS, 1984b). Data-collection activities of hydrologic investigations use a wide variety of procedures and usually are more site specific, more problem-oriented, and of shorter duration than those of basic-data programs. There often is a greater emphasis on understanding hydrologic systems than on data management and record keeping.
The Oregon District has developed a policy and procedures to improve data management for hydrologic investigations. The policy establishes standards for the management and storage of data collected for hydrologic investigations. The procedures delineate the specific activities required to achieve those standards.
The purpose of this report is to describe the policy and procedures used by the Oregon District to manage and store data collected as part of hydrologic investigations. The policy and procedures described herein apply to data collected for hydrologic investigations after Fiscal Year 1998. Hydrologic investigations include District projects having individual account numbers and "spin-off" or short-term projects funded under basic-data-collection account numbers. All data, published and unpublished, are covered by the policy and procedures. In general, data used in analysis and information related to data analysis or interpretations are covered by the policy and procedures. Additional examples are given in the section, "Data-Management Procedures."
Data collected for hydrologic investigations usually are documented, organized, and archived to meet the short-term needs of each project. More thought must be given to the needs of future projects, District-wide uses of the data, and the needs of other users. The problems of meeting only current project needs are readily apparent: lost data, greater effort required to retrieve data, incorrect well numbers and well names, duplicate or triplicate Site-File or water-quality entries, "lost" information (knowledge) when project personnel leave the District, difficulty in responding to information requests, and the inability to verify data stored in computer data bases or published in reports. Incomplete data in computer data bases and paper files limit the utility of the data.
Proper data management will eliminate many of the problems described in the preceding paragraph. Properly managed data will facilitate and improve the retrieval, evaluation, quality assurance, use, and storage of hydrologic and related data. As a result, data users can increase their productivity, and hydrologists and the public can have a higher degree of confidence in USGS data and reports.
It is the policy of the Oregon District that data collected to meet the objectives of projects for hydrologic investigations be documented, organized, and archived in a manner that (1) facilitates retrieval, evaluation, and use by District personnel and others, and (2) enables an independent verification of data contained in all reports and computer data bases.
Procedures for the management of data collected for hydrologic investigations are:
The Site File should contain important information on site identity, characteristics, and location. In the Oregon District, the NWIS data bases are plagued by duplicate Site-File entries for the same site, and data retrieval and use is limited by incomplete Site-File entries. To eliminate the creation of duplicate Site Files and improve the usability of the NWIS data bases in Oregon, four steps are required to establish a data-collection site: 1) determine preliminary site ID and station name, 2) research NWIS Site File to determine if site has already been establised, 3) enter Site File data, 4) add new site to Master GIS Site File.
The person requesting establishment of a new site also suggests a name for the site. The site name will be approved by the DBA who establishes the site. Surface water stations are assigned a descriptive name that identifes the surface water body and the nearest community that can be found on a topographic map (1:24,000 scale). The words, "AT", "NEAR", "ABOVE", or "BELOW" are to be used to describe the relation between the site and the community.
The system used for naming wells and springs in the Oregon District is based on the rectangular PLSS (Public Land Survey System) system for subdivision of public land. The State is divided into 36 square mile townships numbered according to their location relative to the east-west Willamette baseline and a north-south Willamette meridian. The position of a township is given by its north-south "Township" position relative to the baseline and its east-west "Range" position relative to the meridian. Each township is divided into 36 sections approximately one-square-mile, (640-acre) in area and numbered from 1 to 36. For example, a well designated as 01S/03E-33DCA is located in Township 1 south, Range 3 east, section 33. The letters following the section number correspond to the location within the section; the first letter identifies the quarter section (160 acres); the second letter identifies the quarter-quarter section (40 acres); and the third letter identifies the quarter-quarter-quarter section (10-acres). Thus, well 33DCA is located in the NE quarter of the SW quarter of the SE quarter of section 33. When more than one designated well occurs in the quarter-quarter-quarter section, a serial number is included. For wells that were named prior to approximately 1980 (?), a different system of letters following the section number was used for the location within the section. This system assigns a letter to one of 16 quarter-quarter sections (40 acres) that divide the section. The location 33DCA would correspond to the location 33Q. When more than one designated well occurs in the quarter-quarter section, a serial number is included.
Note that for any ground-water site, the person requesting the new site must determine the O regon Water Water Resources Department ID number that was assigned to the drillers report for the well. This number has the form, CCCCNNNNNN, where CCCC is a 4-character abbreviation for the county the driller reported the well to be located in and NNNNNN is a 6-digit sequential number assigned to the well. The OWRD-ID is a unique identifier that links the well to the OWRD Groundwater Resource Information Database (GRID) system. The OWRD-ID will be coded in NWIS component C213, cooperator ID (see table 2).
If a duplicate site is identified or suspected, the appropriate DBA (GWSI or ADAPS), the person requesting establishment of the new site, the Project Chief, and the Discipline Specialists (DS) must decide:
This is a group responsibility.
Old sites commonly are poorly located and the data are incomplete or uncertain. All NWIS data bases should be inventoried to identify existing data stored under a known or suspected duplicate site ID. The person requesting establishment of the new site and the appropriate DBA must attempt to verify the location of the old site and associated data by checking all USGS reports and old project paper files. Information about the site in question should be solicited from all District employees. If the old site and data cannot be verified and if a new site must be established, the value of retaining the old site and data needs to be determined. Factors to consider include the amount, age, source agency, types, and quality of the data.
If data are to be merged from an old site to a new site, or otherwise changed or combined, notice must be given to all District employees. If a decision is made to delete a site, the intention to do so must be announced to all District personnel by the appropriate DBA, and an appropriate time given for a response. If a site or data are deleted because of a lack of confidence in or an inability to verify the data, paper copies of the Site-File information and related NWIS data will be printed and filed by the old site ID in the District Site-File Archive. This is the responsibility of the appropriate DBA and the person requesting establishment of the new site.
Any changes to the site ID, the site name, or any data stored in the Site File must be noted in the computer file, "SITEFILE.CHANGES.LOG" that is maintained by the District NWIS DBA. This file contains the old site ID; the new site ID; the nature and date of the change; the person requesting the change; the reason for the change (for example, "DUPLICATE SITE OF USGS 42424087564301"); and what, if any, data were merged (sample dates, record numbers).
Water-quality data moved to a new site or otherwise combined should state the original site ID in the remarks field of the WATER-QUALITY RECORD HEADER. Annotations also should be made to the paper files. This is the responsibility of the Water-Quality System (QWDATA) DBA. Site-File information moved or combined should be noted in the Remarks Record of the MISCELLANEOUS FILE in GWSI. Annotations also should be made to the paper files. This is the responsibility of the appropriate DBA.
If an old site can be verified from reports, District Site File paper files, or any other means previously discussed, the Site File of the old site must be updated to the maximum extent possible. The source of the verification and the name of the person who made the verification should be entered in the Remarks Record of the MISCELLANEOUS FILE in GWSI. This is the responsibility of the appropriate DBA. Water-quality records that can be verified should be updated to the maximum extent possible. Updating must include the Water Data Storage and Retrieval System (WATSTORE) parameter code 00029 with the correct 5-digit project number. The source of the verification and the initials of the person who made the verification must be added to the remarks field of the WATER QUALITY RECORD HEADER. This is the responsibility of the QWDATA DBA.
Data collection usually requires the use of standardized field forms. The field form, which is usually formatted to meet the needs of the project, must reflect data-base requirements and the needs of other potential users of the data. The field form must include information pertinent to site location and condition, persons performing the field work, project number, reason the field work was performed, weather, instruments, procedures and methods, types of data collected, database parameter codes, and the times associated with various activities. Some examples include: Hydrolab1 1(Any use of trade, product, or firm names in this publicaito ndoes not iply endorsement by the U.S. Government unit number; time of last calibration; type of sampler; sampling method; beginning, end, and midpoint time of sample collection; type of pump used; purging criteria; location of measuring point; and chemical preservatives and their order of use. It is the responsibility of the Project Chief to develop appropriate field forms. To ensure that field forms meet minimum requirements, field forms must be approved by the DS in consultation with the DBA's.
While a project is active, the paper files for each data-collection site are
maintained in Site-File folders in the office of the Project Chief or the project staff. It is
the responsibility of the Project Chief to ensure that the Site-File folders contain all the
information necessary to verify data in a report or NWIS data base. Data to be stored in
Site-File folders include, if applicable:
Any changes in data (for example, water-quality reruns, incorrectly located sites, math errors in
levels or other calculations) are explained, dated, and initialed in the Site-File folders, and
appropriate updates are made to NWIS data bases. It is the responsibility of the Project Chief
to initiate changes to NWIS data bases, to provide the necessary data for updating to the DBA's,
and to document changes made to NWIS data bases in the Site-File folders. After a project report
has been approved for publication or (if no report is to be published) after a project has been
completed or terminated, the Site-File folders will be archived in the District Site File
archive.
Project data bases are defined as any data management and retrieval system, other than NWIS, used for storing data in digital form. Oregon District policy is that any project data for which there is an appropriate component in the NWIS data base, must be stored in the NWIS data base. These data may also be stored in project data bases, however, they must first be entered in the NWIS data base. Additionally, any modifcations to an NWIS data element for an existing site must first be made to the NWIS database. Project data bases may subsequently be updated by retrievals from the NWIS data base. The Project Chief is responsible for ensuring that project data in NWIS data bases are consistent with data stored in project data bases.
The current version of NWIS (3.1) allows the Site File and GWSI components to be queried directly using applications such as ARC-INFO, Arcview, MS Excel and others. The Oregon District is in the process of developing procedures for project staff to use to effieciently query their data in NWIS and, if needed, download it to other applications or project data bases. Future versions of NWIS will make it possible to query the ADAPS, QWDATA, and SWUDS files.
The following sections describe procedures for storing data in NWIS data bases and project data bases. Four of the sections include lists of information that represent the minimum data elements required by the Oregon District for establishing sites and storing data in NWIS data bases. All entries to the Site File and GWSI ground-water data files are coded on form 9-1904-A, revised February 1987 (THIS FORM IS BEING REVISED TO REFLECT RECENT CHANGES AND ADDITIONS TO THE NWIS DATA BASE). It is the responsibility of the Project Chief and staff to compile and verify the required information. It is the responsibility of the DBA's, in consultation with the DS, to review and update these requirements periodically.
DBA's may allow exceptions from one or more required data elements on the basis of the unavailability of the data. For example, a person requesting establishment of a site for an observation well has a driller's log for the well, but the log contains no information on the method or hours of development-two data elements required for establishing a ground-water site. The GWSI DBA may except those two data elements and allow the site to be established without that information. Exceptions from required data will be made by the appropriate DBA on a case-by-case basis.
Information required to be entered in the Site File for all data-collection sites is listed in table 1. Additional information required to be entered in the Site File for all ground-water sites is listed in table 2. All alphabetic information must be entered in CAPITAL letters. Each person requesting establishment of a site or storing data under an existing site must become familiar with these Site-File components (see Mathey, 1989, p. 2-8 to 2-38). It is the responsibility of the Project Chief to input and edit these data. It is the responsibility of the appropriate DBA to review and update these data.
Information required to be entered in the GWSI ground-water data files for all ground-water sites is listed in table 3. All alphabetic information must be entered in CAPITAL letters. Each person requesting establishment of a ground-water site or storing data under an existing ground-water site must become familiar with these components in GWSI (see Mathey, 1989, p. 2-39 to 2-121). It is the responsibility of the Project Chief or the project staff to input and edit these data. It is the responsibility of the GWSI DBA to review and update these data.
Information required to be entered in QWDATA for all water-quality samples is listed in table 4. These data can be obtained from a completed Analytical Services Request form from the National Water-Quality Laboratory. WATSTORE parameter codes required to be entered in QWDATA for all water-quality samples are listed in table 5. The quality-assurance remark code for all these parameter codes is "I" (USGS field value, in review). WATSTORE parameter codes required to be entered in QWDATA for all field alkalinity measurements are listed in table 6.
Each person submitting a water-quality sample must become familiar with the required informa-tion and WATSTORE parameter codes for storing data in QWDATA (see Maddy and others, 1989, p. 2-1 to 2-9 and appendixes A and B). It is the responsibility of the Project Chief to complete the Analytical Services Request form and to log-in each water-quality sample to QWDATA. It is the responsibility of the QWDATA DBA to review this data during or soon after sample login.
Under special circumstances, these data may be entered after a sample is logged in the computer. These circumstances include time required to analyze geologic and hydrologic data in order to determine geologic unit and aquifer type. The QWDATA DBA will not allow the Project Chief or staff access to the analytical results until all pertinent information has been entered in the computer. Project staff should be reminded of the need for prompt review of chemical analyses (especially nutrients). Any delay may compromise the validity and/or quality of the data.
Additional information required to be entered in the Site File for sites with data to be stored in ADAPS is listed in table 7. Each person storing data in ADAPS must become familiar with these Site-File components (see Mathey, 1989, p. 2-23, 2-24, and 2-38).
Also, the ADAPS DBA must be provided with the following information in order to input, edit, and process data in ADAPS. Additional details and explanations for the required information below are described in Dempster (1990, p. 3-1 to 3-36).
It is the responsibility of the Project Chief to provide the requisite data to the ADAPS DBA. It is the responsibility of the ADAPS DBA to review and input these data into ADAPS.
The water-use program is an ongoing effort to make Oregon water-use data available to water managers at the Federal, State, and local level. WHAT ELSE IS NEEDED HERE?
Laboratory printouts must receive timely review so that requests for reruns or verification of laboratory analyses can be performed. For example, nutrients must be rerun within a month from the date of collection. Discrepancies must be resolved, corrected if possible, and all actions documented in the Site-File folders and NWIS data bases. It is the responsibility of the Project Chief to ensure that all data have been correctly recorded on laboratory sheets; and correctly entered into Site-File folders, data bases, and reports. Every report, whether published and in the public domain or administratively released, will contain the site ID's for all data-collection sites used in that report. Tables of water data must include WATSTORE parameter codes. Authors are required to publish their data-not just summary statistics. Senior authors of reports must ensure that the data in their reports are correct and can be verified by data in Site-File folders and NWIS data bases. The senior author is responsible for compiling an errata sheet with corrections for all errors contained in a report.
At the conclusion of a project, the paper file for each data-collection site will be archived in the District Site File archive. Site-File folders will be filed by Federal Information Processing Standards (FIPS) county code, then by site ID within each particular FIPS county code. If a site has been used by two or more different projects, Site-File folders for each project are maintained in separate, but sequential, files. It is the responsibility of the Project Chief to archive the Site-File folders. It is the responsibility of the appropriate DBA to ensure that Site-File folders have been properly archived and to resolve any filing or site-information problems. Data-organization and verification efforts during the project will ensure more rapid, efficient, and quality-assured responses to information requests.
Also, at the conclusion of a project, the project files and data will be archived according to Oregon District guidelines (appendix A).
It is the responsibility of all personnel involved in the collection or compilation of hydrologic and related data to follow the procedures described in this document. It is the responsibility of the Section Chief(s) to ensure that Project Chiefs follow the procedures described in this document. Compliance with these procedures will be reviewed by the Section Chief(s) during quarterly project reviews and before the final project report is submitted for Director's approval. Section Chiefs should consult with DBA's and the DS during appraisal of the Data-Management Element of the employee's Performance Plan to help determine the degree to which an employee has complied with the District data-management procedures.
It is the responsibility of the DBA's to ensure that data added to the data base are complete and meet the requirements described in this document. It is the responsibility of the DBA's to ensure that new data-collection sites are established according to the procedures described in this document. It is the responsibility of the DS to review and approve field forms, assess the need for additional types of information that must be documented and stored, and-in consultation with the DBA's-periodically review and revise the procedures in this document. Additional responsibilities have been described in the "Data-Management Procedures" section.
Data collected for hydrologic investigations in the Oregon District often have been documented, organized, and archived to meet the short-term needs of individual projects. This approach has contributed to many data-related problems: lost data, incorrect site ID's and site names, duplicate or triplicate Site-File or water-quality entries, greater effort required to retrieve data and to respond to information requests by the public, and increased difficulty in verifying data stored in reports and in computer data bases. To address these problems, the Oregon District has adopted the policy that data collected for hydrologic investigations after Fiscal Year 1998 be documented, organized, and archived in a manner that (1) facilitates retrieval, evaluation, and use by District personnel and others; and (2) enables verification of data contained in all reports and computer data bases.
The following procedures have been established to meet the objectives of the District data-management policy:
Biesecker, J.E., 1991, The importance of data management: U.S. Geological Survey Information Systems Developments, v. 8, no. 6, p. 170-171.
Dempster, G.R., Jr., comp., 1990, National water information system user's manual, v. 2, chap. 3, version 91.2--automated data processing system: U.S. Geological Survey Open-File Report 90-116 [variously paged].
Maddy, D.V., Lopp, L.E., Jackson, L.D., Coupe, R.H., and Schertz, T.L., 1989, National water information system user's manual, v. 2, chap. 2, version 90.1--water-quality system: U.S. Geological Survey Open-File Report 89-617, 218 p.
Mathey, S.B, ed., 1989, National water information system user's manual, v. 2, chap. 4, version 90.1--ground-water site inventory system: U.S. Geological Survey Open-File Report 89-587, 242 p.
U.S. Geological Survey, 1984, Water-data program, Water Fact Sheet, 2 p.
U.S. Geological Survey, 1984, Federal-State Cooperative Water-Resources Program, Water Fact Sheet, 2 p.
Role |
Responsibility |
All DBAs |
1) User education
-- teaching users the correct methods for data and |
Surface-Water DBA
|
1) Approve preliminary
8-15 digit site ID given by person establishing site
|
Ground-Water DBA
|
1) Approve preliminary
ground water site name given by person |
Water-Quality DBA
|
1) Return to project
chiefs watlist returns from NWQL
|
NWIS DBA
|
1) Send update JCL's
to Reston biweekly for ADAPS, QWMENU and GWSI
|
Decodes
Administrator |
1) Write fortran programs
to decode all data transmitted from satellites
|
Water-Use DBA
|
1) Maintain all
water-use data collected by the district
|
Project Chief |
1) Ensure that new sites
are accurately field checked and plotted on |
Senior Authors |
1) Publish data and ensure
that data in report is correct and can be |
Section Chief |
1) Ensure that project chiefs follow procedures outlined in this document. |
Discipline Specialist |
1) Approve all
non-standardized field forms before project use
|
Address questions or comments to Charles Merk, CPAC Secretary