Return-Path: rmusson@srv1rvares.er.usgs.gov Received: from mailrvares.er.usgs.gov (srv1rvares.er.usgs.gov [130.11.51.209]) by id02didbse.wr.usgs.GOV (Geomail 1.2.3) with ESMTP id KAA02518 for ; Tue, 1 Jul 1997 10:59:43 -0600 Received: from [130.11.51.244] (miller-mac.er.usgs.gov [130.11.51.244]) by mailrvares.er.usgs.gov (Geomail 1.2.3) with SMTP id NAA14617; Tue, 1 Jul 1997 13:00:23 -0400 Message-Id: <199707011700.NAA14617@mailrvares.er.usgs.gov> Date: Tue, 1 Jul 1997 13:02:12 -0500 To: nawqa91pc@srv1rvares.er.usgs.gov From: "Timothy L Miller, Chief, NAWQA, Reston, VA" (Timothy L. Miller) X-Sender: tlmiller@130.11.51.209 Subject: NWIS data Re-aggregation and SW GIS Cc: "Jeffrey T Armbruster, Asst. Reg. Hydr.-NAWQA, Norcross, GA" , "Ivan C James II, Asst. Reg. Hydrologist-NAWQA, Denver, CO" , "Michael C Yurewicz, Asst. Reg. Hydrol.-NAWQA, Reston, VA" , "Marc A Sylvester, Asst. Reg. Hydrologist-NAWQA, Menlo Park, CA" , "William G Wilber, National Synthesis Coordinator, Reston, VA" , "Robert S Williams Jr., Hydrologist, Denver, CO" , "Dennis R Helsel, Hydrologist, Denver, CO" , "Martin E Gurtz, Hydrologist, Raleigh, NC" , "Robert J Gilliom, Supv. Hydrologist, Sacramento, CA" , "John S Zogorski, Supervisory Hydrologist, Rapid City, SD" , "O. Lehn Franke, Hydrologist, Denver, CO" , "David K Mueller, Hydrologist, Denver, CO" , "Larry J Puckett, Ecologist, Reston, VA" , "Jonathon C Scott, Hydrologist, Oklahoma City, OK" , "Jeffrey D Stoner, Supervisory Hydrologist, Mounds View, MN" , "Sarahnell L Laible, Secretary, Reston, VA" , "Rosemary Musson, Office Automation Assistant, Reston, VA" , "Theresa A Carr, Office Automation Asst., Reston, VA" In Reply Refer To: Mail Stop 413 SENT VIA EMAIL MEMORANDUM July 1, 1997 TO: Study-Unit Chiefs (1991), National Water-Quality Assessment Program FROM: Timothy L. Miller Chief, National Water-Quality Assessment Program SUBJECT: Corrections and Re-aggregation of NWIS Data and SW GIS REPLY REQUESTED BY JULY 31, 1997 BACKGROUND AND OVERVIEW During the past 2 months, National Synthesis teams have been working on a comprehensive review of the national NAWQA database in preparation for finalizing data from the first NAWQA cycle for study-unit summary reports, national synthesis data analysis, and public data release. The scope of review and the number and types of issues encountered with water quality, ancillary, and GIS data have caused this effort to take twice the time originally planned. This has delayed the timeline for finalizing the database by about 1 month so far, and also means that there will be a substantial effort required in most study units to help correct and finalize the database. The NAWQA Leadership Team (NLT) has considered a range of options for dealing with this situation, ranging from accepting the data as they are and staying on the fastest track possible, to slowing the process down enough to accommodate a careful effort to correct all problems. Because this is the last chance to address data issues while the principal investigators are still involved, because this will be the final aggregated database for the first cycle of NAWQA, and because many of our most widely distributed products will be based upon these data; we have decided to make the final database as complete and error free as possible -- and accept the time delay it will cause. Another main purpose of the re-aggregation is to incorporate corrected data that the National Water Quality Laboratory (NWQL) reloaded to district NWIS files during late May and early June. You have probably heard of BAD.QW entries caused by this reload. This NWQL reload, which was completed by June 6, was described in a email from Pete Rogerson on May 21. It affects some of the results for the following parameters (pcodes): pentachloranisole(49460), hexachlorobenzene(49343), lanthanum(34855), europium(34885), uranium(35000), thorium(34980), barium(34805), and tin(34985). Pete's email contained instructions for districts to handle some deletions for 1,3-Dichlorobenzene and 2-Chlorophenol. In addition, new values for radon(82303) were supplied by Kevin Dennehy for districts to change in NWIS. The decision to provide the best possible national database will probably delay the original timeframe by two months for finalizing national data analysis for the summary reports (see revised timeline below). The effort to complete and re-aggregate study unit data sets may require an average of 4-6 person weeks in each study unit, based on experience with pilot efforts. Each study unit will receive a series of database reports and associated requests for checking and revision from national data managers. The primary elements are listed below, although some will be combined as the timing allows: All study units: 1. SW quality data -- checking and resolving how to combine multiple records for the same water sample, verifying national synthesis assignment of sample-collection purpose codes, and verifying extreme concentrations. 2. GW quality data and GWSI -- checking and resolving how to combine multiple records for the same water sample, verifying national synthesis assignment of sample-collection purpose codes, verifying extreme concentrations, and selective verification and updating of GWSI information. Some study units: 3. SW drainage basins -- checking discrepancies between NWIS basin areas and areas derived from GIS basins and providing boundaries for contributing basin in addition to total basin. 4. GW study boundaries -- resolve questions about the derivation of boundaries, especially land use studies, update as needed, and complete documentation so that source materials are cited and study designs could be reproduced. There are also two additional data aggregations: QC data, which is a significant effort, requiring manual entry of missing spike lot numbers and manual selection of data to be included; and continuous temperature (and optionally conductivity) data which will be a simple downloading of available data from Hobo and/or ADAPS files. REVISED SCHEDULE TASKS DUE DATES Matrix view(nonlinear people): NWIS NWIS QC Chems. &Temp. SW GIS GW GIS 1 Pilot guidance tested 6-13 7-15 2 Guidance to SUs on what is requested 6-30 7-31 6-30 7-31 3 SUs respond with data/fixes 7-31 8-30 7-31 8-30 4 Data Aggregated by Jon Scott 9-01 5 Occurrence tables prepared 9-15 Same stuff sorted by due date(linear people): NatSyn data question lists to SUs 6-30 NatSyn submits SW GIS fix needs-4 SUs 6-30 QC & temp. pilot guidance out 7-15 4 SUs respond to SW GIS fix 7-31 NatSyn submits GW GIS fix needs 7-31 SU's correct data & re-submit 7-31 QC & temp. final guidance out 7-31 SUs submit QC & temp. 8-30 4 SUs respond to GW GIS fix 8-30 Jon re-aggregates data 9-01 Occurrence tables prepared 9-15 We recognize that the revised schedule means that costs for some items in producing the study unit summary reports (e.g. printing, finalizing occurrence table and national comparison pages) will be pushed into FY98, even for SUs that will have their summary reports approved in FY97. Please keep your Assistant Regional Hydrologist for NAWQA informed on progress and problems, and they will assist study units in meeting their objectives. PRIORITIES TO BALANCE The NLT recognizes that completing all of the assigned responsibilities will be very difficult for some study units who have lost key staff to other locations or projects. Based on this consideration, the NLT has decided on the following priorities regarding national data aggregation, information archiving, and FY91 study unit reports for the remainder of FY97 and for FY98: 1. Corrections and completion of required fields in the SU chemical data and re-aggregation for the integrity of the national data base. 2. Corrections and updates related to SW and GW GIS data. 3. Completion of the summary report. 3. SU contribution to QC and temperature aggregation. 4. Archiving of critical SU data not in NAWQA national data sets (see NAWQA archive plan guidance--URL below). 5. finishing other SU reports (probably no more than 1 or 2 per FY during the LIP phase). HOW TO GET STUFF Study unit review packages can be fetched via anonymous ftp from: host: mbdokokl.cr.usgs.gov file: ./nawqa/review.SUID.tgz (where SUID=3Dstudy-unit id) URL for 'How to Archive NAWQA Data': http://csdokokl.cr.usgs.gov/nawqa/train/arch/report.html Return-Path: rmusson@srv1rvares.er.usgs.gov Received: from mailrvares.er.usgs.gov (srv1rvares.er.usgs.gov [130.11.51.209]) by id02didbse.wr.usgs.GOV (Geomail 1.2.3) with ESMTP id KAA02518 for ; Tue, 1 Jul 1997 10:59:43 -0600 Received: from [130.11.51.244] (miller-mac.er.usgs.gov [130.11.51.244]) by mailrvares.er.usgs.gov (Geomail 1.2.3) with SMTP id NAA14617; Tue, 1 Jul 1997 13:00:23 -0400 Message-Id: <199707011700.NAA14617@mailrvares.er.usgs.gov> Date: Tue, 1 Jul 1997 13:02:12 -0500 To: nawqa91pc@srv1rvares.er.usgs.gov From: "Timothy L Miller, Chief, NAWQA, Reston, VA" (Timothy L. Miller) X-Sender: tlmiller@130.11.51.209 Subject: NWIS data Re-aggregation and SW GIS Cc: "Jeffrey T Armbruster, Asst. Reg. Hydr.-NAWQA, Norcross, GA" , "Ivan C James II, Asst. Reg. Hydrologist-NAWQA, Denver, CO" , "Michael C Yurewicz, Asst. Reg. Hydrol.-NAWQA, Reston, VA" , "Marc A Sylvester, Asst. Reg. Hydrologist-NAWQA, Menlo Park, CA" , "William G Wilber, National Synthesis Coordinator, Reston, VA" , "Robert S Williams Jr., Hydrologist, Denver, CO" , "Dennis R Helsel, Hydrologist, Denver, CO" , "Martin E Gurtz, Hydrologist, Raleigh, NC" , "Robert J Gilliom, Supv. Hydrologist, Sacramento, CA" , "John S Zogorski, Supervisory Hydrologist, Rapid City, SD" , "O. Lehn Franke, Hydrologist, Denver, CO" , "David K Mueller, Hydrologist, Denver, CO" , "Larry J Puckett, Ecologist, Reston, VA" , "Jonathon C Scott, Hydrologist, Oklahoma City, OK" , "Jeffrey D Stoner, Supervisory Hydrologist, Mounds View, MN" , "Sarahnell L Laible, Secretary, Reston, VA" , "Rosemary Musson, Office Automation Assistant, Reston, VA" , "Theresa A Carr, Office Automation Asst., Reston, VA" In Reply Refer To: Mail Stop 413 SENT VIA EMAIL MEMORANDUM July 1, 1997 TO: Study-Unit Chiefs (1991), National Water-Quality Assessment Program FROM: Timothy L. Miller Chief, National Water-Quality Assessment Program SUBJECT: Corrections and Re-aggregation of NWIS Data and SW GIS REPLY REQUESTED BY JULY 31, 1997 BACKGROUND AND OVERVIEW During the past 2 months, National Synthesis teams have been working on a comprehensive review of the national NAWQA database in preparation for finalizing data from the first NAWQA cycle for study-unit summary reports, national synthesis data analysis, and public data release. The scope of review and the number and types of issues encountered with water quality, ancillary, and GIS data have caused this effort to take twice the time originally planned. This has delayed the timeline for finalizing the database by about 1 month so far, and also means that there will be a substantial effort required in most study units to help correct and finalize the database. The NAWQA Leadership Team (NLT) has considered a range of options for dealing with this situation, ranging from accepting the data as they are and staying on the fastest track possible, to slowing the process down enough to accommodate a careful effort to correct all problems. Because this is the last chance to address data issues while the principal investigators are still involved, because this will be the final aggregated database for the first cycle of NAWQA, and because many of our most widely distributed products will be based upon these data; we have decided to make the final database as complete and error free as possible -- and accept the time delay it will cause. Another main purpose of the re-aggregation is to incorporate corrected data that the National Water Quality Laboratory (NWQL) reloaded to district NWIS files during late May and early June. You have probably heard of BAD.QW entries caused by this reload. This NWQL reload, which was completed by June 6, was described in a email from Pete Rogerson on May 21. It affects some of the results for the following parameters (pcodes): pentachloranisole(49460), hexachlorobenzene(49343), lanthanum(34855), europium(34885), uranium(35000), thorium(34980), barium(34805), and tin(34985). Pete's email contained instructions for districts to handle some deletions for 1,3-Dichlorobenzene and 2-Chlorophenol. In addition, new values for radon(82303) were supplied by Kevin Dennehy for districts to change in NWIS. The decision to provide the best possible national database will probably delay the original timeframe by two months for finalizing national data analysis for the summary reports (see revised timeline below). The effort to complete and re-aggregate study unit data sets may require an average of 4-6 person weeks in each study unit, based on experience with pilot efforts. Each study unit will receive a series of database reports and associated requests for checking and revision from national data managers. The primary elements are listed below, although some will be combined as the timing allows: All study units: 1. SW quality data -- checking and resolving how to combine multiple records for the same water sample, verifying national synthesis assignment of sample-collection purpose codes, and verifying extreme concentrations. 2. GW quality data and GWSI -- checking and resolving how to combine multiple records for the same water sample, verifying national synthesis assignment of sample-collection purpose codes, verifying extreme concentrations, and selective verification and updating of GWSI information. Some study units: 3. SW drainage basins -- checking discrepancies between NWIS basin areas and areas derived from GIS basins and providing boundaries for contributing basin in addition to total basin. 4. GW study boundaries -- resolve questions about the derivation of boundaries, especially land use studies, update as needed, and complete documentation so that source materials are cited and study designs could be reproduced. There are also two additional data aggregations: QC data, which is a significant effort, requiring manual entry of missing spike lot numbers and manual selection of data to be included; and continuous temperature (and optionally conductivity) data which will be a simple downloading of available data from Hobo and/or ADAPS files. REVISED SCHEDULE TASKS DUE DATES Matrix view(nonlinear people): NWIS NWIS QC Chems. &Temp. SW GIS GW GIS 1 Pilot guidance tested 6-13 7-15 2 Guidance to SUs on what is requested 6-30 7-31 6-30 7-31 3 SUs respond with data/fixes 7-31 8-30 7-31 8-30 4 Data Aggregated by Jon Scott 9-01 5 Occurrence tables prepared 9-15 Same stuff sorted by due date(linear people): NatSyn data question lists to SUs 6-30 NatSyn submits SW GIS fix needs-4 SUs 6-30 QC & temp. pilot guidance out 7-15 4 SUs respond to SW GIS fix 7-31 NatSyn submits GW GIS fix needs 7-31 SU's correct data & re-submit 7-31 QC & temp. final guidance out 7-31 SUs submit QC & temp. 8-30 4 SUs respond to GW GIS fix 8-30 Jon re-aggregates data 9-01 Occurrence tables prepared 9-15 We recognize that the revised schedule means that costs for some items in producing the study unit summary reports (e.g. printing, finalizing occurrence table and national comparison pages) will be pushed into FY98, even for SUs that will have their summary reports approved in FY97. Please keep your Assistant Regional Hydrologist for NAWQA informed on progress and problems, and they will assist study units in meeting their objectives. PRIORITIES TO BALANCE The NLT recognizes that completing all of the assigned responsibilities will be very difficult for some study units who have lost key staff to other locations or projects. Based on this consideration, the NLT has decided on the following priorities regarding national data aggregation, information archiving, and FY91 study unit reports for the remainder of FY97 and for FY98: 1. Corrections and completion of required fields in the SU chemical data and re-aggregation for the integrity of the national data base. 2. Corrections and updates related to SW and GW GIS data. 3. Completion of the summary report. 3. SU contribution to QC and temperature aggregation. 4. Archiving of critical SU data not in NAWQA national data sets (see NAWQA archive plan guidance--URL below). 5. finishing other SU reports (probably no more than 1 or 2 per FY during the LIP phase). HOW TO GET STUFF Study unit review packages can be fetched via anonymous ftp from: host: mbdokokl.cr.usgs.gov file: ./nawqa/review.SUID.tgz (where SUID=3Dstudy-unit id) URL for 'How to Archive NAWQA Data': http://csdokokl.cr.usgs.gov/nawqa/train/arch/report.html