DATA--Guidance for the Use of the Program HYDRA to Estimate or Modify Edited Unit Values from ADAPS
In Reply Refer To: April 3, 2003
Mail Stop 415
OFFICE OF SURFACE WATER TECHNICAL MEMORANDUM NO. 2003.02
OFFICE OF WATER QUALITY TECHNICAL MEMORANDUM NO. 2003.02
OFFICE OF GROUND WATER TECHNICAL MEMORANDUM NO. 2003.01
SUBJECT: DATA--Guidance for the Use of the Program HYDRA to
Estimate or Modify Edited Unit Values from ADAPS
BACKGROUND:
The latest release of the National Water Information System (NWIS)
Automated Data Processing System (ADAPS), version 4.3, again allows
graphical editing of unit values using the program HYDRA. This program
allows for mass changes in edited unit values, such as stage or
temperature, with a simple click and draw or click and drag. Because of
this relative ease in “creating edited unit values”, specific guidance is
necessary to define when this feature can be used and when it is
prohibited. This memo includes guidance from the Office of Surface Water,
the Office of Ground Water, and the Office of Water Quality.
As the Water Discipline moves toward unit values being a basic product of
our data collection programs, new tools will be released within the ADAPS
system to allow editing and estimation of output unit values and better
flagging of such estimates within the database. As these tools become
available, this guidance will be revised.
HYDRA is a program originally created during the NWIS-II effort, which was
suspended in the mid 1990’s. It is written in a fairly obscure,
mostly-obsolete language, but had many features the Surface-Water User
Group (SWUG) and program developers deemed useful for processing
time-series data. HYDRA was first available in ADAPS, version 3_2, but
included a caution that original data could be lost if the proper save
procedures were not followed. Despite these warnings, several Districts
realized the enormous advantages of graphical editing and began using this
tool. However, over time many data chiefs and the SWUG became concerned at
the ease with which data points could be “manufactured,” as well as the
possible loss of original data, using HYDRA. In addition, although ADAPS
does document changes to input unit values through the asterisk flag, this
flag does not carry forward to the computed parameter. For these reasons,
in ADAPS version 4.2 the mass editing of unit values was disabled. (It
should be noted that concern regarding loss of original data has been
addressed in ADAPS 4.2 as input unit value data are now stored as
“measured” unit values that cannot be changed. The concern over the ease
of editing unit values and the lack of documentation of those changes
within the system remains however).
Measured unit values are the original values measured by the data
collection instruments and entered into the ADAPS database. Measured unit
values cannot be altered within the ADAPS system. When a mistake occurs,
they must be deleted and the correct values entered. Edited unit values
are those used for records computations. They are the values that are
editable in HYDRA. Output (computed) unit values are calculated by the
primary computations program from other unit values (input or edited unit
values). For example, discharge unit values (output) are computed from
stage unit values (input).
After much discussion with the interested parties, the SWUG made the
decision to reenable mass editing of input unit values in ADAPS version
4.3 with specific guidelines for cases when this option is appropriate and
when it is not appropriate.
POLICY:
The guiding principles to judge appropriate use of HYDRA to estimate or
modify edited unit values are:
1. Unit-Value data should never be created if it did not previously
exist on some recording device. Observation of stage and manual recording
of these observations in field notes is considered a recording device.
2. Original data should not be changed simply to get a correct
final computed result.
3. Existing data can be corrected using best hydrologic judgment,
provided the correction, even if done by directly changing data, is based
on the original data (i.e. the original data, although incorrect, has
value in production of the estimate).
4. It is preferable to estimate output unit-values such as
discharge directly, but the tools to do this do not currently exist in
ADAPS. In the interim, gage height unit values can be estimated.
5. It must be remembered that input unit values such as gage height
and velocity, have intrinsic value to the user community and are not just
a surrogate for discharge.
Appropriate cases:
1. It is appropriate to use the mass editing option of HYDRA for
edited unit values if there are data problems due to orifice problems,
”painting”, or burial and sediment buildup, provided the estimates are
based on the original data and that data indicate adequate contact with
actual stream conditions.
2. It is appropriate to “paste” unit values from backup sources
into the primary data source using the “select and paste” option in HYDRA.
Backup measurement sources must record values at the same site. For
example, stage may be measured at a site using a pressure transducer
(primary sensor) and a float/counterweight (backup sensor).
Inappropriate uses:
1. The mass-editing feature of HYDRA should NEVER be used to alter
correct recorded input unit values simply to generate correct output unit
values. For example, input stage values that are correct, but are ice
affected, should never be “adjusted” to create stage values which when
applied to a rating curve or table, yield the correct discharge unit
values. Similarly, correctly recorded stage values that are affected by
backwater should not be altered or edited to yield correct output
unit-values
2. HYDRA should not be used to “create” unit values for periods of
missing record. Nor should it be used when the original unit values have
no value in the estimation of corrected data (e.g. gas leaks or water
quality sensors exposed to the air during servicing or during extreme low
water events). Daily values may be estimated for these periods, but
missing unit values should not be created and any existing but invalid
unit values should be marked as erroneous.
Although HYDRA also can be used to estimate missing ground-water record,
the Office of Ground Water has a general policy that only measured values
should be stored in NWIS and published in the Annual Data Report.
Estimated values generally should not be published.
Additional flags and/or remark codes need to be added to the database to
further document, through to the computed parameter, the editing or
manipulation done in HYDRA. These flags will be a high priority for the
next ADAPS release. In the meantime, all editing using HYDRA MUST be
carefully documented in the station analysis for that site.
/signed/
Stephen F. Blanchard
Chief, Office of Surface Water
/signed/
Timothy L. Miller
Chief, Office of Water Quality
/signed by/
William L. Cunningham for
William M. Alley
Chief, Office of Ground Water
Distribution Code: All WRD Employees