DATA--Guidance for the Use of the Program HYDRA to Estimate or Modify Edited Unit Values from ADAPS In Reply Refer To: April 3, 2003 Mail Stop 415 OFFICE OF SURFACE WATER TECHNICAL MEMORANDUM NO. 2003.02 OFFICE OF WATER QUALITY TECHNICAL MEMORANDUM NO. 2003.02 OFFICE OF GROUND WATER TECHNICAL MEMORANDUM NO. 2003.01 SUBJECT: DATA--Guidance for the Use of the Program HYDRA to Estimate or Modify Edited Unit Values from ADAPS BACKGROUND: The latest release of the National Water Information System (NWIS) Automated Data Processing System (ADAPS), version 4.3, again allows graphical editing of unit values using the program HYDRA. This program allows for mass changes in edited unit values, such as stage or temperature, with a simple click and draw or click and drag. Because of this relative ease in “creating edited unit values”, specific guidance is necessary to define when this feature can be used and when it is prohibited. This memo includes guidance from the Office of Surface Water, the Office of Ground Water, and the Office of Water Quality. As the Water Discipline moves toward unit values being a basic product of our data collection programs, new tools will be released within the ADAPS system to allow editing and estimation of output unit values and better flagging of such estimates within the database. As these tools become available, this guidance will be revised. HYDRA is a program originally created during the NWIS-II effort, which was suspended in the mid 1990’s. It is written in a fairly obscure, mostly-obsolete language, but had many features the Surface-Water User Group (SWUG) and program developers deemed useful for processing time-series data. HYDRA was first available in ADAPS, version 3_2, but included a caution that original data could be lost if the proper save procedures were not followed. Despite these warnings, several Districts realized the enormous advantages of graphical editing and began using this tool. However, over time many data chiefs and the SWUG became concerned at the ease with which data points could be “manufactured,” as well as the possible loss of original data, using HYDRA. In addition, although ADAPS does document changes to input unit values through the asterisk flag, this flag does not carry forward to the computed parameter. For these reasons, in ADAPS version 4.2 the mass editing of unit values was disabled. (It should be noted that concern regarding loss of original data has been addressed in ADAPS 4.2 as input unit value data are now stored as “measured” unit values that cannot be changed. The concern over the ease of editing unit values and the lack of documentation of those changes within the system remains however). Measured unit values are the original values measured by the data collection instruments and entered into the ADAPS database. Measured unit values cannot be altered within the ADAPS system. When a mistake occurs, they must be deleted and the correct values entered. Edited unit values are those used for records computations. They are the values that are editable in HYDRA. Output (computed) unit values are calculated by the primary computations program from other unit values (input or edited unit values). For example, discharge unit values (output) are computed from stage unit values (input). After much discussion with the interested parties, the SWUG made the decision to reenable mass editing of input unit values in ADAPS version 4.3 with specific guidelines for cases when this option is appropriate and when it is not appropriate. POLICY: The guiding principles to judge appropriate use of HYDRA to estimate or modify edited unit values are: 1. Unit-Value data should never be created if it did not previously exist on some recording device. Observation of stage and manual recording of these observations in field notes is considered a recording device. 2. Original data should not be changed simply to get a correct final computed result. 3. Existing data can be corrected using best hydrologic judgment, provided the correction, even if done by directly changing data, is based on the original data (i.e. the original data, although incorrect, has value in production of the estimate). 4. It is preferable to estimate output unit-values such as discharge directly, but the tools to do this do not currently exist in ADAPS. In the interim, gage height unit values can be estimated. 5. It must be remembered that input unit values such as gage height and velocity, have intrinsic value to the user community and are not just a surrogate for discharge. Appropriate cases: 1. It is appropriate to use the mass editing option of HYDRA for edited unit values if there are data problems due to orifice problems, ”painting”, or burial and sediment buildup, provided the estimates are based on the original data and that data indicate adequate contact with actual stream conditions. 2. It is appropriate to “paste” unit values from backup sources into the primary data source using the “select and paste” option in HYDRA. Backup measurement sources must record values at the same site. For example, stage may be measured at a site using a pressure transducer (primary sensor) and a float/counterweight (backup sensor). Inappropriate uses: 1. The mass-editing feature of HYDRA should NEVER be used to alter correct recorded input unit values simply to generate correct output unit values. For example, input stage values that are correct, but are ice affected, should never be “adjusted” to create stage values which when applied to a rating curve or table, yield the correct discharge unit values. Similarly, correctly recorded stage values that are affected by backwater should not be altered or edited to yield correct output unit-values 2. HYDRA should not be used to “create” unit values for periods of missing record. Nor should it be used when the original unit values have no value in the estimation of corrected data (e.g. gas leaks or water quality sensors exposed to the air during servicing or during extreme low water events). Daily values may be estimated for these periods, but missing unit values should not be created and any existing but invalid unit values should be marked as erroneous. Although HYDRA also can be used to estimate missing ground-water record, the Office of Ground Water has a general policy that only measured values should be stored in NWIS and published in the Annual Data Report. Estimated values generally should not be published. Additional flags and/or remark codes need to be added to the database to further document, through to the computed parameter, the editing or manipulation done in HYDRA. These flags will be a high priority for the next ADAPS release. In the meantime, all editing using HYDRA MUST be carefully documented in the station analysis for that site. /signed/ Stephen F. Blanchard Chief, Office of Surface Water /signed/ Timothy L. Miller Chief, Office of Water Quality /signed by/ William L. Cunningham for William M. Alley Chief, Office of Ground Water Distribution Code: All WRD Employees