Link to USGS home

Office of the Delaware River Master (ODRM)

PROTOCOLS FOR MODELS

SUBMITTED TO THE PARTIES TO THE 1954 U.S. SUPREME COURT DECREE

Paper available in PDF.

Introduction

Computer models have become indispensable for planning and management of complex water systems. This document prescribes general protocols -- principles and guidelines -- to support the development and use of models in water planning and management for the Delaware River Basin. These protocols reflect the views of the Parties to the 1954 U.S. Supreme Court Decree (Decree Parties) and provide a common format for how models intended for consideration by the Parties should be developed and documented. The protocols and guidelines are subject to revision and may be updated as the need arises.

The objective of these protocols is to provide guidance to water stakeholders for model-based evaluation of Delaware River Basin water resources. Adherence to the guidance will facilitate consideration by the Decree Parties by:

Model Development Process

Table 1 presents nine standardized steps that provide a general framework for model development. These steps are intended to ensure that the model addresses the intended problem, reasonably represents the prototype system, and that results are reasonably tested and evaluated. In addition, the model development process helps ensure that the entire process is documented so that others will know what has been done and are clearly informed about the model's limitations for use. Table 1 provides general guidance for models submitted for consideration by the Decree Parties. Specific requirements for submitting models are presented in a subsequent section of this document.

Table 1. Major Steps in Model Development
StepNamePurpose

1. Problem Identification Solving the correct problem
2. Define Modeling Objectives Define use for model and standard of success
3. Formulation of Model Mathematical similarity to the problem system
4. Selection and Study of Numerical Solution Numerical similarity to the mathematical formulation of the problem
5. Model Calibration Input data represents system behavior and characteristics
6. Model Verification Test model base on model behavior
7. Model Validation Test model by comparison with field data
8. Documentation of Model Make model understandable to users
9. Update and Support of Model Maintain and improve the model's usefulness

Use of Models in Planning Studies

Along with model development, the use of models for evaluating planning problems should also be linked to a logical planning process. This process, which is summarized in Table 2, resembles that of a traditional planning study. An expanded discussion of the use of models in planning studies is contained in Appendix 1 of this document.

Table 2. Steps in Model Use for Planning Studies

1.Define study objectives.
2.Define how model outputs relate to the performance of alternatives.
3.Define a base case.
4.Define alternatives.
5.Identify model version and input data.
6.Present model results.
7.Summarize and discuss the performance of each alternative.
8.Discuss study limitations.

Requirements for Models Submitted to the Decree Parties

Models submitted for consideration by the Decree Parties shall address the following specific points:

  1. All assumptions underlying the model must be stated explicitly.
  2. All operating rules incorporated in the model must be stated explicitly. Model developers may elect to work directly with the Decree Parties when designing non-standard operating rules.
  3. Input file development must be clearly explained.
  4. The numerical method used for flow routing must be clearly explained.
  5. Lake Wallenpaupack must be included in the model, and the operating rules simulated must be specified.
  6. Model outputs must address the impacts of the modeled system on reservoir storage and drought frequency. (See Appendix 1, Item 6 for output format requirements.)
  7. The total quantity of water initially available from New York City Delaware Basin reservoirs for meeting instream flow needs in New York State shall be based on the conservation release rates and the thermal bank quantity as defined in the "Interstate Water Management Recommendations…" (Good Faith Agreement) and codified in the Delaware River Basin Commission (DRBC) Comprehensive Plan by DRBC Docket No. D-77-20 CP (Revised), with consideration of amelioration associated with DRBC Resolution No. 2002-33.
  8. The total quantity of water initially available from New York City Delaware Basin reservoirs for meeting Delaware River instream flow needs in the lower basin states shall be based on the Montague flow objectives as expressed in the Good Faith Agreement and codified in the DRBC Comprehensive Plan by DRBC Resolution No. 83-13.
  9. Model runs shall include under normal storage conditions a base case of 800 mgd diversion to New York City, 100 mgd diversion to New Jersey, and a flow objective of 1,750 cfs for the Delaware River at Montague, New Jersey.

APPENDIX 1

Use of Models in Planning Studies

Principles for the use of a model in a planning study are somewhat different from those for model development. A model-based planning study typically has a narrower set of objectives and can rely on much of the documentation and understanding derived during model development. Principles that guide the use of models in a planning study should be aimed at improving the quality of the planning study. Thus, the principles presented here resemble those of a traditional planning study and are consistent with Federal water resources planning guidelines. For a model developed for a specific planning application, much of the following information will appear in the original model development documentation. The use of models in a planning study often follows the following general steps:

  1. Define Study Objectives
  2. The planning study's objectives should affect the type of models chosen and the way models are used for planning. Not all models are appropriate for all objectives, and models often are run differently depending on study objectives. For example, a study of water quality in the Delaware Estuary, requiring close matches of a salinity time series, might require use of detailed hydrodynamic-solute transport models with a great deal of input data. However, a planning study for an upstream storage facility might require only relative comparison of water-quality statistics for different alternative plans, allowing use of a model with fewer computational and data requirements.

  3. Define How Model Outputs Relate to the Performance of Alternatives
  4. Do model outputs indicate how well a proposed plan would perform on specific planning objectives? How close is the correspondence between model output and likely accomplishment of planning objectives?

  5. Define a Base Case
  6. Planning studies are typically more understandable if alternative plans are developed from a base case. The "base case" can be defined as:

  7. Define Alternatives
  8. Alternatives are the competing plans whose performance is to be compared. The definitions of alternatives should include conceptual descriptions (usually as departures from the base case) and detailed descriptions sufficient to allow replication.

  9. Identify Model Version and Input Data
  10. The model version and input data sets should be identified in the study documentation. Modifications made to the basic model, data, and related assumptions should be clearly explained. Sufficient information should be submitted so that the model can be rerun and produce the same results.

  11. Present Model Results
  12. Modeling results for each alternative should be provided and discussed as appropriate to the planning study. The output format should allow for presentation of daily data for each model component of flow, storage, etc., so that results can be more easily verified.

  13. Summarize and Discuss Performance of Each Alternative
  14. Summarizing the performance of each alternative on each planning objective should be part of a planning study.

  15. Discuss Study Limitations
  16. The major limitations of the study results should be identified, and the implications for interpreting the results should be discussed. Note: The degree to which an alternative could have other consequences, including regulatory implications, will be evaluated by the Decree Parties.

APPENDIX 2

Model Simulation Log

Model development and studies need to be documented and archived for quality-assurance purposes. A model archive should contain sufficient information generated during the modeling process so that a third party could adequately perform a post-modeling audit and so that future use of the model is possible. Use of the following model simulation log is required for archiving each significant model run submitted for consideration by the Decree Parties.

Models should be submitted to:

Gary N. Paulachok
Office of the Delaware River Master
U.S. Geological Survey
10 Buist Road, Bldg. 3, Suite 304
Milford, PA 18337
(570) 296-7213 (Voice)
(570) 296-9414 (Fax)
gnpaulac@usgs.gov (email)

Model Simulation Log Form:


Biology | Geology | Geography | Water

Contact us using our online form for feedback and questions