USGS Groundwater Policy > Groundwater Model Archiving > Review > Instructions for Reviewers

Documenting, Archiving, and Public Release of Numerical Groundwater Flow and Transport Models: Instructions for Reviewers

Modeler Workflow - Instructions for Reviewers

This page describes the review requirements for numerical groundwater flow and transport models prior to public release.

Data Review

Review of the model archive is an important part of the release process. The model archive review is considered a Data Review, which is described in more detail on the USGS data management web site. The purpose of the model archive review is to verify the accuracy, completeness, and usability of the product.  The reviewer should confirm that the model archive corresponds to the information given in the model documentation report (tables, figures, text).

The following list is provided for the assistance of model archive reviewers.  Model archive reviewers should have experience with the type of groundwater models contained in the archive.  

In most cases, it will be unreasonable to check every aspect of the model documentation report, so a spot check or a check of carefully selected files will suffice.  In every case, the reviewer should include a description of how much of the model archive they examined.  Colleague reviewers for the model documentation report may also serve as the metadata reviewer for the model archive.

The following is a list of recommended steps for the model archive reviewer:

1. Download model files

• Verify that the compressed files are named correctly
• Verify that the files uncompress (and have not been corrupted) and that the uncompressed model archive structure can be replicated.
• Verify that the uncompressed archive structure is consistent with the required structure.

2. Review selected files

• Check the readme files to make sure they accurately reflect the archive content, provide instructions for running the model(s), provide instructions for reconstructing the model archive from the downloaded model files, and provide a brief description of each model file.
• Load and plot selected shapefiles, if present, to make sure they are properly georegistered
• Does the model archive include metadata in a standard format?

3. Check the archive content

• For MODFLOW-based models, load the model into ModelMuse, FloPy, or another graphical user interface (GUI) and examine the model input and output
• Check for common errors, such as negative hydraulic conductivity, negative cell thicknesses, river bottoms below cell bottom in convertible layers, etc
• Verify that there is no Personally Identifiable Information (PII) or sensitive water-related information (for example, exact locations of public-supply wells), proprietary data, or other information or data that cannot be publicly released in the public archive material.

4. Run the model(s)

• Verify that the readme provides enough information on how to to run the model outside any GUIs used.
• Run the models in the archive being careful not to overwrite the archives’ output files. Run the models outside the GUI. If model can’t be run (too large, runtimes unreasonable, etc) make a note of this in your review (*see note below). Skip step 5-1 but pay close attention to step 5-2 below.
• Verify that the models run to completion with no errors. In some cases, it may be intractable to physically run the model -- for example, a model may have taken days to run in the original simulation -- in this case, examine the provided model output to determine if it ran to completion without errors. Also determine when the run was made (date stamp within the model output); if the run was made relatively recently, this should be a red flag -- the model documentation was probably prepared weeks or months before (including figures, tables, etc). Pay extra close attention to comparing model archive output to the model documentation since changes may have been made to the model in the weeks/months intervening. Make a note that the model was not run and why.

5. Inspect selected model output

• Spot-check simulated output (water budgets, heads, drawdowns, flows) from your run with output in the archive output folder
• Confirm that model output files that are discussed in the model documentation report (tables, figures, text) are included in the model archive. For example, if simulated potentiometric surfaces or hydrographs are presented in the model documentation report then simulated model head output for the presented periods should be included in the model archive.
• If discrepancies are found and you could not run the model (see step 4-2 above), you may have to have the modeler re-run the model and provide you with corrected update.

6. Prepare a written assessment of your review

• Briefly document what was examined.
• Provide a numbered list of comments.

7. Complete Model Archive Verification Form

• Under OGW Technical Memo 2015.02, model reviewers must complete the Model Archive Verification and Approval Form. The author must include the signed form in the IPDS record for the data release.

Metadata Review

Metadata reviewers should refer to the metadata guidance for modelers and the Bureau metadata guidelines and reviewer checklist.