STATISTICAL ANALYSIS OF NUTRIENT CONCENTRATIONS IN Ground Water FOR SUMMARY REPORTS BY 1994 STUDY UNITS

July 12, 2000

Statistical analysis of nutrient concentrations is based on data retrieved in July 2000 from the NAWQA National-Synthesis, Oracle Data Warehouse. These data represent land-use studies and subunit surveys conducted by study units that began in 1991 and 1994. These results supersede those posted in April 2000, which were based on an earlier data aggregation.

Sites selected for data analysis are designed to minimize sampling bias from different sampling strategies. For this reason, the following sites are excluded from this data subset:

springs and drains, because of uncertainties in the source of water and contributing land-use area;
networks with less than 10 wells (unless in undeveloped areas, such as forest land);
spatially co-located (nested) wells, to preclude redundant sampling of an area for purposes of national comparison;
and networks sampled densely within a small area, to preclude over-representation of areas.

For example, albelusag2 was excluded because it consists of domestic wells located next to installed wells comprising albelusag1; and trinlusur2 and trinlusur3 were not used because they represent nested wells sampling deeper ground water beneath network trinlusur1. One well each was retained from densely sampled networks acfbluscr1 and acfbluscr2 and added to the larger overlapping network acfblusag1. Networks ucollusrc1a-e were combined to yield an aggregate land use study with greater than 10 wells.

Nutrient analyses reported here consist of dissolved concentrations of ammonia (P00608), nitrite-plus-nitrate (P00631) (designated “nitrate” because nitrite contribution generally is negligible), and orthophosphate (P00671). To preclude undue influence on results by wells that were sampled several times, only the most recent sample per well was used in statistical analyses.

Nutrient concentrations were statistically analyzed first by determining median concentrations in ground water for each land-use study and subunit survey, then by using these medians as input to additional analyses to compute national statistics such as percentiles and rankings. Ranks were determined for networks grouped by each type of ground water study (agricultural, urban, subunit survey, undeveloped land, and mining). Using network medians as a basis for national statistics negates sampling bias introduced by network size. Some networks have more than 100 wells, but most have 30 wells or less. Medians were used as a measure of central tendency because they are resistant to the effects of outliers typical of skewed data sets. Censored values were set to one-half the detection limit before calculating network medians. In the case of ammonia, which has two reporting levels, the higher one (0.015 mg/L) was used. Exceedance percentages of the nitrate maximum contaminant level (MCL) were determined for each land-use study and subunit survey based on the number of wells in the network with nitrate concentration greater than 10 mg/L.

Table 1 in spreadsheet “gw_summary700.xls” shows the number of observations, median concentrations, and ranks of ammonia, nitrate, and orthophosphate for each land use study and subunit survey. Network medians that are less than the method reporting level are shown in tables 1 and 2 as the negative value of the reporting level. For example, a network median nitrate concentration of 0.025 mg/L (indicating that more than half the samples in the network had nondetectable nitrate concentration) is shown as –0.050.

The percentage of wells with nitrate exceeding the MCL of 10 mg/L is shown in table 1 for each network, but we recommend interpreting exceedance percentages for subunit surveys only. Subunit surveys are assumed to sample deeper ground water (“major aquifers”) commonly used for drinking. In contrast, land use studies sample shallow, recently recharged ground water that might or might not be used for drinking.

Table 2 shows quartiles and other percentiles for each nutrient, based on median concentrations in agricultural and urban land-use studies. For example, the highest median nitrate concentration associated with any agricultural or urban land-use study is 13 mg/L, and the overall median nitrate concentration for these land uses is 1.95 mg/L. Agricultural and urban network medians were lumped in this analysis.

Table 3 shows selected percentiles of nitrate exceedance percentages associated with subunit-surveys. For example, 90 percent of the subunit surveys had a nitrate MCL exceedance percentage of about 10 percent.