In Reply Refer To:
Mail Stop 412

November 14, 2006

Office of Water Quality Technical Memorandum 2007.01

Subject: Policy for the Evaluation and Approval of Analytical Laboratories

The U.S. Geological Survey (USGS) Water Resources Discipline (WRD) has had a long-standing policy that analytical laboratories providing chemical, radiochemical and biological analyses to WRD shall be evaluated, approved and regularly reviewed. The most recent process for laboratory evaluation, documented in the Office of Water Quality (OWQ) Technical Memoranda 98.03R, and 2002.05 are superseded by this OWQ Technical Memorandum 2007.01. This new laboratory evaluation policy and process was developed to address deficiencies with the implementation of 98.03R that made the previous policy ineffective.

The 98.03R policy provided for a broad national approval of laboratories using:
(1) laboratory quality-assurance plans and standard operating procedures as evidence that the reviewed laboratory had a quality system, and (2) a small number (maximum two per year) of blind sample results to compare performance of the laboratory under review to the performance of other laboratories. Laboratories were approved for specified constituents and available for use by any project requiring those constituents, and they were re-evaluated every three years.

The new policy: (1) is project based, assessing a laboratory’s capability to meet specific project needs (as defined by project specific data quality objectives), (2) includes a performance-based laboratory evaluation process using samples specific to the project design, (3) makes it the responsibility of project management to prepare and implement a laboratory evaluation plan that includes submitting quality-control (QC) samples to the laboratory over the life of the project and re-evaluating performance regularly, and (4) makes it the responsibility of the Water Science Center (WSC) Director or in some cases National Program managers to approve the laboratory evaluation plan for each project and assure that it is implemented throughout the project term. The new policy requires more participation from the WSC project personnel than the previous policy; however, the emphasis now is on maintaining data quality required to achieve project objectives. This new policy requires cooperation of the laboratory manager. To assist in this process, the Branch of Quality Systems (BQS) will develop and maintain a database of laboratory information and performance data, and will provide advice and assistance in developing and implementing the laboratory evaluation plan.

The new policy pertains to all laboratories, even when the analytical method being used by a project does not yet have an approved analytical method report. Projects being conducted as a formal part of analytical methods development activities are exempted from the requirement of this policy. However, projects being conducted as a formal part of analytical methods development activities have specific requirements for QC data to accompany each environmental dataset as defined in OWQ Technical Memorandum 2004.01.

The two primary deficiencies corrected by this new policy are: (1) general national approval, which was too generic to be useful at the project level, is now changed to a project specific approach with defined data quality objectives establishing a scientific defensible approach, and (2) approval authority is designated at the level of work with the highest knowledge of project needs and the ability to demonstrate acceptable laboratory performance.

This policy and its procedures (Attachments 1-3) will take some time to implement fully, but they begin in fiscal year (FY) 2007. Given the large number of laboratories used by WSCs, implementation of this policy will occur in a phased approach through FY 2008. By the beginning of FY 2009, all laboratories should be approved under this new policy, and OWQ review of WSCs will include examination of laboratory approval packages. Through FY 2007, projects may choose to have a laboratory approved or re-approved using the procedures in 98.03R; however, they will have to pay the cost of BQS staff time as described in 98.03R.

This OWQ Technical Memorandum includes three attached documents: (1) The Laboratory Policy described in detail with phased implementation, exemptions from the evaluation and approval process, and BQS responsibilities; (2) The Process Description outlining the procedure for evaluation, the procedure to follow for a new laboratory, or to follow for a laboratory in the BQS database; and (3) A description of the BQS database.

Timothy L. Miller /s/
Chief, Office of Water Quality
This memorandum supersedes Office of Water Quality Technical Memoranda 98.03R, and 2002.05.

Distribution: All WRD Employees

Attachments (3)

Attachment 1—Laboratory Evaluation Policy

Analytical laboratories that provide chemical, radiochemical, and biological analyses to the Water Resources Discipline (WRD) must be evaluated relative to the objectives of a project requiring analyses and approved for use for that specific project. Performance samples provide the basis for the initial laboratory approval and the approved laboratory must continue to provide performance sample results during the life of the project.

Process Overview

The decision to select and use a laboratory for analyses of environmental samples for a project is based on the project objectives (data quality objectives) and the performance of the laboratory relative to those objectives. Authority and responsibility for the decision to use a laboratory rest with the Water Science Center (WSC) Director, or a WRD National Program Manager. The Branch of Quality Systems (BQS) can provide assistance in gathering information and evaluating quality-assurance (QA) data as an advisor to the project.

Project objectives should be defined minimally in terms of the analytes of interest and the expected environmental concentration range for each analyte. To initially select a laboratory, project data quality objectives are compared to defined performance (detection levels and estimated bias and variability) of the analytical methods used by the laboratory. The initial step should ensure that the methods available from a laboratory provide appropriate results for the project.

Performance data from a laboratory are required to demonstrate that the laboratory can achieve the performance that has been defined for the analytical services the project requires. Performance data are generally the analytical results from blind samples (concentrations unknown to the analyst). Two phases of performance data are required from the laboratory. Initial performance data are required to show that the laboratory can do what they have said they can do, before any environmental samples are submitted to the laboratory. The initial data are part of the package of information needed to support the selection of a laboratory for a project. Ongoing performance data are required to show that a laboratory continues to meet their performance criteria during the life of the project.

The project chief has the responsibility for collecting the information and data required for selection of a laboratory to provide analytical results to the project. The selection package must be reviewed by at least two individuals independent from the project before it is submitted to the WSC Director (or appropriate National Program Manager) for approval. The WSC Director will either concur or disagree with the project chief’s recommendation to use the laboratory after reviewing the selection package and the reviewer comments. Once the WSC Director concurs with the selection of a laboratory, then the laboratory may be used to analyze environmental samples for the project. Following approval by the WSC Director, the Laboratory Evaluation Package should be kept on file in the WSC, and a copy provided to the Regional Water-Quality Specialist. The detailed steps for preparing a laboratory selection package and the information and data required are documented in Attachment 2.

The goal of this process is to ensure that a laboratory provides data that meet the project needs. It is not a laboratory approval process that sanctions an entire laboratory for multiple uses, which is a very different and more complicated process. Accreditation by the National Environmental Laboratory Accreditation Program (NELAP) was accepted as a substitute for the previous laboratory evaluation process (OWQ Technical Memorandum 2002.05); however, NELAP accreditation cannot be a substitute for this new process, because the NELAP accreditation does not incorporate project specific data requirements. Selection of a laboratory for one project does not mean that it can automatically be selected for another separate project, although every effort should be made to share information and data with multiple projects that use the same laboratory. The key for laboratory use is a match between project specific data quality objectives and laboratory performance addressing those project objectives. A national database will be maintained by the BQS to facilitate sharing of information and data, as well as providing periodic reviews of the ongoing laboratory performance for each project. The BQS database is described in more detail in Attachment 3.

Phased Implementation

There will be a phased implementation of this policy to ensure that it is implemented effectively. Fiscal year (FY) 2007 will be a transition year, and all laboratories should be approved by the end of FY 2008. Beginning in FY 2009, WSC review by the OWQ will include examination of laboratory approval packages.

Transition from the existing procedure to the new procedure:
The procedures supporting this policy and described in the attachments will continue to develop during FY 2007. Projects that choose to use the new procedures can have assistance from BQS, because these first projects will be test cases to help build the laboratory database, software and training material. Through FY 2007, projects may choose to have a laboratory approved or re-approved using the procedures in OWQ Technical Memorandum 98.03R, but they will have to fund the cost of BQS staff time as described in 98.03R.

Development of tools:
The revised Laboratory Evaluation Policy (LEP) requires training material, a modified database, and software to produce useful reports for projects. The training material needs to be available, and the database and software needs to support the basic functionality of data entry and storage, before the new procedure is fully capable of meeting the needs of WRD. These activities will proceed through FY 2007 and into
FY 2008.

Initial sample media and analytical services:
Although water chemistry is the most prominent analytical work that the LEP addresses, the procedures also fit other types of media (i.e., sediment, microbiological, tissue, etc.), and analyses (i.e., radchem and taxonomy, etc.) as well. The initial implementation of the LEP should focus on the water chemistry analytical services. Other matrices and analytical work should be added in later implementation during
FY 2008.

National Water Quality Laboratory (NWQL):
The initial phase of the LEP does not require approval for projects using the NWQL, because abundant QA information is available through the BQS blind sample projects. The NWQL will need time to establish some automated procedures for providing method information in the project worksheets, because the National Laboratory deals with so many different projects. When processes are in place by the end of FY 2008, projects that use NWQL are also required to evaluate the performance of NWQL methods relative to their project data quality objectives. Until then, project staffs are encouraged to apply the principles of this policy to confirm that NWQL meets their project needs. Initial and ongoing NWQL performance data can be accomplished largely with existing blind sample programs in BQS. The data from these BQS programs are already available to projects in WRD and can be used to evaluate performance.

Exemptions from the Lab Evaluation Policy

The procedures described in this policy are not appropriate for some projects that are designed around non-routine analytical work. The most common type of non-routine analytical work is analytical method development. Some characteristics of method development that make these procedures inappropriate are:

• There are no established measurement expectations that control the continuation of analyses (e.g., spike recoveries or calibration rules).
• The process to analyze the sample is not predefined, nor limited to existing procedures.
• The processes may be adapted during the analysis to improve data quality
  (e.g. adjustments for matrix specific conditions).
• The data cannot be compared directly to other data using the same generic methods because the data was subject to extra, undocumented, or novel procedures.
• The data require explanation.
  The procedures for evaluating the results from these types of projects will be an inherent part of the project work plan.

Branch of Quality Systems Responsibilities

The BQS LEP database will be a resource for data and information to projects and programs that require the use of an analytical laboratory. Below are some of the specific tasks that the BQS is responsible for:

• Provide streamlined forms for collecting the appropriate laboratory QA information and data at the start of the approval process
• Assist project staff in evaluating the laboratory information they have obtained
• Assist project staff in locating appropriate performance test samples
• Provide a national repository for the project laboratory evaluation and performance data
• Provide periodic summaries of the laboratory performance data relative to project data quality objectives
• Provide summaries of the performance data from any laboratory already in use by WRD to any project that may want to use the same laboratory
• Provide training material for the laboratory selection and evaluation procedures

Web site for the Laboratory Evaluation Policy
http://bqs.usgs.gov/lep/index.html

Attachment 2 – Process Description

The procedures for selecting and using an analytical laboratory are described below. There are two phases to evaluation of a laboratory, an initial phase for selecting a laboratory and an ongoing phase for continuing to use a laboratory.

Initial Assessment
Selecting a laboratory is the first phase of the process and requires a project compare the project analytical needs to the analytical capabilities of the laboratory. The data and information collected as part of this initial assessment of the laboratory needs to be documented and reviewed and the Water Science Center (WSC) or National Program management must approve the selection of the laboratory, before the laboratory can be used for analysis of environmental samples.

Ongoing Assessment
The continued performance of a laboratory during the life of a project is critical to project success. A laboratory that is providing analytical results to a project is required to provide performance data to demonstrate that they continue to meet the project performance criteria.

The project staff is responsible for obtaining the data and information from a laboratory. The data and information should be entered in the Branch of Quality Systems (BQS) database electronically (see Attachment 3).

Procedure for a single project using a new laboratory

1. Collect and document the project data quality requirements and laboratory information in the Project Worksheet provided by the BQS for the Laboratory Evaluation Policy (LEP):
   1. Project requirements for the analytical results:
      • List of the analytes
      • The range of concentrations expected for each analyte
      • The matrix and any comments to the laboratory regarding special handling or warnings of possible hazards.
      • The analytical method, if a specific method is required by the project.
   2. Method performance information from the laboratory:
      • The method identification number
      • The analytes included in the method
      • The lab method detection level for each analyte
      • The lab reporting level for each analyte
      • The concentration range for each analyte
      • The laboratory estimates for bias and variability for each analyte
2. Obtain quality-control (QC) data that provides an initial demonstration of capability for the laboratory in order of decreasing information provided:
   1. Obtaining double-blind known sample results
   2. Obtaining single-blind known sample results
   The samples and results used for this demonstration of capability must be representative of the expected analytes and concentrations of the project.
   Reference material for some new or specialized methods may be difficult to find. BQS or the analysts running the method may be able to assist the projects in locating reference material. If appropriate reference material cannot be found, then BQS can help design a special case approach.
3. Evaluate the lab capabilities in relation to project objectives: The information and data collected at this point should be sufficient for an initial assessment of how well the project objectives can be met by the laboratory. The information and data collected should be entered into the LEP database and some reports generated to identify any potential mismatches. If the objectives and capability match well, then proceed with the Laboratory Selection Package described in step 4 below. If the laboratory and project do not appear to be a good match or there are some concerns, the project should try to resolve the questions at this point. The project or program staff can seek advice about their concerns from the WSC water-quality specialists, other experts in an office, the Regional Water-Quality Specialists or the BQS. Some negotiation with the laboratory may be possible to resolve existing differences or concerns. If the laboratory and the project are not a good match and the differences cannot be resolved, the project should look for a different laboratory to provide analytical results.
4. Prepare a Laboratory Selection Package to include:
   1. Project information: Describe the project, the type of analytical services needed, including the data quality objectives, how frequently the samples will be submitted, when sample collection will begin and the expected timeframe that the analytical services will be needed.
   2. Laboratory information: Name, address, contact and phone number of the laboratory and references to publications that describe the method(s) that have been selected.
   3. Project Worksheet completed through the initial demonstration of capability.
   4. Results of the Initial demonstration of capability: briefly describe the results of the initial demonstration of capability; does the capability of the laboratory meet expectations?
   5. Plan for testing ongoing capability: define the sample types and frequency for the performance testing that will be conducted during the life of the project.
   6. Data management plan: define how data will be obtained from the laboratory, and where the data from the laboratory will be stored. Include parameter codes that will be used and any other metadata or codes that will be stored with the data.
   7. Recommendation from the project staff to use the laboratory.
5. Submit the Laboratory Selection Package for review. The package should be reviewed by at least two people who are independent of the project. The reviewers may be a WSC water-quality specialist, a Regional Water-Quality Specialist, the BQS LEP support staff, a chemist from a different laboratory, a research chemist, or other selected peer reviewer that has some knowledge of the analytical needs of the project. The project staff will address any questions or concerns from the reviewers.
6. Submit the Laboratory Selection Package to the WSC Director or National Program manager: The Laboratory Selection Package, including the reviews and responses, should be submitted for management approval and signature concurring with the project staff that the laboratory can be used for the project. The approved Laboratory Selection Package should be kept on file in the WSC or National Program office, and a copy should be sent to the corresponding Regional Water-Quality Specialist.
7. Obtain QC data to assess the laboratory performance during the life of the project: Reference material submitted as a double-blind or single-blind sample will be used to provide measures of the laboratory’s analytical performance over time for the project. The project staff should regularly submit QC samples, arrange for QC samples to be submitted, or obtain existing QC data for the laboratory’s ongoing demonstration of capability. The performance samples should be representative of the project samples, cover the timeframe of the project and be submitted frequently enough to provide confidence that the laboratory continues to provide adequate data for the project. The data should be provided to BQS as soon as possible to add to the LEP national database.
8. Review the laboratory performance at least biannually and probably more frequently. BQS will provide the project staff with a Laboratory Evaluation Package with charts and data summaries from data in the BQS national laboratory information database. The data can be compared against the initially documented laboratory limits and the project data requirements. An example for when the laboratory performance might be re-evaluated is during project reviews.

Procedure for a project using a laboratory with existing data in BQS

A project that is planning to use a laboratory that other projects have already used may be able to obtain information about the laboratory methods and performance data from the BQS database. The information in the database needs to be about the same method from the same laboratory and the information needs to be relatively recent to use this source instead of getting new information from the laboratory. The performance data needs to be representative of the new project concentration range to be useful. Retrieving the information from this database will shorten the process for the project, save the laboratory significant time and save money if the performance data can be used.

Procedure for a national or regional program using a laboratory

A national or regional program or many projects from separate WSCs that are seeking to use a laboratory for national or regional objectives, may work with a single Laboratory Selection Package and the data from common performance samples. The project data quality objectives and the performance data need to encompass the range of analytical needs of all the sites and environmental conditions expected.

Attachment 3 – Branch of Quality Systems (BQS) Data Base

Purpose

To support a national laboratory evaluation policy, BQS maintains a database to collect and store laboratory performance data submitted through the BQS website. The database needs to serve two distinct functions. First, the database houses initial and ongoing method-specific performance data for selected laboratories and provides laboratory performance reports to the projects. Second, the database serves as the “yellow pages” for new projects looking for a laboratory with specific methodology, quality features, or specific locales. In some cases, projects may find some efficiency in choosing laboratories already in the database because some of the data collection work is already done. The database functionality, platform, and output will continue to be a work in progress through, at least, FY 2007.

Information content

The general plan for the database content is:

• Project identification, analytes required and expected ranges
• Laboratory identification, method identification, and expected analyte performance
• Laboratory/method/analyte specific blind quality-control (QC) data.

Web based worksheets and batch file capabilities will be available for uploading the Water Science Center project data quality objectives (analytes, range, data needs) and the laboratory information and QC data.

Planned output

There are several obvious uses for the collected data. First, these data can be used to compare laboratory-stated capability to actual performance as part of the initial laboratory selection process. A data summary can be provided for projects just beginning the selection process that alerts the project when there are mismatches between project data quality objectives and laboratory capability or performance.

Second, selected laboratories require on-going review to assure long-term project consistency and quality of project data. The BQS can provide the projects with periodic data summaries of on-going performance. The details of the report are being developed, and will include analyte specific bias and variability analysis.

Third, existing data from selected laboratories can be reviewed by new projects looking for laboratories with specific capabilities.

There will be a user interface, located on the BQS website, where USGS personnel can query the database for their own interest in laboratory performance. Links to pertinent information such as the Standard Reference Sample data, the National Environmental Monitoring Index (NEMI), BQS inorganic blind sample data or organic blind sample data for the National Water Quality Laboratory for comparison, will be included.

Phased implementation

During fiscal year 2007, the focus is on developing the database, the web-based input forms, and assuring that the appropriate data are being collected based on pilot project feedback.

After the basic structure is built, the focus will be on database output. The summary reports assessing initial and ongoing performance should be the first tasks. Feedback from projects as to what types of reports and what kinds of data are useful will be used to develop the BQS database output.

As the scope of laboratory selection expands into other analytical types
(e.g., radiochemistry or taxonomy), the database and the input /output will be expanded and modified to accommodate the needs of different analysis types.

Attachment

*******************************************************
Timothy L. Miller, Chief, Office of Water Quality
U.S. Geological Survey
412 National Center
Reston, VA 20192
703/648-6868
703/648-5722 (fax)
tlmiller@usgs.gov
******************************************************