Policy for the Evaluation and Approval of Production Analytical Laboratories In Reply Refer To: Mail Stop 412 April 16, 1998 OFFICE OF WATER QUALITY TECHNICAL MEMORANDUM 98.03 (REVISED) Subject: Policy for the Evaluation and Approval of Production Analytical Laboratories It is Water Resources Division (WRD) policy that production analytical laboratories which provide chemical, radiochemical, and biological analyses to the Division shall be regularly reviewed, evaluated, and approved. The review process is a necessary prerequisite for: (1) storing the data in a U.S. Geological Survey (USGS) data base that is open to the public; and/or (2) publishing the data in a USGS District annual or other data report. The attachment to this memorandum updates the process--the requirements, specifications, and procedures--that will be used to implement the policy, beginning March 10, 1998. During the remainder of fiscal year (FY) 1998, reviews of production laboratories will be conducted at the request of a District, a national program, or the Office of Water Quality (OWQ). The requesting office will fund the cost of each laboratory review. In FYs 1999 and 2000, one half of the remaining production laboratories will be reviewed each year. Costs for these reviews will be paid by the office that uses or manages the laboratory. The Branch of Quality Systems (BQS) will soon develop a laboratory review schedule for FYs 1999 and 2000 so affected Districts can modify Cooperative Agreements to cover the costs of specific reviews. Beginning in FY 2001, all production laboratories will be reviewed on a 3-year cycle. The District or organizational unit responsible for use or management of the laboratory will pay the cost. The scope, complexity, and number of analyses performed for WRD by each laboratory will largely determine whether the review: (1) can be successfully accomplished, remotely, by examination of documents and telephone interviews, or (2) must be conducted onsite by Division personnel. Remote reviews will be used, wherever practical, to minimize costs. The BQS will organize and conduct the laboratory review process. Under the direction of OWQ, the BQS will develop and prioritize the list of laboratories that need review, establish the schedule, arrange the staffing of reviews, write the review reports, recommend approval or disapproval, develop necessary remedial actions, and--for each disapproved laboratory, work with the requesting WRD office to help achieve approval. BQS will discuss recommendations for approval, disapproval, and remedial actions with the OWQ. The OWQ will then send the results of the review to the organizational unit responsible for use or management of the laboratory. Please join me in ensuring the successful implementation of this policy. We must protect the quality and integrity of our data. Janice R. Ward /s/ Acting Chief, Office of Water Quality Attachments Distribution: A, B, FO, PO District Water-Quality Specialists OWQ Staff Regional Water-Quality Specialists This memorandum supersedes WRD Memorandum No. 92.35. - ------------------ Process for Evaluating and Approving Production Laboratories Which Provide Chemical, Radiochemical, and Biological Data to the WRD Introduction The purpose of this document is to establish the process for evaluating and approving production laboratories which provide chemical, radiochemical, and biological analyses to the Water Resources Division (WRD). This process is needed to ensure that analytical production data are of known and acceptable quality. The quality of all laboratory production data must be fully documented, defensible, and suitable to meet the needs of U.S. Geological Survey (USGS) programs and projects, the public, and other users. This process supports the intent of the "Strategic Plan for the U.S. Geological Survey, 1996 to 2005." As described in this Plan (U.S. Geological Survey, 1996, p. xii): Together, the USGS and its partners will . . . collect and distribute data of the highest possible quality [and] populate USGS national data bases with increasingly large quantities of reliable and timely data . . . . Each production laboratory should be reviewed and approved before WRD submits environmental samples to the laboratory. This will assure that the laboratory is capable of producing data of known and acceptable quality. Then, periodic checks and reviews are required to ensure that data quality is appropriately maintained. Responsibility for the Review Process The Branch of Quality Systems (BQS) will organize and conduct the production laboratory review process. Under the direction of Office of Water Quality (OWQ), the BQS will: o Develop and prioritize the list of laboratories to be reviewed. o Establish the review schedule. o Arrange and participate in the staffing of reviews. o Write the review reports. o Recommend approval or disapproval. o Develop necessary remedial actions. o For each disapproved laboratory work, with the requesting WRD office to help achieve approval. BQS will discuss recommendations for approval, disapproval, and remedial actions with the OWQ. The OWQ will then send the review results to the office requesting use of or responsible for managing the laboratory. If the office is a District, the Regional Hydrologist will be copied. Scope Laboratories affected by this review process include those providing chemical, radiochemical, and biological analyses for production purposes that will be: (1) stored in an USGS data base that is open to the public, and/or (2) published in an USGS district annual or other data report. The laboratory review process covers: o The National Water Quality Laboratory (NWQL). o The Quality of Water Service Unit (QWSU). o Production laboratories under contract to the NWQL. o Production laboratories under contract or Cooperative Agreements with Districts. o Production (non-research) parts of District laboratories. o Production parts of National Research Program (NRP) laboratories. Not covered by this process are laboratories whose sole purpose is to: o Develop analytical techniques. o Provide data for research purposes. o Conduct screening analyses. However, if a NRP or District research laboratory also provides routine production data, that part of its effort is covered by the policy. In 1996, the list of production laboratories used by the Division included: o The NWQL. o The QWSU. o 20 District and NRP laboratories. o 55 Cooperator and contract laboratories. If you are uncertain whether the policy requires your laboratory to be reviewed, please contact the BQS, Laboratory Evaluation Project, at 303/236-1870 x303 or x313. Schedule and Responsibility for Review Costs 1. Fiscal Years (FY) 1998-2000 For the remainder of FY 1998, reviews of production laboratories will be conducted at the request of a District, a national program, or the OWQ. The requesting office will fund the cost of each review. In FYs 1999 and 2000, one half of the remaining production laboratories will be reviewed each year. Costs for these reviews will be paid by the office that uses or manages the laboratory. The BQS will quickly develop a laboratory review schedule for FYs 1999 and 2000 so affected Districts will have time to modify Cooperative Agreements to pay the costs of specific reviews. 2. Beginning FY 2001 Beginning in FY 2001, the complete list of production laboratories will be reviewed on a 3-year cycle. The office that uses or manages the laboratory will pay the cost for each review. The scope, complexity, and number of WRD analyses performed for WRD by each laboratory will largely determine whether a review: (1) can be successfully accomplished, remotely, by examination of documents and telephone interviews, or (2) must be conducted onsite by Division personnel. Remote reviews will be used, wherever practical, to minimize costs. Laboratory Review Criteria The BQS will use the following criteria to evaluate each production analytical laboratory: 1. Appropriate, Approved, and Published Methods All production laboratories must use appropriate analytical methods approved and published by one of the following sources: USGS, U.S. Environmental Protection Agency (EPA), American Public Health Association, American Water Works Association, Water Environment Federation (Standard Methods), or ASTM (formerly American Society for Testing and Materials). The published documentation of an analytical method must include the analytical techniques, chemical processes, and expected data quality. The analytical methods from the sources listed above include a wide range of options. There typically are multiple analytical methods available for any given analyte, each with its own precision, bias, matrix recovery, and interferences. The method should be selected based on the specific objectives and needs of the project. The requesting WRD office is responsible to ensure that the method used is appropriate for the specific environmental matrix and will meet the data-quality needs of the project (see Attachment 1). If a WRD office or the production laboratory requests a method that is not included in the listed sources, that office is responsible for ensuring that the method is approved (based on requirements specified in WRD Memorandum No. 82.28) before the produced analytical data are stored in publicly-accessible data bases or published in USGS district annual or other data reports. Similarly if a laboratory has changed an approved and published method, the new method must be approved according to the specifications in WRD Memorandum No. 82.28 and then published. A method change is defined as any modification that the laboratory expects will change the quality of produced data. 2. Documented Standard Operating Procedures All analytical methods used by a production laboratory must have documented standard operating procedures (SOPs) approved in accordance with the procedures contained in the laboratory quality-assurance (QA) plan. SOPs must contain detailed information not only on how to perform a specific method, but also information related to the preparation of reagents, standards, and reference materials. Changes to SOPs must be fully documented. An SOP change is defined as any modification to the analytical process that is not expected to change the data quality. The implementation of changes to SOPs must follow the process defined in the laboratory QA plan. 3. Approved QA Plan All laboratories must have an approved QA plan that is supplied to WRD customers upon request. The laboratory QA plan must provide internal guidance and documentation that will ensure the laboratory is operating under a standardized, rigorous QA program and is producing analytical results of a known and acceptable quality. The laboratory QA plan must describe QA activities, quality-control (QC) procedures and requirements, QC performance acceptance criteria, and the required corrective actions to be taken if the criteria are not met. Erdmann (1991a) and Taylor (1987) describe the requirements, procedures, and technical information that each production laboratory QA plan should contain. 4. Types and Amount of QC Data and Historical Performance All production laboratories must document their QC procedures and provide data to the WRD offices that allows for continuous tracking of the bias, variability, and recovery over the expected range of environmental data. QC information, such as QC charts, analysis of laboratory QC samples, calibration records, and analyst bench logs should be maintained for at least 5, and preferably 10 years and be available for review by WRD customers. Laboratory QC data are to be fully documented, readily available, and technically defensible. All laboratories must maintain a file of initial performance data for each method and any changes to the method. The file will contain any SOP changes and all accompanying data that define data quality across such changes. On a yearly basis, this file will be copied and sent to the Chief, BQS. This file and all accompanying information will be reviewed as part of each laboratory review. During each review, the BQS will examine the types, amount, and quality of the QC data over at least a three-year period to evaluate the quality and consistency of operations. 5. Participation in the Standard Reference Sample Project The Standard Reference Sample (SRS) Project of WRD conducts a semi-annual round-robin intercomparison study to evaluate and document the analytical performance of laboratories used by the Division and other outside participating laboratories. Each production laboratory used by the Division must analyze, at a minimum, the appropriate SRSs to cover the constituents requested by the Division office. Although the SRS Project presently covers only dissolved inorganic and nutrient analytes, it is developing additional types of SRSs to cover additional analytes and ranges of analytes. Examples of new samples may include various organic SRSs and, also, inorganic SRSs that test a laboratory's capability to measure presence/absence rather than quantifiable results. As new SRSs become available, the SRS project will notify each production laboratory and provide ordering instructions. Acceptance criteria of SRS results are documented in Erdmann (1991b, p. 4) and Farrar (1997, p. 10). If the SRS result for one or more constituents is considered unacceptable (poor rating), the BQS will notify the appropriate WRD office. That office is then responsible to obtain a written explanation of the potential problem and any necessary corrective actions from the laboratory. This documentation will be forwarded to BQS. If two successive unacceptable results are obtained for any constituent (samples are at six-month intervals), the BQS will notify the responsible office, the appropriate Regional Hydrologist, and the OWQ that the laboratory has sustained unacceptable data for those constituents. Until successful corrective action is achieved, data from the laboratory cannot be stored in publicly-accessible USGS data bases nor published in USGS district annual or other data reports. The BQS will evaluate the results of other available performance evaluation analyses (from other external certification processes) along with the SRS results in reaching decisions for corrective action. To enhance the data-review process, each future District review by OWQ will evaluate SRS results from each production laboratory used by the District. 6. Scientific Capability of Personnel The BQS will evaluate the academic education, on-the-job training, knowledge, capabilities, and competence of the production laboratory personnel -- including the laboratory managers, supervisors, and those preparing and analyzing the samples. 7. Laboratory Equipment The BQS will evaluate whether the laboratory has the required preparatory and analytical equipment to appropriately run each method that produces USGS data. Laboratory Evaluation Process The laboratory evaluation process consists of two major activities: 1. Notification of Laboratory Use Near the beginning of each fiscal year, each District or office will be notified to provide BQS, by memorandum through the Regional Hydrologist, the estimated use of individual production analytical laboratories for the following fiscal year. This information will be due to the BQS within the second month of each fiscal year. The BQS will compile the responses, and by mid year, discuss and firm up the estimates with each office. The BQS will use the refined information to prioritize laboratories for review and to select those laboratories to be evaluated during the following fiscal year. The BQS will notify the affected Districts and offices about five months before the beginning of the fiscal year. Then, BQS will work with the Districts/offices to schedule the actual dates for each review. The NWQL and the QWSU will be evaluated annually so notification about their use is not required. Attachment 2 contains the information that BQS will request concerning use of production laboratories. 2. Reviews of Production Laboratories The BQS will conduct the laboratory reviews using the protocols, procedures, and guidelines presented in Erdmann (1991b). Each review will include evaluation of the written documentation (see Attachment 2 and below) and a conference call including BQS, the requesting WRD office, and the production laboratory. The BQS will determine the need for an on-site review based on: o The scope, complexity, and number of analyses performed by the laboratory for WRD. o The completeness and clarity of the written documentation. o Evaluation of the written documentation against the laboratory-review criteria (see Laboratory Review Criteria). Based on preliminary information or concerns, the District/office, the Regional Hydrologist, or OWQ may request an on-site review of a specific production laboratory before or during the initial BQS evaluation. The laboratory review process consists of three steps: Step 1. Review of Written Documentation. The BQS will review and evaluate the following: o Laboratory QA plan. o List of constituents and estimated number of samples requested by the WRD office. o Length of time the laboratory has analyzed for the requested constituents. o List of analytical methods that will be used for the requested constituents, the references for these methods, the principles and analytical range of the methods, and documentation of any modifications or changes to published methods. o Brief resumes of key laboratory managers and supervisors. o Summary of education level, training, and capabilities of personnel conducting the analyses. o Description of relevant major instrumentation used in providing the requested analytical services. o Historical performance evaluation ratings for the SRS intercomparison studies (if available). o Analytical results from recent performance-evaluation sample analyses from other external certification, evaluation, or round-robin studies. o Example copy of the laboratory analytical-results data and laboratory QC data that are available and/or provided to the WRD office. If both the documentation and the performance evaluation results are in order and acceptable, BQS will proceed to step 2. If important documentation is missing or unacceptable, BQS will report this finding to the requesting office and then work with that office and the laboratory to rectify the situation. If performance-evaluation data are unavailable, incomplete, or unsatisfactory, BQS will send appropriate SRSs or arrange for other performance evaluation samples to be sent to the laboratory. These SRSs or other samples must be analyzed by the laboratory and the results provided to and evaluated by the BQS, before step 1 can be completed. Step 2. Conference Call. Once the written documentation and the performance-evaluation results are reviewed and accepted, BQS will arrange a conference call with appropriate personnel in the requesting office and the production laboratory. The purpose is to ensure that adequate technical and QA practices are in place to define and document the quality of the requested analyses. The discussion will include: o Procedures used to handle documentation of laboratory records. o Sample management. o Sample analyses. o The laboratory QA program. o QC activities and information. o Data management. Examples of materials for review may be requested from the laboratory during this call. Following the conference call, if no serious problems or deficiencies are found, the BQS will approve the laboratory as accepted by the WRD for the specified analyses and methods. In this case, results from this step will be documented and forwarded by BQS to the OWQ. OWQ will forward the information to the requesting office and Regional Hydrologist within two weeks following the review. In cases where the laboratory is being requested to perform complex and/or a large number of analyses for WRD, the BQS may request further remote review or even an onsite review to protect the integrity of Division data. If the review discovers serious problems or deficiencies, the BQS will: o Provide the District/office (copy to the Regional Hydrologist) with specific suggestions on how to work with the laboratory to correct the deficiencies. At this point, if the analyses represent new work, the deficiencies must be corrected before the office submits environmental samples to the laboratory. If the laboratory is already analyzing Division environmental samples, no additional results can be stored in publicly accessible data bases or published in USGS district annual or other data reports. o Schedule an on-site laboratory review (see step 3). Once BQS notes serious problems or deficiencies and recommends corrective actions, the requesting WRD office is responsible for working with the production laboratory to achieve resolution. BQS is available to assist the WRD office and the laboratory. However, the requesting WRD office has the ultimate responsibility for ensuring that corrective actions are achievable and implemented by the laboratory. Step 3. On-Site Technical Review. BQS will determine the need for an onsite laboratory review based on evaluation the written documentation, the described laboratory-review criteria, and the performance-evaluation results. The primary determinants are: (1) the scope, complexity, and number of analyses to be performed for WRD, and (2) whether steps 1 and 2 identify serious problems or deficiencies in the production laboratory and its performance capabilities. The objectives of an onsite review are: (1) to ensure that adequate analytical and QC practices are in place and to determine--first hand--how these are used to define and document the quality of the environmental analyses; and (2) ensure that any documented problems and deficiencies have been corrected and are likely to stay under control over the duration of analysis for the WRD. Each on-site review will include: o Inspecting the laboratory facility and instrumentation. o Meeting with selected laboratory personnel. o Examining laboratory processes. o Reviewing various laboratory records and documentation associated with sample management, sample analysis, the QA program, QC activities and information, and data management. At a minimum, a technical expert representing BQS, assisted by a technical representative from the WRD office, will conduct the on-site review. If no serious problems or deficiencies are noted, the BQS will approve the laboratory as accepted by the WRD for the specific analyses and methods. If serious problems or deficiencies are found, BQS will provide the WRD office with: o Specific suggestions on what needs to be changed to remedy identified problems and deficiencies. o Suggestions on how to work with the laboratory to correct deficiencies before submitting environmental samples for analysis or entering additional analytical results into a publicly-accessible data base or publishing the data in an USGS district annual or other data report. Results from this step (acceptance or non-acceptance) will be documented with explanations by BQS and sent to the OWQ. The OWQ will forward the information to the requesting WRD office and the Regional Hydrologist within four weeks after the completion of the onsite review. The WRD office requesting or using the laboratory must work with the laboratory to ensure appropriate corrective actions are implemented. Then the office must document to BQS what actions have been taken. BQS will then re-evaluate the laboratory. If everything is acceptable, BQS will approvethe laboratory as accepted by the WRD for the specific analyses and methods. If the response is inadequate or unclear, BQS will conduct additional review through further evaluation of data and documents, conference calls, or another on-site review. The WRD office using the laboratory has the responsibility to ensure that the corrective actions are implemented by the laboratory. If The Production Laboratory Refuses to Cooperate If a laboratory requested by a WRD office declines to be reviewed, to analyze submitted SRSs or other performance-evaluation samples, or to make needed changes relative to identified problems or deficiencies, the WRD office cannot use the laboratory. Temporary Exemption from the Laboratory Evaluation Process If a WRD office must begin sampling and using the analytical results from a new laboratory before a technical review can be completed by BQS (because of funding-agreement, data-collection, and other needs) the Regional Hydrologist (or designee not from the requesting office) may grant temporary approval based on available knowledge or information about the laboratory. In this situation, the analytical results from the production laboratory are considered provisional until BQS evaluates and approves the laboratory to conduct specific analyses for WRD. To protect the Division's scientific integrity and data quality, this practice should seldomly be used. The WRD office requesting this exception must provide the Regional Hydrologist with as much of the information listed in Attachment 2 as possible, so the competency of the laboratory can be briefly evaluated in advance. The WRD office must also inform BQS that it has requested an exception, along with a scheduled date for participating in the normal approval process. BQS will attempt to honor schedules for reviewing excepted laboratories. Temporary approvals granted by a Regional Hydrologist may not exceed 9 months, and can be granted only once per laboratory. Because this approach is outside the normal technical-review process, a statement of temporary approval by the Regional Hydrologist (or designee) must be included in all USGS-sponsored data reports containing resultant analytical data until final approval is granted. In addition, the resultant data cannot be entered into a publicly-accessible USGS data base. Costs Costs for each review will be funded on a fee-for-service basis. Charges will include salary, travel, and other costs directly associated with performing the reviews. Reviews of the NWQL, the QWSU, and production parts of NRP and District research laboratories will be funded by those laboratories and the costs recovered through analytical charges to customers. Costs associated with technical review of contractor and cooperator laboratories will be funded by the WRD offices requesting use of the laboratories. Costs of reviewing District production laboratories will be paid by the Districts. Technical Support services provided by BQS will include staff support for: o Managing the laboratory review process. o Providing technical assistance to offices participating in the process. o Reporting and documenting the overall usage and performance of production laboratories to the Division. References Cited Erdmann, D.E., 1991a, Quality assurance requirements for water-quality laboratories providing analytical services for the Water Resources Division of the U.S. Geological Survey: U.S. Geological Survey Open-File Report 91-222, 8 p. Erdmann, D.E., 1991b, Technical review of water-quality laboratories providing analytical services for the Water Resources Division of the U.S. Geological Survey: U.S. Geological Survey Open-File Report 91-223, 19 p. Farrar, J.W., 1997, Results of the U.S. Geological Survey's analytical evaluation program for standard reference samples: T-147 (trace constituents), T-149 (trace constituents), M-142 (major constituents), N-53 (nutrient constituents), N-54 (nutrient constituents), P-28 (low ionic strength constituents), GW-1 (ground-water constituents), and Hg-24 (mercury) distributed in April 1997: U.S. Geological Survey Open-File Report 97-553, 184 p. Keith, L.N., 1991, Environmental Sampling and Analysis--A Practical Guide: Chelsea, Michigan, Lewis Publishers, Inc. Shampine, W.J., Pope, L.M., and Coterie, M.T., 1992, Integrating quality assurance in project work plans of the U.S. Geological Survey: U.S. Geological Survey Open-File Report 92-162, 12 p. Taylor, J.K., 1987, Quality Assurance of Chemical Measurements: Chelsea, Michigan, Lewis Publishing, Inc. U.S. Geological Survey, 1996, Strategic plan for the U.S. Geological Survey, 1996 to 2005: U.S. Geological Survey Office of the Director. Attachments Attachment 1: Project Specific Analytical Data Needs Attachment 2: Production Laboratory-Use Information - --------------------- ATTACHMENT 1 -- Project Specific Analytical Data Needs Division offices are responsible to secure appropriate analytical services and to ensure that production laboratories implement appropriate QA procedures that meet the objectives of each project. Laboratory QA procedures and QC data related to the project should be referenced or included in the project work plan or the QA project plan developed at the outset of each project. The project chief and technical specialists then evaluate the QC information generated under the plans to ensure the ongoing suitability and quality of all data--including laboratory analyses--to meet the stated data-quality objectives. Information about integrating QA/QC in project work plans and QA plans for projects is provided in Shampine and others, 1992, Keith (1991), and WRD Memorandum No. 94.17. Each project should also consider using various types of field-based QC samples (blanks, spikes, reference and replicate samples) to help assess the performance of the analytical laboratories. The BQS will assist District, Regional, and other WRD offices in selection of analytical services and development of QA/QC plans. BQS will either provide or arrange for assistance based on discussions with the requestor. If requested, BQS will also review laboratory QC data for specific projects. - --------------------------- ATTACHMENT 2 -- Production Laboratory-Use Information WRD offices will be responsible for annually notifying the BQS about the use of analytical production laboratories for obtaining chemical, radiochemical and biological data that will be placed in publicly-accessible data bases or published in data reports. Two types of information are required: (1) general laboratory use; and (2) constituent specific laboratory use. BQS will use both types to inventory production laboratory use in the WRD, help prioritize and schedule the laboratory reviews, and conduct preliminary screening of the laboratories. The BQS, OWQ, and Regional Water-Quality Specialists will develop and implement a ranking priority system based on the different types and categories of laboratory-use information. To assist the WRD offices in compiling the needed information, BQS will develop a standardized information request that covers the following items: General Laboratory-Use Information: 1. Name and address of laboratory. 2. Type of samples that will be analyzed by the laboratory. 3. Estimated number of samples that will be analyzed in the fiscal year. 4. Estimated cost of laboratory analytical work for the fiscal year. 5. List month/year and brief summary of findings of previous technical reviews of laboratory by WRD. 6. List month/year and historical laboratory performance ratings for SRS intercomparison studies. 7. List type, month/year, and results of any other performance-evaluation studies. 8. Does the laboratory have a documented and complete QA plan and standard operating procedures? 9. Does the WRD project using the laboratory include appropriate laboratory QA and QC information in the project work plan or project QA plan (see Attachment 1)? 10. The type and collection frequency of any field-based QC samples being collected by the WRD project using the laboratory. 11. Brief resumes of key laboratory managers and supervisors. Constituent-Specific Laboratory-Use Information This information will need to be completed for each constituent or group of constituents to be analyzed by the laboratory. 1. Constituent(s) to be analyzed. 2. Length of time (years) the laboratory has analyzed the constituent(s). 3. Planned length of time (years) that the laboratory will be requested to analyze constituent(s) for the WRD office. 4. List source or reference and the name for each analytical method to be used for WRD work. 5. Type, model, and age of instruments used for each analysis. 6. Education level, training, and capabilities of personnel running each method. 7. List month/year and summary of results for the latest SRS study for the constituent(s). ------- End of Forwarded Message