The Quality of Laboratory Testing Today
To assess the analytic quality of laboratory testing in the United States, we obtained proficiency testing survey results from several national programs that comply with Clinical Laboratory Improvement Amendments (CLIA) regulations. We studied regulated tests (cholesterol, glucose, calcium, fibrinogen, and prothrombin time) and nonregulated tests (international normalized ratio [INR], glycohemoglobin, and prostate-specific antigen [PSA]). Quality was assessed on the σ scale with a benchmark for minimum process performance of 3 σ and a goal for world-class quality of 6 σ. Based on the CLIA criteria for acceptable performance in proficiency testing (allowable total errors [TEa]), the national quality of cholesterol testing (TEa = 10%) estimated σ values as 2.9 to 3.0; glucose (TEa = 10%), 2.9 to 3.3; calcium (TEa = 1.0 mg/dL), 2.8 to 3.0; prothrombin time (TEa = 15%), 1.8; INR (TEa = 20%), 2.4 to 3.5; fibrinogen (TEa = 20%), 1.8 to 3.2; glycohemoglobin (TEa = 10%), 1.9 to 2.6; and PSA (TEa = 10%), 1.2 to 1.8. The analytic quality of laboratory tests requires improvement in measurement performance and more intensive quality control monitoring than the CLIA minimum of 2 levels per day.
What is the quality of laboratory tests today? Studies of laboratory errors have documented that a higher percentage of errors occur in the preanalytic and postanalytic processes than in analytic processes. The figures often quoted are 45% for errors in preanalytic processes, 10% for analytic errors, and 45% for postanalytic errors (actual estimates, 45.5%, 7.3%, and 47.2%, respectively) based on a study done in 1988 before the implementation of the current Clinical Laboratory Improvement Amendments (CLIA) regulations. As a consequence of this expected distribution of errors, laboratories are urged to focus their attention on preanalytic and postanalytic processes to improve patient safety.
The final CLIA rule reflects this emphasis on increased quality assessment for preanalytic and postanalytic processes and proposes a reduction in quality control (QC) for analytic processes. This proposal for reducing QC is not found in the regulations but in the State Operations Manual (SOM), which provides interpretive guidelines for implementing the regulations. According to the SOM, laboratories may be able to reduce QC from 2 levels per day to 2 levels per week or even 2 levels per month for measurement procedures and instruments with built-in controls. These so-called equivalent QC (EQC) procedures would be particularly attractive for point-of-care testing applications in which operators often have little laboratory experience and minimum analytic skills.
Is the analytic quality of laboratory tests really so good that only weekly or monthly QC is needed? There are few studies that document current analytic performance relative to the quality required for medical usefulness. The Centers for Medicare & Medicaid Services' (CMS's) own data from laboratory inspections show that as many as 5% to 10% of laboratories are deficient in QC practices, which should raise concerns about the analytic quality achieved. Under CLIA, laboratories also must participate in proficiency testing (PT) surveys, which provide another source of data about the quality of laboratory testing. These data make it possible to provide a more quantitative assessment of the "state of the art" of laboratory testing.
For assessing quality in other industries, Six Sigma Quality Management is gaining momentum as the best approach for providing objective estimates and metrics. Six Sigma requires that tolerance limits be defined for good quality to objectively identify poor quality or defective products (or erroneous test results). Two methods exist, one that inspects process outcome and counts the defects, calculates a defect rate per million, and uses a statistical table to convert defect rate per million to a σ metric. The second makes use of estimates of process variation to predict process performance by calculating a σ metric from the defined tolerance limits and the variation observed for the process. The first method is applicable to preanalytic and postanalytic processes, whereas the second method is particularly suitable for analytic processes in which the precision and accuracy can be determined by experimental procedures. Nevalainen et al demonstrated the application of Six Sigma concepts for characterizing the quality of preanalytic and postanalytic processes on the σ scale. Applications to analytic processes have been described by Westgard.
When assessing quality on the σ scale, the higher the σ metric, the better the quality. According to Nevalainen et al, "average products, regardless of their complexity, have a quality performance value of about 4 σ. The best, or 'world class quality,' products have a level of performance of 6 σ." This corresponds to a process capability index of 2.0, which also has been the goal for industrial production processes. Industry recommends a minimum acceptable process capability of 1.0, which would correspond to a 3 σ process. Thus, common goals across industries are to strive for 6 σ quality and accept a minimum of 3 σ quality. As might be expected, the size of analytic errors that need to be detected by QC will depend on the process capability; therefore, the σ metric also is useful for assessing the adequacy of QC procedures and practices. Thus, with the aid of Six Sigma principles and metrics, it is possible to assess the quality of laboratory testing processes and the QC that is needed to ensure that the desired quality is achieved.
To assess the analytic quality of laboratory testing in the United States, we obtained proficiency testing survey results from several national programs that comply with Clinical Laboratory Improvement Amendments (CLIA) regulations. We studied regulated tests (cholesterol, glucose, calcium, fibrinogen, and prothrombin time) and nonregulated tests (international normalized ratio [INR], glycohemoglobin, and prostate-specific antigen [PSA]). Quality was assessed on the σ scale with a benchmark for minimum process performance of 3 σ and a goal for world-class quality of 6 σ. Based on the CLIA criteria for acceptable performance in proficiency testing (allowable total errors [TEa]), the national quality of cholesterol testing (TEa = 10%) estimated σ values as 2.9 to 3.0; glucose (TEa = 10%), 2.9 to 3.3; calcium (TEa = 1.0 mg/dL), 2.8 to 3.0; prothrombin time (TEa = 15%), 1.8; INR (TEa = 20%), 2.4 to 3.5; fibrinogen (TEa = 20%), 1.8 to 3.2; glycohemoglobin (TEa = 10%), 1.9 to 2.6; and PSA (TEa = 10%), 1.2 to 1.8. The analytic quality of laboratory tests requires improvement in measurement performance and more intensive quality control monitoring than the CLIA minimum of 2 levels per day.
What is the quality of laboratory tests today? Studies of laboratory errors have documented that a higher percentage of errors occur in the preanalytic and postanalytic processes than in analytic processes. The figures often quoted are 45% for errors in preanalytic processes, 10% for analytic errors, and 45% for postanalytic errors (actual estimates, 45.5%, 7.3%, and 47.2%, respectively) based on a study done in 1988 before the implementation of the current Clinical Laboratory Improvement Amendments (CLIA) regulations. As a consequence of this expected distribution of errors, laboratories are urged to focus their attention on preanalytic and postanalytic processes to improve patient safety.
The final CLIA rule reflects this emphasis on increased quality assessment for preanalytic and postanalytic processes and proposes a reduction in quality control (QC) for analytic processes. This proposal for reducing QC is not found in the regulations but in the State Operations Manual (SOM), which provides interpretive guidelines for implementing the regulations. According to the SOM, laboratories may be able to reduce QC from 2 levels per day to 2 levels per week or even 2 levels per month for measurement procedures and instruments with built-in controls. These so-called equivalent QC (EQC) procedures would be particularly attractive for point-of-care testing applications in which operators often have little laboratory experience and minimum analytic skills.
Is the analytic quality of laboratory tests really so good that only weekly or monthly QC is needed? There are few studies that document current analytic performance relative to the quality required for medical usefulness. The Centers for Medicare & Medicaid Services' (CMS's) own data from laboratory inspections show that as many as 5% to 10% of laboratories are deficient in QC practices, which should raise concerns about the analytic quality achieved. Under CLIA, laboratories also must participate in proficiency testing (PT) surveys, which provide another source of data about the quality of laboratory testing. These data make it possible to provide a more quantitative assessment of the "state of the art" of laboratory testing.
For assessing quality in other industries, Six Sigma Quality Management is gaining momentum as the best approach for providing objective estimates and metrics. Six Sigma requires that tolerance limits be defined for good quality to objectively identify poor quality or defective products (or erroneous test results). Two methods exist, one that inspects process outcome and counts the defects, calculates a defect rate per million, and uses a statistical table to convert defect rate per million to a σ metric. The second makes use of estimates of process variation to predict process performance by calculating a σ metric from the defined tolerance limits and the variation observed for the process. The first method is applicable to preanalytic and postanalytic processes, whereas the second method is particularly suitable for analytic processes in which the precision and accuracy can be determined by experimental procedures. Nevalainen et al demonstrated the application of Six Sigma concepts for characterizing the quality of preanalytic and postanalytic processes on the σ scale. Applications to analytic processes have been described by Westgard.
When assessing quality on the σ scale, the higher the σ metric, the better the quality. According to Nevalainen et al, "average products, regardless of their complexity, have a quality performance value of about 4 σ. The best, or 'world class quality,' products have a level of performance of 6 σ." This corresponds to a process capability index of 2.0, which also has been the goal for industrial production processes. Industry recommends a minimum acceptable process capability of 1.0, which would correspond to a 3 σ process. Thus, common goals across industries are to strive for 6 σ quality and accept a minimum of 3 σ quality. As might be expected, the size of analytic errors that need to be detected by QC will depend on the process capability; therefore, the σ metric also is useful for assessing the adequacy of QC procedures and practices. Thus, with the aid of Six Sigma principles and metrics, it is possible to assess the quality of laboratory testing processes and the QC that is needed to ensure that the desired quality is achieved.