Relationship of quality goals and measurement performance to the selection of quality control procedures for multi-channel haematology analysers

An approach is described for selecting QC procedures based on goals for analytical quality, the performance characteristics of the measurement procedure (imprecision, bias, frequency of errors), and the performance characteristics of the control procedure (probabilities for error detection and false rejection). Performance characteristics of stable sample QC procedures and patient data QC procedures (retained patient specimens, Bull's single-rule algorithm, Bull's multi-rule algorithm) are compared to determine when to apply these different QC procedures to multichannel hematology analysers. Precise, stable methods may be controlled using Bull's single-rule algorithm; less precise, less stable methods require Bull's multi-rule algorithm, retained patient specimens, or stable sample QC procedures; imprecise and unstable methods are best controlled using stable sample QC procedures. "Multi-stage" designs may employ stable samples for "startup" testing, retained patient specimens for short term monitoring, and Bull's single-rule algorithm for long term monitoring.

Models for combining random and systematic errors. assumptions and consequences for different models

A series of models for handling and combining systematic and random variations/errors are investigated in order to characterize the different models according to their purpose, their application, and discuss their flaws with regard to their assumptions. The following models are considered 1. linear model, where the random and systematic elements are combined according to a linear concept (TE = absolute value(bias) + z x sigma), where TE is total error, bias is the systematic error component, sigma is the random error component (standard deviation or coefficient of variation) and z is the probability factor; 2. squared model with two sub-models of which one is the classical statistical variance model and the other is the GUM (Guide to Uncertainty in Measurements) model for estimating uncertainty of a measurement; 3. combined model developed for the estimation of analytical quality specifications according to the clinical consequences (clinical outcome) of errors. The consequences of these models are investigated by calculation of the functions of transformation of bias into imprecision according to the assumptions and model calculations. As expected, the functions turn out to be rather different with considerable consequences for these types of transformations. It is concluded that there are at least three models for combining systematic and random variation/errors, each created for its own specific purpose, with its own assumptions and resulting in considerably different results. These models should be used according to their purposes.

Electronic quality control, the total testing process, and the total quality control system

Traditional statistical quality control (QC) using matrix controls is often difficult to implement in point-of-care settings. Alternative QC procedures, such as electronic QC, have been developed by many manufacturers and approved for use by regulatory and accreditation organizations. Electronic QC usually involves the substitution of an electrical signal for the signal that would normally be generated by a sensor responding to an analyte in a specimen; sometimes an artificial nonliquid sample is substituted to cause the sensor to generate an electrical signal. The usefulness of electronic QC can be assessed by identifying the steps in the total testing process that are being monitored. An example is provided for blood gas measurements to illustrate the steps that can be monitored by different types of QC procedures and materials. The need to monitor all of the steps generally requires a combination of procedures and materials, or a total QC system that includes electronic QC, matrix controls, and even real patient specimens. Electronic QC is an essential part of the total QC system, but is not by itself sufficient.

The need for a system of quality standards for modern quality management

The management of analytical quality depends on the careful evaluation of the imprecision (uncertainty) and inaccuracy (trueness) of laboratory methods and the application of statistical quality control procedures to detect medically important analytical errors that may occur during routine analysis. A system of quality standards is recommended to incorporate different types of requirements, such as clinical outcome criteria, analytical outcome criteria, and analytical performance criteria. For practical applications, all need to be translated into operating specifications for the imprecision, inaccuracy, control rules, and number of control measurements that are necessary to assure analytical quality during routine production of test results.

Comparison of NCEP performance specifications for triglycerides, HDL-, and LDL-cholesterol with operating specifications based on NCEP clinical and analytical goals

The National Cholesterol Education Program (NCEP) performance specifications for methods that measure triglycerides, HDL-cholesterol, and LDL-cholesterol have been evaluated by deriving operating specifications from the NCEP analytical total error requirements and the clinical requirements for interpretation of the tests. We determined the maximum imprecision and inaccuracy that would be allowable to control routine methods with commonly used single and multirule quality-control procedures having 2 and 4 control measurements per run, and then compared these estimates with the NCEP guidelines. The NCEP imprecision specifications meet the operating imprecision necessary to assure meeting the NCEP clinical quality requirements for triglycerides and HDL-cholesterol but not for LDL-cholesterol. More importantly, the NCEP imprecision specifications are not adequate to assure meeting the NCEP analytical total error requirements for any of these three tests. Our findings indicate that the NCEP recommendations fail to adequately consider the quality-control requirements necessary to detect medically important systematic errors.

Quality control for qualitative assays: quantitative QC procedure designed to assure analytical quality required for an ELISA of hepatitis B surface antigen

An assay for hepatitis B surface antigen (HBsAg) should reliably detect 0.2 microgram/L, the lowest reported concentration in an asymptomatic blood donor. The difference between this concentration and the assay cutoff defines the analytical quality requirement in a total error format. The design of a statistical QC procedure is critically dependent on the precision of the assay. The precision of a developmental ELISA of HBsAg under study ranged from 17.5% to 9.6% for controls containing 0.07 to 1.50 micrograms/L, respectively. Use of one positive control with the 1(3s), QC rule provided an 85% chance of detecting a critical loss of assay sensitivity; use of two positive controls increased the chance of detecting critical loss of assay sensitivity to nearly 100%. These rules are based on the precision of this developmental assay, and must be developed individually for other assays. The development of the proposed QC procedures illustrates how quantitative QC can be provided for qualitative assays.

QC Validator 2.0: a computer program for automatic selection of statistical QC procedures for applications in healthcare laboratories

A computer program has been developed to help healthcare laboratories select statistical control rules and numbers of control measurements that will assure the quality required by clinical decision interval criteria or analytical total error criteria. The program (QC Validator 2.0 (QC Validator and OPSpecs are registered trademarks of Westgard Quality Corporation, which has applied for a patent for this automatic QC selection process. Windows is a registered trademark of Microsoft Corporation)) runs on IBM compatible personal computers operating under Windows. The user enters information about the method imprecision, inaccuracy, and expected frequency of errors, defines the quality required in terms of a medically important change (clinical decision interval) or an analytical allowable total error, then initiates automatic selection by indicating the number of control materials that are to be analyzed (1, 2, or 3). The program returns with a chart of operating specifications (OPSpecs chart) that displays the selected control rules and numbers of control measurements. The automatic QC selection process is based on user editable criteria for the types of control rules that can be implemented by the laboratory, total numbers of control measurements that are practical, maximum levels of false rejections that can be tolerated and minimum levels of error detection that are acceptable for detection of medically important systematic or random errors.

Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control

Achieving high quality and high productivity with automated testing processes will require process control systems that are optimized for the necessary error detection, minimum false rejection, and maximum run length. This study investigates whether run length could be monitored by average of normals (AON) algorithms that truncate the patient test distribution and estimate the average of a suitable number of patient results. The design of AON algorithms for individual analytes is facilitated by computer-simulated power curves that consider the ratio of the population biological variation (Spop) to the test method variation (Smeas), represent a range of Spop/Smeas ratios from 2 to 15, and include numbers of patient test results from 10 to 600. The potential applications of AON algorithms are assessed for 38 tests whose quality requirements represent the total error criteria from the Ontario Medical Association Laboratory Proficiency Testing Program, Spop/Smeas ratios from 0 to 32, critical systematic shifts from 0.02 to 10.85 Smeas, and test workloads representative of a regional reference laboratory. Approximately half of these tests provide high potential for applying AON algorithms to monitor run length.

Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control

Achieving high quality and high productivity with automated testing processes will require process control systems that are optimized for the necessary error detection, minimum false rejection, and maximum run length. This study investigates whether run length could be monitored by average of normals (AON) algorithms that truncate the patient test distribution and estimate the average of a suitable number of patient results. The design of AON algorithms for individual analytes is facilitated by computer-simulated power curves that consider the ratio of the population biological variation (Spop) to the test method variation (Smeas), represent a range of Spop/Smeas ratios from 2 to 15, and include numbers of patient test results from 10 to 600. The potential applications of AON algorithms are assessed for 38 tests whose quality requirements represent the total error criteria from the Ontario Medical Association Laboratory Proficiency Testing Program, Spop/Smeas ratios from 0 to 32, critical systematic shifts from 0.02 to 10.85 Smeas, and test workloads representative of a regional reference laboratory. Approximately half of these tests provide high potential for applying AON algorithms to monitor run length.

Error budgets for quality management--practical tools for planning and assuring the analytical quality of laboratory testing processes

Analytical quality is often assumed, rather than being assured or guaranteed. Given that it is still essential that laboratories produce reliable test results, managers must continue to improve their skills in analytical quality management. This paper shows managers how to use error budgets and charts of operating specifications (¿OPSpecs¿ charts) to select appropriate control rules and numbers of control measurements, taking into account the analytical or clinical quality required for a test and the imprecision and inaccuracy observed for a method. With currently available tools and a little practice, quality control (QC) procedures can be selected quickly and easily, in just 1 minute or less. Future technology is expected to automate the QC selection process and provide dynamic quality control.

Laboratory precision performance: state of the art versus operating specifications that assure the analytical quality required by clinical laboratory improvement amendments proficiency testing

OBJECTIVE

To estimate the percent of laboratories with precision performance sufficient to satisfy the operating specifications and guarantee the quality required by the proficiency testing criteria defined by the Clinical Laboratory Improvement Amendments of 1988 (CLIA).

DESIGN

Cumulative distributions that describe state-of-the-art laboratory imprecision were obtained for 1500 laboratories participating in the 1990 College of American Pathologists Quality Assurance Service. Allowable imprecision was estimated from the x-intercepts of charts of operating specifications prepared for commonly used single and multirule quality control procedures having two to four control measurements per run.

MAIN OUTCOME MEASURE

The derived values for allowable imprecision were imposed on the cumulative distributions to obtain graphical estimates of the percent of laboratories satisfying the operating specifications.

RESULTS

Up to 28% of laboratories achieve the imprecision allowable for albumin, up to 64% for total bilirubin, 52% for calcium, 35% for chloride, 48% for cholesterol, 28% for cortisol, 84% for creatinine, 9% for digoxin, 61% for glucose, 64% for high-density lipoprotein cholesterol, 88% for hemoglobin, 95% for potassium, 66% for total protein, 18% for sodium, 29% for thyroxine, 87% for triglycerides, 35% for urea nitrogen, and 81% for uric acid.

CONCLUSION

Improvements in precision are still needed for many laboratory tests to assure the analytical quality required by the CLIA proficiency testing total error criteria.

Matrix effects and the performance and selection of quality-control procedures to monitor PO2 measurements

We have assessed how variation in the matrix of control materials would affect error detection and false-rejection characteristics of quality-control (QC) procedures used to monitor PO2 in blood gas measurements. To determine the expected QC performance, we generated power curves for S(mat)/S(meas) ratios of 0.0-4.0. These curves were used to estimate the probabilities of rejecting analytical runs having medically important errors, calculated from the quality required by the CLIA '88 proficiency testing criterion and the precision and accuracy expected for a typical analytical system. When S(mat)/S(meas) ratios are low, the effects of matrix on QC performance are not serious, permitting selections of QC procedures based on simple power curves for a single component of variation. As S(mat)/S(meas) ratios increase, single-rule procedures generally show a loss in error detection, whereas multirule procedures, including the 3(1)s control rule, show an increase in false rejections. An optimized QC design is presented.

Matrix effects and the performance and selection of quality-control procedures to monitor PO2 measurements

We have assessed how variation in the matrix of control materials would affect error detection and false-rejection characteristics of quality-control (QC) procedures used to monitor PO2 in blood gas measurements. To determine the expected QC performance, we generated power curves for S(mat)/S(meas) ratios of 0.0-4.0. These curves were used to estimate the probabilities of rejecting analytical runs having medically important errors, calculated from the quality required by the CLIA '88 proficiency testing criterion and the precision and accuracy expected for a typical analytical system. When S(mat)/S(meas) ratios are low, the effects of matrix on QC performance are not serious, permitting selections of QC procedures based on simple power curves for a single component of variation. As S(mat)/S(meas) ratios increase, single-rule procedures generally show a loss in error detection, whereas multirule procedures, including the 3(1)s control rule, show an increase in false rejections. An optimized QC design is presented.

A method evaluation decision chart (MEDx chart) for judging method performance

OBJECTIVE

To introduce a new graphical tool that improves the process of making decisions about method performance.

DATA SOURCES

Scientific literature, mathematical models, and the author's experience.

STUDY SELECTION

Not applicable.

DATA EXTRACTION

Not applicable.

DATA SYNTHESIS

Relationship to charts of operating specifications (OPSpecs charts) provides guidance for classification of performance as poor, marginal, good, or excellent.

CONCLUSION

Objective decisions on the performance of methods can be made quickly and easily with the aid of a Method Evaluation Decision Chart.

Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria

The allowable imprecision for laboratory tests has been estimated from criteria based on clinical and analytical test outcome. The analytical outcome criteria studied are the Clinical Laboratory Improvement Amendments (CLIA) criteria for proficiency testing. The clinical outcome criteria are estimates of medically significant changes in test results taken from a study in the literature. The estimates of allowable imprecision were obtained from quality-planning models that relate test outcome criteria to the allowable amount of imprecision and inaccuracy and to the quality control that is necessary to assure achievement of the desired outcome criteria in routine operation. These operating specifications for imprecision are consistently more demanding (require lower CVs) than the medically useful CVs originally recommended in the literature because the latter do not properly consider within-subject biological variation. In comparing estimates of allowable imprecision, the CLIA outcome criteria are more demanding than the clinical outcome criteria for aspartate aminotransferase (asymptomatic patients), cholesterol, creatinine (asymptomatic patients), glucose, thyroxine, total protein, urea nitrogen, hematocrit, and prothrombin time. The clinical outcome criteria are more demanding for bilirubin (acute illness), iron, potassium, urea nitrogen (acute illness), and leukocyte count. The estimates of allowable imprecision from analytical and clinical outcome criteria overlap for aspartate aminotransferase (acute illness), bilirubin (asymptomatic patients), calcium, creatinine (acute illness), sodium, triglyceride, and hemoglobin.

Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria

The allowable imprecision for laboratory tests has been estimated from criteria based on clinical and analytical test outcome. The analytical outcome criteria studied are the Clinical Laboratory Improvement Amendments (CLIA) criteria for proficiency testing. The clinical outcome criteria are estimates of medically significant changes in test results taken from a study in the literature. The estimates of allowable imprecision were obtained from quality-planning models that relate test outcome criteria to the allowable amount of imprecision and inaccuracy and to the quality control that is necessary to assure achievement of the desired outcome criteria in routine operation. These operating specifications for imprecision are consistently more demanding (require lower CVs) than the medically useful CVs originally recommended in the literature because the latter do not properly consider within-subject biological variation. In comparing estimates of allowable imprecision, the CLIA outcome criteria are more demanding than the clinical outcome criteria for aspartate aminotransferase (asymptomatic patients), cholesterol, creatinine (asymptomatic patients), glucose, thyroxine, total protein, urea nitrogen, hematocrit, and prothrombin time. The clinical outcome criteria are more demanding for bilirubin (acute illness), iron, potassium, urea nitrogen (acute illness), and leukocyte count. The estimates of allowable imprecision from analytical and clinical outcome criteria overlap for aspartate aminotransferase (acute illness), bilirubin (asymptomatic patients), calcium, creatinine (acute illness), sodium, triglyceride, and hemoglobin.

European specifications for imprecision and inaccuracy compared with operating specifications that assure the quality required by US CLIA proficiency-testing criteria

Proposed and interim European quality specifications for imprecision and inaccuracy have been compared with the US CLIA total error criteria for proficiency testing (PT). To assess the relative demands of separate imprecision and inaccuracy specifications vs total error criteria, we derived the imprecision and inaccuracy that would be allowable if a testing process were to provide 90% assurance of achieving the analytical quality required by CLIA PT criteria. Charts of operating specifications (OPSpecs charts) were prepared for commonly used single-rule and multi-rule quality control procedures with 2 and 4 control measurements per run. Of the 23 tests studied, the proposed European specifications for imprecision and inaccuracy were more demanding than the CLIA requirements for 12 tests (albumin, alkaline phosphatase, amylase, calcium, chloride, creatinine, lactate dehydrogenase, lithium, magnesium, total protein, sodium, and thyroxine). The CLIA total error criteria were more demanding than the proposed European specifications for nine tests (alanine aminotransferase, aspartate aminotransferase, total bilirubin, cholesterol, creatine kinase, iron, triglycerides, uric acid, and urea nitrogen). Two tests (glucose, potassium) showed different requirements at different decision levels. Manufacturers and laboratory analysts need to compare these different quality specifications on a test-by-test basis to guide the development, selection, evaluation, and control of laboratory measurement procedures.

European specifications for imprecision and inaccuracy compared with operating specifications that assure the quality required by US CLIA proficiency-testing criteria

Proposed and interim European quality specifications for imprecision and inaccuracy have been compared with the US CLIA total error criteria for proficiency testing (PT). To assess the relative demands of separate imprecision and inaccuracy specifications vs total error criteria, we derived the imprecision and inaccuracy that would be allowable if a testing process were to provide 90% assurance of achieving the analytical quality required by CLIA PT criteria. Charts of operating specifications (OPSpecs charts) were prepared for commonly used single-rule and multi-rule quality control procedures with 2 and 4 control measurements per run. Of the 23 tests studied, the proposed European specifications for imprecision and inaccuracy were more demanding than the CLIA requirements for 12 tests (albumin, alkaline phosphatase, amylase, calcium, chloride, creatinine, lactate dehydrogenase, lithium, magnesium, total protein, sodium, and thyroxine). The CLIA total error criteria were more demanding than the proposed European specifications for nine tests (alanine aminotransferase, aspartate aminotransferase, total bilirubin, cholesterol, creatine kinase, iron, triglycerides, uric acid, and urea nitrogen). Two tests (glucose, potassium) showed different requirements at different decision levels. Manufacturers and laboratory analysts need to compare these different quality specifications on a test-by-test basis to guide the development, selection, evaluation, and control of laboratory measurement procedures.

Cholesterol--a model system to relate medical needs with analytical performance

The evolution of cholesterol testing provides an example of a systematic approach that developed to relate the medical use of a laboratory test with the analytical performance requirements for that test. Laboratories today have the capability to perform cholesterol testing with the accuracy and precision necessary to meet medical needs. This statement can be made because (a) a standard diagnostic process has been established by the National Cholesterol Education Program; (b) an accuracy base is provided through a reference method that is readily available to manufacturers and laboratories; (c) the precision of analytical systems has been improved by manufacturers; (d) operating specifications for all such systems can be established, with statistical quality-control rules to ensure adequate within-run method performance; and (e) analytical performance is monitored by proficiency testing by using national quality requirements defined by CLIA '88 for acceptability. This cholesterol model provides a logical and scientific approach that should be applicable with other analytes to assure that the analytical performance of the laboratory test satisfies medical needs.

Cholesterol--a model system to relate medical needs with analytical performance

The evolution of cholesterol testing provides an example of a systematic approach that developed to relate the medical use of a laboratory test with the analytical performance requirements for that test. Laboratories today have the capability to perform cholesterol testing with the accuracy and precision necessary to meet medical needs. This statement can be made because (a) a standard diagnostic process has been established by the National Cholesterol Education Program; (b) an accuracy base is provided through a reference method that is readily available to manufacturers and laboratories; (c) the precision of analytical systems has been improved by manufacturers; (d) operating specifications for all such systems can be established, with statistical quality-control rules to ensure adequate within-run method performance; and (e) analytical performance is monitored by proficiency testing by using national quality requirements defined by CLIA '88 for acceptability. This cholesterol model provides a logical and scientific approach that should be applicable with other analytes to assure that the analytical performance of the laboratory test satisfies medical needs.

Analytical Goal-Setting for Monitoring Patients When Two Analytical Methods are Used

Serial results from an individual are often obtained using more than one method. Results should be transferable over time and locale. Every method has inherent analytical error, and goals are required to delineate the maximum allowable random (imprecision) and systematic (inaccuracy, bias) errors to facilitate optimal patient care. Based on Harris's proposal [Am J Clin Pathol 1979;72:374-82] that desirable imprecision should be less than or equal to one-half the within-subject biological variation, if the methods have negligible imprecision, then the maximum allowable bias between two methods used for monitoring is one-third of the within-subject biological variation. A more general model has been developed that relates the analytical imprecisions of two methods, and the bias between them, to biological variation. Applying the general formula derived in specific clinical monitoring situations in which a known change in serial results (occurring at a stated probability) stimulates clinical action allows goals for the imprecisions of the two methods and allows the difference in bias between them to be determined quantitatively.

Analytical goal-setting for monitoring patients when two analytical methods are used

Serial results from an individual are often obtained using more than one method. Results should be transferable over time and locale. Every method has inherent analytical error, and goals are required to delineate the maximum allowable random (imprecision) and systematic (inaccuracy, bias) errors to facilitate optimal patient care. Based on Harris's proposal [Am J Clin Pathol 1979;72:374-82] that desirable imprecision should be less than or equal to one-half the within-subject biological variation, if the methods have negligible imprecision, then the maximum allowable bias between two methods used for monitoring is one-third of the within-subject biological variation. A more general model has been developed that relates the analytical imprecisions of two methods, and the bias between them, to biological variation. Applying the general formula derived in specific clinical monitoring situations in which a known change in serial results (occurring at a stated probability) stimulates clinical action allows goals for the imprecisions of the two methods and allows the difference in bias between them to be determined quantitatively.

Charts of operational process specifications ("OPSpecs charts") for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing performance criteria

“Operational process specifications” have been derived from an analytical quality-planning model to assess the precision, accuracy, and quality control (QC) needed to satisfy Proficiency Testing (PT) criteria. These routine operating specifications are presented in the form of an “OPSpecs chart,” which describes the operational limits for imprecision and inaccuracy when a desired level of quality assurance is provided by a specific QC procedure. OPSpecs charts can be used to compare the operational limits for different QC procedures and to select a QC procedure that is appropriate for the precision and accuracy of a specific measurement procedure. To select a QC procedure, one plots the inaccuracy and imprecision observed for a measurement procedure on the OPSpecs chart to define the current operating point, which is then compared with the operational limits of candidate QC procedures. Any QC procedure whose operational limits are greater than the measurement procedure's operating point will provide a known assurance, with the percent chance specified by the OPSpecs chart, that critical analytical errors will be detected. OPSpecs charts for a 10% PT criterion are presented to illustrate the selection of QC procedures for measurement procedures with different amounts of imprecision and inaccuracy. Normalized OPSpecs charts are presented to permit a more general assessment of the analytical performance required with commonly used QC procedures.

Charts of operational process specifications ("OPSpecs charts") for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing performance criteria

"Operational process specifications" have been derived from an analytical quality-planning model to assess the precision, accuracy, and quality control (QC) needed to satisfy Proficiency Testing (PT) criteria. These routine operating specifications are presented in the form of an "OPSpecs chart," which describes the operational limits for imprecision and inaccuracy when a desired level of quality assurance is provided by a specific QC procedure. OPSpecs charts can be used to compare the operational limits for different QC procedures and to select a QC procedure that is appropriate for the precision and accuracy of a specific measurement procedure. To select a QC procedure, one plots the inaccuracy and imprecision observed for a measurement procedure on the OPSpecs chart to define the current operating point, which is then compared with the operational limits of candidate QC procedures. Any QC procedure whose operational limits are greater than the measurement procedure's operating point will provide a known assurance, with the percent chance specified by the OPSpecs chart, that critical analytical errors will be detected. OPSpecs charts for a 10% PT criterion are presented to illustrate the selection of QC procedures for measurement procedures with different amounts of imprecision and inaccuracy. Normalized OPSpecs charts are presented to permit a more general assessment of the analytical performance required with commonly used QC procedures.

Selection of measurement and control procedures to satisfy the Health Care Financing Administration requirements and provide cost-effective operation

The basic principles of method evaluation and quality control system design have been extensively studied and described. However, the interaction among the various independent variables comprising the complete analytical system is not so well known. In this article, a model of a generalized analytical system is presented and the nature of the interaction among analytical method parameters and quality control system parameters is explored. Implications of recent Health Care Financing Administration regulations for allowable total error are also discussed.

Assuring analytical quality through process planning and quality control

United States governmental regulations describe quality assurance as a system in which quality control (QC) and proficiency testing are to be used to substantiate that routine tests conform to specified performance criteria. Careful planning of analytical processes is necessary to guarantee that routine tests will achieve the analytical performance defined by the Health Care Financing Administration proficiency testing criteria. These proficiency testing criteria are analytical "total error" requirements that need to be translated into operational process specifications for the allowable imprecision, allowable inaccuracy, and the QC procedure (decision rules, number of control measurements) necessary to provide a known or specified level of analytical quality assurance. The level of assurance will depend on the QC procedure's probability for detecting critical sized errors that would cause the quality requirement to be exceeded. Analytical quality can be guaranteed by applying statistical QC to verify, with a specified probability, that a desired quality requirement is actually achieved in routine laboratory testing. Analytical quality assurance, applied in this way, can be a quantitative approach for guaranteeing quality through process planning and QC.

Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing

Current U.S. governmental regulations and requirements for the quality of laboratory tests do not provide a consistent form, comparable numbers, or practical specifications for the routine operation of laboratory testing processes. For cholesterol, as an example, the Health Care Financing Administration provides an analytical performance criterion for proficiency testing to enforce the Clinical Laboratory Improvement Act (CLIA), whereas the U.S. National Cholesterol Education Program (NCEP) provides clinical guidelines for test interpretation, as well as analytical goals for imprecision and inaccuracy. Routine operating process specifications for imprecision, inaccuracy, and quality control can be derived from the analytical and clinical requirements for quality. Use of an analytical "total error" model and a clinical "decision interval" model provides logically consistent and numerically comparable specifications. Studies with these coherent models indicate that a cholesterol testing process properly planned to satisfy the CLIA analytical requirement will also satisfy the NCEP clinical requirement. To provide 90% assurance of detecting systematic shifts of a magnitude that would cause the CLIA analytical requirement to be exceeded, the operational specifications for a cholesterol testing process are an allowable CV of less than or equal to 2%, an allowable bias of less than or equal to 1%, and a control procedure with two measurements per run interpreted by 1(3)s, 1(2.5)s, or 1(3)s/2(2)s/R4s control rules.

Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing

Current U.S. governmental regulations and requirements for the quality of laboratory tests do not provide a consistent form, comparable numbers, or practical specifications for the routine operation of laboratory testing processes. For cholesterol, as an example, the Health Care Financing Administration provides an analytical performance criterion for proficiency testing to enforce the Clinical Laboratory Improvement Act (CLIA), whereas the U.S. National Cholesterol Education Program (NCEP) provides clinical guidelines for test interpretation, as well as analytical goals for imprecision and inaccuracy. Routine operating process specifications for imprecision, inaccuracy, and quality control can be derived from the analytical and clinical requirements for quality. Use of an analytical "total error" model and a clinical "decision interval" model provides logically consistent and numerically comparable specifications. Studies with these coherent models indicate that a cholesterol testing process properly planned to satisfy the CLIA analytical requirement will also satisfy the NCEP clinical requirement. To provide 90% assurance of detecting systematic shifts of a magnitude that would cause the CLIA analytical requirement to be exceeded, the operational specifications for a cholesterol testing process are an allowable CV of less than or equal to 2%, an allowable bias of less than or equal to 1%, and a control procedure with two measurements per run interpreted by 1(3)s, 1(2.5)s, or 1(3)s/2(2)s/R4s control rules.

Implementing total quality management (TQM) in health-care laboratories

Health-care organizations are beginning to apply the principles of total quality management (TQM). Implementing TQM in a health-care laboratory requires incorporating quality improvement (QI) and quality planning (QP) with quality laboratory practices (QLP), quality control (QC), and quality assurance (QA) to provide a complete quality management system. QI and QP can be initiated by developing a strategic plan as a pilot QI project. QI project teams are then introduced to accomplish the highest priority goals. This implementation approach improves strategic planning by using group problem-solving tools and techniques, such as process flow charts, brainstorming, nominal group, fishbone diagrams, consensus decision making, and Pareto analysis. The approach also improves the success of project teams by providing a clear management agenda and a commitment to project-by-project QI.

Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program

We have assessed the laboratory specifications necessary for ensuring that cholesterol testing processes satisfy the quality required by the U.S. National Cholesterol Education Program (NCEP). A model for setting process specifications has been developed to relate the NCEP guidelines for medical interpretation of a cholesterol test to the pre-analytical and analytical variables that can affect a test result. Using this model, we derived specifications for the imprecision (coefficient of variation, CV, or standard deviation, s) and inaccuracy (bias) that are allowable under stable operation, as well as the quality-control procedures (control rules and number of control measurements) that are necessary to detect unstable operation. The NCEP goals of an allowable CV less than or equal to 3% and an allowable bias no greater than +/- 3% are inadequate for assuring the quality of an individual or single cholesterol test when monitoring performance with many of the statistical quality-control procedures currently used in the U.S. With quality-control procedures having two control measurements per run, a CV of 3% is allowable only when bias is zero; a CV less than or equal to 2% is necessary if bias is +/- 3%. With quality-control procedures having four control measurements per run, a CV of 3% is allowable when bias is +/- 1.5%; a CV less than or equal to 2.5% is required if bias is as large as +/- 3%. For two serial tests, the NCEP 3% goals are adequate for current quality-control procedures having four control measurements per run.

Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program

We have assessed the laboratory specifications necessary for ensuring that cholesterol testing processes satisfy the quality required by the U.S. National Cholesterol Education Program (NCEP). A model for setting process specifications has been developed to relate the NCEP guidelines for medical interpretation of a cholesterol test to the pre-analytical and analytical variables that can affect a test result. Using this model, we derived specifications for the imprecision (coefficient of variation, CV, or standard deviation, s) and inaccuracy (bias) that are allowable under stable operation, as well as the quality-control procedures (control rules and number of control measurements) that are necessary to detect unstable operation. The NCEP goals of an allowable CV less than or equal to 3% and an allowable bias no greater than +/- 3% are inadequate for assuring the quality of an individual or single cholesterol test when monitoring performance with many of the statistical quality-control procedures currently used in the U.S. With quality-control procedures having two control measurements per run, a CV of 3% is allowable only when bias is zero; a CV less than or equal to 2% is necessary if bias is +/- 3%. With quality-control procedures having four control measurements per run, a CV of 3% is allowable when bias is +/- 1.5%; a CV less than or equal to 2.5% is required if bias is as large as +/- 3%. For two serial tests, the NCEP 3% goals are adequate for current quality-control procedures having four control measurements per run.

Quality management science in clinical chemistry: a dynamic framework for continuous improvement of quality

Current quality assurance approaches will not be adequate to satisfy the needs for quality in the next decade. Quality management science (QMS), as evolving in industry today, provides the dynamic framework necessary to provide continuous improvement of quality. QMS emphasizes the importance of defining quality goals based on the needs and expectations (implied needs) of customers. The laboratory can develop customer-friendly goals and measures of quality by recognizing that customers' experiences are represented by a totality of results. Quality goals and measures are best communicated as "total performance" by specifying a limit and percentile of the distribution, rather than a mean and standard deviation. Application of quality goals within the laboratory will usually require partitioning the total performance goal into components and translating those components into specifications to guide the operation and management of production processes. QMS also extends beyond technical processes to people processes and provides guidance for improving the quality of worklife and caring for the laboratory's most essential resource--our people.

Predicting effects of quality-control practices on the cost-effective operation of a stable, multitest analytical system

The cost-effective operation of an analytical system depends on quality-control (QC) practices such as the QC procedure itself (control rules, number of control measurements); the batch size or the run length; and the use of bracketed, nonbracketed, or pre-control modes of operation. Predictive value models that predict the defect rate and test yield of each test, as well as of the system as a whole, have been used to study these practices and to develop strategies for improving the quality and productivity of a multitest analyzer. Quality was optimized for most tests by achieving high error detection and low false rejection by the QC procedures. For a few tests where ideal QC performance could not be achieved, predictive models indicate that high quality is achieved, predictive models indicate that high quality is achieved as long as the observed stabilities (low frequencies of errors) of the measurement procedures are maintained. In our laboratories, productivity gains of 2.9% ($17,400/year) were achieved by changing QC procedures. Predictive models indicate that further gains are possible by increasing batch size and changing from bracketed to nonbracketed control operation. In general, the common practice of bracketed control on stable analytical systems may need to be re-examined owing to its effect on the cost of operation.

Quality management science in clinical chemistry: a dynamic framework for continuous improvement of quality

Current quality assurance approaches will not be adequate to satisfy the needs for quality in the next decade. Quality management science (QMS), as evolving in industry today, provides the dynamic framework necessary to provide continuous improvement of quality. QMS emphasizes the importance of defining quality goals based on the needs and expectations (implied needs) of customers. The laboratory can develop customer-friendly goals and measures of quality by recognizing that customers' experiences are represented by a totality of results. Quality goals and measures are best communicated as "total performance" by specifying a limit and percentile of the distribution, rather than a mean and standard deviation. Application of quality goals within the laboratory will usually require partitioning the total performance goal into components and translating those components into specifications to guide the operation and management of production processes. QMS also extends beyond technical processes to people processes and provides guidance for improving the quality of worklife and caring for the laboratory's most essential resource--our people.

Predicting effects of quality-control practices on the cost-effective operation of a stable, multitest analytical system

The cost-effective operation of an analytical system depends on quality-control (QC) practices such as the QC procedure itself (control rules, number of control measurements); the batch size or the run length; and the use of bracketed, nonbracketed, or pre-control modes of operation. Predictive value models that predict the defect rate and test yield of each test, as well as of the system as a whole, have been used to study these practices and to develop strategies for improving the quality and productivity of a multitest analyzer. Quality was optimized for most tests by achieving high error detection and low false rejection by the QC procedures. For a few tests where ideal QC performance could not be achieved, predictive models indicate that high quality is achieved, predictive models indicate that high quality is achieved as long as the observed stabilities (low frequencies of errors) of the measurement procedures are maintained. In our laboratories, productivity gains of 2.9% ($17,400/year) were achieved by changing QC procedures. Predictive models indicate that further gains are possible by increasing batch size and changing from bracketed to nonbracketed control operation. In general, the common practice of bracketed control on stable analytical systems may need to be re-examined owing to its effect on the cost of operation.

Precision requirements for cost-effective operation of analytical processes

Current performance criteria for analytical methods are often based on recommendations developed many years ago. A common criterion for imprecision requires that two times the standard deviation (s) of the method be less than the allowable total error (TEa). Unfortunately, when this criterion is minimally satisfied, commonly used control procedures cannot achieve reliable detection of medically important errors. Studies of the power functions for statistical quality-control (QC) procedures show that the magnitude of medically important errors must be at least 3s to fall near the plateau of the power curves and be readily detected by current QC procedures. For methods that just meet the precision criterion 2s less than TEa, however, medically important errors will fall on the rising portion of the power curves and typically be detected less than half of the time. From a "reverse engineering" perspective, the 2s less than TEa criterion is inadequate because it does not allow for the known performance limitations (lack of sensitivity) of commonly used control procedures. A strong case can be made for using a criterion of at least '4s less than TEa, which calls for a twofold improvement in imprecision over, the current minimum requirements. This recommendation is consistent with current industrial guidelines for process capability and would lead to more reliable detection of medically important errors.

Precision requirements for cost-effective operation of analytical processes

Current performance criteria for analytical methods are often based on recommendations developed many years ago. A common criterion for imprecision requires that two times the standard deviation (s) of the method be less than the allowable total error (TEa). Unfortunately, when this criterion is minimally satisfied, commonly used control procedures cannot achieve reliable detection of medically important errors. Studies of the power functions for statistical quality-control (QC) procedures show that the magnitude of medically important errors must be at least 3s to fall near the plateau of the power curves and be readily detected by current QC procedures. For methods that just meet the precision criterion 2s less than TEa, however, medically important errors will fall on the rising portion of the power curves and typically be detected less than half of the time. From a "reverse engineering" perspective, the 2s less than TEa criterion is inadequate because it does not allow for the known performance limitations (lack of sensitivity) of commonly used control procedures. A strong case can be made for using a criterion of at least '4s less than TEa, which calls for a twofold improvement in imprecision over, the current minimum requirements. This recommendation is consistent with current industrial guidelines for process capability and would lead to more reliable detection of medically important errors.

Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system

Quality-control (QC) procedures (i.e., decision rules used, numbers of control measurements collected per run) have been selected for individual tests of a multitest analyzer, to see that clinical or "medical usefulness" requirements for quality are met. The approach for designing appropriate QC procedures includes the following steps: (a) defining requirements for quality in the form of the "total allowable analytical error" for each test, (b) determining the imprecision of each measurement procedure, (c) calculating the medically important systematic and random errors for each test, and (d) assessing the probabilities for error detection and false rejection for candidate control procedures. In applying this approach to the Hitachi 737 analyzer, a design objective of 90% (or greater) detection of systematic errors was met for most tests (sodium, potassium, glucose, urea nitrogen, creatinine, phosphorus, uric acid, cholesterol, total protein, total bilirubin, gamma-glutamyltransferase, alkaline phosphatase, aspartate aminotransferase, lactate dehydrogenase) by use of 3.5s control limits with two control measurements per run (N). For the remaining tests (albumin, chloride, total CO2, calcium), requirements for QC procedures were more stringent, and 2.5s limits (with N = 2) were selected.

Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system

Quality-control (QC) procedures (i.e., decision rules used, numbers of control measurements collected per run) have been selected for individual tests of a multitest analyzer, to see that clinical or "medical usefulness" requirements for quality are met. The approach for designing appropriate QC procedures includes the following steps: (a) defining requirements for quality in the form of the "total allowable analytical error" for each test, (b) determining the imprecision of each measurement procedure, (c) calculating the medically important systematic and random errors for each test, and (d) assessing the probabilities for error detection and false rejection for candidate control procedures. In applying this approach to the Hitachi 737 analyzer, a design objective of 90% (or greater) detection of systematic errors was met for most tests (sodium, potassium, glucose, urea nitrogen, creatinine, phosphorus, uric acid, cholesterol, total protein, total bilirubin, gamma-glutamyltransferase, alkaline phosphatase, aspartate aminotransferase, lactate dehydrogenase) by use of 3.5s control limits with two control measurements per run (N). For the remaining tests (albumin, chloride, total CO2, calcium), requirements for QC procedures were more stringent, and 2.5s limits (with N = 2) were selected.

Implementation of a multirule, multistage quality control program in a clinical laboratory computer system

We developed a computer subsystem that permits users to implement multirule, multistage quality control procedures. The subsystem runs as a task in the RelationaLABCOM information system and permits the collection of data from on-line instruments, as well as through manual (keyboard) entry. The choices of control rules and their combinations are at the discretion of each laboratory section, with the system automatically administering the chosen protocols for all the technologists working with the computer. Retrospective data analysis and statistics are available for review by laboratory personnel. The subsystem provides a significant improvement in the availability of real-time quality control at the bench.

Quality control of multichannel hematology analyzers: evaluation of Bull's algorithm

Bull's algorithm has been evaluated by computer simulation studies. Varying amounts of systematic analytic error were simulated in either hemoglobin (Hgb), red blood cell count (RBC), or mean corpuscular volume (MCV) with the resulting red blood cell indices averaged in batches of 20 using Bull's algorithm. The number of average indices outside the limits of 0.97 means and 1.03 means (means = stable patient mean index) was tabulated and plotted against the size of the systematic shift, expressed in multiples of the long-term analytic standard deviation (SD). The resulting plots, called power functions, show that Bull's algorithm can detect large shifts effectively and that its power increases with increasing batch number. Shifts less than 2 SD rarely are detected. The minimum error that is detected 50% of the time after nine consecutive batches is shown below: (Formula: see text) The simulation of populations with outlying indices, e.g., neonates and oncology patients, resulted in both decreased and increased power, depending on the proportion of outliers averaged, the index averaged, and the direction of the shift.

Assessment of "Average of Normals" quality control procedures and guidelines for implementation

The capabilities of "Average of Normals" control procedures have been assessed by determining power functions, graphs of the probability of error detection versus the size of analytic error. The power functions indicate that the most important determinants of statistical power are the ratio of the standard deviation of the patient population (Sp) to the analytic standard deviation (Sa), Sp/Sa; the number of data points averaged (N); the control limits (probability for false rejection); truncation limits for selecting the population; and the magnitude of the population lying outside the truncation limits. General guidelines for the implementation of "Average of Normals" are provided, along with a nomogram for the selection of N as a function of Sp/Sa and the probability of error detection. Optimal performance of these procedures may require simulation studies on a per analyte basis.