Analytical performance specifications based on biological variation data - considerations, strengths and limitations

Analytical performance specifications (APS) are typically established through one of three models: (i) outcome studies, (ii) biological variation (BV), or (iii) state-of-the-art. Presently, The APS can, for most measurands that have a stable concentration, be based on BV. BV based APS, defined for imprecision, bias, total allowable error and allowable measurement uncertainty, are applied to many different processes in the laboratory. When calculating APS, it is important to consider the different APS formulae, for what setting they are to be applied and if they are suitable for the intended purpose. In this opinion paper, we elucidate the background, limitations, strengths, and potential intended applications of the different BV based APS formulas. When using BV data to set APS, it is important to consider that all formulae are contingent on accurate and relevant BV estimates. During the last decade, efficient procedures have been established to obtain reliable BV estimates that are presented in the EFLM biological variation database. The database publishes detailed BV data for numerous measurands, global BV estimates derived from meta-analysis of quality-assured studies of similar study design and automatic calculation of BV based APS.

Are analytical performance specifications derived from reference intervals of any use in the veterinary clinical laboratory? A preliminary study on the empirical biological variation model

BACKGROUND

Analytical performance specifications (APS) are vital for method evaluation and quality control validation. However, the limited availability of biological variation (BV) data, regulatory guidelines, and expert opinion (EO) may present challenges in veterinary medicine. The empirical biological variation (EBV) approach, based on population reference intervals (pRI), has emerged as an alternative method to derive APS in human medicine.

OBJECTIVES

This study aimed to assess the practicality and usefulness of the EBV approach in deriving performance limits for various measurands in dogs and cats.

METHODS

Eight hematology and 13 biochemistry measurands were analyzed in dogs and cats. Estimates of combined biologic variation based on traditional biological (CVB ) and EBV-derived (CVE *) formulas were calculated and assessed for evidence of correlation. Performance limits for expanded uncertainty/total error and imprecision were compared among EO, BV, and EBV.

RESULTS

Strong and significant correlations were found between CVB and CVE * for both dogs (r = .86, p < .00001) and cats (r = 0.95, p < .00001). The EBV-derived APS were generally comparable to EO and BV, with a subjective criterion of 1.5% difference for imprecision and 3% for total error/expanded uncertainty.

CONCLUSION

The EBV approach, using pRI, shows promise as a surrogate marker for biological variation and as a practical tool for determining performance limits in dogs and cats. Assuming accurate pRI generated on analyzers with stable analytical performance, this approach could offer benefits when expert recommendations or robust BV studies are lacking or yield conflicting results. Further research is needed to explore the applicability and advantages of the EBV in veterinary medicine.

Evaluation of canine and feline leukocyte differential counts obtained with the scil vCell 5 compared to the Advia 2120 hematology analyzer and a manual method

The vCell 5 (scil Animal Care), a point-of-care hematology analyzer (POCA), was recently introduced to veterinary laboratories. This laser- and impedance-based analyzer is capable of providing a CBC with 5-part WBC differential count (Diff) along with WBC cytograms and flags serving as interpretation aids for numerical results. We compared the scil POCA-Diff to reference methods (i.e., manual differential count, Advia 2120 hematology analyzer [Siemens]) for canine and feline blood samples and considered WBC cytograms and flags. Total observed error (TEo), calculated from CV and bias%, was compared to total allowable error (TEa). Data were analyzed before and after a review process (exclusion of flagged and samples with invalid cytograms). For both species, correlation was good-to-excellent (rs = 0.81-0.97) between both analyzers for all variables, except for feline monocytes (rs = 0.21-0.63) and canine monocyte% (rs = 0.50). Smallest biases were seen for neutrophils (dog: -5.7 to 0.8%; cat: 1.5-9.4%) with both reference methods. Quality requirements (TEo < TEa) were fulfilled for canine and feline neutrophils (TEo = 5.3-10.6%, TEa = 15%) and eosinophils (TEo = 67.1-83%, TEa = (90)-50%) considering at least one reference method. Our review process led to mildly higher rs-values for most variables. Although not completely satisfactory, the scil POCA provides reliable results in compliance with ASVCP quality goals for canine and feline neutrophils and eosinophils. Analyzer flag and cytogram analysis served as useful tools for QA, indicating the necessity for manual review of blood smears, and contributed to improvement of scil POCA performance.

Sigma Metrics in Laboratory Medicine: A Call for Harmonization

BACKGROUND AND AIM

Sigma metrics are applied in clinical laboratories to assess the quality of analytical processes. A parameter associated to a Sigma >6 is considered "world class" whereas a Sigma <3 is "poor" or "unacceptable". The aim of this retrospective study was to quantify the impact of different approaches for Sigma metrics calculation.

MATERIAL AND METHODS

Two IQC levels of 20 different parameters were evaluated for a 12-month period. Sigma metrics were calculated using the formula: (allowable total error (TEa) (%) - bias (%))/(coefficient of variation (CV) (%)). Method precision was calculated monthly or annually. The bias was obtained from peer comparison program (PCP) or external quality assessment program (EQAP), and 9 different TEa sources were included.

RESULTS

There was a substantial monthly variation of Sigma metrics for all combinations, with a median variation of 32% (IQR, 25.6-41.3%). Variation across multiple analyzers and IQC levels were also observed. Furthermore, TEa source had the highest impact on Sigma calculation with proportions of Sigma >6 ranging from 17.5% to 84.4%. The nature of bias was less decisive.

CONCLUSION

In absence of a clear consensus, we recommend that laboratories calculate Sigma metrics on a sufficiently long period of time (>6 months) and carefully evaluate the choice of TEa source.

Analytical performance evaluation of sensitive and old generation reagent in routine practical use: estradiol experience

Evaluation of the analytical performance of tests in medical laboratories is important. Total Error (TE) and sigma analysis have been used as a quantitative indicator of quality for many years. The aim of this study is to evaluate the analytical performance of Beckman Coulter Access Estradiol (E2) and Sensitive E2 reagents. Analytical performance of two reagents were evaluated with TE, six sigma and measurement uncertainty values. Two Beckman Coulter Unicel DxI-800 autoanalyzers (A1 and A2) included in the study. Quality control data between December 2017 and December 2019 were used. CLIA-2019 values were used for total allowable error (TEa) limits. Uncertainty values were calculated with ISO/TS 20914. The median TE of the old generation and sensitive E2 reagent were 27.46% (between 13.49 and 48.88) and 11.16% (between 7.08 and 24.81), respectively (p < .005) The process sigma results were below 3 sigma in all months with the old reagent, whereas with the new reagents it was seen to be above 3 sigma in 11 of 12 months for both autoanalyzers. Uncertainty of old reagent is higher than new reagent. Imprecisions decrease as concentration increases with both reagents. The uncertainty values of low concentration levels are greater than high concentration levels. In conclusion, in both auto analyzers, Sensitive E2 reagent was found to have better performance than old reagent in terms of TE, process sigma and measurement uncertainty. We believe that the sensitive E2 reagent still needs further improvement for patients who have low E2 levels.

Validation study of canine urine cortisol measurement with the Immulite 2000 Xpi cortisol immunoassay

We report here validation of the Immulite 2000 Xpi cortisol immunoassay (Siemens; with kit lot numbers <550) for measurement of urine cortisol in dogs, with characterization of the precision (CV), accuracy (spiking-recovery [SR] bias), and observed total error (TEo = bias + 2CV) across the reportable range. Linearity assessed by simple linear regression was excellent. Imprecision, SR bias, and TEo increased markedly with decreasing urine cortisol concentration. Interlaboratory comparison studies determined range-based (RB) bias and average bias (AB). The 3 biases (SR, RB, and AB) and resulting TEo differed markedly. At 38.6 and 552 nmol/L (1.4 and 20 μg/dL), between-run CVs were 10% and 4.5%, respectively, and TEoRB were ~30% and 20%, respectively, similar to observations in serum in another validation study. These analytical performance parameters should be considered for urine cortisol:creatinine ratio (UCCR) result interpretation, given that, for any hypothetical errorless urine creatinine measurement, the error % on UCCR mirrors the error % on urine cortisol. Importantly, there is no commonly used interpretation threshold for UCCR, given that UCCR varies greatly depending on measurement methods and threshold computation. To date, there is no manufacturer-provided quality control material (QCM) with target values for urine cortisol with an Immulite; for Liquicheck QCM (Bio-Rad), between-run imprecision was ~5% for both QCM levels. Acceptable QC rules are heavily dependent on the desired total allowable error (TEa) for the QCM system, itself limited by the desired clinical TEa.

Validation study of canine serum cortisol measurement with the Immulite 2000 Xpi cortisol immunoassay

We report the results of validation of canine serum cortisol determination with the Immulite 2000 Xpi cortisol immunoassay (Siemens), with characterization of precision (CV), accuracy (spiking-recovery [SR] bias), and observed total error (TEo = bias + 2CV) across the reportable range, specifically at the most common interpretation thresholds for dynamic testing. Imprecision increased at increasing rate with decreasing serum cortisol concentration and bias was low, resulting in increasing TEo with decreasing serum cortisol concentration. Inter-laboratory comparison study allowed for determination of range-based bias (RB) and average bias (AB). At 38.6 and 552 nmol/L (1.4 and 20 μg/dL), between-run CV was 10% and 7.5%, respectively, and TEo ~30% and ~20%, respectively (TEo remained similar regardless of the considered bias: SR, RB, or AB). These analytical performance parameters should be considered in the interpretation of results and for future expert consensus discussions to determine recommendations for allowable total error (TEa). Importantly, the commonly used thresholds for interpretation of results were determined ~40 y ago with different methods of measurements and computation, hence updating is desirable. Quality control material (QCM) had between-run imprecision of 4% for QCM1 and 7% for QCM2; the bias was minimal for both levels. Acceptable QC rules are heavily dependent on the desired TEa for the QCM system (TEaQCM), itself limited by the desired clinical TEa. At low TEaQCM (20-33%), almost no rules were acceptable, whereas at high TEaQCM (50%), almost all rules were acceptable; further investigation is needed to determine which TEaQCM can be guaranteed by simple QC rules.

Analytical quality performance goals for symmetric dimethylarginine in cats

BACKGROUND

Symmetric dimethylarginine (SDMA) reflects the glomerular filtration rate (GFR) in people, dogs, and cats. Initial assays used a liquid chromatography-mass spectroscopy (LC) technique. A veterinary immunoassay has been developed for use in commercial laboratories and point-of-care (POC) laboratory equipment. There have been no independent assessments of these assays, and analytical performance goals for SDMA testing have not been defined.

OBJECTIVES

This study sought to establish analytical performance goals for SDMA in cats from (a) biological variation (BV) data and (b) expert opinion.

METHODS

Analytical performance goals were determined from a prior BV study of SDMA in cats and a survey of veterinary internists who have used SDMA in practice.

RESULTS

Biological variation-based performance goals included an imprecision of ±10% (immunoassay and LC), bias of ±8% (immunoassay and LC), and total error of ±24% (immunoassay and LC). Expert opinion performance goals were ±0.10 μmol/L (±2 μg/dL), or ±0.15 μmol/L (±3 μg/dL), varying with starting SDMA concentrations.

CONCLUSIONS

This study recommends analytical performance goals for SDMA based on BV and expert opinion. Wide dispersion of SDMA results using currently available assays implies that clinicians risk attaching medical significance to small SDMA changes that actually reflect analytical variability.

Fingerstick Precision and Total Error of a Point-of-Care HbA1c Test

BACKGROUND

Point-of-care (POC) HbA1c tests hold the promise of reducing the rates of undiagnosed diabetes, provided they exhibit acceptable analytical performance. The precision and total error of the POC (Afinion™ HbA1c Dx) test were investigated using whole blood samples obtained by fingerstick and venipuncture.

METHODS

Fingerstick samples spanning the assay range were collected from 61 subjects at three representative POC sites. At each site, six fingerstick samples were obtained from each subject and tested on the POC test across two (Afinion AS100) instruments. Repeatability, between-operator, and between-instrument components of variance were calculated using analysis of variance (ANOVA). Four venous samples (low, threshold, medium, and high HbA1c) were measured in duplicate across three instruments using three reagent lots, twice per day over 20-days. Repeatability, between-run, between-day, between-lot, and between-instrument components of variance were calculated. These fingerstick and venous blood results, combined with estimates of imprecision and bias from a prior investigation, allowed for the calculation of the total coefficient of variation (CV) and total error of the POC test using fingerstick and venous whole blood samples.

RESULTS

The total imprecision ranged from 1.30% to 2.03% CV using fingerstick samples and from 1.31% to 1.64% CV using venous samples. The total error ranged from 2.87% to 4.75% using fingerstick samples and from 2.93% to 3.80% using venous samples.

CONCLUSIONS

The POC test evaluated here is precise across its measuring range using both fingerstick and venous whole blood. The calculated total error of the test is well under the accepted quality requirement of ≤6%.

Analysis of a Point-of-Care HbA1c Assay: Is It Time for an HbA1c Error Grid?

In an article in Journal of Diabetes Science and Technology, Arnold et al have presented a thorough study of imprecision components for a point-of-care hemoglobin A1c (HbA1c) assay. An interesting and innovative approach is the combination of data from different studies to arrive at a total error estimate. But total error has the oxymoron feature of estimating performance for most (95%) but not all of the results. An HbA1c error grid would provide the severity for results that exceed the 6% requirement. Since this device is intended for Clinical Laboratory Improvement Amendments waived labs and allows for finger-stick samples, monitoring the Food and Drug Administration adverse event database (MAUDE, Manufacturer and User Facility Device Experience) is recommended.

Study of total error specifications of lymphocyte subsets enumeration using China National EQAS data and Biological Variation Data Critical Appraisal Checklist (BIVAC)-compliant publications

Objectives

It is important to select proper quality specifications for laboratories and external quality assessment (EQA) providers for their quality control and assessment. The aim of this study is to produce new total error (TE) specifications for lymphocyte subset enumeration by analyzing the allowable TE using EQAS data and comparing them with that based on reliable biological variation (BV).

Methods

A total of 54,400 results from 1,716 laboratories were collected from China National EQAS for lymphocyte subset enumeration during the period 2017-2019. The EQA data were grouped according to lower limits of reference intervals for establishing concentration-dependent specifications. The TE value that 80% of laboratories can achieve were considered as TE specifications based on state of the art. The BV studies compliant with Biological Variation Data Critical Appraisal Checklist (BIVAC) were used to calculate the three levels of TE specifications. Then these TE specifications were compared for determining the recommended TE specifications.

Results

Four parameters whose quality specifications could achieve the optimum criteria were as follows: the percentages of CD3+, CD3+CD4+ (high concentration) and CD3-CD16/56+ cells, and the absolute count of CD3-CD16/56+ cells. Only the TE specifications of CD3-CD19+ cells could achieve the minimum criteria. The TE specifications of remaining parameters should reach the desirable criteria.

Conclusions

New TE specifications were established by combining the EQA data and reliable BV data, which could help laboratories to apply proper criteria for continuous improvement of quality control, and EQA providers to use robust acceptance limits for better evaluation of EQAS results.

The top-down approach to measurement uncertainty: which formula should we use in laboratory medicine?

Introduction

By quantifying the measurement uncertainty (MU), both the laboratory and the physician can have an objective estimate of the results’ quality. There is significant flexibility on how to determine the MU in laboratory medicine and different approaches have been proposed by Nordtest, Eurolab and Cofrac to obtain the data and apply them in formulas. The purpose of this study is to compare three different top-down approaches for the estimation of the MU and to suggest which of these approaches could be the most suitable choice for routine use in clinical laboratories.

Materials and methods

Imprecision and bias of the methods were considered as components of the MU. The bias was obtained from certified reference calibrators (CRC), proficiency tests (PT), and inter-laboratory internal quality control scheme (IQCS) programs. The bias uncertainty, the combined and the expanded uncertainty were estimated using the Nordtest, Eurolab and Cofrac approaches.

Results

Using different approaches, the expanded uncertainty estimates ranged from 18.9-40.4%, 18.2-22.8%, 9.3-20.9%, and 7.1-18.6% for cancer antigen (CA) 19-9, testosterone, alkaline phosphatase (ALP), and creatinine, respectively. Permissible values for MU and total error ranged from 16.0-46.1%, 13.1-21.6%, 10.7-26.2%, and 7.5-17.3%, respectively.

Conclusion

The bias was highest using PT, followed by CRC and IQCS data, which were similar. The Cofrac approach showed the highest uncertainties, followed by Eurolab and Nordtest. However, the Eurolab approach requires additional measurements to obtain uncertainty data. In summary, the Nordtest approach using IQCS data was therefore found to be the most practical formula.

Study of the analytical performance at different concentrations of hematological parameters using Spanish EQAS data

Background External quality assessment programs are one of the currently available tools to evaluate the analytical performance of clinical laboratories, where the measurement error (ME) obtained can be compared with quality specifications to evaluate possible deviations. The objective of this work was to analyze the ME behavior over the analytical range to assess the need to establish concentration-dependent specifications. Methods A total of 389,000 results from 585 laboratories and 2628 analyzers were collected from the Spanish external quality assessment schemes (EQAS) in hematology during the years 2015-2016. The parameters evaluated included white blood cells, red blood cells, hemoglobin, hematocrit, mean corpuscular volume, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, platelets, prothrombin time, activated partial thromboplastin time, neutrophils, lymphocytes, monocytes, eosinophils, basophils, reticulocytes, hemoglobin A2, antithrombin, factor VIII, protein C and von Willebrand factor. The 90th percentile of ME was calculated for every concentration evaluated of each parameter. Results We found a significant variation in the analytical performance of leukocytes, platelets, neutrophils, lymphocytes, monocytes, eosinophils, basophils, prothrombin time, reticulocytes, hemoglobin A2, antithrombin and protein C. Furthermore, this ME variation may not allow complying with the same biological variability requirements within the whole analytical range studied. Conclusions Our work shows the importance of implementing concentration-dependent specifications which can help laboratories to use proper criteria for quality specifications selection and for a better external quality control results evaluation.

Uncertainty in measurement and total error: different roads to the same quality destination?

The debate comparing the benefits of measurement uncertainty (uncertainty in measurement, MU) with total error (TE) for the assessment of laboratory performance continues. The summary recently provided in this journal by members of the Task and Finish Group on Total Error (TFG-TE) of the EFLM put the arguments into clear perspective. Even though there is generally strong support for TE in many laboratories, some of the arguments proposed for its on-going support require further comment. In a recent opinion which focused directly on the TFG-TE summary, several potentially confusing statements regarding ISO15189 and the Evaluation of measurement data - Guide to the expression of uncertainty in measurement (GUM) were again promulgated to promote TE methods for assessing uncertainty in laboratory measurement. In this opinion, we present an alternative view of the key issues and outline our views with regard to the relationship between ISO15189, uncertainty in measurement and the GUM.

The use of error and uncertainty methods in the medical laboratory

Error methods - compared with uncertainty methods - offer simpler, more intuitive and practical procedures for calculating measurement uncertainty and conducting quality assurance in laboratory medicine. However, uncertainty methods are preferred in other fields of science as reflected by the guide to the expression of uncertainty in measurement. When laboratory results are used for supporting medical diagnoses, the total uncertainty consists only partially of analytical variation. Biological variation, pre- and postanalytical variation all need to be included. Furthermore, all components of the measuring procedure need to be taken into account. Performance specifications for diagnostic tests should include the diagnostic uncertainty of the entire testing process. Uncertainty methods may be particularly useful for this purpose but have yet to show their strength in laboratory medicine. The purpose of this paper is to elucidate the pros and cons of error and uncertainty methods as groundwork for future consensus on their use in practical performance specifications. Error and uncertainty methods are complementary when evaluating measurement data.

Sigma metrics in laboratory medicine revisited: We are on the right road with the wrong map

Reliable procedures are needed to quantify the performance of instruments and methods in order to increase the quality in clinical laboratories. The Sigma metrics serves that purpose, and in the present study, the current methods for the calculation of the Sigma metrics are critically evaluated. Although the conventional model based on permissible (or allowable) total error is widely used, it has been shown to be flawed. An alternative method is proposed based on the within-subject biological variation. This model is conceptually similar to the model used in industry to quantify measurement performance, based on the concept of the number of distinct categories and consistent with the Six Sigma methodology. The quality of data produced in clinical laboratories is expected, however, to be higher than the quality of industrial products. It is concluded that this model is consistent with Six Sigma theory, original Sigma metrics equation and with the nature of patients' samples. Therefore, it can be used easily to calculate the performance of measurement methods and instruments used in clinical laboratories.

Large molecule run acceptance: Recommendation for best practices and harmonization from the Global Bioanalysis Consortium Harmonization Team

The L1 Global Harmonization Team provides recommendations specifically for run acceptance of ligand binding methods used in bioanalysis of macromolecules in support of pharmacokinetics. The team focused on standard curve calibrators and quality controls for use in both pre-study validation and in-study sample analysis, including their preparation and acceptance criteria. The team also considered standard curve editing and the concept of total error.

The Danger of Using Total Error Models to Compare Glucose Meter Performance

Glucose meter performance specifications provide limits for 95% of results, which is the same as total error. A popular total error model is that total error equals (average) bias plus 2 times imprecision. This model has been used to specify combinations of average bias and imprecision that satisfy total error goals. But this model is incomplete and its conclusions are suspect. It is shown that when interferences occur in glucose meters as exemplified by hematocrit interference, the total error model proposed by Boyd and Bruns cannot distinguish between meters that differ in performance. The CLSI standard EP21-A, does not have this problem because it directly estimates total error bypassing the need for a model. An example illustrates these points.

Simplified method of clinical phenotyping for older men and women using established field-based measures

The purpose of this investigation was to determine body composition classification using field-based testing measurements in healthy elderly men and women. The use of isoperformance curves is presented as a method for this determination. Baseline values from 107 healthy Caucasian men and women, over the age of 65years old, who participated in a separate longitudinal study, were used for this investigation. Field-based measurements of age, height, weight, body mass index (BMI), and handgrip strength were recorded on an individual basis. Relative skeletal muscle index (RSMI) and body fat percentage (FAT%) were determined by dual-energy X-ray absorptiometry (DXA) for each participant. Sarcopenia cut-off values for RSMI of 7.26kg·m(-2) for men and 5.45kg·m(-2) for women and elderly obesity cut-off values for FAT% of 27% for men and 38% for women were used. Individuals above the RSMI cut-off and below the FAT% cut-off were classified in the normal phenotype category, while individuals below the RSMI cut-off and above the FAT% cut-off were classified in the sarcopenic-obese phenotype category. Prediction equations for RSMI and FAT% from sex, BMI, and handgrip strength values were developed using multiple regression analysis. The prediction equations were validated using double cross-validation. The final regression equation developed to predict FAT% from sex, BMI, and handgrip strength resulted in a strong relationship (adjusted R(2)=0.741) to DXA values with a low standard error of the estimate (SEE=3.994%). The final regression equation developed to predict RSMI from the field-based testing measures also resulted in a strong relationship (adjusted R(2)=0.841) to DXA values with a low standard error of the estimate (SEE=0.544kg·m(-2)). Isoperformance curves were developed from the relationship between BMI and handgrip strength for men and women with the aforementioned clinical phenotype classification criteria. These visual representations were used to aid in the classification and evaluation of sarcopenia, obesity, and sarcopenic-obesity in elderly individuals. Future research should replicate the current findings with an increased sample size and the development of tailored interventions for each body composition category.

The new glucose standard POCT12-A3 misses the mark

POCT12-A3 is a Clinical Laboratory Standards Institute standard for hospitals about hospital glucose meter procedures and performance standards. I have reviewed this standard based on the attributes of an ideal performance standard. POCT12-A3 has tighter limits than its predecessor for 95% of results, the limits widen for 98% of results, and there are no limits for 2% of results. It is hard to fathom that 2% of the results are unspecified and could cause life-threatening results, as glucose meters do not perform this poorly. There should be a specification for unreported results since, by definition, point-of-care-testing assays are time sensitive. POCT12-A3 provides useful advice about the glucose testing procedure but provides evaluation guidance only about analytical performance. Moreover, the recommended protocol to assess meter performance is biased and likely to underestimate the observed performance. The guideline would be improved if its specification were based on an error grid and contained evaluation protocols for user errors.

The relationships between common measures of glucose meter performance

BACKGROUND

Glucose meter performance is commonly measured in several different ways, including the relative bias and coefficient of variation (CV), the total error, the mean absolute relative deviation (MARD), and the size of the interval around the reference value that would be necessary to contain a meter measurement at a specified probability. This fourth measure is commonly expressed as a proportion of the reference value and will be referred to as the necessary relative deviation. A deeper understanding of the relationships between these measures may aid health care providers, patients, and regulators in comparing meter performances when different measures are used.

METHODS

The relationships between common measures of glucose meter performance were derived mathematically.

RESULTS

Equations are presented for calculating the total error, MARD, and necessary relative deviation using the reference value, relative bias, and CV when glucose meter measurements are normally distributed. When measurements are also unbiased, the CV, total error, MARD, and necessary relative deviation are linearly related and are therefore equivalent measures of meter performance.

CONCLUSIONS

The relative bias and CV provide more information about meter performance than the other measures considered but may be difficult for some audiences to interpret. Reporting meter performance in multiple ways may facilitate the informed selection of blood glucose meters.

Analysis of the performance of the OneTouch SelectSimple blood glucose monitoring system: why ease of use studies need to be part of accuracy studies

The article entitled "Precision, Accuracy, and User Acceptance of the OneTouch SelectSimple Blood Glucose Monitoring System" by Philis-Tsimikas and colleagues in this issue of Journal of Diabetes Science and Technology demonstrates that the OneTouch® SelectSimple™ glucose meter meets current regulatory expectations for glucose meter performance. These authors describe three studies: precision, accuracy, and ease of use. Accuracy study analysis includes the effects of accuracy and precision. The ease-of-use study was analyzed separately, as recommended by the International Organization for Standardization 15197 glucose standard. The ultimate goal of an evaluation is to estimate the distribution of errors (from any source) that will be experienced in routine use. To accomplish this, ease-of-use results need to be part of the accuracy dataset.

Estimates of total analytical error in consumer and hospital glucose meters contributed by hematocrit, maltose, and ascorbate

BACKGROUND

Patients and physicians expect accurate whole blood glucose monitoring even when patients are anemic, are undergoing peritoneal dialysis, or have slightly elevated ascorbate levels. The objective of this study was to estimate analytical error in two consumer and two hospital glucose meters contributed by variations in hematocrit, maltose, ascorbate, and imprecision.

METHODS

The influence of hematocrit (20-60%), maltose, and ascorbate were tested alone and in combination with each glucose meter and with a reference plasma glucose method at three concentrations of glucose. Precision was determined by consecutive analysis (n=20) at three levels of glucose. Multivariate regression analysis was used to estimate the bias associated with the interferences, alone and in combination. Total analytical error was estimated as |% bias|+1.96 (% imprecision).

RESULTS

Three meters demonstrated hematocrit bias that was dependent upon glucose concentration. Maltose had profound concentration-dependent positive bias on the consumer meters, and the extent of maltose bias was dependent on hematocrit. Ascorbate produced small but statistically significant biases on three meters. Coincident low hematocrit, presence of maltose, and presence of ascorbate increased the observed bias and was summarized by estimation of total analytical error. Among the four glucose meter devices assessed, estimates of total analytical error in glucose measurement ranged from 6 to 68% under the conditions tested.

CONCLUSIONS

The susceptibility of glucose meters to clinically significant analytical biases is highly device-dependent, and low hematocrit exacerbated the observed analytical error.

A review of standards and statistics used to describe blood glucose monitor performance

Glucose performance is reviewed in the context of total error, which includes error from all sources, not just analytical. Many standards require less than 100% of results to be within specific tolerance limits. Analytical error represents the difference between tested glucose and reference method glucose. Medical errors include analytical errors whose magnitude is great enough to likely result in patient harm. The 95% requirements of International Organization for Standardization 15197 and others make little sense, as up to 5% of results can be medically unacceptable. The current American Diabetes Association standard lacks a specification for user error. Error grids can meaningfully specify allowable glucose error. Infrequently, glucose meters do not provide a glucose result; such an occurrence can be devastating when associated with a life-threatening event. Nonreporting failures are ignored by standards. Estimates of analytical error can be classified into the four following categories: imprecision, random patient interferences, protocol-independent bias, and protocol-dependent bias. Methods to estimate total error are parametric, nonparametric, modeling, or direct. The Westgard method underestimates total error by failing to account for random patient interferences. Lawton's method is a more complete model. Bland-Altman, mountain plots, and error grids are direct methods and are easier to use as they do not require modeling. Three types of protocols can be used to estimate glucose errors: method comparison, special studies and risk management, and monitoring performance of meters in the field. Current standards for glucose meter performance are inadequate. The level of performance required in regulatory standards should be based on clinical needs but can only deal with currently achievable performance. Clinical standards state what is needed, whether it can be achieved or not. Rational regulatory decisions about glucose monitors should be based on robust statistical analyses of performance.