Models for combining random and systematic errors. assumptions and consequences for different models

Analytical performance specifications based on biological variation data - considerations, strengths and limitations

Analytical performance specifications (APS) are typically established through one of three models: (i) outcome studies, (ii) biological variation (BV), or (iii) state-of-the-art. Presently, The APS can, for most measurands that have a stable concentration, be based on BV. BV based APS, defined for imprecision, bias, total allowable error and allowable measurement uncertainty, are applied to many different processes in the laboratory. When calculating APS, it is important to consider the different APS formulae, for what setting they are to be applied and if they are suitable for the intended purpose. In this opinion paper, we elucidate the background, limitations, strengths, and potential intended applications of the different BV based APS formulas. When using BV data to set APS, it is important to consider that all formulae are contingent on accurate and relevant BV estimates. During the last decade, efficient procedures have been established to obtain reliable BV estimates that are presented in the EFLM biological variation database. The database publishes detailed BV data for numerous measurands, global BV estimates derived from meta-analysis of quality-assured studies of similar study design and automatic calculation of BV based APS.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC)

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Biological variation: Understanding why it is so important?

This Review will describe the increasing importance of the concepts of biological variation to clinical chemists. The idea of comparison to 'reference' is fundamental in measurement. For the biological measurands, that reference is the relevant patient population, a clinical decision point based on a trial or an individual patient's previous results. The idea of using biological variation to set quality goals was then realised for setting Quality Control (QC) and External Quality Assurance (EQA) limits. The current phase of BV integration into practice is using Patient-Based Real-Time Quality Control (PBRTQC) and Patient Based Quality Assurance (PBQA) to detect a change in assay performance. The challenge of personalised medicine is to determine an individual reference interval. The Athletes Biological Passport may provide the solution.

Errors of measurement for blood volume parameters: a meta-analysis

The volume of red blood cells (V(RBC)) is used routinely in the diagnostic workup of polycythemia, in assessing the efficacy of erythropoietin administration, and to study factors affecting oxygen transport. However, errors of various methods of measurement of V(RBC) and related parameters are not well characterized. We meta-analyzed 346 estimates of error of measurement of V(RBC) for techniques based on Evans blue (V(RBC,Evans)), 51chromium-labeled red blood cells (V(RBC,51Cr)), and carbon monoxide (CO) rebreathing (V(RBC,CO)), as well as hemoglobin mass with the carbon-monoxide method (M(Hb,CO)), in athletes and active and inactive subjects undergoing various experimental and control treatments lasting minutes to months. Subject characteristics and experimental treatments had little effect on error of measurement, but measures with the smallest error showed some increase in error with increasing time between trials. Adjusted to 1 day between trials and expressed as coefficients of variation, mean errors for M(Hb,CO) (2.2%; 90% confidence interval 1.4-3.5%) and V(RBC,51Cr) (2.8%; 2.4-3.2%) were much less than those for V(RBC,Evans) (6.7%; 4.9-9.4%) and V(RBC,CO) (6.7%; 3.4-14%). Most of the error of V(RBC,Evans) was due to error in measurement of volume of plasma via Evans blue dye (6.0%; 4.5-7.8%), which is the basis of V(RBC,Evans). Most of the error in V(RBC,CO) was due to estimates from laboratories with a relatively large error in M(Hb,CO), the basis of V(RBC,CO). V(RBC,51Cr) and M(Hb,CO) are the best measures for research on blood-related changes in oxygen transport. With care, V(RBC,Evans) is suitable for clinical applications of blood-volume measurement.

Comparison of Pooled Fresh Frozen Serum to Proficiency Testing Material in College of American Pathologists Surveys: Cortisol and Immunoglobulin E

Context.—The College of American Pathologists (CAP) provides proficiency testing (PT) surveys to laboratories around the world.

Objectives.—To compare diagnostic assay methods for serum/plasma cortisol and immunoglobulin (Ig) E in terms of their bias and precision, to determine how well CAP PT specimens simulate human serum, and to reassess proficiency test grading criteria in light of these findings.

Design.—A participant-blinded, prospective trial. One vial of pooled fresh frozen serum (FFS) and 4 different admixtures of PT material (PTM) were sent to laboratories participating in PT surveys.

Participants.—Laboratories providing cortisol (>1000) or IgE (>230) results among the subscribers to the CAP surveys, Ligand (General) 2003, set K/KN-A and Chemistry 2003, set C-C.

Main Outcome Measures.—The main outcome measures were (1) bias among laboratories using the same method (peer groups), defined relative to the median of method means (MedMM); (2) imprecision as measured by the SD and coefficient of variation (CV) about each method mean; and (3) total error across laboratories for the FFS cortisol results, defined as |Bias Relative to Reference Method| + 2 SD.

Results.—Cortisol method biases, relative to MedMM, ranged from −22% to 9% for the FFS challenge and from −24% to 36% for comparable PTM challenges. The method biases, relative to the reference method, ranged from −3% to 19% for the FFS challenge. The cortisol method CVs ranged from 4.2% to 13.6% for the FFS challenge and from 4.7% to 12.7% for comparable PTM challenges. Total error across laboratories ranged from 1.4 to 6.9 μg/dL (39 to 190 nmol/L) for the FFS challenge. Immunoglobulin E method biases, relative to MedMM, ranged from −8% to 9% for the FFS challenge and from −7% to 5% for comparable PTM challenges. The IgE method CVs ranged from 3.6% to 6.7% for the FFS challenge and from 3.4% to 9.8% for comparable PTM challenges.

Conclusions.—The bias for cortisol results was less with FFS than with PTM, but imprecision was comparable. The FFS MedMM was 8.5% higher than the reference value. Fresh frozen serum and PTM bias and imprecision for IgE methods were each less than 10%. Because some of the methods demonstrated greater bias when analyzing PTM than FFS, peer group grading of both these analytes is appropriate.

Discordance rate, a new concept for combining diagnostic decisions with analytical performance characteristics. 1. Application in method or sample system comparisons and in defining decision limits

Method comparison studies are usually evaluated by several statistical tests (e.g., regression analysis) which sufficiently describe the analytical (dis)agreement between the results of two procedures. However, they do not provide any information how differences, if observed, influence diagnostic decision making. A novel statistical approach is described to test the clinical relevance of differences between two analytical procedures. The new procedure requires a population-based probability which describes the distribution of values within the population under study and an analytical probability quantifying the risk of errors due to replacing one method by the other. The population probability was derived from 171 subjects from two outpatient departments (internal medicine and dermatology) who were subjected to an oral glucose tolerance test because type 2 diabetes mellitus was suspected. The analytical probability was determined from duplicate glucose measurements in venous and capillary blood, and venous plasma in the fasting and 2 h post-challenge state by the routine method used in a central laboratory (Ebio analyzer) and a (POCT) glucometer (Elite). The two probabilities were combined into one "error rate" (discordance rate). The new concept was applied to three examples. In the first example, a comparison between two analytical systems led to discordance rates above 15%. After transforming the Elite analyzer results by a regression function, the discordance rate decreased below 5%, which was considered to be acceptable for the diagnostic purpose studied. In the second example, discordance rates were estimated by comparing different sample systems with each other. The use of whole blood in comparison with venous plasma led to discordance rates of 5-7% for venous blood and 7-10% for capillary blood. The same data set was also used in a third example to derive decision limits for capillary and venous blood from the established plasma values. The proposed procedure estimates the diagnostic error rate based on analytic performance characteristics and population probabilities. It extends the concept of (un)efficiency by including the effect of variability about a decision limit and the distribution of the measurement values in the patient population.

Description of a generally applicable model for the evaluation of uncertainty of measurement in clinical chemistry

There is a growing pressure on clinical chemistry laboratories to conform to quality standards that require the evaluation and expression of the uncertainty of results of measurement. Nevertheless, there is some reluctance to accept the uncertainty concept in the analytical community due to difficulty in evaluating uncertainty in practice. For example, often the uncertainty of some uncertainty components is not known very well in clinical chemistry measurements, such as those associated with matrix effects or with the values of the calibrators. Moreover, it is not clear how to interpret uncertainty in relation to diagnostic criteria, reference ranges and other decision limits in clinical chemistry practice. Hence, the value of reporting the uncertainty of the measurement result is not obvious. In this paper it is suggested a relatively simple, logical procedure for evaluating measurement uncertainty based on the principles in the Guide for the Expression of Uncertainty of Measurement (GUM). The measurement process is partitioned into elements that are well known to the analyst, namely sampling, calibration, and analysis. The corresponding model function expresses the result of a measurement as the value obtained by the analytical procedure multiplied by the correction factors for sampling bias, for bias caused by the calibrators, and for other types of bias. Under normal conditions, when the measurement procedure is validated and corrected for all known bias, the expected value of each correction factor is one. The uncertainty that remains with regard to sampling, manufacturing of calibrators and other types of bias is combined with the analytical imprecision to yield a combined uncertainty of a result of measurement. The advantages of this approach are: (i) Data from the method validation, internal quality control and from participation in external quality control schemes can be used as input in the uncertainty evaluation process. (ii) The partition of the measurement into well-defined tasks highlights the different responsibilities of the clinical chemistry laboratory and of the manufacturer of reagents and calibrators. (iii) The approach can be used to harmonize the uncertainty evaluation process, which is particularly relevant for laboratories seeking accreditation under ISO 17025. The application of the proposed model is demonstrated by evaluating the uncertainty of a result of a measurement of prolactin in human serum. In the example it is shown how to treat the uncertainty associated with a calibrator supplied with a commercial analytical kit, and how to evaluate the uncertainty associated with matrix effects.