Are analytical performance specifications derived from reference intervals of any use in the veterinary clinical laboratory? A preliminary study on the empirical biological variation model

BACKGROUND

Analytical performance specifications (APS) are vital for method evaluation and quality control validation. However, the limited availability of biological variation (BV) data, regulatory guidelines, and expert opinion (EO) may present challenges in veterinary medicine. The empirical biological variation (EBV) approach, based on population reference intervals (pRI), has emerged as an alternative method to derive APS in human medicine.

OBJECTIVES

This study aimed to assess the practicality and usefulness of the EBV approach in deriving performance limits for various measurands in dogs and cats.

METHODS

Eight hematology and 13 biochemistry measurands were analyzed in dogs and cats. Estimates of combined biologic variation based on traditional biological (CVB ) and EBV-derived (CVE *) formulas were calculated and assessed for evidence of correlation. Performance limits for expanded uncertainty/total error and imprecision were compared among EO, BV, and EBV.

RESULTS

Strong and significant correlations were found between CVB and CVE * for both dogs (r = .86, p < .00001) and cats (r = 0.95, p < .00001). The EBV-derived APS were generally comparable to EO and BV, with a subjective criterion of 1.5% difference for imprecision and 3% for total error/expanded uncertainty.

CONCLUSION

The EBV approach, using pRI, shows promise as a surrogate marker for biological variation and as a practical tool for determining performance limits in dogs and cats. Assuming accurate pRI generated on analyzers with stable analytical performance, this approach could offer benefits when expert recommendations or robust BV studies are lacking or yield conflicting results. Further research is needed to explore the applicability and advantages of the EBV in veterinary medicine.

Verification of sex- and age-specific reference intervals for 13 serum steroids determined by mass spectrometry: evaluation of an indirect statistical approach

OBJECTIVES

Conventionally, reference intervals are established by direct methods, which require a well-characterized, obviously healthy study population. This elaborate approach is time consuming, costly and has rarely been applied to steroid hormones measured by mass spectrometry. In this feasibility study, we investigate whether indirect methods based on routine laboratory results can be used to verify reference intervals from external sources.

METHODS

A total of 11,259 serum samples were used to quantify 13 steroid hormones by mass spectrometry. For indirect estimation of reference intervals, we applied a "modified Hoffmann approach", and verified the results with a more sophisticated statistical method (refineR). We compared our results with those of four recent studies using direct approaches.

RESULTS

We evaluated a total of 81 sex- and age-specific reference intervals, for which at least 120 measurements were available. The overall agreement between indirectly and directly determined reference intervals was surprisingly good as nearly every fourth reference limit could be confirmed by narrow tolerance limits. Furthermore, lower reference limits could be provided for some low concentrated hormones by the indirect method. In cases of substantial deviations, our results matched the underlying data better than reference intervals from external studies.

CONCLUSIONS

Our study shows for the first time that indirect methods are a valuable tool to verify existing reference intervals for steroid hormones. A simple "modified Hoffmann approach" based on the general assumption of a normal or lognormal distribution model is sufficient for screening purposes, while the refineR algorithm may be used for a more detailed analysis.

Review of potentials and limitations of indirect approaches for estimating reference limits/intervals of quantitative procedures in laboratory medicine

Reference intervals (RIs) can be determined by direct and indirect procedures. Both approaches identify a reference population from which the RIs are defined. The crucial difference between direct and indirect methods is that direct methods select particular individuals after individual anamnesis and medical examination have confirmed the absence of pathological conditions. These individuals form a reference subpopulation. Indirect methods select a reference subpopulation in which the individuals are not identified. They isolate a reference population from a mixed population of patients with pathological and non-pathological conditions by statistical reasoning.

At present, the direct procedure internationally recommended is the “gold standard”. It has, however, the disadvantage of high expenses which cannot easily be afforded by most medical laboratories. Therefore, laboratories adopt RIs established by direct methods from external sources requiring a high responsibility for transference problems which are usually neglected by most laboratories. These difficulties can be overcome by indirect procedures which can easily be performed by most laboratories without causing economic problems.

The present review focuses on indirect approaches. Various procedures are presented with their benefits and limitations. Preliminary simulation studies indicate that more recently developed concepts are superior to older approaches.

Indirect estimation of reference intervals using first or last results and results from patients without repeated measurements

Objectives

Indirect methods for the estimation of Reference Limits (RLs) use large data pools stored in modern laboratory information’s systems. To avoid correlation between observations repeated results from each patient should be excluded. Some data pools obtained are anonymized, and thereafter the data cannot be re-identified. The effect of the procedure of data selection on the estimations is not investigated yet.

Methods

We considered four parameters. Data sets were enclosed from two sources: a university hospital and a laboratory primarily reflecting a patient population from medical practitioners. Four algorithms were used for data selection, which generate first, last, all and non-repeated values. RLs were estimated through these data sets and compared.

Results

This study showed the broader reference range estimated by indirect methods if using the whole data set compared to first/last values or non-repeated values.

Conclusions

The use of all data without a filtering step results in a significant bias whereas the choice of first or last values has nearly no impact. The exclusion of repeated measurements results in narrower RLs. This influence confine the use of anonymous data sets where filtering is impossible for the estimation of RLs by indirect methods.

Individualized Correction of the Interference of Hemolysis on Glycated Albumin Determined by the Ketamine Oxidase Method

OBJECTIVE

To investigate the effect of hemolysis on glycated albumin (GA) levels, as determined by the ketamine oxidase method.

METHODS

GA levels and the hemolysis index were determined in nonhemolyzed serum and hemolyzed serum from corresponding patients. We developed an equation to correct the interference of hemolysis on GA, using multiple regression analysis.

RESULTS

The degree of hemolysis was negatively correlated with GA levels (R2 = 0.9500). A correction equation for GA (corrected GA = 2.703 × OD of hemolysis + 1.044 × measured GA -0.906) can revert GA concentrations of hemolyzed specimens to values that were not significantly different from the GA concentration of corresponding nonhemolyzed specimens. The bias of GA concentrations before and after correction was statistically significantly different (P <.01).

CONCLUSIONS

Our results indicate that the level of GA measured through the ketamine oxidase method is negatively affected by hemolysis. The individualized correction of GA results provides increased accuracy in hemolyzed specimens.

Evaluating sample stability in the clinical laboratory with the help of linear and non-linear regression analysis

As it is common practice for laboratories to store patient samples for a predefined period, allowing clinicians to request additional tests on previously collected samples, knowledge about sample stability is indispensable for the laboratorian. A common approach to estimating the maximum storage time is to use a discrete study design, measuring the analyte of interest at various time-points and then checking for significant differences with the help of a statistical test, such as Student's t-test, Wilcoxon's test or an analysis of variance (ANOVA) test. Because only discrete time intervals are considered, stability data can just be approximated. Alternatively, a continuous study design, as described by the Clinical and Laboratory Standards Institute (CLSI) for performing stability experiments for in vitro diagnostic reagents, can also be adopted by the clinical laboratory to evaluate the stability of biological samples. The major advantage of this approach is that it allows laboratories to define individual stability limits for different medical situations and offers more flexibility when choosing time-points for measurements. The intent of this paper is to demonstrate the evaluation of sample stability in the clinical laboratory with a continuous study design implemented with linear or non-linear regression analysis. Appropriate statistical modeling and acceptance criteria are presented, stability functions are described briefly, and checking the overall validity of the results is discussed.

A practical guide to validation and verification of analytical methods in the clinical laboratory

Although the measures to improve quality in the clinical laboratory have been enormous in the past years, not least of all due to the introduction of the ISO standards 15189 and 17025, the handling of validation and verification of method performance often still differs widely from laboratory to laboratory. Much of what is published on the topic contains complex statistics and is difficult to implement in routine laboratories. The result is, that this point is often neglected or implemented incorrectly, which in turn can lead to false conclusions about method performances, potentially compromising patient safety or contributing to incorrect diagnoses. As it has long become a standard requirement for accredited laboratories to evaluate and document the analytical performance of all methods not only prior to their first implementation, but also during ongoing operation, there is a need for clear, standardized and practical guidelines on the subject. This review summarizes the current literature on the topic, focusing on the requirements for method validations, or as the case may be, verifications and describes when to validate, when to verify and which statistical tests are appropriate for each. Proper interpretation of statistical test results and acceptance criteria for each procedure are alluded to. Specific topics, which are addressed, are precision and bias verification of quantitative, qualitative and semi-quantitative procedures, method comparisons with Bland-Altman Plots, Passing-Bablok regression analysis, 2×2 contingency tables and bubble charts, linearity studies, analytical sensitivity and specificity, performing carry-over studies and establishing and confirming reference ranges.

The use of error and uncertainty methods in the medical laboratory

Error methods - compared with uncertainty methods - offer simpler, more intuitive and practical procedures for calculating measurement uncertainty and conducting quality assurance in laboratory medicine. However, uncertainty methods are preferred in other fields of science as reflected by the guide to the expression of uncertainty in measurement. When laboratory results are used for supporting medical diagnoses, the total uncertainty consists only partially of analytical variation. Biological variation, pre- and postanalytical variation all need to be included. Furthermore, all components of the measuring procedure need to be taken into account. Performance specifications for diagnostic tests should include the diagnostic uncertainty of the entire testing process. Uncertainty methods may be particularly useful for this purpose but have yet to show their strength in laboratory medicine. The purpose of this paper is to elucidate the pros and cons of error and uncertainty methods as groundwork for future consensus on their use in practical performance specifications. Error and uncertainty methods are complementary when evaluating measurement data.

Immunofluorescence vs immunochemiluminescent methods: AIA 2000 vs Immulite 2000

The aim of this study was to evaluate the measurement of C-peptide, insulin, sex hormone binding globulin (SHBG), and dehydroepiandrosterone sulfate (DHEAS), tests of specific clinical value that are not frequently measured in routine clinical laboratories, carried out using an automated system, AIA 2000 (Tosoh). The obtained data demonstrated that the evaluated system, characterized by fluorescence system detector provides satisfactory analytical performance comparable to those of different instrument that use the well known and widely diffused chemiluminescent principle (Immulite 2000). These characteristics allow an excellent comparability between methods for the measurement of same specific hormones not frequently used in clinical practice.

LEGO plot for simultaneous application of multiple quality requirements during trueness verification of quantitative laboratory tests

BACKGROUND

We developed a two-dimensional plot for viewing trueness that takes into account potential shift and variable quality requirements to verify trueness using certified reference material (CRM).

METHODS

Glucose, total cholesterol (TC), and creatinine levels were determined by two kinds of assay in two levels of a CRM. Available quality requirements were collected, codified, and sorted in an ascending order in the plot's header row. Centering on the mean of measured values from CRM, the "mean ± US CLIA '88 allowable total error" was located in the header of the leftmost and rightmost columns. Twenty points were created in intervening columns as potential shifts. Uncertainties were calculated according to regression between certified values and uncertainties of CRM, and positioned in the corresponding columns. Cells were assigned different colors where column and row intersected based on comparison of the 95% confidence interval of the percentage bias with each quality requirement.

RESULTS

A glucose assay failed to meet the highest quality criteria, for which shift of +0.13-0.14 mmol/l was required. A TC assay met the quality requirement and a shift of ±0.03 mmol/l was tolerable. A creatinine assay also met the quality requirement but any shift was not tolerable.

CONCLUSION

The plot provides a systematic view of the trueness of quantitative laboratory tests.

The CCLM contribution to improvements in quality and patient safety

Clinical laboratories play an important role in improving patient care. The past decades have seen unbelievable, often unpredictable improvements in analytical performance. Although the seminal concept of the brain-to-brain laboratory loop has been described more than four decades ago, there is now a growing awareness about the importance of extra-analytical aspects in laboratory quality. According to this concept, all phases and activities of the testing cycle should be assessed, monitored and improved in order to decrease the total error rates thereby improving patients’ safety. Clinical Chemistry and Laboratory Medicine (CCLM) not only has followed the shift in perception of quality in the discipline, but has been the catalyst for promoting a large debate on this topic, underlining the value of papers dealing with errors in clinical laboratories and possible remedies, as well as new approaches to the definition of quality in pre-, intra-, and post-analytical steps. The celebration of the 50th anniversary of the CCLM journal offers the opportunity to recall and mention some milestones in the approach to quality and patient safety and to inform our readers, as well as laboratory professionals, clinicians and all the stakeholders of the willingness of the journal to maintain quality issues as central to its interest even in the future.

Permissible performance limits of regression analyses in method comparisons

Method comparisons are indispensable tools for the extensive validation of analytic procedures. Laboratories often only want to know whether an established procedure (x-method) can be replaced by another one (y-method) without interfering with diagnostic purposes. Then split patients' samples are analyzed more or less simultaneously with both procedures designed to measure the same quantity. The measured values are usually presented graphically as a scatter or difference plots. The two methods are considered to be equivalent (comparable) if the data pairs scatter around the line of equality (x=y line) within permissible equivalence lines. It is proposed to derive these limits of permissible imprecision limits which are based on false-positive error rates. If all data pairs are within the limits, both methods lead to comparable false error rates. If one or more data pairs are outside the permissible equivalence limits, the x-method cannot simply be replaced by the y-method and further studies are required. The discordance may be caused either by aberrant values (outliers), non-linearity, bias or a higher variation of e.g., the y-values. The spread around the line of best fit can detect possible interferences if more than 1% of the data pairs are outside permissible spread lines in a scatter plot. Because bias between methods and imprecision can be inter-related, both require specific examinations for their identification.