Should we depend on reference intervals from manufacturer package inserts? Comparing TSH and FT4 reference intervals from four manufacturers with results from modern indirect methods and the direct method

OBJECTIVES

Correct interpretation of thyroid function tests relies on correct reference intervals (RIs) for thyroid-stimulating hormone (TSH) and free thyroxine (FT4). ISO15189 mandates periodic verification of RIs, but laboratories struggle with cost-effective approaches. We investigated whether indirect methods (utilizing historical laboratory data) could replace the direct approach (utilizing healthy reference individuals) and compared results with manufacturer-provided RIs for TSH and FT4.

METHODS

We collected historical data (2008-2022) from 13 Dutch laboratories to re-establish RIs by employing indirect methods, TMC (for TSH) and refineR (for FT4). Laboratories used common automated platforms (Roche, Abbott, Beckman or Siemens). Indirect RIs (IRIs) were determined per laboratory per year and clustered per manufacturer (>1.000.000 data points per manufacturer). Direct RIs (DRIs) were established in 125 healthy individuals per platform.

RESULTS

TSH IRIs remained robust over the years for all manufacturers. FT4 IRIs proved robust for three manufacturers (Roche, Beckman and Siemens), but the IRI upper reference limit (URL) of Abbott showed a decrease of 2 pmol/L from 2015. Comparison of the IRIs and DRIs for TSH and FT4 showed close agreement using adequate age-stratification. Manufacturer-provided RIs, notably Abbott, Roche and Beckman exhibited inappropriate URLs (overall difference of 0.5-1.0 µIU/mL) for TSH. For FT4, the URLs provided by Roche, Abbott and Siemens were overestimated by 1.5-3.5 pmol/L.

CONCLUSIONS

These results underscore the importance of RI verification as manufacturer-provided RIs are often incorrect and RIs may not be robust. Indirect methods offer cost-effective alternatives for laboratory-specific or platform-specific verification of RIs.

Development of next-generation reference interval models to establish reference intervals based on medical data: current status, algorithms and future consideration

Evidence derived from laboratory medicine plays a pivotal role in the diagnosis, treatment monitoring, and prognosis of various diseases. Reference intervals (RIs) are indispensable tools for assessing test results. The accuracy of clinical decision-making relies directly on the appropriateness of RIs. With the increase in real-world studies and advances in computational power, there has been increased interest in establishing RIs using big data. This approach has demonstrated cost-effectiveness and applicability across diverse scenarios, thereby enhancing the overall suitability of the RI to a certain extent. However, challenges persist when tests results are influenced by age and sex. Reliance on a single RI or a grouping of RIs based on age and sex can lead to erroneous interpretation of results with significant implications for clinical decision-making. To address this issue, the development of next generation of reference interval models has arisen at an historic moment. Such models establish a curve relationship to derive continuously changing reference intervals for test results across different age and sex categories. By automatically selecting appropriate RIs based on the age and sex of patients during result interpretation, this approach facilitates clinical decision-making and enhances disease diagnosis/treatment as well as health management practices. Development of next-generation reference interval models use direct or indirect sampling techniques to select reference individuals and then employed curve fitting methods such as splines, polynomial regression and others to establish continuous models. In light of these studies, several observations can be made: Firstly, to date, limited interest has been shown in developing next-generation reference interval models, with only a few models currently available. Secondly, there are a wide range of methods and algorithms for constructing such models, and their diversity may lead to confusion. Thirdly, the process of constructing next-generation reference interval models can be complex, particularly when employing indirect sampling techniques. At present, normative documents pertaining to the development of next-generation reference interval models are lacking. In summary, this review aims to provide an overview of the current state of development of next-generation reference interval models by defining them, highlighting inherent advantages, and addressing existing challenges. It also describes the process, advanced algorithms for model building, the tools required and the diagnosis and validation of models. Additionally, a discussion on the prospects of utilizing big data for developing next-generation reference interval models is presented. The ultimate objective is to equip clinical laboratories with the theoretical framework and practical tools necessary for developing and optimizing next-generation reference interval models to establish next-generation reference intervals while enhancing the use of medical data resources to facilitate precision medicine.

Indirect estimation of pediatric reference interval via density graph deep embedded clustering

Establishing reference intervals (RIs) for pediatric patients is crucial in clinical decision-making, and there is a critical gap of pediatric RIs in China. However, the direct sampling technique for establishing RIs is resource-intensive and ethically challenging. Indirect estimation methods, such as unsupervised clustering algorithms, have emerged as potential alternatives for predicting reference intervals. This study introduces deep graph clustering methods into indirect estimation of pediatric reference intervals. Specifically, we propose a Density Graph Deep Embedded Clustering (DGDEC) algorithm, which incorporates a density feature extractor to enhance sample representation and provides additional perspectives for distinguishing different levels of health status among populations. Additionally, we construct an adjacency matrix by computing the similarity between samples after feature enhancement. The DGDEC algorithm leverages the adjacency matrix to capture the interrelationships between patients and divides patients into different groups, thereby estimating reference intervals for the potential healthy population. The experimental results demonstrate that when compared to other indirect estimation techniques, our method ensures the predicted pediatric reference intervals in different age and gender groups are closer to the true values while maintaining good generalization performance. Additionally, through ablation experiments, our study confirms that the similarity between patients and the multi-scale density features of samples can effectively describe the potential health status of patients.

Comparison of results and age-related changes in establishing reference intervals for CEA, AFP, CA125, and CA199 using four indirect methods

•The reference intervals calculated using RefineR, Kosmic, TMC, and non-parametric methods are similar.•TMC algorithm is more robust, demonstrates a high pass rate among the four methods and has the ability to automatically isolate outliers.•The reference intervals of CA125 and CA199 showed significant differences between age and sex.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC)

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Reference intervals: past, present, and future

Clinical laboratory test results alone are of little value in diagnosing, treating, and monitoring health conditions; as such, a clinically actionable cutoff or reference interval is required to provide context for result interpretation. Healthcare practitioners base their diagnoses, follow-up treatments, and subsequent testing on these reference points. However, they may not be aware of inherent limitations related to the definition and derivation of reference intervals. Laboratorians are responsible for providing the reference intervals they report with results. Yet, the establishment and verification of reference intervals using conventional direct methods are complicated by resource constraints or unique patient demographics. To facilitate standardized reference interval best practices, multiple global scientific societies are actively drafting guidelines and seeking funding to promote these initiatives. Numerous national and international multicenter collaborations demonstrate the ability to leverage combined resources to conduct large reference interval studies by direct methods. However, not all demographics are equally accessible. Novel indirect methods are attractive solutions that utilize computational methods to define reference distributions and reference intervals from mixed data sets of pathologic and non-pathologic patient test results. In an effort to make reference intervals more accurate and personalized, individual-based reference intervals are shown to be more useful than population-based reference intervals in detecting clinically significant analyte changes in a patient that might otherwise go unrecognized when using wider, population-based reference intervals. Additionally, continuous reference intervals can provide more accurate ranges as compared to age-based partitions for individuals that are near the ends of the age partition. The advantages and disadvantages of different reference interval approaches as well as the advancement of non-conventional reference interval studies are discussed in this review.

Current status and challenges in establishing reference intervals based on real-world data

Reference intervals (RIs) are the cornerstone for evaluation of test results in clinical practice and are invaluable in judging patient health and making clinical decisions. Establishing RIs based on clinical laboratory data is a branch of real-world data mining research. Compared to the traditional direct method, this indirect approach is highly practical, widely applicable, and low-cost. Improving the accuracy of RIs requires not only the collection of sufficient data and the use of correct statistical methods, but also proper stratification of heterogeneous subpopulations. This includes the establishment of age-specific RIs and taking into account other characteristics of reference individuals. Although there are many studies on establishing RIs by indirect methods, it is still very difficult for laboratories to select appropriate statistical methods due to the lack of formal guidelines. This review describes the application of real-world data and an approach for establishing indirect reference intervals (iRIs). We summarize the processes for establishing iRIs using real-world data and analyze the principle and applicable scope of the indirect method model in detail. Moreover, we compare different methods for constructing growth curves to establish age-specific RIs, in hopes of providing laboratories with a reference for establishing specific iRIs and giving new insight into clinical laboratory RI research. (201 words).

Verification of Reference Interval of Thyroid Hormones With Manual and Automated Indirect Approaches: Comparison of Hoffman, KOSMIC and refineR Methods

INTRODUCTION

The interpretation of quantitative test results requires the availability of appropriate reference intervals (RIs). Every laboratory has been advised by scientific literature and reagent manufacturers to establish RIs for all analytes. Measuring RIs using direct methods is very costly, and it poses ethical and practical challenges. To overcome these challenges, indirect methods, such as Hoffman, and newer automated approaches, such as KOSMIC and refineR, are used to verify RIs for thyroid hormones.

OBJECTIVE

To verify RIs for thyroid hormones in adult patients using Hoffman, KOSMIC and refineR methods and to compare these with reference ranges given in kit literature or standard textbooks.

MATERIALS AND METHODS

The observed values (results) of thyroid hormone were collected from the LIS (Laboratory Information System) of the Biochemistry Department at the B. J. Medical College and Civil Hospital in Ahmedabad between 1 January 2021 and 31 May 2022. Hoffman, KOSMIC and refineR methods were used to verify the RIs. The computerised Hoffman approach, which Katayev et al. describe, is a simple method for determining RI from hospital data. Zierk et al. pre-validated and suggested the KOSMIC method based on Python programming, whereas refineR was proposed by Tatjana et al. based on R programming language.

RESULTS

Hoffman, KOSMIC and refineR's indirect RI techniques revealed comparable results with kit literature in free T3 and T4, whereas higher upper reference limits of thyroid-stimulating hormone (TSH) compared to kit literature were observed with KOSMIC and refineR methods. However, the computerised Hoffman method revealed comparable results with TSH also.

CONCLUSION

Indirect approaches, such as Hoffman, KOSMIC and refineR, provide reliable RI verification for free T3 and T4 utilising patient samples obtained from LIS. However, the manual Hoffman method provides reliable RI verification for TSH data derived from the hospital population as compared to automated approaches, such as KOSMIC and refineR.

Comparison of reference intervals for biochemical and hematology markers derived by direct and indirect procedures based on the Isfahan cohort study

INTRODUCTION

Indirect methods for reference interval (RI) establishment apply statistical techniques to generate RIs for test result interpretation using stored laboratory data. They present unique advantages relative to traditional direct approaches such as fewer resource requirements; however, there is debate regarding their performance. Herein, we aimed to compare indirect and direct approaches for RI establishment by harnessing data from the Isfahan Cohort Study (ICS). This cohort includes both healthy individuals and those with a history of disease, enabling a direct comparison.

METHODS

Participants were recruited as part of ICS, including 6504 adults aged 34 years and older. Sociodemographic characteristics, anthropometry, blood pressure, various biochemical indices, and hematology parameters were collected. The refineR method was used to establish indirect RIs (before applying exclusion criteria). Direct RIs were calculated using nonparametric methods per CLSI EP28-A3 guidelines (after applying exclusion criteria). Bias ratios were calculated for each parameter to assess significant differences in estimations.

RESULTS

Direct and indirect RI estimations for most hematological and biochemical parameters were comparable. Statistically significant bias ratios between methods were observed for the upper limits of total cholesterol, triglycerides, high-density lipoprotein cholesterol (HDL-C), hemoglobin (female), and platelet count as well as the lower limits of mean corpuscular hemoglobin (female), mean corpuscular volume, hemoglobin, and hematocrit (female).

CONCLUSION

Data presented indicate RIs derived from direct and indirect approaches are similar, but not identical. Further work should focus on the clinical significance of such differences as well as the investigation of necessary data-cleaning criteria before indirect method application.

Definitions and major prerequisites of direct and indirect approaches for estimating reference limits

Reference intervals are established either by direct or indirect approaches. Whereas the definition of direct is well established, the definition of indirect is still a matter of debate. In this paper, a general definition that covers all indirect models presently in use is proposed. With the upcoming popularity of indirect models, it has become evident that further partitioning strategies are required to minimize the risk of patients' false classifications. With indirect methods, such partitions are much easier to execute than with direct methods. The authors believe that the future of reference interval estimation belongs to indirect models with big data pools either from one laboratory or combined from several regional centres (if necessary). Independent of the approach applied, the quality assurance of the pre-analytical and analytical phase, considering biological variables and other confounding factors, is essential.

Age-and sex-specific reference intervals of total cholesterol, LDL cholesterol, HDL cholesterol and non-HDL cholesterol. Comparison of two algorithms for the indirect estimation of reference intervals

Objectives

Reference intervals of total cholesterol concentrations in plasma and of their fractions low-density lipoprotein (LDL)-, high-density lipoprotein (HDL)-and non-HDL concentrations are seldom studied with respect to the relevance of age and sex. Therefore, the effect of age and sex on the reference intervals was reinvestigated with 2 indirect procedures.

Methods

As an indirect approach, the truncated minimum chi-square method was applied. All analyses were performed by computer programs available. The script published on the homepage of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) allows to derive a continuous age dependency of reference intervals together with their confidence and equivalence limits. The results of this approach were compared with those obtained by an indirect method developed more recently, the refineR algorithm.

Results

In the present study, the upper reference limits of total cholesterol varied from 5.1 to 7.8 mmol/L (197–302 mg/dL) depending on various biological variables (as age, sex, inpatients versus outpatients). These upper limits increased with age. Differences between sexes can be neglected except for the age above 80 years. The pattern of reference limits of LDL cholesterol and non-HDL cholesterol paralleled those of total cholesterol. The reference limits of HDL cholesterol were higher in women than in men but were independent of age.

Conclusions

Reference limits for the concentrations of total cholesterol and their fractions LDL-, HDL-and non-HDL concentrations should be stratified for age and sex.

Harmonization of indirect reference intervals calculation by the Bhattacharya method

OBJECTIVES

The aim of this study was to harmonize the criteria for the Bhattacharya indirect method Microsoft Excel Spreadsheet for reference intervals calculation to reduce between-user variability and use these criteria to calculate and evaluate reference intervals for eight analytes in two different years.

METHODS

Anonymized laboratory test results from outpatients were extracted from January 1st 2018 to December 31st 2019. To assure data quality, we examined the monthly results from an external quality control program. Reference intervals were determined by the Bhattacharya method with the St Vincent's hospital Spreadsheet firstly using original criteria and then using additional harmonized criteria defined in this study. Consensus reference intervals using the additional harmonized criteria were calculated as the mean of four users' lower and upper reference interval results. To further test the operation criteria and robustness of the obtained reference intervals, an external user validated the Spreadsheet procedure.

RESULTS

The extracted test results for all selected laboratory tests fulfilled the quality criteria and were included in the present study. Differences between users in calculated reference intervals were frequent when using the Spreadsheet. Therefore, additional criteria for the Spreadsheet were proposed and applied by independent users, such as: to set central bin as the mean of all the data, bin size as small as possible, at least three consecutive bins and a high proportion of bins within the curve.

CONCLUSIONS

The proposed criteria contributed to the harmonization of reference interval calculation between users of the Bhattacharya indirect method Spreadsheet.

Reference Intervals in Coagulation Analysis

Blood coagulation analysis is characterized by the application of a variety of materials, reagents, and analyzers for the determination of the same parameter, or analyte, by different laboratories worldwide. Accordingly, the application of common reference intervals, that, by definition, would represent a "range of values (of a certain analyte) that is deemed normal for a physiological measurement in healthy persons," is difficult to implement without harmonization of procedures. In fact, assay-specific reference intervals are usually established to allow for the discrimination of normal and abnormal values during evaluation of patient results. While such assay-specific reference intervals are often determined by assay manufacturers and subsequently adopted by customer laboratories, verification of transferred values is still mandatory to confirm applicability on site. The same is true for reference intervals that have been adopted from other laboratories, published information, or determined by indirect data mining approaches. In case transferable reference intervals are not available for a specific assay, a direct recruiting approach may or needs to be applied. In comparison to transferred reference interval verification, however, the direct recruiting approach requires a significantly higher number of well-defined samples to be collected and analyzed. In the present review, we aim to give an overview on the above-mentioned aspects and procedures, also with respect to relevant standards, regulations, guidelines, but also challenges for both, assay manufacturers and coagulation laboratories.

RIbench: A Proposed Benchmark for the Standardized Evaluation of Indirect Methods for Reference Interval Estimation

BACKGROUND

Indirect methods leverage real-world data for the estimation of reference intervals. These constitute an active field of research, and several methods have been developed recently. So far, no standardized tool for evaluation and comparison of indirect methods exists.

METHODS

We provide RIbench, a benchmarking suite for quantitative evaluation of any existing or novel indirect method. The benchmark contains simulated test sets for 10 biomarkers mimicking routine measurements of a mixed distribution of non-pathological (reference) values and pathological values. The non-pathological distributions represent 4 common distribution types: normal, skewed, heavily skewed, and skewed-and-shifted. To identify strengths and weaknesses of indirect methods, test sets have varying sample sizes and pathological distributions differ in location, extent of overlap, and fraction. For performance evaluation, we use an overall benchmark score and sub-scores derived from absolute z-score deviations between estimated and true reference limits. We illustrate the application of RIbench by evaluating and comparing the Hoffmann method and 4 modern indirect methods -TML (Truncated-Maximum-Likelihood), kosmic, TMC (Truncated-Minimum-Chi-Square), and refineR- against one another and against a nonparametric direct method (n = 120).

RESULTS

For the modern indirect methods, pathological fraction and sample size had a strong influence on the results: With a pathological fraction up to 20% and a minimum sample size of 5000, most methods achieved results comparable or superior to the direct method.

CONCLUSIONS

We present RIbench, an open-source R-package, for the systematic evaluation of existing and novel indirect methods. RIbench can serve as a tool for enhancement of indirect methods, improving the estimation of reference intervals.

Parametric and non-parametric estimation of reference intervals for routine laboratory tests: an analysis of health check-up data for 260 889 young men in the South Korean military

OBJECTIVES

Determination of reference intervals (RIs) using big data faces several obstacles due to heterogeneity in analysers, period and ethnicity. The present study aimed to establish the RIs for routine common blood count (CBC) and biochemistry laboratory tests in homogeneous, healthy, male Korean soldiers in their 20s using a large health check-up data set, comparing parametric and non-parametric estimation.

DESIGN

A multicentre, cross-sectional study.

SETTING

Seven armed forces hospitals in South Korea.

PARTICIPANTS

A total of 609 649 men underwent health examination when promoted to corporal between January 2015 and September 2021. 260 889 eligible individuals aged 20-25 were included in the analysis.

MAIN OUTCOMES AND MEASURES

The RIs were established by parametric and non-parametric methods. In the parametric approach, maximum likelihood estimation was applied to measure the Box-Cox transformation parameter and the values at the 2.5th and 97.5th percentiles were recalculated. The non-parametric approach adopted the Tukey's exclusion test and the values at the 2.5th and 97.5th percentiles were obtained. Classification by body mass index was also performed.

RESULTS

The obtained RIs for haematology parameters were comparable between devices. If the values followed a Gaussian distribution, parametric and non-parametric methods were well matched for haematology and biochemical markers. When the values were right-skewed, the upper limits were higher with parametric than with non-parametric methods. Participants with obesity showed higher RIs for CBC, some liver function tests and some lipid profiles than participants without obesity.

CONCLUSIONS

Using data from healthy, male Korean soldiers in their 20s, we proposed the RIs for CBC and biochemical parameters, comparing parametric and non-parametric estimation. As such approaches based on large data sets become more prevalent, further studies are needed to discriminate eligible individuals and determine RIs in an extrapolated sample.

Indirect determination of biochemistry reference intervals using outpatient data

The aim of this study was to determine reference intervals in an outpatient population from Vall d'Hebron laboratory using an indirect approach previously described in a Dutch population (NUMBER project). We used anonymized test results from individuals visiting general practitioners and analysed during 2018. Analytical quality was assured by EQA performance, daily average monitoring and by assessing longitudinal accuracy between 2018 and 2020 (using trueness verifiers from Dutch EQA). Per test, outliers by biochemically related tests were excluded, data were transformed to a normal distribution (if necessary) and means and standard deviations were calculated, stratified by age and sex. In addition, the reference limit estimator method was also used to calculate reference intervals using the same dataset. Finally, for standardized tests reference intervals obtained were compared with the published NUMBER results. Reference intervals were calculated using data from 509,408 clinical requests. For biochemical tests following a normal distribution, similar reference intervals were found between Vall d'Hebron and the Dutch study. For creatinine and urea, reference intervals increased with age in both populations. The upper limits of Gamma-glutamyl transferase were markedly higher in the Dutch study compared to Vall d'Hebron results. Creatine kinase and uric acid reference intervals were higher in both populations compared to conventional reference intervals. Medical test results following a normal distribution showed comparable and consistent reference intervals between studies. Therefore a simple indirect method is a feasible and cost-efficient approach for calculating reference intervals. Yet, for generating standardized calculated reference intervals that are traceable to higher order materials and methods, efforts should also focus on test standardization and bias assessment using commutable trueness verifiers.

Comparison of 8 methods for univariate statistical exclusion of pathological subpopulations for indirect reference intervals and biological variation studies

BACKGROUND

Indirect reference intervals and biological variation studies heavily rely on statistical methods to separate pathological and non-pathological subpopulations within the same dataset. In recognition of this, we compare the performance of eight univariate statistical methods for identification and exclusion of values originating from pathological subpopulations.

METHODS

The eight approaches examined were: Tukey's rule with and without Box-Cox transformation; median absolute deviation; double median absolute deviation; Gaussian mixture models; van der Loo (Vdl) methods 1 and 2; and the Kosmic approach. Using four scenarios including lognormal distributions and varying the conditions through the number of pathological populations, central location, spread and proportion for a total of 256 simulated mixed populations. A performance criterion of ±0.05 fractional error from the true underlying lower and upper reference interval was chosen.

RESULTS

Overall, the Kosmic method was a standout with the highest number of scenarios lying within the acceptable error, followed by Vdl method 1 and Tukey's rule. Kosmic and Vdl method 1 appears to discriminate better the non-pathological reference population in the case of log-normal distributed data. When the proportion and spread of pathological subpopulations is high, the performance of statistical exclusion deteriorated considerably.

DISCUSSIONS

It is important that laboratories use a priori defined clinical criteria to minimise the proportion of pathological subpopulation in a dataset prior to analysis. The curated dataset should then be carefully examined so that the appropriate statistical method can be applied.

Big data and reference intervals

Although reference intervals (RIs) play an important role in clinical diagnosis, there remain significant differences with respect to race, gender, age and geographic location. Accordingly, the Clinical Laboratory Standards Institute (CLSI) EP28-A3c has recommended that clinical laboratories establish RIs appropriate to their subject population. Unfortunately, the traditional and direct approach to establish RIs relies on the recruitment of a sufficient number of healthy individuals of various age groups, collection and testing of large numbers of specimens and accurate data interpretation. The advent of the big data era has, however, created a unique opportunity to "mine" laboratory information. Unfortunately, this indirect method lacks standardization, consensus support and CLSI guidance. In this review we provide a historical perspective, comprehensively assess data processing and statistical methods, and post-verification analysis to validate this big data approach in establishing laboratory specific RIs.

The Reference Intervals of Whole Blood Copper, Zinc, Calcium, Magnesium, and Iron in Infants Under 1 Year Old

Trace elements are essential nutrients for the optimal growth, development, and health of infants, and the reference intervals (RIs) from these trace elements in the blood are very important for an accurate assessment of the status of the elements. In this study, blood samples from a total of 13,446 infants (7206 boys and 6240 girls) were used, and the copper (Cu), zinc (Zn), calcium (Ca), magnesium (Mg), and iron (Fe) in their blood were determined using atomic absorption spectrometry. After clearing the data and removing any outliers, the gender- and age-specific RIs obtained from the Cu, Zn, Ca, Mg, and Fe in the infants' blood were established according to the principles of the Clinical and Laboratory Standards Institute (CLSI) C28-A3. In the multivariable analysis, after making the relevant adjustments for the confounding factors, the age of the infants showed a significant positive correlation with the concentrations of Zn, Ca, Mg, and Fe found in the blood (p<0.01). Furthermore, there were obvious differences in the Cu, Zn, and Ca levels in the blood according to the gender of the infants (p<0.01). As infants are in the critical period of their growth and development, the gender- and age-specific RIs may provide helpful guidance for the nutritional status of the Cu, Zn, Ca, Mg, and Fe elements in the infants' blood.

Data mining of pediatric reference intervals

Laboratory tests are essential to assess the health status and to guide patient care in individuals of all ages. The interpretation of quantitative test results requires availability of appropriate reference intervals, and reference intervals in children have to account for the extensive physiological dynamics with age in many biomarkers. Creation of reference intervals using conventional approaches requires the sampling of healthy individuals, which is opposed by ethical and practical considerations in children, due to the need for a large number of blood samples from healthy children of all ages, including neonates and young infants. This limits the availability and quality of pediatric reference intervals, and ultimately negatively impacts pediatric clinical decision-making. Data mining approaches use laboratory test results and clinical information from hospital information systems to create reference intervals. The extensive number of available test results from laboratory information systems and advanced statistical methods enable the creation of pediatric reference intervals with an unprecedented age-related accuracy for children of all ages. Ongoing developments regarding the availability and standardization of electronic medical records and of indirect statistical methods will further improve the benefit of data mining for pediatric reference intervals.

refineR: A Novel Algorithm for Reference Interval Estimation from Real-World Data

Reference intervals are essential for the interpretation of laboratory test results in medicine. We propose a novel indirect approach to estimate reference intervals from real-world data as an alternative to direct methods, which require samples from healthy individuals. The presented refineR algorithm separates the non-pathological distribution from the pathological distribution of observed test results using an inverse approach and identifies the model that best explains the non-pathological distribution. To evaluate its performance, we simulated test results from six common laboratory analytes with a varying location and fraction of pathological test results. Estimated reference intervals were compared to the ground truth, an alternative indirect method (kosmic), and the direct method (N = 120 and N = 400 samples). Overall, refineR achieved the lowest mean percentage error of all methods (2.77%). Analyzing the amount of reference intervals within ± 1 total error deviation from the ground truth, refineR (82.5%) was inferior to the direct method with N = 400 samples (90.1%), but outperformed kosmic (70.8%) and the direct method with N = 120 (67.4%). Additionally, reference intervals estimated from pediatric data were comparable to published direct method studies. In conclusion, the refineR algorithm enables precise estimation of reference intervals from real-world data and represents a viable complement to the direct method.

The importance of correct stratifications when comparing directly and indirectly estimated reference intervals

OBJECTIVES

There are generally two major reasons for the comparison of reference intervals (RIs): when externally determined RIs (from the literature or provided by a manufacturer) are compared with presently used intra-laboratory RIs and when indirectly estimated RIs are compared with directly established RIs. Discrepancies within these comparisons may occur for two reasons: 1. the pre-analytical and/or analytical conditions do not agree and/or 2. biological variables influencing the establishment of RIs have not been considered adequately. If directly and indirectly estimated reference intervals (RIs) are compared with each other, they very often agree. Sometimes, however, a comparison may differ, with the reason for any discrepancy not being further studied. A major reason for differences in the comparison of RIs is that the requirement for stratification has been neglected.

METHODS

The present report outlines the consequences to RI comparison if stratification is neglected during RI determination with the main variables affecting RIs being sex and age. Alanine aminotransferase was chosen as an example in which the RIs depend on both these factors.

RESULTS

Both direct and indirect approaches lead to erroneous RIs if stratification for variables which are known to affect the estimation of RIs is not performed adequately. However, failing to include a required stratification in procedures for directly determined RIs affects the outcome in a different way to indirectly determined RIs.

CONCLUSIONS

The resulting difference between direct and indirect RIs is often misinterpreted as an incorrect RI estimation of the indirect method.

Calculation of indirect reference intervals of plasma lipase activity of adults from existing laboratory data based on the Reference Limit Estimator integrated in the OPUS::L information system

Objectives

Most laboratories have difficulties to determine their own reference intervals for the diagnostic evaluation of patient results by direct methods. Therefore, data is often just taken from the literature or package inserts of the analytical tests.

Methods

The section on Reference Limits of the German Society for Clinical Chemistry and Laboratory Medicine (DGKL) first uploaded the Reference Limit Estimator (RLE) as an R-program with MS Excel-interface on the DGKL home page and now this tool is implemented in the commercial Laboratory Information System OPUS::L (OSM AG Essen, Germany). We used this OPUS::L “Population specific Reference Limits” tool online with our laboratory database. First calculations were done using the example of lipase.

Results

The manufacturer’s original reference interval for lipase 12–53 U/L (adults) was changed to age dependent upper reference limits of <41 U/L (<20 years), <60 U/L (20–80 years) and <70 U/L (>80 years).

Conclusions

By means of the OPUS::L “Population specific Reference Limits” tool we were able to establish our laborarotry specific reference interval for plasma lipase activity. The new reference limits helped to solve an old problem of implausible low elevated lipase values.

Review of potentials and limitations of indirect approaches for estimating reference limits/intervals of quantitative procedures in laboratory medicine

Reference intervals (RIs) can be determined by direct and indirect procedures. Both approaches identify a reference population from which the RIs are defined. The crucial difference between direct and indirect methods is that direct methods select particular individuals after individual anamnesis and medical examination have confirmed the absence of pathological conditions. These individuals form a reference subpopulation. Indirect methods select a reference subpopulation in which the individuals are not identified. They isolate a reference population from a mixed population of patients with pathological and non-pathological conditions by statistical reasoning.

At present, the direct procedure internationally recommended is the “gold standard”. It has, however, the disadvantage of high expenses which cannot easily be afforded by most medical laboratories. Therefore, laboratories adopt RIs established by direct methods from external sources requiring a high responsibility for transference problems which are usually neglected by most laboratories. These difficulties can be overcome by indirect procedures which can easily be performed by most laboratories without causing economic problems.

The present review focuses on indirect approaches. Various procedures are presented with their benefits and limitations. Preliminary simulation studies indicate that more recently developed concepts are superior to older approaches.

Separating disease and health for indirect reference intervals

The indirect approach to defining reference intervals operates ‘a posteriori’, on stored laboratory data. It relies on being able to separate healthy and diseased populations using one or both of clinical techniques or statistical techniques. These techniques are also fundamental in a priori, direct reference interval approaches. The clinical techniques rely on using clinical data that is stored either in the electronic health record or within the laboratory database, to exclude patients with possible disease. It depends on the investigators understanding of the data and the pathological impacts on tests. The statistical technique relies on identifying a dominant, apparently healthy, typically Gaussian distribution, which is unaffected by the overlapping populations with higher (or lower) results. It depends on having large databases to give confidence in the extrapolation of the narrow portion of overall distribution representing unaffected individuals. The statistical issues involved can be complex, and can result in unintended bias, particularly when the impacts of disease and the physiological variations in the data are under appreciated.

Indirect estimation of reference intervals using first or last results and results from patients without repeated measurements

Objectives

Indirect methods for the estimation of Reference Limits (RLs) use large data pools stored in modern laboratory information’s systems. To avoid correlation between observations repeated results from each patient should be excluded. Some data pools obtained are anonymized, and thereafter the data cannot be re-identified. The effect of the procedure of data selection on the estimations is not investigated yet.

Methods

We considered four parameters. Data sets were enclosed from two sources: a university hospital and a laboratory primarily reflecting a patient population from medical practitioners. Four algorithms were used for data selection, which generate first, last, all and non-repeated values. RLs were estimated through these data sets and compared.

Results

This study showed the broader reference range estimated by indirect methods if using the whole data set compared to first/last values or non-repeated values.

Conclusions

The use of all data without a filtering step results in a significant bias whereas the choice of first or last values has nearly no impact. The exclusion of repeated measurements results in narrower RLs. This influence confine the use of anonymous data sets where filtering is impossible for the estimation of RLs by indirect methods.

Big data and reference intervals: rationale, current practices, harmonization and standardization prerequisites and future perspectives of indirect determination of reference intervals using routine data

Reference intervals are commonly used as a decision-making tool. In this review, we provide an overview on "big data" and reference intervals, describing the rationale, current practices including statistical methods, essential prerequisites concerning data quality, including harmonization and standardization, and future perspectives of the indirect determination of reference intervals using routine laboratory data.

Diurnal variation of leukocyte counts affects the indirect estimation of reference intervals

It has been observed that the estimation of reference intervals of leukocytes in whole venous blood leads to higher upper reference limits (uRLs) with indirect methods than has been reported in the literature determined by direct approaches. This phenomenon was reinvestigated with a newer, more advanced indirect method, and could be confirmed. Furthermore, a diurnal variation was observed with lower values during the morning and higher values in the late afternoon and at night. This observation can explain why indirect approaches using samples collected during 24 h lead to higher uRLs than direct methods applied on samples collected presumably in the morning.

Reference limits of high-sensitive cardiac troponin T indirectly estimated by a new approach applying data mining. A special example for measurands with a relatively high percentage of values at or below the detection limit

A new model for the indirect estimation of reference limits (RLs) has been proposed recently and was coined TMC approach (truncated minimum chi-square estimation) which can be performed with R statistic. A spline function is applied to the RLs to get a continuous function if age is graphically presented vs. the RLs avoiding artificial “jumps” between different age groups. Most indirect models assume a power normal distribution and fail if this assumption is not fulfilled as e.g. if a relatively high percentage of measured values is below the detection limit and the data are distributed extremely skewed. This problem is handled by the TMC model. High-sensitive cardiac troponin T (hs cTnT) was chosen as an example. The hs cTnT concentration in serum or plasma is well accepted as a valuable marker in the diagnosis of acute myocardial infarction. Currently, the 99th percentile derived from a “healthy” subpopulation is the decision limit recommended by consensus groups. However, this decision limit is questioned by several authors for many reasons. In the present report, the 97.5th and the 99th percentile limits were reinvestigated by the TMC model with different subpopulations stratified according to age and sex and were finally compared to presently recommended decision limits. In summary, the generally recommended 99th percentile as a fixed decision limit should be reconsidered. It is suggested to apply more specific reference limits stratified for age and sex instead of a fixed decision limit.

Age and sex dependent reference intervals for random plasma/serum glucose concentrations related to different sampling devices and determined by an indirect procedure with data mining

The glucose concentration in plasma or serum is one of the most often requested analytical values in laboratory medicine. Whereas the analytical part of the glucose determination is well standardised, the standardisation of the pre-examination part (pre-analytical phase) is not sufficiently solved, yet. In view of the present controversial discussion regarding the most efficient prevention of pre-analytical glycolysis, the question arises whether the economical and logistic expenses for inhibiting glycolysis determining random glucose concentration are justified. In hospitals with adequate logistics (e.g. pneumatic tube systems for blood tubes) to guarantee a blood sample transport time of about 1 – 2 h, plasma or serum without prevention of glycolysis can be applied for random glucose concentrations if the reference limits are estimated by the laboratory. If such logistics are not available, especially in primary care services, either plasma or serum samples or whole blood in special tubes with anti-glycolytic additives may be sent to the laboratory.

Quantitative laboratory results: normal or lognormal distribution?

The identification of a suitable distribution model is a prerequisite for the parametric estimation of reference intervals and other statistical laboratory tasks. Classification of normal vs. lognormal distributions from healthy populations is easy, but from mixed populations, containing unknown proportions of abnormal results, it is challenging. We demonstrate that Bowley’s skewness coefficient differentiates between normal and lognormal distributions. This classifier is robust and easy to calculate from the quartiles Q1–Q3 according to the formula (Q1 − 2 · Q2 + Q3)/(Q3 − Q1). We validate our algorithm with a more complex procedure, which optimizes the exponent λ of a power transformation. As a practical application, we show that Bowley’s skewness coefficient is suited selecting the adequate distribution model for the estimation of reference limits according to a recent International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) recommendation, especially if the data is right-skewed.

Age- and sex-dependent reference intervals for uric acid estimated by the truncated minimum chi-square (TMC) approach, a new indirect method

Background

Although the concentration of uric acid in serum or plasma is known to depend on sex and age and is subject to diurnal variation, the influence of these covariates on the reference interval (RI) is often neglected. Consequently, the values in the literature vary considerably. Therefore, we reinvestigated the reference limits and their dependence on covariates.

Methods

A new indirect approach was applied which derives a continuous function between age and RIs avoiding the usual “jumps” between various age groups.

Results

It is confirmed that the uric acid concentration in women is lower than in men. The RIs increase with age, in women more than in men. Between 80 and 90 years of age, the upper RI limit (RL) approximately reaches the same level in both sexes. Because the uric acid concentration may indicate renal insufficiency, the concentrations of creatinine and cystatin C were also measured. Both measurands showed the same behaviour as uric acid. Therefore, the age and sex dependency should be considered if the uric acid concentration is used as an indicator for hyperuricaemia (e.g. caused by gout or other metabolic diseases). Furthermore, a diurnal variation was observed.

Conclusions

Due to the variations of various covariates (age, sex, daytime, analytical systems), it is recommended that each laboratory should estimate its own RIs.