Observed, unknown distributions of clinical chemical quantities should be considered to be log-normal: a proposal

Verification of sex- and age-specific reference intervals for 13 serum steroids determined by mass spectrometry: evaluation of an indirect statistical approach

OBJECTIVES

Conventionally, reference intervals are established by direct methods, which require a well-characterized, obviously healthy study population. This elaborate approach is time consuming, costly and has rarely been applied to steroid hormones measured by mass spectrometry. In this feasibility study, we investigate whether indirect methods based on routine laboratory results can be used to verify reference intervals from external sources.

METHODS

A total of 11,259 serum samples were used to quantify 13 steroid hormones by mass spectrometry. For indirect estimation of reference intervals, we applied a "modified Hoffmann approach", and verified the results with a more sophisticated statistical method (refineR). We compared our results with those of four recent studies using direct approaches.

RESULTS

We evaluated a total of 81 sex- and age-specific reference intervals, for which at least 120 measurements were available. The overall agreement between indirectly and directly determined reference intervals was surprisingly good as nearly every fourth reference limit could be confirmed by narrow tolerance limits. Furthermore, lower reference limits could be provided for some low concentrated hormones by the indirect method. In cases of substantial deviations, our results matched the underlying data better than reference intervals from external studies.

CONCLUSIONS

Our study shows for the first time that indirect methods are a valuable tool to verify existing reference intervals for steroid hormones. A simple "modified Hoffmann approach" based on the general assumption of a normal or lognormal distribution model is sufficient for screening purposes, while the refineR algorithm may be used for a more detailed analysis.

Subjective assessment of frequency distribution histograms and consequences on reference interval accuracy for small sample sizes: A computer-simulated study

BACKGROUND

Inaccuracy in estimating reference intervals (RIs) is a problem with small sample sizes.

OBJECTIVES

This study aimed to identify the most accurate statistical methods to estimate RIs based on sample size and population distribution shape. We also studied the accuracy of sample frequency distribution histograms to retrieve the original population distribution and compared strategies based on the histogram and goodness-of-fit test.

METHODS

The statistical methods that best enhanced accuracy were determined for various sample sizes (n = 20-60) and population distributions (Gaussian, log-normal, and left-skewed) were determined by repeated-measures ANOVA and posthoc analyses. Frequency distribution histograms were built from 900 samples of five different sizes randomly extracted from six simulated populations. Three reviewers classified the population distributions from visual assessments of a sample histogram, and the classification error rate was calculated. RI accuracy was compared among the strategies based on the histograms and goodness-of-fit tests.

RESULTS

The parametric, nonparametric, and robust methods enhanced lower reference limit estimation accuracy for Gaussian, log-normal, and left-skewed distributions, respectively. The parametric, nonparametric bootstrap, and nonparametric methods enhanced the upper limit estimation accuracy for Gaussian, log-normal, and left-skewed distributions, respectively. Regardless of sample size, sample histogram assessments properly classified the original population distribution 71% to 93.9% of the time, depending on the reviewers. In this study, the strategy based on histograms assessed by the statistician was significantly more precise and accurate than the strategy based on the goodness-of-fit test (P < 0.001).

CONCLUSIONS

A strategy based on histograms might enhance the accuracy of RI estimations. However, relevant inter-reviewer variations in histogram interpretation were detected. Factors affecting inter-reviewer variations should be further explored.

refineR: A Novel Algorithm for Reference Interval Estimation from Real-World Data

Reference intervals are essential for the interpretation of laboratory test results in medicine. We propose a novel indirect approach to estimate reference intervals from real-world data as an alternative to direct methods, which require samples from healthy individuals. The presented refineR algorithm separates the non-pathological distribution from the pathological distribution of observed test results using an inverse approach and identifies the model that best explains the non-pathological distribution. To evaluate its performance, we simulated test results from six common laboratory analytes with a varying location and fraction of pathological test results. Estimated reference intervals were compared to the ground truth, an alternative indirect method (kosmic), and the direct method (N = 120 and N = 400 samples). Overall, refineR achieved the lowest mean percentage error of all methods (2.77%). Analyzing the amount of reference intervals within ± 1 total error deviation from the ground truth, refineR (82.5%) was inferior to the direct method with N = 400 samples (90.1%), but outperformed kosmic (70.8%) and the direct method with N = 120 (67.4%). Additionally, reference intervals estimated from pediatric data were comparable to published direct method studies. In conclusion, the refineR algorithm enables precise estimation of reference intervals from real-world data and represents a viable complement to the direct method.

Review of potentials and limitations of indirect approaches for estimating reference limits/intervals of quantitative procedures in laboratory medicine

Reference intervals (RIs) can be determined by direct and indirect procedures. Both approaches identify a reference population from which the RIs are defined. The crucial difference between direct and indirect methods is that direct methods select particular individuals after individual anamnesis and medical examination have confirmed the absence of pathological conditions. These individuals form a reference subpopulation. Indirect methods select a reference subpopulation in which the individuals are not identified. They isolate a reference population from a mixed population of patients with pathological and non-pathological conditions by statistical reasoning.

At present, the direct procedure internationally recommended is the “gold standard”. It has, however, the disadvantage of high expenses which cannot easily be afforded by most medical laboratories. Therefore, laboratories adopt RIs established by direct methods from external sources requiring a high responsibility for transference problems which are usually neglected by most laboratories. These difficulties can be overcome by indirect procedures which can easily be performed by most laboratories without causing economic problems.

The present review focuses on indirect approaches. Various procedures are presented with their benefits and limitations. Preliminary simulation studies indicate that more recently developed concepts are superior to older approaches.

Quantitative laboratory results: normal or lognormal distribution?

The identification of a suitable distribution model is a prerequisite for the parametric estimation of reference intervals and other statistical laboratory tasks. Classification of normal vs. lognormal distributions from healthy populations is easy, but from mixed populations, containing unknown proportions of abnormal results, it is challenging. We demonstrate that Bowley’s skewness coefficient differentiates between normal and lognormal distributions. This classifier is robust and easy to calculate from the quartiles Q1–Q3 according to the formula (Q1 − 2 · Q2 + Q3)/(Q3 − Q1). We validate our algorithm with a more complex procedure, which optimizes the exponent λ of a power transformation. As a practical application, we show that Bowley’s skewness coefficient is suited selecting the adequate distribution model for the estimation of reference limits according to a recent International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) recommendation, especially if the data is right-skewed.

Reference Interval Estimation from Mixed Distributions using Truncation Points and the Kolmogorov-Smirnov Distance (kosmic)

Appropriate reference intervals are essential when using laboratory test results to guide medical decisions. Conventional approaches for the establishment of reference intervals rely on large samples from healthy and homogenous reference populations. However, this approach is associated with substantial financial and logistic challenges, subject to ethical restrictions in children, and limited in older individuals due to the high prevalence of chronic morbidities and medication. We implemented an indirect method for reference interval estimation, which uses mixed physiological and abnormal test results from clinical information systems, to overcome these restrictions. The algorithm minimizes the difference between an estimated parametrical distribution and a truncated part of the observed distribution, specifically, the Kolmogorov-Smirnov-distance between a hypothetical Gaussian distribution and the observed distribution of test results after Box-Cox-transformation. Simulations of common laboratory tests with increasing proportions of abnormal test results show reliable reference interval estimations even in challenging simulation scenarios, when <20% test results are abnormal. Additionally, reference intervals generated using samples from a university hospital's laboratory information system, with a gradually increasing proportion of abnormal test results remained stable, even if samples from units with a substantial prevalence of pathologies were included. A high-performance open-source C++ implementation is available at https://gitlab.miracum.org/kosmic.

Patient-Based Real-Time Quality Control: Review and Recommendations

For many years the concept of patient-based quality control (QC) has been discussed and implemented in hematology laboratories; however, the techniques have not been widely implemented in clinical chemistry. This is mainly because of the complexity of this form of QC, as it needs to be optimized for each population and often for each analyte. However, the clear advantages of this form of QC, together with the ongoing realization of the shortcomings of “conventional” QC, have driven a need to provide guidance to laboratories to assist in deploying patient-based QC. This overview describes the components of a patient-based QC system (calculation algorithm, block size, truncation limits, control limits) and the relationship of these to the analyte being controlled. We also discuss the need for patient-based QC system optimization using patient data from the individual testing laboratory to reliably detect systematic errors while ensuring that there are few false alarms. The term patient-based real-time quality control covers many activities that use data from patient samples to detect analytical errors. These activities include the monitoring of patient population parameters such as the mean or median analyte value or using single within-patient changes such as the delta check. In this report, we will restrict the discussion to population-based parameters. This overview is intended to serve as a guide for the implementation of a patient-based QC system. The report does not cover the clinical evaluation of the population.

Reference interval estimation of small sample sizes: A methodologic comparison using a computer-simulation study

BACKGROUND

According to the ASVCP and other guidelines, samples should comprise at least 120 individuals for reference interval (RI) estimation. Unfortunately, this minimum sample size is difficult to achieve in veterinary medicine. Several statistical methods are described to determine RIs from small sample sizes, but it is unclear which method provides the best accuracy.

OBJECTIVES

This study aimed to compare statistical strategies for estimating RIs and determine which strategy best enhances accuracy when the sample size is between 20 and 120.

METHODS

Different sample size groups (n = 120, 100, 80, 60, 40, and 20) were randomly selected 50 times from simulated Gaussian, log-normal, and left-skewed populations of 5000 total values. RIs were calculated using seven different statistical strategies comprising robust, parametric, nonparametric, and bootstrap methods, alone or in combination. RI accuracy was compared among these strategies at each sample size. The strategy that was significantly more accurate than others in the largest number of comparisons was considered as the one that best-enhanced RI accuracy.

RESULTS

The strategies that best-enhanced RI accuracy included using the parametric method when the Shapiro-Wilk P > 0.2 and, otherwise, using the nonparametric method to determine the upper and lower RI limits when there were between 60 and 100 reference individuals, and finding the lower RI limit when there were 40 reference individuals. The Box-Cox transformation parametric method best-enhanced RI accuracy of the upper RI limit when there were 40 reference individuals, and the nonparametric method best-enhanced RI accuracy of both RI limits when there were 20 reference individuals.

CONCLUSIONS

Using the parametric method when the Shapiro-Wilk P > 0.2, and the nonparametric method in other instances, will likely enhance RI accuracy when there are between 40 and 100 reference individuals. For smaller samples, the nonparametric method might be preferred.

Stronger Together: Aggregated Z-values of Traditional Quality Control Measurements and Patient Medians Improve Detection of Biases

BACKGROUND

In clinical chemistry, quality control (QC) often relies on measurements of control samples, but limitations, such as a lack of commutability, compromise the ability of such measurements to detect out-of-control situations. Medians of patient results have also been used for QC purposes, but it may be difficult to distinguish changes observed in the patient population from analytical errors. This study aims to combine traditional control measurements and patient medians for facilitating detection of biases.

METHODS

The software package “rSimLab” was developed to simulate measurements of 5 analytes. Internal QC measurements and patient medians were assessed for detecting impermissible biases. Various control rules combined these parameters. A Westgard-like algorithm was evaluated and new rules that aggregate Z-values of QC parameters were proposed.

RESULTS

Mathematical approximations estimated the required sample size for calculating meaningful patient medians. The appropriate number was highly dependent on the ratio of the spread of sample values to their center. Instead of applying a threshold to each QC parameter separately like the Westgard algorithm, the proposed aggregation of Z-values averaged these parameters. This behavior was found beneficial, as a bias could affect QC parameters unequally, resulting in differences between their Z-transformed values. In our simulations, control rules tended to outperform the simple QC parameters they combined. The inclusion of patient medians substantially improved bias detection for some analytes.

CONCLUSIONS

Patient result medians can supplement traditional QC, and aggregations of Z-values are novel and beneficial tools for QC strategies to detect biases.

Indirect reference intervals of plasma and serum thyrotropin (TSH) concentrations from intra-laboratory data bases from several German and Italian medical centres

BACKGROUND

The dogma of establishing intra-laboratory reference limits (RLs) and their periodic review cannot be fulfilled by most laboratories due to the expenses involved. Thus, most laboratories adopt external sources for their RLs, often neglecting the problems of transferability. This is particularly problematic for analytes with a large diversity of existing RLs, as for example thyrotropin (TSH). Several attempts were taken to derive RLs from the large data pools stored in modern laboratory information systems. These attempts were further developed to a more sophisticated indirect procedure. The new approach can be considered a combined concept because it pre-excludes some subjects by direct criteria a-posterior. In the current study, the applicability of the new concept for modern protein bindings assays was examined for estimating RLs of serum and plasma TSH with data sets from several German and Italian laboratories.

METHODS

A smoothed kernel density function was estimated for the distribution of the total mixed data of the sample group (combined data of non-diseased and diseased subjects). It was assumed that the "central" part of the distribution of all data represents the non-diseased ("healthy") population. The central part was defined by truncation points using an optimisation method, and was used to estimate a Gaussian distribution of the values of presumably non-diseased subjects after Box-Cox transformation of the empirical data. This distribution was now considered as the distribution of the non-diseased subgroup. The percentiles of this parametrical distribution were calculated to obtain RLs.

RESULTS

RLs determined by the indirect combined decomposition technique led to similar RLs as found by several recent study reports using a direct method according to international recommendations. Furthermore, the RLs obtained from 13 laboratories in two different European regions reflected the well-known differences of various analytical procedures. Stratification for gender and age was necessary in contrast to earlier reports. With increasing age, an increase of the upper RL and the reference range was observed. Hospitalisation also affected the RLs. Common RLs appeared acceptable only within the same analytical systems. Some laboratories used RLs which were not appropriate for the population served.

CONCLUSIONS

The proposed strategy of combining exclusion criteria with a resolution technique led to retrospective RLs from intra-laboratory data pools for TSH which were comparable with directly determined RLs. Differences between laboratories were due primarily to the well-known bias of the different analytical procedures and to the status of the population.