An overview of EFLM harmonization activities in Europe

Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI)

OBJECTIVES

ChatGPT, a tool based on natural language processing (NLP), is on everyone's mind, and several potential applications in healthcare have been already proposed. However, since the ability of this tool to interpret laboratory test results has not yet been tested, the EFLM Working group on Artificial Intelligence (WG-AI) has set itself the task of closing this gap with a systematic approach.

METHODS

WG-AI members generated 10 simulated laboratory reports of common parameters, which were then passed to ChatGPT for interpretation, according to reference intervals (RI) and units, using an optimized prompt. The results were subsequently evaluated independently by all WG-AI members with respect to relevance, correctness, helpfulness and safety.

RESULTS

ChatGPT recognized all laboratory tests, it could detect if they deviated from the RI and gave a test-by-test as well as an overall interpretation. The interpretations were rather superficial, not always correct, and, only in some cases, judged coherently. The magnitude of the deviation from the RI seldom plays a role in the interpretation of laboratory tests, and artificial intelligence (AI) did not make any meaningful suggestion regarding follow-up diagnostics or further procedures in general.

CONCLUSIONS

ChatGPT in its current form, being not specifically trained on medical data or laboratory data in particular, may only be considered a tool capable of interpreting a laboratory report on a test-by-test basis at best, but not on the interpretation of an overall diagnostic picture. Future generations of similar AIs with medical ground truth training data might surely revolutionize current processes in healthcare, despite this implementation is not ready yet.

The European Register of Specialists in Clinical Chemistry and Laboratory Medicine: code of conduct, version 3 - 2023

Whilst version 2 focussed on the professional conduct expected of a Specialist in Laboratory Medicine, version 3 builds on the responsibilities for ethical conduct from point of planning to point of care. Particular responsibilities that are outlined include: - The need for evidence when planning a new service, providing assurance that a new test does not do harm - Maintaining respect for patient confidentiality, their religious/ethnic beliefs, the need for informed consent to test, agreement on retrospective use of samples as part of governance envelopes in the pre-analytical phase - Ensuring respect for patient autonomy in the response to untoward results generated in the analytical phase - Supporting the safety of patients in the post-analytical phase through knowledge-based interpretation and presentation of results - The duty of candour to disclose and respond to error across the total testing process - Leading initiatives to harmonise and standardise pre-analytical, analytical and post-analytical phases to ensure more consistent clinical decision making with utilisation of demand management to ensure more equitable access to scarce resources - Working with emerging healthcare providers beyond the laboratory to ensure consistent application of high standards of clinical care In identifying opportunities for wider contributions to resolving ethical challenges across healthcare the need is also highlighted for more external quality assurance schemes and ethics-based quality indicators that span the total testing process.

The harmonization issue in laboratory medicine: the commitment of CCLM

The analytical quality of the clinical laboratory results has shown a significant improvement over the past decades, thanks to the joint efforts of different stakeholders, while the comparability among the results produced by different laboratories and methods still presents some critical issues. During these years, Clinical Chemistry and Laboratory Medicine (CCLM) published several papers on the harmonization issue over all steps in the Total Testing Process, training an important number of laboratory professionals in evaluating and monitoring all the criticisms inherent to the pre-analytical, as well as analytical and post analytical phases: from the consensus statement on the most informative testing in emergency setting, to the prevention and detection of hemolysis or to patients identification and tube labeling procedures, as far as to different approaches to harmonize hormones measurements or to describe new reference methods or to harmonize the laboratory report. During these years the commitment of the journal, devoted to the harmonization processes has allowed to improve the awareness on the topic and to provide specific instruments to monitor the rate of errors and to improve patients safety.

Thoughts and expectations of young professionals about the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)

Objectives

Young laboratory medicine professionals (YLMPs) are the future of clinical laboratories. Although everyday practice shows significant differences among countries, especially during residency training, most of them face the same challenges. Besides promoting scientific, professional and clinical aspects of laboratory medicine in Europe, the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) should take into consideration YLMPs’ concerns and interests to help them achieve excellence. The aim of this survey was to assess the opinion and expectations of YLMPs about their involvement in the activities of EFLM.

Methods

An online survey was distributed to YLMPs in Europe through different channels. The questionnaire consisted of 21 items grouped into five sections: demographic questions, opinion about the current status of YLMPs within EFLM, YLMPs network, suggestions and opportunities, and scientific training and exchange. Where appropriate, responses from residents and specialists were compared.

Results

A total of 329 valid responses were obtained from 53 different countries. Countries with the highest number of participants were Spain, Turkey, Croatia and Romania. A significant percentage would like to know more about EFLM and their activities (86%) and wish EFLM promoted networking and scientific exchanges (95%), for instance by means of a European YLMPs network (93%). EFLMLabX project was widely unknown (75%).

Conclusions

YLMPs demand better connection to share concerns about daily healthcare duties, to keep updated and to advance professionally. EFLM needs to improve their advertising through national societies to increase YLMPs’ participation. In addition to international meetings and congresses, respondents have emphasized that workshops and other small group activities would significantly help promote laboratory medicine practice in Europe.

Performance evaluation of 14 specific proteins measurement checked by an External Quality Assessment Scheme

AIMS OF THE STUDY

To evaluate the state-of-the-art of 14 specific proteins measurement; to evaluate the laboratories' performance and the degree of harmonization in reporting results of participants in the External Quality Assessment Program of the Centre of Biomedical Research (CRB).

METHODS

Overall and system-related inter-laboratory analytical variability (mean CVs%) and between-system differences (mean bias%) were evaluated from data of six EQA cycles 2013-2018. Moreover, we evaluated the analytical performance of participants as well as the units used to express proteins results.

RESULTS

Overall inter-laboratory variability ranged from 3.8% for haptoglobin (HPT) to 12.5% for α₁-antitrypsin (AAT) and decreased for IgA, α₂-macroglobulin (A2M) and transferrin (TRF). Mean CVs% were generally higher for Siemens BN and Beckman Immage immunonephelometric systems, but <7.0% for all proteins. Mean bias > 7.0% was observed for BN (IgA, C4, AAT, transthyretin TTR), Siemens Vista (IgA, C4) and Immage (C4), whereas mean bias < -7.0% was found for Immage (AAT), Beckman AU (IgM) and Roche Cobas (C4, TTR, C-reactive protein). The laboratories' performance within the limits ranged from 85.1% of albumin (ALB) to 97.2% of HPT. The census of units employed in 2018, demonstrated that ~ 70% of laboratories still express the results in mg/dL.

CONCLUSIONS

Despite a reduction in inter-laboratory variability for some proteins, different analytical systems showed both proportional and constant bias between methods. Units used by participants have not been substantially changed and dL is still largely used. The CRB EQA Program, with its performance data sets, is a valuable resource for laboratories and IVD manufacturers and support the goals of harmonization.

Alanine Aminotransferase Results Differ by Analyzer Manufacturer in a National Integrated Health Setting, 2012-2017

CONTEXT.—

Disease guidelines specify universal alanine aminotransferase (ALT) thresholds for clinical decision-making, yet the effect of variability among ALT analyzers remains unclear.

OBJECTIVE.—

To compare ALT results from different analyzers from 2012-2017.

DESIGN.—

Veterans Health Administration (VHA) laboratories perform external ALT proficiency testing using standardized College of American Pathologists (CAP) samples in analyzers by 5 manufacturers. In this operational analysis, we evaluated 22 950 ALT values from 80 independent CAP samples tested at 223 laboratories. Using mixed effects modeling, we estimated the association between analyzer manufacturer and CAP outcome, adjusting for manufacturer, facility, and calendar year. We performed subgroup analyses on CAP samples with overall means near clinical guideline-specified thresholds, including less than 50 U/L (n = 10) and less than 35 U/L (n = 5).

RESULTS.—

The VHA used Abbott Laboratories (n = 3175; 14%), Beckman Coulter Diagnostics (n = 8723; 38%), Roche Diagnostics (n = 2595; 11%), Siemens Healthineers USA (n = 5713; 25%), and Vitros/Ortho Clinical Diagnostics (n = 2744; 12%) analyzers. The CAP samples (n = 80 samples, n = 22 950 tests) covered a wide range of mean ALT values (21-268 U/L). The average difference in mean ALT value per sample between the highest-reading and lowest-reading manufacturers was 15.4 U/L (SD = 1.8) for the 10 samples with mean ALT less than 50 U/L, and it was 10.4 U/L (SD = 3.6) overall (n = 80). In linear mixed effects modeling, we found statistically significant differences in ALT values between the different manufacturers in each year.

CONCLUSIONS.—

We found statistically and clinically meaningful differences between analyzers across the ALT spectrum in each year, including at ALT levels lower than 50 U/L and lower than 35 U/L. Universal ALT thresholds should be avoided as a trigger for clinical action until differences between analyzers can be resolved.