NPS-EQA PART II: Four years' experience in external quality assessment program in Italy for classical and new psychoactive substances analysis in hair

In 2019, the Italian National Institute of Health established an external quality assessment (EQA) program to evaluate the performance of laboratories of collaborative centres participating in the National Early Warning System in hair testing for classical and new psychoactive substances (NPS). The results obtained in the four rounds (2019-2023) and the evolution in hair testing performance for classic drugs of abuse and new psychoactive substances are presented. A total of 11 hair specimens, including 3 blank samples, were prepared by adding different classes of classical and NPS at known concentrations to pre-screened drug-free hair. False negative and false positive results were calculated for the qualitative data evaluation. The quantitative evaluation included the imprecision (as % coefficient of variation, CV%) and the accuracy (as % error, ERR%) of the results with respect to a mean value obtained by reference laboratories and Z-score values were assessed. Over the years, an improvement in false negative results (from 52.4% in the first year to 34.3% in the last one) and false positive results (from 55.0% in the first year to 30.8.% in the last one) was observed. In the first round, the mean ERR% ranged from 6.2% to 112.8% due to NPS determination. However, in the subsequent three rounds, the mean ERR% ranged from 10.4% to 22.4%, The mean CV% in the four rounds was approximately 41.5% (ranging from 44.3% to 53.3%). Between 12.0% and 56.6% of the reported results in all rounds should be considered satisfactory. EQA programs help laboratories to identify and correct problems within their processes by highlighting errors and variations. This ensures that the results produced are accurate and reproducible.

External quality assessment for molecular detection of Ureaplasma urealyticum in China

BACKGROUND AND AIMS

A pilot external quality assessment (EQA) scheme for molecular detection of Ureaplasma urealyticum (UU) was conducted by the National Center for Clinical Laboratories (NCCL) to evaluate the testing capabilities of clinical laboratories and the actual performance of DNA-based nucleic acid amplification tests (NAAT) and RNA-based NAATs when applied in clinical settings.

MATERIALS AND METHODS

The EQA panel contained twelve lyophilized samples, including positive samples containing inactivated cell culture supernatants of UU at different concentrations and sterile saline for negative samples. The positive samples were further divided into three groups of high, moderate and low concentrations. The panels were distributed to the participants and the datasets were analyzed according to the qualitative results.

RESULTS

A total of 365 laboratories participated in the EQA scheme, and 360 results submitted by 338 laboratories were collected, of which 96.11 % (346/360) of the returned results and 95.86 % (324/338) of the laboratories were deemed competent. The positive percentage agreement (PPA) was ≥ 97.5 % for high and moderate concentration samples, but varied significantly for low concentration samples, decreasing from 86.94 % to 51.94 % as the sample concentration decreased. Additionally, for low concentration samples, RNA-based NAAT showed higher PPAs than DNA-based NAATs, but these results were specific to UU supernatants used in this study.

CONCLUSION

Most of UU detection assays employed by the participants were generally consistent with their estimated limit of detection (LOD), and the majority of participants can reliably detect UU samples with high and moderate concentrations, while the poor analytical performance for low concentration samples requires further improvement and optimization.

Evaluation of imprecision in the different detection methods of Zn based on 5 years of data from an external quality assessment program in China

BACKGROUND

This study examines the imprecision of zinc (Zn) measurements across various clinical detection methods by analyzing the external quality assessment (EQA) data from 2018 to 2022. The findings of this study aim to offer recommendations for enhancing Zn measurements.

METHODS

Participating laboratories were grouped into peer categories based on the detection methods. The robust mean and coefficient of variation (CV) of the samples were calculated following ISO 13528 guidelines. The evaluation criteria for optimal, desirable, and minimum allowable imprecision in Zn estimation are 2.50%, 5.05%, and 7.55%, respectively, based on biological variation. Furthermore, the study examined inter-lab CVs, inter-method bias, and the passing rate. The impact of sample concentration on CVs and the pass rate was also investigated.

RESULTS

Over the past five years, 4283 laboratories participated in the EQA program, showing a high pass rate that improved as sample concentration increased. Differential pulse polarography (DPP) demonstrated stable and low CVs (0.61-1.86%). Although differential pulse stripping (DPS) was less stable than DPP, it still exhibited a low CV (0.71-3.10%). Graphite furnace atomic absorption spectrometry (GFAAS) and flame atomic absorption spectrometry (FAAS) performed similarly and displayed stable CVs (2.39-4.42%) within the acceptable range of desirable imprecision (5.05%). However, the CVs for ICP-MS were unacceptable in three out of the five years (5.28-6.20%). In 2022, the number of participating laboratories for DDP, DPS, GFAAS, FAAS and ICP-MS is 131, 35, 35, 820 and 72, respectively.

CONCLUSION

This study provides reliable insights into the imprecision of Zn measurements in clinical laboratories. The findings indicate that additional efforts are required to reduce the imprecision of ICP-MS in Zn measurements.

A small-scale external quality assessment for PCR detection of group B streptococcus in China

BACKGROUND

Group B streptococcus (GBS) is considered a leading cause of maternal and infant morbidity and mortality. Molecular diagnosis is a routinely used approach for GBS screening to protect pregnant women and prevent early-onset GBS neonatal disease. The objective of this study was to identify issues and guarantee the dependability of GBS molecular diagnosis by an external quality assessment (EQA) scheme.

METHODS

The EQA panel comprised eight samples spiked with 10-fold dilutions of GBS suspension (20-2,000,000 copies/mL), and 2 negative control samples. The panels were coded randomly and distributed to participating laboratories for GBS detection.

RESULTS

In total, 44 participating laboratories submitted results with eight commercial GBS PCR assays and one in-house assay. Among them, 36 obtained an acceptable or higher performance score, while 8 required improvement. Among the 440 results returned, 62 (14.1 %) were incorrect, including 5 false positives and 57 false negatives.

CONCLUSIONS

Our small-scale EQA showed that most participating laboratories have reliable diagnostic capacities for GBS PCR detection. Nonetheless, further improvements in the detection performance of some laboratories are required, particularly with low-concentration samples. Our survey also reinforces the use of EQA as an essential tool to evaluate the overall proficiency of clinical laboratories.

On-site evaluation as external quality assessment of microbiology laboratories involved in sentinel laboratory-based antimicrobial resistance surveillance network in Burkina Faso

BACKGROUND

The laboratory-based surveillance of antimicrobial resistance (AMR) is a priority component of the multisectoral national action plan to combat AMR in Burkina Faso. This study aimed to assess the QMS of microbiology laboratories involved in the Sentinel laboratory-based antimicrobial resistance surveillance network in Burkina Faso.

METHODS

A cross-sectional study was conducted from September 1st to November 30th, 2022. The external quality assessment (EQA) method used was on-site evaluation using a checklist that was developed and validated by a technical committee of experts. Teams of two, including an antimicrobial susceptibility test (AST) specialist and a QMS specialist, were trained on this checklist to conduct the assessment. Satisfactory performance was defined as any on-site evaluation score 80% and above with the aim of developing action plans to address gaps.

RESULTS

All 18 laboratories were evaluated. The overall average performance score of the participating laboratories was 40%. The highest overall performance score was 58%, and the lowest overall performance score was 26%. The average overall scores were not significantly different between private and public laboratories (p value = 0.78). The only section of the checklist with the satisfactory performance concerned the "Analytical step of AST", with 76.5% (13/17) of the sentinel laboratories having a score ≥ 80%.

CONCLUSION

The performance of the QMS of the sentinel laboratories in Burkina Faso for AMR surveillance was unsatisfactory, and a corrective action plan was proposed to support these laboratories in improving their QMS over the next 3 years.

External quality assessment for detection of colorectal cancer by Septin9 DNA methylation in clinical laboratories

BACKGROUND AND AIMS

The incidence and mortality rate of colorectal cancer (CRC) are increasing worldwide. Septin9 methylated (mSEPT9) DNA in circulation can be used as a non-invasive detection method to assist in the early diagnosis of CRC; however, the detection methods and procedures are complicated. This study aimed to evaluate the ability of clinical laboratories to detect Septin9 methylation in plasma cell-free DNA (cfDNA).

MATERIALS AND METHODS

We prepared a sample panel consisting of positive and negative Septin9 methylation cells and CRC cells. Three positive samples with different methylation levels, one negative sample and one duplicate sample, two samples containing interference, three different CRC cell samples, and a fictitious case report were included. The panel was distributed to 59 laboratories for mSEPT9 analysis, result comparison, and scoring.

RESULTS

The sample panel, validated by National Medical Products Administration (NMPA)-approved tests and targeted bisulfite sequencing, met expectations and could be used for external quality assessment (EQA). Among the 59 laboratories, 55 (93.22%) correctly reported the mSEPT9 results for all samples, while four (6.79%) reported 15 false negatives and were considered improvable. All false negatives originated from four laboratories using laboratory-developed tests (LDTs), with three failing to detect weakly positive samples, samples containing interference, and samples from different CRC cells, and one reported erroneous results on all positive samples.

CONCLUSION

Our results illustrated that the detection of mSEPT9 in cfDNA is satisfactory in China. EQA is indispensable because it can help improve the diagnostic capability and quality management of the laboratories, and provide suggestions for the problems existing in mSEPT9 detection.

Model of implementing proficiency testing in Vietnam, a developing country

Backgrounds and aims

The aim of this study is to provide a good approach for a quantitative EQA scheme assigned value with limited resources.

Materials and methods

Twelve lyophilized EQA items were distributed to participants in 2021 from North to Southeast Vietnam to measure the concentration of nine parameters, including glucose, urea, creatinine, cholesterol, triglyceride, uric acid, AST, ALT, and GGT. The consensus value of the expert group and all participants were calculated and statistically compared to choose the most appropriate consensus value.

Results

Fifty-nine laboratories attended the EQA scheme, including an expert group using automatic biochemistry analyzers (AAs) and all participants with auto and semi-auto biochemistry (SAA) analyzers. Consensus values of six per nine parameters were different between the two groups for at least two EQA items, including glucose, creatinine, cholesterol, uric acid, AST, and ALT. The coefficients of variation of glucose, urea, creatinine, triglycerides, uric acid, and GGT in the expert group were significantly lower than those in all the participants.

Conclusion

Using the consensus values of expert groups as the assigned values of the EQA program is a relevant strategy to increase testing quality in developing countries with limited resources, such as Vietnam.

A New Strategy for Evaluating the Quality of Laboratory Results for Big Data Research: Using External Quality Assessment Survey Data (2010-2020)

Background

To ensure valid results of big data research in the medical field, the input laboratory results need to be of high quality. We aimed to establish a strategy for evaluating the quality of laboratory results suitable for big data research.

Methods

We used Korean Association of External Quality Assessment Service (KEQAS) data to retrospectively review multicenter data. Seven measurands were analyzed using commutable materials: HbA1c, creatinine (Cr), total cholesterol (TC), triglyceride (TG), alpha-fetoprotein (AFP), prostate-specific antigen (PSA), and cardiac troponin I (cTnI). These were classified into three groups based on their standardization or harmonization status. HbA1c, Cr, TC, TG, and AFP were analyzed with respect to peer group values. PSA and cTnI were analyzed in separate peer groups according to the calibrator type and manufacturer, respectively. The acceptance rate and absolute percentage bias at the medical decision level were calculated based on biological variation criteria.

Results

The acceptance rate (22.5%-100%) varied greatly among the test items, and the mean percentage biases were 0.6%-5.6%, 1.0%-9.6%, and 1.6%-11.3% for all items that satisfied optimum, desirable, and minimum criteria, respectively.

Conclusions

The acceptance rate of participants and their external quality assessment (EQA) results exhibited statistically significant differences according to the quality grade for each criterion. Even when they passed the EQA standards, the test results did not guarantee the quality requirements for big data. We suggest that the KEQAS classification can serve as a guide for building big data.

Classification of "Near-patient" and "Point-of-Care" SARS-CoV-2 Nucleic Acid Amplification Test Systems and a first approach to evaluate their analytical independence of operator activities

BACKGROUND

European legislation defines as "near-patient testing" (NPT) what is popularly and in other legislations specified as "point-of-care testing" (POCT). Systems intended for NPT/POCT use must be characterized by independence from operator activities during the analytic procedure. However, tools for evaluating this are lacking. We hypothesized that the variability of measurement results obtained from identical samples with a larger number of identical devices by different operators, expressed as the method-specific reproducibility of measurement results reported in External Quality Assessment (EQA) schemes, is an indicator for this characteristic.

MATERIALS AND METHODS

Legal frameworks in the EU, the USA and Australia were evaluated about their requirements for NPT/POCT. EQA reproducibility of seven SARS-CoV-2-NAAT systems, all but one designated as "POCT", was calculated from variabilities in Ct values obtained from the respective device types in three different EQA schemes for virus genome detection.

RESULTS

A matrix for characterizing test systems based on their technical complexity and the required operator competence was derived from requirements of the European In Vitro Diagnostic Regulation (IVDR) 2017/746. Good EQA reproducibility of the measurement results of the test systems investigated implies that different users in different locations have no recognizable influence on their measurement results.

CONCLUSION

The fundamental suitability of test systems for NPT/POCT use according to IVDR can be easily verified using the evaluation matrix presented. EQA reproducibility is a specific characteristic indicating independence from operator activities of NPT/POCT assays. EQA reproducibility of other systems than those investigated here remains to be determined.

External quality assessment (EQA) for tumor mutational burden: results of an international IQN path feasibility pilot scheme

Tumor mutational burden (TMB) has recently been approved as an agnostic biomarker for immune checkpoint inhibitors. However, methods for TMB testing have not yet been standardized. The International Quality Network for Pathology (IQNPath) organized a pilot external quality assessment (EQA) scheme for TMB testing. The aim of this program was the validation of the materials and the procedures for the EQA of this complex biomarker. Five formalin-fixed paraffin-embedded (FFPE) cell lines were selected to mimic the various TMB values observed in clinical practice. The FFPE samples were tested with the FoundationOne CDx (F1CDx) assay as the reference test and three commercially available targeted sequencing panels. Following this internal validation, the five cell lines were sent to 29 laboratories selected on the basis of a previous survey. Nineteen of the 23 laboratories that submitted results (82.6%) used targeted sequencing for TMB estimation. Only two laboratories performed whole exome sequencing (WES) and two assessed TMB by clinical exome. A high variability in the reported TMB values was observed. The variability was higher for samples with the highest TMB value according to the F1CDx test. However, good reproducibility of the TMB score was shown by laboratories using the same panel. The majority of laboratories did not indicate a TMB cut-off value for clinical interpretation. In conclusion, this pilot EQA scheme suggests that it is feasible to run such an EQA program for TMB assessment. However, the results of our pilot highlight the numerous challenges for the standardization of this test.

Access and quality of biomarker testing for precision oncology in Europe

BACKGROUND

Predictive biomarkers are essential for selecting the best therapeutic strategy in patients with cancer. The International Quality Network for Pathology, the European Cancer Patient Coalition and the European Federation of Pharmaceuticals Industries and Associations evaluated the access to and quality of biomarker testing across Europe.

METHODS

Data sources included surveys of 141 laboratory managers and 1.665 patients, and 58 in-depth interviews with laboratory managers, physicians and payers. Four access metrics (laboratory access, test availability, test reimbursement, test order rate) and three quality metrics (quality scheme participation, laboratory accreditation, test turnaround time) were applied to rank the results.

RESULTS

The access to precision medicines is higher in countries with public national reimbursement processes in place. Lack of diagnostic laboratory infrastructure, inefficient organization and/or insufficient public reimbursement narrow the access to single biomarker tests in many European countries. In countries with limited public reimbursement, pharma and patients' out of pocket were the primary funding sources for testing. Uptake of multi-biomarker next generation sequencing (NGS) is highly varied, ranging from 0% to >50%. Financial constraints, a lack of NGS testing capabilities and the failure to include NGS testing in the guidelines represent the main barriers to NGS implementation. The quality of biomarker testing is highest in Western and Northern Europe, with more than 90% of laboratories participating in quality assurance schemes.

CONCLUSIONS

Our data clearly indicate the need for a call to action to ensure the clinical implementation of precision medicine in Europe.

Expert opinion on NSCLC small specimen biomarker testing - Part 2: Analysis, reporting, and quality assessment

The diagnostic work-up for non-small cell lung cancer (NSCLC) requires biomarker testing to guide therapy choices. This article is the second of a two-part series. In Part 1, we summarised evidence-based recommendations for obtaining and processing small specimen samples (i.e. pre-analytical steps) from patients with advanced NSCLC. Here, in Part 2, we summarise evidence-based recommendations relating to analytical steps of biomarker testing (and associated reporting and quality assessment) of small specimen samples in NSCLC. As the number of biomarkers for actionable (genetic) targets and approved targeted therapies continues to increase, simultaneous testing of multiple actionable oncogenic drivers using next-generation sequencing (NGS) becomes imperative, as set forth in European Society for Medical Oncology guidelines. This is particularly relevant in advanced NSCLC, where tissue specimens are typically limited and NGS may help avoid tissue exhaustion compared with sequential biomarker testing. Despite guideline recommendations, significant discrepancies in access to NGS persist across Europe, primarily due to reimbursement constraints. The use of increasingly complex testing methods also has implications for the reporting of results. Molecular testing reports should include clinical interpretation with additional commentary on sample adequacy as appropriate. Molecular tumour boards are recommended to facilitate the interpretation of complex genetic information arising from NGS, and to collaboratively determine the optimal treatment for patients with NSCLC. Finally, whichever testing modality is employed, it is essential that adequate internal and external validation and quality control measures are implemented.

Centralisation of surgery for complex cancer diseases: A scoping review of the evidence base on pancreatic cancer

BACKGROUND

Centralisation of cancer surgery is a commonly applied healthcare strategy worldwide. This study aimed to detail the design of centralisation policies, to shed light on the implications of such policies in real practice and to describe the different perspectives taken to deal with difficulties that emerged, taking pancreatic cancer as an example of a complex cancer disease requiring surgery.

METHODOLOGY

A scoping review was conducted using the MEDLINE database. We systematically searched for eligible studies published between January 2000 and December 2018.

RESULTS

In the 33 included studies, centralisation of pancreatic cancer surgery was implemented through three different models: designated hospitals, definition of minimum volumes per provider, and/or recommendations included in protocols and national guidelines. The presence of highly advanced technology and infrastructures, the availability of extensive service coverage and advanced care processes based on expert multidisciplinary teams, and higher caseloads were identified as key components of centralisation policy.

CONCLUSIONS

Centralisation models for pancreatic cancer surgery showed that having expert centres where the care process is comprehensively guided is a foundational policy approach. External quality assessment and the accreditation of centres and professionals performing complex surgical procedures are levers that may positively impact the effectiveness of the measure. POLICY SUMMARY: while we found different experiences and three models of centralisation, all of them were guided by the will to positively impact on pancreatic cancer patients' access to expert care. Clinical research might be able to make progress in the coming years and perhaps contribute to reversing a critical situation of high mortality and growing incidence. However, policymakers must optimise health system responses considering current resources, as suggested by the recommendations proposed in the framework of the EU initiative Bratislava Statement for pancreatic cancer care.

Development and implementation of National External Quality Assurance Programs in a One Health approach: The Armenian experience

INTRODUCTION

Early warning and objective evidence of systematic errors in laboratory diagnosis ensures evidence based corrective and preventive actions that instill patient safety and confidence. External quality assessment contributes significantly to the above as an essential component of laboratory quality assurance. However, implementation of External Quality Assessment in resource-limited settings is challenged by high costs of enrolling in international schemes. To ensure sustainability, a National External Quality Assessment Program in Armenia was developed using a One Health approach.

METHODS

Through engagement of stakeholders from Ministry of Health and Department of Agriculture under Ministry of Economy the government of Armenia started the implementation of the Armenia Laboratory External Quality Assessment (ALEQA) program. Policies and procedures were defined, a web interface for return of results and feedback reporting was created. A training was offered for characterization of simulated samples for bacterial pathogens. Following a pilot survey, the program was successfully scaled up, with later addition of a Brucella serology discipline.

RESULTS

The return rate of results was 100% for all surveys. There was an improvement in the performance of the laboratories from the 2015 to the 2019 surveys. The bacterial pathogens EQA survey's, was interrupted between 2017 and 2019. The Brucella Serology survey showed 77% of the 26 participating laboratories had satisfactory performance.

CONCLUSION

This is one of the few National EQA Programs that have embraced the One Health approach to improve reach of EQA Programs in resource-limited settings in both human and veterinary laboratories.

Improving the accuracy of chloride measurements through participation in regular external quality assessment programme

BACKGROUND

A chloride test is an integral part of a basic metabolic panel that is essential for the assessment of a patient's acid-base and electrolyte status. While many methods are available commercially for the routine measurement of chloride, there is a need to address the accuracy and variability among the measurement results, especially with the prevalence of patients seeking treatment across different healthcare providers for alternative opinions.

METHOD

A method based on sector field inductively coupled plasma isotope dilution mass spectrometry (SF-ICP-IDMS) was developed for the measurement of chloride in human serum. The SF-ICP-IDMS method was then used to assign the target values in the Health Sciences Authority (HSA) External Quality Assessment (EQA) Programme to evaluate the results of chloride test from participating clinical laboratories.

RESULTS

The accuracy of the measurements was evaluated by comparing the results with the certified values of Electrolytes in Frozen Human Serum Certified Reference Materials (SRM 956c and SRM 956d) from the National Institute of Standards and Technology (NIST) at different chloride concentration levels. Over a five-year period from 2014-2018, the number of clinical laboratories which participated in the EQA Programme increased from 23 to 33. Comparison of robust means from the laboratories' results with our assigned target values revealed a reduction in relative deviation over time. The relationship between the deviation of each brand of clinical analysers and the chloride levels was established, where a larger deviation was uncovered at low chloride concentration. The SF-ICP-IDMS method was further demonstrated to be comparable with methods used by other metrology institutes in an international comparison organised by HSA under the auspice of the Consultative Committee for Amount of Substance - Metrology in Chemistry and Biology (CCQM).

CONCLUSION

The use of metrologically traceable assigned target values enabled the study of method biasness from a small pool of dataset in each of the four brands of clinical analysers in HSA EQA Programme. This work underscores the need to improve the accuracy of chloride measurements by regular participation in an accuracy-based EQA Programme.

Non-invasive prenatal screening for foetal trisomy: An assessment of reliability and reporting

OBJECTIVE

Non-invasive prenatal screening (NIPS) has been introduced as a routine screening technique for aneuploidies in the clinic. To evaluate its reliability and reporting standardization, the National Center for Clinical Laboratories launched an external quality assessment (EQA) program based on highly simulated samples.

METHODS

Maternal and child paired cell lines were digested by enzymes to obtain DNA fragments for the analysis panel, which were composed of 5% T21, 5% T18, 10% T13, 10% euploid, and 20% T18 samples. The samples were validated and distributed to laboratories along with scenarios and questionnaires for analysis.

RESULTS

Out of 350 participating laboratories, 98.6% correctly identified all samples. The concurrence rates of laboratories for the 5% T21, 5% T18, 10% T13, 10% euploid, and 20% T18 samples were 98.9%, 99.7%, 99.7%, 100%, and 100%, respectively. Enrichment increased the foetal fraction (FF) values by 2 ∼ 3-fold, but the z scores generated by the enrichment group fluctuated greatly. Other FF estimation techniques, such as the size-based and FF-QuantSC methods, generated slightly different FF values from the chr Y-based method. Furthermore, some laboratories omitted the suggestions of results in reports.

CONCLUSIONS

The participating laboratories provided highly reliable results for samples with relatively higher FF values. However, the absence of performance validation, laboratory errors, and low FF values were potential reasons for false-negative results. In addition, enrichment operations should be validated and normalized to guarantee NIPS reproducibility, plus further efforts are required to standardize the NIPS reports.

Comparability of sample results and commutability of reference materials among different measurement procedures for protein C activity assays

BACKGROUND AND AIMS

Several types of measurement procedures (MPs) for protein C activity assays are currently available. Clinical sample (CS) results among different MPs should be comparable. The commutability of reference materials (RMs) is an essential requirement to achieve comparability of CS results.

MATERIALS AND METHODS

Considering the total error calculated using reliable biological variation (BV) data and external quality assessment (EQA) criteria, we chose the allowable limits of comparability and criterion of commutability. According to Clinical and Laboratory Standardization Institute EP9 and our previous studies, 92 CSs were used to evaluate the comparability among the three MPs (Sysmex CS-5100, IL ACL TOP 700, and STA-R Evolution). The difference in bias method recommended by International Federation of Clinical Chemistry and Laboratory Medicine was used to assess the commutability of six RMs, including World Health Organization (WHO) IS 02/342.

RESULTS

The compliance rates of CSs were 94.6-100% with the corresponding calibration mode. WHO IS, HemosIL calibration plasma, and candidate RMs, PC20201 and PC20202, were commutable between each pair of the three MPs.

CONCLUSION

It is feasible to set the allowable limits of comparability and the criterion of commutability based on the BV and EQA criteria.

External quality assessment of molecular testing of 9 viral encephalitis-related viruses in China

BACKGROUND

Eastern equine encephalitis virus (EEEV), Western equine encephalitis virus (WEEV), Venezuelan equine encephalitis virus (VEEV), Hendra virus (HeV), Nipah virus (NiV), Yellow fever virus (YFV), West Nile virus (WNV), Saint Louis encephalitis virus (SLEV) and Tick-borne encephalitis virus (TBEV) have been detected in travelers returning to China and potentially pose a serious threat to public health. Real-time reverse transcription polymerase chain reaction (rRT-PCR) plays an important role in the detection of these viruses. Although these viruses are not mainly prevalent in China, occasionally imported cases have been reported with the increase in population mobility and entry-exit activities. Therefore, it is necessary to monitor the ability of major domestic laboratories to detect and identify exotic arbovirus infections in travelers.

METHODS

An external quality assessment program for the molecular detection of EEEV, VEEV, WEEV, SLEV, WNV, YFV, TBEV, HeV and NiV was organized. The assessment panel included 26 negative and positive samples with different concentrations of virus-like particles and distributed to 31 laboratories to evaluate the accuracy of virus detection.

RESULTS

At the laboratory level, 87.5% (7/8, EEEV), 85.7% (12/14, WEEV), 100% (13/13, VEEV), 87.5% (7/8, HeV), 76.5% (13/17, NiV), 92.6% (25/27, YFV), 81.3% (13/16, WNV), 100% (5/5, SLEV) and 75.0% (6/8, TBEV) of the participants were considered "competent". Of all the results, the false-positive and false-negative rates were 0.3% and 0.7%, respectively. The sensitivity of most detection assays (15/17, 88.2%) was more than 90%. In addition, we observed significantly different cycle threshold values when using primer-probe sets in different target regions to detect EEEV and SLEV.

CONCLUSIONS

Most laboratories have reliable virus detection capabilities. However, laboratory testing capabilities need to be improved to avoid cross-contamination and to better manage undetected false-negative samples.

Development of a robust blood smear preparation procedure for external quality assessment

Introduction

The external quality assessment (EQA) scheme is particularly important for laboratory performance evaluation. Peripheral blood smears are necessary to identify morphological features, and the procedure for preparing such smears must be robust to meet the ISO 15189 standard. Although blood smear preparation is a routine activity in medical laboratories, an appropriate procedure for preparing a series of blood smears with high homogeneity and durable stability for EQA purposes has not yet been published elsewhere. For this reason, a robust procedure was developed and validated in this study.

Methods

Various factors affecting blood smear preparation, such as the amount of time collected blood samples are stored before fixation, suitable reagents, and specification parameters for each step, including fixation, staining, and timing of the staining steps, were studied. Each experiment was evaluated based on homogeneity and stability characteristics.

Results

Whole blood mixed with EDTA anticoagulant was used to make the blood smears. Samples were fixed with pure anhydrous alcohol and stained in Coplin jars using the Wright-Giemsa method.

Conclusion

The homogeneity and stability of two lots of smears suitable for EQA purposes was confirmed based on intact morphology of the smears for more than 8 months at room temperature.

Standardization Status of Total Cholesterol Concentration Measurement: Analysis of Korean External Quality Assessment Data

Background

Total cholesterol concentration measurement is important in the diagnosis of dyslipidemia and evaluation of cardiovascular disease risk factors. Measurement reliability for obtaining an accurate total cholesterol concentration requires procedure standardization. We evaluated the standardization status for total cholesterol concentration measurement through Korean external quality assessment (EQA) data analysis.

Methods

This study involved 1,670 laboratories that participated in the EQA of total cholesterol concentration measurements in 2019 for 32 products from different manufacturers. The target concentrations of three quality control (QC) materials (samples A, B, and C) were measured using the reference method and compared with EQA data. The performance criteria for total cholesterol concentration measurement were based on the National Cholesterol Education Program guidelines, with ±3% inaccuracy.

Results

The target values and inaccuracies of the QC material based on the reference method measurements were 254.65±7.64, 108.30±3.25, and 256.29±7.69 mg/dL (6.59±0.20, 2.80±0.08, and 6.63±0.20 mmol/L) for samples A, B, and C, respectively. The performance criteria were not met in 42.7% laboratories for sample A, 68.4% of laboratories for sample B, and 38.0% laboratories for sample C.

Conclusions

Despite significant efforts to accurately measure total cholesterol concentrations, further actions are needed for measurement standardization. Manufacturers reporting values that differ from target values should check calibrator traceability; additional efforts to accurately measure total cholesterol concentrations are required for laboratories that use products from these manufacturers.

A second external quality assessment of isolation and identification of influenza viruses in cell culture in the Asia Pacific region highlights improved performance by participating laboratories

Influenza viruses must be amplified in cell culture for detailed antigenic analysis and for phenotypic assays assessing susceptibility to antiviral drugs or for other assays. Following on from the first external quality assessment (EQA) for isolation and identification of influenza viruses using cell culture techniques in 2016, a follow up EQA was performed in 2019 for National Influenza Centres (NICs) in the World Health Organization (WHO) South East Asia and Western Pacific Regions. Nineteen WHO NICs performed influenza virus isolation and identification techniques on an EQA panel comprising 16 samples, containing influenza A or B viruses and negative control samples. One sample was used exclusively to assess capacity to measure a hemagglutination titer and the other 15 samples were used for virus isolation and subsequent identification. Virus isolation from EQA samples was generally detected by assessment of cytopathic effect and/or hemagglutination assay while virus identification was determined by real time RT-PCR, hemagglutination inhibition and/or immunofluorescence assays. For virus isolation from EQA samples, 6/19 participating laboratories obtained 15/15 correct results in the first EQA (2016) compared to 11/19 in the follow up (2019). For virus identification in isolates derived from EQA samples, 6/19 laboratories obtained 15/15 correct results in 2016 compared to 13/19 in 2019. Overall, NIC laboratories in the Asia Pacific Region showed a significant improvement between 2016 and 2019 in terms of the correct results reported for isolation from EQA samples and identification of virus in isolates derived from EQA samples (p=0.01 and p=0.02, respectively).

Status quo of ALK testing in lung cancer: results of an EQA scheme based on in-situ hybridization, immunohistochemistry, and RNA/DNA sequencing

With this external quality assessment (EQA) scheme, we aim to investigate the diagnostic performance of the currently available methods for the detection of ALK alterations in non-small cell lung cancer on a national scale, namely, in situ hybridization (ISH), immunohistochemistry (IHC), and RNA/DNA sequencing (NGS). The EQA scheme cohort consisted of ten specimens, including four ALK positive and six ALK negative samples, which were thoroughly pretested using IHC, ISH, and RNA/DNA NGS. Unstained tumor sections were provided to the 57 participants, and the results were retrieved via an online questionnaire. ISH was used by 29, IHC by 38, and RNA/DNA sequencing by 19 participants. Twenty-eight institutions (97%) passed the ring trial using ISH, 33 (87%) by using IHC, and 18 (95%) by using NGS. The highest sensitivity and interrater agreement (Fleiss ‘ kappa) was observed for RNA/DNA sequencing (99%, 0.975), followed by ISH (94%, 0.898) and IHC (92%, 0.888). However, the proportion of samples that were not evaluable due to bad tissue quality was also higher for RNA/DNA sequencing (4%) compared with ISH (0.7%) and IHC (0.5%). While all three methods produced reliable results between the different institutions, the highest sensitivity and concordance were observed for RNA/DNA sequencing. These findings encourage the broad implementation of this method in routine diagnostic, although the application might be limited by technical capacity, economical restrictions, and tissue quality of formalin-fixed samples.

Performance of Mali's biosafety level 3 laboratory in the external quality assessment in preparedness of laboratory accreditation and support to clinical trials

Background:

The external quality assessment (EQA) or external quality control is an evaluation conducted by a certified external organization to inquire about the quality of the results provided by a laboratory. The primary role of EQA is to verify the accuracy of laboratory results. This is essential in research because research data should be published in international peer-reviewed journals, and laboratory results must be repeatable. In 2007, the University Clinical Research Center (UCRC’s) biosafety level 3 (BSL-3) laboratory joined the EQA program with the College of American Pathologists in acid-fast staining and culture and identification of mycobacteria as per laboratory accreditation preparedness. Thus, after 11 years of participation, the goal of our study was to evaluate the performance of our laboratory during the different interlaboratory surveys.

Methods:

We conducted a descriptive retrospective study to evaluate the results of UCRC mycobacteriology laboratory from surveys conducted during 2007 and 2017.

Results:

Of the 22 evaluations, the laboratory had satisfactory (100% of concordance results) in 18 (81.8%) and good (80% of concordance results) in 4 (18.2%). Overall, the laboratory was above the commended/accepted limits of 75%.

Conclusion:

So far, UCRC’s BSL-3 performed well during the first 11 years of survey participation, and efforts should be deployed to maintain this high quality in the preparedness for laboratory accreditation and support to clinical trials.

The performance of human cytomegalovirus digital PCR reference measurement procedure in seven external quality assessment schemes over four years

A candidate digital PCR (dPCR)-based reference measurement procedure for quantification of human cytomegalovirus (hCMV) was evaluated in 10 viral load comparison schemes (seven external quality assessment (EQA) and three additional training schemes) organized by INSTAND e.V. over four years (between September 2014 and March 2018). Four metrology institutes participated in these schemes using the same extraction method and dPCR measurement procedure for the hCMV specific target sequence of UL54 gene. The calibration independent reference measurement procedure results from the metrology institutes were compared to the results of the clinical diagnostic laboratories applying hCMV qPCR measurement procedures calibrated to reference materials. While the criteria for the acceptable deviation from the target value interval for INSTAND's EQA schemes is from -0.8 log₁₀ to +0.8 log₁₀, the majority of dPCR results were between -0.2 log₁₀ to +0.2 log₁₀. Only 4 out of 45 results exceeded this interval with the maximum deviation of -0.542 log₁₀. In the training schemes containing samples with lower hCMV concentrations, more than half of the results deviated less than ±0.2 log₁₀ from the target value, while more than 95% deviated less than ±0.4 log₁₀ from the target value. Evaluation of intra- and inter-laboratory variation of dPCR results confirmed high reproducibility and trueness of the method. This work demonstrates that dPCR has the potential to act as a calibration independent reference measurement procedure for the value assignment of hCMV calibration and reference materials to support qPCR calibration as well as ultimately for routine hCMV load testing.

An assessment of the reproducibility of reverse transcription digital PCR quantification of HIV-1

Viral load monitoring in human immunodeficiency virus type 1 (HIV-1) infection is often performed using reverse transcription quantitative PCR (RT-qPCR) to observe response to treatment and identify the development of resistance. Traceability is achieved using a calibration hierarchy traceable to the International Unit (IU). IU values are determined using consensus agreement derived from estimations by different laboratories. Such a consensus approach is necessary due to the fact that there are currently no reference measurement procedures available that can independently assign a reference value to viral reference materials for molecular in vitro diagnostic tests. Digital PCR (dPCR) is a technique that has the potential to be used for this purpose. In this paper, we investigate the ability of reverse transcriptase dPCR (RT-dPCR) to quantify HIV-1 genomic RNA without calibration. Criteria investigated included the performance of HIV-1 RNA extraction steps, choice of reverse transcription approach and selection of target gene with assays performed in both single and duplex format. We developed a protocol which was subsequently applied by two independent laboratories as part of an external quality assurance (EQA) scheme for HIV-1 genome detection. Our findings suggest that RT-dPCR could be used as reference measurement procedure to aid the value assignment of HIV-1 reference materials to support routine calibration of HIV-1 viral load testing by RT-qPCR.

Evaluation of a worldwide EQA scheme for complex clonality analysis of clinical lymphoproliferative cases demonstrates a learning effect

Clonality analysis of immunoglobulin (IG) or T-cell receptor (TR) gene rearrangements is routine practice to assist diagnosis of lymphoid malignancies. Participation in external quality assessment (EQA) aids laboratories in identifying systematic shortcomings. The aim of this study was to evaluate laboratories' improvement in IG/TR analysis and interpretation during five EQA rounds between 2014 and 2018. Each year, participants received a total of five cases for IG and five cases for TR testing. Paper-based cases were included for analysis of the final molecular conclusion that should be interpreted based on the integration of the individual PCR results. Wet cases were distributed for analysis of their routine protocol as well as evaluation of the final molecular conclusion. In total, 94.9% (506/533) of wet tests and 97.9% (829/847) of paper tests were correctly analyzed for IG, and 96.8% (507/524) wet tests and 93.2% (765/821) paper tests were correctly analyzed for TR. Analysis scores significantly improved when laboratories participated to more EQA rounds (p=0.001). Overall performance was significantly lower (p=0.008) for non-EuroClonality laboratories (95% for IG and 93% for TR) compared to EuroClonality laboratories (99% for IG and 97% for TR). The difference was not related to the EQA scheme year, anatomic origin of the sample, or final clinical diagnosis. This evaluation showed that repeated EQA participation helps to reduce performance differences between laboratories (EuroClonality versus non-EuroClonality) and between sample types (paper versus wet). The difficulties in interpreting oligoclonal cases highlighted the need for continued education by meetings and EQA schemes.

Preparation, stability and commutability of candidate reference materials for lactate dehydrogenase (LDH)

BACKGROUND

Lactate dehydrogenase (LDH) is a key enzyme that functions as a marker of cell damage. Its activity can be measured by a variety of laboratory methods. To eliminate inter-method bias and enhance equivalence among different measurement procedures, LDH detection needs to be standardized.

METHODS

Optimized sequences coding for lactate dehydrogenase subunit A (LDH-A) and subunit B (LDH-B) were synthesized and cloned into the pRSFDuet vector, and then the constructed 6His-LDHA-pRSFDuet, 6His-LDHB-pRSFDuet, and 6His-LDHA-LDHB-pRSFDuet plasmids were transformed into Escherichia coli BL21 (DE3) for expression. The enzyme activity and specific activity of recombinant LDHs were detected. Electrophoresis of LDH isoenzymes was performed. The stability of recombinant LDHs and serum LDH was evaluated. Commutability of recombinant LDH-B was studied by the IFCC reference method and six routine methods.

RESULTS

Three plasmids were constructed and three highly concentrated recombinant LDH isoenzymes were obtained. The specific activities of LDH-A, LDH-AB, and LDH-B were 18.08 U/mg, 21.74 U/mg, and 14.18 U/mg, respectively. There was a desirable linear correlation between the activities of recombinant LDH isoenzymes and their protein concentrations. Electrophoresis of LDH isoenzymes showed that the recombinant LDH-B corresponded to LDH1 and it demonstrated good stability at 4 °C and 25 °C for 5 weeks. LDH-B formulations in saline-bovine serum albumin solution and human serum matrix were commutable for six routine methods.

CONCLUSION

Human recombinant LDH-B has great potential to become an improved and less expensive standard or reference material in external quality assessment for clinical LDH measurement.

Measurement harmonization and traceability for trace element analyses across the Children's Health Exposure Analysis Resource laboratory network

Harmonization and traceability are related metrological principles that are indispensable to assuring measurement comparability across different biomonitoring studies. The Children's Health Exposure Analysis Resource (CHEAR) was established in 2015 with six laboratories providing environmental exposure measurements on biospecimens. To ensure harmonization across studies for trace elements, CHEAR used a multi-faceted approach that included: 1) an initial interlaboratory validation exercise based on the analysis of certified blood and urine reference materials; 2) frequent participation in an established interlaboratory proficiency program for trace elements; and 3) analysis of a common pool of well-characterized biological reference materials with each analytical batch. Method accuracy and precision were established for each laboratory via analysis of NIST SRM 955c Toxic Elements in Caprine Blood, SRM 2668 Toxic Elements in Frozen Human Urine and SRM 3668 Mercury, Perchlorate, and Iodide in Frozen Human Urine. The differences among the six laboratories for As, Cd, Hg, and Mn in urine and Cd, Hg, and Pb in whole blood were judged to be fit-for-purpose. Interlaboratory performance over a 5-year period demonstrated an improvement in performance, such that for 2018-2019, >99% of challenges for urine As, Cd, Hg, and Mn, and 95% for whole blood Cd, Hg, Pb, and Mn, were found to be satisfactory. The CHEAR common reference materials were analyzed by at least 5 laboratories for 22 elements in urine and 13-14 elements in whole blood, thus providing a rich source of data to assess intra- and inter-run performance. The suite of trace elements with assigned values in both blood and urine matrices are more comprehensive than similar reference materials from other sources, and is reflective of the concentrations necessary to support biomonitoring studies. While some areas for future improvement were identified, significant progress was made to improve harmonization of trace element measurements in biological matrices among the CHEAR network labs.

PD-L1 immunohistochemistry in non-small-cell lung cancer: unraveling differences in staining concordance and interpretation

Programmed death ligand 1 (PD-L1) immunohistochemistry (IHC) is accepted as a predictive biomarker for the selection of immune checkpoint inhibitors. We evaluated the staining quality and estimation of the tumor proportion score (TPS) in non-small-cell lung cancer during two external quality assessment (EQA) schemes by the European Society of Pathology. Participants received two tissue micro-arrays with three (2017) and four (2018) cases for PD-L1 IHC and a positive tonsil control, for staining by their routine protocol. After the participants returned stained slides to the EQA coordination center, three pathologists assessed each slide and awarded an expert staining score from 1 to 5 points based on the staining concordance. Expert scores significantly (p < 0.01) improved between EQA schemes from 3.8 (n = 67) to 4.3 (n = 74) on 5 points. Participants used 32 different protocols: the majority applied the 22C3 (56.7%) (Dako), SP263 (19.1%) (Ventana), and E1L3N (Cell Signaling) (7.1%) clones. Staining artifacts consisted mainly of very weak or weak antigen demonstration (63.0%) or excessive background staining (19.8%). Participants using CE-IVD kits reached a higher score compared with those using laboratory-developed tests (LDTs) (p < 0.05), mainly attributed to a better concordance of SP263. The TPS was under- and over-estimated in 20/423 (4.7%) and 24/423 (5.7%) cases, respectively, correlating to a lower expert score. Additional research is needed on the concordance of less common protocols, and on reasons for lower LDT concordance. Laboratories should carefully validate all test methods and regularly verify their performance. EQA participation should focus on both staining concordance and interpretation of PD-L1 IHC.

Causes behind error rates for predictive biomarker testing: the utility of sending post-EQA surveys

External quality assessment (EQA) schemes assess the performance of predictive biomarker testing in lung and colorectal cancer and have previously demonstrated variable error rates. No information is currently available on the underlying causes of incorrect EQA results in the laboratories. Participants in EQA schemes by the European Society of Pathology between 2014 and 2018 for lung and colorectal cancer were contacted to complete a survey if they had at least one analysis error or test failure in the provided cases. Of the 791 surveys that were sent, 325 were completed including data from 185 unique laboratories on 514 incorrectly analyzed or failed cases. For the digital cases and immunohistochemistry, the majority of errors were interpretation-related. For fluorescence in situ hybridization, problems with the EQA materials were reported frequently. For variant analysis, the causes were mainly methodological for lung cancer but variable for colorectal cancer. Post-analytical (clerical and interpretation) errors were more likely detected after release of the EQA results compared to pre-analytical and analytical issues. Accredited laboratories encountered fewer reagent problems and more often responded to the survey. A recent change in test methodology resulted in method-related problems. Testing more samples annually introduced personnel errors and lead to a lower performance in future schemes. Participation to quality improvement projects is important to reduce deviating test results in laboratories, as the different error causes differently affect the test performance. EQA providers could benefit from requesting root cause analyses behind errors to offer even more tailored feedback, subschemes, and cases.

Commutability of possible external quality assessment materials for progesterone measurement

BACKGROUND

The commutability of control materials used for external quality assessment (EQA) programs is of great importance. Evaluating the commutability of control materials is crucial to assess their suitability for EQA programs.

METHODS

Forty-eight individual patient serum samples, commercial EQA samples, human serum pools (HSPs), commercially available sterile filtered charcoal stripped serum (CS) and swine serum were analyzed using the isotope dilution liquid chromatography-tandem mass spectrometry (ID LC-MS/MS) comparative method and six immunoassays for progesterone. The commutability was assessed according to the EP14-A2 guideline and the difference in bias approach, respectively.

RESULTS

According to the EP14-A2 guideline, HSPs and CS were commutable for all the tested immunoassays, while swine serum showed positive matrix effects in some assays. Based on the difference in bias approach, a large number of inconclusive and noncommutable results appeared.

CONCLUSIONS

The commutability of the processed materials varied depending on which evaluation approach and criterion was applied. Noncommutability of the EQA materials was observed. And HSPs and CS were possible commutable candidate control materials according to the EP14-A2 guideline.

Biomarker testing in oncology - Requirements for organizing external quality assessment programs to improve the performance of laboratory testing: revision of an expert opinion paper on behalf of IQNPath ABSL

In personalized medicine, predictive biomarker testing is the basis for an appropriate choice of therapy for patients with cancer. An important tool for laboratories to ensure accurate results is participation in external quality assurance (EQA) programs. Several providers offer predictive EQA programs for different cancer types, test methods, and sample types. In 2013, a guideline was published on the requirements for organizing high-quality EQA programs in molecular pathology. Now, after six years, steps were taken to further harmonize these EQA programs as an initiative by IQNPath ABSL, an umbrella organization founded by various EQA providers. This revision is based on current knowledge, adds recommendations for programs developed for predictive biomarkers by in situ methodologies (immunohistochemistry and in situ hybridization), and emphasized transparency and an evidence-based approach. In addition, this updated version also has the aim to give an overview of current practices from various EQA providers.

Quality in practice: applying the patient inventory method at a Danish psychiatric hospital

QUALITY PROBLEM

Patient care pathways should be organized according to the needs of the patients. This requires methods to assess whether the specific pathways ensure the right care for the right person at the right time and in the right setting.

INITIAL ASSESSMENT

Previous investigations indicate that ~25% of the patients in Danish hospitals experience inappropriate elements in their care pathways.

CHOICE OF SOLUTION

This study applied the Patient Inventory method to identify inappropriate elements in care pathways in 15 psychiatric in-patient wards in Denmark.

IMPLEMENTATION

The pathway for 201 patients was systematically evaluated by the clinical staff to identify whether the admission of the patient was avoidable, the hospitalization was unnecessarily prolonged or if the patient could receive more relevant treatment elsewhere. A subsequent meeting between the clinical staff and management qualified the assessment and identified possible solutions to problems.

EVALUATION

A total of 54 (26.9%) of the included patients were assessed to have inappropriate elements in their care pathways, some with more than one type, resulting in a total of 65 episodes.Eight of these episodes (13.1%) were admissions considered to be avoidable, 26 (42.2%) were unnecessary prolongation of admissions, and 31 (58.1%) were patients assessed to be able to receive more relevant care elsewhere.

LESSONS LEARNED

One out of four assessed patients admitted to a psychiatric ward was exposed to an inappropriate element in their care pathway. The Patient Inventory tool can assist in a structured dialogue between clinical staff and management to identify focus areas for improvement efforts.

Implementation of external quality assessment of microscopy for improved parasite detection and confirmatory diagnosis of malaria in Tanzanian Military health facilities

Objective

Good quality microscopy is critical for accurate detection and confirmation of malaria parasite infections. Microscopy relies on the skills of technicians to prepare and read slides, high quality reagents, and a good program of internal and external quality control (EQA), which are lacking in most malaria endemic settings. This study was undertaken between January 2016 and December 2018 to pilot an EQA of microscopy for improved diagnosis of malaria and patient care in Tanzanian Military health facilities.

Results

Of all blood smears crosschecked (n = 4000) at baseline, only 38.5% were incorrectly diagnosed by laboratory staff with false positive and negative rates of 46.7% and 16.4%, respectively. During the implementation of EQA, false positive and negative results decreased due to increased quality index of slide preparation and reading through supportive supervision, and retraining of laboratory personnel. There was a gradual increase of quarterly and annual total quality index for all laboratories, from 60% in 2016 to 78% in 2017 and 90% in 2018. The mean proficiency testing performance scores also increased from 75% in 2016 to 82% in 2017 and to 90% in 2018. Poor blood smear preparation and staining contributed to high false positive and negative rates while EQA helped in improvement of diagnostics.

Assessors assemble: the need for harmonised external quality assessment schemes for emerging diagnostic methodologies in the field of parasitology

Global travel and migration trends have meant a huge increase in the numbers of people exposed to tropical parasitic diseases. Thus, there is an increasing need for robust, reproducible and reliable diagnostic techniques in the field. Advanced molecular and lateral flow techniques have pushed the boundaries of clinical parasite diagnostics with their enhanced sensitivities and specificities. These emerging technologies are, however, not without their challenges, and recently there has been multiple evidence of a lack of consensus among protocols and results obtained by quality assessment of these novel technologies. This commentary discusses findings from some recent quality assessment studies in the field of blood and faecal parasitology. The article also makes recommendations for a unified and harmonised approach towards delivering high-quality clinical parasitology diagnoses, especially through the use of proficiency testing.

First results of a national external quality assessment scheme for the detection of SARS-CoV-2 genome sequences

BACKGROUND

Broad and decentralised testing of SARS-CoV-2 RNA genomes is a WHO-recommended strategy to contain the SARS-CoV-2 pandemic by identifying infected cases in order to minimize onward transmission. With the need to increase the test capacities in Austria, nation-wide numerous laboratories rapidly implemented assays for molecular detection of SARS-CoV-2 based on real-time RT-PCR assays. The objective of this study was to monitor reliability of the laboratory results for SARS-CoV-2 RNA detection through an external quality assessment (EQA) scheme.

METHODS

For this, the Center for Virology, Medical University of Vienna was tasked by the Federal Ministry of Social Affairs, Health, Care and Consumer Protection to perform the first Austrian EQA on SARS-CoV-2 which was organised in cooperation with the Austrian Association for Quality Assurance and Standardization of Medical and Diagnostic Tests (ÖQUASTA). Data were analysed on the basis of qualitative outcome of testing in relation to the nucleic acid (NA) extraction and detection methods used.

RESULTS AND CONCLUSION

A total of 52 laboratories participated, contributing results from 67 test panels comprising 42 distinct combinations of NA extraction and PCR reagents. By testing 3 positive (CT values: S1, 28.4; S2, 33.6; S3, 38.5) and 1 negative sample, no false-positive results were obtained by any of the laboratories. Otherwise, 40/67 tests (60 %) detected all positive samples correctly as positive, but 25/67 tests (37 %) did not detect the weakest positive sample (S3), and 3 % reported S2 and S3 as false-negative. Improvement in test sensitivity by focusing on NA extraction and/or PCR-based detection is recommended.

Comparison among different measurement systems for fibrinogen using fresh samples and frozen samples

BACKGROUND

Many laboratories in China have several types of coagulation analyzers. Differences in fibrinogen results among different systems may cause inappropriate medical decisions. Our aim was to set the comparability evaluation criteria and evaluate comparability of different fibrinogen measurement systems using fresh and frozen samples.

METHODS

Biological variation (BV) publications on fibrinogen were reviewed. Total error based on reliable BV data and external quality assessment (EQA) criteria were combined to set allowable limit. The compliance rate of samples for the limit should achieve at least 80% if the results obtained from different systems were comparable. Fifty-seven samples before and after freeze-thaw were measured by three measurement systems and the percentage of compliant samples was calculated.

RESULTS

The allowable limit was 11.3%. The compliance rates of fresh samples were 78.2-84.2%, and the rates of frozen samples were 54.5-93.0%. The comparability results were different using two kinds of samples.

CONCLUSIONS

It is feasible to set allowable limits of comparability based on BV and state of the art; and fresh samples are recommended for evaluating systems comparability. The results of comparability are related to samples' concentrations distribution, which should range over the concentration ranges in routine laboratory tests.

Laboratory reporting on the clinical spectrum of CFTR p.Arg117His: Still room for improvement

BACKGROUND

The clinical spectrum associated with cystic fibrosis transmembrane conductance regulator (CFTR) variant p.Arg117His is highly variable, ranging from full-blown cystic fibrosis (CF) in a small number of cases to CFTR-related disorders (CFTR-RDs) or no symptoms at all. Therefore, taking into account phenotype variability is essential for interpretation. External quality assessment (EQA) schemes can help laboratories to objectively assess the quality of genotyping and reporting by the laboratory.

METHODS

We performed a retrospective longitudinal data analysis on laboratory performance regarding the interpretation of p.Arg117His during CF EQA scheme participation. Completeness and accuracy of reporting on two mock clinical cases were each compared over time (case 1: 2005, 2007 and 2012; case 2: 2015 and 2018). These cases concerned subjects compound heterozygous for p.Phe508del and p.Arg117His in cis with 7T, but with different clinical backgrounds (family planning (case 1) versus diagnostic testing for a child (case 2)). Furthermore, we analyzed the influence of previous participations, annual test volume, accreditation status and laboratory setting on overall performance.

RESULTS

Overall performance improved over time, except during the 2007 CF EQA scheme. In addition, previous participations had a beneficial effect on laboratory performance. Accreditation status, annual test volume and laboratory setting did not significantly influence total interpretation scores.

CONCLUSIONS

In general, laboratories performed well on both cases, although reporting on the variable clinical spectrum of p.Arg117His in cis with 7T and on the disease liability of individual CFTR variants can still improve. Moreover, this study underlined the educational role of CF EQA schemes.

Sensitive detection methods are key to identify secondary EGFR c.2369C>T p.(Thr790Met) in non-small cell lung cancer tissue samples

Background

Correct identification of the EGFR c.2369C>T p.(Thr790Met) variant is key to decide on a targeted therapeutic strategy for patients with acquired EGFR TKI resistance in non-small cell lung cancer. The aim of this study was to evaluate the correct detection of this variant in 12 tumor tissue specimens tested by 324 laboratories participating in External Quality Assessment (EQA) schemes.

Methods

Data from EQA schemes were evaluated between 2013 and 2018 from cell lines (6) and resections (6) containing the EGFR c.2369C>T p.(Thr790Met) mutation. Adequate performance was defined as the percentage of tests for which an outcome was available and correct. Additional data on the used test method were collected from the participants. Chi-squared tests on contingency tables and a biserial rank correlation were applied by IBM SPSS Statistics version 25 (IBM, Armonk, NY, USA).

Results

In 26 of the 1190 tests (2.2%) a technical failure occurred. For the remaining 1164 results, 1008 (86.6%) were correct, 151 (12.9%) were false-negative and 5 (0.4%) included incorrect mutations. Correct p.(Thr790Met) detection improved over time and for repeated scheme participations. In-house non-next-generation sequencing (NGS) techniques performed worse (81.1%, n = 293) compared to non-NGS commercial kits (85.2%, n = 656) and NGS (97.0%, n = 239). Over time there was an increase in the users of NGS. Resection specimens performed worse (82.6%, n = 610 tests) compared to cell line material (90.9%, n = 578 tests), except for NGS (96.3%, n = 344 for resections and 98.6%, n = 312 for cell lines). Samples with multiple mutations were more difficult compared to samples with the single p.(Thr790Met) variant. A change of the test method was shown beneficial to reduce errors but introduced additional analysis failures.

Conclusions

A significant number of laboratories that offer p.(Thr790Met) testing did not detect this relevant mutation compared to the other EQA participants. However, correct identification of this variant is improving over time and was higher for NGS users. Revising the methodology might be useful to resolve errors, especially for resection specimens with low frequency or multiple variants. EQA providers should include challenging resections in the scheme.

WHO malaria nucleic acid amplification test external quality assessment scheme: results of distribution programmes one to three

Background

The World Health Organization (WHO) recommends parasite-based diagnosis of malaria. In recent years, there has been surge in the use of various kinds of nucleic-acid amplification based tests (NAATs) for detection and identification of Plasmodium spp. to support clinical care in high-resource settings and clinical and epidemiological research worldwide. However, these tests are not without challenges, including lack (or limited use) of standards and lack of reproducibility, due in part to variation in protocols amongst laboratories. Therefore, there is a need for rigorous quality control, including a robust external quality assessment (EQA) scheme targeted towards malaria NAATs. To this effect, the WHO Global Malaria Programme worked with the UK National External Quality Assessment Scheme (UK NEQAS) Parasitology and with technical experts to launch a global NAAT EQA scheme in January 2017.

Methods

Panels of NAAT EQA specimens containing five major species of human-infecting Plasmodium at various parasite concentrations and negative samples were created in lyophilized blood (LB) and dried blood spot (DBS) formats. Two distributions per year were sent, containing five LB and five DBS specimens. Samples were tested and validated by six expert referee laboratories prior to distribution. Between 37 and 45 laboratories participated in each distribution and submitted results using the online submission portal of UK NEQAS. Participants were scored based on their laboratory’s stated capacity to identify Plasmodium species, and individual laboratory reports were sent which included performance comparison with anonymized peers.

Results

Analysis of the first three distributions revealed that the factors that most significantly affected performance were sample format (DBS vs LB), species and parasite density, while laboratory location and the reported methodology used (type of nucleic acid extraction, amplification, or DNA vs RNA target) did not significantly affect performance. Referee laboratories performed better than non-referee laboratories.

Conclusions

Globally, malaria NAAT assays now inform a range of clinical, epidemiological and research investigations. EQA schemes offer a way for laboratories to assess and improve their performance, which is critical to safeguarding the reliability of data and diagnoses especially in situations where various NAAT methodologies and protocols are in use.

External Quality Assessment on CD4+ T-Cell Count Using in-House Proficiency Testing Panels for CD4 Count Laboratories in Addis Ababa, Ethiopia

Background

CD4+ T-cell count External Quality Assessment program is important for the evaluation of performance of CD4 count laboratories. The aim of this study was to assess the quality of CD4count laboratory performance using in-house Proficiency testing panels that perform routine CD4 counts in Addis Ababa, Ethiopia, 2013/14.

Methods

Participating laboratories were 20, 23 and 25 in trials 1, 2 and 3, respectively. In-house prepared fresh whole blood samples both with “normal” and “low” CD4 values were sent to participating laboratories. Percentage and absolute counts of CD4+ T-lymphocytes were done using their routine procedures. Data were analyzed for each trial including trimmed mean, standard deviation (SD), percent coefficient of variation (%CV), residual, and standard deviation index (SDI) values for both absolute counts and percentages of CD4+ lymphocytes (%CD4).

Results

Most participating laboratories produced results that were within 2SD of the mean. Average inter-laboratory precision (trimmed %CV) was 10.87% and 5.14% for CD4 absolute counts and %CD4, respectively. For normal material, the trimmed mean %CV was 9.59% and3.23% for CD4 absolute counts and %CD4, respectively. For low material, the trimmed mean % CV was 12.15% and 7.05% for CD4 absolute counts and %CD4 respectively. BDFACSCount™ users showed the best accuracy and precision as evidenced by longitudinal analysis.

Conclusion

This study was found to help facilities in early identifying their gaps with regard to their CD4 count performance and in avoiding the challenges encountered during participation in external EQA providers like the high cost, transportation problem, feedback delay and CD4laboratory coverage.

Inter-laboratory proficiency testing scheme for tumour next-generation sequencing in Ontario: a pilot study

Background

A pilot inter-laboratory proficiency scheme for 5 Ontario clinical laboratories testing tumour samples for the Ontario-wide Cancer Targeted Nucleic Acid Evaluation (octane) study was undertaken to assess proficiency in the identification and reporting of next-generation sequencing (ngs) test results in solid tumour testing from archival formalin-fixed, paraffin-embedded (ffpe) tissue.

Methods

One laboratory served as the reference centre and provided samples to 4 participating laboratories. An analyte-based approach was applied: each participating laboratory received 10 ffpe tissue specimens profiled at the reference centre, with tumour site and histology provided. Laboratories performed testing per their standard ngs tumour test protocols. Items returned for assessment included genes and variants that would be typically reported in routine clinical testing and variant call format (vcf) files to allow for assessment of ngs technical quality.

Results

Two main aspects were assessed:■ Technical quality and accuracy of identification of exonic variants■ Site-specific reporting practicesTechnical assessment included evaluation of exonic variant identification, quality assessment of the vcf files to evaluate base calling, variant allele frequency, and depth of coverage for all exonic variants. Concordance at 100% was observed from all sites in the technical identification of 98 exonic variants across the 10 cases. Variability between laboratories in the choice of variants considered clinically reportable was significant. Of the 38 variants reported as clinically relevant by at least 1 site, only 3 variants were concordantly reported by all participating centres as clinically relevant.

Conclusions

Although excellent technical concordance for ngs tumour profiling was observed across participating institutions, differences in the reporting of clinically relevant variants were observed, highlighting reporting as a gap where consensus on the part of Ontario laboratories is needed.

Can hospital accreditation enhance patient experience? Longitudinal evidence from a Hong Kong hospital patient experience survey

BACKGROUND

Hospital accreditation is expected to improve health care quality and patient satisfaction. However, little and conflicting evidence is currently available to support its effect on patient outcomes, particularly patient experience. Hong Kong recently launched a pilot programme to test an infrastructure for accreditation of both private and public hospitals with the Australian Council on Healthcare Standards. This study aims to evaluate the longitudinal impact of hospital accreditation on patient experience in a publicly-funded university teaching hospital in Hong Kong.

METHODS

Three cross-sectional surveys were conducted at three time points: 9 months pre- accreditation as baseline (T1), three (T2) and fifteen months (T3) post-accreditation. Acute care inpatients aged 18 to 80 were recruited on the second day of hospital admission to complete the Picker Patient Experience Questionnaire-15 (PPE-15). Baseline data was first compared to the 2005 Hong Kong average for public hospitals using t-tests. Data was then analyzed using ANOVA and multiple linear regression to evaluate differences across the three cross-sections and examine the effect of accreditation over time while controlling for covariates.

RESULTS

3083 patients (T1 = 896, T2 = 1093, T3 = 1094) completed the survey for a response rate of 83.5, 86.1, and 83.8%, respectively. The hospital baseline domain and summary patient experience scores differed from the Hong Kong public hospital average obtained from the 2005 Thematic Household Survey. All domain and summary patient experience scores declined (improved) over the study period (T1 to T3). The multiple regression results confirmed the time point score comparisons with declining (improving) parameter estimates for T2 and T3 for all domain and summary scores except the 'continuity and transition' domain, for which the declining coefficient was only significant at T3.

CONCLUSIONS

While hospital accreditation has not been shown to improve patient outcomes, this study suggests the accreditation exercise may enhance patient experience. Moreover, it suggests the quality improvement initiatives associated with accreditation may address areas of concern emphasized by Hong Kong patients, such as involvement in care and emotional support from providers.

To Win the Battle, First Know Your Enemy: Error Rates in Immunohematology External Quality Assessment Results

Background

As some errors in pretransfusion testing remain unrecognized, error rates and the resulting need for corrective measures are probably underestimated. External quality assessment (EQA) schemes could provide valuable input for identifying error-prone laboratory tests because they are designed to monitor test performance and errors. So far, however, there are only limited published data on error rates in such schemes.

Methods

The types and incidence of incorrect results in an EQA scheme for red cell immunohematology with 187 participating laboratories were examined. The results of 58 distributions between 1999 and 2017 were evaluated, considering also the employed determination methods.

Results

Out of a total of 58,726 results, 563 (0.96%) were incorrect. Error rates were 5.45% for antibody identification, 1.39% for Rh phenotyping, 0.83% for serologic cross-match, 0.60% for direct antiglobulin test, 0.20% for Kell phenotyping, 0.16% for antibody screening, and 0.14% for ABO phenotyping. During the observation period, 53 participants reported error-free results, while 37 reported one incorrect result and 97 repeatedly reported incorrect results for one or more analytes. Error rates obtained by manual methods significantly surpassed those obtained by automated methods (1.04 vs. 0.42%). The introduction of double testing with two different systems reduced error rates in Rh phenotyping from 1.55 to 0.50%.

Conclusion

Risk assessment should consider that error rates in pretransfusion test results vary. These data delineate the error risk potential of individual laboratory tests and thus should aid in tailoring appropriate improvement measures.

European follow-up of incorrect biomarker results for colorectal cancer demonstrates the importance of quality improvement projects

Biomarker analysis for colorectal cancer has been shown to be reliable in Europe with 97% of samples tested by EQA participants to be correctly classified. This study focuses on errors during the annual EQA assessment. The aim was to explore the causes and actions related to the observed errors and to provide feedback and assess any improvement between 2016 and 2017. An electronic survey was sent to all laboratories with minimum one genotyping error or technical failure on ten tumor samples. A workshop was organized based on 2016 survey responses. Improvement of performance in 2017 was assessed for returning participants (n = 76), survey respondents (n = 13) and workshop participants (n = 4). Survey respondents and workshop participants improved in terms of (maximum) analysis score, successful participation, and genotyping errors compared to all returning participants. In 2016, mostly pre- and post-analytical errors (both 25%) were observed caused by unsuitability of the tumor tissue for molecular analysis. In 2017, most errors were due to analytical problems (50.0%) caused by methodological problems. The most common actions taken (n = 58) were protocol revisions (34.5%) and staff training (15.5%). In 24.1% of issues identified no action was performed. Corrective actions were linked to an improved performance, especially if performed by the pathologist. Although biomarker testing has improved over time, error occurrence at different phases stresses the need for quality improvement throughout the test process. Participation to quality improvement projects and a close collaboration with the pathologist can have a positive influence on performance.

Desirable quality-control materials for the establishment of qualified external quality assessment on prenatal diagnosis of chromosomal aneuploidies

OBJECTIVE

To prepare desirable quality-control materials for the establishement of qualified external quality assessment on fluorescence in situ hybridization (FISH)-detected prenatal diagnosis of chromosomal aneuploidies.

METHODS

Four types of amniotic fluid cell suspensions (13-trisomy, 18-trisomy, 21-trisomy and 47,XXY) were mixed together by ratio to produce mosaicism with the percentages of each aneuploidy as 10%, 20%, 30% and 40%, respectively. After being stored in liquid nitrogen of -196 °C for six months, randomly selected samples were incubated in 37 °C water, followed by cultivation, hypo-osmosis and fixation. Finally, FISH detetion was applied on them before and after external laboratory mailing, in step with detection on conventional case samples.

RESULTS

Before mailing, the positive rates of each aneuploidy described above were 12.8%, 23.6%, 33.8%, 44.0%, while 12.6%, 23.8%, 34.0%, 43.5% after mailing. t-test, criteria for stability assessment of quality-control materials in CANS-GL03:2006, showed no significant effect of external mailing on mosaicism since corresponding t values are lower than threshold with significance level α as 0.05 and degree of freedom as 10.

CONCLUSION

As FISH detection showed, the mosaic cell strains prepared in current study exhibited excellent stabilities after cryopreservation in -196 °C, subculture, hypo-osmosis, fixation and external laboratory mailing, demonstrating them as reliable and promising quality-control materials for the establishment of a qualified external quality assessment on prenatal diagnosis of chromosomal aneuploidies.

External quality assessment of malaria microscopy diagnosis in selected health facilities in Western Oromia, Ethiopia

Background

Accurate early diagnosis and prompt treatment are one of the key strategies to control and prevent malaria disease. External quality assessment is the most effective method for evaluation of the quality of malaria microscopy diagnosis. The aim of this study was to assess the quality of malaria microscopy diagnosis and its associated factors in selected public health facility laboratories in East Wollega Zone, Western Ethiopia.

Methods

Facility-based cross-sectional study design was conducted in 30 randomly selected public health facility laboratories from November 2014 to January 2015 in East Wollega Zone, Western Ethiopia. Ten validated stained malaria panel slides with known Plasmodium species, developmental stage and parasite density were distributed. Data were captured; cleaned and analyzed using SPSS version 20 statistical software-multivariate logistic regressions and the agreement in reading between the peripheral diagnostic centers and the reference laboratory were done using kappa statistics.

Results

A total of 30 health facility laboratories were involved in the study and the overall quality of malaria microscopy diagnosis was poor (62.3%). The associated predictors of quality in this diagnosis were in-service training [(AOR = 16, 95% CI (1.3, 1.96)], smearing quality [(AOR = 24, 95% CI (1.8, 3.13)], staining quality [(AOR = 15, 95% CI (2.35, 8.61), parasite detection [(AOR = 9, 95% CI (1.1, 8.52)] and identification skills [(AOR = 8.6, 95% CI (1.21, 1.63)]. Eighteen (60%) of health facility laboratories had in-service trained laboratory professionals on malaria microscopy diagnosis.

Conclusion

Overall quality of malaria microscopy diagnosis was poor and a significant gap in this service was observed that could impact on its diagnostic services.

Misdiagnosis of pulmonary tuberculosis and associated factors in peripheral laboratories: a retrospective study, Addis Ababa, Ethiopia

Objective

Sputum smear microscopy reading errors are likely to result in failure to detect persons with infectious TB. This study was intended to review misdiagnosis of pulmonary TB and associated factors in peripheral laboratories.

Results

During the study period 1033 (10.5%) sputum smear positive and 8783 (89.5%) smear negative slides were reported by peripheral laboratories. The slides were re-read by the central referral laboratories (CRLs) as the reference standard reading. Of 1033 positive slides reported by peripheral laboratories, 25 (2.4%) were false positive. Out of 8783 smear negative slides reported by peripheral laboratories, 35 (0.4%) were false negative. The sensitivity, specificity, positive predictive value and negative predictive value of peripheral laboratories were 96.64, 99.72, 97.58, and 99.61% respectively. The peripheral laboratories and CRLs have an observed agreement (Po) of 0.9939. Of 135 peripheral laboratories, 93 (68.9%) read negative and positive slides correctly, 49 (36.3%) did not have lens cleaning tissue papers, 11 (8.1%) lacked frosted slides, and 14 (10.4%) had shortage of reagents. As conclusions, the peripheral laboratories and CRLs had high agreement for sputum smear microscopy reading. However, a few TB cases were misdiagnosed despite having the disease; these individuals might continue to spread the infection in the community.

Quality control of malaria microscopy reveals misdiagnosed non-falciparum species and other microscopically detectable pathogens in Senegal

Background

In developing countries, malaria diagnosis relies on microscopy and rapid diagnostic tests. In Senegal, national malaria control program (NMCP) regularly conducts supervisory visits in health services where malaria microscopy is performed. In this study, expert microscopists assessed the performance of laboratory technicians in malaria microscopy.

Methods

The present external quality assessment (EQA) was conducted in three different areas of malaria transmission. Participants were laboratory technicians previously trained by NMCP on malaria microscopy. Stored read slides were randomly collected for blinded re-checking by expert microscopists. At the same time a set of 8 slides (3 positive P. falciparum and 5 negative slides) were submitted to participants for proficiency testing. Microscopists performance were evaluated on the basis of the errors rates on slide reading—high false positive (HFP), high false negative (HFN), low false positive (LFP) and low false negative (LFN)—and the calculation of their sensitivities and specificities relative to expert microscopy. Data were entered and analysed using Microsoft Excel software.

Results

A total of 450 stored slides were collected from 17 laboratories for re-checking. Eight laboratories scored 100% of correct reading. Only one major error was recorded (HFP). Six laboratories recorded LFN results: Borrelia, P. ovale, and low parasite densities (95 and 155 p/μl) were missed. Two P. falciparum slides were misidentified as P. malariae and one P. ovale slide as P. vivax. The overall sensitivities and specificities for all participants against expert microscopists were 97.8 and 98.2% respectively; Sensitivities and specificities of hospital microscopists (96.7 and 98.9%) were statistically similar to those of health centre microscopists (98.5 and 97.8% respectively) (p = 0.3993 and p = 0.9412 respectively). Overall, a very good agreement was noted with kappa value of 0.96 (CI_95% 93.4–98.6%) relative to expert microscopy. Proficiency testing showed that among the 17 participants, 11 laboratories scored 100% of correct reading. Three LFN and four LFP results were recorded respectively. The P. falciparum slide with Maurer dots was misidentified as P. ovale in 1 centre and the same slide was misread as P. vivax in another centre; No major error (HFP or HFN) was noted.

Conclusion

EQA of malaria microscopy showed an overall good performance especially regarding P. falciparum detection. However, efforts need to be made addressing the ability to detect non-falciparum species and others endemic blood pathogens such as Borrelia. The further NMCP training sessions and evaluations should consider those aspects to expect high quality-assured capacity for malaria microscopy.