Point-of-care testing of group A streptococcal antigen: performance evaluated by external quality assessment

Improving diagnostic performance of coronavirus disease 2019 rapid antigen testing through computer-based feedback training using open-source experimental psychology software

INTRODUCTION

Rapid antigen testing (RAT) results are visually read as whether colored line is present or absent. The subjective interpretation potentially misses detecting weak lines due to lower analyte concentration in samples tested, requiring training. Although routine test experience has improved the result readout skills, it consumes time and resources. Therefore, we created a computer-based feedback training method using open-source experimental psychology software, wherein participants accumulate RAT result readout experience by repeatedly responding positive/negative to randomly presented pictures showing RAT results; then, they receive feedback on their answers as correct or incorrect and are asked to stare at the pictures again with the knowledge of correct answer. This study aimed to examine the training effects in improving the skills, using coronavirus disease 2019 (COVID-19) RAT.

METHODS

Twenty-two medical technologists were randomly divided into two groups: the feedback-training and test-experience groups. Using several pictures showing positive and negative results of COVID-19 RAT, after examination of their initial result readout skills, feedback-training group received the feedback training, whereas test-experience group performed an equal number of tests without feedback to accumulate test experience, and their skills were examined again. The ratio of "positive" answers to the pictures showing positive results (i.e., hit rate) was statistically analyzed.

RESULTS

The feedback-training group showed a significantly higher hit rate after their training, whereas the test-experience group did not. The feedback training effects were manifested in weak line detection.

CONCLUSIONS

This computer-based feedback training method can be an effective tool for improving RAT result readout skills.

Evaluation of the Serological Point-of-Care Testing of Infectious Mononucleosis by Data of External Quality Control Samples

Timely and reliable laboratory diagnostics is a necessity for patient safety and good patient management. Success in external quality assessment (EQA) reflects on the everyday work in a clinical laboratory. This study evaluated the reliability of serological point-of-care (POC) testing for the Epstein-Barr virus (EBV) that causes infectious mononucleosis (IM). Data from the results of 95 external quality control (EQC) samples, altogether 18 885 results during an eight-year period (2010-2017) were collected from 273 Finnish testing sites. Diagnosing acute infectious mononucleosis (EBV IM) is based on clinical, haematological and serological findings. Heterophile antibody tests are used for this purpose because they can be carried out at POC and are cheap and robust to perform. In this study, the data showed that the testing sites used 3 test methods and 17 different test kits; of the kits, 4 were used during the whole study period. The most commonly used test methods were immunochromatographic assays (12 test kits, 17 959 EQC results). Latex agglutination (4 test kits, 504 results) and immunofiltration test methods (one kit, 422 results) were also used. The overall success rate was 99.3% (for positive samples 99.6%, for negative samples 99.1%). The success rates of the different test methods varied from 94.3% for the immunofiltration method to 99.6% for the latex agglutination method. The lowest success rates were found for negative samples: 82.0% (QuickVue, Quidel [immunochromatographic method]), 91.3% (RDT EBV IgM Assay, Bio-Rad [immunofiltration method]). The results of the negative samples that represented old EBV immunity were the most difficult to interpret with a success rate of 98.9% compared to success rates of clearly positive (99.6%) and negative (99.5%) samples (P < .001). Especially the immunofiltration method (RDT EBV IgM Assay) produced 13.7% false positive results for samples of old immunity. The data showed that 42 of the studied 95 EBV IM EQA rounds were reported as expected (true positive or true negative) by all testing sites.

Comparison of the ID Now Influenza A & B 2, Cobas Influenza A/B, and Xpert Xpress Flu Point-of-Care Nucleic Acid Amplification Tests for Influenza A/B Virus Detection in Children

Early diagnosis of influenza (Flu) is critical for patient management and infection control. The ID Now influenza A & B 2 (ID Now) assay (Abbott Laboratories), Cobas influenza A/B nucleic acid test (LIAT; Roche Molecular Systems, Inc.), and Xpert Xpress Flu (Xpert; Cepheid) are rapid, point-of-care molecular assays for Flu virus detection. The study aim was to compare the performances of these three commercially available Clinical Laboratory Improvement Amendments (CLIA)-waived Flu virus assays.

Early diagnosis of influenza (Flu) is critical for patient management and infection control. The ID Now influenza A & B 2 (ID Now) assay (Abbott Laboratories), Cobas influenza A/B nucleic acid test (LIAT; Roche Molecular Systems, Inc.), and Xpert Xpress Flu (Xpert; Cepheid) are rapid, point-of-care molecular assays for Flu virus detection. The study aim was to compare the performances of these three commercially available Clinical Laboratory Improvement Amendments (CLIA)-waived Flu virus assays. We prospectively enrolled 201 children <18 years old from January to April 2018 and collected nasopharyngeal swab specimens in viral medium. Aliquots were frozen for testing on different diagnostic platforms, as per the manufacturers’ instructions. CDC Flu A/B PCR was used as a reference method to evaluate the performances of these three platforms. Among the 201 specimens tested, the CDC Flu A/B PCR assay detected Flu A/B virus in 107 samples (Flu A virus, 73 samples; Flu B virus, 36 samples; dual Flu A/B virus positive, 2 samples), while the ID Now virus detected 102 samples (Flu A virus, 69 samples; Flu B virus, 37 samples; dual Flu A/B virus positive, 4 samples; invalid rate, 1/201 [0.5%]), the LIAT detected 112 samples (Flu A virus, 74 samples; Flu B virus, 38 samples; invalid rate, 11/201 [5.5%]), and the Xpert assay detected 112 samples (Flu A virus, 76 samples; Flu B virus, 36 samples; invalid rate, 6/201 [3.0%]). The overall sensitivities for the ID Now assay, LIAT, and Xpert assay for Flu A virus detection (93.2%, 100%, and 100%, respectively) and Flu B virus detection (97.2%, 94.4%, and 91.7%, respectively) were comparable. The specificity for Flu A and B virus detection by all methods was >97%. These molecular assays had higher sensitivity than did a historical standard-of-care test from the BD Veritor antigen test (Flu A virus, 79.5%; Flu B virus, 66.7%).

Operator Influence on Blinded Diagnostic Accuracy of Point-of-Care Antigen Testing for Group A Streptococcal Pharyngitis

Background. Acute pharyngitis caused by Group A Streptococcus (GAS) is a common presentation to pediatric emergency departments (ED). Diagnosis with conventional throat culture requires 18–24 hours, which prevents point-of-care treatment decisions. Rapid antigen detection tests (RADT) are faster, but previous reports demonstrate significant operator influence on performance. Objective. To measure operator influence on the diagnostic accuracy of a RADT when performed by pediatric ED nurses and clinical microbiology laboratory technologists, using conventional culture as the reference standard. Methods. Children presenting to a pediatric ED with suspected acute pharyngitis were recruited. Three pharyngeal swabs were collected at once. One swab was used to perform the RADT in the ED, and two were sent to the clinical microbiology laboratory for RADT and conventional culture testing. Results. The RADT when performed by technologists compared to nurses had a 5.1% increased sensitivity (81.4% versus 76.3%) (p = 0.791) (95% CI for difference between technologists and nurses = −11% to +21%) but similar specificity (97.7% versus 96.6%). Conclusion. The performance of the RADT was similar between technologists and ED nurses, although adequate power was not achieved. RADT may be employed in the ED without clinically significant loss of sensitivity.

Rapid antigen detection test for group A streptococcus in children with pharyngitis

BACKGROUND

Group A streptococcus (GAS) accounts for 20% to 40% of cases of pharyngitis in children; the remaining cases are caused by viruses. Compared with throat culture, rapid antigen detection tests (RADTs) offer diagnosis at the point of care (within five to 10 minutes).

OBJECTIVES

To determine the diagnostic accuracy of RADTs for diagnosing GAS in children with pharyngitis. To assess the relative diagnostic accuracy of the two major types of RADTs (enzyme immunoassays (EIA) and optical immunoassays (OIA)) by indirect and direct comparison.

SEARCH METHODS

We searched CENTRAL, MEDLINE, EMBASE, Web of Science, CDSR, DARE, MEDION and TRIP (January 1980 to July 2015). We also conducted related citations tracking via PubMed, handsearched reference lists of included studies and relevant review articles, and screened all articles citing included studies via Google Scholar.

SELECTION CRITERIA

We included studies that compared RADT for GAS pharyngitis with throat culture on a blood agar plate in a microbiology laboratory in children seen in ambulatory care.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened titles and abstracts for relevance, assessed full texts for inclusion, and carried out data extraction and quality assessment using the QUADAS-2 tool. We used bivariate meta-analysis to estimate summary sensitivity and specificity, and to investigate heterogeneity across studies. We compared the accuracy of EIA and OIA tests using indirect and direct evidence.

MAIN RESULTS

We included 98 unique studies in the review (116 test evaluations; 101,121 participants). The overall methodological quality of included studies was poor, mainly because many studies were at high risk of bias regarding patient selection and the reference standard used (in 73% and 43% of test evaluations, respectively). In studies in which all participants underwent both RADT and throat culture (105 test evaluations; 58,244 participants; median prevalence of participants with GAS was 29.5%), RADT had a summary sensitivity of 85.6%; 95% confidence interval (CI) 83.3 to 87.6 and a summary specificity of 95.4%; 95% CI 94.5 to 96.2. There was substantial heterogeneity in sensitivity across studies; specificity was more stable. There was no evidence of a trade-off between sensitivity and specificity. Heterogeneity in accuracy was not explained by study-level characteristics such as whether an enrichment broth was used before plating, mean age and clinical severity of participants, and GAS prevalence. The sensitivity of EIA and OIA tests was comparable (summary sensitivity 85.4% versus 86.2%). Sensitivity analyses showed that summary estimates of sensitivity and specificity were stable in low risk of bias studies.

AUTHORS' CONCLUSIONS

In a population of 1000 children with a GAS prevalence of 30%, 43 patients with GAS will be missed. Whether or not RADT can be used as a stand-alone test to rule out GAS will depend mainly on the epidemiological context. The sensitivity of EIA and OIA tests seems comparable. RADT specificity is sufficiently high to ensure against unnecessary use of antibiotics. Based on these results, we would expect that amongst 100 children with strep throat, 86 would be correctly detected with the rapid test while 14 would be missed and not receive antibiotic treatment.

External quality assessment of point-of-care methods: model for combined assessment of method bias and single-participant performance by the use of native patient samples and noncommutable control materials

BACKGROUND

An important objective in external quality assessment (EQA) is to evaluate systematic deviations between methods. However, this is not possible when noncommutable control materials are used. The aim of this study was to develop an EQA model that incorporates a method bias evaluation using native patient samples into EQA schemes in which noncommutable materials are used.

METHODS

The model was applied twice in a point-of-care (POC) international normalized ratio survey among 1341 and 1578 participants. To estimate bias, about 100 native patient samples for each POC method were analyzed by a selected group of "expert" primary healthcare centers and on a designated comparison method. In addition, the expert centers as well as all the other EQA participants analyzed 2 noncommutable control materials, and method-specific target values were established. Both method bias and the deviation of a single-participant result from the method target value were evaluated against analytical quality specifications, making combined assessment possible. The best-case scenario occurred when both results were within the quality specifications.

RESULTS

Two POC methods fulfilled the quality specification for bias, whereas one did not. The best-case scenario was achieved by more than 90% of the participants using the methods with no bias, whereas none of the participants using the method with unacceptable bias achieved this result.

CONCLUSIONS

We propose an EQA model for which the bias of POC methods can be evaluated in situations in which commutable control materials are not available.

Methods for the rapid screening of group A streptococci: fluorescent in situ hybridization versus immunochromatography

OBJECTIVE

To evaluate the accuracy of detection for screening group A streptococci (GAS) in pediatric clinics using fluorescent in situ hybridization (FISH) and immunochromatographic assay (ICA).

SUBJECTS AND METHODS

A group of 630 children who attended an outpatient clinic with signs and symptoms of acute upper respiratory tract infection were enrolled in this study. Specimens were collected using a double-swab collection device. Culturing methods were employed as the gold standard for comparison. Discordant results (i.e., positive results for FISH or ICA along with negative culture results) were further evaluated by using Todd-Hewitt broth (THB) with subsequent subculture for group A selective streptococcus agar. True-positive or false-positive of GAS was determined by the presence or absence of THB subculture. The diagnostic characteristics of FISH and ICA were assessed.

RESULTS

After discrepant analysis, the sensitivity, specificity and positive and negative predictive values were as follows: 89.2, 100, 100 and 95.4%, respectively, for FISH; corresponding values for ICA were 76.8, 98.6, 96.1 and 90.5%.

CONCLUSION

The results demonstrated that the FISH assay had a higher detection sensitivity than ICA and is suitable for rapid and accurate GAS screening in pediatric clinics.

Diagnosing infections--current and anticipated technologies for point-of-care diagnostics and home-based testing

In recent years, we have witnessed many transitions in healthcare systems around the globe. For example, population expansion and ageing, and the human immunodeficiency virus (HIV)-AIDS epidemics, have exerted pressure to decentralize the practice of healthcare outside of traditional settings to bring care to those in need. Upstream of patient management, diagnosis is aimed at adequately orienting medical decisions, and considerable efforts have been made to make this process faster and more efficient. However, there are several diseases and medical conditions that may/will benefit from technologies and tests that can be performed closer to the patient, at the point of care or even in the home. In this review, and in light of the paradox that technology and assay developers and healthcare officials must take into consideration for advancing human health in developed and developing countries, we present an overview of rapid diagnosis of infectious diseases at the point of care and of technologies that may contribute to enhancement of the worldwide point-of-care testing market.

In vitro evaluation of five rapid antigen detection tests for group A beta-haemolytic streptococcal sore throat infections

BACKGROUND

Using accurate and easy to use rapid antigen detection tests (RADTs) to identify group A beta-haemolytic Streptococci (GABHS) sore throat infections could reduce unnecessary antibiotic prescribing and antimicrobial resistance. Although there is no international consensus on the use of RADTs, these kits have been widely adopted in Finland, France and the USA. Yet in the UK, the Clinical Knowledge Summaries, that provide the main online guidance for GPs, discourage RADTs use, citing their poor sensitivity and inability to impact on prescribing decisions in acute sore throat infections.

OBJECTIVE

The purpose of this study was to evaluate the ease of use and in vitro accuracy (sensitivity and specificity) of the five most commonly used RADTs in Europe (OSOM Ultra, Quickvue Dipstick, Streptatest, Clearview Exact Strep A and IMI Test Pack).

METHODS

To ensure the RADTs were evaluated objectively, a standardized in vitro method using known concentrations of GABHS was used to remove the inherent biases associated with clinical studies.

RESULTS

The IMI Test Pack was the easiest RADT to use overall. The ability to detect all positive GABHS (sensitivity) varied considerably between kits from 95% [95% confidence interval (CI): 88-98%], for the IMI Test Pack and OSOM, to 62% (95% CI: 51-72%) for Clearview, at the highest GABHS concentration. None of the RADTs gave any false-positive results with commensal flora-they were 100% specific.

CONCLUSIONS

The IMI Test Pack is most suitable for use in primary care, as it had high sensitivity, high specificity and was easy to use.