Compbdt: an R program to compare two binary diagnostic tests subject to a paired design

Discriminability of the Beck Depression Inventory and its Abbreviations in an Adolescent Psychiatric Sample

Background

The Beck Depression Inventory (BDI) is a widely acknowledged self-report screening tool for evaluating the presence and intensity of depressive symptoms. The BDI-IA, although an older version, is highly correlated with the updated BDI-II, remains clinically valuable, and is widely used due to its free availability.

Aim

This study aimed to examine the psychometric properties of the BDI-IA and compare its diagnostic accuracy with the abbreviated BDI-SF, BDI-PC, and BDI-6 versions against gold-standard research diagnoses in a representative Finnish adolescent clinical population.

Methods

The participants were referred outpatient adolescents aged 13-20 years (N = 752, 73% female). We investigated structural validity with item factor analysis and evaluated the criterion validity of mean scores and factor scores with various diagnostic measures. Sample-optimal cut-offs (criterion unweighted Cohen's kappa) were estimated with a bootstrap procedure.

Results

The sample-optimal cut-off for the full BDI was 19, slightly higher than that suggested by the previous literature. The abbreviations of the BDI-IA were demonstrated to be as good as the full scale in detecting depressive symptoms in all three diagnostic categorizations.

Conclusion

The use of brief and user-friendly questionnaires such as the BDI-PC or BDI-6 is recommended to ensure optimal depression screening and minimize the administrative burden, especially in primary care settings where clinical decision-making and referrals often need to occur within a limited time frame.

testCompareR: an R package to compare two binary diagnostic tests using paired data

Background

Binary diagnostic tests are commonly used in medicine to answer a question about a patient's clinical status, most commonly, do they or do they not have some disease. Recent advances in statistical methodologies for performing inferential statistics to compare commonly used test metrics for two diagnostic tests have not yet been implemented in a statistical package.

Methods

Up-to-date statistical methods to compare the test metrics achieved by two binary diagnostic tests are implemented in the new R package testCompareR. The output and efficiency of testCompareR is compared to the only other available package which performs this function, DTComPair, as well as an open-source program, compbdt, using a motivating example.

Results

testCompareR achieves similar results to DTComPair using statistical methods with improved coverage and asymptotic performance. Further, testCompareR is faster than the currently available package and requires fewer pre-processing steps in order to produce accurate results.

Conclusions

testCompareR provides a new tool to compare the test metrics for two binary diagnostic tests compared with the gold standard. This tool allows flexible inputs, which minimises the need for data pre-processing, and operates in very few steps, so that it is easy to use even for those less experienced with R. testCompareR achieves results comparable to those computed by DTComPair, using optimised statistical methods and with improved computational efficiency.

Comparison of the modified Mallampati classification score versus the best visible Mallampati score in the prediction of difficult tracheal intubation: a single-centre prospective observational study

PURPOSE

The modified Mallampati classification is part of the preoperative airway risk assessment. Inconsistency in the way it is examined may contribute to heterogeneity in its diagnostic performance. The best visible Mallampati score could reduce interobserver heterogeneity but its diagnostic characteristics remain unknown.

METHODS

During preoperative anesthesia consultation of adult patients with a senior anesthesiologist, we compared the modified Mallampati classification score vs the best visible Mallampati score using cervical extension, tongue in, and phonation. The primary outcome was the diagnostic characteristic of the modified Mallampati classification score vs the best visible Mallampati score as predictors of difficult orotracheal intubation (more than two direct laryngoscopies or need for an alternate device). We performed a multivariable analysis to identify independent predictors of difficult orotracheal intubation in the tested cohort.

RESULTS

Difficult orotracheal intubation occurred in 77/3,243 (2.4%) patients. A best visible Mallampati score was obtained in 1,596 (49.2%) patients. Sensitivity and specificity of the modified Mallampati classification score for predicting difficult orotracheal intubation were 0.56 (95% confidence interval [CI], 0.44 to 0.66) and 0.69 (95% CI, 0.68 to 0.71), respectively. By comparison, the best visible Mallampati score was less sensitive (difference, -0.30; 95% CI, -0.19 to -0.30; P < 0.001) but more specific (difference, 0.24; 95% CI, 0.22 to -0.25; P < 0.001). In patients with difficult orotracheal intubation, 53% were incorrectly reclassified as low risk by the best visible Mallampati score.

CONCLUSION

Compared with the modified Mallampati classification score, the best visible Mallampati score decreased sensitivity for predicting difficult orotracheal intubation and falsely classified half of the patients with difficult orotracheal intubation. Taking the risks associated with difficult airways into account, our findings indicate that a careful examination of the modified Mallampati classification is required during the global preoperative airway examination.

STUDY REGISTRATION

ClinicalTrials.gov ( NCT02788253 ); 9 February 2016.

Head-to-Head Comparison of Tau and Amyloid Positron Emission Tomography Visual Reads for Differential Diagnosis of Neurodegenerative Disorders: An International, Multicenter Study

OBJECTIVE

We compared the accuracy of amyloid and [18F]Flortaucipir (FTP) tau positron emission tomography (PET) visual reads for distinguishing patients with mild cognitive impairment (MCI) or dementia with fluid biomarker support of Alzheimer's disease (AD).

METHODS

Participants with FTP-PET, amyloid-PET, and diagnosis of dementia-AD (n = 102), MCI-AD (n = 41), non-AD diseases (n = 76), and controls (n = 20) were included. AD status was determined independent of PET by cerebrospinal fluid or plasma biomarkers. The mean age was 66.9 years, and 44.8% were women. Three readers interpreted scans blindly and independently. Amyloid-PET was classified as positive/negative using tracer-specific criteria. FTP-PET was classified as positive with medial temporal lobe (MTL) binding as the minimum uptake indicating AD tau (tau-MTL+), positive with posterolateral temporal or extratemporal cortical binding in an AD-like pattern (tau-CTX+), or negative. The majority of scan interpretations were used to calculate diagnostic accuracy of visual reads in detecting MCI/dementia with fluid biomarker support for AD (MCI/dementia-AD).

RESULTS

Sensitivity of amyloid-PET for MCI/dementia-AD was 95.8% (95% confidence interval 91.1-98.4%), which was comparable to tau-CTX+ 92.3% (86.7-96.1%, p = 0.67) and tau-MTL+ 97.2% (93.0-99.2%, p = 0.27). Specificity of amyloid-PET for biomarker-negative healthy and disease controls was 84.4% (75.5-91.0%), which was like tau-CTX+ 88.5% (80.4-94.1%, p = 0.34), and trended toward being higher than tau-MTL+ 75.0% (65.1-83.3%, p = 0.08). Tau-CTX+ had higher specificity than tau-MTL+ (p = 0.0002), but sensitivity was lower (p = 0.02), driven by decreased sensitivity for MCI-AD (80.5% [65.1-91.2] vs. 95.1% [83.5-99.4], p = 0.03).

INTERPRETATION

Amyloid- and tau-PET visual reads have similar sensitivity/specificity for detecting AD in cognitively impaired patients. Visual tau-PET interpretations requiring cortical binding outside MTL increase specificity, but lower sensitivity for MCI-AD. ANN NEUROL 2024;96:476-487.

The Accuracy of the Learning-Curve Cumulative Sum Method in Assessing Brachial Plexus Block Competency

BACKGROUND

The learning-curve cumulative sum method (LC-CUSUM) and its risk-adjusted form (RA-LC-CUSUM) have been proposed as performance-monitoring methods to assess competency during the learning phase of procedural skills. However, scarce data exist about the method's accuracy. This study aimed to compare the accuracy of LC-CUSUM forms using historical data consisting of sequences of successes and failures in brachial plexus blocks (BPBs) performed by anesthesia residents.

METHODS

Using historical data from 1713 BPB performed by 32 anesthesia residents, individual learning curves were constructed using the LC-CUSUM and RA-LC-CUSUM methods. A multilevel logistic regression model predicted the procedure-specific risk of failure incorporated in the RA-LC-CUSUM calculations. Competency was defined as a maximum 15% cumulative failure rate and was used as the reference for determining the accuracy of both methods.

RESULTS

According to the LC-CUSUM method, 22 residents (84.61%) attained competency after a median of 18.5 blocks (interquartile range [IQR], 14-23), while the RA-LC-CUSUM assigned competency to 20 residents (76.92%) after a median of 17.5 blocks (IQR, 14-25, P = .001). The median failure rate at reaching competency was 6.5% (4%-9.75%) under the LC-CUSUM and 6.5% (4%-9%) for the RA-LC-CUSUM method ( P = .37). The sensitivity of the LC-CUSUM (85%; 95% confidence interval [CI], 71%-98%) was similar to the RA-LC-CUSUM method (77%; 95% CI, 61%-93%; P = .15). Identical specificity values were found for both methods (67%; 95% CI, 29%-100%, P = 1).

CONCLUSIONS

The LC-CUSUM and RA-LC-CUSUM methods were associated with substantial false-positive and false-negative rates. Also, small lower limits for the 95% CIs around the accuracy measures were observed, indicating that the methods may be inaccurate for high-stakes decisions about resident competency at BPBs.

Protocol for evaluating the fitness for purpose of an artificial intelligence product for radiology reporting in the BreastScreen New South Wales breast cancer screening programme

INTRODUCTION

Radiologist shortages threaten the sustainability of breast cancer screening programmes. Artificial intelligence (AI) products that can interpret mammograms could mitigate this risk. While previous studies have suggested this technology has accuracy comparable to radiologists most have been limited by using 'enriched' datasets and/or not considering the interaction between the algorithm and human readers. This study will address these limitations by comparing the accuracy of a workflow using AI alongside radiologists on a large consecutive cohort of examinations from a breast cancer screening programme. The study will combine the strengths of a large retrospective design with the benefit of prospective data collection. It will test this technology without risk to screening programme participants nor the need to wait for follow-up data. With a sample of 2 years of consecutive screening examinations, it is likely the largest test of this technology to date. The study will help determine whether this technology can safely be introduced into the BreastScreen New South Wales (NSW) population-based screening programme to address radiology workforce risks without compromising cancer detection rates or increasing false-positive recalls.

METHODS AND ANALYSIS

A retrospective, consecutive cohort of digital mammography screens from 658 207 examinations from BreastScreen NSW will be reinterpreted by the Lunit Insight MMG AI product. The cohort includes 4383 screen-detected and 1171 interval cancers. The results will be compared with radiologist single reading and the AI results will also be used to replace the second reader in a double-reading model. New adjudication reading will be performed where the AI disagrees with the first reader. Recall rates and cancer detection rates of combined AI-radiologist reading will be compared with the rates obtained at the time of screening.

ETHICS AND DISSEMINATION

This study has ethical approval from the NSW Health Population Health Services Research Ethics Committee (2022/ETH02397). Findings will be published in peer-reviewed journals and presented at conferences. The findings of this evaluation will be provided to programme managers, governance bodies and other stakeholders in Australian breast cancer screening programmes.

Diagnosis and subtyping of idiopathic inflammatory myopathies: caution required in the use of myositis autoantibodies

Detection of myositis autoantibodies (MAs) has utility in both the diagnosis and subtyping of idiopathic inflammatory myopathies (IIMs). Multiplex assays such as the Euroimmun line immunoassay (LIA) have significant limitations in rare diseases like IIM. A retrospective cohort study was performed on positive MA detected on LIA in 171 patients using the manufacturer's recommended cut-off. Only 16.7% were deemed true positive after clinical correlation. Autoantibody-specific cut-offs were created and applied to the original cohort, along with generically applied higher cut-offs. Positive predictive value (PPV) improved, but there was variable increase in false negatives. False positive MA results are common using LIA, but locally derived cut-offs can improve performance. Clinicians must be aware of the limitations of LIA, which is the commonest method for MA detection in Australasia.

Sensitivity and specificity of LDBio Aspergillus ICT lateral flow assay for diagnosing allergic bronchopulmonary aspergillosis in adult asthmatics

BACKGROUND

Aspergillus fumigatus-specific IgG estimation is crucial for diagnosing allergic bronchopulmonary aspergillosis (ABPA). A point-of-care LDBio immunochromatographic lateral flow assay (LFA) had 0%-90% sensitivity to detect IgG/IgM antibodies against A. fumigatus.

OBJECTIVE

To assess the accuracy of LDBio-LFA in diagnosing ABPA, using the modified ISHAM-ABPA working group criteria as the reference standard. The secondary objective was to compare the diagnostic performance between LDBio-LFA and A. fumigatus-specific IgG (cut-offs, 27 and 40 mgA/L), using a multidisciplinary team (blinded to A. fumigatus-IgG and LDBio-LFA results) diagnosis of ABPA as the reference standard.

METHODS

We prospectively enrolled adult subjects with asthma and ABPA. We performed the LDBio-LFA per the manufacturer's recommendations. We used the commercially available automated fluorescent enzyme immunoassay for measuring serum A. fumigatus-specific IgG. We used the same serum sample to perform both index tests. The tests were performed by technicians blinded to the results of other tests and clinical diagnoses.

RESULTS

We included 123 asthmatic and 166 ABPA subjects, with a mean ± SD age of 37.4 ± 14.4 years. Bronchiectasis and high-attenuation mucus were seen in 93.6% (146/156) and 24.3% (38/156) of the ABPA subjects. The sensitivity and specificity of LDBio-LFA in diagnosing ABPA were 84.9% and 82.9%, respectively. The sensitivity of serum A. fumigatus-specific IgG ≥27 mgA/L was 13% better than LDBio-LFA, with no difference in specificity. There was no significant difference in sensitivity and specificity between LDBio-LFA and serum A. fumigatus-IgG ≥40 mgA/L.

CONCLUSION

LDBio-LFA is a valuable test for diagnosing ABPA. However, a negative test should be confirmed using an enzyme immunoassay.

Comparison of commercially available, rapid, point-of-care C-reactive protein assays among children with febrile illness in southwestern Uganda

In Uganda, children with febrile illness are often treated with antibiotics even though most have self-limiting, likely viral, infections. C-reactive protein (CRP) measurement can help identify those who are more likely to have a bacterial infection and therefore need antibiotic treatment. Implementation of a CRP rapid diagnostic test (RDT) at the point-of-care in resource-constrained settings with minimal laboratory infrastructure could reduce unnecessary antibiotic use. In this study, we evaluated the performance of three semi-quantitative CRP RDTs (Actim, BTNX, Duo) against a reference CRP assay requiring an electrically powered analyzer (Afinion). While both tests demonstrated substantial agreement with Afinion, Actim had slightly higher agreement than BTNX. The sensitivity was higher for the BTNX test, whereas the Actim test had a higher specificity, at cut-offs of 40 mg/L and 80 mg/L. At a cut-off of 20 mg/L, Duo demonstrated substantial agreement with the Afinion test as well. Our results demonstrate the reliability of CRP RDTs when compared to a reference standard. CRP RDTs without the need for a laboratory-based analyzer are promising tools for optimizing antibiotic use in low-resource settings.

Pulse Oximetry as an Aid to Rule Out Pneumonia among Patients with a Lower Respiratory Tract Infection in Primary Care

Guidelines recommend chest X-rays (CXRs) to diagnose pneumonia and guide antibiotic treatment. This study aimed to identify clinical predictors of pneumonia that are visible on a chest X-ray (CXR+) which could support ruling out pneumonia and avoiding unnecessary CXRs, including oxygen saturation. A secondary analysis was performed in a clinical trial that included patients with suspected pneumonia in Swiss primary care. CXRs were reviewed by two radiologists. We evaluated the association between clinical signs (heart rate > 100/min, respiratory rate ≥ 24/min, temperature ≥ 37.8 °C, abnormal auscultation, and oxygen saturation < 95%) and CXR+ using multivariate analysis. We also calculated the diagnostic performance of the associated clinical signs combined in a clinical decision rule (CDR), as well as a CDR derived from a large meta-analysis (at least one of the following: heart rate > 100/min, respiratory rate ≥ 24/min, temperature ≥ 37.8 °C, or abnormal auscultation). Out of 469 patients from the initial trial, 107 had a CXR and were included in this study. Of these, 26 (24%) had a CXR+. We found that temperature and oxygen saturation were associated with CXR+. A CDR based on the presence of either temperature ≥ 37.8 °C and/or an oxygen saturation level < 95% had a sensitivity of 69% and a negative likelihood ratio (LR-) of 0.45. The CDR from the meta-analysis had a sensitivity of 92% and an LR- of 0.37. The addition of saturation < 95% to this CDR increased the sensitivity (96%) and decreased the LR- (0.21). In conclusion, this study suggests that pulse oximetry could be added to a simple CDR to decrease the probability of pneumonia to an acceptable level and avoid unnecessary CXRs.

Assessing Liver Fibrosis Using 2D-SWE Liver Ultrasound Elastography and Dynamic Liver Scintigraphy with 99mTc-mebrofenin: A Comparative Prospective Single-Center Study

Background and Objectives: Many quantitative imaging modalities are available that quantify chronic liver disease, although only a few of them are included in clinical guidelines. Many more imaging options are still competing to find their place in the area of diagnosing chronic liver disease. We report our first prospective single-center study evaluating different imaging modalities that stratify viral hepatitis-associated liver fibrosis in a treatment-naïve patient group. Materials and Methods: The aim of our study is to compare and to combine already employed 2D shear wave elastography (2D-SWE) with dynamic liver scintigraphy with 99mTc-mebrofenin in chronic viral hepatitis patients for the staging of liver fibrosis. Results: Seventy-two patients were enrolled in the study. We found that both 2D-SWE ultrasound imaging, with dynamic liver scintigraphy with 99mTc-mebrofenin are able to stratify CLD patients into different liver fibrosis categories based on histological examination findings. We did not find any statistically significant difference between these imaging options, which means that dynamic liver scintigraphy with 99mTc-mebrofenin is not an inferior imaging technique. A combination of these imaging modalities showed increased accuracy in the non-invasive staging of liver cirrhosis. Conclusions: Our study presents that 2D-SWE and dynamic liver scintigraphy with 99mTc-mebrofenin could be used for staging liver fibrosis, both in singular application and in a combined way, adding a potential supplementary value that represents different aspects of liver fibrosis in CLD.

Head-to-Head Comparison Between Rabbit Sign and EANM/SNMMI Criteria for the 18F-DOPA Visual Assessment of Parkinsonian Syndromes in PET/MRI: A Multiple Expert-Based and Blinded Controlled Study

PURPOSE

The aim of this study was to compare the diagnostic performance of the rabbit visual pattern versus the one endorsed by the EANM/SNMMI for the diagnosis of parkinsonian syndromes in PET/MRI.

PATIENTS AND METHODS

The 18F-DOPA PET images of 129 consecutive patients (65 Park+ and 64 controls) with 1 year of clinical follow-up were reviewed independently by 5 experienced readers on the same imaging workstation, blinded to the final clinical diagnosis. Two visual methods were assessed independently, with several days to months of interval: the criteria endorsed by EANM/SNMMI and the "rabbit" shape of the striate assessed on 3D MIP images. The sensitivities, specificities, likelihood ratios, and predictive values of the 2 diagnostic tests were estimated simultaneously by using the "comparison of 2 binary diagnostic tests to a paired design" method.

RESULTS

The estimated 95% confidence interval (CI) of sensitivities and specificities ranged from 49.4% to 76.5% and from 83.2% to 97.7%, respectively. The 95% CI estimates of positive and negative likelihood ratios ranged from 3.8 to 26.7 and from 0.26 to 0.56, respectively. The 95% CI estimates of the positive and negative predictive values ranged from 78.1% to 96.7% and from 60.3% to 81.4%, respectively. For all the parameters, no statistical difference was observed between the 2 methods (P > 0.05). The rabbit sign reduced the readers' discrepancies by 25%, while maintaining the same performance.

CONCLUSIONS

The rabbit visual pattern appears at least comparable to the current EANM/SNMMI reference procedure for the assessment of parkinsonian syndromes in daily clinical practice, without the need of any image postprocessing. Further multicenter prospective studies would be of relevance to validate these findings.

Effectiveness and reliability of the four-step STANDING algorithm performed by interns and senior emergency physicians for predicting central causes of vertigo

BACKGROUND

For emergency physicians (EPs), acute vertigo is a challenging complaint and learning a reliable clinical approach is needed. STANDING is a four-step bedside algorithm that requires (1) identifying spontaneous nystagmus with Frenzel glasses or, alternatively, a positional nystagmus; (2) characterizing the nystagmus direction; (3) assessing the vestibuloocular reflex (head impulse test); and (4) assessing the gait. The objective was to determine its accuracy for diagnosing central vertigo when using by naïve examiners as such as interns and its agreement with senior EPs.

METHODS

This was a prospective 1-year diagnostic cohort study among patients with vertigo, vestibulovisual symptoms, or postural symptoms seen by 20 interns trained in the four-step examination. The algorithm was performed first by an intern and second by a senior EP and categorized as either worrisome when indicating a central diagnosis and benign or inconclusive when indicating a peripheral diagnosis. The reference test was diffusion-weighted brain magnetic resonance imaging.

RESULTS

Among 312 patients included, 57 had a central diagnosis including 33 ischemic strokes (10.5%). The main etiology was benign paroxysmal positional vertigo (32.7%). The likelihood ratios were 4.63 and 10.33 for a worrisome STANDING, 0.09 and 0.01 for a benign STANDING, and 0.21 and 0.35 for an inconclusive STANDING, for interns and senior EPs, respectively. The algorithm showed sensitivities of 84.8% (95% CI 75.6%-93.9%) and 89.8% (95% CI 82.1%-97.5%), negative predictive values of 96.2% (95% CI 93.7%-98.6%) and 97.5% (95% CI 95.5%-99.5%), specificities of 88.9% (95% CI 85.1%-92.8%) and 91.3% (95% CI 87.8%-94.8%), and positive predictive values of 64.1% (95% CI 53.5%-74.8%) and 70.7% (95% CI 60.4%-81.0%), respectively. The agreement between interns and senior EPs was very substantial (B-statistic coefficient: 0.77) and almost perfect for each step: (1) 0.87, (2) 0.98, (3) 0.95, and (4) 0.99.

CONCLUSIONS

With a single training session, the algorithm reached high accuracy and reliability for ruling out central causes of vertigo in the hands of both novices and experienced EPs. A future multicenter randomized controlled trial should further its impact on unnecessary neuroimaging use and patient's satisfaction.

A machine-learning model for reducing misdiagnosis in heparin-induced thrombocytopenia: A prospective, multicenter, observational study

BACKGROUND

Diagnosing heparin-induced thrombocytopenia (HIT) at the bedside remains challenging, exposing a significant number of patients at risk of delayed diagnosis or overtreatment. We hypothesized that machine-learning algorithms could be utilized to develop a more accurate and user-friendly diagnostic tool that integrates diverse clinical and laboratory information and accounts for complex interactions.

METHODS

We conducted a prospective cohort study including 1393 patients with suspected HIT between 2018 and 2021 from 10 study centers. Detailed clinical information and laboratory data were collected, and various immunoassays were conducted. The washed platelet heparin-induced platelet activation assay (HIPA) served as the reference standard.

FINDINGS

HIPA diagnosed HIT in 119 patients (prevalence 8.5%). The feature selection process in the training dataset (75% of patients) yielded the following predictor variables: (1) immunoassay test result, (2) platelet nadir, (3) unfractionated heparin use, (4) CRP, (5) timing of thrombocytopenia, and (6) other causes of thrombocytopenia. The best performing models were a support vector machine in case of the chemiluminescent immunoassay (CLIA) and the ELISA, as well as a gradient boosting machine in particle-gel immunoassay (PaGIA). In the validation dataset (25% of patients), the AUROC of all models was 0.99 (95% CI: 0.97, 1.00). Compared to the currently recommended diagnostic algorithm (4Ts score, immunoassay), the numbers of false-negative patients were reduced from 12 to 6 (-50.0%; ELISA), 9 to 3 (-66.7%, PaGIA) and 14 to 5 (-64.3%; CLIA). The numbers of false-positive individuals were reduced from 87 to 61 (-29.8%; ELISA), 200 to 63 (-68.5%; PaGIA) and increased from 50 to 63 (+29.0%) for the CLIA.

INTERPRETATION

Our user-friendly machine-learning algorithm for the diagnosis of HIT (https://toradi-hit.org) was substantially more accurate than the currently recommended diagnostic algorithm. It has the potential to reduce delayed diagnosis and overtreatment in clinical practice. Future studies shall validate this model in wider settings.

FUNDING

Swiss National Science Foundation (SNSF), and International Society on Thrombosis and Haemostasis (ISTH).

Assessment of a combined musculoskeletal and chest deep learning-based detection solution in an emergency setting

Rationale and objectives

Triage and diagnostic deep learning-based support solutions have started to take hold in everyday emergency radiology practice with the hope of alleviating workflows. Although previous works had proven that artificial intelligence (AI) may increase radiologist and/or emergency physician reading performances, they were restricted to finding, bodypart and/or age subgroups, without evaluating a routine emergency workflow composed of chest and musculoskeletal adult and pediatric cases. We aimed at evaluating a multiple musculoskeletal and chest radiographic findings deep learning-based commercial solution on an adult and pediatric emergency workflow, focusing on discrepancies between emergency and radiology physicians.

Material and methods

This retrospective, monocentric and observational study included 1772 patients who underwent an emergency radiograph between July and October 2020, excluding spine, skull and plain abdomen procedures. Emergency and radiology reports, obtained without AI as part of the clinical workflow, were collected and discordant cases were reviewed to obtain the radiology reference standard. Case-level AI outputs and emergency reports were compared to the reference standard. DeLong and Wald tests were used to compare ROC-AUC and Sensitivity/Specificity, respectively.

Results

Results showed an overall AI ROC-AUC of 0.954 with no difference across age or body part subgroups. Real-life emergency physicians' sensitivity was 93.7 %, not significantly different to the AI model (P = 0.105), however in 172/1772 (9.7 %) cases misdiagnosed by emergency physicians. In this subset, AI accuracy was 90.1 %.

Conclusion

This study highlighted that multiple findings AI solution for emergency radiographs is efficient and complementary to emergency physicians, and could help reduce misdiagnosis in the absence of immediate radiological expertize.

NOL4 is a novel nuclear marker of small cell carcinoma and other neuroendocrine neoplasms

Neuroendocrine neoplasms (NENs) such as small cell lung cancer (SCLC) and large cell neuroendocrine carcinoma (LCNEC) have characteristic histologies, but immunohistochemistry using neuroendocrine markers is still desirable to confirm diagnosis. CD56 is the most sensitive marker, but also stains various normal tissues and other tumors. Recently, we reported that nucleolar protein 4 (NOL4) is present in the blood of SCLC patients and found it was stained in the SCLC nuclei. In this study, we compared expressions of NOL4 and CD56, using 64 cases of SCLC, 18 cases of LCNEC, 6 cases of atypical carcinoid tumor, 7 cases of typical carcinoid tumor, 68 cases of lung adenocarcinoma, and 62 cases of lung squamous cell carcinoma. For primary lung NENs, sensitivity, specificity, positive predictive value (PPV) and negative predictive value of NOL4 were 77.5%, 95.8%, 93.2%, and 85.1%, respectively, while those of CD56 were 92.1%, 93.3%, 91.1%, and 94.1%. The specificity and PPV of NOL4 were higher than those of CD56, although the differences were not statistically significant. However, NOL4 retains its nuclear immunoreactivity in areas of crush artifact or necrosis. Furthermore, NOL4 was not expressed in adjacent normal tissues including bronchial cells and pneumocytes. Therefore, a combination of NOL4 staining with other cytoplasmic or membranous neuroendocrine markers might enhance diagnostic utility for SCLC and other NENs.

Head-to-head comparison of biparametric versus multiparametric MRI of the prostate before robot-assisted transperineal fusion prostate biopsy

PURPOSE

Prostate biparametric magnetic resonance imaging (bpMRI) including T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) might be an alternative to multiparametric MRI (mpMRI, including dynamic contrast imaging, DCE) to detect and guide targeted biopsy in patients with suspected prostate cancer (PCa). However, there is no upgrading peripheral zone PI-RADS 3 to PI-RADS 4 without DCE in bpMRI. The aim of this study was to evaluate bpMRI against mpMRI in biopsy-naïve men with elevated prostate-specific antigen (PSA) scheduled for robot-assisted-transperineal fusion-prostate biopsy (RA-TB).

METHODS

Retrospective single-center-study of 563 biopsy-naïve men (from 01/2015 to 09/2018, mean PSA 9.7 ± 6.5 ng/mL) with PI-RADSv2.1 conform mpMRI at 3 T before RA-TB. Clinically significant prostate cancer (csPCa) was defined as ISUP grade ≥ 2 in any core. Two experienced readers independently evaluated images according to PI-RADSv2.1 criteria (separate readings for bpMRI and mpMRI sequences, 6-month interval). Reference standard was histology from RA-TB.

RESULTS

PI-RADS 2 was scored in 5.1% of cases (3.4% cancer/3.4% csPCa), PI-RADS 3 in 16.9% (32.6%/3.2%), PI-RADS 4 in 57.6% (66.1%/58.3%) and PI-RADS 5 in 20.4% of cases (79.1%/74.8%). For mpMRI/bpMRI test comparison, sensitivity was 99.0%/97.1% (p < 0.001), specificity 47.5%/61.2% (p < 0.001), PPV 69.5%/75.1% (p < 0.001) and NPV 97.6%/94.6% (n.s.). csPCa was considered gold standard. 35 cases without cancer were upgraded to PI-RADS 4 (mpMRI) and six PI-RADS 3 cases with csPCa were not upgraded (bpMRI).

CONCLUSION

In patients planned for RA-TB with elevated PSA and clinical suspicion for PCa, specificity was higher in bpMRI vs. mpMRI, which could solve constrains regarding time and contrast agent.

Detection of Early Ischemic Changes with Virtual Noncontrast Dual-Energy CT in Acute Ischemic Stroke: A Noninferiority Analysis

BACKGROUND AND PURPOSE

Dual-energy virtual NCCT has the potential to replace conventional NCCT to detect early ischemic changes in acute ischemic stroke. In this study, we evaluated whether virtual NCCT is noninferior compared with standard linearly blended NCCT, a surrogate of conventional NCCT, regarding the detection of early ischemic changes with ASPECTS.

MATERIALS AND METHODS

Adult patients who presented with suspected acute ischemic stroke and who underwent dual-energy NCCT and CTA and brain MR imaging within 48 hours were included. Standard linearly blended images were reconstructed to match a conventional NCCT. Virtual NCCT images were reconstructed from CTA. ASPECTS was evaluated on conventional NCCT, virtual NCCT, and DWI, which served as the reference standard. Agreement between CT assessments and the reference standard was evaluated with the Lin concordance correlation coefficient. Noninferiority was assessed with bootstrapped estimates of the differences in ASPECTS between conventional and virtual NCCT with 95% CIs.

RESULTS

Of the 193 included patients, 100 patients (52%) had ischemia on DWI. Compared with the reference standard, the ASPECTS concordance correlation coefficient for conventional and virtual NCCT was 0.23 (95% CI, 0.15-0.32) and 0.44 (95% CI, 0.33-0.53), respectively. The difference in the concordance correlation coefficient between virtual and conventional NCCT was 0.20 (95% CI, 0.01-0.39) and did not cross the prespecified noninferiority margin of -0.10.

CONCLUSIONS

Dual-energy virtual NCCT is noninferior compared with conventional NCCT for the detection of early ischemic changes with ASPECTS.

Commentary: Statistical analysis of 2 x 2 tables in Biomarker studies 2) Study design and statistical tests

Abstract/SummaryIn this article the importance of study design will be emphasized, statistical methods for analysing the data are described and some of the implications of these method are discussed. How these issues relate to the use of SpIN and SnOUT rules are reviewed.

Prospective study of Na[18F]F PET/CT for cancer staging in morbidly obese patients compared with [99mTc]Tc-MDP whole-body planar, SPECT and SPECT/CT

PURPOSE

This prospective study aims to assess the diagnostic test characteristics of Na[¹⁸F]F PET/CT for the skeletal staging of cancer in morbidly obese patients compared with ^99mTc-methylene diphosphonate (MDP), whole-body planar (WBS), SPECT, and SPECT/CT acquisitions.

MATERIAL AND METHODS

One hundred seventeen obese patients (BMI 46.5 ± 6.1 kg/m² and mean age, 59.0 years; range 32-89 years) with BMI > 40 kg/m² were prospectively enrolled and underwent [^99mTc]Tc-MDP WBS, SPECT, SPECT/CT, and Na[¹⁸F]F PET/CT within two weeks for the osseous staging of a malignancy. Images were assessed qualitatively using a 3-point scale. Patient and lesion-based diagnostic test characteristics were estimated using an optimistic and pessimistic dichotomization method.

RESULTS

Bone metastases were confirmed in 44 patients. Patient-based optimistic diagnostic test characteristics were (sensitivity, specificity, overall accuracy): Na[¹⁸F]F PET/CT (95.5%, 95.9%, 95.7%), [^99mTc]Tc-MDP WBS (52.3%, 71.2%, 64.1%), SPECT (61.4%, 80.8%, 73.5%) and SPECT/CT (65.9%, 91.8%, 82.1%). Lesion-based optimistic diagnostic test characteristics were: Na[¹⁸F]F PET/CT (97.7%, 97.9%, 97.7%), [^99mTc]Tc-MDP WBS (39%, 67%, 48.9%), SPECT (52.9%, 93.6%, 67.3%) and SPECT/CT (65.9%, 91.8%, 82.1%). There was no significant difference in the specificity of Na[¹⁸F]F and SPECT/CT. All other pairwise comparisons were significant (p<.001). ROC curve analysis showed a high overall accuracy of Na[¹⁸F]F with significantly higher AUCs for Na[¹⁸F]F PET/CT compared to [^99mTc]Tc-MDP WBS, SPECT, and SPECT/CT on both patient and lesion-based analysis (p<.001). Moreover, Na[¹⁸F]F PET/CT changed patient management in 38% of patients.

CONCLUSIONS

Na[¹⁸F]F PET/CT may be the preferred imaging modality for skeletal staging in morbidly obese patients. The technique provides excellent diagnostic test characteristics superior to [^99mTc]Tc-MDP bone scan (including SPECT/CT), impacts patient management, has an acceptable radiation exposure profile, and is well-tolerated. Further cost-effectiveness evaluations are warranted.

Comparison of the Diagnostic Performance of Myocardial Perfusion Scintigraphy with and Without Attenuation Correction

Objectives:

Myocardial perfusion scintigraphy (MPS) is an important diagnostic test for detecting of coronary artery stenosis (CAS); however, tissue attenuation can lead to a difference in accuracy. We evaluated the diagnostic accuracy of attenuation-corrected (AC) and non-attenuation-corrected (NC) MPS for the detection of CAS.

Methods:

We retrospectively recruited patients who underwent invasive coronary angiography within 10 months after Tc-99m sestamibi MPS. The AC and NC perfusion images were analyzed separately, and each myocardial segment was scored based on relative uptake from 0 to 4. The summed stress score (SSS), summed rest score (SRS), and summed difference score (SDS) were calculated. The diagnostic performances were analyzed using the area under the curve (AUC) of the receiver operating characteristic curve.

Results:

From 117 patients, significant coronary stenosis was present in 66 patients (56%). The SSS and SRS obtained from NC-images were higher than those from AC, supporting the presence of attenuation artifacts in NC images. The AUC of SSS and SDS were significantly higher than those of SRS in both AC- and NC-images, but no significant difference was found between the AUC of SSS, and those of SDS. The optimal cut-offs were >12 for AC-SSS, >15 for NC-SSS, >4 for AC-SDS and >3 for NC-SDS. There was no statistically significant difference in the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy among AC-SSS, NC-SSS, AC-SDS, and NC-SDS.

Conclusion:

NC-based Tc-99m-sestamibi MPS promised comparable accuracy to AC images by using different cut-off values for diagnosis.

The diagnostic accuracy of MRI and nonenhanced CT for high-risk vertebral artery anatomy for subaxial anterior cervical spine surgery safety

OBJECTIVE

Medial migration of the vertebral artery (VA) can be a risk factor for injury during anterior procedures. CT angiography (CTA) has been considered the gold standard for the evaluation of various areas of the arterial anatomy. MRI and nonenhanced CT are more commonly used as routine preoperative imaging studies, but it is unclear if these modalities can safely exclude the anomalous course of the VA. The aims of this cross-sectional observational study were to investigate risk factors for medially migrated VA on CTA and to evaluate the diagnostic accuracy of MRI and nonenhanced CT for high-risk VA anatomy in the subaxial cervical spine.

METHODS

The records of 248 patients who underwent CTA for any reason at a single academic institution between 2007 and 2018 were reviewed. The authors included MRI and nonenhanced CT taken within 1 year before or after CTA. An axial VA position classification was used to grade VA anomalies in the subaxial cervical spine. The multivariable linear regression analysis with mixed models was performed to identify the risk factors for medialized VA. The sensitivity and specificity of MRI and nonenhanced CT for high-risk VA positions were calculated.

RESULTS

A total of 175 CTA sequences met the inclusion criteria. The mean age was 63.8 years. Advanced age, disc and pedicle levels, lower cervical levels, and left side were independent risk factors for medially migrated VA. The sensitivities of MRI and nonenhanced CT for the detection of grade 1 or higher VA position were only fair, and the sensitivity of MRI was lower than that of nonenhanced CT (0.31 vs 0.37, p < 0.001), but the specificities were similarly high for both modalities (0.97 vs 0.97). With the combination of MRI and nonenhanced CT, the sensitivity significantly increased to 0.50 (p < 0.001 vs MRI and vs CT alone) with a minimal decrease in specificity.

CONCLUSIONS

Axial images of MRI and nonenhanced CT demonstrated high specificities but only fair sensitivities. Nonenhanced CT demonstrated better diagnostic value than MRI. When combining both modalities the sensitivity improved, but a substantial proportion of medialized VAs could not be diagnosed.

Differentiating central from peripheral causes of acute vertigo in an emergency setting with the HINTS, STANDING, and ABCD2 tests: A diagnostic cohort study

BACKGROUND

Diagnosing stroke in dizzy patients remains a challenge in emergency medicine. The accuracy of the neuroophthalmologic examination HINTS performed by emergency physicians (EPs) is unknown. Our objective was to determine the accuracy of the HINTS examination performed by trained EPs for diagnosing central cause of acute vertigo and unsteadiness and to compare it with another bedside clinical tool, STANDING, and with the history-based score ABCD2.

METHODS

This was a prospective diagnostic cohort study among patients with isolated vertigo and unsteadiness seen in a single emergency department (ED). Trained EPs performed HINTS and STANDING tests blinded to attending physicians. ABCD2 ≥ 4 was used as the threshold and was calculated retrospectively. The criterion standard was diffusion-weighted brain magnetic resonance imaging (MRI). Peripheral diagnoses were established by a normal MRI, and etiologies were further refined by an otologic examination.

RESULTS

We included 300 patients of whom 62 had a central lesion on neuroimaging including 49 strokes (79%). Of the 238 peripheral diagnoses, 159 were vestibulopathies, mainly benign paroxysmal positional vertigo (40%). HINTS and STANDING tests reached high sensitivities at 97% and 94% and NPVs at 99% and 98%, respectively. The ABCD2 score failed to predict half of central vertigo cases and had a sensitivity of 55% and a NPV of 87%. The STANDING test was more specific and had a better positive predictive value (PPV; 75% and 49%, respectively; positive likelihood ratio [LR+] = 3.71, negative likelihood ratio [LR-] = 0.09) than the HINTS test (67% and 44%, respectively; LR+ = 2.96, LR- = 0.04). The ABCD2 score was specific (82%, LR+ = 3.04, LR- = 0.56) but had a very low PPV (44%).

CONCLUSIONS

In the hands of EPs, HINTS and STANDING tests outperformed ABCD2 in identifying central causes of vertigo. For diagnosing peripheral disorders, the STANDING algorithm is more specific than the HINTS test. HINTS and STANDING could be useful tools saving both time and costs related to unnecessary neuroimaging use.

Evaluation of computer-aided diagnosis in breast ultrasonography: Improvement in diagnostic performance of inexperienced radiologists

PURPOSE

To evaluate if a computer-aided diagnosis (CAD) system on ultrasound (US) can improve the diagnostic performance of inexperienced radiologists.

METHODS

We collected ultrasound images of 256 breast lesions taken between March and May 2020. We asked two experienced and two inexperienced radiologists to retrospectively review the US features of each breast lesion according to the Breast Imaging Reporting and Data System (BI-RADS) categories. A CAD examination with S-Detect™ software (Samsung Healthcare, Seoul, South Korea) was conducted retrospectively by another uninvolved radiologist blinded to the BIRADS values previously attributed to the lesions. Diagnostic performances of experienced and inexperienced radiologists and CAD were compared and the inter-observer agreement among radiologists was calculated.

RESULTS

The diagnostic performance of the experienced group in terms of sensitivity was significantly higher than CAD (p < 0.001). Conversely, the diagnostic performance of inexperienced group in terms of both sensitivity and specificity was significantly lower than CAD (p < 0.001). We obtained an excellent agreement in the evaluation of the lesions among the two expert radiologists (Kappa coefficient: 88.7%), and among the two non-expert radiologists (Kappa coefficient: 84.9%).

CONCLUSION

The US CAD system is a useful additional tool to improve the diagnostic performance of the inexperienced radiologists, eventually reducing the number of unnecessary biopsies. Moreover, it is a valid second opinion in case of experienced radiologists.

Comparison of Different Investigation Strategies to Defer Cardiac Testing in Patients With Stable Chest Pain

OBJECTIVES

This study aimed to compare the current 5 investigation strategies to defer cardiac testing in patients with stable chest pain.

BACKGROUND

For the clinical management of stable chest pain, the identification of patients unlikely to benefit from further cardiac testing is important, but the most appropriate investigation strategy is unknown.

METHODS

A total of 4,207 patients referred to coronary computed tomography angiography for stable chest pain were classified into low- and high-risk groups according to the 2016 National Institute of Health and Care Excellence (NICE) guideline-determined strategy; PROMISE (Prospective Multicenter Imaging Study for Evaluation of Chest Pain) minimal risk tool-based strategy; 2019 European Society of Cardiology (ESC) guideline-determined strategy; and coronary artery calcium score (CACS), either in isolation (the CACS strategy) or as part of a weighted clinical likelihood model-based strategy (the CACS-CL strategy). The associations of obstructive coronary artery disease on coronary computed tomography angiography, major adverse cardiovascular events, and subsequent clinical management with risk groups according to different strategies were evaluated and compared.

RESULTS

The NICE, PROMISE, ESC, CACS, and CACS-CL strategies classified a proportion (22.63%, 29.21%, 41.84%, 46.76%, and 51.41%, respectively) of patients into low-risk groups. Compared with the NICE, PROMISE, ESC, and CACS strategies, the CACS-CL strategy had a stronger association between risk groups and obstructive coronary artery disease (odd ratios: 16.00 vs 2.93, 5.53, 7.94, and 10.39, respectively), major adverse cardiovascular events (HRs: 6.83 vs 1.90, 2.94, 4.23, and 5.13, respectively) and intensive subsequent clinical management as well as better metrics of diagnostic accuracy and positive net reclassification improvement.

CONCLUSIONS

Among contemporary strategies used to identify patients with stable chest pain at low risk, the use of CACS, especially when combined with clinical risk features, showed the strongest potential to effectively defer cardiac testing.

Evaluation of Early Response to Preoperative Accelerated Partial Breast Irradiation (PAPBI) by Histopathology, Magnetic Resonance Imaging, and 18F-fluorodexoyglucose Positron Emission Tomography/Computed Tomography (FDG PET/CT)

PURPOSE

This study aimed to find indicators for early response to radiation therapy in breast cancer. These would be of help in tailoring treatment for individual patients.

METHODS AND MATERIALS

We analyzed 66 patients with low-risk breast cancer (≥60 years; cT1-2pN0) treated within the Preoperative Accelerated Partial Breast Irradiation (PAPBI) trial. Patients received radiation therapy (RT; 10 x 4 Gray or 5 x 6 Gray), followed by a wide local excision after 6 weeks. Patients underwent magnetic resonance imaging (MRI) and 18F-fluorodexoyglucose (FDG) positron emission tomography/computed tomography (PET/CT) before RT and 5 weeks after RT, before surgery. We assessed the response to PAPBI using a histopathologic assessment and correlated this with responses on MRI and FDG PET/CT. We calculated the positive predictive values (PPVs) of MRI and PET/CT as the number of true positives (complete response on MRI/normalized at visual evaluation on PET/CT and pathologic complete response) divided by the number of patients with a complete response on MRI/normalized at visual evaluation on PET/CT. Similarly, the negative predictive values (NPVs) of MRI and PET/CT were calculated.

RESULTS

The pathologic response was (nearly) complete in 15 (23%) of the 66 patients and partially complete in 28 (42%). The remaining 23 patients (35%) were nonresponders. The PPV of MRI (Response evaluation criteria in solid tumors [RECIST]) was 87.5% and the NPV was 85%. The PPV and NPV of PET/CT were 25% and 92%, respectively.

CONCLUSIONS

The most accurate method to predict a response and residual disease after preoperative RT in low-risk breast cancer was MRI, using RECIST.

An Automated Correction Algorithm (ALPACA) for ddPCR Data Using Adaptive Limit of Blank and Correction of False Positive Events Improves Specificity of Mutation Detection

BACKGROUND

Bio-Rad droplet-digital PCR is a highly sensitive method that can be used to detect tumor mutations in circulating cell-free DNA (cfDNA) of patients with cancer. Correct interpretation of ddPCR results is important for optimal sensitivity and specificity. Despite its widespread use, no standardized method to interpret ddPCR data is available, nor have technical artifacts affecting ddPCR results been widely studied.

METHODS

False positive rates were determined for 6 ddPCR assays at variable amounts of input DNA, revealing polymerase induced false positive events (PIFs) and other false positives. An in silico correction algorithm, known as the adaptive LoB and PIFs: an automated correction algorithm (ALPACA), was developed to remove PIFs and apply an adaptive limit of blank (LoB) to individual samples. Performance of ALPACA was compared to a standard strategy (no PIF correction and static LoB = 3) using data from commercial reference DNA, healthy volunteer cfDNA, and cfDNA from a real-life cohort of 209 patients with stage IV nonsmall cell lung cancer (NSCLC) whose tumor and cfDNA had been molecularly profiled.

RESULTS

Applying ALPACA reduced false positive results in healthy cfDNA compared to the standard strategy (specificity 98 vs 88%, P = 10-5) and stage IV NSCLC patient cfDNA (99 vs 93%, P = 10-11), while not affecting sensitivity in commercial reference DNA (70 vs 68% P = 0.77) or patient cfDNA (82 vs 88%, P = 0.13). Overall accuracy in patient samples was improved (98 vs 92%, P = 10-7).

CONCLUSIONS

Correction of PIFs and application of an adaptive LoB increases specificity without a loss of sensitivity in ddPCR, leading to a higher accuracy in a real-life cohort of patients with stage IV NSCLC.