Comparison of chest X-ray interpretation by Emergency Department clinicians and radiologists in suspected COVID-19 infection: a retrospective cohort study

Chest radiograph classification and severity of suspected COVID-19 by different radiologist groups and attending clinicians: multi-reader, multi-case study

OBJECTIVES

To quantify reader agreement for the British Society of Thoracic Imaging (BSTI) diagnostic and severity classification for COVID-19 on chest radiographs (CXR), in particular agreement for an indeterminate CXR that could instigate CT imaging, from single and paired images.

METHODS

Twenty readers (four groups of five individuals)-consultant chest (CCR), general consultant (GCR), and specialist registrar (RSR) radiologists, and infectious diseases clinicians (IDR)-assigned BSTI categories and severity in addition to modified Covid-Radiographic Assessment of Lung Edema Score (Covid-RALES), to 305 CXRs (129 paired; 2 time points) from 176 guideline-defined COVID-19 patients. Percentage agreement with a consensus of two chest radiologists was calculated for (1) categorisation to those needing CT (indeterminate) versus those that did not (classic/probable, non-COVID-19); (2) severity; and (3) severity change on paired CXRs using the two scoring systems.

RESULTS

Agreement with consensus for the indeterminate category was low across all groups (28-37%). Agreement for other BSTI categories was highest for classic/probable for the other three reader groups (66-76%) compared to GCR (49%). Agreement for normal was similar across all radiologists (54-61%) but lower for IDR (31%). Agreement for a severe CXR was lower for GCR (65%), compared to the other three reader groups (84-95%). For all groups, agreement for changes across paired CXRs was modest.

CONCLUSION

Agreement for the indeterminate BSTI COVID-19 CXR category is low, and generally moderate for the other BSTI categories and for severity change, suggesting that the test, rather than readers, is limited in utility for both deciding disposition and serial monitoring.

KEY POINTS

• Across different reader groups, agreement for COVID-19 diagnostic categorisation on CXR varies widely. • Agreement varies to a degree that may render CXR alone ineffective for triage, especially for indeterminate cases. • Agreement for serial CXR change is moderate, limiting utility in guiding management.

The role of chest imaging in the diagnosis, management, and monitoring of coronavirus disease 2019 (COVID-19)

Coronavirus disease 2019 (COVID-19) pandemic has posed a major public health crisis all over the world. The role of chest imaging, especially computed tomography (CT), has evolved during the pandemic paralleling the accumulation of scientific evidence. In the early stage of the pandemic, the performance of chest imaging for COVID-19 has widely been debated especially in the context of comparison to real-time reverse transcription polymerase chain reaction. Current evidence is against the use of chest imaging for routine screening of COVID-19 contrary to the initial expectations. It still has an integral role to play, however, in its work up and staging, especially when assessing complications or disease progression. Chest CT is gold standard imaging modality for COVID-19 pneumonia; in some situations, chest X-ray or ultrasound may be an effective alternative. The most important role of radiologists in this context is to be able to identify those patients at greatest risk of imminent clinical decompensation by learning to stratify cases of COVID-19 on the basis of radiologic imaging in the most efficient and timely fashion possible. The present availability of multiple and more refined CT grading systems and classification is now making this task easier and thereby contributing to the recent improvements achieved in COVID-19 treatment and outcomes. In this article, evidence of chest imaging regarding diagnosis, management and monitoring of COVID-19 will be chronologically reviewed.

Diagnosis of COVID-19 in Patients with Negative Nasopharyngeal Swabs: Reliability of Radiological and Clinical Diagnosis and Accuracy Versus Serology

Background: The diagnosis of Coronavirus disease 2019 (COVID-19) relies on the positivity of nasopharyngeal swab. However, a significant percentage of symptomatic patients may test negative. We evaluated the reliability of COVID-19 diagnosis made by radiologists and clinicians and its accuracy versus serology in a sample of patients hospitalized for suspected COVID-19 with multiple negative swabs. Methods: Admission chest CT-scans and clinical records of swab-negative patients, treated according to the COVID-19 protocol or deceased during hospitalization, were retrospectively evaluated by two radiologists and two clinicians, respectively. Results: Of 254 patients, 169 swab-confirmed cases and one patient without chest CT-scan were excluded. A total of 84 patients were eligible for the reliability study. Of these, 21 patients died during hospitalization; the remaining 63 underwent serological testing and were eligible for the accuracy evaluation. Of the 63, 26 patients showed anti-Sars-Cov-2 antibodies, while 37 did not. The inter-rater agreement was “substantial” (kappa 0.683) between radiologists, “moderate” (kappa 0.454) between clinicians, and only “fair” (kappa 0.341) between radiologists and clinicians. Both radiologic and clinical evaluations showed good accuracy compared to serology. Conclusions: The radiologic and clinical diagnosis of COVID-19 for swab-negative patients proved to be sufficiently reliable and accurate to allow a diagnosis of COVID-19, which needs to be confirmed by serology and follow-up.