Suboptimal Quality and High Risk of Bias in Diagnostic Test Accuracy Studies at Chest Radiography and CT in the Acute Setting of the COVID-19 Pandemic: A Systematic Review

An Umbrella Review With Meta-Analysis of Chest Computed Tomography for Diagnosis of COVID-19: Considerations for Trauma Patient Management

Background

RT-PCR testing is the standard for diagnosis of COVID-19, although it has its suboptimal sensitivity. Chest computed tomography (CT) has been proposed as an additional tool with diagnostic value, and several reports from primary and secondary studies that assessed its diagnostic accuracy are already available. To inform recommendations and practice regarding the use of chest CT in the in the trauma setting, we sought to identify, appraise, and summarize the available evidence on the diagnostic accuracy of chest CT for diagnosis of COVID-19, and its application in emergency trauma surgery patients; overcoming limitations of previous reports regarding chest CT accuracy and discussing important considerations regarding its role in this setting.

Methods

We conducted an umbrella review using Living Overview of Evidence platform for COVID-19, which performs regular automated searches in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and more than 30 other sources. The review was conducted following the JBI methodology for systematic reviews. The Grading of Recommendations, Assessment, Development, and Evaluation approach for grading the certainty of the evidence is reported (registered in International Prospective Register of Systematic Reviews, CRD42020198267).

Results

Thirty studies that fulfilled selection criteria were included; 19 primary studies provided estimates of sensitivity (0.91, 95%CI = [0.88–0.93]) and specificity (0.73, 95%CI = [0.61; 0.82]) of chest CT for COVID-19. No correlation was found between sensitivities and specificities (ρ = 0.22, IC95% [–0.33; 0.66]). Diagnostic odds ratio was estimated at: DOR = 27.5, 95%CI (14.7; 48.5). Evidence for sensitivity estimates was graded as MODERATE, and for specificity estimates it was graded as LOW.

Conclusion

The value of chest CT appears to be that of an additional screening tool that can easily detect PCR false negatives, which are reportedly highly frequent. Upon the absence of PCR testing and impossibility to perform RT-PCR in trauma patients, chest CT can serve as a substitute with increased value and easy implementation.

Systematic Review Registration

[www.crd.york.ac.uk/prospero], identifier [CRD42020198267].

Diagnostic performance of the RSNA-proposed classification for COVID-19 pneumonia versus pre-pandemic controls

Objective

To evaluate the diagnostic accuracy of the Radiological Society of North America (RSNA) classification system for coronavirus disease 2019 (COVID-19) pneumonia compared to pre-pandemic chest computed tomography (CT) scan images to mitigate the risk of bias regarding the reference standard.

Materials and methods

This was a retrospective, cross-sectional, diagnostic test accuracy study. Chest CT scans, carried out from May 1 to June 30, 2020, and from May 1 to July 17, 2017, were consecutively selected for the COVID-19 (positive reverse transcription-polymerase chain reaction [RT-PCR] for severe acute respiratory syndrome coronavirus 2 result) and control (pre-pandemic) groups, respectively. Four expert thoracic radiologists blindly interpreted each CT scan image. Sensitivity and specificity were calculated.

Results

A total of 160 chest CT scan images were included: 79 in the COVID-19 group (56 [43.5–67] years old, 41 men) and 81 in the control group (62 [52–72] years old, 44 men). Typically, an estimated specificity of 98.5% (95% confidence interval [CI] 98.1%–98.4%) was obtained. For the indeterminate classification as a diagnostic threshold, an estimated sensitivity of 88.3% (95% CI 84.7%–91.7%) and a specificity of 79.0% (95% CI 74.5%–83.4%), with an area under the curve of 0.865 (95% CI 0.838–0.895), were obtained.

Conclusion

The RSNA classification system shows strong diagnostic accuracy for COVID-19 pneumonia, even against pre-pandemic controls. It can be an important aid in clinical decision-making, especially when a typical or indeterminate pattern is found, possibly advising retesting following an initial negative RT-PCR result and streamlining early management and isolation.

Chest computed tomography in the diagnosis of COVID-19 in patients with false negative RT-PCR

Objective

To evaluate the role of chest computed tomography in patients with COVID-19 who presented initial negative result in reverse transcriptase-polymerase chain reaction (RT-PCR).

Methods

A single-center, retrospective study that evaluated 39 patients with negative RT-PCR for COVID-19, who underwent chest computed tomography and had a final clinical or serological diagnosis of COVID-19. The visual tomographic classification was evaluated according to the Consensus of the Radiological Society of North America and software developed with artificial intelligence for automatic detection of findings and chance estimation of COVID-19.

Results

In the visual tomographic analysis, only one of them (3%) presented computed tomography classified as negative, 69% were classified as typical and 28% as indeterminate. In the evaluation using the software, only four (about 10%) had a probability of COVID-19 <25%.

Conclusion

Computed tomography can play an important role in management of suspected cases of COVID-19 with initial negative results in RT-PCR, especially considering those patients outside the ideal window for sample collection for RT-PCR.

The role of chest imaging in the diagnosis, management, and monitoring of coronavirus disease 2019 (COVID-19)

Coronavirus disease 2019 (COVID-19) pandemic has posed a major public health crisis all over the world. The role of chest imaging, especially computed tomography (CT), has evolved during the pandemic paralleling the accumulation of scientific evidence. In the early stage of the pandemic, the performance of chest imaging for COVID-19 has widely been debated especially in the context of comparison to real-time reverse transcription polymerase chain reaction. Current evidence is against the use of chest imaging for routine screening of COVID-19 contrary to the initial expectations. It still has an integral role to play, however, in its work up and staging, especially when assessing complications or disease progression. Chest CT is gold standard imaging modality for COVID-19 pneumonia; in some situations, chest X-ray or ultrasound may be an effective alternative. The most important role of radiologists in this context is to be able to identify those patients at greatest risk of imminent clinical decompensation by learning to stratify cases of COVID-19 on the basis of radiologic imaging in the most efficient and timely fashion possible. The present availability of multiple and more refined CT grading systems and classification is now making this task easier and thereby contributing to the recent improvements achieved in COVID-19 treatment and outcomes. In this article, evidence of chest imaging regarding diagnosis, management and monitoring of COVID-19 will be chronologically reviewed.

The rapid, massive growth of COVID-19 authors in the scientific literature

We examined the extent to which the scientific workforce in different fields was engaged in publishing COVID-19-related papers. According to Scopus (data cut, 1 August 2021), 210 183 COVID-19-related publications included 720 801 unique authors, of which 360 005 authors had published at least five full papers in their career and 23 520 authors were at the top 2% of their scientific subfield based on a career-long composite citation indicator. The growth of COVID-19 authors was far more rapid and massive compared with cohorts of authors historically publishing on H1N1, Zika, Ebola, HIV/AIDS and tuberculosis. All 174 scientific subfields had some specialists who had published on COVID-19. In 109 of the 174 subfields of science, at least one in 10 active, influential (top 2% composite citation indicator) authors in the subfield had authored something on COVID-19. Fifty-three hyper-prolific authors had already at least 60 (and up to 227) COVID-19 publications each. Among the 300 authors with the highest composite citation indicator for their COVID-19 publications, most common countries were USA (n = 67), China (n = 52), UK (n = 32) and Italy (n = 18). The rapid and massive involvement of the scientific workforce in COVID-19-related work is unprecedented and creates opportunities and challenges. There is evidence for hyper-prolific productivity.

Highlighting COVID-19: What the imaging exams show about the disease

Coronavirus disease 2019 (COVID-19), a global emergency, is caused by severe acute respiratory syndrome coronavirus 2. The gold standard for its diagnosis is the reverse transcription polymerase chain reaction, but considering the high number of infected people, the low availability of this diagnostic tool in some contexts, and the limitations of the test, other tools that aid in the identification of the disease are necessary. In this scenario, imaging exams such as chest X-ray (CXR) and computed tomography (CT) have played important roles. CXR is useful for assessing disease progression because it allows the detection of extensive consolidations, besides being a fast and cheap method. On the other hand, CT is more sensitive for detecting lung changes in the early stages of the disease and is also useful for assessing disease progression. Of note, ground-glass opacities are the main COVID-19-related CT findings. Positron emission tomography combined with CT can be used to evaluate chronic and substantial damage to the lungs and other organs; however, it is an expensive test. Lung ultrasound (LUS) has been shown to be a promising technique in that context as well, being useful in the screening and monitoring of patients, disease classification, and management related to mechanical ventilation. Moreover, LUS is an inexpensive alternative available at the bedside. Finally, magnetic resonance imaging, although not usually requested, allows the detection of pulmonary, cardiovascular, and neurological abnormalities associated with COVID-19. Furthermore, it is important to consider the challenges faced in the radiology field in the adoption of control measures to prevent infection and in the follow-up of post-COVID-19 patients.

The RSNA International COVID-19 Open Radiology Database (RICORD)

The coronavirus disease 2019 (COVID-19) pandemic is a global health care emergency. Although reverse-transcription polymerase chain reaction testing is the reference standard method to identify patients with COVID-19 infection, chest radiography and CT play a vital role in the detection and management of these patients. Prediction models for COVID-19 imaging are rapidly being developed to support medical decision making. However, inadequate availability of a diverse annotated data set has limited the performance and generalizability of existing models. To address this unmet need, the RSNA and Society of Thoracic Radiology collaborated to develop the RSNA International COVID-19 Open Radiology Database (RICORD). This database is the first multi-institutional, multinational, expert-annotated COVID-19 imaging data set. It is made freely available to the machine learning community as a research and educational resource for COVID-19 chest imaging. Pixel-level volumetric segmentation with clinical annotations was performed by thoracic radiology subspecialists for all COVID-19-positive thoracic CT scans. The labeling schema was coordinated with other international consensus panels and COVID-19 data annotation efforts, the European Society of Medical Imaging Informatics, the American College of Radiology, and the American Association of Physicists in Medicine. Study-level COVID-19 classification labels for chest radiographs were annotated by three radiologists, with majority vote adjudication by board-certified radiologists. RICORD consists of 240 thoracic CT scans and 1000 chest radiographs contributed from four international sites. It is anticipated that RICORD will ideally lead to prediction models that can demonstrate sustained performance across populations and health care systems.

Diagnostic Performance of CO-RADS and the RSNA Classification System in Evaluating COVID-19 at Chest CT: A Meta-Analysis

PURPOSE

To determine the diagnostic performance of the COVID-19 Reporting and Data System (CO-RADS) and the Radiological Society of North America (RSNA) categorizations in patients with clinically suspected coronavirus disease 2019 (COVID-19) infection.

MATERIALS AND METHODS

In this meta-analysis, studies from 2020, up to August 24, 2020 were assessed for inclusion criteria of studies that used CO-RADS or the RSNA categories for scoring chest CT in patients with suspected COVID-19. A total of 186 studies were identified. After review of abstracts and text, a total of nine studies were included in this study. Patient information (n¸ age, sex), CO-RADS and RSNA scoring categories, and other study characteristics were extracted. Study quality was assessed with the QUADAS-2 tool. Meta-analysis was performed with a random effects model.

RESULTS

Nine studies (3283 patients) were included. Overall study quality was good, except for risk of non-performance of repeated reverse transcriptase polymerase chain reaction (RT-PCR) after negative initial RT-PCR and persistent clinical suspicion in four studies. Pooled COVID-19 frequencies in CO-RADS categories were: 1, 8.8%; 2, 11.1%; 3, 24.6%; 4, 61.9%; and 5, 89.6%. Pooled COVID-19 frequencies in RSNA classification categories were: negative 14.4%; atypical, 5.7%; indeterminate, 44.9%; and typical, 92.5%. Pooled pairs of sensitivity and specificity using CO-RADS thresholds were the following: at least 3, 92.5% (95% CI: 87.1, 95.7) and 69.2% (95%: CI: 60.8, 76.4); at least 4, 85.8% (95% CI: 78.7, 90.9) and 84.6% (95% CI: 79.5, 88.5); and 5, 70.4% (95% CI: 60.2, 78.9) and 93.1% (95% CI: 87.7, 96.2). Pooled pairs of sensitivity and specificity using RSNA classification thresholds for indeterminate were 90.2% (95% CI: 87.5, 92.3) and 75.1% (95% CI: 68.9, 80.4) and for typical were 65.2% (95% CI: 37.0, 85.7) and 94.9% (95% CI: 86.4, 98.2).

CONCLUSION

COVID-19 infection frequency was higher in patients categorized with higher CORADS and RSNA classification categories.