A virtual clinical trial using projection-based nodule insertion to determine radiologist reader performance in lung cancer screening CT

Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance

We present a method to generate synthetic thorax radiographs with realistic nodules from CT scans, and a perfect ground truth knowledge. We evaluated the detection performance of nine radiologists and two convolutional neural networks in a reader study. Nodules were artificially inserted into the lung of a CT volume and synthetic radiographs were obtained by forward-projecting the volume. Hence, our framework allowed for a detailed evaluation of CAD systems' and radiologists' performance due to the availability of accurate ground-truth labels for nodules from synthetic data. Radiographs for network training (U-Net and RetinaNet) were generated from 855 CT scans of a public dataset. For the reader study, 201 radiographs were generated from 21 nodule-free CT scans with altering nodule positions, sizes and nodule counts of inserted nodules. Average true positive detections by nine radiologists were 248.8 nodules, 51.7 false positive predicted nodules and 121.2 false negative predicted nodules. The best performing CAD system achieved 268 true positives, 66 false positives and 102 false negatives. Corresponding weighted alternative free response operating characteristic figure-of-merits (wAFROC FOM) for the radiologists range from 0.54 to 0.87 compared to a value of 0.81 (CI 0.75-0.87) for the best performing CNN. The CNN did not perform significantly better against the combined average of the 9 readers (p = 0.49). Paramediastinal nodules accounted for most false positive and false negative detections by readers, which can be explained by the presence of more tissue in this area.

Deep-learning lesion and noise insertion for virtual clinical trial in Chest CT

Accurate and objective image quality assessment is essential for the task of radiation dose optimization in clinical CT. Standard method relies on multi-reader multi-case (MRMC) studies in which radiologists are tasked to interpret diagnostic image quality of many carefully-collected positive and negative cases. The efficiency of such MRMC studies is frequently challenged by the lengthy and expensive procedure of case collection and the establishment of clinical reference standard. To address this challenge, multiple methods of virtual clinical trial to synthesize patient cases at different conditions have been proposed. Projection-domain lesion- / noise-insertion methods require the access to patient raw data and vendor-specific proprietary tools which are frequently not accessible to most users. The conventional image-domain noise-insertion methods are often challenged by the over-simplified lesion models and CT system models which may not represent conditions in real scans. In this work, we developed deep-learning lesion and noise insertion techniques that can synthesize chest CT images at different dose levels with and without lung nodules using existing patient cases. The proposed method involved a nodule-insertion convolutional neural network (CNN) and a noise-insertion CNN. Both CNNs demonstrated comparable quality to our previously-validated projection domain lesion- / noise-insertion techniques: mean structural similarity index (SSIM) of inserted nodules 0.94 (routine dose), and mean percent noise difference ~5% (50% of routine dose). The proposed deep-learning techniques for chest CT virtual clinical trial operate directly on image domain, which is more widely applicable than projection-domain techniques.

Deep-learning-based model observer for a lung nodule detection task in computed tomography

Purpose: Task-based image quality assessment using model observers (MOs) is an effective approach to radiation dose and scanning protocol optimization in computed tomography (CT) imaging, once the correlation between MOs and radiologists can be established in well-defined clinically relevant tasks. Conventional MO studies were typically simplified to detection, classification, or localization tasks using tissue-mimicking phantoms, as traditional MOs cannot be readily used in complex anatomical background. However, anatomical variability can affect human diagnostic performance. Approach: To address this challenge, we developed a deep-learning-based MO (DL-MO) for localization tasks and validated in a lung nodule detection task, using previously validated projection-based lesion-/noise-insertion techniques. The DL-MO performance was compared with 4 radiologist readers over 12 experimental conditions, involving varying radiation dose levels, nodule sizes, nodule types, and reconstruction types. Each condition consisted of 100 trials (i.e., 30 images per trial) generated from a patient cohort of 50 cases. DL-MO was trained using small image volume-of-interests extracted across the entire volume of training cases. For each testing trial, the nodule searching of DL-MO was confined to a 3-mm thick volume to improve computational efficiency, and radiologist readers were tasked to review the entire volume. Results: A strong correlation between DL-MO and human readers was observed (Pearson's correlation coefficient: 0.980 with a 95% confidence interval of [0.924, 0.994]). The averaged performance bias between DL-MO and human readers was 0.57%. Conclusion: The experimental results indicated the potential of using the proposed DL-MO for diagnostic image quality assessment in realistic chest CT tasks.

Local complexity metrics to quantify the effect of anatomical noise on detectability of lung nodules in chest CT imaging

The purpose of this study is to (1) develop metrics to characterize the regional anatomical complexity of the lungs, and (2) relate these metrics with lung nodule detection in chest CT. A free-scrolling reader-study with virtually inserted nodules (13 radiologists × 157 total nodules = 2041 responses) is used to characterize human detection performance. Metrics of complexity based on the local density and orientation of distracting vasculature are developed for two-dimensional (2-D) and three-dimensional (3-D) considerations of the image volume. Assessed characteristics included the distribution of 2-D/3-D vessel structures of differing orientation (dubbed "2-D/3-D and dot-like/line-like distractor indices"), contiguity of inserted nodules with local vasculature, mean local gray-level surrounding each nodule, the proportion of lung voxels to total voxels in each section, and 3-D distance of each nodule from the trachea bifurcation. A generalized linear mixed-effects statistical model is used to determine the influence of each these metrics on nodule detectability. In order of decreasing effect size: 3-D line-like distractor index, 2-D line-like distractor index, 2-D dot-like distractor index, local mean gray-level, contiguity with 2-D dots, lung area, and contiguity with 3-D lines all significantly affect detectability ( P < 0.05 ). These data demonstrate that local lung complexity degrades detection of lung nodules.