Localization of liver lesions in abdominal CT imaging: II. Mathematical model observer performance correlates with human observer performance for localization of liver lesions in abdominal CT imaging

Low-contrast detectability of photon-counting-detector CT at different scan modes and image types in comparison with energy-integrating-detector CT

Purpose

We aim to compare the low-contrast detectability of a clinical whole-body photon-counting-detector (PCD)-CT at different scan modes and image types with an energy-integrating-detector (EID)-CT.

Approach

We used a channelized Hotelling observer (CHO) previously optimized for quality control purposes. An American College of Radiology CT accreditation phantom was scanned on both PCD-CT and EID-CT with 10 phantom positionings. For PCD-CT, images were generated using two scan modes, standard resolution (SR) and ultra-high-resolution (UHR); two image types, virtual monochromatic images at 70 keV and low-energy threshold (T3D); both filtered-back-projection (FBP) and iterative reconstruction (IR) reconstruction methods; and three reconstruction kernels. For each positioning, three repeated scans were acquired for each scan mode, image type, and CTDIvol of 6, 12, and 24 mGy. For EID-CT, images acquired from scans (10 positionings × 3 repeats × 3 doses) were reconstructed using the closest counterpart FBP and IR kernels. CHO was applied to calculate the index of detectability (d ' ) on both scanners.

Results

With the smooth Br44 kernel, the d ' of UHR was mostly comparable with that of the SR mode (difference: -11.4% to 8.3%, p = 0.020 to 0.956), and the T3D images had a higher d ' (difference: 0.7% to 25.6%) than 70 keV images on PCD-CT. Compared with the EID-CT, UHR-T3D of PCD-CT had non-inferior d ' (difference: -2.7% to 12.9%) with IR and non-superior d ' (difference: 0.8% to 11.2%) with FBP using the Br44 kernel. PCD-CT produced higher d ' than EID-CT by 61.8% to 247.1% with the sharper reconstruction kernels.

Conclusions

The comparison between PCD-CT and EID-CT was significantly influenced by the reconstruction method and kernel. With a smooth kernel that is typically used in low-contrast detection tasks, the PCD-CT demonstrated low-contrast detectability that was comparable to EID-CT with IR and showed no superiority when using FBP. With the use of sharper kernels, the PCD-CT significantly outperformed EID-CT in low-contrast detectability.

Channelized hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom: Part II. Repeatability study

BACKGROUND

Objective and quantitative evaluation for low-contrast detectability that correlates with human observer performance is lacking for routine CT quality control testing. Channelized Hotelling observer (CHO) is considered a strong candidate to fill the need but has long been deemed impractical to implement due to its requirement of a large number of repeated scans in order to provide accurate and precise estimates of index of detectability (d'). In our previous work, we optimized a CHO model observer on the American College of Radiology (ACR) CT accreditation phantom and achieved accurate measurement of d' with only 1-3 repeat scans.

PURPOSE

In this work, we aim to validate the repeatability of the proposed CHO-based low-contrast evaluation on four scanner models using the ACR CT accreditation phantom.

METHODS

The repeatability test was performed on four different scanners from two major CT manufacturers: Siemens Force and Alpha; Canon Prism and Prime SP. An ACR CT phantom was scanned 10 times, each time after repositioning of the phantom. For each repositioning, 3 repeated scans were acquired at 24, 12, and 6 mGy on all four scanner models. CHO was applied at the measured dose levels for different low-contrast object sizes (4-6 mm). The CHO was also applied to images created using deep learning-based reconstructions on Canon Prism and to four different scan/reconstruction modes on the Siemens Alpha, a photon-counting-detector (PCD)-CT. The repeatability was evaluated by the probability that a measurement would fall within the ±15% tolerance (P<15% ).

RESULTS

With the CHO setting optimized for the ACR phantom and the use of 3 repeated scans and 9 non-overlapping slices per scan, the CHO measurement could provide high repeatability with P<15% of 98.8%-99.9% at 12 mGy with IR reconstruction on all four scanners. On scanner A, P<15% were 91.5%-99.9% at the three dose levels and for all three object sizes while the numbers were 93.6%-99.998% on scanner B. P<15% were 96.5%-97.2% for the two deep learning reconstructions and 97.0%-99.97% for the four scan/reconstruction modes on the PCD-CT.

CONCLUSION

The CHO provided highly repeatable measurements with over 95% probability that a CHO measurement would lie within the ±15% tolerance for most of the dose levels and object sizes on the ACR phantom. The repeatability was maintained when the CHO was applied to images created with a commercial deep learning-based reconstruction and various scan/reconstruction modes on a PCD-CT. This study demonstrates that practical implementation of CHO for routine quality control and performance evaluation is feasible.

Accurate and efficient measurement of channelized Hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom

BACKGROUND

Current CT quality control (QC) for low-contrast detectability relies on visual inspection and measurement of contrast-to-noise ratio (CNR). However, CNR numbers become unreliable when it comes to nonlinear methods, such as iterative reconstruction (IR) and deep-learning-based techniques. Image quality metrics using channelized Hotelling observer (CHO) have been validated to be well correlated with human observer performance on phantom-based and patient-based tasks, but it has not been widely used in routine CT QC mainly because the CHO calculation typically requires a large number of repeated scans in order to provide accurate and precise estimate of index of detectability (d').

PURPOSE

The main goal of this work is to optimize channel filters and other CHO parameters and accurately estimate the low-contrast detectability with minimum number of repeated scans for the widely used American College of Radiology (ACR) CT accreditation phantom so that it can become practically feasible for routine CT QC tests.

METHODS

To provide a converged d' value, an ACR phantom was repeatedly scanned 100 times at three dose levels (24, 12, and 6 mGy). Images were reconstructed with two kernels (FBP Br44 and IR Br44-3). d' as a function of number of repeated scans was determined for different number of background regions of interest (ROIs), different number of low-contrast objects, different number of slices per each object, and different channel filter options. A reference d' was established using the optimized CHO setting, and the bias of d' was quantified using the d' calculated from all 100 repeated scans. The variation of d' at each condition was estimated using a resampling method combining random subsampling among 100 repeated scans and bootstrapping of the ensembles of signal and background ROIs.

RESULTS

Optimized parameters in CHO calculation were determined: two background ROIs per object, four objects per low-contrast object size, nine non-overlapping slices per object, and a 4-channel Gabor filter. The bias and uncertainty were estimated at different numbers of repeated scans using these parameters. When only one single scan was used in the CHO calculation, the bias of d' was below 6.2% and the uncertainty 15.6-19.6% for the 6, 5, and 4 mm objects, while with three repeated scans the bias was below 2.0% and uncertainty 8.7-10.9% for the three object sizes.

CONCLUSION

With optimized parameter settings in CHO, efficient and accurate measurement of low-contrast detectability on the commonly used ACR phantom becomes feasible, which could potentially lead to adoption of CHO-based low-contrast evaluation in routine QC tests.

Deep-learning model observer for a low-contrast hepatic metastases localization task in computed tomography

PURPOSE

Conventional model observers (MO) in CT are often limited to a uniform background or varying background that is random and can be modeled in an analytical form. It is unclear if these conventional MOs can be readily generalized to predict human observer performance in clinical CT tasks that involve realistic anatomical background. Deep-learning-based model observers (DL-MO) have recently been developed, but have not been validated for challenging low contrast diagnostic tasks in abdominal CT. We consequently sought to validate a DL-MO for a low-contrast hepatic metastases localization task.

METHODS

We adapted our recently developed DL-MO framework for the liver metastases localization task. Our previously-validated projection-domain lesion-/noise-insertion techniques were used to synthesize realistic positive and low-dose abdominal CT exams, using the archived patient projection data. Ten experimental conditions were generated, which involved different lesion sizes/contrasts, radiation dose levels, and image reconstruction types. Each condition included 100 trials generated from a patient cohort of 7 cases. Each trial was presented as liver image patches (160×160×5 voxels). The DL-MO performance was calculated for each condition and was compared with human observer performance, which was obtained by three sub-specialized radiologists in an observer study. The performance of DL-MO and radiologists was gauged by the area under localization receiver-operating-characteristic curves. The generalization performance of the DL-MO was estimated with the repeated twofold cross-validation method over the same set of trials used in the human observer study. A multi-slice Channelized Hoteling Observers (CHO) was compared with the DL-MO across the same experimental conditions.

RESULTS

The performance of DL-MO was highly correlated to that of radiologists (Pearson's correlation coefficient: 0.987; 95% CI: [0.942, 0.997]). The performance level of DL-MO was comparable to that of the grouped radiologists, that is, the mean performance difference was -3.3%. The CHO performance was poorer than the grouped radiologist performance, before internal noise could be added. The correlation between CHO and radiologists was weaker (Pearson's correlation coefficient: 0.812, and 95% CI: [0.378, 0.955]), and the corresponding performance bias (-29.5%) was statistically significant.

CONCLUSION

The presented study demonstrated the potential of using the DL-MO for image quality assessment in patient abdominal CT tasks.

Image texture, low contrast liver lesion detectability and impact on dose: Deep learning algorithm compared to partial model-based iterative reconstruction

OBJECTIVES

To compare deep learning (True Fidelity, TF) and partial model based Iterative Reconstruction (ASiR-V) algorithm for image texture, low contrast lesion detectability and potential dose reduction.

METHODS

Anthropomorphic phantoms (mimicking non-overweight and overweight patient), containing lesions of 6 mm in diameter with 20HU contrast, were scanned at five different dose levels (2,6,10,15,20 mGy) on a CT system, using clinical routine protocols for liver lesion detection. Images were reconstructed using ASiR-V 0% (surrogate for FBP), 60 % and TF at low, medium and high strength. Noise texture was characterized by computing a normalized Noise Power Spectrum filtered by an eye filter. The similarity against FBP texture was evaluated using peak frequency difference (PFD) and root mean square deviation (RMSD). Low contrast detectability was assessed using a channelized Hotelling observer and the area under the ROC curve (AUC) was used as figure of merit. Potential dose reduction was calculated to obtain the same AUC for TF and ASiR-V.

RESULTS

FBP-like noise texture was more preserved with TF (PFD from -0.043mm-1 to -0.09mm-1, RMSD from 0.12mm-1 to 0.21mm-1) than with ASiR-V (PFD equal to 0.12 mm-1, RMSD equal to 0.53mm-1), resulting in a sharper image. AUC was always higher with TF than ASIR-V. In average, TF compared to ASiR-V, enabled a radiation dose reduction potential of 7%, 25 % and 33 % for low, medium and high strength respectively.

CONCLUSION

Compared to ASIR-V, TF at high strength does not impact noise texture and maintains low contrast liver lesions detectability at significant lower dose.

Toward a comparison and an optimization of CT protocols using new metrics of dose and image quality part I: prediction of human observers using a model observer for detection and discrimination tasks in low-dose CT images in various scanning conditions

In the context of reducing the patient dose coming from CT scanner examinations without penalizing the diagnosis, the assessment of both patient dose and image quality (IQ) with relevant metrics is crucial. The present study represents the first stage in a larger work, aiming to compare and optimize CT protocols using dose and IQ new metrics. We proposed here to evaluate the capacity of the Non-PreWhitening matched filter with an eye (NPWE) model observer to be a robust and accurate estimation of IQ. We focused our work on two types of clinical tasks: a low contrast detection task and a discrimination task. We designed a torso-shaped phantom, including Plastic Water^®slabs with cylindrical inserts of different diameters, sections and compositions. We led a human observer study with 13 human observers on images acquired in multiple irradiation and reconstruction scanning conditions (voltage, pitch, slice thickness, noise level of the reconstruction algorithm, energy level in dual-energy mode and dose), to evaluate the behavior of the model observer compared to the human responses faced to changing conditions. The model observer presented the same trends as the human observers with generally better results. We rescaled the NPWE model on the human responses by scanning conditions (kVp, pitch, slice thickness) to obtain the best agreement between both observer types, estimated using the Bland-Altman method. The impact of some scanning parameters was estimated using the correct answer rate given by the rescaled NPWE model, for both tasks and each insert size. In particular, the comparison between the dual-energy mode at 74 keV and the single-energy mode at 120 kVp showed that, if the 120 kVp voltage provided better results for the smallest insert at the lower doses for both tasks, their responses were equivalent in many cases.

A Web-Based Software Platform for Efficient and Quantitative CT Image Quality Assessment and Protocol Optimization

Channelized Hotelling observer (CHO), which has been shown to be well correlated with human observer performance in many clinical CT tasks, has a great potential to become the method of choice for objective image quality assessment. However, the use of CHO in clinical CT is still quite limited, mainly due to its complexity in measurement and calculation in practice, and the lack of access to an efficient and validated software tool for most clinical users. In this work, a web-based software platform for CT image quality assessment and protocol optimization (CTPro) was introduced. A validated CHO tool, along with other common image quality assessment tools, was made readily accessible through this web platform for clinical users and researchers without the need of installing additional software. An example of its application to evaluation of convolutional-neural-network (CNN)-based denoising was demonstrated.

Low-dose CT image and projection dataset

PURPOSE

To describe a large, publicly available dataset comprising computed tomography (CT) projection data from patient exams, both at routine clinical doses and simulated lower doses.

ACQUISITION AND VALIDATION METHODS

The library was developed under local ethics committee approval. Projection and image data from 299 clinically performed patient CT exams were archived for three types of clinical exams: noncontrast head CT scans acquired for acute cognitive or motor deficit, low-dose noncontrast chest scans acquired to screen high-risk patients for pulmonary nodules, and contrast-enhanced CT scans of the abdomen acquired to look for metastatic liver lesions. Scans were performed on CT systems from two different CT manufacturers using routine clinical protocols. Projection data were validated by reconstructing the data using several different reconstruction algorithms and through use of the data in the 2016 Low Dose CT Grand Challenge. Reduced dose projection data were simulated for each scan using a validated noise-insertion method. Radiologists marked location and diagnosis for detected pathologies. Reference truth was obtained from the patient medical record, either from histology or subsequent imaging.

DATA FORMAT AND USAGE NOTES

Projection datasets were converted into the previously developed DICOM-CT-PD format, which is an extended DICOM format created to store CT projections and acquisition geometry in a nonproprietary format. Image data are stored in the standard DICOM image format and clinical data in a spreadsheet. Materials are provided to help investigators use the DICOM-CT-PD files, including a dictionary file, data reader, and user manual. The library is publicly available from The Cancer Imaging Archive (https://doi.org/10.7937/9npb-2637).

POTENTIAL APPLICATIONS

This CT data library will facilitate the development and validation of new CT reconstruction and/or denoising algorithms, including those associated with machine learning or artificial intelligence. The provided clinical information allows evaluation of task-based diagnostic performance.

Observer Performance for Detection of Pulmonary Nodules at Chest CT over a Large Range of Radiation Dose Levels

Background There is a wide variation in radiation dose levels that can be used with chest CT in order to detect indeterminate pulmonary nodules. Purpose To compare the performance of lower-radiation-dose chest CT with that of routine dose in the detection of indeterminate pulmonary nodules 5 mm or greater. Materials and Methods In this retrospective study, CT projection data from 83 routine-dose chest CT examinations performed in 83 patients (120 kV, 70 quality reference mAs [QRM]) were collected between November 2013 and April 2014. Reference indeterminate pulmonary nodules were identified by two nonreader thoracic radiologists. By using validated noise insertion, five lower-dose data sets were reconstructed with filtered back projection (FBP) or iterative reconstruction (IR; 30 QRM with FBP, 10 QRM with IR, 5 QRM with FBP, 5 QRM with IR, and 2.5 QRM with IR). Three thoracic radiologists circled pulmonary nodules, rating confidence that the nodule was a 5-mm-or-greater indeterminate pulmonary nodule, and graded image quality. Analysis was performed on a per-nodule basis by using jackknife alternative free-response receiver operating characteristic figure of merit (FOM) and noninferiority limit of -0.10. Results There were 66 indeterminate pulmonary nodules (mean size, 8.6 mm ± 3.4 [standard deviation]; 21 part-solid nodules) in 42 patients (mean age, 51 years ± 17; 21 men and 21 women). Compared with the FOM for routine-dose CT (size-specific dose estimate, 6.5 mGy ± 1.8; FOM, 0.86 [95% confidence interval: 0.80, 0.91]), FOM was noninferior for all lower-dose configurations except for 2.5 QRM with IR. The sensitivity for subsolid nodules at 70 QRM was 60% (range, 48%-72%) and was significantly worse at a dose of 5 QRM and lower, whether or not IR was used (P < .05). Diagnostic image quality decreased with decreasing dose (P < .001) and was better with IR at 5 QRM (P < .05). Conclusion CT images reconstructed at dose levels down to 10 quality reference mAs (size-specific dose estimate, 0.9 mGy) had noninferior performance compared with routine dose in depicting pulmonary nodules. Iterative reconstruction improved subjective image quality but not performance at low dose levels. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by White and Kazerooni in this issue.