The “perfect” reader study

Clinical Super-Resolution Computed Tomography of Bone Microstructure: Application in Musculoskeletal and Dental Imaging

PURPOSE

Clinical cone-beam computed tomography (CBCT) devices are limited to imaging features of half a millimeter in size and cannot quantify the tissue microstructure. We demonstrate a robust deep-learning method for enhancing clinical CT images, only requiring a limited set of easy-to-acquire training data.

METHODS

Knee tissue from five cadavers and six total knee replacement patients, and 14 teeth from eight patients were scanned using laboratory CT as training data for the developed super-resolution (SR) technique. The method was benchmarked against ex vivo test set, 52 osteochondral samples are imaged with clinical and laboratory CT. A quality assurance phantom was imaged with clinical CT to quantify the technical image quality. To visually assess the clinical image quality, musculoskeletal and maxillofacial CBCT studies were enhanced with SR and contrasted to interpolated images. A dental radiologist and surgeon reviewed the maxillofacial images.

RESULTS

The SR models predicted the bone morphological parameters on the ex vivo test set more accurately than conventional image processing. The phantom analysis confirmed higher spatial resolution on the SR images than interpolation, but image grayscales were modified. Musculoskeletal and maxillofacial CBCT images showed more details on SR than interpolation; however, artifacts were observed near the crown of the teeth. The readers assessed mediocre overall scores for both SR and interpolation. The source code and pretrained networks are publicly available.

CONCLUSION

Model training with laboratory modalities could push the resolution limit beyond state-of-the-art clinical musculoskeletal and dental CBCT. A larger maxillofacial training dataset is recommended for dental applications.

Feasibility of intra-operative image guidance in burn excision surgery with multispectral imaging and deep learning

BACKGROUND

Exposing a healthy wound bed for skin grafting is an important step during burn surgery to ensure graft take and maintain good functional outcomes. Currently, the removal of non-viable tissue in the burn wound bed during excision is determined by expert clinician judgment. Using a porcine model of tangential burn excision, we investigated the effectiveness of an intraoperative multispectral imaging device combined with artificial intelligence to aid clinician judgment for the excision of non-viable tissue.

METHODS

Multispectral imaging data was obtained from serial tangential excisions of thermal burn injuries and used to train a deep learning algorithm to identify the presence and location of non-viable tissue in the wound bed. Following algorithm development, we studied the ability of two surgeons to estimate wound bed viability, both unaided and aided by the imaging device.

RESULTS

The deep learning algorithm was 87% accurate in identifying the viability of a burn wound bed. When paired with the surgeons, this device significantly improved their abilities to determine the viability of the wound bed by 25% (p = 0.03). Each time a surgeon changed their decision after seeing the AI model output, it was always a change from an incorrect decision to excise more tissue to a correct decision to stop excision.

CONCLUSION

This study provides insight into the feasibility of image-guided burn excision, its effect on surgeon decision making, and suggests further investigation of a real-time imaging system for burn surgery could reduce over-excision of burn wounds.

Development of an artificial intelligence bacteremia prediction model and evaluation of its impact on physician predictions focusing on uncertainty

Prediction of bacteremia is a clinically important but challenging task. An artificial intelligence (AI) model has the potential to facilitate early bacteremia prediction, aiding emergency department (ED) physicians in making timely decisions and reducing unnecessary medical costs. In this study, we developed and externally validated a Bayesian neural network-based AI bacteremia prediction model (AI-BPM). We also evaluated its impact on physician predictive performance considering both AI and physician uncertainties using historical patient data. A retrospective cohort of 15,362 adult patients with blood cultures performed in the ED was used to develop the AI-BPM. The AI-BPM used structured and unstructured text data acquired during the early stage of ED visit, and provided both the point estimate and 95% confidence interval (CI) of its predictions. High AI-BPM uncertainty was defined as when the predetermined bacteremia risk threshold (5%) was included in the 95% CI of the AI-BPM prediction, and low AI-BPM uncertainty was when it was not included. In the temporal validation dataset (N = 8,188), the AI-BPM achieved area under the receiver operating characteristic curve (AUC) of 0.754 (95% CI 0.737-0.771), sensitivity of 0.917 (95% CI 0.897-0.934), and specificity of 0.340 (95% CI 0.330-0.351). In the external validation dataset (N = 7,029), the AI-BPM's AUC was 0.738 (95% CI 0.722-0.755), sensitivity was 0.927 (95% CI 0.909-0.942), and specificity was 0.319 (95% CI 0.307-0.330). The AUC of the post-AI physicians predictions (0.703, 95% CI 0.654-0.753) was significantly improved compared with that of the pre-AI predictions (0.639, 95% CI 0.585-0.693; p-value < 0.001) in the sampled dataset (N = 1,000). The AI-BPM especially improved the predictive performance of physicians in cases with high physician uncertainty (low subjective confidence) and low AI-BPM uncertainty. Our results suggest that the uncertainty of both the AI model and physicians should be considered for successful AI model implementation.

Assessment of an artificial intelligence aid for the detection of appendicular skeletal fractures in children and young adults by senior and junior radiologists

BACKGROUND

As the number of conventional radiographic examinations in pediatric emergency departments increases, so, too, does the number of reading errors by radiologists.

OBJECTIVE

The aim of this study is to investigate the ability of artificial intelligence (AI) to improve the detection of fractures by radiologists in children and young adults.

MATERIALS AND METHODS

A cohort of 300 anonymized radiographs performed for the detection of appendicular fractures in patients ages 2 to 21 years was collected retrospectively. The ground truth for each examination was established after an independent review by two radiologists with expertise in musculoskeletal imaging. Discrepancies were resolved by consensus with a third radiologist. Half of the 300 examinations showed at least 1 fracture. Radiographs were read by three senior pediatric radiologists and five radiology residents in the usual manner and then read again immediately after with the help of AI.

RESULTS

The mean sensitivity for all groups was 73.3% (110/150) without AI; it increased significantly by almost 10% (P<0.001) to 82.8% (125/150) with AI. For junior radiologists, it increased by 10.3% (P<0.001) and for senior radiologists by 8.2% (P=0.08). On average, there was no significant change in specificity (from 89.6% to 90.3% [+0.7%, P=0.28]); for junior radiologists, specificity increased from 86.2% to 87.6% (+1.4%, P=0.42) and for senior radiologists, it decreased from 95.1% to 94.9% (-0.2%, P=0.23). The stand-alone sensitivity and specificity of the AI were, respectively, 91% and 90%.

CONCLUSION

With the help of AI, sensitivity increased by an average of 10% without significantly decreasing specificity in fracture detection in a predominantly pediatric population.

The Role of Machine Learning in Cardiovascular Pathology

Machine learning has seen slow but steady uptake in diagnostic pathology over the past decade to assess digital whole-slide images. Machine learning tools have incredible potential to standardise, and likely even improve, histopathologic diagnoses, but they are not yet widely used in clinical practice. We describe the principles of these tools and technologies and some successful preclinical and pretranslational efforts in cardiovascular pathology, as well as a roadmap for moving forward. In nonhuman animal models, one proof-of-principle application is in rodent progressive cardiomyopathy, which is of particular significance to drug toxicity studies. Basic science successes include screening the quality of differentiated stem cells and characterising cardiomyocyte developmental stages, with potential applications for research and toxicology/drug safety screening using derived or native human pluripotent stem cells differentiated into cardiomyocytes. Translational studies of particular note include those with success in diagnosing the various forms of heart allograft rejection. For fully realising the value of these tools in clinical cardiovascular pathology, we identify 3 essential challenges. First is image quality standardisation to ensure that algorithms can be developed and implemented on robust, consistent data. The second is consensus diagnosis; experts don't always agree, and thus "truth" may be difficult to establish, but the algorithms themselves may provide a solution. The third is the need for large-enough data sets to facilitate robust algorithm development, necessitating large cross-institutional shared image databases. The power of histopathology-based machine learning technologies is tremendous, and we outline the next steps needed to capitalise on this power.

Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study

BACKGROUND

Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model.

METHODS

In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior.

FINDINGS

Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings.

INTERPRETATION

This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice.

FUNDING

Annalise.ai.

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists

BACKGROUND

Artificial intelligence (AI) systems performing at radiologist-like levels in the evaluation of digital mammography (DM) would improve breast cancer screening accuracy and efficiency. We aimed to compare the stand-alone performance of an AI system to that of radiologists in detecting breast cancer in DM.

METHODS

Nine multi-reader, multi-case study datasets previously used for different research purposes in seven countries were collected. Each dataset consisted of DM exams acquired with systems from four different vendors, multiple radiologists' assessments per exam, and ground truth verified by histopathological analysis or follow-up, yielding a total of 2652 exams (653 malignant) and interpretations by 101 radiologists (28 296 independent interpretations). An AI system analyzed these exams yielding a level of suspicion of cancer present between 1 and 10. The detection performance between the radiologists and the AI system was compared using a noninferiority null hypothesis at a margin of 0.05.

RESULTS

The performance of the AI system was statistically noninferior to that of the average of the 101 radiologists. The AI system had a 0.840 (95% confidence interval [CI] = 0.820 to 0.860) area under the ROC curve and the average of the radiologists was 0.814 (95% CI = 0.787 to 0.841) (difference 95% CI = -0.003 to 0.055). The AI system had an AUC higher than 61.4% of the radiologists.

CONCLUSIONS

The evaluated AI system achieved a cancer detection accuracy comparable to an average breast radiologist in this retrospective setting. Although promising, the performance and impact of such a system in a screening setting needs further investigation.

Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study

Purpose

To study the feasibility of automatically identifying normal digital mammography (DM) exams with artificial intelligence (AI) to reduce the breast cancer screening reading workload.

Methods and materials

A total of 2652 DM exams (653 cancer) and interpretations by 101 radiologists were gathered from nine previously performed multi-reader multi-case receiver operating characteristic (MRMC ROC) studies. An AI system was used to obtain a score between 1 and 10 for each exam, representing the likelihood of cancer present. Using all AI scores between 1 and 9 as possible thresholds, the exams were divided into groups of low- and high likelihood of cancer present. It was assumed that, under the pre-selection scenario, only the high-likelihood group would be read by radiologists, while all low-likelihood exams would be reported as normal. The area under the reader-averaged ROC curve (AUC) was calculated for the original evaluations and for the pre-selection scenarios and compared using a non-inferiority hypothesis.

Results

Setting the low/high-likelihood threshold at an AI score of 5 (high likelihood > 5) results in a trade-off of approximately halving (− 47%) the workload to be read by radiologists while excluding 7% of true-positive exams. Using an AI score of 2 as threshold yields a workload reduction of 17% while only excluding 1% of true-positive exams. Pre-selection did not change the average AUC of radiologists (inferior 95% CI > − 0.05) for any threshold except at the extreme AI score of 9.

Conclusion

It is possible to automatically pre-select exams using AI to significantly reduce the breast cancer screening reading workload.

Key Points

• There is potential to use artificial intelligence to automatically reduce the breast cancer screening reading workload by excluding exams with a low likelihood of cancer.

• The exclusion of exams with the lowest likelihood of cancer in screening might not change radiologists’ breast cancer detection performance.

• When excluding exams with the lowest likelihood of cancer, the decrease in true-positive recalls would be balanced by a simultaneous reduction in false-positive recalls.

Novel Structured Reporting Systems for Theranostic Radiotracers

Standardized reporting is more and more routinely implemented in clinical practice, and such structured reports have a major impact on a large variety of medical fields, such as laboratory medicine, pathology, and, recently, radiology. Notably, the field of nuclear medicine is constantly evolving as novel radiotracers for numerous clinical applications are developed. Thus, framework systems for standardized reporting in this field may increase clinical acceptance of new radiotracers, allow for inter- and intracenter comparisons for quality assurance, and be used in global multicenter studies to ensure comparable results and enable efficient data abstraction. In the last couple of years, several standardized framework systems for PET radiotracers with potential theranostic applications have been proposed. These include systems for prostate-specific membrane antigen-targeted PET agents to diagnose and treat prostate cancer, and systems for somatostatin receptor-targeted PET agents to diagnose and treat neuroendocrine neoplasia. In the present review, the framework systems for these 2 types of cancer will be briefly introduced, followed by an overview of their advantages and limitations. In addition, potential applications will be defined, approaches to validate such concepts will be proposed, and future perspectives will be discussed.

Generative Adversarial Networks for the Creation of Realistic Artificial Brain Magnetic Resonance Images

Even as medical data sets become more publicly accessible, most are restricted to specific medical conditions. Thus, data collection for machine learning approaches remains challenging, and synthetic data augmentation, such as generative adversarial networks (GAN), may overcome this hurdle. In the present quality control study, deep convolutional GAN (DCGAN)–based human brain magnetic resonance (MR) images were validated by blinded radiologists. In total, 96 T1-weighted brain images from 30 healthy individuals and 33 patients with cerebrovascular accident were included. A training data set was generated from the T1-weighted images and DCGAN was applied to generate additional artificial brain images. The likelihood that images were DCGAN-created versus acquired was evaluated by 5 radiologists (2 neuroradiologists [NRs], vs 3 non-neuroradiologists [NNRs]) in a binary fashion to identify real vs created images. Images were selected randomly from the data set (variation of created images, 40%–60%). None of the investigated images was rated as unknown. Of the created images, the NRs rated 45% and 71% as real magnetic resonance imaging images (NNRs, 24%, 40%, and 44%). In contradistinction, 44% and 70% of the real images were rated as generated images by NRs (NNRs, 10%, 17%, and 27%). The accuracy for the NRs was 0.55 and 0.30 (NNRs, 0.83, 0.72, and 0.64). DCGAN-created brain MR images are similar enough to acquired MR images so as to be indistinguishable in some cases. Such an artificial intelligence algorithm may contribute to synthetic data augmentation for “data-hungry” technologies, such as supervised machine learning approaches, in various clinical applications.