Combination of generative adversarial network and convolutional neural network for automatic subcentimeter pulmonary adenocarcinoma classification

TDASD: Generating medically significant fine-grained lung adenocarcinoma nodule CT images based on stable diffusion models with limited sample size

BACKGROUND AND OBJECTIVES

Spread through air spaces (STAS) is an emerging lung cancer infiltration pattern. Predicting its spread through CT scans is crucial. However, limited STAS data makes this prediction task highly challenging. Stable diffusion is capable of generating more diverse and higher-quality images compared to traditional GAN models, surpassing the dominating GAN family models in image synthesis over the past few years. To alleviate the issue of limited STAS data, we propose a method TDASD based on stable diffusion, which is able to generate high-resolution CT images of pulmonary nodules corresponding to specific nodular signs according to the medical professionals.

METHODS

First, we apply the stable diffusion method for fine-tuning training on publicly available lung datasets. Subsequently, we extract nodules from our hospital's lung adenocarcinoma data and apply slight rotations to the original nodule CT slices within a reasonable range before undergoing another round of fine-tuning through stable diffusion. Finally, employing DDIM and Ksample sampling methods, we generate lung adenocarcinoma nodule CT images with signs based on prompts provided by doctors. The method we propose not only safeguards patient privacy but also enhances the diversity of medical images under limited data conditions. Furthermore, our approach to generating medical images incorporates medical knowledge, resulting in images that exhibit pertinent medical features, thus holding significant value in tumor discrimination diagnostics.

RESULTS

Our TDASD method has the capability to generate medically meaningful images by optimizing input prompts based on medical descriptions provided by experts. The images generated by our method can improve the model's classification accuracy. Furthermore, Utilizing solely the data generated by our method for model training, the test results on the original real dataset reveal an accuracy rate that closely aligns with the testing accuracy achieved through training on real data.

CONCLUSIONS

The method we propose not only safeguards patient privacy but also enhances the diversity of medical images under limited data conditions. Furthermore, our approach to generating medical images incorporates medical knowledge, resulting in images that exhibit pertinent medical features, thus holding significant value in tumor discrimination diagnostics.

Research progress on deep learning in magnetic resonance imaging-based diagnosis and treatment of prostate cancer: a review on the current status and perspectives

Multiparametric magnetic resonance imaging (mpMRI) has emerged as a first-line screening and diagnostic tool for prostate cancer, aiding in treatment selection and noninvasive radiotherapy guidance. However, the manual interpretation of MRI data is challenging and time-consuming, which may impact sensitivity and specificity. With recent technological advances, artificial intelligence (AI) in the form of computer-aided diagnosis (CAD) based on MRI data has been applied to prostate cancer diagnosis and treatment. Among AI techniques, deep learning involving convolutional neural networks contributes to detection, segmentation, scoring, grading, and prognostic evaluation of prostate cancer. CAD systems have automatic operation, rapid processing, and accuracy, incorporating multiple sequences of multiparametric MRI data of the prostate gland into the deep learning model. Thus, they have become a research direction of great interest, especially in smart healthcare. This review highlights the current progress of deep learning technology in MRI-based diagnosis and treatment of prostate cancer. The key elements of deep learning-based MRI image processing in CAD systems and radiotherapy of prostate cancer are briefly described, making it understandable not only for radiologists but also for general physicians without specialized imaging interpretation training. Deep learning technology enables lesion identification, detection, and segmentation, grading and scoring of prostate cancer, and prediction of postoperative recurrence and prognostic outcomes. The diagnostic accuracy of deep learning can be improved by optimizing models and algorithms, expanding medical database resources, and combining multi-omics data and comprehensive analysis of various morphological data. Deep learning has the potential to become the key diagnostic method in prostate cancer diagnosis and treatment in the future.

An interpretable deep learning approach for designing nanoporous silicon nitride membranes with tunable mechanical properties

The high permeability and strong selectivity of nanoporous silicon nitride (NPN) membranes make them attractive in a broad range of applications. Despite their growing use, the strength of NPN membranes needs to be improved for further extending their biomedical applications. In this work, we implement a deep learning framework to design NPN membranes with improved or prescribed strength values. We examine the predictions of our framework using physics-based simulations. Our results confirm that the proposed framework is not only able to predict the strength of NPN membranes with a wide range of microstructures, but also can design NPN membranes with prescribed or improved strength. Our simulations further demonstrate that the microstructural heterogeneity that our framework suggests for the optimized design, lowers the stress concentration around the pores and leads to the strength improvement of NPN membranes as compared to conventional membranes with homogenous microstructures.

Convolutional neural network of age-related trends digital radiographs of medial clavicle in a Thai population: a preliminary study

Age at death estimation has always been a crucial yet challenging part of identification process in forensic field. The use of human skeletons have long been explored using the principle of macro and micro-architecture change in correlation with increasing age. The clavicle is recommended as the best candidate for accurate age estimation because of its accessibility, time to maturation and minimal effect from weight. Our study applies pre-trained convolutional neural network in order to achieve the most accurate and cost effective age estimation model using clavicular bone. The total of 988 clavicles of Thai population with known age and sex were radiographed using Kodak 9000 Extra-oral Imaging System. The radiographs then went through preprocessing protocol which include region of interest selection and quality assessment. Additional samples were generated using generative adversarial network. The total clavicular images used in this study were 3,999 which were then separated into training and test set, and the test set were subsequently categorized into 7 age groups. GoogLeNet was modified at two layers and fine tuned the parameters. The highest validation accuracy was 89.02% but the test set achieved only 30% accuracy. Our results show that the use of medial clavicular radiographs has a potential in the field of age at death estimation, thus, further study is recommended.

Deep learning for predicting epidermal growth factor receptor mutations of non-small cell lung cancer on PET/CT images

Background

Predicting the mutation status of the epidermal growth factor receptor (EGFR) gene based on an integrated positron emission tomography/computed tomography (PET/CT) image of non-small cell lung cancer (NSCLC) is a noninvasive, low-cost method which is valuable for targeted therapy. Although deep learning has been very successful in robotic vision, it is still challenging to predict gene mutations in PET/CT-derived studies because of the small amount of medical data and the different parameters of PET/CT devices.

Methods

We used the advanced EfficientNet-V2 model to predict the EGFR mutation based on fused PET/CT images. First, we extracted 3-dimensional (3D) pulmonary nodules from PET and CT as regions of interest (ROIs). We then fused each single PET and CT image. The network model was used to predict the mutation status of lung nodules by the new data after fusion, and the model was weighted adaptively. The EfficientNet-V2 model used multiple channels to represent nodules comprehensively.

Results

We trained the EfficientNet-V2 model through our PET/CT fusion algorithm using a dataset of 150 patients. The prediction accuracy of EGFR and non-EGFR mutations was 86.25% in the training dataset, and the accuracy rate was 81.92% in the validation set.

Conclusions

Combined with experiments, the demonstrated PET/CT fusion algorithm outperformed radiomics methods in predicting EGFR and non-EGFR mutations in NSCLC.

Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in image synthesis, Generative Adversarial Networks (GANs), and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

Deep learning based dental implant failure prediction from periapical and panoramic films

Background

Dental implant failure is a critical condition that can seriously compromise therapeutic efficacy. Insufficient bone volume, unfavorable bone quality, periodontal bone loss, and systemic conditions, including osteopenia/osteoporosis and diabetes mellitus, have been associated with implant failure. Early indicators of potential implant failure could help mitigate the risk of severe complications. This study aimed to develop an effective implant outcome prediction model using dental periapical and panoramic films.

Methods

A total of 248 patients (89 with failed implants and 159 with successful implants) were examined. A total of 529 periapical images and 551 panoramic images were collected from the patients for a deep learning-based model. Based on radiographic peri-implant alveolar bone pattern, implant outcome was divided into three categories: implant failure with marginal bone loss, implant failure without marginal bone loss, and implant success. We extracted features using a deep convolutional neural network (CNN) and built a hybrid model to combine periapical and panoramic images. A comparison among three categories of receiver operating characteristic (ROC) curves was performed. The diagnostic accuracy, precision, recall and F1-score of the dataset were assessed.

Results

Our model achieved an AUC (area under the ROC curve) of 0.972 for failure with marginal bone loss, 0.947 for failure without marginal bone loss and 0.975 for success. In all conditions, for periapical images alone, the diagnostic accuracy was 78.6%; the precision was 0.84, recall was 0.73, and F1-score was 0.75. For panoramic images alone, the diagnostic accuracy was 78.7%; the precision was 0.87, recall was 0.63, and F1-score was 0.66. Both periapical and panoramic images were used in our novel method, and the prediction accuracy was 87%. The precision was 0.85, recall was 0.88, and F1-score was 0.85.

Conclusions

The deep learning model used features from periapical and panoramic images to effectively predict the occurrence of implant failure and might facilitate early clinical intervention for potential dental implant failures.

How does the artificial intelligence-based image-assisted technique help physicians in diagnosis of pulmonary adenocarcinoma? A randomized controlled experiment of multicenter physicians in China

OBJECTIVE

Although artificial intelligence (AI) has achieved high levels of accuracy in the diagnosis of various diseases, its impact on physicians' decision-making performance in clinical practice is uncertain. This study aims to assess the impact of AI on the diagnostic performance of physicians with differing levels of self-efficacy under working conditions involving different time pressures.

MATERIALS AND METHODS

A 2 (independent diagnosis vs AI-assisted diagnosis) × 2 (no time pressure vs 2-minute time limit) randomized controlled experiment of multicenter physicians was conducted. Participants diagnosed 10 pulmonary adenocarcinoma cases and their diagnostic accuracy, sensitivity, and specificity were evaluated. Data analysis was performed using multilevel logistic regression.

RESULTS

One hundred and four radiologists from 102 hospitals completed the experiment. The results reveal (1) AI greatly increases physicians' diagnostic accuracy, either with or without time pressure; (2) when no time pressure, AI significantly improves physicians' diagnostic sensitivity but no significant change in specificity, while under time pressure, physicians' diagnostic sensitivity and specificity are both improved with the aid of AI; (3) when no time pressure, physicians with low self-efficacy benefit from AI assistance thus improving diagnostic accuracy but those with high self-efficacy do not, whereas physicians with low and high levels of self-efficacy both benefit from AI under time pressure.

DISCUSSION

This study is one of the first to provide real-world evidence regarding the impact of AI on physicians' decision-making performance, taking into account 2 boundary factors: clinical time pressure and physicians' self-efficacy.

CONCLUSION

AI-assisted diagnosis should be prioritized for physicians working under time pressure or with low self-efficacy.

Performance of a generative adversarial network using ultrasound images to stage liver fibrosis and predict cirrhosis based on a deep-learning radiomics nomogram

AIM

To investigate the performance of a generative adversarial network (GAN) model for staging liver fibrosis and its radiomics-based nomogram for predicting cirrhosis.

MATERIALS AND METHODS

This two-centre retrospective study included 434 patients for whom input data of ultrasound images and histopathological data (obtained within 1 month of ultrasound examinations) were assigned to the training cohort (249 patients), the internal cohort (92 patients), and the external (93 patients) cohort. A data augmentation method based on a GAN model was used. The discriminative performance was evaluated for classifying fibrosis of S4 and ≥S3. Deep-learning radiomics features were extracted for the prediction of cirrhosis (S4). To perform feature reduction and selection, the least absolute shrinkage and selection operator (LASSO) algorithm was applied. Radiomics scores, along with clinical factors, were incorporated into a nomogram using multivariable logistic regression analysis. The performance of the models was estimated with respect to discrimination power, calibration, and clinical benefits.

RESULTS

The areas under the receiver operating characteristic curve (AUCs) values of the GAN were 0.832/0.762 (≥S3), and 0.867/0.835 (S4) for internal/external test sets, respectively. The radiomics nomogram that intergrated radiomics scores and clinical factors showed good calibration and discrimination ability of 0.922 (AUC) in the training dataset, 0.896 in the internal dataset, and 0.861 in the external dataset. Decision curve analysis (DCA) demonstrated that the nomogram outperformed radiologist and haematological indices in terms of the most clinical benefits.

CONCLUSIONS

The GAN model could be applied to discriminate fibrosis stages, and a favourable predictive accuracy for diagnosing cirrhosis was achieved using a deep-learning radiomics nomogram.

Pix2Pix generative adversarial network for low dose myocardial perfusion SPECT denoising

BACKGROUND

Myocardial perfusion (MP) SPECT is a well-established method for diagnosing cardiac disease, yet its radiation risk poses safety concern. This study aims to apply and evaluate the use of Pix2Pix generative adversarial network (Pix2Pix GAN) in denoising low dose MP SPECT images.

METHODS

One hundred male and female patients with different 99mTc-sestamibi activity distributions, organ and body sizes were simulated by a population of digital 4D Extended Cardiac Torso (XCAT) phantoms. Realistic noisy SPECT projections of full dose of 987 MBq injection and 16 min acquisition, and low dose ranged from 1/20 to 1/2 of the full dose, were generated by an analytical projector from the right anterior oblique (RAO) to the left posterior oblique (LPO) positions. Additionally, twenty patients underwent ~1,184 MBq 99mTc-sestamibi stress SPECT/CT scan were also retrospectively recruited for the study. For each patient, low dose SPECT images (7/10 to 1/10 of full dose) were generated from the full dose list mode data. Our Pix2Pix GAN model was trained with full dose and low dose reconstructed SPECT image pairs. Normalized mean square error (NMSE), structural similarity index (SSIM), coefficient of variation (CV), full-width-at-half-maximum (FWHM) and relative defect size differences (RSD) of Pix2Pix GAN processed images were evaluated along with a reference convolutional auto encoder (CAE) network and post-reconstruction filters.

RESULTS

NMSE values of 0.0233±0.004 vs. 0.0249±0.004 and 0.0313±0.007 vs. 0.0579±0.016 were obtained on 1/2 and 1/20 dose level for Pix2Pix GAN and CAE in the simulation study, while they were 0.0376±0.010 vs. 0.0433±0.010 and 0.0907±0.020 vs. 0.1186±0.025 on 7/10 and 1/10 dose level in the clinical study. Similar results were also obtained from the SSIM, CV, FWHM and RSD values. Overall, the use of Pix2Pix GAN was superior to other denoising methods in all physical indices, particular in the lower dose levels in the simulation and clinical study.

CONCLUSIONS

The Pix2Pix GAN method is effective to reduce the noise level of low dose MP SPECT. Further studies on clinical performance are warranted to demonstrate its full clinical effectiveness.

Design and Implementation of Obstetric Central Monitoring System Based on Medical Image Segmentation Algorithm

At present, the incidence of emergencies in obstetric care environment is gradually increasing, and different obstetric wards often have a variety of situations. Therefore, it can provide great help in clinical medicine to give early warning and plan coping plans according to different situations. This paper studied an obstetrics central surveillance system based on a medical image segmentation algorithm. Images obtained by central obstetrics monitoring are segmented, magnified in detail, and image features are extracted, collated, and trained. The normal distribution rule is used to classify the features, which are included in the feature library of the obstetric central monitoring system. In the gray space of the medical image, the statistical distribution of gray features of the medical image is described by the mixture model of Rayleigh distribution and Gaussian distribution. In the gray space of the medical image, Taylor series expansion is used to describe the linear geometric structure of medicine. The eigenvalues of Hessian matrix are introduced to obtain high-order multiscale features of medicine. The multiscale feature energy function is introduced into Markov random energy objective function to realize medical image segmentation. Compared with other segmentation algorithms, the accuracy and sensitivity of the proposed algorithm are 87.98% and 86.58%, respectively, which can clearly segment small medical features.

[Chinese Experts Consensus on Artificial Intelligence Assisted Management for Pulmonary Nodule (2022 Version)]

Low-dose computed tomography (CT) for lung cancer screening has been proven to reduce lung cancer deaths in the screening group compared with the control group. The increasing number of pulmonary nodules being detected by CT scans significantly increase the workload of the radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of pulmonary nodule discrimination and has been tested in preliminary studies for nodule management. As more and more artificial AI products are commercialized, the consensus statement has been organized in a collaborative effort by Thoracic Surgery Committee, Department of Simulated Medicine, Wu Jieping Medical Foundation to aid clinicians in the application of AI-assisted management for pulmonary nodules.
.

Models of Artificial Intelligence-Assisted Diagnosis of Lung Cancer Pathology Based on Deep Learning Algorithms

In this article, in order to explore the application of a diagnosis system for lung cancer, we use an auxiliary diagnostic system to predict and diagnose the good and evil attributes of chest CT pulmonary nodules. This research improves the new diagnosis method based on the convolutional neural network (CNN) and the recurrent neural network (RNN) and combines the dual effects of the two algorithms to process the classification of benign and malignant nodules. By collecting H-E-stained pathological slices of 652 patients' lung lesions from two hospitals between January 2018 and January 2019, the output results of the improved 3D U-net system and the consistent results of two-person reading were compared. This article analyzes the sensitivity, specificity, positive flammability rate, and negative flammability rate of different lung nodule detection methods. In addition, the artificial intelligence system's and the radiologist's judgment results of benign and malignant pulmonary nodules are used to draw ROC curves for further analysis. The improved model has an accuracy rate of 92.3% for predicting malignant lung nodules and an accuracy rate of 82.8% for benign lung nodules. The new diagnostic method using the convolutional neural network and the recurrent neural network can be very effective for improving the accuracy of predicting lung cancer diagnosis. It can play a very effective role in the disease prediction of lung cancer patients, thereby improving the treatment effect.

Special Convolutional Neural Network for Identification and Positioning of Interstitial Lung Disease Patterns in Computed Tomography Images

In this paper, automated detection of interstitial lung disease patterns in high resolution computed tomography images is achieved by developing a faster region-based convolutional network based detector with GoogLeNet as a backbone. GoogLeNet is simplified by removing few inception models and used as the backbone of the detector network. The proposed framework is developed to detect several interstitial lung disease patterns without doing lung field segmentation. The proposed method is able to detect the five most prevalent interstitial lung disease patterns: fibrosis, emphysema, consolidation, micronodules and ground-glass opacity, as well as normal. Five-fold cross-validation has been used to avoid bias and reduce over-fitting. The proposed framework performance is measured in terms of F-score on the publicly available MedGIFT database. It outperforms state-of-the-art techniques. The detection is performed at slice level and could be used for screening and differential diagnosis of interstitial lung disease patterns using high resolution computed tomography images.

A review of medical image data augmentation techniques for deep learning applications

Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning-based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren't typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state-of-the-art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.

Multi-channel multi-task deep learning for predicting EGFR and KRAS mutations of non-small cell lung cancer on CT images

BACKGROUND

Predicting the mutation statuses of 2 essential pathogenic genes [epidermal growth factor receptor (EGFR) and Kirsten rat sarcoma (KRAS)] in non-small cell lung cancer (NSCLC) based on CT is valuable for targeted therapy because it is a non-invasive and less costly method. Although deep learning technology has realized substantial computer vision achievements, CT imaging being used to predict gene mutations remains challenging due to small dataset limitations.

METHODS

We propose a multi-channel and multi-task deep learning (MMDL) model for the simultaneous prediction of EGFR and KRAS mutation statuses based on CT images. First, we decomposed each 3D lung nodule into 9 views. Then, we used the pre-trained inception-attention-resnet model for each view to learn the features of the nodules. By combining 9 inception-attention-resnet models to predict the types of gene mutations in lung nodules, the models were adaptively weighted, and the proposed MMDL model could be trained end-to-end. The MMDL model utilized multiple channels to characterize the nodule more comprehensively and integrate patient personal information into our learning process.

RESULTS

We trained the proposed MMDL model using a dataset of 363 patients collected by our partner hospital and conducted a multi-center validation on 162 patients in The Cancer Imaging Archive (TCIA) public dataset. The accuracies for the prediction of EGFR and KRAS mutations were, respectively, 79.43% and 72.25% in the training dataset and 75.06% and 69.64% in the validation dataset.

CONCLUSIONS

The experimental results demonstrated that the proposed MMDL model outperformed the latest methods in predicting EGFR and KRAS mutations in NSCLC.