Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning

Machine learning: an advancement in biochemical engineering

One of the most remarkable techniques recently introduced into the field of bioprocess engineering is machine learning. Bioprocess engineering has drawn much attention due to its vast application in different domains like biopharmaceuticals, fossil fuel alternatives, environmental remediation, and food and beverage industry, etc. However, due to their unpredictable mechanisms, they are very often challenging to optimize. Furthermore, biological systems are extremely complicated; hence, machine learning algorithms could potentially be utilized to improve and build new biotechnological processes. Gaining insight into the fundamental mathematical understanding of commonly used machine learning algorithms, including Support Vector Machine, Principal Component Analysis, Partial Least Squares and Reinforcement Learning, the present study aims to discuss various case studies related to the application of machine learning in bioprocess engineering. Recent advancements as well as challenges posed in this area along with their potential solutions are also presented.

BrainDAS: Structure-aware domain adaptation network for multi-site brain network analysis

In the medical field, datasets are mostly integrated across sites due to difficult data acquisition and insufficient data at a single site. The domain shift problem caused by the heterogeneous distribution among multi-site data makes autism spectrum disorder (ASD) hard to identify. Recently, domain adaptation has received considerable attention as a promising solution. However, domain adaptation on graph data like brain networks has not been fully studied. It faces two major challenges: (1) complex graph structure; and (2) multiple source domains. To overcome the issues, we propose an end-to-end structure-aware domain adaptation framework for brain network analysis (BrainDAS) using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed approach contains two stages: supervision-guided multi-site graph domain adaptation with dynamic kernel generation and graph classification with attention-based graph pooling. We evaluate our BrainDAS on a public dataset provided by Autism Brain Imaging Data Exchange (ABIDE) which includes 871 subjects from 17 different sites, surpassing state-of-the-art algorithms in several different evaluation settings. Furthermore, our promising results demonstrate the interpretability and generalization of the proposed method. Our code is available at https://github.com/songruoxian/BrainDAS.

The Role and Applications of Artificial Intelligence in the Treatment of Chronic Pain

PURPOSE OF REVIEW

This review aims to explore the interface between artificial intelligence (AI) and chronic pain, seeking to identify areas of focus for enhancing current treatments and yielding novel therapies.

RECENT FINDINGS

In the United States, the prevalence of chronic pain is estimated to be upwards of 40%. Its impact extends to increased healthcare costs, reduced economic productivity, and strain on healthcare resources. Addressing this condition is particularly challenging due to its complexity and the significant variability in how patients respond to treatment. Current options often struggle to provide long-term relief, with their benefits rarely outweighing the risks, such as dependency or other side effects. Currently, AI has impacted four key areas of chronic pain treatment and research: (1) predicting outcomes based on clinical information; (2) extracting features from text, specifically clinical notes; (3) modeling 'omic data to identify meaningful patient subgroups with potential for personalized treatments and improved understanding of disease processes; and (4) disentangling complex neuronal signals responsible for pain, which current therapies attempt to modulate. As AI advances, leveraging state-of-the-art architectures will be essential for improving chronic pain treatment. Current efforts aim to extract meaningful representations from complex data, paving the way for personalized medicine. The identification of unique patient subgroups should reveal targets for tailored chronic pain treatments. Moreover, enhancing current treatment approaches is achievable by gaining a more profound understanding of patient physiology and responses. This can be realized by leveraging AI on the increasing volume of data linked to chronic pain.

Identifying and training deep learning neural networks on biomedical-related datasets

This manuscript describes the development of a resources module that is part of a learning platform named 'NIGMS Sandbox for Cloud-based Learning' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox at the beginning of this Supplement. This module delivers learning materials on implementing deep learning algorithms for biomedical image data in an interactive format that uses appropriate cloud resources for data access and analyses. Biomedical-related datasets are widely used in both research and clinical settings, but the ability for professionally trained clinicians and researchers to interpret datasets becomes difficult as the size and breadth of these datasets increases. Artificial intelligence, and specifically deep learning neural networks, have recently become an important tool in novel biomedical research. However, use is limited due to their computational requirements and confusion regarding different neural network architectures. The goal of this learning module is to introduce types of deep learning neural networks and cover practices that are commonly used in biomedical research. This module is subdivided into four submodules that cover classification, augmentation, segmentation and regression. Each complementary submodule was written on the Google Cloud Platform and contains detailed code and explanations, as well as quizzes and challenges to facilitate user training. Overall, the goal of this learning module is to enable users to identify and integrate the correct type of neural network with their data while highlighting the ease-of-use of cloud computing for implementing neural networks. This manuscript describes the development of a resource module that is part of a learning platform named ``NIGMS Sandbox for Cloud-based Learning'' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox [1] at the beginning of this Supplement. This module delivers learning materials on the analysis of bulk and single-cell ATAC-seq data in an interactive format that uses appropriate cloud resources for data access and analyses.

Machine-Learning-Based Characterization and Inverse Design of Metamaterials

Metamaterials, characterized by unique structures, exhibit exceptional properties applicable across various domains. Traditional methods like experiments and finite-element methods (FEM) have been extensively utilized to characterize these properties. However, exploring an extensive range of structures using these methods for designing desired structures with excellent properties can be time-intensive. This paper formulates a machine-learning-based approach to expedite predicting effective metamaterial properties, leading to the discovery of microstructures with diverse and outstanding characteristics. The process involves constructing 2D and 3D microstructures, encompassing porous materials, solid-solid-based materials, and fluid-solid-based materials. Finite-element methods are then employed to determine the effective properties of metamaterials. Subsequently, the Random Forest (RF) algorithm is applied for training and predicting effective properties. Additionally, the Aquila Optimizer (AO) method is employed for a multiple optimization task in inverse design. The regression model generates accurate estimation with a coefficient of determination higher than 0.98, a mean absolute percentage error lower than 0.088, and a root mean square error lower than 0.03, indicating that the machine-learning-based method can accurately characterize the metamaterial properties. An optimized structure with a high Young's modulus and low thermal conductivity is designed by AO within the first 30 iterations. This approach accelerates simulating the effective properties of metamaterials and can design microstructures with multiple excellent performances. The work offers guidance to design microstructures in various practical applications such as vibration energy absorbers.

A Scientometric Worldview of Artificial Intelligence in Musculoskeletal Diseases Since the 21st Century

Purpose

Over the past 24 years, significant advancements have been made in applying artificial intelligence (AI) to musculoskeletal (MSK) diseases. However, there is a lack of analytical and descriptive investigations on the trajectory, essential research directions, current research scenario, pivotal focuses, and future perspectives. This research aims to provide a thorough update on the progress in AI for MSK diseases over the last 24 years.

Methods

Data from the Web of Science database, covering January 1, 2000, to March 1, 2024, was analyzed. Using advanced analytical tools, we conducted comprehensive scientometric and visual analyses.

Results

The findings highlight the predominant influence of the USA, which accounts for 28.53% of the total publications and plays a key role in shaping research in this field. Notable productivity was seen at institutions such as the University of California, San Francisco, Harvard Medical School, and Seoul National University. Valentina Pedoia is identified as the most prolific contributor. Scientific Reports had the highest number of publications in this area. The five most significant diseases are joint diseases, bone fractures, bone tumors, cartilage diseases, and spondylitis.

Conclusion

This comprehensive scientometric assessment benefits both experienced researchers and newcomers, providing quick access to essential information and fostering the development of innovative concepts in this field.

Frequency-hopping along with resolution-turning for fast and enhanced reconstruction in ultrasound tomography

The distorted Born iterative (DBI) method is considered to obtain images with high-contrast and resolution. Besides satisfying the Born approximation condition, the frequency-hopping (FH) technique is necessary to gradually update the sound contrast from the first iteration and progress to the actual sound contrast of the imaged object in subsequent iterations. Inspired by the fact that the higher the frequency, the higher the resolution. Because low-frequency allows for low-resolution object imaging, hence for high-resolution imaging requirements, using low-frequency to possess a high-resolution image from the first iteration will be less efficient. For an effective reconstruction, the object's resolution at low frequencies should be small. And similarly, with high frequencies, the object resolution should be larger. Therefore, in this paper, the FH, and the resolution-turning (RT) technique are proposed to obtain object images with high-contrast and -resolution. The convergence speed in the initial iterations is rapidly achieved by utilizing low frequency in the frequency-turning technique and low image resolution in the resolution-turning technique. It is crucial to ensure accurate object reconstruction for subsequent iterations. The desired spatial resolution is attained by employing high frequency and large image resolution. The resolution-turning distorted Born iterative (RT-DBI) and frequency-hopping distorted Born iterative (FH-DBI) solutions are thoroughly investigated to exploit their best performance. This makes sense because if it is not good to choose the number of iterations for the frequency f1 in FH-DBI and for the resolution of N1 × N1 in RT-DBI, then these solutions give even worse quality than traditional DBI. After that, the RT-FH-DBI integration was investigated in two sub-solutions. We found that the lower frequency f1 used both before and after the RT would get the best performance. Consequently, compared to the traditional DBI approaches, the normalized error and total runtime for the reconstruction process were dramatically decreased, at 83.6% and 18.6%, respectively. Besides fast and quality imaging, the proposed solution RT-FH-DBI is promised to produce high-contrast and high-resolution object images, aiming at object reconstruction at the biological tissue. The development of 3D imaging and experimental verification will be studied further.

Application of convolutional neural network for differentiating ovarian thecoma-fibroma and solid ovarian cancer based on MRI

BACKGROUND

Ovarian thecoma-fibroma and solid ovarian cancer have similar clinical and imaging features, and it is difficult for radiologists to differentiate them. Since the treatment and prognosis of them are different, accurate characterization is crucial.

PURPOSE

To non-invasively differentiate ovarian thecoma-fibroma and solid ovarian cancer by convolutional neural network based on magnetic resonance imaging (MRI), and to provide the interpretability of the model.

MATERIAL AND METHODS

A total of 156 tumors, including 86 ovarian thecoma-fibroma and 70 solid ovarian cancer, were split into the training set, the validation set, and the test set according to the ratio of 8:1:1 by stratified random sampling. In this study, we used four different networks, two different weight modes, two different optimizers, and four different sizes of regions of interest (ROI) to test the model performance. This process was repeated 10 times to calculate the average performance of the test set. The gradient weighted class activation mapping (Grad-CAM) was used to explain how the model makes classification decisions by visual location map.

RESULTS

ResNet18, which had pre-trained weight, using Adam and one multiple ROI circumscribed rectangle, achieved best performance. The average accuracy, precision, recall, and AUC were 0.852, 0.828, 0.848, and 0.919 (P < 0.01), respectively. Grad-CAM showed areas associated with classification appeared on the edge or interior of ovarian thecoma-fibroma and the interior of solid ovarian cancer.

CONCLUSION

This study shows that convolution neural network based on MRI can be helpful for radiologists in differentiating ovarian thecoma-fibroma and solid ovarian cancer.

An electrocardiogram-based AI algorithm for early detection of pulmonary hypertension

BACKGROUND

Early diagnosis of pulmonary hypertension (PH) is critical for effective treatment and management. We aimed to develop and externally validate an artificial intelligence algorithm that could serve as a PH screening tool, based on analysis of a standard 12-lead ECG.

METHODS

The PH Early Detection Algorithm (PH-EDA) is a convolutional neural network developed using retrospective ECG voltage-time data, with patients classified as "PH-likely" or "PH-unlikely" (controls) based on right heart catheterisation or echocardiography. In total, 39 823 PH-likely patients and 219 404 control patients from Mayo Clinic were randomly split into training (48%), validation (12%) and test (40%) sets. ECGs taken within 1 month of PH diagnosis (diagnostic dataset) were used to train the PH-EDA at Mayo Clinic. Performance was tested on diagnostic ECGs within the test sets from Mayo Clinic (n=16 175/87 998 PH-likely/controls) and Vanderbilt University Medical Center (VUMC; n=6045/24 256 PH-likely/controls). In addition, performance was tested on ECGs taken 6-18 months (pre-emptive dataset), and up to 5 years prior to a PH diagnosis at both sites.

RESULTS

Performance testing yielded an area under the receiver operating characteristic curve (AUC) of 0.92 and 0.88 in the diagnostic test sets at Mayo Clinic and VUMC, respectively, and 0.86 and 0.81, respectively, in the pre-emptive test sets. The AUC remained a minimum of 0.79 at Mayo Clinic and 0.73 at VUMC up to 5 years before diagnosis.

CONCLUSION

The PH-EDA can detect PH at diagnosis and 6-18 months prior, demonstrating the potential to accelerate diagnosis and management of this debilitating disease.

Research on ultrasound-based radiomics: a bibliometric analysis

Background

A large number of studies related to ultrasound-based radiomics have been published in recent years; however, a systematic bibliometric analysis of this topic has not yet been conducted. In this study, we attempted to identify the hotspots and frontiers in ultrasound-based radiomics through bibliometrics and to systematically characterize the overall framework and characteristics of studies through mapping and visualization.

Methods

A literature search was carried out in Web of Science Core Collection (WoSCC) database from January 2016 to December 2023 according to a predetermined search formula. Bibliometric analysis and visualization of the results were performed using CiteSpace, VOSviewer, R, and other platforms.

Results

Ultimately, 466 eligible papers were included in the study. Publication trend analysis showed that the annual publication trend of journals in ultrasound-based radiomics could be divided into three phases: there were no more than five documents published in this field in any year before 2018, a small yearly increase in the number of annual publications occurred between 2018 and 2022, and a high, stable number of publications appeared after 2022. In the analysis of publication sources, China was found to be the main contributor, with a much higher number of publications than other countries, and was followed by the United States and Italy. Frontiers in Oncology was the journal with the highest number of papers in this field, publishing 60 articles. Among the academic institutions, Fudan University, Sun Yat-sen University, and the Chinese Academy of Sciences ranked as the top three in terms of the number of documents. In the analysis of authors and cocited authors, the author with the most publications was Yuanyuan Wang, who has published 19 articles in 8 years, while Philippe Lambin was the most cited author, with 233 citations. Visualization of the results from the cocitation analysis of the literature revealed a strong centrality of the subject terms papillary thyroid cancer, biological behavior, potential biomarkers, and comparative assessment, which may be the main focal points of research in this subject. Based on the findings of the keyword analysis and cluster analysis, the keywords can be categorized into two major groups: (I) technological innovations that enable the construction of radiomics models such as machine learning and deep learning and (II) applications of predictive models to support clinical decision-making in certain diseases, such as papillary thyroid cancer, hepatocellular carcinoma (HCC), and breast cancer.

Conclusions

Ultrasound-based radiomics has received widespread attention in the medical field and has been gradually been applied in clinical research. Radiomics, a relatively late development in medical technology, has made substantial contributions to the diagnosis, prediction, and prognostic evaluation of diseases. Additionally, the coupling of artificial intelligence techniques with ultrasound imaging has yielded a number of promising tools that facilitate clinical decision-making and enable the practice of precision medicine. Finally, the development of ultrasound-based radiomics requires multidisciplinary cooperation and joint efforts from the field biomedicine, information technology, statistics, and clinical medicine.

Dual Prototypical Self-Supervised Learning for One-shot Medical Image Segmentation

Medical image segmentation using deep learning typically requires a large quantity of well-annotated data. However, the acquisition of pixel-level annotations is arduous and expensive, often requiring the expertise of experienced medical professionals. Recent advancements in few-shot learning can help address label scarcity in medical image segmentation by leveraging a small amount of labeled data. Most research on prototypical learning for medical image segmentation involves prototypes averaged across the entire class, ignoring fine details. In this work, we propose a novel dual prototype network and introduce part prototypes to supplement local prototypes for extracting fine-grained features and enhancing model performance in one-shot medical image segmentation. Extensive experiments on the CHAOS dataset demonstrate the effectiveness of the proposed method on one-shot segmentation, outperforming other methods by a significant margin.Clinical relevance- The proposed method can address the problem of prototypical self-supervised learning in the one-shot medical image segmentation scenario, saving annotation costs in clinical practice.

Connectional-style-guided contextual representation learning for brain disease diagnosis

Structural magnetic resonance imaging (sMRI) has shown great clinical value and has been widely used in deep learning (DL) based computer-aided brain disease diagnosis. Previous DL-based approaches focused on local shapes and textures in brain sMRI that may be significant only within a particular domain. The learned representations are likely to contain spurious information and have poor generalization ability in other diseases and datasets. To facilitate capturing meaningful and robust features, it is necessary to first comprehensively understand the intrinsic pattern of the brain that is not restricted within a single data/task domain. Considering that the brain is a complex connectome of interlinked neurons, the connectional properties in the brain have strong biological significance, which is shared across multiple domains and covers most pathological information. In this work, we propose a connectional style contextual representation learning model (CS-CRL) to capture the intrinsic pattern of the brain, used for multiple brain disease diagnosis. Specifically, it has a vision transformer (ViT) encoder and leverages mask reconstruction as the proxy task and Gram matrices to guide the representation of connectional information. It facilitates the capture of global context and the aggregation of features with biological plausibility. The results indicate that CS-CRL achieves superior accuracy in multiple brain disease diagnosis tasks across six datasets and three diseases and outperforms state-of-the-art models. Furthermore, we demonstrate that CS-CRL captures more brain-network-like properties, and better aggregates features, is easier to optimize, and is more robust to noise, which explains its superiority in theory.

Reinforced Computer-Aided Framework for Diagnosing Thyroid Cancer

Thyroid cancer is the most pervasive disease in the endocrine system and is getting extensive attention. The most prevalent method for an early check is ultrasound examination. Traditional research mainly concentrates on promoting the performance of processing a single ultrasound image using deep learning. However, the complex situation of patients and nodules often makes the model dissatisfactory in terms of accuracy and generalization. Imitating the diagnosis process in reality, a practical diagnosis-oriented computer-aided diagnosis (CAD) framework towards thyroid nodules is proposed, using collaborative deep learning and reinforcement learning. Under the framework, the deep learning model is trained collaboratively with multiparty data; afterward classification results are fused by a reinforcement learning agent to decide the final diagnosis result. Within the architecture, multiparty collaborative learning with privacy-preserving on large-scale medical data brings robustness and generalization, and diagnostic information is modeled as a Markov decision process (MDP) to get final precise diagnosis results. Moreover, the framework is scalable and capable of containing more diagnostic information and multiple sources to pursue a precise diagnosis. A practical dataset of two thousand thyroid ultrasound images is collected and labeled for collaborative training on classification tasks. The simulated experiments have shown the advancement of the framework in promising performance.

Enhancing automatic prediction of clinically significant prostate cancer with deep transfer learning 2.5-dimensional segmentation on bi-parametric magnetic resonance imaging (bp-MRI)

Background

The aggressiveness of prostate cancer (PCa) is crucial in determining treatment method. The purpose of this study was to establish a 2.5-dimensional (2.5D) deep transfer learning (DTL) detection model for the automatic detection of clinically significant PCa (csPCa) based on bi-parametric magnetic resonance imaging (bp-MRI).

Methods

A total of 231 patients, including 181 with csPCa and 50 with non-clinically significant PCa (non-csPCa), were enrolled. Stratified random sampling was then employed to divide all participants into a training set [185] and a test set [46]. The DTL model was obtained through image acquisition, image segmentation, and model construction. Finally, the diagnostic performance of the 2.5D and 2-dimensional (2D) models in predicting the aggressiveness of PCa was evaluated and compared using receiver operating characteristic (ROC) curves.

Results

DTL models based on 2D and 2.5D segmentation were established and validated to assess the aggressiveness of PCa. The results demonstrated that the diagnostic efficiency of the DTL model based on 2.5D was superior to that of the 2D model, regardless of whether in a single or combined sequence. Particularly, the 2.5D combined model outperformed other models in differentiating csPCa from non-csPCa. The area under the curve (AUC) values for the 2.5D combined model in the training and test sets were 0.960 and 0.949, respectively. Furthermore, the T2-weighted imaging (T2WI) model showed superiority over the apparent diffusion coefficient (ADC) model, but was not as effective as the combined model, whether based on 2.5D or 2D.

Conclusions

A DTL model based on 2.5D segmentation was developed to automatically evaluate PCa aggressiveness on bp-MRI, improving the diagnostic performance of the 2D model. The results indicated that the continuous information between adjacent layers can enhance the detection rate of lesions and reduce the misjudgment rate based on the DTL model.

Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.

Classification of pain expression images in elderly with hip fractures based on improved ResNet50 network

The aim of this study is designed an improved ResNet 50 network to achieve automatic classification model for pain expressions by elderly patients with hip fractures. This study built a dataset by combining the advantages of deep learning in image recognition, using a hybrid of the Multi-Task Cascaded Convolutional Neural Networks (MTCNN). Based on ResNet50 network framework utilized transfer learning to implement model function. This study performed the hyperparameters by Bayesian optimization in the learning process. This study calculated intraclass correlation between visual analog scale scores provided by clinicians independently and those provided by pain expression evaluation assistant(PEEA). The automatic pain expression recognition model in elderly patients with hip fractures, which constructed using the algorithm. The accuracy achieved 99.6% on the training set, 98.7% on the validation set, and 98.2% on the test set. The substantial kappa coefficient of 0.683 confirmed the efficacy of PEEA in clinic. This study demonstrates that the improved ResNet50 network can be used to construct an automatic pain expression recognition model for elderly patients with hip fractures, which has higher accuracy.

Brain age gap estimation using attention-based ResNet method for Alzheimer's disease detection

This study investigates the correlation between brain age and chronological age in healthy individuals using brain MRI images, aiming to identify potential biomarkers for neurodegenerative diseases like Alzheimer's. To achieve this, a novel attention-based ResNet method, 3D-Attention-Resent-SVR, is proposed to accurately estimate brain age and distinguish between Cognitively Normal (CN) and Alzheimer's disease (AD) individuals by computing the brain age gap (BAG). Unlike conventional methods, which often rely on single datasets, our approach addresses potential biases by employing four datasets for training and testing. The results, based on a combined dataset from four public sources comprising 3844 data points, demonstrate the model's efficacy with a mean absolute error (MAE) of 2.05 for brain age gap estimation. Moreover, the model's generalizability is showcased by training on three datasets and testing on a separate one, yielding a remarkable MAE of 2.4. Furthermore, leveraging BAG as the sole biomarker, our method achieves an accuracy of 92% and an AUC of 0.87 in Alzheimer's disease detection on the ADNI dataset. These findings underscore the potential of our approach in assisting with early detection and disease monitoring, emphasizing the strong correlation between BAG and AD.

Pattern classification of interstitial lung diseases from computed tomography images using a ResNet-based network with a split-transform-merge strategy and split attention

In patients with interstitial lung disease (ILD), accurate pattern assessment from their computed tomography (CT) images could help track lung abnormalities and evaluate treatment efficacy. Based on excellent image classification performance, convolutional neural networks (CNNs) have been massively investigated for classifying and labeling pathological patterns in the CT images of ILD patients. However, previous studies rarely considered the three-dimensional (3D) structure of the pathological patterns of ILD and used two-dimensional network input. In addition, ResNet-based networks such as SE-ResNet and ResNeXt with high classification performance have not been used for pattern classification of ILD. This study proposed a SE-ResNeXt-SA-18 for classifying pathological patterns of ILD. The SE-ResNeXt-SA-18 integrated the multipath design of the ResNeXt and the feature weighting of the squeeze-and-excitation network with split attention. The classification performance of the SE-ResNeXt-SA-18 was compared with the ResNet-18 and SE-ResNeXt-18. The influence of the input patch size on classification performance was also evaluated. Results show that the classification accuracy was increased with the increase of the patch size. With a 32 × 32 × 16 input, the SE-ResNeXt-SA-18 presented the highest performance with average accuracy, sensitivity, and specificity of 0.991, 0.979, and 0.994. High-weight regions in the class activation maps of the SE-ResNeXt-SA-18 also matched the specific pattern features. In comparison, the performance of the SE-ResNeXt-SA-18 is superior to the previously reported CNNs in classifying the ILD patterns. We concluded that the SE-ResNeXt-SA-18 could help track or monitor the progress of ILD through accuracy pattern classification.

Prediction of treatment response in major depressive disorder using a hybrid of convolutional recurrent deep neural networks and effective connectivity based on EEG signal

In this study, we have developed a novel method based on deep learning and brain effective connectivity to classify responders and non-responders to selective serotonin reuptake inhibitors (SSRIs) antidepressants in major depressive disorder (MDD) patients prior to the treatment using EEG signal. The effective connectivity of 30 MDD patients was determined by analyzing their pretreatment EEG signals, which were then concatenated into delta, theta, alpha, and beta bands and transformed into images. Using these images, we then fine tuned a hybrid Convolutional Neural Network that is enhanced with bidirectional Long Short-Term Memory cells based on transfer learning. The Inception-v3, ResNet18, DenseNet121, and EfficientNet-B0 models are implemented as base models. Finally, the models are followed by BiLSTM and dense layers in order to classify responders and non-responders to SSRI treatment. Results showed that the EfficiencyNet-B0 has the highest accuracy of 98.33, followed by DensNet121, ResNet18 and Inception-v3. Therefore, a new method was proposed in this study that uses deep learning models to extract both spatial and temporal features automatically, which will improve classification results. The proposed method provides accurate identification of MDD patients who are responding, thereby reducing the cost of medical facilities and patient care.

Multi-scale feature fusion and class weight loss for skin lesion classification

Skin cancer is one of the common types of cancer. It spreads quickly and is not easy to detect in the early stages, posing a major threat to human health. In recent years, deep learning methods have attracted widespread attention for skin cancer detection in dermoscopic images. However, training a practical classifier becomes highly challenging due to inter-class similarity and intra-class variation in skin lesion images. To address these problems, we propose a multi-scale fusion structure that combines shallow and deep features for more accurate classification. Simultaneously, we implement three approaches to the problem of class imbalance: class weighting, label smoothing, and resampling. In addition, the HAM10000_RE dataset strips out hair features to demonstrate the role of hair features in the classification process. We demonstrate that the region of interest is the most critical classification feature for the HAM10000_SE dataset, which segments lesion regions. We evaluated the effectiveness of our model using the HAM10000 and ISIC2019 dataset. The results showed that this method performed well in dermoscopic classification tasks, with ACC and AUC of 94.0% and 99.3%, on the HAM10000 dataset and ACC of 89.8% for the ISIC2019 dataset. The overall performance of our model is excellent in comparison to state-of-the-art models.

Predicting mortality after transcatheter aortic valve replacement using preprocedural CT

Transcatheter aortic valve replacement (TAVR) is a widely used intervention for patients with severe aortic stenosis. Identifying high-risk patients is crucial due to potential postprocedural complications. Currently, this involves manual clinical assessment and time-consuming radiological assessment of preprocedural computed tomography (CT) images by an expert radiologist. In this study, we introduce a probabilistic model that predicts post-TAVR mortality automatically using unprocessed, preprocedural CT and 25 baseline patient characteristics. The model utilizes CT volumes by automatically localizing and extracting a region of interest around the aortic root and ascending aorta. It then extracts task-specific features with a 3D deep neural network and integrates them with patient characteristics to perform outcome prediction. As missing measurements or even missing CT images are common in TAVR planning, the proposed model is designed with a probabilistic structure to allow for marginalization over such missing information. Our model demonstrates an AUROC of 0.725 for predicting all-cause mortality during postprocedure follow-up on a cohort of 1449 TAVR patients. This performance is on par with what can be achieved with lengthy radiological assessments performed by experts. Thus, these findings underscore the potential of the proposed model in automatically analyzing CT volumes and integrating them with patient characteristics for predicting mortality after TAVR.

Applying a deep learning pipeline to classify land cover from low-quality historical RGB imagery

Land use and land cover (LULC) classification is becoming faster and more accurate thanks to new deep learning algorithms. Moreover, new high spectral- and spatial-resolution datasets offer opportunities to classify land cover with greater accuracy and class specificity. However, deploying deep learning algorithms to characterize present-day, modern land cover based on state-of-the-art data is insufficient for understanding trends in land cover change and identifying changes in and drivers of ecological and social variables of interest. These identifications require characterizing past land cover, for which imagery is often lower-quality. We applied a deep learning pipeline to classify land cover from historical, low-quality RGB aerial imagery, using a case study of Vancouver, Canada. We deployed an atrous convolutional neural network from DeepLabv3+ (which has previously shown to outperform other networks) and trained it on modern Maxar satellite imagery using a modern land cover classification. We fine-tuned the resultant model using a small dataset of manually annotated and augmented historical imagery. This final model accurately predicted historical land cover classification at rates similar to other studies that used high-quality imagery. These predictions indicate that Vancouver has lost vegetative cover from 1995-2021, including a decrease in conifer cover, an increase in pavement cover, and an overall decrease in tree and grass cover. Our workflow may be harnessed to understand historical land cover and identify land cover change in other regions and at other times.

Real-Time Precise Targeting of the Subthalamic Nucleus via Transfer Learning in a Rat Model of Parkinson's Disease Based on Microelectrode Arrays

In neurodegenerative disorders, neuronal firing patterns and oscillatory activity are remarkably altered in specific brain regions, which can serve as valuable biomarkers for the identification of deep brain regions. The subthalamic nucleus (STN) has been the primary target for DBS in patients with Parkinson's disease (PD). In this study, changes in the spike firing patterns and spectral power of local field potentials (LFPs) in the pre-STN (zona incerta, ZI) and post-STN (cerebral peduncle, cp) regions were investigated in PD rats, providing crucial evidence for the functional localization of the STN. Sixteen-channel microelectrode arrays (MEAs) with sites distributed at different depths and widths were utilized to record neuronal activities. The spikes in the STN exhibited higher firing rates than those in the ZI and cp. Furthermore, the LFP power in the delta band in the STN was the greatest, followed by that in the ZI, and was greater than that in the cp. Additionally, increased LFP power was observed in the beta bands in the STN. To identify the best performing classification model, we applied various convolutional neural networks (CNNs) based on transfer learning to analyze the recorded raw data, which were processed using the Gram matrix of the spikes and the fast Fourier transform of the LFPs. The best transfer learning model achieved an accuracy of 95.16%. After fusing the spike and LFP classification results, the time precision for processing the raw data reached 500 ms. The pretrained model, utilizing raw data, demonstrated the feasibility of employing transfer learning for training models on neural activity. This approach highlights the potential for functional localization within deep brain regions.

Application of deep learning to pressure injury staging

OBJECTIVE

Accurate assessment of pressure injuries (PIs) is necessary for a good outcome. Junior and non-specialist nurses have less experience with PIs and lack clinical practice, and so have difficulty staging them accurately. In this work, a deep learning-based system for PI staging and tissue classification is proposed to help improve its accuracy and efficiency in clinical practice, and save healthcare costs.

METHOD

A total of 1610 cases of PI and their corresponding photographs were collected from clinical practice, and each sample was accurately staged and the tissues labelled by experts for training a Mask Region-based Convolutional Neural Network (Mask R-CNN, Facebook Artificial Intelligence Research, Meta, US) object detection and instance segmentation network. A recognition system was set up to automatically stage and classify the tissues of the remotely uploaded PI photographs.

RESULTS

On a test set of 100 samples, the average precision of this model for stage recognition reached 0.603, which exceeded that of the medical personnel involved in the comparative evaluation, including an enterostomal therapist.

CONCLUSION

In this study, the deep learning-based PI staging system achieved the evaluation performance of a nurse with professional training in wound care. This low-cost system could help overcome the difficulty of identifying PIs by junior and non-specialist nurses, and provide valuable auxiliary clinical information.

CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation

Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.

The State of Artificial Intelligence in Pediatric Surgery: A Systematic Review

BACKGROUND

Artificial intelligence (AI) has been recently shown to improve clinical workflows and outcomes - yet its potential in pediatric surgery remains largely unexplored. This systematic review details the use of AI in pediatric surgery.

METHODS

Nine medical databases were searched from inception until January 2023, identifying articles focused on AI in pediatric surgery. Two authors reviewed full texts of eligible articles. Studies were included if they were original investigations on the development, validation, or clinical application of AI models for pediatric health conditions primarily managed surgically. Studies were excluded if they were not peer-reviewed, were review articles, editorials, commentaries, or case reports, did not focus on pediatric surgical conditions, or did not employ at least one AI model. Extracted data included study characteristics, clinical specialty, AI method and algorithm type, AI model (algorithm) role and performance metrics, key results, interpretability, validation, and risk of bias using PROBAST and QUADAS-2.

RESULTS

Authors screened 8178 articles and included 112. Half of the studies (50%) reported predictive models (for adverse events [25%], surgical outcomes [16%] and survival [9%]), followed by diagnostic (29%) and decision support models (21%). Neural networks (44%) and ensemble learners (36%) were the most commonly used AI methods across application domains. The main pediatric surgical subspecialties represented across all models were general surgery (31%) and neurosurgery (25%). Forty-four percent of models were interpretable, and 6% were both interpretable and externally validated. Forty percent of models had a high risk of bias, and concerns over applicability were identified in 7%.

CONCLUSIONS

While AI has wide potential clinical applications in pediatric surgery, very few published AI algorithms were externally validated, interpretable, and unbiased. Future research needs to focus on developing AI models which are prospectively validated and ultimately integrated into clinical workflows.

LEVEL OF EVIDENCE

2A.

Deep learning for computer-aided abnormalities classification in digital mammogram: A data-centric perspective

Breast cancer is the most common type of cancer in women, and early abnormality detection using mammography can significantly improve breast cancer survival rates. Diverse datasets are required to improve the training and validation of deep learning (DL) systems for autonomous breast cancer diagnosis. However, only a small number of mammography datasets are publicly available. This constraint has created challenges when comparing different DL models using the same dataset. The primary contribution of this study is the comprehensive description of a selection of currently available public mammography datasets. The information available on publicly accessible datasets is summarized and their usability reviewed to enable more effective models to be developed for breast cancer detection and to improve understanding of existing models trained using these datasets. This study aims to bridge the existing knowledge gap by offering researchers and practitioners a valuable resource to develop and assess DL models in breast cancer diagnosis.

Geographical discrimination of Asian red pepper powders using 1H NMR spectroscopy and deep learning-based convolution neural networks

This study investigated an innovative approach to discriminate the geographical origins of Asian red pepper powders by analyzing one-dimensional 1H NMR spectra through a deep learning-based convolution neural network (CNN). 1H NMR spectra were collected from 300 samples originating from China, Korea, and Vietnam and used as input data. Principal component analysis - linear discriminant analysis and support vector machine models were employed for comparison. Bayesian optimization was used for hyperparameter optimization, and cross-validation was performed to prevent overfitting. As a result, all three models discriminated the origins of the test samples with over 95 % accuracy. Specifically, the CNN models achieved a 100 % accuracy rate. Gradient-weighted class activation mapping analysis verified that the CNN models recognized the origins of the samples based on variations in metabolite distributions. This research demonstrated the potential of deep learning-based classification of 1H NMR spectra as an accurate and reliable approach for determining the geographical origins of various foods.

Toxicity prediction and classification of Gunqile-7 with small sample based on transfer learning method

Drug-induced diseases are the most important component of iatrogenic disease. It is the duty of doctors to provide a reasonable and safe dose of medication. Gunqile-7 is a Mongolian medicine with analgesic and anti-inflammatory effects. As a foreign substance in the body, even with reasonable medication, it may produce varying degrees of adverse reactions or toxic side effects. Since the cost of collecting Gunqile-7 for pharmacological animal trials is high and the data sample is small, this paper employs transfer learning and data augmentation methods to study the toxicity of Gunqile-7. More specifically, to reduce the necessary number of training samples, the data augmentation approach is employed to extend the data set. Then, the transfer learning method and one-dimensional convolutional neural network are utilized to train the network. In addition, we use the support vector machine-recursive feature elimination method for feature selection to reduce features that have adverse effects on model predictions. Furthermore, due to the important role of the pre-trained model of transfer learning, we select a quantitative toxicity prediction model as the pre-trained model, which is consistent with the purpose of this paper. Lastly, the experimental results demonstrate the efficiency of the proposed method. Our method can improve accuracy by up to 9 percentage points compared to the method without transfer learning on a small sample set.

Self-supervised learning for medical image data with anatomy-oriented imaging planes

Self-supervised learning has emerged as a powerful tool for pretraining deep networks on unlabeled data, prior to transfer learning of target tasks with limited annotation. The relevance between the pretraining pretext and target tasks is crucial to the success of transfer learning. Various pretext tasks have been proposed to utilize properties of medical image data (e.g., three dimensionality), which are more relevant to medical image analysis than generic ones for natural images. However, previous work rarely paid attention to data with anatomy-oriented imaging planes, e.g., standard cardiac magnetic resonance imaging views. As these imaging planes are defined according to the anatomy of the imaged organ, pretext tasks effectively exploiting this information can pretrain the networks to gain knowledge on the organ of interest. In this work, we propose two complementary pretext tasks for this group of medical image data based on the spatial relationship of the imaging planes. The first is to learn the relative orientation between the imaging planes and implemented as regressing their intersecting lines. The second exploits parallel imaging planes to regress their relative slice locations within a stack. Both pretext tasks are conceptually straightforward and easy to implement, and can be combined in multitask learning for better representation learning. Thorough experiments on two anatomical structures (heart and knee) and representative target tasks (semantic segmentation and classification) demonstrate that the proposed pretext tasks are effective in pretraining deep networks for remarkably boosted performance on the target tasks, and superior to other recent approaches.

Application of convolutional neural networks in medical images: a bibliometric analysis

Background

In the field of medical imaging, the rapid rise of convolutional neural networks (CNNs) has presented significant opportunities for conserving healthcare resources. However, with the wide spread application of CNNs, several challenges have emerged, such as enormous data annotation costs, difficulties in ensuring user privacy and security, weak model interpretability, and the consumption of substantial computational resources. The fundamental challenge lies in optimizing and seamlessly integrating CNN technology to enhance the precision and efficiency of medical diagnosis.

Methods

This study sought to provide a comprehensive bibliometric overview of current research on the application of CNNs in medical imaging. Initially, bibliometric methods were used to calculate the frequency statistics, and perform the cluster analysis and the co-citation analysis of countries, institutions, authors, keywords, and references. Subsequently, the latent Dirichlet allocation (LDA) method was employed for the topic modeling of the literature. Next, an in-depth analysis of the topics was conducted, and the topics in the medical field, technical aspects, and trends in topic evolution were summarized. Finally, by integrating the bibliometrics and LDA results, the developmental trajectory, milestones, and future directions in this field were outlined.

Results

A data set containing 6,310 articles in this field published from January 2013 to December 2023 was complied. With a total of 55,538 articles, the United States led in terms of the citation count, while in terms of the publication volume, China led with 2,385 articles. Harvard University emerged as the most influential institution, boasting an average of 69.92 citations per article. Within the realm of CNNs, residual neural network (ResNet) and U-Net stood out, receiving 1,602 and 1,419 citations, respectively, which highlights the significant attention these models have received. The impact of coronavirus disease 2019 (COVID-19) was unmistakable, as reflected by the publication of 597 articles, making it a focal point of research. Additionally, among various disease topics, with 290 articles, brain-related research was the most prevalent. Computed tomography (CT) imaging dominated the research landscape, representing 73% of the 30 different topics.

Conclusions

Over the past 11 years, CNN-related research in medical imaging has grown exponentially. The findings of the present study provide insights into the field's status and research hotspots. In addition, this article meticulously chronicled the development of CNNs and highlighted key milestones, starting with LeNet in 1989, followed by a challenging 20-year exploration period, and culminating in the breakthrough moment with AlexNet in 2012. Finally, this article explored recent advancements in CNN technology, including semi-supervised learning, efficient learning, trustworthy artificial intelligence (AI), and federated learning methods, and also addressed challenges related to data annotation costs, diagnostic efficiency, model performance, and data privacy.

Branched Convolutional Neural Networks for Receiver Channel Recovery in High-Frame-Rate Sparse-Array Ultrasound Imaging

For high-frame-rate ultrasound imaging, it remains challenging to implement on compact systems as a sparse imaging configuration with limited array channels. One key issue is that the resulting image quality is known to be mediocre not only because unfocused plane-wave excitations are used but also because grating lobes would emerge in sparse-array configurations. In this article, we present the design and use of a new channel recovery framework to infer full-array plane-wave channel datasets for periodically sparse arrays that operate with as few as one-quarter of the full-array aperture. This framework is based on a branched encoder-decoder convolutional neural network (CNN) architecture, which was trained using full-array plane-wave channel data collected from human carotid arteries (59 864 training acquisitions; 5-MHz imaging frequency; 20-MHz sampling rate; plane-wave steering angles between -15° and 15° in 1° increments). Three branched encoder-decoder CNNs were separately trained to recover missing channels after differing degrees of channelwise downsampling (2, 3, and 4 times). The framework's performance was tested on full-array and downsampled plane-wave channel data acquired from an in vitro point target, human carotid arteries, and human brachioradialis muscle. Results show that when inferred full-array plane-wave channel data were used for beamforming, spatial aliasing artifacts in the B-mode images were suppressed for all degrees of channel downsampling. In addition, the image contrast was enhanced compared with B-mode images obtained from beamforming with downsampled channel data. When the recovery framework was implemented on an RTX-2080 GPU, the three investigated degrees of downsampling all achieved the same inference time of 4 ms. Overall, the proposed framework shows promise in enhancing the quality of high-frame-rate ultrasound images generated using a sparse-array imaging setup.

CD-Net: Cascaded 3D Dilated convolutional neural network for pneumonia lesion segmentation

COVID-19 is a global pandemic that has caused significant global, social, and economic disruption. To effectively assist in screening and monitoring diagnosed cases, it is crucial to accurately segment lesions from Computer Tomography (CT) scans. Due to the lack of labeled data and the presence of redundant parameters in 3D CT, there are still significant challenges in diagnosing COVID-19 in related fields. To address the problem, we have developed a new model called the Cascaded 3D Dilated convolutional neural network (CD-Net) for directly processing CT volume data. To reduce memory consumption when cutting volume data into small patches, we initially design a cascade architecture in CD-Net to preserve global information. Then, we construct a Multi-scale Parallel Dilated Convolution (MPDC) block to aggregate features of different sizes and simultaneously reduce the parameters. Moreover, to alleviate the shortage of labeled data, we employ classical transfer learning, which requires only a small amount of data while achieving better performance. Experimental results conducted on the different public-available datasets verify that the proposed CD-Net has reduced the negative-positive ratio and outperformed other existing segmentation methods while requiring less data.

Ultrasound deep learning radiomics and clinical machine learning models to predict low nuclear grade, ER, PR, and HER2 receptor status in pure ductal carcinoma in situ

Background

Low nuclear grade ductal carcinoma in situ (DCIS) patients can adopt proactive management strategies to avoid unnecessary surgical resection. Different personalized treatment modalities may be selected based on the expression status of molecular markers, which is also predictive of different outcomes and risks of recurrence. DCIS ultrasound findings are mostly non mass lesions, making it difficult to determine boundaries. Currently, studies have shown that models based on deep learning radiomics (DLR) have advantages in automatic recognition of tumor contours. Machine learning models based on clinical imaging features can explain the importance of imaging features.

Methods

The available ultrasound data of 349 patients with pure DCIS confirmed by surgical pathology [54 low nuclear grade, 175 positive estrogen receptor (ER+), 163 positive progesterone receptor (PR+), and 81 positive human epidermal growth factor receptor 2 (HER2+)] were collected. Radiologists extracted ultrasonographic features of DCIS lesions based on the 5th Edition of Breast Imaging Reporting and Data System (BI-RADS). Patient age and BI-RADS characteristics were used to construct clinical machine learning (CML) models. The RadImageNet pretrained network was used for extracting radiomics features and as an input for DLR modeling. For training and validation datasets, 80% and 20% of the data, respectively, were used. Logistic regression (LR), support vector machine (SVM), random forest (RF), and eXtreme Gradient Boosting (XGBoost) algorithms were performed and compared for the final classification modeling. Each task used the area under the receiver operating characteristic curve (AUC) to evaluate the effectiveness of DLR and CML models.

Results

In the training dataset, low nuclear grade, ER+, PR+, and HER2+ DCIS lesions accounted for 19.20%, 65.12%, 61.21%, and 30.19%, respectively; the validation set, they consisted of 19.30%, 62.50%, 57.14%, and 30.91%, respectively. In the DLR models we developed, the best AUC values for identifying features were 0.633 for identifying low nuclear grade, completed by the XGBoost Classifier of ResNet50; 0.618 for identifying ER, completed by the RF Classifier of InceptionV3; 0.755 for identifying PR, completed by the XGBoost Classifier of InceptionV3; and 0.713 for identifying HER2, completed by the LR Classifier of ResNet50. The CML models had better performance than DLR in predicting low nuclear grade, ER+, PR+, and HER2+ DCIS lesions. The best AUC values by classification were as follows: for low nuclear grade by RF classification, AUC: 0.719; for ER+ by XGBoost classification, AUC: 0.761; for PR+ by XGBoost classification, AUC: 0.780; and for HER2+ by RF classification, AUC: 0.723.

Conclusions

Based on small-scale datasets, our study showed that the DLR models developed using RadImageNet pretrained network and CML models may help predict low nuclear grade, ER+, PR+, and HER2+ DCIS lesions so that patients benefit from hierarchical and personalized treatment.

Development and external validation of a transfer learning-based system for the pathological diagnosis of colorectal cancer: a large emulated prospective study

Background

The progress in Colorectal cancer (CRC) screening and management has resulted in an unprecedented caseload for histopathological diagnosis. While artificial intelligence (AI) presents a potential solution, the predominant emphasis on slide-level aggregation performance without thorough verification of cancer in each location, impedes both explainability and transparency. Effectively addressing these challenges is crucial to ensuring the reliability and efficacy of AI in histology applications.

Method

In this study, we created an innovative AI algorithm using transfer learning from a polyp segmentation model in endoscopy. The algorithm precisely localized CRC targets within 0.25 mm² grids from whole slide imaging (WSI). We assessed the CRC detection capabilities at this fine granularity and examined the influence of AI on the diagnostic behavior of pathologists. The evaluation utilized an extensive dataset comprising 858 consecutive patient cases with 1418 WSIs obtained from an external center.

Results

Our results underscore a notable sensitivity of 90.25% and specificity of 96.60% at the grid level, accompanied by a commendable area under the curve (AUC) of 0.962. This translates to an impressive 99.39% sensitivity at the slide level, coupled with a negative likelihood ratio of <0.01, signifying the dependability of the AI system to preclude diagnostic considerations. The positive likelihood ratio of 26.54, surpassing 10 at the grid level, underscores the imperative for meticulous scrutiny of any AI-generated highlights. Consequently, all four participating pathologists demonstrated statistically significant diagnostic improvements with AI assistance.

Conclusion

Our transfer learning approach has successfully yielded an algorithm that can be validated for CRC histological localizations in whole slide imaging. The outcome advocates for the integration of the AI system into histopathological diagnosis, serving either as a diagnostic exclusion application or a computer-aided detection (CADe) tool. This integration has the potential to alleviate the workload of pathologists and ultimately benefit patients.

A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image Classification

Deep semi-supervised learning (DSSL) is a machine learning paradigm that blends supervised and unsupervised learning techniques to improve the performance of various models in computer vision tasks. Medical image classification plays a crucial role in disease diagnosis, treatment planning, and patient care. However, obtaining labeled medical image data is often expensive and time-consuming for medical practitioners, leading to limited labeled datasets. DSSL techniques aim to address this challenge, particularly in various medical image tasks, to improve model generalization and performance. DSSL models leverage both the labeled information, which provides explicit supervision, and the unlabeled data, which can provide additional information about the underlying data distribution. That offers a practical solution to resource-intensive demands of data annotation, and enhances the model’s ability to generalize across diverse and previously unseen data landscapes. The present study provides a critical review of various DSSL approaches and their effectiveness and challenges in enhancing medical image classification tasks. The study categorized DSSL techniques into six classes: consistency regularization method, deep adversarial method, pseudo-learning method, graph-based method, multi-label method, and hybrid method. Further, a comparative analysis of performance for six considered methods is conducted using existing studies. The referenced studies have employed metrics such as accuracy, sensitivity, specificity, AUC-ROC, and F1 score to evaluate the performance of DSSL methods on different medical image datasets. Additionally, challenges of the datasets, such as heterogeneity, limited labeled data, and model interpretability, were discussed and highlighted in the context of DSSL for medical image classification. The current review provides future directions and considerations to researchers to further address the challenges and take full advantage of these methods in clinical practices.

Fertility Preservation in the Era of Immuno-Oncology: Lights and Shadows

In recent years, immuno-oncology has revolutionized the cancer treatment field by harnessing the immune system's power to counteract cancer cells. While this innovative approach holds great promise for improving cancer outcomes, it also raises important considerations related to fertility and reproductive toxicity. In fact, most young females receiving gonadotoxic anti-cancer treatments undergo iatrogenic ovarian exhaustion, resulting in a permanent illness that precludes the vocation of motherhood as a natural female sexual identity. Although commonly used, oocyte cryopreservation for future in vitro fertilization and even ovarian cortex transplantation are considered unsafe procedures in cancer patients due to their oncogenic risks; whereas, ovarian stem cells might support neo-oogenesis, providing a novel stemness model of regenerative medicine for future fertility preservation programs in oncology. Recent scientific evidence has postulated that immune checkpoint inhibitors (ICIs) might in some way reduce fertility by inducing either primary or secondary hypogonadism, whose incidence and mechanisms are not yet known. Therefore, considering the lack of data, it is currently not possible to define the most suitable FP procedure for young patients who are candidates for ICIs. In this report, we will investigate the few available data concerning the molecular regulation of ICI therapy and their resulting gonadal toxicity, to hypothesize the most suitable fertility preservation strategy for patients receiving these drugs.

Investigating the detection of breast cancer with deep transfer learning using ResNet18 and ResNet34

A lot of underdeveloped nations particularly in Africa struggle with cancer-related, deadly diseases. Particularly in women, the incidence of breast cancer is rising daily because of ignorance and delayed diagnosis. Only by correctly identifying and diagnosing cancer in its very early stages of development can be effectively treated. The classification of cancer can be accelerated and automated with the aid of computer-aided diagnosis and medical image analysis techniques. This research provides the use of transfer learning from a Residual Network 18 (ResNet18) and Residual Network 34 (ResNet34) architectures to detect breast cancer. The study examined how breast cancer can be identified in breast mammography pictures using transfer learning from ResNet18 and ResNet34, and developed a demo app for radiologists using the trained models with the best validation accuracy. 1, 200 datasets of breast x-ray mammography images from the National Radiological Society's (NRS) archives were employed in the study. The dataset was categorised as implant cancer negative, implant cancer positive, cancer negative and cancer positive in order to increase the consistency of x-ray mammography images classification and produce better features. For the multi-class classification of the images, the study gave an average accuracy for binary classification of benign or malignant cancer cases of 86.7% validation accuracy for ResNet34 and 92% validation accuracy for ResNet18. A prototype web application showcasing ResNet18 performance has been created. The acquired results show how transfer learning can improve the accuracy of breast cancer detection, providing invaluable assistance to medical professionals, particularly in an African scenario.

Exploring Types of Photonic Neural Networks for Imaging and Computing-A Review

Photonic neural networks (PNNs), utilizing light-based technologies, show immense potential in artificial intelligence (AI) and computing. Compared to traditional electronic neural networks, they offer faster processing speeds, lower energy usage, and improved parallelism. Leveraging light's properties for information processing could revolutionize diverse applications, including complex calculations and advanced machine learning (ML). Furthermore, these networks could address scalability and efficiency challenges in large-scale AI systems, potentially reshaping the future of computing and AI research. In this comprehensive review, we provide current, cutting-edge insights into diverse types of PNNs crafted for both imaging and computing purposes. Additionally, we delve into the intricate challenges they encounter during implementation, while also illuminating the promising perspectives they introduce to the field.

Evaluation of the consistency of the MRI- based AI segmentation cartilage model using the natural tibial plateau cartilage

OBJECTIVE

The study aims to evaluate the accuracy of an MRI-based artificial intelligence (AI) segmentation cartilage model by comparing it to the natural tibial plateau cartilage.

METHODS

This study included 33 patients (41 knees) with severe knee osteoarthritis scheduled to undergo total knee arthroplasty (TKA). All patients had a thin-section MRI before TKA. Our study is mainly divided into two parts: (i) In order to evaluate the MRI-based AI segmentation cartilage model's 2D accuracy, the natural tibial plateau was used as gold standard. The MRI-based AI segmentation cartilage model and the natural tibial plateau were represented in binary visualization (black and white) simulated photographed images by the application of Simulation Photography Technology. Both simulated photographed images were compared to evaluate the 2D Dice similarity coefficients (DSC). (ii) In order to evaluate the MRI-based AI segmentation cartilage model's 3D accuracy. Hand-crafted cartilage model based on knee CT was established. We used these hand-crafted CT-based knee cartilage model as gold standard to evaluate 2D and 3D consistency of between the MRI-based AI segmentation cartilage model and hand-crafted CT-based cartilage model. 3D registration technology was used for both models. Correlations between the MRI-based AI knee cartilage model and CT-based knee cartilage model were also assessed with the Pearson correlation coefficient.

RESULTS

The AI segmentation cartilage model produced reasonably high two-dimensional DSC. The average 2D DSC between MRI-based AI cartilage model and the tibial plateau cartilage is 0.83. The average 2D DSC between the AI segmentation cartilage model and the CT-based cartilage model is 0.82. As for 3D consistency, the average 3D DSC between MRI-based AI cartilage model and CT-based cartilage model is 0.52. However, the quantification of cartilage segmentation with the AI and CT-based models showed excellent correlation (r = 0.725; P values < 0.05).

CONCLUSION

Our study demonstrated that our MRI-based AI cartilage model can reliably extract morphologic features such as cartilage shape and defect location of the tibial plateau cartilage. This approach could potentially benefit clinical practices such as diagnosing osteoarthritis. However, in terms of cartilage thickness and three-dimensional accuracy, MRI-based AI cartilage model underestimate the actual cartilage volume. The previous AI verification methods may not be completely accurate and should be verified with natural cartilage images. Combining multiple verification methods will improve the accuracy of the AI model.

Deep transfer learning with gravitational search algorithm for enhanced plant disease classification

Plant diseases annually cause damage and loss of much of the crop, if not its complete destruction, and this constitutes a significant challenge for farm owners, governments, and consumers alike. Therefore, identifying and classifying diseases at an early stage is very important in order to sustain local and global food security. In this research, we designed a new method to identify plant diseases by combining transfer learning and Gravitational Search Algorithm (GSA). Two state-of-the-art pretrained models have been adopted for extracting features in this study, which are MobileNetV2 and ResNe50V2. Multilayer feature extraction is applied in this study to ensure representations of plant leaves from different levels of abstraction for precise classification. These features are then concatenated and passed to GSA for optimizing them. Finally, optimized features are passed to Multinomial Logistic Regression (MLR) for final classification. This integration is essential for categorizing 18 different types of infected and healthy leaf samples. The performance of our approach is strengthened by a comparative analysis that incorporates features optimized by the Genetic Algorithm (GA). Additionally, the MLR algorithm is contrasted with K-Nearest Neighbors (KNN). The empirical findings indicate that our model, which has been refined using GSA, achieves very high levels of precision. Specifically, the average precision for MLR is 99.2%, while for KNN it is 98.6%. The resulting results significantly exceed those achieved with GA-optimized features, thereby highlighting the superiority of our suggested strategy. One important result of our study is that we were able to decrease the number of features by more than 50%. This reduction greatly reduces the processing requirements without sacrificing the quality of the diagnosis. This work presents a robust and efficient approach to the early detection of plant diseases. The work demonstrates the utilization of sophisticated computational methods in agriculture, enabling the development of novel data-driven strategies for plant health management, therefore enhancing worldwide food security.

Tumor Segmentation in Intraoperative Fluorescence Images Based on Transfer Learning and Convolutional Neural Networks

OBJECTIVE

To propose a transfer learning based method of tumor segmentation in intraoperative fluorescence images, which will assist surgeons to efficiently and accurately identify the boundary of tumors of interest.

METHODS

We employed transfer learning and deep convolutional neural networks (DCNNs) for tumor segmentation. Specifically, we first pre-trained four networks on the ImageNet dataset to extract low-level features. Subsequently, we fine-tuned these networks on two fluorescence image datasets (ABFM and DTHP) separately to enhance the segmentation performance of fluorescence images. Finally, we tested the trained models on the DTHL dataset. The performance of this approach was compared and evaluated against DCNNs trained end-to-end and the traditional level-set method.

RESULTS

The transfer learning-based UNet++ model achieved high segmentation accuracies of 82.17% on the ABFM dataset, 95.61% on the DTHP dataset, and 85.49% on the DTHL test set. For the DTHP dataset, the pre-trained Deeplab v3 + network performed exceptionally well, with a segmentation accuracy of 96.48%. Furthermore, all models achieved segmentation accuracies of over 90% when dealing with the DTHP dataset.

CONCLUSION

To the best of our knowledge, this study explores tumor segmentation on intraoperative fluorescent images for the first time. The results show that compared to traditional methods, deep learning has significant advantages in improving segmentation performance. Transfer learning enables deep learning models to perform better on small-sample fluorescence image data compared to end-to-end training. This discovery provides strong support for surgeons to obtain more reliable and accurate image segmentation results during surgery.

Cervical lymph node metastasis prediction from papillary thyroid carcinoma US videos: a prospective multicenter study

BACKGROUND

Prediction of lymph node metastasis (LNM) is critical for individualized management of papillary thyroid carcinoma (PTC) patients to avoid unnecessary overtreatment as well as undesired under-treatment. Artificial intelligence (AI) trained by thyroid ultrasound (US) may improve prediction performance.

METHODS

From September 2017 to December 2018, patients with suspicious PTC from the first medical center of the Chinese PLA general hospital were retrospectively enrolled to pre-train the multi-scale, multi-frame, and dual-direction deep learning (MMD-DL) model. From January 2019 to July 2021, PTC patients from four different centers were prospectively enrolled to fine-tune and independently validate MMD-DL. Its diagnostic performance and auxiliary effect on radiologists were analyzed in terms of receiver operating characteristic (ROC) curves, areas under the ROC curve (AUC), accuracy, sensitivity, and specificity.

RESULTS

In total, 488 PTC patients were enrolled in the pre-training cohort, and 218 PTC patients were included for model fine-tuning (n = 109), internal test (n = 39), and external validation (n = 70). Diagnostic performances of MMD-DL achieved AUCs of 0.85 (95% CI: 0.73, 0.97) and 0.81 (95% CI: 0.73, 0.89) in the test and validation cohorts, respectively, and US radiologists significantly improved their average diagnostic accuracy (57% vs. 60%, P = 0.001) and sensitivity (62% vs. 65%, P < 0.001) by using the AI model for assistance.

CONCLUSIONS

The AI model using US videos can provide accurate and reproducible prediction of cervical lymph node metastasis in papillary thyroid carcinoma patients preoperatively, and it can be used as an effective assisting tool to improve diagnostic performance of US radiologists.

TRIAL REGISTRATION

We registered on the Chinese Clinical Trial Registry website with the number ChiCTR1900025592.

Automated detection and classification of concealed objects using infrared thermography and convolutional neural networks

This paper presents a study on the effectiveness of a convolutional neural network (CNN) in classifying infrared images for security scanning. Infrared thermography was explored as a non-invasive security scanner for stand-off and walk-through concealed object detection. Heat generated by human subjects radiates off the clothing surface, allowing detection by an infrared camera. However, infrared lacks in penetration capability compared to longer electromagnetic waves, leading to less obvious visuals on the clothing surface. ResNet-50 was used as the CNN model to automate the classification process of thermal images. The ImageNet database was used to pre-train the model, which was further fine-tuned using infrared images obtained from experiments. Four image pre-processing approaches were explored, i.e., raw infrared image, subject cropped region-of-interest (ROI) image, K-means, and Fuzzy-c clustered images. All these approaches were evaluated using the receiver operating characteristic curve on an internal holdout set, with an area-under-the-curve of 0.8923, 0.9256, 0.9485, and 0.9669 for the raw image, ROI cropped, K-means, and Fuzzy-c models, respectively. The CNN models trained using various image pre-processing approaches suggest that the prediction performance can be improved by the removal of non-decision relevant information and the visual highlighting of features.

Classification of Chest CT Lung Nodules Using Collaborative Deep Learning Model

Background

Early detection of lung cancer through accurate diagnosis of malignant lung nodules using chest CT scans offers patients the highest chance of successful treatment and survival. Despite advancements in computer vision through deep learning algorithms, the detection of malignant nodules faces significant challenges due to insufficient training datasets.

Methods

This study introduces a model based on collaborative deep learning (CDL) to differentiate between cancerous and non-cancerous nodules in chest CT scans with limited available data. The model dissects a nodule into its constituent parts using six characteristics, allowing it to learn detailed features of lung nodules. It utilizes a CDL submodel that incorporates six types of feature patches to fine-tune a network previously trained with ResNet-50. An adaptive weighting method learned through error backpropagation enhances the process of identifying lung nodules, incorporating these CDL submodels for improved accuracy.

Results

The CDL model demonstrated a high level of performance in classifying lung nodules, achieving an accuracy of 93.24%. This represents a significant improvement over current state-of-the-art methods, indicating the effectiveness of the proposed approach.

Conclusion

The findings suggest that the CDL model, with its unique structure and adaptive weighting method, offers a promising solution to the challenge of accurately detecting malignant lung nodules with limited data. This approach not only improves diagnostic accuracy but also contributes to the early detection and treatment of lung cancer, potentially saving lives.

Automatic and real-time tissue sensing for autonomous intestinal anastomosis using hybrid MLP-DC-CNN classifier-based optical coherence tomography

Anastomosis is a common and critical part of reconstructive procedures within gastrointestinal, urologic, and gynecologic surgery. The use of autonomous surgical robots such as the smart tissue autonomous robot (STAR) system demonstrates an improved efficiency and consistency of the laparoscopic small bowel anastomosis over the current da Vinci surgical system. However, the STAR workflow requires auxiliary manual monitoring during the suturing procedure to avoid missed or wrong stitches. To eliminate this monitoring task from the operators, we integrated an optical coherence tomography (OCT) fiber sensor with the suture tool and developed an automatic tissue classification algorithm for detecting missed or wrong stitches in real time. The classification results were updated and sent to the control loop of STAR robot in real time. The suture tool was guided to approach the object by a dual-camera system. If the tissue inside the tool jaw was inconsistent with the desired suture pattern, a warning message would be generated. The proposed hybrid multilayer perceptron dual-channel convolutional neural network (MLP-DC-CNN) classification platform can automatically classify eight different abdominal tissue types that require different suture strategies for anastomosis. In MLP, numerous handcrafted features (∼1955) were utilized including optical properties and morphological features of one-dimensional (1D) OCT A-line signals. In DC-CNN, intensity-based features and depth-resolved tissues' attenuation coefficients were fully exploited. A decision fusion technique was applied to leverage the information collected from both classifiers to further increase the accuracy. The algorithm was evaluated on 69,773 testing A-line data. The results showed that our model can classify the 1D OCT signals of small bowels in real time with an accuracy of 90.06%, a precision of 88.34%, and a sensitivity of 87.29%, respectively. The refresh rate of the displayed A-line signals was set as 300 Hz, the maximum sensing depth of the fiber was 3.6 mm, and the running time of the image processing algorithm was ∼1.56 s for 1,024 A-lines. The proposed fully automated tissue sensing model outperformed the single classifier of CNN, MLP, or SVM with optimized architectures, showing the complementarity of different feature sets and network architectures in classifying intestinal OCT A-line signals. It can potentially reduce the manual involvement of robotic laparoscopic surgery, which is a crucial step towards a fully autonomous STAR system.

A transformer-based pyramid network for coronary calcified plaque segmentation in intravascular optical coherence tomography images

Characterizing coronary calcified plaque (CCP) provides essential insight into diagnosis and treatment of atherosclerosis. Intravascular optical coherence tomography (OCT) offers significant advantages for detecting CCP and even automated segmentation with recent advances in deep learning techniques. Most of current methods have achieved promising results by adopting existing convolution neural networks (CNNs) in computer vision domain. However, their performance can be detrimentally affected by unseen plaque patterns and artifacts due to inherent limitation of CNNs in contextual reasoning. To overcome this obstacle, we proposed a Transformer-based pyramid network called AFS-TPNet for robust, end-to-end segmentation of CCP from OCT images. Its encoder is built upon CSWin Transformer architecture, allowing for better perceptual understanding of calcified arteries at a higher semantic level. Specifically, an augmented feature split (AFS) module and residual convolutional position encoding (RCPE) mechanism are designed to effectively enhance the capability of Transformer in capturing both fine-grained features and global contexts. Extensive experiments showed that AFS-TPNet trained using Lovasz Loss achieved superior performance in segmentation CCP under various contexts, surpassing prior state-of-the-art CNN and Transformer architectures by more than 6.58% intersection over union (IoU) score. The application of this promising method to extract CCP features is expected to enhance clinical intervention and translational research using OCT.

Deep learning for the identification of ridge deficiency around dental implants

OBJECTIVES

This study aimed to use a deep learning (DL) approach for the automatic identification of the ridge deficiency around dental implants based on an image slice from cone-beam computerized tomography (CBCT).

MATERIALS AND METHODS

Single slices crossing the central long-axis of 630 mandibular and 845 maxillary virtually placed implants (4-5 mm diameter, 10 mm length) in 412 patients were used. The ridges were classified based on the intraoral bone-implant support and sinus floor location. The slices were either preprocessed by alveolar ridge homogenizing prior to DL (preprocessed) or left unpreprocessed. A convolutional neural network with ResNet-50 architecture was employed for DL.

RESULTS

The model achieved an accuracy of >98.5% on the unpreprocessed image slices and was found to be superior to the accuracy observed on the preprocessed slices. On the mandible, model accuracy was 98.91 ± 1.45%, and F1 score, a measure of a model's accuracy in binary classification tasks, was lowest (97.30%) on the ridge with a combined horizontal-vertical defect. On the maxilla, model accuracy was 98.82 ± 1.11%, and the ridge presenting an implant collar-sinus floor distance of 5-10 mm with a dehiscence defect had the lowest F1 score (95.86%). To achieve >90% model accuracy, ≥441 mandibular slices or ≥592 maxillary slices were required.

CONCLUSIONS

The ridge deficiency around dental implants can be identified using DL from CBCT image slices without the need for preprocessed homogenization. The model will be further strengthened by implementing more clinical expertise in dental implant treatment planning and incorporating multiple slices to classify 3-dimensional implant-ridge relationships.

Automatic liver segmentation and assessment of liver fibrosis using deep learning with MR T1-weighted images in rats

OBJECTIVES

To validate the performance of nnU-Net in segmentation and CNN in classification for liver fibrosis using T1-weighted images.

MATERIALS AND METHODS

In this prospective study, animal models of liver fibrosis were induced by injecting subcutaneously a mixture of Carbon tetrachloride and olive oil. A total of 99 male Wistar rats were successfully induced and underwent MR scanning with no contrast agent to get T1-weighted images. The regions of interest (ROIs) of the whole liver were delineated layer by layer along the liver edge by 3D Slicer. For segmentation task, all T1-weighted images were randomly divided into training and test cohorts in a ratio of 7:3. For classification, images containing the hepatic maximum diameter of every rat were selected and 80% images of no liver fibrosis (NLF), early liver fibrosis (ELF) and progressive liver fibrosis (PLF) stages were randomly selected for training, while the rest were used for testing. Liver segmentation was performed by the nnU-Net model. The convolutional neural network (CNN) was used for classification task of liver fibrosis stages. The Dice similarity coefficient was used to evaluate the segmentation performance of nnU-Net. Confusion matrix, ROC curve and accuracy were used to show the classification performance of CNN.

RESULTS

A total of 2628 images were obtained from 99 Wistar rats by MR scanning. For liver segmentation by nnU-Net, the Dice similarity coefficient in the test set was 0.8477. The accuracies of CNN in staging NLF, ELF and PLF were 0.73, 0.89 and 0.84, respectively. The AUCs were 0.76, 0.88 and 0.79, respectively.

CONCLUSION

The nnU-Net architecture is of high accuracy for liver segmentation and CNN for assessment of liver fibrosis with T1-weighted images.