Establishment of a novel tumor neoantigen prediction tool for personalized vaccine design

The personalized neoantigen nanovaccine (PNVAC) platform for patients with gastric cancer we established previously exhibited promising anti-tumor immunoreaction. However, limited by the ability of traditional neoantigen prediction tools, a portion of epitopes failed to induce specific immune response. In order to filter out more neoantigens to optimize our PNVAC platform, we develop a novel neoantigen prediction model, NUCC. This prediction tool trained through a deep learning approach exhibits better neoantigen prediction performance than other prediction tools, not only in two independent epitope datasets, but also in a totally new epitope dataset we construct from scratch, including 25 patients with advance gastric cancer and 150 candidate mutant peptides, 13 of which prove to be neoantigen by immunogenicity test in vitro. Our work lay the foundation for the improvement of our PNVAC platform for gastric cancer in the future.

Artificial intelligence for skin permeability prediction: deep learning

BACKGROUND AND OBJECTIVE

Researchers have put in significant laboratory time and effort in measuring the permeability coefficient (Kp) of xenobiotics. To develop alternative approaches to this labour-intensive procedure, predictive models have been employed by scientists to describe the transport of xenobiotics across the skin. Most quantitative structure-permeability relationship (QSPR) models are derived statistically from experimental data. Recently, artificial intelligence-based computational drug delivery has attracted tremendous interest. Deep learning is an umbrella term for machine-learning algorithms consisting of deep neural networks (DNNs). Distinct network architectures, like convolutional neural networks (CNNs), feedforward neural networks (FNNs), and recurrent neural networks (RNNs), can be employed for prediction.

METHODS

In this project, we used a convolutional neural network, feedforward neural network, and recurrent neural network to predict skin permeability coefficients from a publicly available database reported by Cheruvu et al. The dataset contains 476 records of 145 chemicals, xenobiotics, and pharmaceuticals, administered on the human epidermis in vitro from aqueous solutions of constant concentration either saturated in infinite dose quantities or diluted. All the computations were conducted with Python under Anaconda and Jupyterlab environment after importing the required Python, Keras, and Tensorflow modules.

RESULTS

We used a convolutional neural network, feedforward neural network, and recurrent neural network to predict log kp.

CONCLUSION

This research work shows that deep learning networks can be successfully used to digitally screen and predict the skin permeability of xenobiotics.

SwinD-Net: a lightweight segmentation network for laparoscopic liver segmentation

The real-time requirement for image segmentation in laparoscopic surgical assistance systems is extremely high. Although traditional deep learning models can ensure high segmentation accuracy, they suffer from a large computational burden. In the practical setting of most hospitals, where powerful computing resources are lacking, these models cannot meet the real-time computational demands. We propose a novel network SwinD-Net based on Skip connections, incorporating Depthwise separable convolutions and Swin Transformer Blocks. To reduce computational overhead, we eliminate the skip connection in the first layer and reduce the number of channels in shallow feature maps. Additionally, we introduce Swin Transformer Blocks, which have a larger computational and parameter footprint, to extract global information and capture high-level semantic features. Through these modifications, our network achieves desirable performance while maintaining a lightweight design. We conduct experiments on the CholecSeg8k dataset to validate the effectiveness of our approach. Compared to other models, our approach achieves high accuracy while significantly reducing computational and parameter overhead. Specifically, our model requires only 98.82 M floating-point operations (FLOPs) and 0.52 M parameters, with an inference time of 47.49 ms per image on a CPU. Compared to the recently proposed lightweight segmentation network UNeXt, our model not only outperforms it in terms of the Dice metric but also has only 1/3 of the parameters and 1/22 of the FLOPs. In addition, our model achieves a 2.4 times faster inference speed than UNeXt, demonstrating comprehensive improvements in both accuracy and speed. Our model effectively reduces parameter count and computational complexity, improving the inference speed while maintaining comparable accuracy. The source code will be available at https://github.com/ouyangshuiming/SwinDNet.

Resolution-enhanced multi-core fiber imaging learned on a digital twin for cancer diagnosis

Significance

Deep learning enables label-free all-optical biopsies and automated tissue classification. Endoscopic systems provide intraoperative diagnostics to deep tissue and speed up treatment without harmful tissue removal. However, conventional multi-core fiber (MCF) endoscopes suffer from low resolution and artifacts, which hinder tumor diagnostics.

Aim

We introduce a method to enable unpixelated, high-resolution tumor imaging through a given MCF with a diameter of around 0.65 mm and arbitrary core arrangement and inhomogeneous transmissivity.

Approach

Image reconstruction is based on deep learning and the digital twin concept of the single-reference-based simulation with inhomogeneous optical properties of MCF and transfer learning on a small experimental dataset of biological tissue. The reference provided physical information about the MCF during the training processes.

Results

For the simulated data, hallucination caused by the MCF inhomogeneity was eliminated, and the averaged peak signal-to-noise ratio and structural similarity were increased from 11.2 dB and 0.20 to 23.4 dB and 0.74, respectively. By transfer learning, the metrics of independent test images experimentally acquired on glioblastoma tissue ex vivo can reach up to 31.6 dB and 0.97 with 14 fps computing speed.

Conclusions

With the proposed approach, a single reference image was required in the pre-training stage and laborious acquisition of training data was bypassed. Validation on glioblastoma cryosections with transfer learning on only 50 image pairs showed the capability for high-resolution deep tissue retrieval and high clinical feasibility.

The new era of artificial intelligence in neuroradiology: current research and promising tools

Radiology has a number of characteristics that make it an especially suitable medical discipline for early artificial intelligence (AI) adoption. These include having a well-established digital workflow, standardized protocols for image storage, and numerous well-defined interpretive activities. The more than 200 commercial radiologic AI-based products recently approved by the Food and Drug Administration (FDA) to assist radiologists in a number of narrow image-analysis tasks such as image enhancement, workflow triage, and quantification, corroborate this observation. However, in order to leverage AI to boost efficacy and efficiency, and to overcome substantial obstacles to widespread successful clinical use of these products, radiologists should become familiarized with the emerging applications in their particular areas of expertise. In light of this, in this article we survey the existing literature on the application of AI-based techniques in neuroradiology, focusing on conditions such as vascular diseases, epilepsy, and demyelinating and neurodegenerative conditions. We also introduce some of the algorithms behind the applications, briefly discuss a few of the challenges of generalization in the use of AI models in neuroradiology, and skate over the most relevant commercially available solutions adopted in clinical practice. If well designed, AI algorithms have the potential to radically improve radiology, strengthening image analysis, enhancing the value of quantitative imaging techniques, and mitigating diagnostic errors.

Integration of a deep learning basal cell carcinoma detection and tumor mapping algorithm into the Mohs micrographic surgery workflow and effects on clinical staffing: A simulated, retrospective study

Background

Artificial intelligence (AI) enabled tools have been proposed as 1 solution to improve health care delivery. However, research on downstream effects of AI integration into the clinical workflow is lacking.

Objective

We aim to analyze how integration of an automated basal cell carcinoma detection and tumor mapping algorithm in a Mohs micrographic surgery unit impacts the work efficiency of clinical and laboratory staff.

Methods

Slide, staff, and histotechnician waiting times were analyzed over a 20-day period in a Mohs micrographic surgery unit. A simulated AI workflow was created and the time differences between the real and simulated workflows were compared.

Results

Simulated nonautonomous algorithm integration led to savings of 35.6% of slide waiting time, 18.4% of staff waiting time, and 18.6% of histotechnician waiting time per day. Algorithm integration on days with increased reconstruction complexity resulted in the greatest time savings.

Limitations

One Mohs micrographic surgery unit was analyzed and simulated AI integration was performed retrospectively.

Conclusions

AI integration results in reduced staff waiting times, enabling increased productivity and a streamlined clinical workflow. Schedules containing surgical cases with either increased repair complexity or numerous tumor removal stages stand to benefit most. However, significant logistical challenges must be addressed before broad adoption into clinical practice is realistic.

Domain Adaptation-Based Deep Learning Model for Forecasting and Diagnosis of Glaucoma Disease

The main factor causing irreversible blindness is glaucoma. Early detection greatly reduces the risk of further vision loss. To address this problem, we developed a domain adaptation-based deep learning model called Glaucoma Domain Adaptation (GDA) based on 66,742 fundus photographs collected from 3272 eyes of 1636 subjects. GDA learns domain-invariant and domain-specific representations to extract both general and specific features. We also developed a progressive weighting mechanism to accurately transfer the source domain knowledge while mitigating the transfer of negative knowledge from the source to the target domain. We employed low-rank coding for aligning the source and target distributions. We trained GDA based on three different scenarios including eyes annotated as glaucoma due to 1) optic disc abnormalities regardless of visual field abnormalities, 2) optic disc or visual field abnormalities except ones that are glaucoma due to both optic disc and visual field abnormalities at the same time, and 3) visual field abnormalities regardless of optic disc abnormalities We then evaluate the generalizability of GDA based on two independent datasets. The AUCs of GDA in forecasting glaucoma based on the first, second, and third scenarios were 0.90, 0.88, and 0.80 and the Accuracies were 0.82, 0.78, and 0.72, respectively. The AUCs of GDA in diagnosing glaucoma based on the first, second, and third scenarios were 0.98, 0.96, and 0.93 and the Accuracies were 0.93, 0.91, and 0.88, respectively. The proposed GDA model achieved high performance and generalizability for forecasting and diagnosis of glaucoma disease from fundus photographs. GDA may augment glaucoma research and clinical practice in identifying patients with glaucoma and forecasting those who may develop glaucoma thus preventing future vision loss.

Automatic grading of intervertebral disc degeneration in lumbar dog spines

Background

Intervertebral disc degeneration is frequent in dogs and can be associated with symptoms and functional impairments. The degree of disc degeneration can be assessed on T2-weighted MRI scans using the Pfirrmann classification scheme, which was developed for the human spine. However, it could also be used to quantify the effectiveness of disc regeneration therapies. We developed and tested a deep learning tool able to automatically score the degree of disc degeneration in dog spines, starting from an existing model designed to process images of human patients.

Methods

MRI midsagittal scans of 5991 lumbar discs of dog patients were collected and manually evaluated with the Pfirrmann scheme and a modified scheme with transitional grades. A deep learning model was trained to classify the disc images based on the two schemes and tested by comparing its performance with the model processing human images.

Results

The determination of the Pfirrmann grade showed sensitivities higher than 83% for all degeneration grades, except for grade 5, which is rare in dog spines, and high specificities. In comparison, the correspondent human model had slightly higher sensitivities, on average 90% versus 85% for the canine model. The modified scheme with the fractional grades did not show significant advantages with respect to the original Pfirrmann grades.

Conclusions

The novel tool was able to accurately and reliably score the severity of disc degeneration in dogs, although with a performance inferior than that of the human model. The tool has potential in the clinical management of disc degeneration in canine patients as well as in longitudinal studies evaluating regenerative therapies in dogs used as animal models of human disorders.

A comparison of two learning approach inventories and their utility in predicting examination performance and study habits

The revised two-factor Study Process Questionnaire and the Approaches and Study Skills Inventory for Students are two instruments commonly used to measure student learning approach. Although they are designed to measure similar constructs, it is unclear whether the metrics they provide differ in terms of their real-world classification of learning approach. The purpose of this study is to compare outcomes of these two inventories in a study population from an undergraduate (baccalaureate) human anatomy course. The three central goals of this study are to compare the inventories in terms of 1) how students are classified, 2) the relationship between examination performance, time spent studying, and learning approach, and 3) instrument reliability. Results demonstrate that student classifications of corresponding scales of each inventory are significantly correlated, suggesting they measure similar constructs. Although the inventories had similar reliability, neither was consistently strong in predicting examination performance or study habits. Overall, these results suggest that the two inventories are comparable in terms of how they measure learning approach, but the lack of correspondence between learning approach scores and measurement outcomes questions their validity as tools that can be used universally in classrooms.NEW & NOTEWORTHY Although learning approach inventories have been used extensively in education research, there has been no direct comparison of how student classification differs between instruments or how classification influences the interpretation of how learning approach impacts student performance. This is especially relevant in light of recent research questioning the validity of the Study Process Questionnaire (LoGiudice AB, Norman GR, Manzoor S, Monteiro S. Adv Health Sci Educ Theory Pract 28: 47-63, 2023; Johnson SN, Gallagher ED, Vagnozzi AM. PLoS One 16: e0250600, 2021).

JUST-Net: Jointly unrolled cross-domain optimization based spatio-temporal reconstruction network for accelerated 3D myelin water imaging

PURPOSE

We introduced a novel reconstruction network, jointly unrolled cross-domain optimization-based spatio-temporal reconstruction network (JUST-Net), aimed at accelerating 3D multi-echo gradient-echo (mGRE) data acquisition and improving the quality of resulting myelin water imaging (MWI) maps.

METHOD

An unrolled cross-domain spatio-temporal reconstruction network was designed. The main idea is to combine frequency and spatio-temporal image feature representations and to sequentially implement convolution layers in both domains. The k-space subnetwork utilizes shared information from adjacent frames, whereas the image subnetwork applies separate convolutions in both spatial and temporal dimensions. The proposed reconstruction network was evaluated for both retrospectively and prospectively accelerated acquisition. Furthermore, it was assessed in simulation studies and real-world cases with k-space corruptions to evaluate its potential for motion artifact reduction.

RESULTS

The proposed JUST-Net enabled highly reproducible and accelerated 3D mGRE acquisition for whole-brain MWI, reducing the acquisition time from fully sampled 15:23 to 2:22 min within a 3-min reconstruction time. The normalized root mean squared error of the reconstructed mGRE images increased by less than 4.0%, and the correlation coefficients for MWI showed a value of over 0.68 when compared to the fully sampled reference. Additionally, the proposed method demonstrated a mitigating effect on both simulated and clinical motion-corrupted cases.

CONCLUSION

The proposed JUST-Net has demonstrated the capability to achieve high acceleration factors for 3D mGRE-based MWI, which is expected to facilitate widespread clinical applications of MWI.

Unsupervised deep learning with convolutional neural networks for static parallel transmit design: A retrospective study

PURPOSE

To mitigateB 1 + $$ {B}_1^{+} $$ inhomogeneity at 7T for multi-channel transmit arrays using unsupervised deep learning with convolutional neural networks (CNNs).

METHODS

Deep learning parallel transmit (pTx) pulse design has received attention, but such methods have relied on supervised training and did not use CNNs for multi-channelB 1 + $$ {B}_1^{+} $$ maps. In this work, we introduce an alternative approach that facilitates the use of CNNs with multi-channelB 1 + $$ {B}_1^{+} $$ maps while performing unsupervised training. The multi-channelB 1 + $$ {B}_1^{+} $$ maps are concatenated along the spatial dimension to enable shift-equivariant processing amenable to CNNs. Training is performed in an unsupervised manner using a physics-driven loss function that minimizes the discrepancy of the Bloch simulation with the target magnetization, which eliminates the calculation of reference transmit RF weights. The training database comprises 3824 2D sagittal, multi-channelB 1 + $$ {B}_1^{+} $$ maps of the healthy human brain from 143 subjects.B 1 + $$ {B}_1^{+} $$ data were acquired at 7T using an 8Tx/32Rx head coil. The proposed method is compared to the unregularized magnitude least-squares (MLS) solution for the target magnetization in static pTx design.

RESULTS

The proposed method outperformed the unregularized MLS solution for RMS error and coefficient-of-variation and had comparable energy consumption. Additionally, the proposed method did not show local phase singularities leading to distinct holes in the resulting magnetization unlike the unregularized MLS solution.

CONCLUSION

Proposed unsupervised deep learning with CNNs performs better than unregularized MLS in static pTx for speed and robustness.

Automated curation of large-scale cancer histopathology image datasets using deep learning

BACKGROUND

Artificial intelligence (AI) has numerous applications in pathology, supporting diagnosis and prognostication in cancer. However, most AI models are trained on highly selected data, typically one tissue slide per patient. In reality, especially for large surgical resection specimens, dozens of slides can be available for each patient. Manually sorting and labelling whole-slide images (WSIs) is a very time-consuming process, hindering the direct application of AI on the collected tissue samples from large cohorts. In this study we addressed this issue by developing a deep-learning (DL)-based method for automatic curation of large pathology datasets with several slides per patient.

METHODS

We collected multiple large multicentric datasets of colorectal cancer histopathological slides from the United Kingdom (FOXTROT, N = 21,384 slides; CR07, N = 7985 slides) and Germany (DACHS, N = 3606 slides). These datasets contained multiple types of tissue slides, including bowel resection specimens, endoscopic biopsies, lymph node resections, immunohistochemistry-stained slides, and tissue microarrays. We developed, trained, and tested a deep convolutional neural network model to predict the type of slide from the slide overview (thumbnail) image. The primary statistical endpoint was the macro-averaged area under the receiver operating curve (AUROCs) for detection of the type of slide.

RESULTS

In the primary dataset (FOXTROT), with an AUROC of 0.995 [95% confidence interval [CI]: 0.994-0.996] the algorithm achieved a high classification performance and was able to accurately predict the type of slide from the thumbnail image alone. In the two external test cohorts (CR07, DACHS) AUROCs of 0.982 [95% CI: 0.979-0.985] and 0.875 [95% CI: 0.864-0.887] were observed, which indicates the generalizability of the trained model on unseen datasets. With a confidence threshold of 0.95, the model reached an accuracy of 94.6% (7331 classified cases) in CR07 and 85.1% (2752 classified cases) for the DACHS cohort.

CONCLUSION

Our findings show that using the low-resolution thumbnail image is sufficient to accurately classify the type of slide in digital pathology. This can support researchers to make the vast resource of existing pathology archives accessible to modern AI models with only minimal manual annotations.

Edge Computing Transformers for Fall Detection in Older Adults

The global trend of increasing life expectancy introduces new challenges with far-reaching implications. Among these, the risk of falls among older adults is particularly significant, affecting individual health and the quality of life, and placing an additional burden on healthcare systems. Existing fall detection systems often have limitations, including delays due to continuous server communication, high false-positive rates, low adoption rates due to wearability and comfort issues, and high costs. In response to these challenges, this work presents a reliable, wearable, and cost-effective fall detection system. The proposed system consists of a fit-for-purpose device, with an embedded algorithm and an Inertial Measurement Unit (IMU), enabling real-time fall detection. The algorithm combines a Threshold-Based Algorithm (TBA) and a neural network with low number of parameters based on a Transformer architecture. This system demonstrates notable performance with 95.29% accuracy, 93.68% specificity, and 96.66% sensitivity, while only using a 0.38% of the trainable parameters used by the other approach.

Automatic Identification and Severity Classification of Retinal Biomarkers in SD-OCT Using Dilated Depthwise Separable Convolution ResNet with SVM Classifier

PURPOSE

Diagnosis of Uveitic Macular Edema (UME) using Spectral Domain OCT (SD-OCT) is a promising method for early detection and monitoring of sight-threatening visual impairment. Viewing multiple B-scans and identifying biomarkers is challenging and time-consuming for clinical practitioners. To overcome these challenges, this paper proposes an image classification hybrid framework for predicting the presence of biomarkers such as intraretinal cysts (IRC), hyperreflective foci (HRF), hard exudates (HE) and neurosensory detachment (NSD) in OCT B-scans along with their severity.

METHODS

A dataset of 10880 B-scans from 85 Uveitic patients is collected and graded by two board-certified ophthalmologists for the presence of biomarkers. A novel image classification framework, Dilated Depthwise Separable Convolution ResNet (DDSC-RN) with SVM classifier, is developed to achieve network compression with a larger receptive field that captures both low and high-level features of the biomarkers without loss of classification accuracy. The severity level of each biomarker is predicted from the feature map, extracted by the proposed DDSC-RN network.

RESULTS

The proposed hybrid model is evaluated using ground truth labels from the hospital. The deep learning model initially, identified the presence of biomarkers in B-scans. It achieved an overall accuracy of 98.64%, which is comparable to the performance of other state-of-the-art models, such as DRN-C-42 and ResNet-34. The SVM classifier then predicted the severity of each biomarker, achieving an overall accuracy of 89.3%.

CONCLUSIONS

A new hybrid model accurately identifies four retinal biomarkers on a tissue map and predicts their severity. The model outperforms other methods for identifying multiple biomarkers in complex OCT B-scans. This helps clinicians to screen multiple B-scans of UME more effectively, leading to better treatment outcomes.

Deep hybrid modeling of a HEK293 process: Combining long short-term memory networks with first principles equations

The combination of physical equations with deep learning is becoming a promising methodology for bioprocess digitalization. In this paper, we investigate for the first time the combination of long short-term memory (LSTM) networks with first principles equations in a hybrid workflow to describe human embryonic kidney 293 (HEK293) culture dynamics. Experimental data of 27 extracellular state variables in 20 fed-batch HEK293 cultures were collected in a parallel high throughput 250 mL cultivation system in an industrial process development setting. The adaptive moment estimation method with stochastic regularization and cross-validation were employed for deep learning. A total of 784 hybrid models with varying deep neural network architectures, depths, layers sizes and node activation functions were compared. In most scenarios, hybrid LSTM models outperformed classical hybrid Feedforward Neural Network (FFNN) models in terms of training and testing error. Hybrid LSTM models revealed to be less sensitive to data resampling than FFNN hybrid models. As disadvantages, Hybrid LSTM models are in general more complex (higher number of parameters) and have a higher computation cost than FFNN hybrid models. The hybrid model with the highest prediction accuracy consisted in a LSTM network with seven internal states connected in series with dynamic material balance equations. This hybrid model correctly predicted the dynamics of the 27 state variables (R2 = 0.93 in the test data set), including biomass, key substrates, amino acids and metabolic by-products for around 10 cultivation days.

Many direct-to-consumer canine genetic tests can identify the breed of purebred dogs

OBJECTIVE

To compare pedigree documentation and genetic test results to evaluate whether user-provided photographs influence the breed ancestry predictions of direct-to-consumer (DTC) genetic tests for dogs.

ANIMALS

12 registered purebred pet dogs representing 12 different breeds.

METHODS

Each dog owner submitted 6 buccal swabs, 1 to each of 6 DTC genetic testing companies. Experimenters registered each sample per manufacturer instructions. For half of the dogs, the registration included a photograph of the DNA donor. For the other half of the dogs, photographs were swapped between dogs. DNA analysis and breed ancestry prediction were conducted by each company. The effect of condition (ie, matching vs shuffled photograph) was evaluated for each company's breed predictions. As a positive control, a convolutional neural network was also used to predict breed based solely on the photograph.

RESULTS

Results from 5 of the 6 tests always included the dog's registered breed. One test and the convolutional neural network were unlikely to identify the registered breed and frequently returned results that were more similar to the photograph than the DNA. Additionally, differences in the predictions made across all tests underscored the challenge of identifying breed ancestry, even in purebred dogs.

CLINICAL RELEVANCE

Veterinarians are likely to encounter patients who have conducted DTC genetic testing and may be asked to explain the results of genetic tests they did not order. This systematic comparison of commercially available tests provides context for interpreting results from consumer-grade DTC genetic testing kits.

Predicting dynamic, motion-related changes in B0 field in the brain at a 7T MRI using a subject-specific fine-trained U-net

PURPOSE

Subject movement during the MR examination is inevitable and causes not only image artifacts but also deteriorates the homogeneity of the main magnetic field (B0 ), which is a prerequisite for high quality data. Thus, characterization of changes to B0 , for example induced by patient movement, is important for MR applications that are prone to B0 inhomogeneities.

METHODS

We propose a deep learning based method to predict such changes within the brain from the change of the head position to facilitate retrospective or even real-time correction. A 3D U-net was trained on in vivo gradient-echo brain 7T MRI data. The input consisted of B0 maps and anatomical images at an initial position, and anatomical images at a different head position (obtained by applying a rigid-body transformation on the initial anatomical image). The output consisted of B0 maps at the new head positions. We further fine-trained the network weights to each subject by measuring a limited number of head positions of the given subject, and trained the U-net with these data.

RESULTS

Our approach was compared to established dynamic B0 field mapping via interleaved navigators, which suffer from limited spatial resolution and the need for undesirable sequence modifications. Qualitative and quantitative comparison showed similar performance between an interleaved navigator-equivalent method and proposed method.

CONCLUSION

It is feasible to predict B0 maps from rigid subject movement and, when combined with external tracking hardware, this information could be used to improve the quality of MR acquisitions without the use of navigators.

Optimization-enabled deep learning model for disease detection in IoT platform

Nowadays, Internet of things (IoT) and IoT platforms are extensively utilized in several healthcare applications. The IoT devices produce a huge amount of data in healthcare field that can be inspected on an IoT platform. In this paper, a novel algorithm, named artificial flora optimization-based chameleon swarm algorithm (AFO-based CSA), is developed for optimal path finding. Here, data are collected by the sensors and transmitted to the base station (BS) using the proposed AFO-based CSA, which is derived by integrating artificial flora optimization (AFO) in chameleon swarm algorithm (CSA). This integration refers to the AFO-based CSA model enhancing the strengths and features of both AFO and CSA for optimal routing of medical data in IoT. Moreover, the proposed AFO-based CSA algorithm considers factors such as energy, delay, and distance for the effectual routing of data. At BS, prediction is conducted, followed by stages, like pre-processing, feature dimension reduction, adopting Pearson's correlation, and disease detection, done by recurrent neural network, which is trained by the proposed AFO-based CSA. Experimental result exhibited that the performance of the proposed AFO-based CSA is superior to competitive approaches based on the energy consumption (0.538 J), accuracy (0.950), sensitivity (0.965), and specificity (0.937).

Stenosis Detection and Quantification of Coronary Artery Using Machine Learning and Deep Learning

The aim of this review is to introduce some applications of artificial intelligence (AI) algorithms for the detection and quantification of coronary stenosis using computed tomography angiography (CTA). The realization of automatic/semi-automatic stenosis detection and quantification includes the following steps: vessel central axis extraction, vessel segmentation, stenosis detection, and quantification. Many new AI techniques, such as machine learning and deep learning, have been widely used in medical image segmentation and stenosis detection. This review also summarizes the recent progress regarding coronary stenosis detection and quantification, and discusses the development trends in this field. Through evaluation and comparison, researchers can better understand the research frontier in related fields, compare the advantages and disadvantages of various methods, and better optimize the new technologies. Machine learning and deep learning will promote the process of automatic detection and quantification of coronary artery stenosis. However, the machine learning and the deep learning methods need a large amount of data, so they also face some challenges because of the lack of professional image annotations (manually add labels by experts).

DLgram cloud service for deep-learning analysis of microscopy images

To analyze images in various fields of science and technology, it is often necessary to count observed objects and determine their parameters. This can be quite labor-intensive and time-consuming. This article presents DLgram, a universal, user-friendly cloud service that is developed for this purpose. It is based on deep learning technologies and does not require programming skills. The user labels several objects in the image and uploads it to the cloud where the neural network is trained to recognize the objects being studied. The user receives recognition results, which if necessary, can be corrected, errors removed, or missing objects added. In addition, it is possible to carry out mathematical processing of the data obtained to get information about the sizes, areas, and coordinates of the observed objects. The article describes the service features and discusses examples of its application. The DLgram service allows to reduce significantly the time spent on quantitative image analysis, reduce subjective factor influence, and increase the accuracy of analysis. RESEARCH HIGHLIGHTS: DLgram automatically recognizes and counts the number of objects in images and their parameters. DLgram is a universal service, which was created on the basis of the latest deep learning developments and does not require programming skills.

Automatic Recognition of Concealed Fish Bones under Laryngoscopy: A Practical AI Model Based on YOLO-V5

BACKGROUND

Fish bone impaction is one of the most common problems encountered in otolaryngology emergencies. Due to their small and transparent nature, as well as the complexity of pharyngeal anatomy, identifying fish bones efficiently under laryngoscopy requires substantial clinical experience. This study aims to create an AI model to assist clinicians in detecting pharyngeal fish bones more efficiently under laryngoscopy.

METHODS

Totally 3133 laryngoscopic images related to fish bones were collected for model training and validation. The images in the training dataset were trained using the YOLO-V5 algorithm model. After training, the model was validated and its performance was evaluated using a test dataset. The model's predictions were compared to those of human experts. Seven laryngoscopic videos related to fish bone were used to validate real-time target detection by the model.

RESULTS

The model trained in YOLO-V5 demonstrated good generalization and performance, with an average precision of 0.857 when the intersection over union (IOU) threshold was set to 0.5. The precision, recall rate, and F1 scores of the model are 0.909, 0.818, and 0.87, respectively. The overall accuracy of the model in the validation set was 0.821, comparable to that of ENT specialists. The model processed each image in 0.012 s, significantly faster than human processing (p < 0.001). Furthermore, the model exhibited outstanding performance in video recognition.

CONCLUSION

Our AI model based on YOLO-V5 effectively identifies and localizes fish bone foreign bodies in static laryngoscopic images and dynamic videos. It shows great potential for clinical application.

LEVEL OF EVIDENCE

3 Laryngoscope, 134:2162-2169, 2024.

Multitask Deep Learning-Based Whole-Process System for Automatic Diagnosis of Breast Lesions and Axillary Lymph Node Metastasis Discrimination from Dynamic Contrast-Enhanced-MRI: A Multicenter Study

BACKGROUND

Accurate diagnosis of breast lesions and discrimination of axillary lymph node (ALN) metastases largely depend on radiologist experience.

PURPOSE

To develop a deep learning-based whole-process system (DLWPS) for segmentation and diagnosis of breast lesions and discrimination of ALN metastasis.

STUDY TYPE

Retrospective.

POPULATION

1760 breast patients, who were divided into training and validation sets (1110 patients), internal (476 patients), and external (174 patients) test sets.

FIELD STRENGTH/SEQUENCE

3.0T/dynamic contrast-enhanced (DCE)-MRI sequence.

ASSESSMENT

DLWPS was developed using segmentation and classification models. The DLWPS-based segmentation model was developed by the U-Net framework, which combined the attention module and the edge feature extraction module. The average score of the output scores of three networks was used as the result of the DLWPS-based classification model. Moreover, the radiologists' diagnosis without and with the DLWPS-assistance was explored. To reveal the underlying biological basis of DLWPS, genetic analysis was performed based on RNA-sequencing data.

STATISTICAL TESTS

Dice similarity coefficient (DI), area under receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and kappa value.

RESULTS

The segmentation model reached a DI of 0.828 and 0.813 in the internal and external test sets, respectively. Within the breast lesions diagnosis, the DLWPS achieved AUCs of 0.973 in internal test set and 0.936 in external test set. For ALN metastasis discrimination, the DLWPS achieved AUCs of 0.927 in internal test set and 0.917 in external test set. The agreement of radiologists improved with the DLWPS-assistance from 0.547 to 0.794, and from 0.848 to 0.892 in breast lesions diagnosis and ALN metastasis discrimination, respectively. Additionally, 10 breast cancers with ALN metastasis were associated with pathways of aerobic electron transport chain and cytoplasmic translation.

DATA CONCLUSION

The performance of DLWPS indicates that it can promote radiologists in the judgment of breast lesions and ALN metastasis and nonmetastasis.

LEVEL OF EVIDENCE

4 TECHNICAL EFFICACY STAGE: 3.

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

PURPOSE

This study aimed to perform multimodal analysis by vision transformer (vViT) in predicting O6-methylguanine-DNA methyl transferase (MGMT) promoter status among adult patients with diffuse glioma using demographics (sex and age), radiomic features, and MRI.

METHODS

The training and test datasets contained 122 patients with 1,570 images and 30 patients with 484 images, respectively. The radiomic features were extracted from enhancing tumors (ET), necrotic tumor cores (NCR), and the peritumoral edematous/infiltrated tissues (ED) using contrast-enhanced T1-weighted images (CE-T1WI) and T2-weighted images (T2WI). The vViT had 9 sectors; 1 demographic sector, 6 radiomic sectors (CE-T1WI ET, CE-T1WI NCR, CE-T1WI ED, T2WI ET, T2WI NCR, and T2WI ED), 2 image sectors (CE-T1WI, and T2WI). Accuracy and area under the curve of receiver-operating characteristics (AUC-ROC) were calculated for the test dataset. The performance of vViT was compared with AlexNet, GoogleNet, VGG16, and ResNet by McNemar and Delong test. Permutation importance (PI) analysis with the Mann-Whitney U test was performed.

RESULTS

The accuracy was 0.833 (95% confidence interval [95%CI]: 0.714-0.877) and the area under the curve of receiver-operating characteristics was 0.840 (0.650-0.995) in the patient-based analysis. The vViT had higher accuracy than VGG16 and ResNet, and had higher AUC-ROC than GoogleNet (p<0.05). The ED radiomic features extracted from the T2-weighted image demonstrated the highest importance (PI=0.239, 95%CI: 0.237-0.240) among all other sectors (p<0.0001).

CONCLUSION

The vViT is a competent deep learning model in predicting MGMT status. The ED radiomic features of the T2-weighted image demonstrated the most dominant contribution.

Synthesized 7T MPRAGE From 3T MPRAGE Using Generative Adversarial Network and Validation in Clinical Brain Imaging: A Feasibility Study

BACKGROUND

Ultra-high field 7T MRI can provide excellent tissue contrast and anatomical details, but is often cost prohibitive, and is not widely accessible in clinical practice.

PURPOSE

To generate synthetic 7T images from widely acquired 3T images with deep learning and to evaluate the feasibility of this approach for brain imaging.

STUDY TYPE

Prospective.

POPULATION

33 healthy volunteers and 89 patients with brain diseases, divided into training, and evaluation datasets in the ratio 4:1.

SEQUENCE AND FIELD STRENGTH

T1-weighted nonenhanced or contrast-enhanced magnetization-prepared rapid acquisition gradient-echo sequence at both 3T and 7T.

ASSESSMENT

A generative adversarial network (SynGAN) was developed to produce synthetic 7T images from 3T images as input. SynGAN training and evaluation were performed separately for nonenhanced and contrast-enhanced paired acquisitions. Qualitative image quality of acquired 3T and 7T images and of synthesized 7T images was evaluated by three radiologists in terms of overall image quality, artifacts, sharpness, contrast, and visualization of vessel using 5-point Likert scales.

STATISTICAL TESTS

Wilcoxon signed rank tests to compare synthetic 7T images with acquired 7T and 3T images and intraclass correlation coefficients to evaluate interobserver variability. P < 0.05 was considered significant.

RESULTS

Of the 122 paired 3T and 7T MRI scans, 66 were acquired without contrast agent and 56 with contrast agent. The average time to generate synthetic images was ~11.4 msec per slice (2.95 sec per participant). The synthetic 7T images achieved significantly improved tissue contrast and sharpness in comparison to 3T images in both nonenhanced and contrast-enhanced subgroups. Meanwhile, there was no significant difference between acquired 7T and synthetic 7T images in terms of all the evaluation criteria for both nonenhanced and contrast-enhanced subgroups (P ≥ 0.180).

DATA CONCLUSION

The deep learning model has potential to generate synthetic 7T images with similar image quality to acquired 7T images.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 1.

Artificial intelligence in serum protein electrophoresis: history, state of the art, and perspective

Serum protein electrophoresis (SPEP) is a valuable laboratory test that separates proteins from the blood based on their electrical charge and size. The test can detect and analyze various protein abnormalities, and the interpretation of graphic SPEP features plays a crucial role in the diagnosis and monitoring of conditions, such as myeloma. Furthermore, the advancement of artificial intelligence (AI) technology presents an opportunity to enhance the organization and optimization of analytical procedures by streamlining the process and reducing the potential for human error in SPEP analysis, thereby making the process more efficient and reliable. For instance, AI can assist in the identification of protein peaks, the calculation of their relative proportions, and the detection of abnormalities or inconsistencies. This review explores the characteristics and limitations of AI in SPEP, and the role of standardization in improving its clinical utility. It also offers guidance on the rational ordering and interpreting of SPEP results in conjunction with AI. Such integration can effectively reduce the time and resources required for manual analysis while improving the accuracy and consistency of the results.

Prediction of immunochemotherapy response for diffuse large B-cell lymphoma using artificial intelligence digital pathology

Diffuse large B-cell lymphoma (DLBCL) is a heterogeneous and prevalent subtype of aggressive non-Hodgkin lymphoma that poses diagnostic and prognostic challenges, particularly in predicting drug responsiveness. In this study, we used digital pathology and deep learning to predict responses to immunochemotherapy in patients with DLBCL. We retrospectively collected 251 slide images from 216 DLBCL patients treated with rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone (R-CHOP), with their immunochemotherapy response labels. The digital pathology images were processed using contrastive learning for feature extraction. A multi-modal prediction model was developed by integrating clinical data and pathology image features. Knowledge distillation was employed to mitigate overfitting on gigapixel histopathology images to create a model that predicts responses based solely on pathology images. Based on the importance derived from the attention mechanism of the model, we extracted histological features that were considered key textures associated with drug responsiveness. The multi-modal prediction model achieved an impressive area under the ROC curve of 0.856, demonstrating significant associations with clinical variables such as Ann Arbor stage, International Prognostic Index, and bulky disease. Survival analyses indicated their effectiveness in predicting relapse-free survival. External validation using TCGA datasets supported the model's ability to predict survival differences. Additionally, pathology-based predictions show promise as independent prognostic indicators. Histopathological analysis identified centroblastic and immunoblastic features to be associated with treatment response, aligning with previous morphological classifications and highlighting the objectivity and reproducibility of artificial intelligence-based diagnosis. This study introduces a novel approach that combines digital pathology and clinical data to predict the response to immunochemotherapy in patients with DLBCL. This model shows great promise as a diagnostic and prognostic tool for clinical management of DLBCL. Further research and genomic data integration hold the potential to enhance its impact on clinical practice, ultimately improving patient outcomes.

The macular retinal ganglion cell layer as a biomarker for diagnosis and prognosis in multiple sclerosis: A deep learning approach

PURPOSE

The macular ganglion cell layer (mGCL) is a strong potential biomarker of axonal degeneration in multiple sclerosis (MS). For this reason, this study aims to develop a computer-aided method to facilitate diagnosis and prognosis in MS.

METHODS

This paper combines a cross-sectional study of 72 MS patients and 30 healthy control subjects for diagnosis and a 10-year longitudinal study of the same MS patients for the prediction of disability progression, during which the mGCL was measured using optical coherence tomography (OCT). Deep neural networks were used as an automatic classifier.

RESULTS

For MS diagnosis, greatest accuracy (90.3%) was achieved using 17 features as inputs. The neural network architecture comprised the input layer, two hidden layers and the output layer with softmax activation. For the prediction of disability progression 8 years later, accuracy of 81.9% was achieved with a neural network comprising two hidden layers and 400 epochs.

CONCLUSION

We present evidence that by applying deep learning techniques to clinical and mGCL thickness data it is possible to identify MS and predict the course of the disease. This approach potentially constitutes a non-invasive, low-cost, easy-to-implement and effective method.

AI-guided histopathology predicts brain metastasis in lung cancer patients

Brain metastases can occur in nearly half of patients with early and locally advanced (stage I-III) non-small cell lung cancer (NSCLC). There are no reliable histopathologic or molecular means to identify those who are likely to develop brain metastases. We sought to determine if deep learning (DL) could be applied to routine H&E-stained primary tumor tissue sections from stage I-III NSCLC patients to predict the development of brain metastasis. Diagnostic slides from 158 patients with stage I-III NSCLC followed for at least 5 years for the development of brain metastases (Met+, 65 patients) versus no progression (Met-, 93 patients) were subjected to whole-slide imaging. Three separate iterations were performed by first selecting 118 cases (45 Met+, 73 Met-) to train and validate the DL algorithm, while 40 separate cases (20 Met+, 20 Met-) were used as the test set. The DL algorithm results were compared to a blinded review by four expert pathologists. The DL-based algorithm was able to distinguish the eventual development of brain metastases with an accuracy of 87% (p < 0.0001) compared with an average of 57.3% by the four pathologists and appears to be particularly useful in predicting brain metastases in stage I patients. The DL algorithm appears to focus on a complex set of histologic features. DL-based algorithms using routine H&E-stained slides may identify patients who are likely to develop brain metastases from those who will remain disease free over extended (>5 year) follow-up and may thus be spared systemic therapy. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Semiautomatic Assessment of Facet Tropism From Lumbar Spine MRI Using Deep Learning: A Northern Finland Birth Cohort Study

STUDY DESIGN

This is a retrospective, cross-sectional, population-based study that automatically measured the facet joint (FJ) angles from T2-weighted axial magnetic resonance imagings (MRIs) of the lumbar spine using deep learning (DL).

OBJECTIVE

This work aimed to introduce a semiautomatic framework that measures the FJ angles using DL and study facet tropism (FT) in a large Finnish population-based cohort.

SUMMARY OF DATA

T2-weighted axial MRIs of the lumbar spine (L3/4 through L5/S1) for (n=1288) in the NFBC1966 Finnish population-based cohort were used for this study.

MATERIALS AND METHODS

A DL model was developed and trained on 430 participants' MRI images. The authors computed FJ angles from the model's prediction for each level, that is, L3/4 through L5/S1, for the male and female subgroups. Inter-rater and intrarater reliability was analyzed for 60 participants using annotations made by two radiologists and a musculoskeletal researcher. With the developed method, we examined FT in the entire NFBC1966 cohort, adopting the literature definitions of FT thresholds at 7° and 10°. The rater agreement was evaluated both for the annotations and the FJ angles computed based on the annotations. FJ asymmetry ( - was used to evaluate the agreement and correlation between the raters. Bland-Altman analysis was used to assess the agreement and systemic bias in the FJ asymmetry. The authors used the Dice score as the metric to compare the annotations between the raters. The authors evaluated the model predictions on the independent test set and compared them against the ground truth annotations.

RESULTS

This model scored Dice (92.7±0.1) and intersection over union (87.1±0.2) aggregated across all the regions of interest, that is, vertebral body (VB), FJs, and posterior arch (PA). The mean FJ angles measured for the male and female subgroups were in agreement with the literature findings. Intrarater reliability was high, with a Dice score of VB (97.3), FJ (82.5), and PA (90.3). The inter-rater reliability was better between the radiologists with a Dice score of VB (96.4), FJ (75.5), and PA (85.8) than between the radiologists and the musculoskeletal researcher. The prevalence of FT was higher in the male subgroup, with L4/5 found to be the most affected region.

CONCLUSION

The authors developed a DL-based framework that enabled us to study FT in a large cohort. Using the proposed method, the authors present the prevalence of FT in a Finnish population-based cohort.

Detection of peach soluble solids based on near-infrared spectroscopy with High Order Spatial Interaction network

BACKGROUND

Due to the scalability of deep learning technology, researchers have applied it to the non-destructive testing of peach internal quality. In addition, the soluble solids content (SSC) is an important internal quality indicator that determines the quality of peaches. Peaches with high SSC have a sweeter taste and better texture, making them popular in the market. Therefore, SSC is an important indicator for measuring peach internal quality and making harvesting decisions.

RESULTS

This article presents the High Order Spatial Interaction Network (HOSINet), which combines the Position Attention Module (PAM) and Channel Attention Module (CAM). Additionally, a feature wavelength selection algorithm similar to the Group-based Clustering Subspace Representation (GCSR-C) is used to establish the Position and Channel Attention Module-High Order Spatial Interaction (PC-HOSI) model for peach SSC prediction. The accuracy of this model is compared with traditional machine learning and traditional deep learning models. Finally, the permutation algorithm is combined with deep learning models to visually evaluate the importance of feature wavelengths. Increasing the order of the PC-HOSI model enhances its ability to learn spatial correlations in the dataset, thus improving its predictive performance.

CONCLUSION

The optimal model, PC-HOSI model, performed well with an order of 3 (PC-HOSI-3), with a root mean square error of 0.421 °Brix and a coefficient of determination of 0.864. Compared with traditional machine learning and deep learning algorithms, the coefficient of determination for the prediction set was improved by 0.07 and 0.39, respectively. The permutation algorithm also provided interpretability analysis for the predictions of the deep learning model, offering insights into the importance of spectral bands. These results contribute to the accurate prediction of SSC in peaches and support research on interpretability of neural network models for prediction. © 2024 Society of Chemical Industry.

Artificial intelligence for cervical cancer screening: Scoping review, 2009-2022

BACKGROUND

The intersection of artificial intelligence (AI) with cancer research is increasing, and many of the advances have focused on the analysis of cancer images.

OBJECTIVES

To describe and synthesize the literature on the diagnostic accuracy of AI in early imaging diagnosis of cervical cancer following Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR).

SEARCH STRATEGY

Arksey and O'Malley methodology was used and PubMed, Scopus, and Google Scholar databases were searched using a combination of English and Spanish keywords.

SELECTION CRITERIA

Identified titles and abstracts were screened to select original reports and cross-checked for overlap of cases.

DATA COLLECTION AND ANALYSIS

A descriptive summary was organized by the AI algorithm used, total of images analyzed, data source, clinical comparison criteria, and diagnosis performance.

MAIN RESULTS

We identified 32 studies published between 2009 and 2022. The primary sources of images were digital colposcopy, cervicography, and mobile devices. The machine learning/deep learning (DL) algorithms applied in the articles included support vector machine (SVM), random forest classifier, k-nearest neighbors, multilayer perceptron, C4.5, Naïve Bayes, AdaBoost, XGboots, conditional random fields, Bayes classifier, convolutional neural network (CNN; and variations), ResNet (several versions), YOLO+EfficientNetB0, and visual geometry group (VGG; several versions). SVM and DL methods (CNN, ResNet, VGG) showed the best diagnostic performances, with an accuracy of over 97%.

CONCLUSION

We concluded that the use of AI for cervical cancer screening has increased over the years, and some results (mainly from DL) are very promising. However, further research is necessary to validate these findings.

Analysing the impact of comorbid conditions and media coverage on online symptom search data: a novel AI-based approach for COVID-19 tracking

BACKGROUND

Web search data have proven to bea valuable early indicator of COVID-19 outbreaks. However, the influence of co-morbid conditions with similar symptoms and the effect of media coverage on symptom-related searches are often overlooked, leading to potential inaccuracies in COVID-19 simulations.

METHOD

This study introduces a machine learning-based approach to estimate the magnitude of the impact of media coverage and comorbid conditions with similar symptoms on online symptom searches, based on two scenarios with quantile levels 10-90 and 25-75. An incremental batch learning RNN-LSTM model was then developed for the COVID-19 simulation in Australia and New Zealand, allowing the model to dynamically simulate different infection rates and transmissibility of SARS-CoV-2 variants.

RESULT

The COVID-19 infected person-directed symptom searches were found to account for only a small proportion of the total search volume (on average 33.68% in Australia vs. 36.89% in New Zealand) compared to searches influenced by media coverage and comorbid conditions (on average 44.88% in Australia vs. 50.94% in New Zealand). The proposed method, which incorporates estimated symptom component ratios into the RNN-LSTM embedding model, significantly improved COVID-19 simulation performance.

CONCLUSION

Media coverage and comorbid conditions with similar symptoms dominate the total number of online symptom searches, suggesting that direct use of online symptom search data in COVID-19 simulations may overestimate COVID-19 infections. Our approach provides new insights into the accurate estimation of COVID-19 infections using online symptom searches, thereby assisting governments in developing complementary methods for public health surveillance.

Deep-Learning Automation of Preoperative Radiographic Parameters Associated With Early Periprosthetic Femur Fracture After Total Hip Arthroplasty

BACKGROUND

The radiographic assessment of bone morphology impacts implant selection and fixation type in total hip arthroplasty (THA) and is important to minimize the risk of periprosthetic femur fracture (PFF). We utilized a deep-learning algorithm to automate femoral radiographic parameters and determined which automated parameters were associated with early PFF.

METHODS

Radiographs from a publicly available database and from patients undergoing primary cementless THA at a high-volume institution (2016 to 2020) were obtained. A U-Net algorithm was trained to segment femoral landmarks for bone morphology parameter automation. Automated parameters were compared against that of a fellowship-trained surgeon and compared in an independent cohort of 100 patients who underwent THA (50 with early PFF and 50 controls matched by femoral component, age, sex, body mass index, and surgical approach).

RESULTS

On the independent cohort, the algorithm generated 1,710 unique measurements for 95 images (5% lesser trochanter identification failure) in 22 minutes. Medullary canal width, femoral cortex width, canal flare index, morphological cortical index, canal bone ratio, and canal calcar ratio had good-to-excellent correlation with surgeon measurements (Pearson's correlation coefficient: 0.76 to 0.96). Canal calcar ratios (0.43 ± 0.08 versus 0.40 ± 0.07) and canal bone ratios (0.39 ± 0.06 versus 0.36 ± 0.06) were higher (P < .05) in the PFF cohort when comparing the automated parameters.

CONCLUSIONS

Deep-learning automated parameters demonstrated differences in patients who had and did not have early PFF after cementless primary THA. This algorithm has the potential to complement and improve patient-specific PFF risk-prediction tools.

Vein segmentation and visualization of upper and lower extremities using convolution neural network

OBJECTIVES

The study focused on developing a reliable real-time venous localization, identification, and visualization framework based upon deep learning (DL) self-parametrized Convolution Neural Network (CNN) algorithm for segmentation of the venous map for both lower and upper limb dataset acquired under unconstrained conditions using near-infrared (NIR) imaging setup, specifically to assist vascular surgeons during venipuncture, vascular surgeries, or Chronic Venous Disease (CVD) treatments.

METHODS

A portable image acquisition setup has been designed to collect venous data (upper and lower extremities) from 72 subjects. A manually annotated image dataset was used to train and compare the performance of existing well-known CNN-based architectures such as ResNet and VGGNet with self-parameterized U-Net, improving automated vein segmentation and visualization.

RESULTS

Experimental results indicated that self-parameterized U-Net performs better at segmenting the unconstrained dataset in comparison with conventional CNN feature-based learning models, with a Dice score of 0.58 and displaying 96.7 % accuracy for real-time vein visualization, making it appropriate to locate veins in real-time under unconstrained conditions.

CONCLUSIONS

Self-parameterized U-Net for vein segmentation and visualization has the potential to reduce risks associated with traditional venipuncture or CVD treatments by outperforming conventional CNN architectures, providing vascular assistance, and improving patient care and treatment outcomes.

Application of a novel deep learning-based 3D videography workflow to bat flight

Studying the detailed biomechanics of flying animals requires accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor-intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning-powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m3 wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m3 enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human-to-human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.

Deep learning prediction of triplet-triplet annihilation parameters in blue fluorescent organic light-emitting diodes

The triplet-triplet annihilation (TTA) ratio and the rate coefficient (kTT) of TTA are key factors in estimating the contribution of triplet excitons to radiative singlet excitons in fluorescent TTA organic light-emitting diodes. In this study, we implemented deep learning models to predict key factors from transient electroluminescence (trEL) data using new numerical equations. A new TTA model was developed that considers both polaron and exciton dynamics, enabling the distinction between prompt and delayed singlet decays with a fundamental understanding of the mechanism. In addition, deep learning models for predicting the kinetic coefficients and TTA ratio were established. After comprehensive optimization inspired by photophysics, we achieved determination coefficient values of 0.992 and 0.999 in the prediction of kTT and TTA ratio, respectively, indicating a nearly perfect prediction. The contribution of each kinetic parameter of polaron and exciton dynamics to the trEL curve was discussed using various deep learning models. This article is protected by copyright. All rights reserved.

Pixelated High-Q Metasurfaces for in Situ Biospectroscopy and Artificial Intelligence-Enabled Classification of Lipid Membrane Photoswitching Dynamics

Nanophotonic devices excel at confining light into intense hot spots of electromagnetic near fields, creating exceptional opportunities for light-matter coupling and surface-enhanced sensing. Recently, all-dielectric metasurfaces with ultrasharp resonances enabled by photonic bound states in the continuum (BICs) have unlocked additional functionalities for surface-enhanced biospectroscopy by precisely targeting and reading out the molecular absorption signatures of diverse molecular systems. However, BIC-driven molecular spectroscopy has so far focused on end point measurements in dry conditions, neglecting the crucial interaction dynamics of biological systems. Here, we combine the advantages of pixelated all-dielectric metasurfaces with deep learning-enabled feature extraction and prediction to realize an integrated optofluidic platform for time-resolved in situ biospectroscopy. Our approach harnesses high-Q metasurfaces specifically designed for operation in a lossy aqueous environment together with advanced spectral sampling techniques to temporally resolve the dynamic behavior of photoswitchable lipid membranes. Enabled by a software convolutional neural network, we further demonstrate the real-time classification of the characteristic cis and trans membrane conformations with 98% accuracy. Our synergistic sensing platform incorporating metasurfaces, optofluidics, and deep learning reveals exciting possibilities for studying multimolecular biological systems, ranging from the behavior of transmembrane proteins to the dynamic processes associated with cellular communication.

Application of Artificial Intelligence in Tissue Engineering

Tissue engineering, a crucial approach in medical research and clinical applications, aims to regenerate damaged organs. By combining stem cells, biochemical factors, and biomaterials, it encounters challenges in designing complex 3D structures. Artificial intelligence (AI) enhances tissue engineering through computational modeling, biomaterial design, cell culture optimization, and personalized medicine. This review explores AI applications in organ tissue engineering (bone, heart, nerve, skin, cartilage), employing various machine learning (ML) algorithms for data analysis, prediction, and optimization. Each section discusses common ML algorithms and specific applications, emphasizing the potential and challenges in advancing regenerative therapies.

Effect of MR head coil geometry on deep-learning-based MR image reconstruction

PURPOSE

To investigate whether parallel imaging-imposed geometric coil constraints can be relaxed when using a deep learning (DL)-based image reconstruction method as opposed to a traditional non-DL method.

THEORY AND METHODS

Traditional and DL-based MR image reconstruction approaches operate in fundamentally different ways: Traditional methods solve a system of equations derived from the image data whereas DL methods use data/target pairs to learn a generalizable reconstruction model. Two sets of head coil profiles were evaluated: (1) 8-channel and (2) 32-channel geometries. A DL model was compared to conjugate gradient SENSE (CG-SENSE) and L1-wavelet compressed sensing (CS) through quantitative metrics and visual assessment as coil overlap was increased.

RESULTS

Results were generally consistent between experiments. As coil overlap increased, there was a significant (p < 0.001) decrease in performance in most cases for all methods. The decrease was most pronounced for CG-SENSE, and the DL models significantly outperformed (p < 0.001) their non-DL counterparts in all scenarios. CS showed improved robustness to coil overlap and signal-to-noise ratio (SNR) versus CG-SENSE, but had quantitatively and visually poorer reconstructions characterized by blurriness as compared to DL. DL showed virtually no change in performance across SNR and very small changes across coil overlap.

CONCLUSION

The DL image reconstruction method produced images that were robust to coil overlap and of higher quality than CG-SENSE and CS. This suggests that geometric coil design constraints can be relaxed when using DL reconstruction methods.

DF-QSM: Data Fidelity based Hybrid Approach for Improved Quantitative Susceptibility Mapping of the Brain

Quantitative Susceptibility Mapping (QSM) is an advanced magnetic resonance imaging (MRI) technique to quantify the magnetic susceptibility of the tissue under investigation. Deep learning methods have shown promising results in deconvolving the susceptibility distribution from the measured local field obtained from the MR phase. Although existing deep learning based QSM methods can produce high-quality reconstruction, they are highly biased toward training data distribution with less scope for generalizability. This work proposes a hybrid two-step reconstruction approach to improve deep learning based QSM reconstruction. The susceptibility map prediction obtained from the deep learning methods has been refined in the framework developed in this work to ensure consistency with the measured local field. The developed method was validated on existing deep learning and model-based deep learning methods for susceptibility mapping of the brain. The developed method resulted in improved reconstruction for MRI volumes obtained with different acquisition settings, including deep learning models trained on constrained (limited) data settings.

Early identification of patients at risk for iron-deficiency anemia using deep learning techniques

OBJECTIVES

Iron-deficiency anemia (IDA) is a common health problem worldwide, and up to 10% of adult patients with incidental IDA may have gastrointestinal cancer. A diagnosis of IDA can be established through a combination of laboratory tests, but it is often underrecognized until a patient becomes symptomatic. Based on advances in machine learning, we hypothesized that we could reduce the time to diagnosis by developing an IDA prediction model. Our goal was to develop 3 neural networks by using retrospective longitudinal outpatient laboratory data to predict the risk of IDA 3 to 6 months before traditional diagnosis.

METHODS

We analyzed retrospective outpatient electronic health record data between 2009 and 2020 from an academic medical center in northern Texas. We included laboratory features from 30,603 patients to develop 3 types of neural networks: artificial neural networks, long short-term memory cells, and gated recurrent units. The classifiers were trained using the Adam Optimizer across 200 random training-validation splits. We calculated accuracy, area under the receiving operating characteristic curve, sensitivity, and specificity in the testing split.

RESULTS

Although all models demonstrated comparable performance, the gated recurrent unit model outperformed the other 2, achieving an accuracy of 0.83, an area under the receiving operating characteristic curve of 0.89, a sensitivity of 0.75, and a specificity of 0.85 across 200 epochs.

CONCLUSIONS

Our results showcase the feasibility of employing deep learning techniques for early prediction of IDA in the outpatient setting based on sequences of laboratory data, offering a substantial lead time for clinical intervention.

Computer vision analysis of mother-infant interaction identified efficient pup retrieval in V1b receptor knockout mice

Close contact between lactating rodent mothers and their infants is essential for effective nursing. Whether the mother's effort to retrieve the infants to their nest requires the vasopressin-signaling via V1b receptor has not been fully defined. To address this question, V1b receptor knockout (V1bKO) and control mice were analyzed in pup retrieval test. Because an exploring mother in a new test cage randomly accessed to multiple infants in changing backgrounds over time, a computer vision-based deep learning analysis was applied to continuously calculate the distances between the mother and the infants as a parameter of their relationship. In an open-field, a virgin female V1bKO mice entered fewer times into the center area and moved shorter distances than wild-type (WT). While this behavioral pattern persisted in V1bKO mother, the pup retrieval test demonstrated that total distances between a V1bKO mother and infants came closer in a shorter time than with a WT mother. Moreover, in the medial preoptic area, parts of the V1b receptor transcripts were detected in galanin- and c-fos-positive neurons following maternal stimulation by infants. This research highlights the effectiveness of deep learning analysis in evaluating the mother-infant relationship and the critical role of V1b receptor in pup retrieval during the early lactation phase.

Artificial intelligence assistance for the measurement of full alignment parameters in whole-spine lateral radiographs

BACKGROUND

Measuring spinal alignment with radiological parameters is essential in patients with spinal conditions likely to be treated surgically. These evaluations are not usually included in the radiological report. As a result, spinal surgeons commonly perform the measurement, which is time-consuming and subject to errors. We aim to develop a fully automated artificial intelligence tool to assist in measuring alignment parameters in whole-spine lateral radiograph (WSL X-rays).

MATERIAL AND METHODS

We developed a tool called Vertebrai that automatically calculates the global spinal parameters (GSP): Pelvic incidence (PI), sacral slope (SS), pelvic tilt (PT), L1-L4 angle, L4-S1 lumbo-pelvic angle, T1 pelvic angle (TPA), sagittal vertical axis (SVA), cervical lordosis (CL), C1-C2 lordosis, lumbar lordosis (LL), mid-thoracic kyphosis (MTK), proximal thoracic kyphosis (PTK), global thoracic kyphosis (GTK), T1 slope (T1S), C2-C7 plummet (cSVA), spino-sacral angle (SSA), C7 tilt (C7T), global tilt (GT), spinopelvic tilt (T1SPi) and hip odontoid axis (OD-HA). We assessed human-AI interaction instead of AI performance alone. We compared the time to measure GSP and inter-rater agreement with and without AI assistance. Two institutional datasets were created with 2267 multilabel images for classification, and 784 WSL X-rays with reference standard landmark labeled by spinal surgeons.

RESULTS

Vertebrai significantly reduced the measurement time comparing spine surgeons with AI assistance and the AI algorithm alone, without human intervention (3 minutes vs 0.26 minutes; p < 0.05). Vertebrai achieved an average accuracy of 83% in detecting abnormal alignment values, with the SS parameter exhibiting the lowest accuracy at 61.5% and T1SPi demonstrating the highest accuracy at 100%. Intra-class correlation analysis revealed a high level of correlation and consistency in the global alignment parameters.

CONCLUSIONS

Vertebrai's measurements can accurately detect alignment parameters, making it a promising tool for measuring GSP automatically.

Automated scoring of glomerular injury in TNS2-deficient nephropathy

Several artificial intelligence (AI) systems have been developed for glomerular pathology analysis in clinical settings. However, the application of AI systems in nonclinical fields remains limited. In this study, we trained a convolutional neural network model, which is an AI algorithm, to classify the severity of Tensin 2 (TNS2)-deficient nephropathy into seven categories. A dataset consisting of 803 glomerular images was generated from kidney sections of TNS2-deficient and wild-type mice. Manual evaluations of the images were conducted to assess their glomerular injury scores. The trained AI achieved approximately 70% accuracy in predicting the glomerular injury score for TNS2-deficient nephropathy. However, the AI achieved approximately 100% accuracy when considering predictions within one score of the true label as correct. The AI's predicted mean score closely matched the true mean score. In conclusion, while the AI model may not replace human judgment entirely, it can serve as a reliable second assessor in scoring glomerular injury, offering potential benefits in enhancing the accuracy and objectivity of such assessments.

Haplotype function score improves biological interpretation and cross-ancestry polygenic prediction of human complex traits

We propose a new framework for human genetic association studies: at each locus, a deep learning model (in this study, Sei) is used to calculate the functional genomic activity score for two haplotypes per individual. This score, defined as the Haplotype Function Score (HFS), replaces the original genotype in association studies. Applying the HFS framework to 14 complex traits in the UK Biobank, we identified 3619 independent HFS-trait associations with a significance of p < 5 × 10-8. Fine-mapping revealed 2699 causal associations, corresponding to a median increase of 63 causal findings per trait compared with single-nucleotide polymorphism (SNP)-based analysis. HFS-based enrichment analysis uncovered 727 pathway-trait associations and 153 tissue-trait associations with strong biological interpretability, including 'circadian pathway-chronotype' and 'arachidonic acid-intelligence'. Lastly, we applied least absolute shrinkage and selection operator (LASSO) regression to integrate HFS prediction score with SNP-based polygenic risk scores, which showed an improvement of 16.1-39.8% in cross-ancestry polygenic prediction. We concluded that HFS is a promising strategy for understanding the genetic basis of human complex traits.

Evaluating the relationship between magnetic resonance image quality metrics and deep learning-based segmentation accuracy of brain tumors

BACKGROUND

Magnetic resonance imaging (MRI) scans are known to suffer from a variety of acquisition artifacts as well as equipment-based variations that impact image appearance and segmentation performance. It is still unclear whether a direct relationship exists between magnetic resonance (MR) image quality metrics (IQMs) (e.g., signal-to-noise, contrast-to-noise) and segmentation accuracy.

PURPOSE

Deep learning (DL) approaches have shown significant promise for automated segmentation of brain tumors on MRI but depend on the quality of input training images. We sought to evaluate the relationship between IQMs of input training images and DL-based brain tumor segmentation accuracy toward developing more generalizable models for multi-institutional data.

METHODS

We trained a 3D DenseNet model on the BraTS 2020 cohorts for segmentation of tumor subregions enhancing tumor (ET), peritumoral edematous, and necrotic and non-ET on MRI; with performance quantified via a 5-fold cross-validated Dice coefficient. MRI scans were evaluated through the open-source quality control tool MRQy, to yield 13 IQMs per scan. The Pearson correlation coefficient was computed between whole tumor (WT) dice values and IQM measures in the training cohorts to identify quality measures most correlated with segmentation performance. Each selected IQM was used to group MRI scans as "better" quality (BQ) or "worse" quality (WQ), via relative thresholding. Segmentation performance was re-evaluated for the DenseNet model when (i) training on BQ MRI images with validation on WQ images, as well as (ii) training on WQ images, and validation on BQ images. Trends were further validated on independent test sets derived from the BraTS 2021 training cohorts.

RESULTS

For this study, multimodal MRI scans from the BraTS 2020 training cohorts were used to train the segmentation model and validated on independent test sets derived from the BraTS 2021 cohort. Among the selected IQMs, models trained on BQ images based on inhomogeneity measurements (coefficient of variance, coefficient of joint variation, coefficient of variation of the foreground patch) and the models trained on WQ images based on noise measurement peak signal-to-noise ratio (SNR) yielded significantly improved tumor segmentation accuracy compared to their inverse models.

CONCLUSIONS

Our results suggest that a significant correlation may exist between specific MR IQMs and DenseNet-based brain tumor segmentation performance. The selection of MRI scans for model training based on IQMs may yield more accurate and generalizable models in unseen validation.

Unmasking bias in artificial intelligence: a systematic review of bias detection and mitigation strategies in electronic health record-based models

OBJECTIVES

Leveraging artificial intelligence (AI) in conjunction with electronic health records (EHRs) holds transformative potential to improve healthcare. However, addressing bias in AI, which risks worsening healthcare disparities, cannot be overlooked. This study reviews methods to handle various biases in AI models developed using EHR data.

MATERIALS AND METHODS

We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines, analyzing articles from PubMed, Web of Science, and IEEE published between January 01, 2010 and December 17, 2023. The review identified key biases, outlined strategies for detecting and mitigating bias throughout the AI model development, and analyzed metrics for bias assessment.

RESULTS

Of the 450 articles retrieved, 20 met our criteria, revealing 6 major bias types: algorithmic, confounding, implicit, measurement, selection, and temporal. The AI models were primarily developed for predictive tasks, yet none have been deployed in real-world healthcare settings. Five studies concentrated on the detection of implicit and algorithmic biases employing fairness metrics like statistical parity, equal opportunity, and predictive equity. Fifteen studies proposed strategies for mitigating biases, especially targeting implicit and selection biases. These strategies, evaluated through both performance and fairness metrics, predominantly involved data collection and preprocessing techniques like resampling and reweighting.

DISCUSSION

This review highlights evolving strategies to mitigate bias in EHR-based AI models, emphasizing the urgent need for both standardized and detailed reporting of the methodologies and systematic real-world testing and evaluation. Such measures are essential for gauging models' practical impact and fostering ethical AI that ensures fairness and equity in healthcare.

Deep learning-based optimization of field geometry for total marrow irradiation delivered with volumetric modulated arc therapy

BACKGROUND

Total marrow (lymphoid) irradiation (TMI/TMLI) is a radiotherapy treatment used to selectively target the bone marrow and lymph nodes in conditioning regimens for allogeneic hematopoietic stem cell transplantation. A complex field geometry is needed to cover the large planning target volume (PTV) of TMI/TMLI with volumetric modulated arc therapy (VMAT). Five isocenters and ten overlapping fields are needed for the upper body, while, for patients with large anatomical conformation, two specific isocenters are placed on the arms. The creation of a field geometry is clinically challenging and is performed by a medical physicist (MP) specialized in TMI/TMLI.

PURPOSE

To develop convolutional neural networks (CNNs) for automatically generating the field geometry of TMI/TMLI.

METHODS

The dataset comprised 117 patients treated with TMI/TMLI between 2011 and 2023 at our Institute. The CNN input image consisted of three channels, obtained by projecting along the sagittal plane: (1) average CT pixel intensity within the PTV; (2) PTV mask; (3) brain, lungs, liver, bowel, and bladder masks. This "averaged" frontal view combined the information analyzed by the MP when setting the field geometry in the treatment planning system (TPS). Two CNNs were trained to predict the isocenters coordinates and jaws apertures for patients with (CNN-1) and without (CNN-2) isocenters on the arms. Local optimization methods were used to refine the models output based on the anatomy of the patient. Model evaluation was performed on a test set of 15 patients in two ways: (1) by computing the root mean squared error (RMSE) between the CNN output and ground truth; (2) with a qualitative assessment of manual and generated field geometries-scale: 1 = not adequate, 4 = adequate-carried out in blind mode by three MPs with different expertise in TMI/TMLI. The Wilcoxon signed-rank test was used to evaluate the independence of the given scores between manual and generated configurations (p < 0.05 significant).

RESULTS

The average and standard deviation values of RMSE for CNN-1 and CNN-2 before/after local optimization were 15 ± 2/13 ± 3 mm and 16 ± 2/18 ± 4 mm, respectively. The CNNs were integrated into a planning automation software for TMI/TMLI such that the MPs could analyze in detail the proposed field geometries directly in the TPS. The selection of the CNN model to create the field geometry was based on the PTV width to approximate the decision process of an experienced MP and provide a single option of field configuration. We found no significant differences between the manual and generated field geometries for any MP, with median values of 4 versus 4 (p = 0.92), 3 versus 3 (p = 0.78), 4 versus 3 (p = 0.48), respectively. Starting from October 2023, the generated field geometry has been introduced in our clinical practice for prospective patients.

CONCLUSIONS

The generated field geometries were clinically acceptable and adequate, even for an MP with high level of expertise in TMI/TMLI. Incorporating the knowledge of the MPs into the development cycle was crucial for optimizing the models, especially in this scenario with limited data.

Suppressing label noise in medical image classification using mixup attention and self-supervised learning

Deep neural networks (DNNs) have been widely applied in medical image classification and achieve remarkable classification performance. These achievements heavily depend on large-scale accurately annotated training data. However, label noise is inevitably introduced in the medical image annotation, as the labeling process heavily relies on the expertise and experience of annotators. Meanwhile, DNNs suffer from overfitting noisy labels, degrading the performance of models. Therefore, in this work, we innovatively devise noise-robust training approach to mitigate the adverse effects of noisy labels in medical image classification. Specifically, we incorporate contrastive learning and intra-group attention mixup strategies into the vanilla supervised learning. The contrastive learning for feature extractor helps to enhance visual representation of DNNs. The intra-group attention mixup module constructs groups and assigns self-attention weights for group-wise samples, and subsequently interpolates massive noisy-suppressed samples through weighted mixup operation. We conduct comparative experiments on both synthetic and real-world noisy medical datasets under various noise levels. Rigorous experiments validate that our noise-robust method with contrastive learning and attention mixup can effectively handle with label noise, and is superior to state-of-the-art methods. An ablation study also shows that both components contribute to boost model performance. The proposed method demonstrates its capability of curb label noise and has certain potential toward real-world clinic applications.