Weakly supervised classification of aortic valve malformations using unlabeled cardiac MRI sequences

Analysis of 3D pathology samples using weakly supervised AI

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

The Role of Artificial Intelligence in Cardiac Imaging

Artificial intelligence (AI) is having a significant impact in medical imaging, advancing almost every aspect of the field, from image acquisition and postprocessing to automated image analysis with outreach toward supporting decision making. Noninvasive cardiac imaging is one of the main and most exciting fields for AI development. The aim of this review is to describe the main applications of AI in cardiac imaging, including CT and MR imaging, and provide an overview of recent advancements and available clinical applications that can improve clinical workflow, disease detection, and prognostication in cardiac disease.

Screening and diagnosis of cardiovascular disease using artificial intelligence-enabled cardiac magnetic resonance imaging

Cardiac magnetic resonance imaging (CMR) is the gold standard for cardiac function assessment and plays a crucial role in diagnosing cardiovascular disease (CVD). However, its widespread application has been limited by the heavy resource burden of CMR interpretation. Here, to address this challenge, we developed and validated computerized CMR interpretation for screening and diagnosis of 11 types of CVD in 9,719 patients. We propose a two-stage paradigm consisting of noninvasive cine-based CVD screening followed by cine and late gadolinium enhancement-based diagnosis. The screening and diagnostic models achieved high performance (area under the curve of 0.988 ± 0.3% and 0.991 ± 0.0%, respectively) in both internal and external datasets. Furthermore, the diagnostic model outperformed cardiologists in diagnosing pulmonary arterial hypertension, demonstrating the ability of artificial intelligence-enabled CMR to detect previously unidentified CMR features. This proof-of-concept study holds the potential to substantially advance the efficiency and scalability of CMR interpretation, thereby improving CVD screening and diagnosis.

Deep learning for classification of late gadolinium enhancement lesions based on the 16-segment left ventricular model

PURPOSE

This study aimed to develop and validate a deep learning-based method that allows for segmental analysis of myocardial late gadolinium enhancement (LGE) lesions.

METHODS

Cardiac LGE data from 170 patients with coronary artery disease and non-ischemic heart disease were used for training, validation, and testing. Short-axis images were transformed to polar space after identification of the left ventricular (LV) center point and anterior right ventricular (RV) insertion point. Images were obtained after dividing the polar transformed images into segments based on the 16-segment LV model. Five different deep convolutional neural network (CNN) models were developed and validated using the labeled data, where the image after the division corresponded to a segment, and the lesion labeling was based on the 16-segment LV model. Unseen testing data were used to evaluate the performance of the lesion classification.

RESULTS

Without manual lesion segmentation and annotation, the proposed method showed an area under the curve (AUC) of 0.875, and a precision, recall, and F1-score of 0.723, 0.783, and 0.752, respectively for the lesion class when the pretrained ResNet50 model was tested for all slice images. The two pretrained models of ResNet50 and EfficientNet-B0 outperformed the three non-pretrained CNN models in terms of AUCs (0.873-0.875 vs. 0.834-0.841).

CONCLUSION

The proposed method is based on learning a deep CNN model from polar transformed images to predict LGE lesions with good accuracy and does not require time-consuming annotation procedures such as lesion segmentation.

Multi-target landmark detection with incomplete images via reinforcement learning and shape prior embedding

Medical images are generally acquired with limited field-of-view (FOV), which could lead to incomplete regions of interest (ROI), and thus impose a great challenge on medical image analysis. This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets. Based on learning a navigation policy, instead of predicting targets directly, reinforcement learning (RL)-based methods have the potential to tackle this challenge in an efficient manner. Inspired by this, in this work we propose a multi-agent RL framework for simultaneous multi-target landmark detection. This framework is aimed to learn from incomplete or (and) complete images to form an implicit knowledge of global structure, which is consolidated during the training stage for the detection of targets from either complete or incomplete test images. To further explicitly exploit the global structural information from incomplete images, we propose to embed a shape model into the RL process. With this prior knowledge, the proposed RL model can not only localize dozens of targets simultaneously, but also work effectively and robustly in the presence of incomplete images. We validated the applicability and efficacy of the proposed method on various multi-target detection tasks with incomplete images from practical clinics, using body dual-energy X-ray absorptiometry (DXA), cardiac MRI and head CT datasets. Results showed that our method could predict whole set of landmarks with incomplete training images up to 80% missing proportion (average distance error 2.29 cm on body DXA), and could detect unseen landmarks in regions with missing image information outside FOV of target images (average distance error 6.84 mm on 3D half-head CT). Our code will be released via https://zmiclab.github.io/projects.html.

The effect of image resolution on convolutional neural networks in breast ultrasound

Purpose

The objective of this research was to investigate the efficacy of various parameter combinations of Convolutional Neural Networks (CNNs) models, namely MobileNet and DenseNet121, and different input image resolutions (REZs) ranging from 64×64 to 512×512 pixels, for diagnosing breast cancer.

Materials and methods

During the period of June 2015 to November 2020, two hospitals were involved in the collection of two-dimensional ultrasound breast images for this retrospective multicenter study. The diagnostic performance of the computer models MobileNet and DenseNet 121 was compared at different resolutions.

Results

The results showed that MobileNet had the best breast cancer diagnosis performance at 320×320pixel REZ and DenseNet121 had the best breast cancer diagnosis performance at 448×448pixel REZ.

Conclusion

Our study reveals a significant correlation between image resolution and breast cancer diagnosis accuracy. Through the comparison of MobileNet and DenseNet121, it is highlighted that lightweight neural networks (LW-CNNs) can achieve model performance similar to or even slightly better than large neural networks models (HW-CNNs) in ultrasound images, and LW-CNNs' prediction time per image is lower.

Deep Learning Synthetic Strain: Quantitative Assessment of Regional Myocardial Wall Motion at MRI

Purpose

To assess the feasibility of a newly developed algorithm, called deep learning synthetic strain (DLSS), to infer myocardial velocity from cine steady-state free precession (SSFP) images and detect wall motion abnormalities in patients with ischemic heart disease.

Materials and Methods

In this retrospective study, DLSS was developed by using a data set of 223 cardiac MRI examinations including cine SSFP images and four-dimensional flow velocity data (November 2017 to May 2021). To establish normal ranges, segmental strain was measured in 40 individuals (mean age, 41 years ± 17 [SD]; 30 men) without cardiac disease. Then, DLSS performance in the detection of wall motion abnormalities was assessed in a separate group of patients with coronary artery disease, and these findings were compared with consensus results of four independent cardiothoracic radiologists (ground truth). Algorithm performance was evaluated by using receiver operating characteristic curve analysis.

Results

Median peak segmental radial strain in individuals with normal cardiac MRI findings was 38% (IQR: 30%-48%). Among patients with ischemic heart disease (846 segments in 53 patients; mean age, 61 years ± 12; 41 men), the Cohen κ among four cardiothoracic readers for detecting wall motion abnormalities was 0.60-0.78. DLSS achieved an area under the receiver operating characteristic curve of 0.90. Using a fixed 30% threshold for abnormal peak radial strain, the algorithm achieved a sensitivity, specificity, and accuracy of 86%, 85%, and 86%, respectively.

Conclusion

The deep learning algorithm had comparable performance with subspecialty radiologists in inferring myocardial velocity from cine SSFP images and identifying myocardial wall motion abnormalities at rest in patients with ischemic heart disease.Keywords: Neural Networks, Cardiac, MR Imaging, Ischemia/Infarction Supplemental material is available for this article. © RSNA, 2023.

Weakly Supervised Classification of Vital Sign Alerts as Real or Artifact

A significant proportion of clinical physiologic monitoring alarms are false. This often leads to alarm fatigue in clinical personnel, inevitably compromising patient safety. To combat this issue, researchers have attempted to build Machine Learning (ML) models capable of accurately adjudicating Vital Sign (VS) alerts raised at the bedside of hemodynamically monitored patients as real or artifact. Previous studies have utilized supervised ML techniques that require substantial amounts of hand-labeled data. However, manually harvesting such data can be costly, time-consuming, and mundane, and is a key factor limiting the widespread adoption of ML in healthcare (HC). Instead, we explore the use of multiple, individually imperfect heuristics to automatically assign probabilistic labels to unlabeled training data using weak supervision. Our weakly supervised models perform competitively with traditional supervised techniques and require less involvement from domain experts, demonstrating their use as efficient and practical alternatives to supervised learning in HC applications of ML.

Deep learning-enabled analysis of medical images identifies cardiac sphericity as an early marker of cardiomyopathy and related outcomes

BACKGROUND

Quantification of chamber size and systolic function is a fundamental component of cardiac imaging. However, the human heart is a complex structure with significant uncharacterized phenotypic variation beyond traditional metrics of size and function. Examining variation in cardiac shape can add to our ability to understand cardiovascular risk and pathophysiology.

METHODS

We measured the left ventricle (LV) sphericity index (short axis length/long axis length) using deep learning-enabled image segmentation of cardiac magnetic resonance imaging data from the UK Biobank. Subjects with abnormal LV size or systolic function were excluded. The relationship between LV sphericity and cardiomyopathy was assessed using Cox analyses, genome-wide association studies, and two-sample Mendelian randomization.

FINDINGS

In a cohort of 38,897 subjects, we show that a one standard deviation increase in sphericity index is associated with a 47% increased incidence of cardiomyopathy (hazard ratio [HR]: 1.47, 95% confidence interval [CI]: 1.10-1.98, p = 0.01) and a 20% increased incidence of atrial fibrillation (HR: 1.20, 95% CI: 1.11-1.28, p < 0.001), independent of clinical factors and traditional magnetic resonance imaging (MRI) measurements. We identify four loci associated with sphericity at genome-wide significance, and Mendelian randomization supports non-ischemic cardiomyopathy as causal for LV sphericity.

CONCLUSIONS

Variation in LV sphericity in otherwise normal hearts predicts risk for cardiomyopathy and related outcomes and is caused by non-ischemic cardiomyopathy.

FUNDING

This study was supported by grants K99-HL157421 (D.O.) and KL2TR003143 (S.L.C.) from the National Institutes of Health.

Diagnostic performance of deep learning-based vessel extraction and stenosis detection on coronary computed tomography angiography for coronary artery disease: a multi-reader multi-case study

BACKGROUND

Post-processing and interpretation of coronary CT angiography (CCTA) imaging are time-consuming and dependent on the reader's experience. An automated deep learning (DL)-based imaging reconstruction and diagnosis system was developed to improve diagnostic accuracy and efficiency.

METHODS

Our study including 374 cases from five sites, inviting 12 radiologists, assessed the DL-based system in diagnosing obstructive coronary disease with regard to diagnostic performance, imaging post-processing and reporting time of radiologists, with invasive coronary angiography as a standard reference. The diagnostic performance of DL system and DL-assisted human readers was compared with the traditional method of human readers without DL system.

RESULTS

Comparing the diagnostic performance of human readers without DL system versus with DL system, the AUC was improved from 0.81 to 0.82 (p < 0.05) at patient level and from 0.79 to 0.81 (p < 0.05) at vessel level. An increase in AUC was observed in inexperienced radiologists (p < 0.05), but was absent in experienced radiologists. Regarding diagnostic efficiency, comparing the DL system versus human reader, the average post-processing and reporting time was decreased from 798.60 s to 189.12 s (p < 0.05). The sensitivity and specificity of using DL system alone were 93.55% and 59.57% at patient level and 83.23% and 79.97% at vessel level, respectively.

CONCLUSIONS

With the DL system serving as a concurrent reader, the overall post-processing and reading time was substantially reduced. The diagnostic accuracy of human readers, especially for inexperienced readers, was improved. DL-assisted human reader had the potential of being the reading mode of choice in clinical routine.

Bimodal CNN for cardiovascular disease classification by co-training ECG grayscale images and scalograms

This study aimed to develop a bimodal convolutional neural network (CNN) by co-training grayscale images and scalograms of ECG for cardiovascular disease classification. The bimodal CNN model was developed using a 12-lead ECG database collected from Chapman University and Shaoxing People's Hospital. The preprocessed database contains 10,588 ECG data and 11 heart rhythms labeled by a specialist physician. The preprocessed one-dimensional ECG signals were converted into two-dimensional grayscale images and scalograms, which are fed simultaneously to the bimodal CNN model as dual input images. The proposed model aims to improve the performance of CVDs classification by making use of ECG grayscale images and scalograms. The bimodal CNN model consists of two identical Inception-v3 backbone models, which were pre-trained on the ImageNet database. The proposed model was fine-tuned with 6780 dual-input images, validated with 1694 dual-input images, and tested on 2114 dual-input images. The bimodal CNN model using two identical Inception-v3 backbones achieved best AUC (0.992), accuracy (95.08%), sensitivity (0.942), precision (0.946) and F1-score (0.944) in lead II. Ensemble model of all leads obtained AUC (0.994), accuracy (95.74%), sensitivity (0.950), precision (0.953), and F1-score (0.952). The bimodal CNN model showed better diagnostic performance than logistic regression, XGBoost, LSTM, single CNN model training with grayscale images alone or with scalograms alone. The proposed bimodal CNN model would be of great help in diagnosing cardiovascular diseases.

Adaptation of Autoencoder for Sparsity Reduction From Clinical Notes Representation Learning

When dealing with clinical text classification on a small dataset, recent studies have confirmed that a well-tuned multilayer perceptron outperforms other generative classifiers, including deep learning ones. To increase the performance of the neural network classifier, feature selection for the learning representation can effectively be used. However, most feature selection methods only estimate the degree of linear dependency between variables and select the best features based on univariate statistical tests. Furthermore, the sparsity of the feature space involved in the learning representation is ignored.

GOAL

Our aim is, therefore, to access an alternative approach to tackle the sparsity by compressing the clinical representation feature space, where limited French clinical notes can also be dealt with effectively.

METHODS

This study proposed an autoencoder learning algorithm to take advantage of sparsity reduction in clinical note representation. The motivation was to determine how to compress sparse, high-dimensional data by reducing the dimension of the clinical note representation feature space. The classification performance of the classifiers was then evaluated in the trained and compressed feature space.

RESULTS

The proposed approach provided overall performance gains of up to 3% for each test set evaluation. Finally, the classifier achieved 92% accuracy, 91% recall, 91% precision, and 91% f1-score in detecting the patient's condition. Furthermore, the compression working mechanism and the autoencoder prediction process were demonstrated by applying the theoretic information bottleneck framework. Clinical and Translational Impact Statement- An autoencoder learning algorithm effectively tackles the problem of sparsity in the representation feature space from a small clinical narrative dataset. Significantly, it can learn the best representation of the training data because of its lossless compression capacity compared to other approaches. Consequently, its downstream classification ability can be significantly improved, which cannot be done using deep learning models.

Improving classification of correct and incorrect protein-protein docking models by augmenting the training set

Motivation

Protein-protein interactions drive many relevant biological events, such as infection, replication and recognition. To control or engineer such events, we need to access the molecular details of the interaction provided by experimental 3D structures. However, such experiments take time and are expensive; moreover, the current technology cannot keep up with the high discovery rate of new interactions. Computational modeling, like protein-protein docking, can help to fill this gap by generating docking poses. Protein-protein docking generally consists of two parts, sampling and scoring. The sampling is an exhaustive search of the tridimensional space. The caveat of the sampling is that it generates a large number of incorrect poses, producing a highly unbalanced dataset. This limits the utility of the data to train machine learning classifiers.

Results

Using weak supervision, we developed a data augmentation method that we named hAIkal. Using hAIkal, we increased the labeled training data to train several algorithms. We trained and obtained different classifiers; the best classifier has 81% accuracy and 0.51 Matthews' correlation coefficient on the test set, surpassing the state-of-the-art scoring functions.

Availability and implementation

Docking models from Benchmark 5 are available at https://doi.org/10.5281/zenodo.4012018. Processed tabular data are available at https://repository.kaust.edu.sa/handle/10754/666961. Google colab is available at https://colab.research.google.com/drive/1vbVrJcQSf6\_C3jOAmZzgQbTpuJ5zC1RP?usp=sharing.

Supplementary information

Supplementary data are available at Bioinformatics Advances online.

Deep Learning Staging of Liver Iron Content From Multiecho MR Images

BACKGROUND

MRI represents the most established liver iron content (LIC) evaluation approach by estimation of liver T2* value, but it is dependent on the choice of the measurement region and the software used for image analysis.

PURPOSE

To develop a deep-learning method for unsupervised classification of LIC from magnitude T2* multiecho MR images.

STUDY TYPE

Retrospective.

POPULATION/SUBJECTS

A total of 1069 thalassemia major patients enrolled in the core laboratory of the Myocardial Iron Overload in Thalassemia (MIOT) network, which were included in the training (80%) and test (20%) sets. Twenty patients from different MRI vendors included in the external test set.

FIELD STRENGTH/SEQUENCE

A5 T, T2* multiecho magnitude images.

ASSESSMENT

Four deep-learning convolutional neural networks (HippoNet-2D, HippoNet-3D, HippoNet-LSTM, and an ensemble network HippoNet-Ensemble) were used to achieve unsupervised staging of LIC using five classes (normal, borderline, middle, moderate, severe). The training set was employed to construct the deep-learning model. The performance of the LIC staging model was evaluated in the test set and in the external test set. The model's performances were assessed by evaluating the accuracy, sensitivity, and specificity with respect to the ground truth labels obtained by T2* measurements and by comparison with operator-induced variability originating from different region of interest (ROI) placements.

STATISTICAL TESTS

The network's performances were evaluated by single-class accuracy, specificity, and sensitivity and compared by one-way repeated measures analysis of variance (ANOVA) and one-way ANOVA.

RESULTS

HippoNet-Ensemble reached an accuracy significantly higher than the other networks, and a sensitivity and specificity higher than HippoNet-LSTM. Accuracy, sensitivity, and specificity values for the LIC stages were: normal: 0.96/0.93/0.97, borderline: 0.95/0.85/0.98, mild: 0.96/0.88/0.98, moderate: 0.95/0.89/0.97, severe: 0.97/0.95/0.98. Correctly staging of cases was in the range of 85%-95%, depending on the LIC class. Multiclass accuracy was 0.90 against 0.92 for the interobserver variability.

DATA CONCLUSION

The proposed HippoNet-Ensemble network can perform unsupervised LIC staging and achieves good prognostic performance.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 2.

Synthesis of patient-specific multipoint 4D flow MRI data of turbulent aortic flow downstream of stenotic valves

We propose to synthesize patient-specific 4D flow MRI datasets of turbulent flow paired with ground truth flow data to support training of inference methods. Turbulent blood flow is computed based on the Navier-Stokes equations with moving domains using realistic boundary conditions for aortic shapes, wall displacements and inlet velocities obtained from patient data. From the simulated flow, synthetic multipoint 4D flow MRI data is generated with user-defined spatiotemporal resolutions and reconstructed with a Bayesian approach to compute time-varying velocity and turbulence maps. For MRI data synthesis, a fixed hypothetical scan time budget is assumed and accordingly, changes to spatial resolution and time averaging result in corresponding scaling of signal-to-noise ratios (SNR). In this work, we focused on aortic stenotic flow and quantification of turbulent kinetic energy (TKE). Our results show that for spatial resolutions of 1.5 and 2.5 mm and time averaging of 5 ms as encountered in 4D flow MRI in practice, peak total turbulent kinetic energy downstream of a 50, 75 and 90% stenosis is overestimated by as much as 23, 15 and 14% (1.5 mm) and 38, 24 and 23% (2.5 mm), demonstrating the importance of paired ground truth and 4D flow MRI data for assessing accuracy and precision of turbulent flow inference using 4D flow MRI exams.

Harnessing feature extraction capacities from a pre-trained convolutional neural network (VGG-16) for the unsupervised distinction of aortic outflow velocity profiles in patients with severe aortic stenosis

Aims

Hypothesizing that aortic outflow velocity profiles contain more valuable information about aortic valve obstruction and left ventricular contractility than can be captured by the human eye, features of the complex geometry of Doppler tracings from patients with severe aortic stenosis (AS) were extracted by a convolutional neural network (CNN).

Methods and results

After pre-training a CNN (VGG-16) on a large data set (ImageNet data set; 14 million images belonging to 1000 classes), the convolutional part was employed to transform Doppler tracings to 1D arrays. Among 366 eligible patients [age: 79.8 ± 6.77 years; 146 (39.9%) women] with pre-procedural echocardiography and right heart catheterization prior to transcatheter aortic valve replacement (TAVR), good quality Doppler tracings from 101 patients were analysed. The convolutional part of the pre-trained VGG-16 model in conjunction with principal component analysis and k-means clustering distinguished two shapes of aortic outflow velocity profiles. Kaplan-Meier analysis revealed that mortality in patients from Cluster 2 (n = 40, 39.6%) was significantly increased [hazard ratio (HR) for 2-year mortality: 3; 95% confidence interval (CI): 1-8.9]. Apart from reduced cardiac output and mean aortic valve gradient, patients from Cluster 2 were also characterized by signs of pulmonary hypertension, impaired right ventricular function, and right atrial enlargement. After training an extreme gradient boosting algorithm on these 101 patients, validation on the remaining 265 patients confirmed that patients assigned to Cluster 2 show increased mortality (HR for 2-year mortality: 2.6; 95% CI: 1.4-5.1, P-value: 0.004).

Conclusion

Transfer learning enables sophisticated pattern recognition even in clinical data sets of limited size. Importantly, it is the left ventricular compensation capacity in the face of increased afterload, and not so much the actual obstruction of the aortic valve, that determines fate after TAVR.

Strategies to Address the Lack of Labeled Data for Supervised Machine Learning Training With Electronic Health Records: Case Study for the Extraction of Symptoms From Clinical Notes

BACKGROUND

Automated extraction of symptoms from clinical notes is a challenging task owing to the multidimensional nature of symptom description. The availability of labeled training data is extremely limited owing to the nature of the data containing protected health information. Natural language processing and machine learning to process clinical text for such a task have great potential. However, supervised machine learning requires a great amount of labeled data to train a model, which is at the origin of the main bottleneck in model development.

OBJECTIVE

The aim of this study is to address the lack of labeled data by proposing 2 alternatives to manual labeling for the generation of training labels for supervised machine learning with English clinical text. We aim to demonstrate that using lower-quality labels for training leads to good classification results.

METHODS

We addressed the lack of labels with 2 strategies. The first approach took advantage of the structured part of electronic health records and used diagnosis codes (International Classification of Disease-10th revision) to derive training labels. The second approach used weak supervision and data programming principles to derive training labels. We propose to apply the developed framework to the extraction of symptom information from outpatient visit progress notes of patients with cardiovascular diseases.

RESULTS

We used >500,000 notes for training our classification model with International Classification of Disease-10th revision codes as labels and >800,000 notes for training using labels derived from weak supervision. We show that the dependence between prevalence and recall becomes flat provided a sufficiently large training set is used (>500,000 documents). We further demonstrate that using weak labels for training rather than the electronic health record codes derived from the patient encounter leads to an overall improved recall score (10% improvement, on average). Finally, the external validation of our models shows excellent predictive performance and transferability, with an overall increase of 20% in the recall score.

CONCLUSIONS

This work demonstrates the power of using a weak labeling pipeline to annotate and extract symptom mentions in clinical text, with the prospects to facilitate symptom information integration for a downstream clinical task such as clinical decision support.

Weak Supervision for Affordable Modeling of Electrocardiogram Data

Analysing electrocardiograms (ECGs) is an inexpensive and non-invasive, yet powerful way to diagnose heart disease. ECG studies using Machine Learning to automatically detect abnormal heartbeats so far depend on large, manually annotated datasets. While collecting vast amounts of unlabeled data can be straightforward, the point-by-point annotation of abnormal heartbeats is tedious and expensive. We explore the use of multiple weak supervision sources to learn diagnostic models of abnormal heartbeats via human designed heuristics, without using ground truth labels on individual data points. Our work is among the first to define weak supervision sources directly on time series data. Results show that with as few as six intuitive time series heuristics, we are able to infer high quality probabilistic label estimates for over 100,000 heartbeats with little human effort, and use the estimated labels to train competitive classifiers evaluated on held out test data.

AI in health and medicine

Artificial intelligence (AI) is poised to broadly reshape medicine, potentially improving the experiences of both clinicians and patients. We discuss key findings from a 2-year weekly effort to track and share key developments in medical AI. We cover prospective studies and advances in medical image analysis, which have reduced the gap between research and deployment. We also address several promising avenues for novel medical AI research, including non-image data sources, unconventional problem formulations and human-AI collaboration. Finally, we consider serious technical and ethical challenges in issues spanning from data scarcity to racial bias. As these challenges are addressed, AI's potential may be realized, making healthcare more accurate, efficient and accessible for patients worldwide.

Labels in a haystack: Approaches beyond supervised learning in biomedical applications

Recent advances in biomedical machine learning demonstrate great potential for data-driven techniques in health care and biomedical research. However, this potential has thus far been hampered by both the scarcity of annotated data in the biomedical domain and the diversity of the domain's subfields. While unsupervised learning is capable of finding unknown patterns in the data by design, supervised learning requires human annotation to achieve the desired performance through training. With the latter performing vastly better than the former, the need for annotated datasets is high, but they are costly and laborious to obtain. This review explores a family of approaches existing between the supervised and the unsupervised problem setting. The goal of these algorithms is to make more efficient use of the available labeled data. The advantages and limitations of each approach are addressed and perspectives are provided.

Application of machine learning to predict the occurrence of arrhythmia after acute myocardial infarction

Background

Early identification of the occurrence of arrhythmia in patients with acute myocardial infarction plays an essential role in clinical decision-making. The present study attempted to use machine learning (ML) methods to build predictive models of arrhythmia after acute myocardial infarction (AMI).

Methods

A total of 2084 patients with acute myocardial infarction were enrolled in this study. (All data is available on Github: https://github.com/wangsuhuai/AMI-database1.git). The primary outcome is whether tachyarrhythmia occurred during admission containing atrial arrhythmia, ventricular arrhythmia, and supraventricular tachycardia. All data is randomly divided into a training set (80%) and an internal testing set (20%). Apply three machine learning algorithms: decision tree, random forest (RF), and artificial neural network (ANN) to learn the training set to build a model, then use the testing set to evaluate the prediction performance, and compare it with the model built by the Global Registry of Acute Coronary Events (GRACE) risk variable set.

Results

Three ML models predict the occurrence of tachyarrhythmias after AMI. After variable selection, the artificial neural network (ANN) model has reached the highest accuracy rate, which is better than the model constructed using the Grace variable set. After applying SHapley Additive exPlanations (SHAP) to make the model interpretable, the most important features are abnormal wall motion, lesion location, bundle branch block, age, and heart rate. Among them, RBBB (odds ratio [OR]: 4.21; 95% confidence interval [CI]: 2.42–7.02), ≥ 2 ventricular walls motion abnormal (OR: 3.26; 95% CI: 2.01–4.36) and right coronary artery occlusion (OR: 3.00; 95% CI: 1.98–4.56) are significant factors related to arrhythmia after AMI.

Conclusions

We used advanced machine learning methods to build prediction models for tachyarrhythmia after AMI for the first time (especially the ANN model that has the best performance). The current study can supplement the current AMI risk score, provide a reliable evaluation method for the clinic, and broaden the new horizons of ML and clinical research.

Trial registration Clinical Trial Registry No.: ChiCTR2100041960.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01667-8.

Separating Hope from Hype: Artificial Intelligence Pitfalls and Challenges in Radiology

Although recent scientific studies suggest that artificial intelligence (AI) could provide value in many radiology applications, much of the hard engineering work required to consistently realize this value in practice remains to be done. In this article, we summarize the various ways in which AI can benefit radiology practice, identify key challenges that must be overcome for those benefits to be delivered, and discuss promising avenues by which these challenges can be addressed.

Clinician-driven artificial intelligence in ophthalmology: resources enabling democratization

PURPOSE OF REVIEW

This article aims to discuss the current state of resources enabling the democratization of artificial intelligence (AI) in ophthalmology.

RECENT FINDINGS

Open datasets, efficient labeling techniques, code-free automated machine learning (AutoML) and cloud-based platforms for deployment are resources that enable clinicians with scarce resources to drive their own AI projects.

SUMMARY

Clinicians are the use-case experts who are best suited to drive AI projects tackling patient-relevant outcome measures. Taken together, open datasets, efficient labeling techniques, code-free AutoML and cloud platforms break the barriers for clinician-driven AI. As AI becomes increasingly democratized through such tools, clinicians and patients stand to benefit greatly.

Improving broad-coverage medical entity linking with semantic type prediction and large-scale datasets

OBJECTIVES

Biomedical natural language processing tools are increasingly being applied for broad-coverage information extraction-extracting medical information of all types in a scientific document or a clinical note. In such broad-coverage settings, linking mentions of medical concepts to standardized vocabularies requires choosing the best candidate concepts from large inventories covering dozens of types. This study presents a novel semantic type prediction module for biomedical NLP pipelines and two automatically-constructed, large-scale datasets with broad coverage of semantic types.

METHODS

We experiment with five off-the-shelf biomedical NLP toolkits on four benchmark datasets for medical information extraction from scientific literature and clinical notes. All toolkits adopt a staged approach of mention detection followed by two stages of medical entity linking: (1) generating a list of candidate concepts, and (2) picking the best concept among them. We introduce a semantic type prediction module to alleviate the problem of overgeneration of candidate concepts by filtering out irrelevant candidate concepts based on the predicted semantic type of a mention. We present MedType, a fully modular semantic type prediction model which we integrate into the existing NLP toolkits. To address the dearth of broad-coverage training data for medical information extraction, we further present WikiMed and PubMedDS, two large-scale datasets for medical entity linking.

RESULTS

Semantic type filtering improves medical entity linking performance across all toolkits and datasets, often by several percentage points of F-1. Further, pretraining MedType on our novel datasets achieves state-of-the-art performance for semantic type prediction in biomedical text.

CONCLUSIONS

Semantic type prediction is a key part of building accurate NLP pipelines for broad-coverage information extraction from biomedical text. We make our source code and novel datasets publicly available to foster reproducible research.

The Automatic Detection of Dataset Names in Scientific Articles

We study the task of recognizing named datasets in scientific articles as a Named Entity Recognition (NER) problem. Noticing that available annotated datasets were not adequate for our goals, we annotated 6000 sentences extracted from four major AI conferences, with roughly half of them containing one or more named datasets. A distinguishing feature of this set is the many sentences using enumerations, conjunctions and ellipses, resulting in long BI+ tag sequences. On all measures, the SciBERT NER tagger performed best and most robustly. Our baseline rule based tagger performed remarkably well and better than several state-of-the-art methods. The gold standard dataset, with links and offsets from each sentence to the (open access available) articles together with the annotation guidelines and all code used in the experiments, is available on GitHub.

Progress and Research Priorities in Imaging Genomics for Heart and Lung Disease: Summary of an NHLBI Workshop

Imaging genomics is a rapidly evolving field that combines state-of-the-art bioimaging with genomic information to resolve phenotypic heterogeneity associated with genomic variation, improve risk prediction, discover prevention approaches, and enable precision diagnosis and treatment. Contemporary bioimaging methods provide exceptional resolution generating discrete and quantitative high-dimensional phenotypes for genomics investigation. Despite substantial progress in combining high-dimensional bioimaging and genomic data, methods for imaging genomics are evolving. Recognizing the potential impact of imaging genomics on the study of heart and lung disease, the National Heart, Lung, and Blood Institute convened a workshop to review cutting-edge approaches and methodologies in imaging genomics studies, and to establish research priorities for future investigation. This report summarizes the presentations and discussions at the workshop. In particular, we highlight the need for increased availability of imaging genomics data in diverse populations, dedicated focus on less common conditions, and centralization of efforts around specific disease areas.

ACE: the Advanced Cohort Engine for searching longitudinal patient records

OBJECTIVE

To propose a paradigm for a scalable time-aware clinical data search, and to describe the design, implementation and use of a search engine realizing this paradigm.

MATERIALS AND METHODS

The Advanced Cohort Engine (ACE) uses a temporal query language and in-memory datastore of patient objects to provide a fast, scalable, and expressive time-aware search. ACE accepts data in the Observational Medicine Outcomes Partnership Common Data Model, and is configurable to balance performance with compute cost. ACE's temporal query language supports automatic query expansion using clinical knowledge graphs. The ACE API can be used with R, Python, Java, HTTP, and a Web UI.

RESULTS

ACE offers an expressive query language for complex temporal search across many clinical data types with multiple output options. ACE enables electronic phenotyping and cohort-building with subsecond response times in searching the data of millions of patients for a variety of use cases.

DISCUSSION

ACE enables fast, time-aware search using a patient object-centric datastore, thereby overcoming many technical and design shortcomings of relational algebra-based querying. Integrating electronic phenotype development with cohort-building enables a variety of high-value uses for a learning health system. Tradeoffs include the need to learn a new query language and the technical setup burden.

CONCLUSION

ACE is a tool that combines a unique query language for time-aware search of longitudinal patient records with a patient object datastore for rapid electronic phenotyping, cohort extraction, and exploratory data analyses.

Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset

The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in a large chest X-ray dataset. We characterized the effectiveness of data Shapley in identifying low quality versus valuable data for pneumonia detection. We found that removing training data with high Shapley values decreased the pneumonia detection performance, whereas removing data with low Shapley values improved the model performance. Furthermore, there were more mislabeled examples in low Shapley value data and more true pneumonia cases in high Shapley value data. Our results suggest that low Shapley value indicates mislabeled or poor quality images, whereas high Shapley value indicates data that are valuable for pneumonia detection. Our method can serve as a framework for using data Shapley to denoise large-scale medical imaging datasets.

Ontology-driven weak supervision for clinical entity classification in electronic health records

In the electronic health record, using clinical notes to identify entities such as disorders and their temporality (e.g. the order of an event relative to a time index) can inform many important analyses. However, creating training data for clinical entity tasks is time consuming and sharing labeled data is challenging due to privacy concerns. The information needs of the COVID-19 pandemic highlight the need for agile methods of training machine learning models for clinical notes. We present Trove, a framework for weakly supervised entity classification using medical ontologies and expert-generated rules. Our approach, unlike hand-labeled notes, is easy to share and modify, while offering performance comparable to learning from manually labeled training data. In this work, we validate our framework on six benchmark tasks and demonstrate Trove's ability to analyze the records of patients visiting the emergency department at Stanford Health Care for COVID-19 presenting symptoms and risk factors.

Multi-task weak supervision enables anatomically-resolved abnormality detection in whole-body FDG-PET/CT

Computational decision support systems could provide clinical value in whole-body FDG-PET/CT workflows. However, limited availability of labeled data combined with the large size of PET/CT imaging exams make it challenging to apply existing supervised machine learning systems. Leveraging recent advancements in natural language processing, we describe a weak supervision framework that extracts imperfect, yet highly granular, regional abnormality labels from free-text radiology reports. Our framework automatically labels each region in a custom ontology of anatomical regions, providing a structured profile of the pathologies in each imaging exam. Using these generated labels, we then train an attention-based, multi-task CNN architecture to detect and estimate the location of abnormalities in whole-body scans. We demonstrate empirically that our multi-task representation is critical for strong performance on rare abnormalities with limited training data. The representation also contributes to more accurate mortality prediction from imaging data, suggesting the potential utility of our framework beyond abnormality detection and location estimation.

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Linking clinical narratives to standardized vocabularies and coding systems is a key component of unlocking the information in medical text for analysis. However, many domains of medical concepts, such as functional outcomes and social determinants of health, lack well-developed terminologies that can support effective coding of medical text. We present a framework for developing natural language processing (NLP) technologies for automated coding of medical information in under-studied domains, and demonstrate its applicability through a case study on physical mobility function. Mobility function is a component of many health measures, from post-acute care and surgical outcomes to chronic frailty and disability, and is represented as one domain of human activity in the International Classification of Functioning, Disability, and Health (ICF). However, mobility and other types of functional activity remain under-studied in the medical informatics literature, and neither the ICF nor commonly-used medical terminologies capture functional status terminology in practice. We investigated two data-driven paradigms, classification and candidate selection, to link narrative observations of mobility status to standardized ICF codes, using a dataset of clinical narratives from physical therapy encounters. Recent advances in language modeling and word embedding were used as features for established machine learning models and a novel deep learning approach, achieving a macro-averaged F-1 score of 84% on linking mobility activity reports to ICF codes. Both classification and candidate selection approaches present distinct strengths for automated coding in under-studied domains, and we highlight that the combination of (i) a small annotated data set; (ii) expert definitions of codes of interest; and (iii) a representative text corpus is sufficient to produce high-performing automated coding systems. This research has implications for continued development of language technologies to analyze functional status information, and the ongoing growth of NLP tools for a variety of specialized applications in clinical care and research.

Artificial Intelligence and Transcatheter Interventions for Structural Heart Disease: A glance at the (near) future

With innovations in therapeutic technologies and changes in population demographics, transcatheter interventions for structural heart disease have become the preferred treatment and will keep growing. Yet, a thorough clinical selection and efficient pathway from diagnosis to treatment and follow-up are mandatory. In this review we reflect on how artificial intelligence may help to improve patient selection, pre-procedural planning, procedure execution and follow-up so to establish efficient and high quality health care in an increasing number of patients.

Promise and Peril of Population Genomics for the Development of Genome-First Approaches in Mendelian Cardiovascular Disease

The rich tradition of cardiovascular genomics has placed the field in prime position to extend our knowledge toward a genome-first approach to diagnosis and therapy. Population-scale genomic data has enabled exponential improvements in our ability to adjudicate variant pathogenicity based on allele rarity, and there has been a significant effort to employ these sizeable data in the investigation of rare disease. Certainly, population genomics data has great potential to aid the development of a genome-first approach to Mendelian cardiovascular disease, but its use in the clinical and investigative decision making is limited by the characteristics of the populations studied, and the evolutionary constraints on human Mendelian variation. To truly empower clinicians and patients, the successful implementation of a genome-first approach to rare cardiovascular disease will require the nuanced incorporation of population-based discovery with detailed investigation of rare disease cohorts and prospective variant evaluation.

Machine Learning and the Future of Cardiovascular Care: JACC State-of-the-Art Review

The role of physicians has always been to synthesize the data available to them to identify diagnostic patterns that guide treatment and follow response. Today, increasingly sophisticated machine learning algorithms may grow to support clinical experts in some of these tasks. Machine learning has the potential to benefit patients and cardiologists, but only if clinicians take an active role in bringing these new algorithms into practice. The aim of this review is to introduce clinicians who are not data science experts to key concepts in machine learning that will allow them to better understand the field and evaluate new literature and developments. The current published data in machine learning for cardiovascular disease is then summarized, using both a bibliometric survey, with code publicly available to enable similar analysis for any research topic of interest, and select case studies. Finally, several ways that clinicians can and must be involved in this emerging field are presented.

Cardiac Imaging of Aortic Valve Area From 34 287 UK Biobank Participants Reveals Novel Genetic Associations and Shared Genetic Comorbidity With Multiple Disease Phenotypes

Background:

The aortic valve is an important determinant of cardiovascular physiology and anatomic location of common human diseases.

Methods:

From a sample of 34 287 white British ancestry participants, we estimated functional aortic valve area by planimetry from prospectively obtained cardiac magnetic resonance imaging sequences of the aortic valve. Aortic valve area measurements were submitted to genome-wide association testing, followed by polygenic risk scoring and phenome-wide screening, to identify genetic comorbidities.

Results:

A genome-wide association study of aortic valve area in these UK Biobank participants showed 3 significant associations, indexed by rs71190365 (chr13:50764607, DLEU1 , P =1.8×10 −9 ), rs35991305 (chr12:94191968, CRADD , P =3.4×10 −8 ), and chr17:45013271:C:T ( GOSR2 , P =5.6×10 −8 ). Replication on an independent set of 8145 unrelated European ancestry participants showed consistent effect sizes in all 3 loci, although rs35991305 did not meet nominal significance. We constructed a polygenic risk score for aortic valve area, which in a separate cohort of 311 728 individuals without imaging demonstrated that smaller aortic valve area is predictive of increased risk for aortic valve disease (odds ratio, 1.14; P =2.3×10 −6 ). After excluding subjects with a medical diagnosis of aortic valve stenosis (remaining n=308 683 individuals), phenome-wide association of >10 000 traits showed multiple links between the polygenic score for aortic valve disease and key health-related comorbidities involving the cardiovascular system and autoimmune disease. Genetic correlation analysis supports a shared genetic etiology with between aortic valve area and birth weight along with other cardiovascular conditions.

Conclusions:

These results illustrate the use of automated phenotyping of cardiac imaging data from the general population to investigate the genetic etiology of aortic valve disease, perform clinical prediction, and uncover new clinical and genetic correlates of cardiac anatomy.

Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis

Supervised training of deep learning models requires large labeled datasets. There is a growing interest in obtaining such datasets for medical image analysis applications. However, the impact of label noise has not received sufficient attention. Recent studies have shown that label noise can significantly impact the performance of deep learning models in many machine learning and computer vision applications. This is especially concerning for medical applications, where datasets are typically small, labeling requires domain expertise and suffers from high inter- and intra-observer variability, and erroneous predictions may influence decisions that directly impact human health. In this paper, we first review the state-of-the-art in handling label noise in deep learning. Then, we review studies that have dealt with label noise in deep learning for medical image analysis. Our review shows that recent progress on handling label noise in deep learning has gone largely unnoticed by the medical image analysis community. To help achieve a better understanding of the extent of the problem and its potential remedies, we conducted experiments with three medical imaging datasets with different types of label noise, where we investigated several existing strategies and developed new methods to combat the negative effect of label noise. Based on the results of these experiments and our review of the literature, we have made recommendations on methods that can be used to alleviate the effects of different types of label noise on deep models trained for medical image analysis. We hope that this article helps the medical image analysis researchers and developers in choosing and devising new techniques that effectively handle label noise in deep learning.

Development and application of artificial intelligence in cardiac imaging

In this review, we describe the technical aspects of artificial intelligence (AI) in cardiac imaging, starting with radiomics, basic algorithms of deep learning and application tasks of algorithms, until recently the availability of the public database. Subsequently, we conducted a systematic literature search for recently published clinically relevant studies on AI in cardiac imaging. As a result, 24 and 14 studies using CT and MRI, respectively, were included and summarized. From these studies, it can be concluded that AI is widely applied in cardiac applications in the clinic, including coronary calcium scoring, coronary CT angiography, fractional flow reserve CT, plaque analysis, left ventricular myocardium analysis, diagnosis of myocardial infarction, prognosis of coronary artery disease, assessment of cardiac function, and diagnosis and prognosis of cardiomyopathy. These advancements show that AI has a promising prospect in cardiac imaging.

Multispectral Imaging for Automated Tissue Identification of Normal Human Surgical Specimens

OBJECTIVE

Safe surgery requires the accurate discrimination of tissue intraoperatively. We assess the feasibility of using multispectral imaging and deep learning to enhance surgical vision by automated identification of normal human head and neck tissues.

STUDY DESIGN

Construction and feasibility testing of novel multispectral imaging system for surgery.

SETTING

Academic university hospital.

SUBJECTS AND METHODS

Multispectral images of fresh-preserved human cadaveric tissues were captured with our adapted digital operating microscope. Eleven tissue types were sampled, each sequentially exposed to 6 lighting conditions. Two convolutional neural network machine learning models were developed to classify tissues based on multispectral and white-light color images (ARRInet-M and ARRInet-W, respectively). Blinded otolaryngology residents were asked to identify tissue specimens from white-light color images, and their performance was compared with that of the ARRInet models.

RESULTS

A novel multispectral imaging system was developed with minimal adaptation to an existing digital operating microscope. With 81.8% accuracy in tissue identification of full-size images, the multispectral ARRInet-M classifier outperformed the white-light-only ARRInet-W model (45.5%) and surgical residents (69.7%). Challenges with discrimination occurred with parotid vs fat and blood vessels vs nerve.

CONCLUSIONS

A deep learning model using multispectral imaging outperformed a similar model and surgical residents using traditional white-light imaging at the task of classifying normal human head and neck tissue ex vivo. These results suggest that multispectral imaging can enhance surgical vision and augment surgeons' ability to identify tissues during a procedure.

Ontology-driven weak supervision for clinical entity classification in electronic health records

In the electronic health record, using clinical notes to identify entities such as disorders and their temporality (e.g. the order of an event relative to a time index) can inform many important analyses. However, creating training data for clinical entity tasks is time consuming and sharing labeled data is challenging due to privacy concerns. The information needs of the COVID-19 pandemic highlight the need for agile methods of training machine learning models for clinical notes. We present Trove, a framework for weakly supervised entity classification using medical ontologies and expert-generated rules. Our approach, unlike hand-labeled notes, is easy to share and modify, while offering performance comparable to learning from manually labeled training data. In this work, we validate our framework on six benchmark tasks and demonstrate Trove's ability to analyze the records of patients visiting the emergency department at Stanford Health Care for COVID-19 presenting symptoms and risk factors.

Clinical applications of artificial intelligence in cardiology on the verge of the decade

Artificial intelligence (AI) has been hailed as the fourth industrial revolution and its influence on people's lives is increasing. The research on AI applications in medicine is progressing rapidly. This revolution shows promise for more precise diagnoses, streamlined workflows, increased accessibility to healthcare services and new insights into ever-growing population-wide datasets. While some applications have already found their way into contemporary patient care, we are still in the early days of the AI-era in medicine. Despite the popularity of these new technologies, many practitioners lack an understanding of AI methods, their benefits, and pitfalls. This review aims to provide information about the general concepts of machine learning (ML) with special focus on the applications of such techniques in cardiovascular medicine. It also sets out the current trends in research related to medical applications of AI. Along with new possibilities, new threats arise - acknowledging and understanding them is as important as understanding the ML methodology itself. Therefore, attention is also paid to the current opinions and guidelines regarding the validation and safety of AI-powered tools.

Automatic T Staging Using Weakly Supervised Deep Learning for Nasopharyngeal Carcinoma on MR Images

BACKGROUND

Recent studies have shown that deep learning can help tumor staging automatically. However, automatic nasopharyngeal carcinoma (NPC) staging is difficult due to the lack of large and slice-level annotated datasets.

PURPOSE

To develop a weakly-supervised deep-learning method to predict NPC patients' T stage without additional annotations.

STUDY TYPE

Retrospective.

POPULATION/SUBJECTS

In all, 1138 cases with NPC from 2010 to 2012 were enrolled, including a training set (n = 712) and a validation set (n = 426).

FIELD STRENGTH/SEQUENCE

1.5T, T₁ -weighted images (T₁ WI), T₂ -weighted images (T₂ WI), contrast-enhanced T₁ -weighted images (CE-T₁ WI).

ASSESSMENT

We used a weakly-supervised deep-learning network to achieve automated T staging of NPC. T usually refers to the size and extent of the main tumor. The training set was employed to construct the deep-learning model. The performance of the automated T staging model was evaluated in the validation set. The accuracy of the model was assessed by the receiver operating characteristic (ROC) curve. To further assess the performance of the deep-learning-based T score, the progression-free survival (PFS) and overall survival (OS) were performed.

STATISTICAL TESTS

The Sklearn package in Python was applied to calculate the area under the curve (AUC) of the ROC. The survcomp package was used for calculations and comparisons between C-indexes. The software SPSS was employed to conduct survival analysis and chi-square tests.

RESULTS

The accuracy of the deep-learning model was 75.59% in the validation set. The average AUC of the ROC curve of different stages was 0.943. There were no significant differences in the C-indexes of PFS and OS from the deep-learning model and those from TNM staging, with P values of 0.301 and 0.425, respectively.

DATA CONCLUSION

This weakly-supervised deep-learning approach can perform fully automated T staging of NPC and achieve good prognostic performance.

LEVEL OF EVIDENCE

3 Technical Efficacy Stage: 2 J. Magn. Reson. Imaging 2020;52:1074-1082.

Artificial intelligence and automation in valvular heart diseases

Artificial intelligence (AI) is gradually changing every aspect of social life, and healthcare is no exception. The clinical procedures that were supposed to, and could previously only be handled by human experts can now be carried out by machines in a more accurate and efficient way. The coming era of big data and the advent of supercomputers provides great opportunities to the development of AI technology for the enhancement of diagnosis and clinical decision-making. This review provides an introduction to AI and highlights its applications in the clinical flow of diagnosing and treating valvular heart diseases (VHDs). More specifically, this review first introduces some key concepts and subareas in AI. Secondly, it discusses the application of AI in heart sound auscultation and medical image analysis for assistance in diagnosing VHDs. Thirdly, it introduces using AI algorithms to identify risk factors and predict mortality of cardiac surgery. This review also describes the state-of-the-art autonomous surgical robots and their roles in cardiac surgery and intervention.

How Will Machine Learning Inform the Clinical Care of Atrial Fibrillation?

Machine learning applications in cardiology have rapidly evolved in the past decade. With the availability of machine learning tools coupled with vast data sources, the management of atrial fibrillation (AF), a common chronic disease with significant associated morbidity and socioeconomic impact, is undergoing a knowledge and practice transformation in the increasingly complex healthcare environment. Among other advances, deep-learning machine learning methods, including convolutional neural networks, have enabled the development of AF screening pathways using the ubiquitous 12-lead ECG to detect asymptomatic paroxysmal AF in at-risk populations (such as those with cryptogenic stroke), the refinement of AF and stroke prediction schemes through comprehensive digital phenotyping using structured and unstructured data abstraction from the electronic health record or wearable monitoring technologies, and the optimization of treatment strategies, ranging from stroke prophylaxis to monitoring of antiarrhythmic drug (AAD) therapy. Although the clinical and population-wide impact of these tools continues to be elucidated, such transformative progress does not come without challenges, such as the concerns about adopting black box technologies, assessing input data quality for training such models, and the risk of perpetuating rather than alleviating health disparities. This review critically appraises the advances of machine learning related to the care of AF thus far, their potential future directions, and its potential limitations and challenges.

The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions

UK Biobank is a population-based cohort of half a million participants aged 40-69 years recruited between 2006 and 2010. In 2014, UK Biobank started the world's largest multi-modal imaging study, with the aim of re-inviting 100,000 participants to undergo brain, cardiac and abdominal magnetic resonance imaging, dual-energy X-ray absorptiometry and carotid ultrasound. The combination of large-scale multi-modal imaging with extensive phenotypic and genetic data offers an unprecedented resource for scientists to conduct health-related research. This article provides an in-depth overview of the imaging enhancement, including the data collected, how it is managed and processed, and future directions.

Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging

Machine learning models for medical image analysis often suffer from poor performance on important subsets of a population that are not identified during training or testing. For example, overall performance of a cancer detection model may be high, but the model may still consistently miss a rare but aggressive cancer subtype. We refer to this problem as hidden stratification, and observe that it results from incompletely describing the meaningful variation in a dataset. While hidden stratification can substantially reduce the clinical efficacy of machine learning models, its effects remain difficult to measure. In this work, we assess the utility of several possible techniques for measuring hidden stratification effects, and characterize these effects both via synthetic experiments on the CIFAR-100 benchmark dataset and on multiple real-world medical imaging datasets. Using these measurement techniques, we find evidence that hidden stratification can occur in unidentified imaging subsets with low prevalence, low label quality, subtle distinguishing features, or spurious correlates, and that it can result in relative performance differences of over 20% on clinically important subsets. Finally, we discuss the clinical implications of our findings, and suggest that evaluation of hidden stratification should be a critical component of any machine learning deployment in medical imaging.

Challenges of developing a digital scribe to reduce clinical documentation burden

Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout. Advances in artificial intelligence (AI) and machine learning (ML) open the possibility of automating clinical documentation with digital scribes, using speech recognition to eliminate manual documentation by clinicians or medical scribes. However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations. This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts using speech recognition, inducing topic structure from conversation data, extracting medical concepts, generating clinically meaningful summaries of conversations, and obtaining clinical data for AI and ML algorithms.

Snorkel: rapid training data creation with weak supervision

Labeling training data is increasingly the largest bottleneck in deploying machine learning systems. We present Snorkel, a first-of-its-kind system that enables users to train state-of-the-art models without hand labeling any training data. Instead, users write labeling functions that express arbitrary heuristics, which can have unknown accuracies and correlations. Snorkel denoises their outputs without access to ground truth by incorporating the first end-to-end implementation of our recently proposed machine learning paradigm, data programming. We present a flexible interface layer for writing labeling functions based on our experience over the past year collaborating with companies, agencies, and research laboratories. In a user study, subject matter experts build models 2.8 × faster and increase predictive performance an average 45.5 % versus seven hours of hand labeling. We study the modeling trade-offs in this new setting and propose an optimizer for automating trade-off decisions that gives up to 1.8 × speedup per pipeline execution. In two collaborations, with the US Department of Veterans Affairs and the US Food and Drug Administration, and on four open-source text and image data sets representative of other deployments, Snorkel provides 132 % average improvements to predictive performance over prior heuristic approaches and comes within an average 3.60 % of the predictive performance of large hand-curated training sets.