Retinal imaging for the assessment of stroke risk: a systematic review

BACKGROUND

Stroke is a leading cause of morbidity and mortality. Retinal imaging allows non-invasive assessment of the microvasculature. Consequently, retinal imaging is a technology which is garnering increasing attention as a means of assessing cardiovascular health and stroke risk.

METHODS

A biomedical literature search was performed to identify prospective studies that assess the role of retinal imaging derived biomarkers as indicators of stroke risk.

RESULTS

Twenty-four studies were included in this systematic review. The available evidence suggests that wider retinal venules, lower fractal dimension, increased arteriolar tortuosity, presence of retinopathy, and presence of retinal emboli are associated with increased likelihood of stroke. There is weaker evidence to suggest that narrower arterioles and the presence of individual retinopathy traits such as microaneurysms and arteriovenous nicking indicate increased stroke risk. Our review identified three models utilizing artificial intelligence algorithms for the analysis of retinal images to predict stroke. Two of these focused on fundus photographs, whilst one also utilized optical coherence tomography (OCT) technology images. The constructed models performed similarly to conventional risk scores but did not significantly exceed their performance. Only two studies identified in this review used OCT imaging, despite the higher dimensionality of this data.

CONCLUSION

Whilst there is strong evidence that retinal imaging features can be used to indicate stroke risk, there is currently no predictive model which significantly outperforms conventional risk scores. To develop clinically useful tools, future research should focus on utilization of deep learning algorithms, validation in external cohorts, and analysis of OCT images.

A Generative Shape Compositional Framework to Synthesize Populations of Virtual Chimeras

Generating virtual organ populations that capture sufficient variability while remaining plausible is essential to conduct in silico trials (ISTs) of medical devices. However, not all anatomical shapes of interest are always available for each individual in a population. The imaging examinations and modalities used can vary between subjects depending on their individualized clinical pathways. Different imaging modalities may have various fields of view and are sensitive to signals from other tissues/organs, or both. Hence, missing/partially overlapping anatomical information is often available across individuals. We introduce a generative shape model for multipart anatomical structures, learnable from sets of unpaired datasets, i.e., where each substructure in the shape assembly comes from datasets with missing or partially overlapping substructures from disjoint subjects of the same population. The proposed generative model can synthesize complete multipart shape assemblies coined virtual chimeras (VCs). We applied this framework to build VCs from databases of whole-heart shape assemblies that each contribute samples for heart substructures. Specifically, we propose a graph neural network-based generative shape compositional framework, which comprises two components, a part-aware generative shape model that captures the variability in shape observed for each structure of interest in the training population and a spatial composition network that assembles/composes the structures synthesized by the former into multipart shape assemblies (i.e., VCs). We also propose a novel self-supervised learning scheme that enables the spatial composition network to be trained with partially overlapping data and weak labels. We trained and validated our approach using shapes of cardiac structures derived from cardiac magnetic resonance (MR) images in the UK Biobank (UKBB). When trained with complete and partially overlapping data, our approach significantly outperforms a principal component analysis (PCA)-based shape model (trained with complete data) in terms of generalizability and specificity. This demonstrates the superiority of the proposed method, as the synthesized cardiac virtual populations are more plausible and capture a greater degree of shape variability than those generated by the PCA-based shape model.

Unsupervised ensemble-based phenotyping enhances discoverability of genes related to left-ventricular morphology

Recent genome-wide association studies have successfully identified associations between genetic variants and simple cardiac morphological parameters derived from cardiac magnetic resonance images. However, the emergence of large databases, including genetic data linked to cardiac magnetic resonance facilitates the investigation of more nuanced patterns of cardiac shape variability than those studied so far. Here we propose a framework for gene discovery coined unsupervised phenotype ensembles. The unsupervised phenotype ensemble builds a redundant yet highly expressive representation by pooling a set of phenotypes learnt in an unsupervised manner, using deep learning models trained with different hyperparameters. These phenotypes are then analysed via genome-wide association studies, retaining only highly confident and stable associations across the ensemble. We applied our approach to the UK Biobank database to extract geometric features of the left ventricle from image-derived three-dimensional meshes. We demonstrate that our approach greatly improves the discoverability of genes that influence left ventricle shape, identifying 49 loci with study-wide significance and 25 with suggestive significance. We argue that our approach would enable more extensive discovery of gene associations with image-derived phenotypes for other organs or image modalities.

Deep segmentation of OCTA for evaluation and association of changes of retinal microvasculature with Alzheimer's disease and mild cognitive impairment

BACKGROUND

Optical coherence tomography angiography (OCTA) enables fast and non-invasive high-resolution imaging of retinal microvasculature and is suggested as a potential tool in the early detection of retinal microvascular changes in Alzheimer's Disease (AD). We developed a standardised OCTA analysis framework and compared their extracted parameters among controls and AD/mild cognitive impairment (MCI) in a cross-section study.

METHODS

We defined and extracted geometrical parameters of retinal microvasculature at different retinal layers and in the foveal avascular zone (FAZ) from segmented OCTA images obtained using well-validated state-of-the-art deep learning models. We studied these parameters in 158 subjects (62 healthy control, 55 AD and 41 MCI) using logistic regression to determine their potential in predicting the status of our subjects.

RESULTS

In the AD group, there was a significant decrease in vessel area and length densities in the inner vascular complexes (IVC) compared with controls. The number of vascular bifurcations in AD is also significantly lower than that of healthy people. The MCI group demonstrated a decrease in vascular area, length densities, vascular fractal dimension and the number of bifurcations in both the superficial vascular complexes (SVC) and the IVC compared with controls. A larger vascular tortuosity in the IVC, and a larger roundness of FAZ in the SVC, can also be observed in MCI compared with controls.

CONCLUSION

Our study demonstrates the applicability of OCTA for the diagnosis of AD and MCI, and provides a standard tool for future clinical service and research. Biomarkers from retinal OCTA images can provide useful information for clinical decision-making and diagnosis of AD and MCI.

Beyond images: an integrative multi-modal approach to chest x-ray report generation

Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. In this paper, we present a novel multi-modal deep neural network framework for generating chest x-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes. We introduce a conditioned cross-multi-head attention module to fuse these heterogeneous data modalities, bridging the semantic gap between visual and textual data. Experiments demonstrate substantial improvements from using additional modalities compared to relying on images alone. Notably, our model achieves the highest reported performance on the ROUGE-L metric compared to relevant state-of-the-art models in the literature. Furthermore, we employed both human evaluation and clinical semantic similarity measurement alongside word-overlap metrics to improve the depth of quantitative analysis. A human evaluation, conducted by a board-certified radiologist, confirms the model's accuracy in identifying high-level findings, however, it also highlights that more improvement is needed to capture nuanced details and clinical context.

Accelerated simulation methodologies for computational vascular flow modelling

Vascular flow modelling can improve our understanding of vascular pathologies and aid in developing safe and effective medical devices. Vascular flow models typically involve solving the nonlinear Navier-Stokes equations in complex anatomies and using physiological boundary conditions, often presenting a multi-physics and multi-scale computational problem to be solved. This leads to highly complex and expensive models that require excessive computational time. This review explores accelerated simulation methodologies, specifically focusing on computational vascular flow modelling. We review reduced order modelling (ROM) techniques like zero-/one-dimensional and modal decomposition-based ROMs and machine learning (ML) methods including ML-augmented ROMs, ML-based ROMs and physics-informed ML models. We discuss the applicability of each method to vascular flow acceleration and the effectiveness of the method in addressing domain-specific challenges. When available, we provide statistics on accuracy and speed-up factors for various applications related to vascular flow simulation acceleration. Our findings indicate that each type of model has strengths and limitations depending on the context. To accelerate real-world vascular flow problems, we propose future research on developing multi-scale acceleration methods capable of handling the significant geometric variability inherent to such problems.

A generalised deep meta-learning model for automated quality control of cardiovascular magnetic resonance images

BACKGROUND AND OBJECTIVES

Cardiovascular magnetic resonance (CMR) imaging is a powerful modality in functional and anatomical assessment for various cardiovascular diseases. Sufficient image quality is essential to achieve proper diagnosis and treatment. A large number of medical images, the variety of imaging artefacts, and the workload of imaging centres are amongst the factors that reveal the necessity of automatic image quality assessment (IQA). However, automated IQA requires access to bulk annotated datasets for training deep learning (DL) models. Labelling medical images is a tedious, costly and time-consuming process, which creates a fundamental challenge in proposing DL-based methods for medical applications. This study aims to present a new method for CMR IQA when there is limited access to annotated datasets.

METHODS

The proposed generalised deep meta-learning model can evaluate the quality by learning tasks in the prior stage and then fine-tuning the resulting model on a small labelled dataset of the desired tasks. This model was evaluated on the data of over 6,000 subjects from the UK Biobank for five defined tasks, including detecting respiratory motion, cardiac motion, Aliasing and Gibbs ringing artefacts and images without artefacts.

RESULTS

The results of extensive experiments show the superiority of the proposed model. Besides, comparing the model's accuracy with the domain adaptation model indicates a significant difference by using only 64 annotated images related to the desired tasks.

CONCLUSION

The proposed model can identify unknown artefacts in images with acceptable accuracy, which makes it suitable for medical applications and quality assessment of large cohorts. CODE AVAILABILITY: https://github.com/HosseinSimchi/META-IQA-CMRImages.

Simultaneous Hip Implant Segmentation and Gruen Landmarks Detection

The assessment of implant status and complications of Total Hip Replacement (THR) relies mainly on the clinical evaluation of the X-ray images to analyse the implant and the surrounding rigid structures. Current clinical practise depends on the manual identification of important landmarks to define the implant boundary and to analyse many features in arthroplasty X-ray images, which is time-consuming and could be prone to human error. Semantic segmentation based on the Convolutional Neural Network (CNN) has demonstrated successful results in many medical segmentation tasks. However, these networks cannot define explicit properties that lead to inaccurate segmentation, especially with the limited size of image datasets. Our work integrates clinical knowledge with CNN to segment the implant and detect important features simultaneously. This is instrumental in the diagnosis of complications of arthroplasty, particularly for loose implant and implant-closed bone fractures, where the location of the fracture in relation to the implant must be accurately determined. In this work, we define the points of interest using Gruen zones that represent the interface of the implant with the surrounding bone to build a Statistical Shape Model (SSM). We propose a multitask CNN that combines regression of pose and shape parameters constructed from the SSM and semantic segmentation of the implant. This integrated approach has improved the estimation of implant shape, from 74% to 80% dice score, making segmentation realistic and allowing automatic detection of Gruen zones. To train and evaluate our method, we generated a dataset of annotated hip arthroplasty X-ray images that will be made available.

Measuring cardiomyocyte cellular characteristics in cardiac hypertrophy using diffusion-weighted MRI

PURPOSE

This paper presents a hierarchical modeling approach for estimating cardiomyocyte major and minor diameters and intracellular volume fraction (ICV) using diffusion-weighted MRI (DWI) data in ex vivo mouse hearts.

METHODS

DWI data were acquired on two healthy controls and two hearts 3 weeks post transverse aortic constriction (TAC) using a bespoke diffusion scheme with multiple diffusion times (Δ $$ \Delta $$ ), q-shells and diffusion encoding directions. Firstly, a bi-exponential tensor model was fitted separately at each diffusion time to disentangle the dependence on diffusion times from diffusion weightings, that is, b-values. The slow-diffusing component was attributed to the restricted diffusion inside cardiomyocytes. ICV was then extrapolated atΔ = 0 $$ \Delta =0 $$ using linear regression. Secondly, given the secondary and the tertiary diffusion eigenvalue measurements for the slow-diffusing component obtained at different diffusion times, major and minor diameters were estimated assuming a cylinder model with an elliptical cross-section (ECS). High-resolution three-dimensional synchrotron X-ray imaging (SRI) data from the same specimen was utilized to evaluate the biophysical parameters.

RESULTS

Estimated parameters using DWI data were (control 1/control 2 vs. TAC 1/TAC 2): major diameter-17.4μ $$ \mu $$ m/18.0μ $$ \mu $$ m versus 19.2μ $$ \mu $$ m/19.0μ $$ \mu $$ m; minor diameter-10.2μ $$ \mu $$ m/9.4μ $$ \mu $$ m versus 12.8μ $$ \mu $$ m/13.4μ $$ \mu $$ m; and ICV-62%/62% versus 68%/47%. These findings were consistent with SRI measurements.

CONCLUSION

The proposed method allowed for accurate estimation of biophysical parameters suggesting cardiomyocyte diameters as sensitive biomarkers of hypertrophy in the heart.

Fully Automatic initialization and segmentation of left and right ventricles for large-scale cardiac MRI using a deeply supervised network and 3D-ASM

BACKGROUND AND OBJECTIVE

The sheer volume of data generated by population imaging studies is unparalleled by current capabilities to extract objective and quantitative cardiac phenotypes; subjective and time-consuming manual image analysis remains the gold standard. Automated image analytics to compute quantitative imaging biomarkers of cardiac function are desperately needed. Data volumes and their variability pose a challenge to most state-of-the-art methods for endo- and epicardial contours, which lack robustness when applied to very large datasets. Our aim is to develop an analysis pipeline for the automatic quantification of cardiac function from cine magnetic resonance imaging data.

METHOD

This work adopt 4,638 cardiac MRI cases coming from UK Biobank with ground truth available for left and RV contours. A hybrid and robust algorithm is proposed to improve the accuracy of automatic left and right ventricle segmentation by harnessing the localization accuracy of deep learning and the morphological accuracy of 3D-ASM (three-dimensional active shape models). The contributions of this paper are three-fold. First, a fully automatic method is proposed for left and right ventricle initialization and cardiac MRI segmentation by taking full advantage of spatiotemporal constraint. Second, a deeply supervised network is introduced to train and segment the heart. Third, the 3D-ASM image search procedure is improved by combining image intensity models with convolutional neural network (CNN) derived distance maps improving endo- and epicardial edge localization.

RESULTS

The proposed architecture outperformed the state of the art for cardiac MRI segmentation from UK Biobank. The statistics of RV landmarks detection errors for Triscuspid valve and RV apex are 4.17 mm and 5.58 mm separately. The overlap metric, mean contour distance, Hausdorff distance and cardiac functional parameters are calculated for the LV (Left Ventricle) and RV (Right Ventricle) contour segmentation. Bland-Altman analysis for clinical parameters shows that the results from our automated image analysis pipelines are in good agreement with results from expert manual analysis.

CONCLUSIONS

Our hybrid scheme combines deep learning and statistical shape modeling for automatic segmentation of the LV/RV from cardiac MRI datasets is effective and robust and can compute cardiac functional indexes from population imaging.

Hemodynamics of thrombus formation in intracranial aneurysms: An in silico observational study

How prevalent is spontaneous thrombosis in a population containing all sizes of intracranial aneurysms? How can we calibrate computational models of thrombosis based on published data? How does spontaneous thrombosis differ in normo- and hypertensive subjects? We address the first question through a thorough analysis of published datasets that provide spontaneous thrombosis rates across different aneurysm characteristics. This analysis provides data for a subgroup of the general population of aneurysms, namely, those of large and giant size (>10 mm). Based on these observed spontaneous thrombosis rates, our computational modeling platform enables the first in silico observational study of spontaneous thrombosis prevalence across a broader set of aneurysm phenotypes. We generate 109 virtual patients and use a novel approach to calibrate two trigger thresholds: residence time and shear rate, thus addressing the second question. We then address the third question by utilizing this calibrated model to provide new insight into the effects of hypertension on spontaneous thrombosis. We demonstrate how a mechanistic thrombosis model calibrated on an intracranial aneurysm cohort can help estimate spontaneous thrombosis prevalence in a broader aneurysm population. This study is enabled through a fully automatic multi-scale modeling pipeline. We use the clinical spontaneous thrombosis data as an indirect population-level validation of a complex computational modeling framework. Furthermore, our framework allows exploration of the influence of hypertension in spontaneous thrombosis. This lays the foundation for in silico clinical trials of cerebrovascular devices in high-risk populations, e.g., assessing the performance of flow diverters in aneurysms for hypertensive patients.

Virtual high-resolution MR angiography from non-angiographic multi-contrast MRIs: synthetic vascular model populations for in-silico trials

Despite success on multi-contrast MR image synthesis, generating specific modalities remains challenging. Those include Magnetic Resonance Angiography (MRA) that highlights details of vascular anatomy using specialised imaging sequences for emphasising inflow effect. This work proposes an end-to-end generative adversarial network that can synthesise anatomically plausible, high-resolution 3D MRA images using commonly acquired multi-contrast MR images (e.g. T1/T2/PD-weighted MR images) for the same subject whilst preserving the continuity of vascular anatomy. A reliable technique for MRA synthesis would unleash the research potential of very few population databases with imaging modalities (such as MRA) that enable quantitative characterisation of whole-brain vasculature. Our work is motivated by the need to generate digital twins and virtual patients of cerebrovascular anatomy for in-silico studies and/or in-silico trials. We propose a dedicated generator and discriminator that leverage the shared and complementary features of multi-source images. We design a composite loss function for emphasising vascular properties by minimising the statistical difference between the feature representations of the target images and the synthesised outputs in both 3D volumetric and 2D projection domains. Experimental results show that the proposed method can synthesise high-quality MRA images and outperform the state-of-the-art generative models both qualitatively and quantitatively. The importance assessment reveals that T2 and PD-weighted images are better predictors of MRA images than T1; and PD-weighted images contribute to better visibility of small vessel branches towards the peripheral regions. In addition, the proposed approach can generalise to unseen data acquired at different imaging centres with different scanners, whilst synthesising MRAs and vascular geometries that maintain vessel continuity. The results show the potential for use of the proposed approach to generating digital twin cohorts of cerebrovascular anatomy at scale from structural MR images typically acquired in population imaging initiatives.

RecON: Online learning for sensorless freehand 3D ultrasound reconstruction

Sensorless freehand 3D ultrasound (US) reconstruction based on deep networks shows promising advantages, such as large field of view, relatively high resolution, low cost, and ease of use. However, existing methods mainly consider vanilla scan strategies with limited inter-frame variations. These methods thus are degraded on complex but routine scan sequences in clinics. In this context, we propose a novel online learning framework for freehand 3D US reconstruction under complex scan strategies with diverse scanning velocities and poses. First, we devise a motion-weighted training loss in training phase to regularize the scan variation frame-by-frame and better mitigate the negative effects of uneven inter-frame velocity. Second, we effectively drive online learning with local-to-global pseudo supervisions. It mines both the frame-level contextual consistency and the path-level similarity constraint to improve the inter-frame transformation estimation. We explore a global adversarial shape before transferring the latent anatomical prior as supervision. Third, we build a feasible differentiable reconstruction approximation to enable the end-to-end optimization of our online learning. Experimental results illustrate that our freehand 3D US reconstruction framework outperformed current methods on two large, simulated datasets and one real dataset. In addition, we applied the proposed framework to clinical scan videos to further validate its effectiveness and generalizability.

High-throughput 3DRA segmentation of brain vasculature and aneurysms using deep learning

BACKGROUND AND OBJECTIVES

Automatic segmentation of the cerebral vasculature and aneurysms facilitates incidental detection of aneurysms. The assessment of aneurysm rupture risk assists with pre-operative treatment planning and enables in-silico investigation of cerebral hemodynamics within and in the vicinity of aneurysms. However, ensuring precise and robust segmentation of cerebral vessels and aneurysms in neuroimaging modalities such as three-dimensional rotational angiography (3DRA) is challenging. The vasculature constitutes a small proportion of the image volume, resulting in a large class imbalance (relative to surrounding brain tissue). Additionally, aneurysms and vessels have similar image/appearance characteristics, making it challenging to distinguish the aneurysm sac from the vessel lumen.

METHODS

We propose a novel multi-class convolutional neural network to tackle these challenges and facilitate the automatic segmentation of cerebral vessels and aneurysms in 3DRA images. The proposed model is trained and evaluated on an internal multi-center dataset and an external publicly available challenge dataset.

RESULTS

On the internal clinical dataset, our method consistently outperformed several state-of-the-art approaches for vessel and aneurysm segmentation, achieving an average Dice score of 0.81 (0.15 higher than nnUNet) and an average surface-to-surface error of 0.20 mm (less than the in-plane resolution (0.35 mm/pixel)) for aneurysm segmentation; and an average Dice score of 0.91 and average surface-to-surface error of 0.25 mm for vessel segmentation. In 223 cases of a clinical dataset, our method accurately segmented 190 aneurysm cases.

CONCLUSIONS

The proposed approach can help address class imbalance problems and inter-class interference problems in multi-class segmentation. Besides, this method performs consistently on clinical datasets from four different sources and the generated results are qualified for hemodynamic simulation. Code available at https://github.com/cistib/vessel-aneurysm-segmentation.

Multicenter and Multichannel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network

For significant memory concern (SMC) and mild cognitive impairment (MCI), their classification performance is limited by confounding features, diverse imaging protocols, and limited sample size. To address the above limitations, we introduce a dual-modality fused brain connectivity network combining resting-state functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), and propose three mechanisms in the current graph convolutional network (GCN) to improve classifier performance. First, we introduce a DTI-strength penalty term for constructing functional connectivity networks. Stronger structural connectivity and bigger structural strength diversity between groups provide a higher opportunity for retaining connectivity information. Second, a multi-center attention graph with each node representing a subject is proposed to consider the influence of data source, gender, acquisition equipment, and disease status of those training samples in GCN. The attention mechanism captures their different impacts on edge weights. Third, we propose a multi-channel mechanism to improve filter performance, assigning different filters to features based on feature statistics. Applying those nodes with low-quality features to perform convolution would also deteriorate filter performance. Therefore, we further propose a pooling mechanism, which introduces the disease status information of those training samples to evaluate the quality of nodes. Finally, we obtain the final classification results by inputting the multi-center attention graph into the multi-channel pooling GCN. The proposed method is tested on three datasets (i.e., an ADNI 2 dataset, an ADNI 3 dataset, and an in-house dataset). Experimental results indicate that the proposed method is effective and superior to other related algorithms, with a mean classification accuracy of 93.05% in our binary classification tasks. Our code is available at: https://github.com/Xuegang-S.

Toxicity Prediction in Pelvic Radiotherapy Using Multiple Instance Learning and Cascaded Attention Layers

Modern radiotherapy delivers treatment plans optimised on an individual patient level, using CT-based 3D models of patient anatomy. This optimisation is fundamentally based on simple assumptions about the relationship between radiation dose delivered to the cancer (increased dose will increase cancer control) and normal tissue (increased dose will increase rate of side effects). The details of these relationships are still not well understood, especially for radiation-induced toxicity. We propose a convolutional neural network based on multiple instance learning to analyse toxicity relationships for patients receiving pelvic radiotherapy. A dataset comprising of 315 patients were included in this study; with 3D dose distributions, pre-treatment CT scans with annotated abdominal structures, and patient-reported toxicity scores provided for each participant. In addition, we propose a novel mechanism for segregating the attentions over space and dose/imaging features independently for a better understanding of the anatomical distribution of toxicity. Quantitative and qualitative experiments were performed to evaluate the network performance. The proposed network could predict toxicity with 80% accuracy. Attention analysis over space demonstrated that there was a significant association between radiation dose to the anterior and right iliac of the abdomen and patient-reported toxicity. Experimental results showed that the proposed network had outstanding performance for toxicity prediction, localisation and explanation with the ability of generalisation for an unseen dataset.

Contribution of Shape Features to Intradiscal Pressure and Facets Contact Pressure in L4/L5 FSUs: An In-Silico Study

Finite element models (FEMs) of the spine commonly use a limited number of simplified geometries. Nevertheless, the geometric features of the spine are important in determining its FEM outcomes. The link between a spinal segment's shape and its biomechanical response has been studied, but the co-variances of the shape features have been omitted. We used a principal component (PCA)-based statistical shape modelling (SSM) approach to investigate the contribution of shape features to the intradiscal pressure (IDP) and the facets contact pressure (FCP) in a cohort of synthetic L4/L5 functional spinal units under axial compression. We quantified the uncertainty in the FEM results, and the contribution of individual shape modes to these results. This parameterisation approach is able to capture the variability in the correlated anatomical features in a real population and sample plausible synthetic geometries. The first shape mode ([Formula: see text]) explained 22.6% of the shape variation in the subject-specific cohort used to train the SSM, and had the largest correlation with, and contribution to IDP (17%) and FCP (11%). The largest geometric variation in ([Formula: see text]) was in the annulus-nucleus ratio.

DragNet: Learning-based deformable registration for realistic cardiac MR sequence generation from a single frame

Deformable image registration (DIR) can be used to track cardiac motion. Conventional DIR algorithms aim to establish a dense and non-linear correspondence between independent pairs of images. They are, nevertheless, computationally intensive and do not consider temporal dependencies to regulate the estimated motion in a cardiac cycle. In this paper, leveraging deep learning methods, we formulate a novel hierarchical probabilistic model, termed DragNet, for fast and reliable spatio-temporal registration in cine cardiac magnetic resonance (CMR) images and for generating synthetic heart motion sequences. DragNet is a variational inference framework, which takes an image from the sequence in combination with the hidden states of a recurrent neural network (RNN) as inputs to an inference network per time step. As part of this framework, we condition the prior probability of the latent variables on the hidden states of the RNN utilised to capture temporal dependencies. We further condition the posterior of the motion field on a latent variable from hierarchy and features from the moving image. Subsequently, the RNN updates the hidden state variables based on the feature maps of the fixed image and the latent variables. Different from traditional methods, DragNet performs registration on unseen sequences in a forward pass, which significantly expedites the registration process. Besides, DragNet enables generating a large number of realistic synthetic image sequences given only one frame, where the corresponding deformations are also retrieved. The probabilistic framework allows for computing spatio-temporal uncertainties in the estimated motion fields. Our results show that DragNet performance is comparable with state-of-the-art methods in terms of registration accuracy, with the advantage of offering analytical pixel-wise motion uncertainty estimation across a cardiac cycle and being a motion generator. We will make our code publicly available.

Three-dimensional micro-structurally informed in silico myocardium-Towards virtual imaging trials in cardiac diffusion weighted MRI

In silico tissue models (viz. numerical phantoms) provide a mechanism for evaluating quantitative models of magnetic resonance imaging. This includes the validation and sensitivity analysis of imaging biomarkers and tissue microstructure parameters. This study proposes a novel method to generate a realistic numerical phantom of myocardial microstructure. The proposed method extends previous studies by accounting for the variability of the cardiomyocyte shape, water exchange between the cardiomyocytes (intercalated discs), disorder class of myocardial microstructure, and four sheetlet orientations. In the first stage of the method, cardiomyocytes and sheetlets are generated by considering the shape variability and intercalated discs in cardiomyocyte-cardiomyocyte connections. Sheetlets are then aggregated and oriented in the directions of interest. The morphometric study demonstrates no significant difference (p>0.01) between the distribution of volume, length, and primary and secondary axes of the numerical and real (literature) cardiomyocyte data. Moreover, structural correlation analysis validates that the in-silico tissue is in the same class of disorderliness as the real tissue. Additionally, the absolute angle differences between the simulated helical angle (HA) and input HA (reference value) of the cardiomyocytes (4.3°±3.1°) demonstrate a good agreement with the absolute angle difference between the measured HA using experimental cardiac diffusion tensor imaging (cDTI) and histology (reference value) reported by (Holmes et al., 2000) (3.7°±6.4°) and (Scollan et al. 1998) (4.9°±14.6°). Furthermore, the angular distance between eigenvectors and sheetlet angles of the input and simulated cDTI is much smaller than those between measured angles using structural tensor imaging (as a gold standard) and experimental cDTI. Combined with the qualitative results, these results confirm that the proposed method can generate richer numerical phantoms for the myocardium than previous studies.

Dynamic feature learning for COVID-19 segmentation and classification

Since December 2019, coronavirus SARS-CoV-2 (COVID-19) has rapidly developed into a global epidemic, with millions of patients affected worldwide. As part of the diagnostic pathway, computed tomography (CT) scans are used to help patient management. However, parenchymal imaging findings in COVID-19 are non-specific and can be seen in other diseases. In this work, we propose to first segment lesions from CT images, and further, classify COVID-19 patients from healthy persons and common pneumonia patients. In detail, a novel Dynamic Fusion Segmentation Network (DFSN) that automatically segments infection-related pixels is first proposed. Within this network, low-level features are aggregated to high-level ones to effectively capture context characteristics of infection regions, and high-level features are dynamically fused to model multi-scale semantic information of lesions. Based on DFSN, Dynamic Transfer-learning Classification Network (DTCN) is proposed to distinguish COVID-19 patients. Within DTCN, a pre-trained DFSN is transferred and used as the backbone to extract pixel-level information. Then the pixel-level information is dynamically selected and used to make a diagnosis. In this way, the pre-trained DFSN is utilized through transfer learning, and clinical significance of segmentation results is comprehensively considered. Thus DTCN becomes more sensitive to typical signs of COVID-19. Extensive experiments are conducted to demonstrate effectiveness of the proposed DFSN and DTCN frameworks. The corresponding results indicate that these two models achieve state-of-the-art performance in terms of segmentation and classification.

The association between native myocardial T1 relaxation times and left atrial phasic structure and function: the UK Biobank Imaging Enhancement study

Introduction

Left ventricular (LV) myocardial fibrosis is posited to result in left atrial (LA) changes via LV remodelling and diastolic dysfunction, though the association remains poorly characterised. Native myocardial T1 mapping is a non-invasive modality that quantifies diffuse myocardial fibrosis. This study examines the relationship between LV fibrosis (quantified by native T1 times) and LA function, drawing upon data from the UK Biobank.

Methods

40,818 participants underwent cardiovascular magnetic resonance (CMR) using steady-state free precession imaging at 1.5 Tesla. Native T1-mapping was performed using the Shortened Modified Look-Locker Inversion recovery technique (ShMOLLI), with global myocardial T1 estimated by an automatic segmentation framework. Nine parameters of LA phasic function were calculated (representing global, reservoir, conduit and booster components) from normalised LA volume-time curves. LV parameters (LV Mass, end-diastolic volume and ejection fraction) were extracted by a convolutional neural network. Multivariable logistic regression models were used to assess the association between T1 (exposure) and LA function (outcome). Mediation analysis was performed to assess the role of LV parameters as a mediator for the association between T1 and LA function. Lastly, potential non-linear relationships between T1 and LA function were investigated using Restrictive Cubic Spline (RCS) modelling, with model fit assessed via the Akaike Information Criterion (AIC).

Results

Higher T1 values were positively associated with larger LA volumes, and negatively associated with markers of LA global, reservoir and booster function. In the fully adjusted model, T1 was positively associated with larger LA minimum size (Beta: +0.034 SD per T1 SD; Confidence Interval (CI): 0.024, 0.045), and negatively associated with LA emptying volume (Beta: −0.017; CI: −0.027, −0.006), LA booster volume (Beta: −0.019; CI: −0.030, −0.008), LA emptying fraction (Beta: −0.052; CI: −0.062, −0.041), and LA reservoir function (Beta: −0.028; CI: −0.039, −0.017). Though adjustment for LV parameters did not fully attenuate the above relationships, LV parameters were consistent mediators between T1 and LA function, with proportional mediative effects ranging from 15% to 75%. Lastly, there is evidence of an inverted J-shaped relationship between T1 and LA function, with the associations becoming more apparent in the upper half of T1 ranges (turning points within 925–950 ms, median T1 = 930 ms) (p<0.05).

Conclusion

This study demonstrates a consistent association between higher native T1 values (as a marker of myocardial fibrosis) and lower LA global and phasic functions. We also highlighted an interplay between T1 values, LV remodelling and LA dysfunction. These findings will facilitate our understanding of the disease processes underlying cardiac dysfunction and myocardial remodelling at an early, subclinical stage.

Funding Acknowledgement

Type of funding sources: Public hospital(s). Main funding source(s): This work was part of the portfolio of translational research of the National Institute for Health Research Biomedical Research Centre at Barts and The London School of Medicine and DentistryDr Nay Aung is supported by a Wellcome Trust Research Training Fellowship (203553/Z/16/Z)

Quantitating Age-Related BMD Textural Variation from DXA Region-Free-Analysis: A Study of Hip Fracture Prediction in Three Cohorts

The risk of osteoporotic fracture is inversely related to bone mineral density (BMD), but how spatial BMD pattern influences fracture risk remains incompletely understood. This study used a pixel-level spatiotemporal atlas of proximal femoral BMD in 13,338 white European women (age 20-97 years) to quantitate age-related texture variation in BMD maps and generate a "reference" map of bone aging. We introduce a new index, called Densitometric Bone Age (DBA), as the age at which an individual site-specific BMD map (the proximal femur is studied here) best matches the median aging trajectory at that site in terms of the root mean squared error (RMSE). The ability of DBA to predict incident hip fracture and hip fracture pattern over 5 years following baseline BMD was compared against conventional region-based BMD analysis in a subset of 11,899 women (age 45-97 years), for which follow-up fracture records exist. There were 208 subsequent incident hip fractures in the study populations (138 femoral necks [FNs], 52 trochanteric [TR], 18 sites unspecified). DBA had modestly better performance compared to the conventional FN-BMD, TR-BMD, and total hip (TOT)-BMD in identifying hip fractures measured as the area under the curve (AUC) using receiver operating characteristics (ROC) curve analysis by 2% (95% confidence interval [CI], -0.5% to 3.5%), 3% (95% CI, 1.0% to 4.0%), and 1% (95% CI, 0.4% to 1.6%), respectively. Compared to FN-BMD T-score, DBA improved the ROC-AUC for predicting TR fractures by ~5% (95% CI, 1.1% to 9.8%) with similar performance in identifying FN fractures. Compared to TR-BMD T-score, DBA improved the ROC-AUC for the prediction of FN fractures by ~3% (95% CI, 1.1% to 4.9%), with similar performance in identifying TR fractures. Our findings suggest that DBA may provide a spatially sensitive measure of proximal femoral fragility that is not captured by FN-BMD or TR-BMD alone. © 2022 The Authors. Journal of Bone and Mineral Research published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research (ASBMR).

Utility of pre-treatment FDG PET/CT-derived machine learning models for outcome prediction in classical Hodgkin lymphoma

Objectives

Relapse occurs in ~20% of patients with classical Hodgkin lymphoma (cHL) despite treatment adaption based on 2-deoxy-2-[¹⁸F]fluoro-d-glucose positron emission tomography/computed tomography response. The objective was to evaluate pre-treatment FDG PET/CT–derived machine learning (ML) models for predicting outcome in patients with cHL.

Methods

All cHL patients undergoing pre-treatment PET/CT at our institution between 2008 and 2018 were retrospectively identified. A 1.5 × mean liver standardised uptake value (SUV) and a fixed 4.0 SUV threshold were used to segment PET/CT data. Feature extraction was performed using PyRadiomics with ComBat harmonisation. Training (80%) and test (20%) cohorts stratified around 2-year event-free survival (EFS), age, sex, ethnicity and disease stage were defined. Seven ML models were trained and hyperparameters tuned using stratified 5-fold cross-validation. Area under the curve (AUC) from receiver operator characteristic analysis was used to assess performance.

Results

A total of 289 patients (153 males), median age 36 (range 16–88 years), were included. There was no significant difference between training (n = 231) and test cohorts (n = 58) (p value > 0.05). A ridge regression model using a 1.5 × mean liver SUV segmentation had the highest performance, with mean training, validation and test AUCs of 0.82 ± 0.002, 0.79 ± 0.01 and 0.81 ± 0.12. However, there was no significant difference between a logistic model derived from metabolic tumour volume and clinical features or the highest performing radiomic model.

Conclusions

Outcome prediction using pre-treatment FDG PET/CT–derived ML models is feasible in cHL patients. Further work is needed to determine optimum predictive thresholds for clinical use.

Key points

• A fixed threshold segmentation method led to more robust radiomic features.

• A radiomic-based model for predicting 2-year event-free survival in classical Hodgkin lymphoma patients is feasible.

• A predictive model based on ridge regression was the best performing model on our dataset.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-022-09039-0.

Automatic 3D+t four-chamber CMR quantification of the UK biobank: integrating imaging and non-imaging data priors at scale

Accurate 3D modelling of cardiac chambers is essential for clinical assessment of cardiac volume and function, including structural, and motion analysis. Furthermore, to study the correlation between cardiac morphology and other patient information within a large population, it is necessary to automatically generate cardiac mesh models of each subject within the population. In this study, we introduce MCSI-Net (Multi-Cue Shape Inference Network), where we embed a statistical shape model inside a convolutional neural network and leverage both phenotypic and demographic information from the cohort to infer subject-specific reconstructions of all four cardiac chambers in 3D. In this way, we leverage the ability of the network to learn the appearance of cardiac chambers in cine cardiac magnetic resonance (CMR) images, and generate plausible 3D cardiac shapes, by constraining the prediction using a shape prior, in the form of the statistical modes of shape variation learned a priori from a subset of the population. This, in turn, enables the network to generalise to samples across the entire population. To the best of our knowledge, this is the first work that uses such an approach for patient-specific cardiac shape generation. MCSI-Net is capable of producing accurate 3D shapes using just a fraction (about 23% to 46%) of the available image data, which is of significant importance to the community as it supports the acceleration of CMR scan acquisitions. Cardiac MR images from the UK Biobank were used to train and validate the proposed method. We also present the results from analysing 40,000 subjects of the UK Biobank at 50 time-frames, totalling two million image volumes. Our model can generate more globally consistent heart shape than that of manual annotations in the presence of inter-slice motion and shows strong agreement with the reference ranges for cardiac structure and function across cardiac ventricles and atria.

Parkinson's Disease Classification and Clinical Score Regression via United Embedding and Sparse Learning From Longitudinal Data

Parkinson's disease (PD) is known as an irreversible neurodegenerative disease that mainly affects the patient's motor system. Early classification and regression of PD are essential to slow down this degenerative process from its onset. In this article, a novel adaptive unsupervised feature selection approach is proposed by exploiting manifold learning from longitudinal multimodal data. Classification and clinical score prediction are performed jointly to facilitate early PD diagnosis. Specifically, the proposed approach performs united embedding and sparse regression, which can determine the similarity matrices and discriminative features adaptively. Meanwhile, we constrain the similarity matrix among subjects and exploit the l_2,p norm to conduct sparse adaptive control for obtaining the intrinsic information of the multimodal data structure. An effective iterative optimization algorithm is proposed to solve this problem. We perform abundant experiments on the Parkinson's Progression Markers Initiative (PPMI) data set to verify the validity of the proposed approach. The results show that our approach boosts the performance on the classification and clinical score regression of longitudinal data and surpasses the state-of-the-art approaches.

The Pitfalls of Using Open Data to Develop Deep Learning Solutions for COVID-19 Detection in Chest X-Rays

Since the emergence of COVID-19, deep learning models have been developed to identify COVID-19 from chest X-rays. With little to no direct access to hospital data, the AI community relies heavily on public data comprising numerous data sources. Model performance results have been exceptional when training and testing on open-source data, surpassing the reported capabilities of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study impactful models are trained on a widely used open-source data and tested on an external test set and a hospital dataset, for the task of classifying chest X-rays into one of three classes: COVID-19, non-COVID pneumonia and no-pneumonia. Classification performance of the models investigated is evaluated through ROC curves, confusion matrices and standard classification metrics. Explainability modules are implemented to explore the image features most important to classification. Data analysis and model evalutions show that the popular open-source dataset COVIDx is not representative of the real clinical problem and that results from testing on this are inflated. Dependence on open-source data can leave models vulnerable to bias and confounding variables, requiring careful analysis to develop clinically useful/viable AI tools for COVID-19 detection in chest X-rays.

What is next for screening for undiagnosed atrial fibrillation? Artificial intelligence may hold the key

Atrial fibrillation (AF) is increasingly common, though often undiagnosed, leaving many people untreated and at elevated risk of ischaemic stroke. Current European guidelines do not recommend systematic screening for AF, even though a number of studies have shown that periods of serial or continuous rhythm monitoring in older people in the general population increase detection of AF and the prescription of oral anticoagulation. This article discusses the conflicting results of two contemporary landmark trials, STROKESTOP and the LOOP, which provided the first evidence on whether screening for AF confers a benefit for people in terms of clinical outcomes. The benefit and efficiency of systematic screening for AF in the general population could be optimized by targeting screening to only those at higher risk of developing AF. For this purpose, evidence is emerging that prediction models developed using artificial intelligence in routinely collected electronic health records can provide strong discriminative performance for AF and increase detection rates when combined with rhythm monitoring in a clinical study. We consider future directions for investigation in this field and how this could be best aligned to the current evidence base to target screening in people at elevated risk of stroke.

MICaps: Multi-instance capsule network for machine inspection of Munro's microabscess

Munro's Microabscess (MM) is the diagnostic hallmark of psoriasis. Neutrophil detection in the Stratum Corneum (SC) of the skin epidermis is an integral part of MM detection in skin biopsy. The microscopic inspection of skin biopsy is a tedious task and staining variations in skin histopathology often hinder human performance to differentiate neutrophils from skin keratinocytes. Motivated from this, we propose a computational framework that can assist human experts and reduce potential errors in diagnosis. The framework first segments the SC layer, and multiple patches are sampled from the segmented regions which are classified to detect neutrophils. Both UNet and CapsNet are used for segmentation and classification. Experiments show that of the two choices, CapsNet, owing to its robustness towards better hierarchical object representation and localisation ability, appears as a better candidate for both segmentation and classification tasks and hence, we termed our framework as MICaps. The training algorithm explores both minimisation of Dice Loss and Focal Loss and makes a comparative study between the two. The proposed framework is validated with our in-house dataset consisting of 290 skin biopsy images. Two different experiments are considered. Under the first protocol, only 3-fold cross-validation is done to directly compare the current results with the state-of-the-art ones. Next, the performance of the system on a held-out data set is reported. The experimental results show that MICaps improves the state-of-the-art diagnosis performance by 3.27% (maximum) and reduces the number of model parameters by 50%.

Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG

Musculoskeletal models have been widely used for detailed biomechanical analysis to characterise various functional impairments given their ability to estimate movement variables (i.e., muscle forces and joint moments) which cannot be readily measured in vivo. Physics-based computational neuromusculoskeletal models can interpret the dynamic interaction between neural drive to muscles, muscle dynamics, body and joint kinematics and kinetics. Still, such set of solutions suffers from slowness, especially for the complex models, hindering the utility in real-time applications. In recent years, data-driven methods have emerged as a promising alternative due to the benefits in speedy and simple implementation, but they cannot reflect the underlying neuromechanical processes. This paper proposes a physics-informed deep learning framework for musculoskeletal modelling, where physics-based domain knowledge is brought into the data-driven model as soft constraints to penalise/regularise the data-driven model. We use the synchronous muscle forces and joint kinematics prediction from surface electromyogram (sEMG) as the exemplar to illustrate the proposed framework. Convolutional neural network (CNN) is employed as the deep neural network to implement the proposed framework. Simultaneously, the physics law between muscle forces and joint kinematics is used the soft constraint. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The experimental results demonstrate the effectiveness and robustness of the proposed framework.

Medicine-Based Evidence in Congenital Heart Disease: How Artificial Intelligence Can Guide Treatment Decisions for Individual Patients

Built on the foundation of the randomized controlled trial (RCT), Evidence Based Medicine (EBM) is at its best when optimizing outcomes for homogeneous cohorts of patients like those participating in an RCT. Its weakness is a failure to resolve a clinical quandary: patients appear for care individually, each may differ in important ways from an RCT cohort, and the physician will wonder each time if following EBM will provide best guidance for this unique patient. In an effort to overcome this weakness, and promote higher quality care through a more personalized approach, a new framework has been proposed: Medicine-Based Evidence (MBE). In this approach, big data and deep learning techniques are embraced to interrogate treatment responses among patients in real-world clinical practice. Such statistical models are then integrated with mechanistic disease models to construct a “digital twin,” which serves as the real-time digital counterpart of a patient. MBE is thereby capable of dynamically modeling the effects of various treatment decisions in the context of an individual's specific characteristics. In this article, we discuss how MBE could benefit patients with congenital heart disease, a field where RCTs are difficult to conduct and often fail to provide definitive solutions because of a small number of subjects, their clinical complexity, and heterogeneity. We will also highlight the challenges that must be addressed before MBE can be embraced in clinical practice and its full potential can be realized.

Origami: Single-cell 3D shape dynamics oriented along the apico-basal axis of folding epithelia from fluorescence microscopy data

A common feature of morphogenesis is the formation of three-dimensional structures from the folding of two-dimensional epithelial sheets, aided by cell shape changes at the cellular-level. Changes in cell shape must be studied in the context of cell-polarised biomechanical processes within the epithelial sheet. In epithelia with highly curved surfaces, finding single-cell alignment along a biological axis can be difficult to automate in silico. We present 'Origami', a MATLAB-based image analysis pipeline to compute direction-variant cell shape features along the epithelial apico-basal axis. Our automated method accurately computed direction vectors denoting the apico-basal axis in regions with opposing curvature in synthetic epithelia and fluorescence images of zebrafish embryos. As proof of concept, we identified different cell shape signatures in the developing zebrafish inner ear, where the epithelium deforms in opposite orientations to form different structures. Origami is designed to be user-friendly and is generally applicable to fluorescence images of curved epithelia.

Predicting patient-level new-onset atrial fibrillation from population-based nationwide electronic health records: protocol of FIND-AF for developing a precision medicine prediction model using artificial intelligence

INTRODUCTION

Atrial fibrillation (AF) is a major cardiovascular health problem: it is common, chronic and incurs substantial healthcare expenditure because of stroke. Oral anticoagulation reduces the risk of thromboembolic stroke in those at higher risk; but for a number of patients, stroke is the first manifestation of undetected AF. There is a rationale for the early diagnosis of AF, before the first complication occurs, but population-based screening is not recommended. Previous prediction models have been limited by their data sources and methodologies. An accurate model that uses existing routinely collected data is needed to inform clinicians of patient-level risk of AF, inform national screening policy and highlight predictors that may be amenable to primary prevention.

METHODS AND ANALYSIS

We will investigate the application of a range of deep learning techniques, including an adapted convolutional neural network, recurrent neural network and Transformer, on routinely collected primary care data to create a personalised model predicting the risk of new-onset AF over a range of time periods. The Clinical Practice Research Datalink (CPRD)-GOLD dataset will be used for derivation, and the CPRD-AURUM dataset will be used for external geographical validation. Both comprise a sizeable representative population and are linked at patient-level to secondary care databases. The performance of the deep learning models will be compared against classic machine learning and traditional statistical predictive modelling methods. We will only use risk factors accessible in primary care and endow the model with the ability to update risk prediction as it is presented with new data, to make the model more useful in clinical practice.

ETHICS AND DISSEMINATION

Permissions for CPRD-GOLD and CPRD-AURUM datasets were obtained from CPRD (ref no: 19_076). The CPRD ethical approval committee approved the study. The results will be submitted as a research paper for publication to a peer-reviewed journal and presented at peer-reviewed conferences.

TRIAL REGISTRATION DETAILS

A systematic review to incorporate within the overall project was registered on PROSPERO (registration number CRD42021245093). The study was registered on ClinicalTrials.gov (NCT04657900).

A probabilistic deep motion model for unsupervised cardiac shape anomaly assessment

Automatic shape anomaly detection in large-scale imaging data can be useful for screening suboptimal segmentations and pathologies altering the cardiac morphology without intensive manual labour. We propose a deep probabilistic model for local anomaly detection in sequences of heart shapes, modelled as point sets, in a cardiac cycle. A deep recurrent encoder-decoder network captures the spatio-temporal dependencies to predict the next shape in the cycle and thus derive the outlier points that are attributed to excessive deviations from the network prediction. A predictive mixture distribution models the inlier and outlier classes via Gaussian and uniform distributions, respectively. A Gibbs sampling Expectation-Maximisation (EM) algorithm computes soft anomaly scores of the points via the posterior probabilities of each class in the E-step and estimates the parameters of the network and the predictive distribution in the M-step. We demonstrate the versatility of the method using two shape datasets derived from: (i) one million biventricular CMR images from 20,000 participants in the UK Biobank (UKB), and (ii) routine diagnostic imaging from Multi-Centre, Multi-Vendor, and Multi-Disease Cardiac Image (M&Ms). Experiments show that the detected shape anomalies in the UKB dataset are mostly associated with poor segmentation quality, and the predicted shape sequences show significant improvement over the input sequences. Furthermore, evaluations on U-Net based shapes from the M&Ms dataset reveals that the anomalies are attributable to the underlying pathologies that affect the ventricles. The proposed model can therefore be used as an effective mechanism to sift shape anomalies in large-scale cardiac imaging pipelines for further analysis.

Auto-weighted centralised multi-task learning via integrating functional and structural connectivity for subjective cognitive decline diagnosis

Early diagnosis and intervention of mild cognitive impairment (MCI) and its early stage (i.e., subjective cognitive decline (SCD)) is able to delay or reverse the disease progression. However, discrimination between SCD, MCI and healthy subjects accurately remains challenging. This paper proposes an auto-weighted centralised multi-task (AWCMT) learning framework for differential diagnosis of SCD and MCI. AWCMT is based on structural and functional connectivity information inferred from magnetic resonance imaging (MRI). To be specific, we devise a novel multi-task learning algorithm to combine neuroimaging functional and structural connective information. We construct a functional brain network through a sparse and low-rank machine learning method, and also a structural brain network via fibre bundle tracking. Those two networks are constructed separately and independently. Multi-task learning is then used to identify features integration of functional and structural connectivity. Hence, we can learn each task's significance automatically in a balanced way. By combining the functional and structural information, the most informative features of SCD and MCI are obtained for diagnosis. The extensive experiments on the public and self-collected datasets demonstrate that the proposed algorithm obtains better performance in classifying SCD, MCI and healthy people than traditional algorithms. The newly proposed method has good interpretability as it is able to discover the most disease-related brain regions and their connectivity. The results agree well with current clinical findings and provide new insights into early AD detection based on the multi-modal neuroimaging technique.

Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds

Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a variety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac motion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications involving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art techniques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spatial resolution is ∼1.8×1.8×10mm³). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysis.

Interdisciplinary research: shaping the healthcare of the future

The hospitals of the future will be shaped by scientific and technical advances made across a wide range of disciplines because complex problems in healthcare cannot be addressed successfully by a single discipline. This paper considers how interdisciplinary research is being promoted and the prospects for developing stronger and deeper collaborations between medicine, health and other disciplines, drawing on case studies from mathematics, physics and engineering. The anticipated impact of greater interdisciplinarity on clinical training and the provision of care is also reviewed. While the role and training of clinicians in the provision of care will continue to evolve, they will remain leading members of a much broader and more diverse interdisciplinary team, alert to the value of deep and sustained interdisciplinary research.

Medical imaging and computational image analysis in COVID-19 diagnosis: A review

Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus. The disease presents with symptoms such as shortness of breath, fever, dry cough, and chronic fatigue, amongst others. The disease may be asymptomatic in some patients in the early stages, which can lead to increased transmission of the disease to others. This study attempts to review papers on the role of imaging and medical image computing in COVID-19 diagnosis. For this purpose, PubMed, Scopus and Google Scholar were searched to find related studies until the middle of 2021. The contribution of this study is four-fold: 1) to use as a tutorial of the field for both clinicians and technologists, 2) to comprehensively review the characteristics of COVID-19 as presented in medical images, 3) to examine automated artificial intelligence-based approaches for COVID-19 diagnosis, 4) to express the research limitations in this field and the methods used to overcome them. Using machine learning-based methods can diagnose the disease with high accuracy from medical images and reduce time, cost and error of diagnostic procedure. It is recommended to collect bulk imaging data from patients in the shortest possible time to improve the performance of COVID-19 automated diagnostic methods.

Diagnosis of early Alzheimer's disease based on dynamic high order networks

Machine learning methods have been widely used for early diagnosis of Alzheimer's disease (AD) via functional connectivity networks (FCNs) analysis from neuroimaging data. The conventional low-order FCNs are obtained by time-series correlation of the whole brain based on resting-state functional magnetic resonance imaging (R-fMRI). However, FCNs overlook inter-region interactions, which limits application to brain disease diagnosis. To overcome this drawback, we develop a novel framework to exploit the high-level dynamic interactions among brain regions for early AD diagnosis. Specifically, a sliding window approach is employed to generate some R-fMRI sub-series. The correlations among these sub-series are then used to construct a series of dynamic FCNs. High-order FCNs based on the topographical similarity between each pair of the dynamic FCNs are then constructed. Afterward, a local weight clustering method is used to extract effective features of the network, and the least absolute shrinkage and selection operation method is chosen for feature selection. A support vector machine is employed for classification, and the dynamic high-order network approach is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Our experimental results demonstrate that the proposed approach not only achieves promising results for AD classification, but also successfully recognizes disease-related biomarkers.

Real-time coronary artery stenosis detection based on modern neural networks

Invasive coronary angiography remains the gold standard for diagnosing coronary artery disease, which may be complicated by both, patient-specific anatomy and image quality. Deep learning techniques aimed at detecting coronary artery stenoses may facilitate the diagnosis. However, previous studies have failed to achieve superior accuracy and performance for real-time labeling. Our study is aimed at confirming the feasibility of real-time coronary artery stenosis detection using deep learning methods. To reach this goal we trained and tested eight promising detectors based on different neural network architectures (MobileNet, ResNet-50, ResNet-101, Inception ResNet, NASNet) using clinical angiography data of 100 patients. Three neural networks have demonstrated superior results. The network based on Faster-RCNN Inception ResNet V2 is the most accurate and it achieved the mean Average Precision of 0.95, F1-score 0.96 and the slowest prediction rate of 3 fps on the validation subset. The relatively lightweight SSD MobileNet V2 network proved itself as the fastest one with a low mAP of 0.83, F1-score of 0.80 and a mean prediction rate of 38 fps. The model based on RFCN ResNet-101 V2 has demonstrated an optimal accuracy-to-speed ratio. Its mAP makes up 0.94, F1-score 0.96 while the prediction speed is 10 fps. The resultant performance-accuracy balance of the modern neural networks has confirmed the feasibility of real-time coronary artery stenosis detection supporting the decision-making process of the Heart Team interpreting coronary angiography findings.

Super-Resolution of Cardiac MR Cine Imaging using Conditional GANs and Unsupervised Transfer Learning

High-resolution (HR), isotropic cardiac Magnetic Resonance (MR) cine imaging is challenging since it requires long acquisition and patient breath-hold times. Instead, 2D balanced steady-state free precession (SSFP) sequence is widely used in clinical routine. However, it produces highly-anisotropic image stacks, with large through-plane spacing that can hinder subsequent image analysis. To resolve this, we propose a novel, robust adversarial learning super-resolution (SR) algorithm based on conditional generative adversarial nets (GANs), that incorporates a state-of-the-art optical flow component to generate an auxiliary image to guide image synthesis. The approach is designed for real-world clinical scenarios and requires neither multiple low-resolution (LR) scans with multiple views, nor the corresponding HR scans, and is trained in an end-to-end unsupervised transfer learning fashion. The designed framework effectively incorporates visual properties and relevant structures of input images and can synthesise 3D isotropic, anatomically plausible cardiac MR images, consistent with the acquired slices. Experimental results show that the proposed SR method outperforms several state-of-the-art methods both qualitatively and quantitatively. We show that subsequent image analyses including ventricle segmentation, cardiac quantification, and non-rigid registration can benefit from the super-resolved, isotropic cardiac MR images, to produce more accurate quantitative results, without increasing the acquisition time. The average Dice similarity coefficient (DSC) for the left ventricular (LV) cavity and myocardium are 0.95 and 0.81, respectively, between real and synthesised slice segmentation. For non-rigid registration and motion tracking through the cardiac cycle, the proposed method improves the average DSC from 0.75 to 0.86, compared to the original resolution images.

Baseline PET/CT imaging parameters for prediction of treatment outcome in Hodgkin and diffuse large B cell lymphoma: a systematic review

Purpose

To systematically review the literature evaluating clinical utility of imaging metrics derived from baseline fluorine-18 fluorodeoxyglucose positron emission tomography/computed tomography (PET/CT) for prediction of progression-free (PFS) and overall survival (OS) in patients with classical Hodgkin lymphoma (HL) and diffuse large B cell lymphoma (DLBCL).

Methods

A search of MEDLINE/PubMed, Web of Science, Cochrane, Scopus and clinicaltrials.gov databases was undertaken for articles evaluating PET/CT imaging metrics as outcome predictors in HL and DLBCL. PRISMA guidelines were followed. Risk of bias was assessed using the Quality in Prognosis Studies (QUIPS) tool.

Results

Forty-one articles were included (31 DLBCL, 10 HL). Significant predictive ability was reported in 5/20 DLBCL studies assessing SUVmax (PFS: HR 0.13–7.35, OS: HR 0.83–11.23), 17/19 assessing metabolic tumour volume (MTV) (PFS: HR 2.09–11.20, OS: HR 2.40–10.32) and 10/13 assessing total lesion glycolysis (TLG) (PFS: HR 1.078–11.21, OS: HR 2.40–4.82). Significant predictive ability was reported in 1/4 HL studies assessing SUVmax (HR not reported), 6/8 assessing MTV (PFS: HR 1.2–10.71, OS: HR 1.00–13.20) and 2/3 assessing TLG (HR not reported). There are 7/41 studies assessing the use of radiomics (4 DLBCL, 2 HL); 5/41 studies had internal validation and 2/41 included external validation. All studies had overall moderate or high risk of bias.

Conclusion

Most studies are retrospective, underpowered, heterogenous in their methodology and lack external validation of described models. Further work in protocol harmonisation, automated segmentation techniques and optimum performance cut-off is required to develop robust methodologies amenable for clinical utility.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05233-2.

OpenMandible: An open-source framework for highly realistic numerical modelling of lower mandible physiology

OBJECTIVE

Computer modeling of lower mandible physiology remains challenging because prescribing realistic material characteristics and boundary conditions from medical scans requires advanced equipment and skill sets. The objective of this study is to provide a framework that could reduce simplifications made and inconsistency (in terms of geometry, materials, and boundary conditions) among further studies on the topic.

METHODS

The OpenMandible framework offers: 1) the first publicly available multiscale model of the mandible developed by combining cone beam computerized tomography (CBCT) and μCT imaging modalities, and 2) a C++ software tool for the generation of simulation-ready models (tet4 and hex8 elements). In addition to the application of conventional (Neumann and Dirichlet) boundary conditions, OpenMandible introduces a novel geodesic wave propagation - based approach for incorporating orthotropic micromechanical characteristics of cortical bone, and a unique algorithm for modeling muscles as uniformly directed vectors. The base intact model includes the mandible (spongy and compact bone), 14 teeth (comprising dentin, enamel, periodontal ligament, and pulp), simplified temporomandibular joints, and masticatory muscles (masseter, temporalis, medial, and lateral pterygoid).

RESULTS

The complete source code, executables, showcases, and sample data are freely available on the public repository: https://github.com/ArsoVukicevic/OpenMandible. It has been demonstrated that by slightly editing the baseline model, one can study different "virtual" treatments or diseases, including tooth restoration, placement of implants, mandible bone degradation, and others.

SIGNIFICANCE

OpenMandible eases the community to undertake a broad range of studies on the topic, while increasing their consistency and reproducibility. At the same time, the needs for dedicated equipment and skills for developing realistic simulation models are significantly reduced.

Graph convolution network with similarity awareness and adaptive calibration for disease-induced deterioration prediction

Graph convolution networks (GCN) have been successfully applied in disease prediction tasks as they capture interactions (i.e., edges and edge weights on the graph) between individual elements. The interactions in existing works are constructed by fusing similarity between imaging information and distance between non-imaging information, whereas disregarding the disease status of those individuals in the training set. Besides, the similarity is being evaluated by computing the correlation distance between feature vectors, which limits prediction performance, especially for predicting significant memory concern (SMC) and mild cognitive impairment (MCI). In this paper, we propose three mechanisms to improve GCN, namely similarity-aware adaptive calibrated GCN (SAC-GCN), for predicting SMC and MCI. First, we design a similarity-aware graph using different receptive fields to consider disease status. The labelled subjects on the graph are only connected with those labelled subjects with the same status. Second, we propose an adaptive mechanism to evaluate similarity. Specifically, we construct initial GCN with evaluating similarity by using traditional correlation distance, then pre-train the initial GCN by using training samples and use it to score all subjects. Then, the difference between these scores replaces correlation distance to update similarity. Last, we devise a calibration mechanism to fuse functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) information into edges. The proposed method is tested on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Experimental results demonstrate that our proposed method is useful to predict disease-induced deterioration and superior to other related algorithms, with a mean classification accuracy of 86.83% in our prediction tasks.