Cine Cardiac MRI Motion Artifact Reduction Using a Recurrent Neural Network

Complex-valued neural networks to speed-up MR thermometry during hyperthermia using Fourier PD and PDUNet

Hyperthermia (HT) in combination with radio- and/or chemotherapy has become an accepted cancer treatment for distinct solid tumour entities. In HT, tumour tissue is exogenously heated to temperatures between 39 and 43 °C for 60 min. Temperature monitoring can be performed non-invasively using dynamic magnetic resonance imaging (MRI). However, the slow nature of MRI leads to motion artefacts in the images due to the movements of patients during image acquisition. By discarding parts of the data, the speed of the acquisition can be increased - known as undersampling. However, due to the invalidation of the Nyquist criterion, the acquired images might be blurry and can also produce aliasing artefacts. The aim of this work was, therefore, to reconstruct highly undersampled MR thermometry acquisitions with better resolution and with fewer artefacts compared to conventional methods. The use of deep learning in the medical field has emerged in recent times, and various studies have shown that deep learning has the potential to solve inverse problems such as MR image reconstruction. However, most of the published work only focuses on the magnitude images, while the phase images are ignored, which are fundamental requirements for MR thermometry. This work, for the first time, presents deep learning-based solutions for reconstructing undersampled MR thermometry data. Two different deep learning models have been employed here, the Fourier Primal-Dual network and the Fourier Primal-Dual UNet, to reconstruct highly undersampled complex images of MR thermometry. MR images of 44 patients with different sarcoma types who received HT treatment in combination with radiotherapy and/or chemotherapy were used in this study. The method reduced the temperature difference between the undersampled MRIs and the fully sampled MRIs from 1.3 to 0.6 °C in full volume and 0.49 °C to 0.06 °C in the tumour region for a theoretical acceleration factor of 10.

Review of the Current State of Artificial Intelligence in Pediatric Cardiovascular Magnetic Resonance Imaging

Cardiovascular magnetic resonance (CMR) imaging is essential for the management of congenital heart disease (CHD), due to the ability to perform anatomic and physiologic assessments of patients. However, CMR scans can be time-consuming to perform and analyze, creating roadblocks to broader use of CMR in CHD. Recent publications have shown artificial intelligence (AI) has the potential to increase efficiency, improve image quality, and reduce errors. This review examines the use of AI techniques to improve CMR in CHD, by focusing on deep learning techniques applied to image acquisition and reconstruction, image processing and reporting, clinical use cases, and future directions.

Investigating the Use of Generative Adversarial Networks-Based Deep Learning for Reducing Motion Artifacts in Cardiac Magnetic Resonance

Objective

To evaluate the effectiveness of deep learning technology based on generative adversarial networks (GANs) in reducing motion artifacts in cardiac magnetic resonance (CMR) cine sequences.

Methods

The training and testing datasets consisted of 2000 and 200 pairs of clear and blurry images, respectively, acquired through simulated motion artifacts in CMR cine sequences. These datasets were used to establish and train a deep learning GAN model. To assess the efficacy of the deep learning network in mitigating motion artifacts, 100 images with simulated motion artifacts and 37 images with real-world motion artifacts encountered in clinical practice were selected. Image quality pre- and post-optimization was assessed using metrics including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Leningrad Focus Measure, and a 5-point Likert scale.

Results

After GAN optimization, notable improvements were observed in the PSNR, SSIM, and focus measure metrics for the 100 images with simulated artifacts. These metrics increased from initial values of 23.85±2.85, 0.71±0.08, and 4.56±0.67, respectively, to 27.91±1.74, 0.83±0.05, and 7.74±0.39 post-optimization. Additionally, the subjective assessment scores significantly improved from 2.44±1.08 to 4.44±0.66 (P<0.001). For the 37 images with real-world artifacts, the Tenengrad Focus Measure showed a significant enhancement, rising from 6.06±0.91 to 10.13±0.48 after artifact removal. Subjective ratings also increased from 3.03±0.73 to 3.73±0.87 (P<0.001).

Conclusion

GAN-based deep learning technology effectively reduces motion artifacts present in CMR cine images, demonstrating significant potential for clinical application in optimizing CMR motion artifact management.

Joint suppression of cardiac bSSFP cine banding and flow artifacts using twofold phase-cycling and a dual-encoder neural network

BACKGROUND

Cardiac balanced steady state free precession (bSSFP) cine imaging suffers from banding and flow artifacts induced by off-resonance. The work aimed to develop a twofold phase cycling sequence with a neural network-based reconstruction (2P-SSFP+Network) for a joint suppression of banding and flow artifacts in cardiac cine imaging.

METHODS

A dual-encoder neural network was trained on 1620 pairs of phase-cycled left ventricular (LV) cine images collected from 18 healthy subjects. Twenty healthy subjects and 25 patients were prospectively scanned using the proposed 2P-SSFP sequence. bSSFP cine of a single RF phase increment (1P-SSFP), bSSFP cine of a single radiofrequency (RF) phase increment with a network-based artifact reduction (1P-SSFP+Network), the averaging of the two phase-cycled images (2P-SSFP+Average), and the proposed method were mutually compared, in terms of artifact suppression performance in the LV, generalizability over altered scan parameters and scanners, suppression of large-area banding artifacts in the left atrium (LA), and accuracy of downstream segmentation tasks.

RESULTS

In the healthy subjects, 2P-SSFP+Network showed robust suppressions of artifacts across a range of phase combinations. Compared with 1P-SSFP and 2P-SSFP+Average, 2P-SSFP+Network improved banding artifacts (3.85 ± 0.67 and 4.50 ± 0.45 vs 5.00 ± 0.00, P < 0.01 and P = 0.02, respectively), flow artifacts (3.35 ± 0.78 and 2.10 ± 0.77 vs 4.90 ± 0.20, both P < 0.01), and overall image quality (3.25 ± 0.51 and 2.30 ± 0.60 vs 4.75 ± 0.25, both P < 0.01). 1P-SSFP+Network and 2P-SSFP+Network achieved a similar artifact suppression performance, yet the latter had fewer hallucinations (two-chamber, 4.25 ± 0.51 vs 4.85 ± 0.45, P = 0.04; four-chamber, 3.45 ± 1.21 vs 4.65 ± 0.50, P = 0.03; and left atrium (LA), 3.35 ± 1.00 vs 4.65 ± 0.45, P < 0.01). Furthermore, in the pulmonary veins and LA, 1P-SSFP+Network could not eliminate banding artifacts since they occupied a large area, whereas 2P-SSFP+Network reliably suppressed the artifacts. In the downstream automated myocardial segmentation task, 2P-SSFP+Network achieved more accurate segmentations than 1P-SSFP with different phase increments.

CONCLUSIONS

2P-SSFP+Network jointly suppresses banding and flow artifacts while manifesting a good generalizability against variations of anatomy and scan parameters. It provides a feasible solution for robust suppression of the two types of artifacts in bSSFP cine imaging.

Evaluation of super resolution technology for digestive endoscopic images

Object

This study aims to evaluate the value of super resolution (SR) technology in augmenting the quality of digestive endoscopic images.

Methods

In the retrospective study, we employed two advanced SR models, i.e., SwimIR and ESRGAN. Two discrete datasets were utilized, with training conducted using the dataset of the First Affiliated Hospital of Soochow University (12,212 high-resolution images) and evaluation conducted using the HyperKvasir dataset (2,566 low-resolution images). Furthermore, an assessment of the impact of enhanced low-resolution images was conducted using a 5-point Likert scale from the perspectives of endoscopists. Finally, two endoscopic image classification tasks were employed to evaluate the effect of SR technology on computer vision (CV).

Results

SwinIR demonstrated superior performance, which achieved a PSNR of 32.60, an SSIM of 0.90, and a VIF of 0.47 in test set. 90 % of endoscopists supported that SR preprocessing moderately ameliorated the readability of endoscopic images. For CV, enhanced images bolstered the performance of convolutional neural networks, whether in the classification task of Barrett's esophagus (improved F1-score: 0.04) or Mayo Endoscopy Score (improved F1-score: 0.04).

Conclusions

SR technology demonstrates the capacity to produce high-resolution endoscopic images. The approach enhanced clinical readability and CV models' performance of low-resolution endoscopic images.

Comprehensive review of artifacts in cardiac MRI and their mitigation

Cardiac magnetic resonance imaging (CMR) is an important clinical tool that obtains high-quality images for assessment of cardiac morphology, function, and tissue characteristics. However, the technique may be prone to artifacts that may limit the diagnostic interpretation of images. This article reviews common artifacts which may appear in CMR exams by describing their appearance, the challenges they mitigate true pathology, and offering possible solutions to reduce their impact. Additionally, this article acts as an update to previous CMR artifacts reports by including discussion about new CMR innovations.

The role of artificial intelligence in cardiovascular magnetic resonance imaging

Cardiovascular magnetic resonance (CMR) imaging is the gold standard test for myocardial tissue characterization and chamber volumetric and functional evaluation. However, manual CMR analysis can be time-consuming and is subject to intra- and inter-observer variability. Artificial intelligence (AI) is a field that permits automated task performance through the identification of high-level and complex data relationships. In this review, we review the rapidly growing role of AI in CMR, including image acquisition, sequence prescription, artifact detection, reconstruction, segmentation, and data reporting and analysis including quantification of volumes, function, myocardial infarction (MI) and scar detection, and prediction of outcomes. We conclude with a discussion of the emerging challenges to widespread adoption and solutions that will allow for successful, broader uptake of this powerful technology.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Stop moving: MR motion correction as an opportunity for artificial intelligence

Subject motion is a long-standing problem of magnetic resonance imaging (MRI), which can seriously deteriorate the image quality. Various prospective and retrospective methods have been proposed for MRI motion correction, among which deep learning approaches have achieved state-of-the-art motion correction performance. This survey paper aims to provide a comprehensive review of deep learning-based MRI motion correction methods. Neural networks used for motion artifacts reduction and motion estimation in the image domain or frequency domain are detailed. Furthermore, besides motion-corrected MRI reconstruction, how estimated motion is applied in other downstream tasks is briefly introduced, aiming to strengthen the interaction between different research areas. Finally, we identify current limitations and point out future directions of deep learning-based MRI motion correction.

STADNet: Spatial-Temporal Attention-Guided Dual-Path Network for cardiac cine MRI super-resolution

Cardiac cine magnetic resonance imaging (MRI) is a commonly used clinical tool for evaluating cardiac function and morphology. However, its diagnostic accuracy may be compromised by the low spatial resolution. Current methods for cine MRI super-resolution reconstruction still have limitations. They typically rely on 3D convolutional neural networks or recurrent neural networks, which may not effectively capture long-range or non-local features due to their limited receptive fields. Optical flow estimators are also commonly used to align neighboring frames, which may cause information loss and inaccurate motion estimation. Additionally, pre-warping strategies may involve interpolation, leading to potential loss of texture details and complicated anatomical structures. To overcome these challenges, we propose a novel Spatial-Temporal Attention-Guided Dual-Path Network (STADNet) for cardiac cine MRI super-resolution. We utilize transformers to model long-range dependencies in cardiac cine MR images and design a cross-frame attention module in the location-aware spatial path, which enhances the spatial details of the current frame by using complementary information from neighboring frames. We also introduce a recurrent flow-enhanced attention module in the motion-aware temporal path that exploits the correlation between cine MRI frames and extracts the motion information of the heart. Experimental results demonstrate that STADNet outperforms SOTA approaches and has significant potential for clinical practice.

A cardiac MRI motion artifact reduction method based on edge enhancement network

Cardiac magnetic resonance imaging (MRI) usually requires a long acquisition time. The movement of the patients during MRI acquisition will produce image artifacts. Previous studies have shown that clear MR image texture edges are of great significance for pathological diagnosis. In this paper, a motion artifact reduction method for cardiac MRI based on edge enhancement network is proposed. Firstly, the four-plane normal vector adaptive fractional differential mask is applied to extract the edge features of blurred images. The four-plane normal vector method can reduce the noise information in the edge feature maps. The adaptive fractional order is selected according to the normal mean gradient and the local Gaussian curvature entropy of the images. Secondly, the extracted edge feature maps and blurred images are input into the de-artifact network. In this network, the edge fusion feature extraction network and the edge fusion transformer network are specially designed. The former combines the edge feature maps with the fuzzy feature maps to extract the edge feature information. The latter combines the edge attention network and the fuzzy attention network, which can focus on the blurred image edges. Finally, extensive experiments show that the proposed method can obtain higher peak signal-to-noise ratio and structural similarity index measure compared to state-of-art methods. The de-artifact images have clear texture edges.

Motion Artifact Reduction Using U-Net Model with Three-Dimensional Simulation-Based Datasets for Brain Magnetic Resonance Images

This study aimed to remove motion artifacts from brain magnetic resonance (MR) images using a U-Net model. In addition, a simulation method was proposed to increase the size of the dataset required to train the U-Net model while avoiding the overfitting problem. The volume data were rotated and translated with random intensity and frequency, in three dimensions, and were iterated as the number of slices in the volume data. Then, for every slice, a portion of the motion-free k-space data was replaced with motion k-space data, respectively. In addition, based on the transposed k-space data, we acquired MR images with motion artifacts and residual maps and constructed datasets. For a quantitative evaluation, the root mean square error (RMSE), peak signal-to-noise ratio (PSNR), coefficient of correlation (CC), and universal image quality index (UQI) were measured. The U-Net models for motion artifact reduction with the residual map-based dataset showed the best performance across all evaluation factors. In particular, the RMSE, PSNR, CC, and UQI improved by approximately 5.35×, 1.51×, 1.12×, and 1.01×, respectively, and the U-Net model with the residual map-based dataset was compared with the direct images. In conclusion, our simulation-based dataset demonstrates that U-Net models can be effectively trained for motion artifact reduction.

Manifold-based denoising for Ferumoxytol-enhanced 3D cardiac cine MRI

The two-dimensional (2D) cine cardiovascular magnetic resonance (CMR) technique is the reference standard for assessing cardiac function. However, one challenge with 2D cine is that the acquisition time for the whole cine stack is long and requires multiple breath holds, which may not be feasible for pediatric or ill patients. Though single breath-hold multi-slice cine may address the issue, it can only acquire low-resolution images, and hence, affect the accuracy of cardiac function assessment. To address these challenges, a Ferumoxytol-enhanced, free breathing, isotropic high-resolution 3D cine technique was developed. The method produces high-contrast cine images with short acquisition times by using compressed sensing together with a manifold-based method for image denoising. This study included fifteen patients (9.1 $ \pm $ 5.6 yrs.) who were referred for clinical cardiovascular magnetic resonance imaging (MRI) with Ferumoxytol contrast and were prescribed the 3D cine sequence. The data was acquired on a 1.5T scanner. Statistical analysis shows that the manifold-based denoised 3D cine can accurately measure ventricular function with no significant differences when compared to the conventional 2D breath-hold (BH) cine. The multiplanar reconstructed images of the proposed 3D cine method are visually comparable to the golden standard 2D BH cine method in terms of clarity, contrast, and anatomical precision. The proposed method eliminated the need for breath holds, reduced scan times, enabled multiplanar reconstruction within an isotropic data set, and has the potential to be used as an effective tool to access cardiovascular conditions.

Deep Learning for Retrospective Motion Correction in MRI: A Comprehensive Review

Motion represents one of the major challenges in magnetic resonance imaging (MRI). Since the MR signal is acquired in frequency space, any motion of the imaged object leads to complex artefacts in the reconstructed image in addition to other MR imaging artefacts. Deep learning has been frequently proposed for motion correction at several stages of the reconstruction process. The wide range of MR acquisition sequences, anatomies and pathologies of interest, and motion patterns (rigid vs. deformable and random vs. regular) makes a comprehensive solution unlikely. To facilitate the transfer of ideas between different applications, this review provides a detailed overview of proposed methods for learning-based motion correction in MRI together with their common challenges and potentials. This review identifies differences and synergies in underlying data usage, architectures, training and evaluation strategies. We critically discuss general trends and outline future directions, with the aim to enhance interaction between different application areas and research fields.

Data-driven system matrix manipulation enabling fast functional imaging and intra-image nonrigid motion correction in tomography

Tomographic imaging modalities are described by large system matrices. Sparse sampling and tissue motion degrade system matrix and image quality. Various existing techniques improve the image quality without correcting the system matrices. Here, we compress the system matrices to improve computational efficiency (e.g., 42 times) using singular value decomposition and fast Fourier transform. Enabled by the efficiency, we propose (1) fast sparsely sampling functional imaging by incorporating a densely sampled prior image into the system matrix, which maintains the critical linearity while mitigating artifacts and (2) intra-image nonrigid motion correction by incorporating the motion as subdomain translations into the system matrix and reconstructing the translations together with the image iteratively. We demonstrate the methods in 3D photoacoustic computed tomography with significantly improved image qualities and clarify their applicability to X-ray CT and MRI or other types of imperfections due to the similarities in system matrices.

TT U-Net: Temporal Transformer U-Net for Motion Artifact Reduction Using PAD (Pseudo All-Phase Clinical-Dataset) in Cardiac CT

Involuntary motion of the heart remains a challenge for cardiac computed tomography (CT) imaging. Although the electrocardiogram (ECG) gating strategy is widely adopted to perform CT scans at the quasi-quiescent cardiac phase, motion-induced artifacts are still unavoidable for patients with high heart rates or irregular rhythms. Dynamic cardiac CT, which provides functional information of the heart, suffers even more severe motion artifacts. In this paper, we develop a deep learning based framework for motion artifact reduction in dynamic cardiac CT. First, we build a PAD (Pseudo All-phase clinical-Dataset) based on a whole-heart motion model and single-phase cardiac CT images. This dataset provides dynamic CT images with realistic-looking motion artifacts that help to develop data-driven approaches. Second, we formulate the problem of motion artifact reduction as a video deblurring task according to its dynamic nature. A novel TT U-Net (Temporal Transformer U-Net) is proposed to excavate the spatiotemporal features for better motion artifact reduction. The self-attention mechanism along the temporal dimension effectively encodes motion information and thus aids image recovery. Experiments show that the TT U-Net trained on the proposed PAD performs well on clinical CT scans, which substantiates the effectiveness and fine generalization ability of our method. The source code, trained models, and dynamic demo will be available at https://github.com/ivy9092111111/TT-U-Net.

Mitigation of motion-induced artifacts in cone beam computed tomography using deep convolutional neural networks

BACKGROUND

Cone beam computed tomography (CBCT) is often employed on radiation therapy treatment devices (linear accelerators) used in image-guided radiation therapy (IGRT). For each treatment session, it is necessary to obtain the image of the day in order to accurately position the patient and to enable adaptive treatment capabilities including auto-segmentation and dose calculation. Reconstructed CBCT images often suffer from artifacts, in particular those induced by patient motion. Deep-learning based approaches promise ways to mitigate such artifacts.

PURPOSE

We propose a novel deep-learning based approach with the goal to reduce motion induced artifacts in CBCT images and improve image quality. It is based on supervised learning and includes neural network architectures employed as pre- and/or post-processing steps during CBCT reconstruction.

METHODS

Our approach is based on deep convolutional neural networks which complement the standard CBCT reconstruction, which is performed either with the analytical Feldkamp-Davis-Kress (FDK) method, or with an iterative algebraic reconstruction technique (SART-TV). The neural networks, which are based on refined U-net architectures, are trained end-to-end in a supervised learning setup. Labeled training data are obtained by means of a motion simulation, which uses the two extreme phases of 4D CT scans, their deformation vector fields, as well as time-dependent amplitude signals as input. The trained networks are validated against ground truth using quantitative metrics, as well as by using real patient CBCT scans for a qualitative evaluation by clinical experts.

RESULTS

The presented novel approach is able to generalize to unseen data and yields significant reductions in motion induced artifacts as well as improvements in image quality compared with existing state-of-the-art CBCT reconstruction algorithms (up to +6.3 dB and +0.19 improvements in peak signal-to-noise ratio, PSNR, and structural similarity index measure, SSIM, respectively), as evidenced by validation with an unseen test dataset, and confirmed by a clinical evaluation on real patient scans (up to 74% preference for motion artifact reduction over standard reconstruction).

CONCLUSIONS

For the first time, it is demonstrated, also by means of clinical evaluation, that inserting deep neural networks as pre- and post-processing plugins in the existing 3D CBCT reconstruction and trained end-to-end yield significant improvements in image quality and reduction of motion artifacts.

The role of AI in prostate MRI quality and interpretation: Opportunities and challenges

Prostate MRI plays an important role in imaging the prostate gland and surrounding tissues, particularly in the diagnosis and management of prostate cancer. With the widespread adoption of multiparametric magnetic resonance imaging in recent years, the concerns surrounding the variability of imaging quality have garnered increased attention. Several factors contribute to the inconsistency of image quality, such as acquisition parameters, scanner differences and interobserver variabilities. While efforts have been made to standardize image acquisition and interpretation via the development of systems, such as PI-RADS and PI-QUAL, the scoring systems still depend on the subjective experience and acumen of humans. Artificial intelligence (AI) has been increasingly used in many applications, including medical imaging, due to its ability to automate tasks and lower human error rates. These advantages have the potential to standardize the tasks of image interpretation and quality control of prostate MRI. Despite its potential, thorough validation is required before the implementation of AI in clinical practice. In this article, we explore the opportunities and challenges of AI, with a focus on the interpretation and quality of prostate MRI.

Generalizable Deep Learning Method for Suppressing Unseen and Multiple MRI Artifacts Using Meta-learning

Magnetic Resonance (MR) images suffer from various types of artifacts due to motion, spatial resolution, and under-sampling. Conventional deep learning methods deal with removing a specific type of artifact, leading to separately trained models for each artifact type that lack the shared knowledge generalizable across artifacts. Moreover, training a model for each type and amount of artifact is a tedious process that consumes more training time and storage of models. On the other hand, the shared knowledge learned by jointly training the model on multiple artifacts might be inadequate to generalize under deviations in the types and amounts of artifacts. Model-agnostic meta-learning (MAML), a nested bi-level optimization framework is a promising technique to learn common knowledge across artifacts in the outer level of optimization, and artifact-specific restoration in the inner level. We propose curriculum-MAML (CMAML), a learning process that integrates MAML with curriculum learning to impart the knowledge of variable artifact complexity to adaptively learn restoration of multiple artifacts during training. Comparative studies against Stochastic Gradient Descent and MAML, using two cardiac datasets reveal that CMAML exhibits (i) better generalization with improved PSNR for 83% of unseen types and amounts of artifacts and improved SSIM in all cases, and (ii) better artifact suppression in 4 out of 5 cases of composite artifacts (scans with multiple artifacts).Clinical relevance- Our results show that CMAML has the potential to minimize the number of artifact-specific models; which is essential to deploy deep learning models for clinical use. Furthermore, we have also taken another practical scenario of an image affected by multiple artifacts and show that our method performs better in 80% of cases.

A bidirectional registration neural network for cardiac motion tracking using cine MRI images

Using cine magnetic resonance imaging (cine MRI) images to track cardiac motion helps users to analyze the myocardial strain, and is of great importance in clinical applications. At present, most of the automatic deep learning-based motion tracking methods compare two images without considering temporal information between MRI frames, which easily leads to the lack of consistency of the generated motion fields. Even though a small number of works take into account the temporal factor, they are usually computationally intensive or have limitations on image length. To solve this problem, we propose a bidirectional convolution neural network for motion tracking of cardiac cine MRI images. This network leverages convolutional blocks to extract spatial features from three-dimensional (3D) image registration pairs, and models the temporal relations through a bidirectional recurrent neural network to obtain the Lagrange motion field between the reference image and other images. Compared with previous pairwise registration methods, the proposed method can automatically learn spatiotemporal information from multiple images with fewer parameters. We evaluated our model on three public cardiac cine MRI datasets. The experimental results demonstrated that the proposed method can significantly improve the motion tracking accuracy. The average Dice coefficient between estimated segmentation and manual segmentation has reached almost 0.85 on the widely used Automatic Cardiac Diagnostic Challenge (ACDC) dataset.

The Synergy between Deep Learning and Organs-on-Chips for High-Throughput Drug Screening: A Review

Organs-on-chips (OoCs) are miniature microfluidic systems that have arguably become a class of advanced in vitro models. Deep learning, as an emerging topic in machine learning, has the ability to extract a hidden statistical relationship from the input data. Recently, these two areas have become integrated to achieve synergy for accelerating drug screening. This review provides a brief description of the basic concepts of deep learning used in OoCs and exemplifies the successful use cases for different types of OoCs. These microfluidic chips are of potential to be assembled as highly potent human-on-chips with complex physiological or pathological functions. Finally, we discuss the future supply with perspectives and potential challenges in terms of combining OoCs and deep learning for image processing and automation designs.

Interpretable laryngeal tumor grading of histopathological images via depth domain adaptive network with integration gradient CAM and priori experience-guided attention

Tumor grading and interpretability of laryngeal cancer is a key yet challenging task in the clinical diagnosis, mainly because of the commonly used low-magnification pathological images lack fine cellular structure information and accurate localization, the diagnosis results of pathologists are different from those of attentional convolutional network -based methods, and the gradient-weighted class activation mapping method cannot be optimized to create the best visualization map. To address this problem, we propose an end-to-end depth domain adaptive network (DDANet) with integration gradient CAM and priori experience-guided attention to improve the tumor grading performance and interpretability by introducing the pathologist's a priori experience in high-magnification into the depth model. Specifically, a novel priori experience-guided attention (PE-GA) method is developed to solve the traditional unsupervised attention optimization problem. Besides, a novel integration gradient CAM is proposed to mitigate overfitting, information redundancies and low sparsity of the Grad-CAM graphs generated by the PE-GA method. Furthermore, we establish a set of quantitative evaluation metric systems for model visual interpretation. Extensive experimental results show that compared with the state-of-the-art methods, the average grading accuracy is increased to 88.43% (↑4.04%), the effective interpretable rate is increased to 52.73% (↑11.45%). Additionally, it effectively reduces the difference between CV-based method and pathology in diagnosis results. Importantly, the visualized interpretive maps are closer to the region of interest of concern by pathologists, and our model outperforms pathologists with different levels of experience.

3D-MedTranCSGAN: 3D Medical Image Transformation using CSGAN

Computer vision techniques are a rapidly growing area of transforming medical images for various specific medical applications. In an end-to-end application, this paper proposes a 3D Medical Image Transformation Using a CSGAN model named a 3D-MedTranCSGAN. The 3D-MedTranCSGAN model is an integration of non-adversarial loss components and the Cyclic Synthesized Generative Adversarial Networks. The proposed model utilizes PatchGAN's discriminator network, to penalize the difference between the synthesized image and the original image. The model also computes the non-adversary loss functions such as content, perception, and style transfer losses. 3DCascadeNet is a new generator architecture introduced in the paper, which is used to enhance the perceptiveness of the transformed medical image by encoding-decoding pairs. We use the 3D-MedTranCSGAN model to do various tasks without modifying specific applications: PET to CT image transformation; reconstruction of CT to PET; modification of movement artefacts in MR images; and removing noise in PET images. We found that 3D-MedTranCSGAN outperformed other transformation methods in our experiments. For the first task, the proposed model yields SSIM is 0.914, PSNR is 26.12, MSE is 255.5, VIF is 0.4862, UQI is 0.9067 and LPIPs is 0.2284. For the second task, the model yields 0.9197, 25.7, 257.56, 0.4962, 0.9027, 0.2262. For the third task, the model yields 0.8862, 24.94, 0.4071, 0.6410, 0.2196. For the final task, the model yields 0.9521, 33.67, 33.57, 0.6091, 0.9255, 0.0244. Based on the result analysis, the proposed model outperforms the other techniques.

Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge

Dynamic imaging is a beneficial tool for interventions to assess physiological changes. Nonetheless during dynamic MRI, while achieving a high temporal resolution, the spatial resolution is compromised. To overcome this spatio-temporal trade-off, this research presents a super-resolution (SR) MRI reconstruction with prior knowledge based fine-tuning to maximise spatial information while reducing the required scan-time for dynamic MRIs. A U-Net based network with perceptual loss is trained on a benchmark dataset and fine-tuned using one subject-specific static high resolution MRI as prior knowledge to obtain high resolution dynamic images during the inference stage. 3D dynamic data for three subjects were acquired with different parameters to test the generalisation capabilities of the network. The method was tested for different levels of in-plane undersampling for dynamic MRI. The reconstructed dynamic SR results after fine-tuning showed higher similarity with the high resolution ground-truth, while quantitatively achieving statistically significant improvement. The average SSIM of the lowest resolution experimented during this research (6.25% of the k-space) before and after fine-tuning were 0.939 ± 0.008 and 0.957 ± 0.006 respectively. This could theoretically result in an acceleration factor of 16, which can potentially be acquired in less than half a second. The proposed approach shows that the super-resolution MRI reconstruction with prior-information can alleviate the spatio-temporal trade-off in dynamic MRI, even for high acceleration factors.