Gradual back-projection residual attention network for magnetic resonance image super-resolution

ResNet and its application to medical image processing: Research progress and challenges

BACKGROUND AND OBJECTIVE

Deep learning, a novel approach and subset of machine learning, has drawn a growing amount of attention from computer vision researchers in recent years. This method has drawn a lot of interest because of its extraordinary ability to interpret medical pictures, especially when combined with residual neural networks, which have helped to progress the field.

METHODS

In this paper, the following research is carried out on the residual network. First, the research status of ResNet in the medical field is introduced. The fundamental idea behind the residual neural network is then explained, along with the residual unit, its many structures, and the network architecture. Second, four aspects of the widespread use of residual neural networks in medical image processing are discussed: lung tumor, diagnosis of skin diseases, diagnosis of breast diseases, and diagnosis of diseases of the brain. Finally, the main issues and ResNet's future development in the area of processing medical images are discussed.

RESULTS

In the area of medical graph processing, residual neural networks have made strides and have had success in the clinical auxiliary diagnosis of serious illnesses such as lung tumors, breast cancer, skin conditions, and cardiovascular and cerebrovascular diseases.

CONCLUSION

We thoroughly sorted out the most recent developments in residual neural network research and their use in medical image processing, which serves as a crucial point of reference for this field of study. It offers a helpful reference for further promoting the application and research of the ResNet model in the field of medical image processing by summarising the application status and issues of the ResNet model in the field of medical image processing and putting forwards some future development directions.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Multi-omics integration method based on attention deep learning network for biomedical data classification

BACKGROUND AND OBJECTIVE

Integrating multi-omics data for the comprehensive analysis of the biological processes in human diseases has become one of the most challenging tasks of bioinformatics. Deep learning (DL) algorithms have recently become one of the most promising multi-omics data integration analysis methods. However, existing DL-based studies almost integrate the multi-omics data by concatenation in the input data space or the learned feature space, ignoring the correlations between patients and omics.

METHODS

We propose a novel multi-omics integration method, called Multi-omics Attention Deep Learning Network (MOADLN), which is used for biomedical data classification. Firstly, for each type of omics data, we use three fully-connected layers and the self-attention mechanism to reduce dimensionality, and construct the correlations between patients, respectively. Then, we apply the feature vector learned from self-attention to generate the initial category labels. Secondly, for the initial label predicted of each omics data, we use an effective Multi-Omics Correlation Discovery Network (MOCDN) to learn the cross-omic correlations in the label space. Finally, we use the softmax classifier for label prediction.

RESULTS

We demonstrate that our method outperforms several state-of-the-art methods on two datasets with mRNA expression data, DNA methylation data, and miRNA expression data. In addition, we identified essential biomarkers of relevant diseases by MOADLN, and the generality of MOADLN is also demonstrated in the KIRP and KIRC datasets.

CONCLUSIONS

MOADLN jointly explores correlations between patients in intra-omics and correlations of cross-omics in label space, which is an effective DL-based classification of biomedical data.

Feedback attention network for cardiac magnetic resonance imaging super-resolution

BACKGROUND AND OBJECTIVE

Atrial fibrillation (AF) is a common clinical arrhythmia with a high disability and mortality rate. Improving the resolution of atrial structure and its changes in patients with AF is very important for understanding and treating AF.

METHODS

Aiming at the problems of previous deep learning-based image super-resolution (SR) reconstruction methods simply deepening the network, loss of upsampling information, and difficulty in the reconstruction of high-frequency information, we propose the Feedback Attention Network (FBAN) for cardiac magnetic resonance imaging (CMRI) super-resolution. The network comprises a preprocessing module, a multi-scale residual group module, an upsampling module, and a reconstruction module. The preprocessing module uses a convolutional layer to extract shallow features and dilate the number of channels of the feature map. The multi-scale residual group module adds a multi-channel network, a mixed attention mechanism, and a long and short skip connection to expand the receptive field of the feature map, improve the multiplexing of multi-scale features and strengthen the reconstruction of high-frequency information. The upsampling module adopts the sub-pixel method to upsample the feature map to the target image size. The reconstruction module consists of a convolutional layer, which is used to restore the number of channels of the feature map to the original number to obtain the reconstructed high-resolution (HR) image.

RESULTS

Furthermore, the test results on the public dataset of CMRI show that the HR images reconstructed by the FBAN method not only have a good improvement in reconstructed edge and texture information but also have a good improvement in the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) objective evaluation indicators.

CONCLUSION

Compared with the local magnified image, the edge information of the FBAN method reconstructed image has been enhanced, more high-frequency information of the CMRI is restored, the texture details are less lost, and the reconstructed image is less blurry. Overall, the reconstructed image has a lighter feeling of smearing, and the visual experience is more apparent and sharper.

Channel attention generative adversarial network for super-resolution of glioma magnetic resonance image

BACKGROUND AND OBJECTIVE

Glioma is the most common primary craniocerebral tumor caused by the cancelation of glial cells in the brain and spinal cord, with a high incidence and cure rate. Magnetic resonance imaging (MRI) is a common technique for detecting and analyzing brain tumors. Due to improper hardware and operation, the obtained brain MRI images are low-resolution, making it difficult to detect and grade gliomas accurately. However, super-resolution reconstruction technology can improve the clarity of MRI images and help experts accurately detect and grade glioma.

METHODS

We propose a glioma magnetic resonance image super-resolution reconstruction method based on channel attention generative adversarial network (CGAN). First, we replace the base block of SRGAN with a residual dense block based on the channel attention mechanism. Second, we adopt a relative average discriminator to replace the discriminator in standard GAN. Finally, we add the mean squared error loss to the training, consisting of the mean squared error loss, the L1 norm loss, and the generator's adversarial loss to form the generator loss function.

RESULTS

On the Set5, Set14, Urban100, and glioma datasets, compared with the state-of-the-art algorithms, our proposed CGAN method has improved peak signal-to-noise ratio and structural similarity, and the reconstructed glioma images are more precise than other algorithms.

CONCLUSION

The experimental results show that our CGAN method has apparent improvements in objective evaluation indicators and subjective visual effects, indicating its effectiveness and superiority.

Multi-modal brain MRI images enhancement based on framelet and local weights super-resolution

Magnetic resonance (MR) image enhancement technology can reconstruct high-resolution image from a low-resolution image, which is of great significance for clinical application and scientific research. T1 weighting and T2 weighting are the two common magnetic resonance imaging modes, each of which has its own advantages, but the imaging time of T2 is much longer than that of T1. Related studies have shown that they have very similar anatomical structures in brain images, which can be utilized to enhance the resolution of low-resolution T2 images by using the edge information of high-resolution T1 images that can be rapidly imaged, so as to shorten the imaging time needed for T2 images. In order to overcome the inflexibility of traditional methods using fixed weights for interpolation and the inaccuracy of using gradient threshold to determine edge regions, we propose a new model based on previous studies on multi-contrast MR image enhancement. Our model uses framelet decomposition to finely separate the edge structure of the T2 brain image, and uses the local regression weights calculated from T1 image to construct a global interpolation matrix, so that our model can not only guide the edge reconstruction more accurately where the weights are shared, but also carry out collaborative global optimization for the remaining pixels and their interpolated weights. Experimental results on a set of simulated MR data and two sets of real MR images show that the enhanced images obtained by the proposed method are superior to the compared methods in terms of visual sharpness or qualitative indicators.

Dual attention mechanism network for lung cancer images super-resolution

BACKGROUND AND OBJECTIVE

Currently, the morbidity and mortality of lung cancer rank first among malignant tumors worldwide. Improving the resolution of thin-slice CT of the lung is particularly important for the early diagnosis of lung cancer screening.

METHODS

Aiming at the problems of network training difficulty and low utilization of feature information caused by the deepening of network layers in super-resolution (SR) reconstruction technology, we propose the dual attention mechanism network for single image super-resolution (SISR). Firstly, the feature of a low-resolution image is extracted directly to retain the feature information. Secondly, several independent dual attention mechanism modules are constructed to extract high-frequency details. The introduction of residual connections can effectively solve the gradient disappearance caused by network deepening, and long and short skip connections can effectively enhance the data features. Furthermore, a hybrid loss function speeds up the network's convergence and improves image SR restoration ability. Finally, through the upsampling operation, the reconstructed high-resolution image is obtained.

RESULTS

The results on the Set5 dataset for 4 × enlargement show that compared with traditional SR methods such as Bicubic, VDSR, and DRRN, the average PSNR/SSIM is increased by 3.33 dB / 0.079, 0.41 dB / 0.007 and 0.22 dB / 0.006 respectively. The experimental data fully show that DAMN can better restore the image contour features, obtain higher PSNR, SSIM, and better visual effect.

CONCLUSION

Through the DAMN reconstruction method, the image quality can be improved without increasing radiation exposure and scanning time. Radiologists can enhance their confidence in diagnosing early lung cancer, provide a basis for clinical experts to choose treatment plans, formulate follow-up strategies, and benefit patients in the early stage.

Improved generative adversarial network for retinal image super-resolution

BACKGROUND AND OBJECTIVE

The retina is the only organ in the body that can use visible light for non-invasive observation. By analyzing retinal images, we can achieve early screening, diagnosis and prevention of many ophthalmological and systemic diseases, helping patients avoid the risk of blindness. Due to the powerful feature extraction capabilities, many deep learning super-resolution reconstruction networks have been applied to retinal image analysis and achieved excellent results.

METHODS

Given the lack of high-frequency information and poor visual perception in the current reconstruction results of super-resolution reconstruction networks under large-scale factors, we present an improved generative adversarial network (IGAN) algorithm for retinal image super-resolution reconstruction. Firstly, we construct a novel residual attention block, improving the reconstruction results lacking high-frequency information and texture details under large-scale factors. Secondly, we remove the Batch Normalization layer that affects the quality of image generation in the residual network. Finally, we use the more robust Charbonnier loss function instead of the mean square error loss function and the TV regular term to smooth the training results.

RESULTS

Experimental results show that our proposed method significantly improves objective evaluation indicators such as peak signal-to-noise ratio and structural similarity. The obtained image has rich texture details and a better visual experience than the state-of-the-art image super-resolution methods.

CONCLUSION

Our proposed method can better learn the mapping relationship between low-resolution and high-resolution retinal images. This method can be effectively and stably applied to the analysis of retinal images, providing an effective basis for early clinical treatment.

End-to-end residual attention mechanism for cataractous retinal image dehazing

BACKGROUND AND OBJECTIVE

Cataract is one of the most common causes of vision loss. Light scattering due to clouding of the lens in cataract patients makes it extremely difficult to image the retina of cataract patients with fundus cameras, resulting in a serious decrease in the quality of the retinal images taken. Furthermore, the age of cataract patients is generally too old, in addition to cataracts, the patients often have other retinal diseases, which brings great challenges to experts in the clinical diagnosis of cataract patients using retinal imaging.

METHODS

In this paper, we present the End-to-End Residual Attention Mechanism (ERAN) for Cataractous Retinal Image Dehazing, which it includes four modules: encoding module, multi-scale feature extraction module, feature fusion module, and decoding module. The encoding module encodes the input cataract haze image into an image, facilitating subsequent feature extraction and reducing memory usage. The multi-scale feature extraction module includes a hole convolution module, a residual block, and an adaptive skip connection, which can expand the receptive field and extract features of different scales through weighted screening for fusion. The feature fusion module uses adaptive skip connections to enhance the network's ability to extract haze density images to make haze removal more thorough. Furthermore, the decoding module performs non-linear mapping on the fused features to obtain the haze density image, and then restores the haze-free image.

RESULTS

The experimental results show that the proposed method has achieved better objective and subjective evaluation results, and has a better dehazing effect.

CONCLUSION

We proposed ERAN method not only provides visually better images, but also helps experts better diagnose other retinal diseases in cataract patients, leading to better care and treatment.

Super-resolution reconstruction of biometric features recognition based on manifold learning and deep residual network

BACKGROUND AND OBJECTIVE

In daily life, face information has the characteristics of uniqueness and universality. However, in a real-world scene, the image information of the face acquired by the acquisition device often contains noises such as blurring and sharpening. As such, super-resolution reconstruction of face features recognition based on manifold learning is proposed in this paper.

METHODS

We reconstruct low-resolution facial expression images, introduce a simplified residual block network and manifold learning, and propose joint supervision through a new hybrid loss function, which not only retains the color and characteristics of the image, but also retains the high-frequency information. The ResNet50 network uses the weight feature of information entropy to optimize the information of the pooling layer, and the esNet50 network uses the improved PSO algorithm to optimize the initial weight of the error back-propagation phase.

RESULTS

In the case of inputting extremely low resolution (6 × 6) facial expression images, the accuracy rate is increased by 9.091%. The accuracy of the high-resolution facial expressions after reconstruction with a size of 12×12 is 96.970%. The accuracy rate for happy expressions is 100%, the accuracy rate for anger, disgust, sadness, and surprise recognition is 97%, the accuracy rate for contempt is 94%, and the accuracy rate for fear is 88%.

CONCLUSIONS

The experimental results verify the feasibility and superiority of the system, and effectively improve the accuracy of low-resolution facial expressions.

Nonconvex Nonlocal Tucker Decomposition for 3D Medical Image Super-Resolution

Limited by hardware conditions, imaging devices, transmission efficiency, and other factors, high-resolution (HR) images cannot be obtained directly in clinical settings. It is expected to obtain HR images from low-resolution (LR) images for more detailed information. In this article, we propose a novel super-resolution model for single 3D medical images. In our model, nonlocal low-rank tensor Tucker decomposition is applied to exploit the nonlocal self-similarity prior knowledge of data. Different from the existing methods that use a convex optimization for tensor Tucker decomposition, we use a tensor folded-concave penalty to approximate a nonlocal low-rank tensor. Weighted 3D total variation (TV) is used to maintain the local smoothness across different dimensions. Extensive experiments show that our method outperforms some state-of-the-art (SOTA) methods on different kinds of medical images, including MRI data of the brain and prostate and CT data of the abdominal and dental.

Propofol Anesthesia Depth Monitoring Based on Self-Attention and Residual Structure Convolutional Neural Network

Methods

We compare nine index values, select CNN+EEG, which has good correlation with BIS index, as an anesthesia state observation index to identify the parameters of the model, and establish a model based on self-attention and dual resistructure convolutional neural network. The data of 93 groups of patients were selected and randomly grouped into three parts: training set, validation set, and test set, and compared the best and worst results predicted by BIS.

Result

The best result is that the model's accuracy of predicting BLS on the test set has an overall upward trend, eventually reaching more than 90%. The overall error shows a gradual decrease and eventually approaches zero. The worst result is that the model's accuracy of predicting BIS on the test set has an overall upward trend. The accuracy rate is relatively stable without major fluctuations, but the final accuracy rate is above 70%.

Conclusion

The prediction of BIS indicators by the deep learning method CNN algorithm shows good results in statistics.

Medical expert and machine learning analysis of lumbar disc herniation based on magnetic resonance imaging

OBJECTIVE

Observation and statistical analysis was used to evaluate the ability of lumbar disc magnetic resonance imaging (MRI) to obtain the smallest size of Al₂O₃ spots (calcified foci) and lumbar disc fiber signals.

METHODS

First, we perform image acquisition of the MRI, perform the statistical analysis using five different sizes of Al₂O₃ spots and lumbar disc fibers on the imaging plate (IP), use a molybdenum target MRI machine 26 kV, adjust the milliampere amount, select the appropriate image processing parameters, and obtain the experimental image of the density value (D=0.70±0.05), the 5-point judgment method is used to obtain the total score of 10 lines of signals composed of 5 signals and noise, and a group is computed using the statistical analysis that is built from human observation and machine prediction (based on machine learning), which are then compared. In particular, we implemented a convolutional neural network algorithm to evaluate the medical condition against human observers, so as to study the structure of the lumbar intervertebral disc. We compute the true positive probability P(S/s) and false positive probability P(S/n) values, draw ROC curve, and compute the judgment probability value of each signal P_det. We then use SPSS 10.0 statistical single factor analysis of variance software to process the data, and obtain the smallest calcified focus and lumbar disc mass focus.

RESULTS

Using probability statistical methods to obtain the data of the ROC curve and the average value of the judgment probability P_det, among 5 different sizes Al₂O₃ spots (calcifications), 0.20mm P_det= 0.6250minimum, 0.55mm P_det = 0.9000 the largest, but the difference between 0.20mm and 0.25mm P_det is not statistically significant, and the difference is statistically significant; among the five types of lumbar disc fibers (tumor foci) of different sizes, 0.45mm P_det= 0.5313minimum, 1.00mm P_det =0.8813 is the largest, while the difference between 0.45mm and 0.60mm is not statistically significant, and the difference between 0.45mm and other is statistically significant. We note that the human observation and machine learning prediction is not significantly different (P<0.05).

CONCLUSIONS

The computation of the ROC curve and that of the probability of judgment using the statistical analysis based on a deep learning platform is simple and fast, and approximates that of human observation. It is suitable for the evaluation of image quality control carried out by daily clinical work.