Smoke veil prior regularized surgical field desmoking without paired in-vivo data

Though deep learning-based surgical smoke removal methods have shown significant improvements in effectiveness and efficiency, the lack of paired smoke and smoke-free images in real surgical scenarios limits the performance of these methods. Therefore, methods that can achieve good generalization performance without paired in-vivo data are in high demand. In this work, we propose a smoke veil prior regularized two-stage smoke removal framework based on the physical model of smoke image formation. More precisely, in the first stage, we leverage a reconstruction loss, a consistency loss and a smoke veil prior-based regularization term to perform fully supervised training on a synthetic paired image dataset. Then a self-supervised training stage is deployed on the real smoke images, where only the consistency loss and the smoke veil prior-based loss are minimized. Experiments show that the proposed method outperforms the state-of-the-art ones on synthetic dataset. The average PSNR, SSIM and RMSE values are 21.99±2.34, 0.9001±0.0252 and 0.2151±0.0643, respectively. The qualitative visual inspection on real dataset further demonstrates the effectiveness of the proposed method.

A neural network based framework for effective laparoscopic video quality assessment

Video quality assessment is a challenging problem having a critical significance in the context of medical imaging. For instance, in laparoscopic surgery, the acquired video data suffers from different kinds of distortion that not only hinder surgery performance but also affect the execution of subsequent tasks in surgical navigation and robotic surgeries. For this reason, we propose in this paper neural network-based approaches for distortion classification as well as quality prediction. More precisely, a Residual Network (ResNet) based approach is firstly developed for simultaneous ranking and classification task. Then, this architecture is extended to make it appropriate for the quality prediction task by using an additional Fully Connected Neural Network (FCNN). To train the overall architecture (ResNet and FCNN models), transfer learning and end-to-end learning approaches are investigated. Experimental results, carried out on a new laparoscopic video quality database, have shown the efficiency of the proposed methods compared to recent conventional and deep learning based approaches.

Robust Brain Age Estimation based on sMRI via Nonlinear Age-Adaptive Ensemble Learning

Precise prediction on brain age is urgently needed by many biomedical areas including mental rehabilitation prognosis as well as various medicine or treatment trials. People began to realize that contrasting physical (real) age and predicted brain age can help to highlight brain issues and evaluate if patients' brains are healthy or not. Such age prediction is often challenging for single model-based prediction, while the conditions of brains vary drastically over age. In this work, we present an age-adaptive ensemble model that is based on the combination of four different machine learning algorithms, including a support vector machine (SVR), a convolutional neural network (CNN) model, and the popular GoogLeNet and ResNet deep networks. The ensemble model proposed here is nonlinearly adaptive, where age is taken as a key factor in the nonlinear combination of various single-algorithm-based independent models. In our age-adaptive ensemble method, the weights of each model are learned automatically as nonlinear functions over age instead of fixed values, while brain age estimation is based on such an age-adaptive integration of various single models. The quality of the model is quantified by the mean absolute errors (MAE) and spearman correlation between the predicted age and the actual age, with the least MAE and the highest Spearman correlation representing the highest accuracy in age prediction. By testing on the Predictive Analysis Challenge 2019 (PAC 2019) dataset, our novel ensemble model has achieved a MAE down to 3.19, which is a significantly increased accuracy in this brain age competition. If deployed in the real world, our novel ensemble model having an improved accuracy could potentially help doctors to identify the risk of brain diseases more accurately and quickly, thus helping pharmaceutical companies develop drugs or treatments precisely, and potential offer a new powerful tool for researchers in the field of brain science.

An Expanded MLP Neural Network for Fast Angiodysplasia Lesions Segmentation in Capsule Endoscopy Images

In this paper, an expanded multilayer perceptron (EMLP) neural network is proposed to automatically segment angiodysplasia regions in wireless capsule endoscopy (WCE) images The main idea is to minimize the distance between the input image and the corresponding binary ground truth, i.e., the mask image. After the training phase, when a test image is given to the network, the lesion pixels will be close to “one” and the other pixels will be close to “zero” and finally, the lesion area can be segmented using thresholding. Since angiodysplasia lesions appear in images with different spectrums of red color, the classical MLP neural network cannot be trained with a wide range of red color, hence leads to undesirable network accuracy. To solve this problem, we proposed an EMLP neural network for image segmentation. In the EMLP neural network, neurons are divided into several groups, each of which is for learning a spectrum of the lesion. The EMLP is able to learn a wider range of red colors. The proposed method is able to segment WCE images containing angiodysplasia faster than the existing methods. Our investigation shows that our method also outperforms existing methods in terms of segmentation scores.

A Computer-Aided Method for Digestive System Abnormality Detection in WCE Images

Wireless capsule endoscopy (WCE) is a powerful tool for the diagnosis of gastrointestinal diseases. The output of this tool is in video with a length of about eight hours, containing about 8000 frames. It is a difficult task for a physician to review all of the video frames. In this paper, a new abnormality detection system for WCE images is proposed. The proposed system has four main steps: (1) preprocessing, (2) region of interest (ROI) extraction, (3) feature extraction, and (4) classification. In ROI extraction, at first, distinct areas are highlighted and nondistinct areas are faded by using the joint normal distribution; then, distinct areas are extracted as an ROI segment by considering a threshold. The main idea is to extract abnormal areas in each frame. Therefore, it can be used to extract various lesions in WCE images. In the feature extraction step, three different types of features (color, texture, and shape) are employed. Finally, the features are classified using the support vector machine. The proposed system was tested on the Kvasir-Capsule dataset. The proposed system can detect multiple lesions from WCE frames with high accuracy.

A combined multiple action recognition and summarization for surveillance video sequences

Human action recognition and video summarization represent challenging tasks for several computer vision applications including video surveillance, criminal investigations, and sports applications. For long videos, it is difficult to search within a video for a specific action and/or person. Usually, human action recognition approaches presented in the literature deal with videos that contain only a single person, and they are able to recognize his action. This paper proposes an effective approach to multiple human action detection, recognition, and summarization. The multiple action detection extracts human bodies’ silhouette, then generates a specific sequence for each one of them using motion detection and tracking method. Each of the extracted sequences is then divided into shots that represent homogeneous actions in the sequence using the similarity between each pair frames. Using the histogram of the oriented gradient (HOG) of the Temporal Difference Map (TDMap) of the frames of each shot, we recognize the action by performing a comparison between the generated HOG and the existed HOGs in the training phase which represents all the HOGs of many actions using a set of videos for training. Also, using the TDMap images we recognize the action using a proposed CNN model. Action summarization is performed for each detected person. The efficiency of the proposed approach is shown through the obtained results for mainly multi-action detection and recognition.

Efficient Enhancement of Stereo Endoscopic Images Based on Joint Wavelet Decomposition and Binocular Combination

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality.

Variational based smoke removal in laparoscopic images

Background

In laparoscopic surgery, image quality can be severely degraded by surgical smoke, which not only introduces errors for the image processing algorithms (used in image guided surgery), but also reduces the visibility of the observed organs and tissues. To overcome these drawbacks, this work aims to remove smoke in laparoscopic images using an image preprocessing method based on a variational approach.

Methods

In this paper, we present the physical smoke model where the degraded image is separated into two parts: direct attenuation and smoke veil and propose an efficient variational-based desmoking method for laparoscopic images. To estimate the smoke veil, the proposed method relies on the observation that smoke veil has low contrast and low inter-channel differences. A cost function is defined based on this prior knowledge and is solved using an augmented Lagrangian method. The obtained smoke veil is then subtracted from the original degraded image, resulting in the direct attenuation part. Finally, the smoke free image is computed using a linear intensity transformation of the direct attenuation part.

Results

The performance of the proposed method is evaluated quantitatively and qualitatively using three datasets: two public real smoked laparoscopic datasets and one generated synthetic dataset. No-reference and reduced-reference image quality assessment metrics are used with the two real datasets, and show that the proposed method outperforms the state-of-the-art ones. Besides, standard full-reference ones are employed with the synthetic dataset, and indicate also the good performance of the proposed method. Furthermore, the qualitative visual inspection of the results shows that our method removes smoke effectively from the laparoscopic images.

Conclusion

All the obtained results show that the proposed approach reduces the smoke effectively while preserving the important perceptual information of the image. This allows to provide a better visualization of the operation field for surgeons and improve the image guided laparoscopic surgery procedure.

A novel method for planning liver resections using deformable Bézier surfaces and distance maps

BACKGROUND AND OBJECTIVE

For more than a decade, computer-assisted surgical systems have been helping surgeons to plan liver resections. The most widespread strategies to plan liver resections are: drawing traces in individual 2D slices, and using a 3D deformable plane. In this work, we propose a novel method which requires low level of user interaction while keeping high flexibility to specify resections.

METHODS

Our method is based on the use of Bézier surfaces, which can be deformed using a grid of control points, and distance maps as a base to compute and visualize resection margins (indicators of safety) in real-time. Projection of resections in 2D slices, as well as computation of resection volume statistics are also detailed.

RESULTS

The method was evaluated and compared with state-of-the-art methods by a group of surgeons (n=5, 5-31 years of experience). Our results show that theproposed method presents planning times as low as state-of-the-art methods (174 s median time) with high reproducibility of results in terms of resected volume. In addition, our method not only leads to smooth virtual resections easier to perform surgically compared to other state-of-the-art methods, but also shows superior preservation of resection margins.

CONCLUSIONS

Our method provides clinicians with a robust and easy-to-use method for planning liver resections with high reproducibility, smoothness of resection and preservation of resection margin. Our results indicate the ability of the method to represent any type of resection and being integrated in real clinical work-flows.

A new look to multichannel blind image deconvolution

The aim of this paper is to propose a new look to MBID, examine some known approaches, and provide a new MC method for restoring blurred and noisy images. First, the direct image restoration problem is briefly revisited. Then a new method based on inverse filtering for perfect image restoration in the noiseless case is proposed. The noisy case is addressed by introducing a regularization term into the objective function in order to avoid noise amplification. Second, the filter identification problem is considered in the MC context. A new robust solution to estimate the degradation matrix filter is then derived and used in conjunction with a total variation approach to restore the original image. Simulation results and performance evaluations using recent image quality metrics are provided to assess the effectiveness of the proposed methods.

A new image smoothing method based on a simple model of spatial processing in the early stages of human vision

The difficulty of preserving edges is central to the problem of smoothing images. The main problem is that of distinguishing between meaningful contours and noise, so that the image can be smoothed without loss of details. Substantial efforts have been devoted to solving this difficult problem, and a plethora of filtering methods have been proposed in the literature. Non-linear filters have proved to be more efficient than their linear counterparts. Here, a new nonlinear filter for noise smoothing is introduced. This filter is based on the psychophysical phenomenon of human visual contrast sensitivity. Results on real images are presented to demonstrate the validity of our approach compared to other known filtering methods.

Analysis of dynamic nuclear cardiac images by covariance function

A new method using the covariance function as a measure of functional similarity is presented for dynamic analysis of a sequence of scintigraphic cardiac images taken throughout the cardiac cycle. The similarity between the temporal response of pixels in a reference region of the scintigraphic image series and the temporal response of the remaining pixels in the image sequence is calculated. The resulting covariance image is a functional image representing regions with different temporal dynamics. A box-plot representation of this image permits better interpretation for clinical decision making. This analysis allows visualization of the ventricular emptying pattern, which may be useful in studying motion or conduction abnormalities. Compared to Fourier analysis, our method does not make assumption that the data are periodic and that the transition between the first and the last frame of the study is smooth. The proposed method has been performed in one normal patient and twenty patients with abnormal ventricular contraction patterns, and there is no computational difficulty in its implementation. A comparison with the Fourier analysis is performed.

A noise-filtering method using a local information measure

A nonlinear-noise filtering method for image processing, based on the entropy concept is developed and compared to the well-known median filter and to the center weighted median filter (CWM). The performance of the proposed method is evaluated through subjective and objective criteria. It is shown that this method performs better than the classical median for different types of noise and can perform better than the CWM filter in some cases.

Entropic contrast enhancement

A technique for the contrast enhancement of a picture is proposed. The method is derived from the entropy concept of information theory. The originality of the algorithm rests on the use of a local contrast to define the digital entropy. The basic idea of the treatment is to enhance the contrast by transforming the global entropy. The same technique can also be used for an adaptive smoothing processing.