A fine-grained recognition technique for identifying Chinese food images

As a crucial area of research in the field of computer vision, food recognition technology has become a core technology in many food-related fields, such as unmanned restaurants and food nutrition analysis, which are closely related to our healthy lives. Obtaining accurate classification results is the most important task in food recognition. Food classification is a fine-grained recognition process, which involves extracting features from a group of objects with similar appearances and accurately classifying them into different categories. In a such usage environment, the network is required to not only overview the overall image, but also capture the subtle details within it. In addition, since Chinese food images have unique texture features, the model needs to extract texture information from the image. However, existing CNN methods have not focused on and processed this information. To classify food as accurately as possible, this paper introduces the Laplace pyramid into the convolution layer and proposes a bilinear network that can perceive image texture features and multi-scale features (LMB-Net). The proposed model was evaluated on a public dataset, and the results demonstrate that LMB-Net achieves state-of-the-art classification performance.

Fast fetal head compounding from multi-view 3D ultrasound

The diagnostic value of ultrasound images may be limited by the presence of artefacts, notably acoustic shadows, lack of contrast and localised signal dropout. Some of these artefacts are dependent on probe orientation and scan technique, with each image giving a distinct, partial view of the imaged anatomy. In this work, we propose a novel method to fuse the partially imaged fetal head anatomy, acquired from numerous views, into a single coherent 3D volume of the full anatomy. Firstly, a stream of freehand 3D US images is acquired using a single probe, capturing as many different views of the head as possible. The imaged anatomy at each time-point is then independently aligned to a canonical pose using a recurrent spatial transformer network, making our approach robust to fast fetal and probe motion. Secondly, images are fused by averaging only the most consistent and salient features from all images, producing a more detailed compounding, while minimising artefacts. We evaluated our method quantitatively and qualitatively, using image quality metrics and expert ratings, yielding state of the art performance in terms of image quality and robustness to misalignments. Being online, fast and fully automated, our method shows promise for clinical use and deployment as a real-time tool in the fetal screening clinic, where it may enable unparallelled insight into the shape and structure of the face, skull and brain.

Good and bad boundaries in ultrasound compounding: preserving anatomic boundaries while suppressing artifacts

Purpose

Ultrasound compounding is to combine sonographic information captured from different angles and produce a single image. It is important for multi-view reconstruction, but as of yet there is no consensus on best practices for compounding. Current popular methods inevitably suppress or altogether leave out bright or dark regions that are useful and potentially introduce new artifacts. In this work, we establish a new algorithm to compound the overlapping pixels from different viewpoints in ultrasound.

Methods

Inspired by image fusion algorithms and ultrasound confidence, we uniquely leverage Laplacian and Gaussian pyramids to preserve the maximum boundary contrast without overemphasizing noise, speckles, and other artifacts in the compounded image, while taking the direction of the ultrasound probe into account. Besides, we designed an algorithm that detects the useful boundaries in ultrasound images to further improve the boundary contrast.

Results

We evaluate our algorithm by comparing it with previous algorithms both qualitatively and quantitatively, and we show that our approach not only preserves both light and dark details, but also somewhat suppresses noise and artifacts, rather than amplifying them. We also show that our algorithm can improve the performance of downstream tasks like segmentation.

Conclusion

Our proposed method that is based on confidence, contrast, and both Gaussian and Laplacian pyramids appears to be better at preserving contrast at anatomic boundaries while suppressing artifacts than any of the other approaches we tested. This algorithm may have future utility with downstream tasks such as 3D ultrasound volume reconstruction and segmentation.

LatLRR-FCNs: Latent Low-Rank Representation With Fully Convolutional Networks for Medical Image Fusion

Medical image fusion, which aims to derive complementary information from multi-modality medical images, plays an important role in many clinical applications, such as medical diagnostics and treatment. We propose the LatLRR-FCNs, which is a hybrid medical image fusion framework consisting of the latent low-rank representation (LatLRR) and the fully convolutional networks (FCNs). Specifically, the LatLRR module is used to decompose the multi-modality medical images into low-rank and saliency components, which can provide fine-grained details and preserve energies, respectively. The FCN module aims to preserve both global and local information by generating the weighting maps for each modality image. The final weighting map is obtained using the weighted local energy and the weighted sum of the eight-neighborhood-based modified Laplacian method. The fused low-rank component is generated by combining the low-rank components of each modality image according to the guidance provided by the final weighting map within pyramid-based fusion. A simple sum strategy is used for the saliency components. The usefulness and efficiency of the proposed framework are thoroughly evaluated on four medical image fusion tasks, including computed tomography (CT) and magnetic resonance (MR), T1- and T2-weighted MR, positron emission tomography and MR, and single-photon emission CT and MR. The results demonstrate that by leveraging the LatLRR for image detail extraction and the FCNs for global and local information description, we can achieve performance superior to the state-of-the-art methods in terms of both objective assessment and visual quality in some cases. Furthermore, our method has a competitive performance in terms of computational costs compared to other baselines.

Multimodal medical image fusion via laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy

BACKGROUND

In recent years, numerous fusion algorithms have been proposed for multimodal medical images. The Laplacian pyramid is one type of multiscale fusion method. Although the pyramid-based fusion algorithm can fuse images well, it has the disadvantages of edge degradation, detail loss and image smoothing as the number of decomposition layers increase, which is harmful for medical diagnosis and analysis.

METHOD

This paper proposes a medical image fusion algorithm based on the Laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy, which can greatly improve the edge quality. First, multimodal medical images are reconstructed through convolutional neural network. Then, the Laplacian pyramid is applied in the decomposition and fusion process. The optimal number of decomposition layers is determined by experiments. In addition, a local gradient energy fusion strategy is utilized to fuse the coefficients in each layer. Finally, the fused image is output through Laplacian inverse transformation.

RESULTS

Compared with existing algorithms, our fusion results represent better vision quality performance. Furthermore, our algorithm is considerably superior to the compared algorithms in objective indicators. In addition, in our fusion results of Alzheimer and Glioma, the disease details are much clearer than those of compared algorithms, which can provide a reliable basis for doctors to analyze disease and make pathological diagnoses.

A practical PET/CT data visualization method with dual-threshold PET colorization and image fusion

Multi-modal medical imaging has emerged as a general trend in clinical diagnosis and treatment planning. In recent years, great efforts have been made to investigate and develop dual-modality scanners, among which PET/CT is the most widespread one in clinical practice. In this paper, we propose a simple yet effective PET/CT data visualization method that can integrate these two modalities into composite data for better observation. The proposed method consists of three main steps. First, a PET data colorization approach is presented based on a dual-threshold scheme, which applies a pair of high and low thresholds to colorize the PET image. Then, to extract functional information from the PET image more adequately, unlike traditional blending fashion that directly uses the CT image as underlay, we merge the CT and the PET images with a Laplacian pyramid (LP)-based image fusion approach to generate the underlay. Finally, the visualization result is obtained by blending the fused image and the colorized PET image. Experiments are conducted on 5 sets of PET/CT scans that contain 200 paired slices in total. The ClearCanvas software and the method using the presented PET colorization approach but with the CT image as underlay are adopted for comparison. Experimental results demonstrate that the proposed method can achieve more promising performance in terms of both visual perception and quantitative assessment. The code of the proposed method has been made available online athttps://github.com/yuliu316316/Visualization.

A robust multi-scale approach to quantitative susceptibility mapping

Quantitative Susceptibility Mapping (QSM), best known as a surrogate for tissue iron content, is becoming a highly relevant MRI contrast for monitoring cellular and vascular status in aging, addiction, traumatic brain injury and, in general, a wide range of neurological disorders. In this study we present a new Bayesian QSM algorithm, named Multi-Scale Dipole Inversion (MSDI), which builds on the nonlinear Morphology-Enabled Dipole Inversion (nMEDI) framework, incorporating three additional features: (i) improved implementation of Laplace's equation to reduce the influence of background fields through variable harmonic filtering and subsequent deconvolution, (ii) improved error control through dynamic phase-reliability compensation across spatial scales, and (iii) scalewise use of the morphological prior. More generally, this new pre-conditioned QSM formalism aims to reduce the impact of dipole-incompatible fields and measurement errors such as flow effects, poor signal-to-noise ratio or other data inconsistencies that can lead to streaking and shadowing artefacts. In terms of performance, MSDI is the first algorithm to rank in the top-10 for all metrics evaluated in the 2016 QSM Reconstruction Challenge. It also demonstrated lower variance than nMEDI and more stable behaviour in scan-rescan reproducibility experiments for different MRI acquisitions at 3 and 7 Tesla. In the present work, we also explored new forms of susceptibility MRI contrast making explicit use of the differential information across spatial scales. Specifically, we show MSDI-derived examples of: (i) enhanced anatomical detail with susceptibility inversions from short-range dipole fields (hereby referred to as High-Pass Susceptibility Mapping or HPSM), (ii) high specificity to venous-blood susceptibilities for highly regularised HPSM (making a case for MSDI-based Venography or VenoMSDI), (iii) improved tissue specificity (and possibly statistical conditioning) for Macroscopic-Vessel Suppressed Susceptibility Mapping (MVSSM), and (iv) high spatial specificity and definition for HPSM-based Susceptibility-Weighted Imaging (HPSM-SWI) and related intensity projections.

An improved nonlinear diffusion in Laplacian pyramid domain for cone beam CT denoising during image-guided vascular intervention

BACKGROUND

Cone-beam computed tomography (CBCT) acquisition during endovascular aneurysm repair is an emergent technology with more and more applications. It may provide 3-D information to achieve guidance of intervention. However, there is growing concern on the overall radiation doses delivered to patients, thus a low dose protocol is called when scanning. But CBCT images with a low dose protocol are degraded, resulting in streak artifacts and decreased contrast-to-noise ratio (CNR). In this paper, a Laplacian pyramid-based nonlinear diffusion is proposed to improve the quality of CBCT images.

METHOD

We first transform the CBCT image into its pyramid domain, then a modified nonlinear diffusion is performed in each level to remove noise across edges while keeping edges as far as possible. The improved diffusion coefficient is a function of the gradient magnitude image; the threshold in the modified diffusion function is estimated using the median absolute deviation (MAD) estimator; the time step is automatically determined by iterative image changes and the iteration is stopped according to mean absolute error between two adjacent diffusions. Finally, we reconstruct the Laplacian pyramid using the processed pyramid images in each level.

RESULT

Results from simulation show that the filtered image from the proposed method has the highest peak signal-noise ratio (81.92), the highest correlation coefficient (99.77%) and the lowest mean square error (27.61), compared with the other four methods. In addition, it has highest contrast-to-noise ratio and sharpness in ROIs. Results from real CBCT images show that the proposed method shows better smoothness in homogeneous regions meanwhile keeps bony structures clear.

CONCLUSION

Simulation and patient studies show that the proposed method has a good tradeoff between noise/artifacts suppression and edge preservation.

A Streaming Motion Magnification Core for Smart Image Sensors

This paper proposes a modified Eulerian Video Magnification (EVM) algorithm and a hardware implementation of a motion magnification core for smart image sensors. Compared to the original EVM algorithm, we perform the pixel-wise temporal bandpass filtering only once rather than multiple times on all scale layers, to reduce the memory and multiplier requirement for hardware implementation. A pixel stream processing architecture with pipelined blocks is proposed for the magnification core, enabling it to readily fit common image sensing components with streaming pixel output, while achieving higher performance with lower system cost. We implemented an FPGA-based prototype that is able to process up to 90M pixels per second and magnify subtle motion. The motion magnification results are comparable to the original algorithm running on PC.

Robust Quantitative Assessment of Trabecular Microarchitecture in Extremity Cone-Beam CT Using Optimized Segmentation Algorithms

Purpose

In-vivo evaluation of bone microarchitecture remains challenging because of limited resolution of conventional orthopaedic imaging modalities. We investigate the performance of flat-panel detector extremity Cone-Beam CT (CBCT) in quantitative analysis of trabecular bone. To enable accurate morphometry of fine trabecular bone architecture, advanced CBCT pre-processing and segmentation algorithms are developed.

Methods

The study involved 35 transilliac bone biopsy samples imaged on extremity CBCT (voxel size 75 μm, imaging dose ~13 mGy) and gold standard μCT (voxel size 7.67 μm). CBCT image segmentation was performed using (i) global Otsu's thresholding, (ii) Bernsen's local thresholding, (iii) Bernsen's local thresholding with additional histogram-based global pre-thresholding, and (iv) the same as (iii) but combined with contrast enhancement using a Laplacian Pyramid. Correlations between extremity CBCT with the different segmentation algorithms and gold standard μCT were investigated for measurements of Bone Volume over Total Volume (BV/TV), Trabecular Thickness (Tb.Th), Trabecular Spacing (Tb.Sp), and Trabecular Number (Tb.N).

Results

The combination of local thresholding with global pre-thresholding and Laplacian contrast enhancement outperformed other CBCT segmentation methods. Using this optimal segmentation scheme, strong correlation between extremity CBCT and μCT was achieved, with Pearson coefficients of 0.93 for BV/TV, 0.89 for Tb.Th, 0.91 for Tb.Sp, and 0.88 for Tb.N (all results statistically significant). Compared to a simple global CBCT segmentation using Otsu's algorithm, the advanced segmentation method achieved ~20% improvement in the correlation coefficient for Tb.Th and ~50% improvement for Tb.Sp.

Conclusions

Extremity CBCT combined with advanced image pre-processing and segmentation achieves high correlation with gold standard μCT in measurements of trabecular microstructure. This motivates ongoing development of clinical applications of extremity CBCT in in-vivo evaluation of bone health e.g. in early osteoarthritis and osteoporosis.