1
|
Waida H, Yamazaki K, Tokuhisa A, Wada M, Wada Y. Investigating self-supervised image denoising with denaturation. Neural Netw 2025; 184:106966. [PMID: 39700824 DOI: 10.1016/j.neunet.2024.106966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2024] [Revised: 10/08/2024] [Accepted: 11/25/2024] [Indexed: 12/21/2024]
Abstract
Self-supervised learning for image denoising problems in the presence of denaturation for noisy data is a crucial approach in machine learning. However, theoretical understanding of the performance of the approach that uses denatured data is lacking. To provide better understanding of the approach, in this paper, we analyze a self-supervised denoising algorithm that uses denatured data in depth through theoretical analysis and numerical experiments. Through the theoretical analysis, we discuss that the algorithm finds desired solutions to the optimization problem with the population risk, while the guarantee for the empirical risk depends on the hardness of the denoising task in terms of denaturation levels. We also conduct several experiments to investigate the performance of an extended algorithm in practice. The results indicate that the algorithm training with denatured images works, and the empirical performance aligns with the theoretical results. These results suggest several insights for further improvement of self-supervised image denoising that uses denatured data in future directions.
Collapse
Affiliation(s)
- Hiroki Waida
- Department of Mathematical and Computing Science, Institute of Science Tokyo, 2-12-1 Ookayama, Meguro-ku, Tokyo, 152-8550, Japan
| | - Kimihiro Yamazaki
- Fujitsu Limited, 4-1-1 Kamikodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa, 211-8588, Japan
| | - Atsushi Tokuhisa
- RIKEN Center for Computational Science, 7-1-26 Minatojima-minami-machi, Chuo-ku, Kobe, Hyogo, 650-0047, Japan
| | - Mutsuyo Wada
- Fujitsu Limited, 4-1-1 Kamikodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa, 211-8588, Japan
| | - Yuichiro Wada
- Fujitsu Limited, 4-1-1 Kamikodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa, 211-8588, Japan; RIKEN Center for Advanced Intelligence Project, Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan.
| |
Collapse
|
2
|
Fröch JE, Chakravarthula P, Sun J, Tseng E, Colburn S, Zhan A, Miller F, Wirth-Singh A, Tanguy QAA, Han Z, Böhringer KF, Heide F, Majumdar A. Beating spectral bandwidth limits for large aperture broadband nano-optics. Nat Commun 2025; 16:3025. [PMID: 40155619 PMCID: PMC11953342 DOI: 10.1038/s41467-025-58208-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2024] [Accepted: 03/13/2025] [Indexed: 04/01/2025] Open
Abstract
Flat optics have been proposed as an attractive approach for the implementation of new imaging and sensing modalities to replace and augment refractive optics. However, chromatic aberrations impose fundamental limitations on diffractive flat optics. As such, true broadband high-quality imaging has thus far been out of reach for fast f-numbers, large aperture, flat optics. In this work, we overcome intrinsic spectral bandwidth limitations, achieving broadband imaging in the visible wavelength range with a flat meta-optic, co-designed with computational reconstruction. We derive the necessary conditions for a broadband, 1 cm aperture, f/2 flat optic, with a diagonal field of view of 30° and average system MTF contrast of 20% or larger for a spatial frequency of 100 lp/mm in the visible band (>30% for <70 lp/mm). Finally, we use a coaxial, dual-aperture system to train the broadband imaging meta-optic with a learned reconstruction method operating on pair-wise captured imaging data. Fundamentally, our work challenges the entrenched belief of the inability of capturing high-quality, full-color images using a single large aperture meta-optic.
Collapse
Affiliation(s)
- Johannes E Fröch
- Department of Physics, University of Washington, Seattle, WA, USA.
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA.
| | - Praneeth Chakravarthula
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - Jipeng Sun
- Department of Computer Science, Princeton University, Princeton, NJ, USA
| | - Ethan Tseng
- Department of Computer Science, Princeton University, Princeton, NJ, USA
| | - Shane Colburn
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
| | - Alan Zhan
- Tunoptix, 4000 Mason Road 300, Fluke Hall, Seattle, WA, USA
| | - Forrest Miller
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
| | - Anna Wirth-Singh
- Department of Physics, University of Washington, Seattle, WA, USA
| | - Quentin A A Tanguy
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
| | - Zheyi Han
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
| | - Karl F Böhringer
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
- Department of Bioengineering, University of Washington, Seattle, WA, USA
- Institute for Nano-Engineered Systems, University of Washington, Seattle, WA, USA
| | - Felix Heide
- Department of Computer Science, Princeton University, Princeton, NJ, USA
| | - Arka Majumdar
- Department of Physics, University of Washington, Seattle, WA, USA.
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA.
| |
Collapse
|
3
|
Jia L, Jia B, Li Z, Zhang Y, Gui Z. A cross-type multi-dimensional network based on feature enhancement and triple interactive attention for LDCT denoising. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2025; 33:393-404. [PMID: 39973791 DOI: 10.1177/08953996241306696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/21/2025]
Abstract
BackgroundNumerous deep leaning methods for low-dose computed technology (CT) image denoising have been proposed, achieving impressive results. However, issues such as loss of structure and edge information and low denoising efficiency still exist.ObjectiveTo improve image denoising quality, an enhanced multi-dimensional hybrid attention LDCT image denoising network based on edge detection is proposed in this paper.MethodsIn our network, we employ a trainable Sobel convolution to design an edge enhancement module and fuse an enhanced triplet attention network (ETAN) after each 3 × 3 convolutional layer to extract richer features more comprehensively and suppress useless information. During the training process, we adopt a strategy that combines total variation loss (TVLoss) with mean squared error (MSE) loss to reduce high-frequency artifacts in image reconstruction and balance image denoising and detail preservation.ResultsCompared with other advanced algorithms (CT-former, REDCNN and EDCNN), our proposed model achieves the best PSNR and SSIM values in CT image of the abdomen, which are 34.8211and 0.9131, respectively.ConclusionThrough comparative experiments with other related algorithms, it can be seen that the algorithm proposed in this article has achieved significant improvements in both subjective vision and objective indicators.
Collapse
Affiliation(s)
- Lina Jia
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan, China
- Shanxi Key Laboratory of Wireless Communication and Detection, Taiyuan, China
| | - Beibei Jia
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan, China
| | - Zongyang Li
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan, China
| | - Yizhuo Zhang
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan, China
| | - Zhiguo Gui
- State Key Laboratory of Dynamic Measurement Technology, North University of China, Taiyuan, China
| |
Collapse
|
4
|
Dehdab R, Brendel JM, Streich S, Ladurner R, Stenzl B, Mueck J, Gassenmaier S, Krumm P, Werner S, Herrmann J, Nikolaou K, Afat S, Brendlin A. Evaluation of a Deep Learning Denoising Algorithm for Dose Reduction in Whole-Body Photon-Counting CT Imaging: A Cadaveric Study. Acad Radiol 2025:S1076-6332(24)01040-7. [PMID: 39818525 DOI: 10.1016/j.acra.2024.12.052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2024] [Revised: 12/21/2024] [Accepted: 12/22/2024] [Indexed: 01/18/2025]
Abstract
RATIONALE AND OBJECTIVES Photon Counting CT (PCCT) offers advanced imaging capabilities with potential for substantial radiation dose reduction; however, achieving this without compromising image quality remains a challenge due to increased noise at lower doses. This study aims to evaluate the effectiveness of a deep learning (DL)-based denoising algorithm in maintaining diagnostic image quality in whole-body PCCT imaging at reduced radiation levels, using real intraindividual cadaveric scans. MATERIALS AND METHODS Twenty-four cadaveric human bodies underwent whole-body CT scans on a PCCT scanner (NAEOTOM Alpha, Siemens Healthineers) at four different dose levels (100%, 50%, 25%, and 10% mAs). Each scan was reconstructed using both ADMIRE level 2 and a DL algorithm (ClariCT.AI, ClariPi Inc.), resulting in 192 datasets. Objective image quality was assessed by measuring CT value stability, image noise, and contrast-to-noise ratio (CNR) across consistent regions of interest (ROIs) in the liver parenchyma. Two radiologists independently evaluated subjective image quality based on overall image clarity, sharpness, and contrast. Inter-rater agreement was determined using Spearman's correlation coefficient, and statistical analysis included mixed-effects modeling to assess objective and subjective image quality. RESULTS Objective analysis showed that the DL denoising algorithm did not significantly alter CT values (p ≥ 0.9975). Noise levels were consistently lower in denoised datasets compared to the Original (p < 0.0001). No significant differences were observed between the 25% mAs denoised and the 100% mAs original datasets in terms of noise and CNR (p ≥ 0.7870). Subjective analysis revealed strong inter-rater agreement (r ≥ 0.78), with the 50% mAs denoised datasets rated superior to the 100% mAs original datasets (p < 0.0001) and no significant differences detected between the 25% mAs denoised and 100% mAs original datasets (p ≥ 0.9436). CONCLUSION The DL denoising algorithm maintains image quality in PCCT imaging while enabling up to a 75% reduction in radiation dose. This approach offers a promising method for reducing radiation exposure in clinical PCCT without compromising diagnostic quality.
Collapse
Affiliation(s)
- Reza Dehdab
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.).
| | - Jan M Brendel
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Sebastian Streich
- Institute of Clinical Anatomy and Cell Analysis, Eberhard Karls University of Tuebingen, Elfriede-Aulhorn-Str 8, 72076, Tuebingen, Germany (S.S., R.L.)
| | - Ruth Ladurner
- Institute of Clinical Anatomy and Cell Analysis, Eberhard Karls University of Tuebingen, Elfriede-Aulhorn-Str 8, 72076, Tuebingen, Germany (S.S., R.L.)
| | - Benedikt Stenzl
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Jonas Mueck
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Sebastian Gassenmaier
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Patrick Krumm
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Sebastian Werner
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Judith Herrmann
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Konstantin Nikolaou
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Saif Afat
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| | - Andreas Brendlin
- Department of Radiology, University Hospital Tuebingen, Hoppe-Seyler-Str. 3, 72076 Tuebingen, Germany (R.D., J.M.B., B.S., J.M., S.G., P.K., S.W., J.H., K.N., S.A., A.B.)
| |
Collapse
|
5
|
Wirtensohn S, Schmid C, Berthe D, John D, Heck L, Taphorn K, Flenner S, Herzen J. Self-supervised denoising of grating-based phase-contrast computed tomography. Sci Rep 2024; 14:32169. [PMID: 39741166 DOI: 10.1038/s41598-024-83517-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Accepted: 12/16/2024] [Indexed: 01/02/2025] Open
Abstract
In the last decade, grating-based phase-contrast computed tomography (gbPC-CT) has received growing interest. It provides additional information about the refractive index decrement in the sample. This signal shows an increased soft-tissue contrast. However, the resolution dependence of the signal poses a challenge: its contrast enhancement is overcompensated by the low resolution in low-dose applications such as clinical computed tomography. As a result, the implementation of gbPC-CT is currently tied to a higher dose. To reduce the dose, we introduce the self-supervised deep learning network Noise2Inverse into the field of gbPC-CT. We evaluate the behavior of the Noise2Inverse parameters on the phase-contrast results. Afterward, we compare its results with other denoising methods, namely the Statistical Iterative Reconstruction, Block Matching 3D, and Patchwise Phase Retrieval. In the example of Noise2Inverse, we show that deep learning networks can deliver superior denoising results with respect to the investigated image quality metrics. Their application allows to increase the resolution while maintaining the dose. At higher resolutions, gbPC-CT can naturally deliver higher contrast than conventional absorption-based CT. Therefore, the application of machine learning-based denoisers shifts the dose-normalized image quality in favor of gbPC-CT, bringing it one step closer to medical application.
Collapse
Affiliation(s)
- Sami Wirtensohn
- Research Group Biomedical Imaging Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany.
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany.
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany.
- Institute of Materials Physics, Helmholtz-Zentrum Hereon, 21502, Geesthacht, Germany.
| | - Clemens Schmid
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
- Paul Scherer Institute, Forschungsstrasse 111, 5232, Villigen, Switzerland
| | - Daniel Berthe
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
| | - Dominik John
- Research Group Biomedical Imaging Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
- Institute of Materials Physics, Helmholtz-Zentrum Hereon, 21502, Geesthacht, Germany
| | - Lisa Heck
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
| | - Kirsten Taphorn
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
| | - Silja Flenner
- Institute of Materials Physics, Helmholtz-Zentrum Hereon, 21502, Geesthacht, Germany
| | - Julia Herzen
- Research Group Biomedical Imaging Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Chair of Biomedical Physics, Department of Physics, TUM School of Natural Sciences, Technical University of Munich, 85748, Garching, Germany
- Munich Institute of Biomedical Engineering, Technical University of Munich, 85748, Garching, Germany
| |
Collapse
|
6
|
Yamamoto A, Yamanaka A, Iida K, Shimada Y, Hata S. Integrating machine learning with advanced processing and characterization for polycrystalline materials: a methodology review and application to iron-based superconductors. SCIENCE AND TECHNOLOGY OF ADVANCED MATERIALS 2024; 26:2436347. [PMID: 39845724 PMCID: PMC11753020 DOI: 10.1080/14686996.2024.2436347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/21/2024] [Revised: 11/22/2024] [Accepted: 11/26/2024] [Indexed: 01/24/2025]
Abstract
In this review, we present a new set of machine learning-based materials research methodologies for polycrystalline materials developed through the Core Research for Evolutionary Science and Technology project of the Japan Science and Technology Agency. We focus on the constituents of polycrystalline materials (i.e. grains, grain boundaries [GBs], and microstructures) and summarize their various aspects (experimental synthesis, artificial single GBs, multiscale experimental data acquisition via electron microscopy, formation process modeling, property description modeling, 3D reconstruction, and data-driven design methods). Specifically, we discuss a mechanochemical process involving high-energy milling, in situ observation of microstructural formation using 3D scanning transmission electron microscopy, phase-field modeling coupled with Bayesian data assimilation, nano-orientation analysis via scanning precession electron diffraction, semantic segmentation using neural network models, and the Bayesian-optimization-based process design using BOXVIA software. As a proof of concept, a researcher- and data-driven process design methodology is applied to a polycrystalline iron-based superconductor to evaluate its bulk magnet properties. Finally, future challenges and prospects for data-driven material development and iron-based superconductors are discussed.
Collapse
Affiliation(s)
- Akiyasu Yamamoto
- Department of Applied Physics, Tokyo University of Agriculture and Technology, Tokyo, Japan
- JST-CREST, Saitama, Japan
| | - Akinori Yamanaka
- JST-CREST, Saitama, Japan
- Department of Mechanical System Engineering, Tokyo University of Agriculture and Technology, Tokyo, Japan
| | - Kazumasa Iida
- JST-CREST, Saitama, Japan
- College of Industrial Technology, Nihon University, Chiba, Japan
| | - Yusuke Shimada
- JST-CREST, Saitama, Japan
- Department of Advanced Materials Science and Engineering, Kyushu University, Fukuoka, Japan
| | - Satoshi Hata
- JST-CREST, Saitama, Japan
- Department of Advanced Materials Science and Engineering, Kyushu University, Fukuoka, Japan
- The Ultramicroscopy Research Center, Kyushu University, Fukuoka, Japan
| |
Collapse
|
7
|
Hallett JE, Bruza P, Jermyn M, Li K, Pogue BW. Noise & mottle suppression methods for cumulative Cherenkov images of radiation therapy delivery. Phys Med Biol 2024; 69:225015. [PMID: 39474803 PMCID: PMC11639195 DOI: 10.1088/1361-6560/ad8c93] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2024] [Revised: 09/25/2024] [Accepted: 10/29/2024] [Indexed: 11/13/2024]
Abstract
Purpose.Cherenkov imaging during radiotherapy provides a real time visualization of beam delivery on patient tissue, which can be used dynamically for incident detection or to review a summary of the delivered surface signal for treatment verification. Very few photons form the images, and one limitation is that the noise level per frame can be quite high, and mottle in the cumulative processed images can cause mild overall noise. This work focused on removing or suppressing noise via image postprocessing.Approach.Images were analyzed for peak-signal-to-noise and spatial frequencies present, and several established noise/mottle reduction algorithms were chosen based upon these observations. These included total variation minimization (TV-L1), non-local means filter (NLM), block-matching 3D (BM3D), alpha (adaptive) trimmed mean (ATM), and bilateral filtering. Each were applied to images acquired using a BeamSite camera (DoseOptics) imaged signal from 6x photons from a TrueBeam linac delivering dose at 600 MU min-1incident on an anthropomorphic phantom and tissue slab phantom in various configurations and beam angles. The standard denoised images were tested for PSNR, noise power spectrum (NPS) and image sharpness.Results.The average peak-signal-to-noise ratio (PSNR) increase was 17.4% for TV-L1. NLM denoising increased the average PSNR by 19.1%, BM3D processing increased it by12.1% and the bilateral filter increased the average PSNR by 19.0%. Lastly, the ATM filter resulted in the lowest average PSNR increase of 10.9%. Of all of these, the NLM and bilateral filters produced improved edge sharpness with, generally, the lowest NPS curve.Conclusion.For cumulative image Cherenkov data, NLM and the bilateral filter yielded optimal denoising with the TV-L1 algorithm giving comparable results. Single video frame Cherenkov images exhibit much higher noise levels compared to cumulative images. Noise suppression algorithms for these frame rates will likely be a different processing pipeline involving these filters incorporated with machine learning.
Collapse
Affiliation(s)
- Jeremy E Hallett
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, United States of America
| | - Petr Bruza
- Thayer School of Engineering at Dartmouth, Hanover, NH, United States of America
- DoseOptics LLC, Lebanon, NH, United States of America
| | - Michael Jermyn
- Thayer School of Engineering at Dartmouth, Hanover, NH, United States of America
- DoseOptics LLC, Lebanon, NH, United States of America
| | - Ke Li
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, United States of America
| | - Brian W Pogue
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, United States of America
- Thayer School of Engineering at Dartmouth, Hanover, NH, United States of America
- DoseOptics LLC, Lebanon, NH, United States of America
| |
Collapse
|
8
|
Marcos L, Babyn P, Alirezaie J. Pure Vision Transformer (CT-ViT) with Noise2Neighbors Interpolation for Low-Dose CT Image Denoising. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:2669-2687. [PMID: 38622385 PMCID: PMC11522238 DOI: 10.1007/s10278-024-01108-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 03/26/2024] [Accepted: 03/26/2024] [Indexed: 04/17/2024]
Abstract
Convolutional neural networks (CNN) have been used for a wide variety of deep learning applications, especially in computer vision. For medical image processing, researchers have identified certain challenges associated with CNNs. These challenges encompass the generation of less informative features, limitations in capturing both high and low-frequency information within feature maps, and the computational cost incurred when enhancing receptive fields by deepening the network. Transformers have emerged as an approach aiming to address and overcome these specific limitations of CNNs in the context of medical image analysis. Preservation of all spatial details of medical images is necessary to ensure accurate patient diagnosis. Hence, this research introduced the use of a pure Vision Transformer (ViT) for a denoising artificial neural network for medical image processing specifically for low-dose computed tomography (LDCT) image denoising. The proposed model follows a U-Net framework that contains ViT modules with the integration of Noise2Neighbor (N2N) interpolation operation. Five different datasets containing LDCT and normal-dose CT (NDCT) image pairs were used to carry out this experiment. To test the efficacy of the proposed model, this experiment includes comparisons between the quantitative and visual results among CNN-based (BM3D, RED-CNN, DRL-E-MP), hybrid CNN-ViT-based (TED-Net), and the proposed pure ViT-based denoising model. The findings of this study showed that there is about 15-20% increase in SSIM and PSNR when using self-attention transformers than using the typical pure CNN. Visual results also showed improvements especially when it comes to showing fine structural details of CT images.
Collapse
Affiliation(s)
- Luella Marcos
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada
| | - Paul Babyn
- Department of Medical Imaging, University of Saskatchewan, 105 Administration Pl, Saskatoon, SK S7N0W8, Saskatchewan, Canada
| | - Javad Alirezaie
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada.
| |
Collapse
|
9
|
Sharma V, Awate SP. Adversarial EM for variational deep learning: Application to semi-supervised image quality enhancement in low-dose PET and low-dose CT. Med Image Anal 2024; 97:103291. [PMID: 39121545 DOI: 10.1016/j.media.2024.103291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2022] [Revised: 07/23/2024] [Accepted: 07/25/2024] [Indexed: 08/12/2024]
Abstract
In positron emission tomography (PET) and X-ray computed tomography (CT), reducing radiation dose can cause significant degradation in image quality. For image quality enhancement in low-dose PET and CT, we propose a novel theoretical adversarial and variational deep neural network (DNN) framework relying on expectation maximization (EM) based learning, termed adversarial EM (AdvEM). AdvEM proposes an encoder-decoder architecture with a multiscale latent space, and generalized-Gaussian models enabling datum-specific robust statistical modeling in latent space and image space. The model robustness is further enhanced by including adversarial learning in the training protocol. Unlike typical variational-DNN learning, AdvEM proposes latent-space sampling from the posterior distribution, and uses a Metropolis-Hastings scheme. Unlike existing schemes for PET or CT image enhancement which train using pairs of low-dose images with their corresponding normal-dose versions, we propose a semi-supervised AdvEM (ssAdvEM) framework that enables learning using a small number of normal-dose images. AdvEM and ssAdvEM enable per-pixel uncertainty estimates for their outputs. Empirical analyses on real-world PET and CT data involving many baselines, out-of-distribution data, and ablation studies show the benefits of the proposed framework.
Collapse
Affiliation(s)
- Vatsala Sharma
- Computer Science and Engineering (CSE) Department, Indian Institute of Technology (IIT) Bombay, Mumbai, India.
| | - Suyash P Awate
- Computer Science and Engineering (CSE) Department, Indian Institute of Technology (IIT) Bombay, Mumbai, India
| |
Collapse
|
10
|
Meader E, Walcheck MT, Leder MR, Jing R, Wrighton PJ, Sugden WW, Najia MA, Oderberg IM, Taylor VM, LeBlanc ZC, Quenzer ED, Lim SE, Daley GQ, Goessling W, North TE. Bnip3lb-driven mitophagy sustains expansion of the embryonic hematopoietic stem cell pool. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.09.23.614531. [PMID: 39386657 PMCID: PMC11463499 DOI: 10.1101/2024.09.23.614531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/12/2024]
Abstract
Embryonic hematopoietic stem and progenitor cells (HSPCs) have the unique ability to undergo rapid proliferation while maintaining multipotency, a clinically-valuable quality which currently cannot be replicated in vitro. Here, we show that embryonic HSPCs achieve this state by precise spatio-temporal regulation of reactive oxygen species (ROS) via Bnip3lb-associated developmentally-programmed mitophagy, a distinct autophagic regulatory mechanism from that of adult HSPCs. While ROS drives HSPC specification in the dorsal aorta, scRNAseq and live-imaging of Tg(ubi:mitoQC) zebrafish indicate that mitophagy initiates as HSPCs undergo endothelial-to-hematopoietic transition and colonize the caudal hematopoietic tissue (CHT). Knockdown of bnip3lb reduced mitophagy and HSPC numbers in the CHT by promoting myeloid-biased differentiation and apoptosis, which was rescued by anti-oxidant exposure. Conversely, induction of mitophagy enhanced both embryonic HSPC and lymphoid progenitor numbers. Significantly, mitophagy activation improved ex vivo functional capacity of hematopoietic progenitors derived from human-induced pluripotent stem cells (hiPSCs), enhancing serial-replating hematopoietic colony forming potential. HIGHLIGHTS ROS promotes HSPC formation in the dorsal aorta but negatively affects maintenance thereafter.HSPCs colonizing secondary niches control ROS levels via Bnip3lb-directed mitophagy.Mitophagy protects nascent HSPCs from ROS-associated apoptosis and maintains multipotency.Induction of mitophagy enhances long-term hematopoietic potential of iPSC-derived HSPCs.
Collapse
|
11
|
Hein D, Holmin S, Szczykutowicz T, Maltz JS, Danielsson M, Wang G, Persson M. Noise suppression in photon-counting computed tomography using unsupervised Poisson flow generative models. Vis Comput Ind Biomed Art 2024; 7:24. [PMID: 39311990 PMCID: PMC11420411 DOI: 10.1186/s42492-024-00175-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 09/01/2024] [Indexed: 09/26/2024] Open
Abstract
Deep learning (DL) has proven to be important for computed tomography (CT) image denoising. However, such models are usually trained under supervision, requiring paired data that may be difficult to obtain in practice. Diffusion models offer unsupervised means of solving a wide range of inverse problems via posterior sampling. In particular, using the estimated unconditional score function of the prior distribution, obtained via unsupervised learning, one can sample from the desired posterior via hijacking and regularization. However, due to the iterative solvers used, the number of function evaluations (NFE) required may be orders of magnitudes larger than for single-step samplers. In this paper, we present a novel image denoising technique for photon-counting CT by extending the unsupervised approach to inverse problem solving to the case of Poisson flow generative models (PFGM)++. By hijacking and regularizing the sampling process we obtain a single-step sampler, that is NFE = 1. Our proposed method incorporates posterior sampling using diffusion models as a special case. We demonstrate that the added robustness afforded by the PFGM++ framework yields significant performance gains. Our results indicate competitive performance compared to popular supervised, including state-of-the-art diffusion-style models with NFE = 1 (consistency models), unsupervised, and non-DL-based image denoising techniques, on clinical low-dose CT data and clinical images from a prototype photon-counting CT system developed by GE HealthCare.
Collapse
Affiliation(s)
- Dennis Hein
- Department of Physics, KTH Royal Institute of Technology, Stockholm, 1142, Sweden.
- MedTechLabs, Karolinska University Hospital, Stockholm, 17164, Sweden.
| | - Staffan Holmin
- Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, 17164, Sweden
- Department of Neuroradiology, Karolinska University Hospital, Stockholm, 17164, Sweden
| | - Timothy Szczykutowicz
- Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, 53705, United States
| | | | - Mats Danielsson
- Department of Physics, KTH Royal Institute of Technology, Stockholm, 1142, Sweden
- MedTechLabs, Karolinska University Hospital, Stockholm, 17164, Sweden
| | - Ge Wang
- Department of Biomedical Engineering, School of Engineering, Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY, 12180, United States
| | - Mats Persson
- Department of Physics, KTH Royal Institute of Technology, Stockholm, 1142, Sweden
- MedTechLabs, Karolinska University Hospital, Stockholm, 17164, Sweden
| |
Collapse
|
12
|
Zhang Y, Li Z, Tong X, Xie Z, Huang S, Zhang YE, Ke HB, Wang WH, Zhou J. Three-dimensional atomic insights into the metal-oxide interface in Zr-ZrO 2 nanoparticles. Nat Commun 2024; 15:7624. [PMID: 39223157 PMCID: PMC11369257 DOI: 10.1038/s41467-024-52026-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Accepted: 08/23/2024] [Indexed: 09/04/2024] Open
Abstract
Metal-oxide interfaces with poor coherency have specific properties comparing to bulk materials and offer broad applications in heterogeneous catalysis, battery, and electronics. However, current understanding of the three-dimensional (3D) atomic metal-oxide interfaces remains limited because of their inherent structural complexity and the limitations of conventional two-dimensional imaging techniques. Here, we determine the 3D atomic structure of metal-oxide interfaces in zirconium-zirconia nanoparticles using atomic-resolution electron tomography. We quantitatively analyze the atomic concentration and the degree of oxidation, and find the coherency and translational symmetry of the interfaces are broken. Atoms at the interface have low structural ordering, low coordination, and elongated bond length. Moreover, we observe porous structures such as Zr vacancies and nano-pores, and investigate their distribution. Our findings provide a clear 3D atomic picture of metal-oxide interface with direct experimental evidence. We anticipate this work could encourage future studies on fundamental problems of oxides, such as interfacial structures in semiconductor and atomic motion during oxidation process.
Collapse
Affiliation(s)
- Yao Zhang
- Beijing National Laboratory for Molecular Sciences, Center for Integrated Spectroscopy, College of Chemistry and Molecular Engineering, Peking University, Beijing, 100871, China
| | - Zezhou Li
- Beijing National Laboratory for Molecular Sciences, Center for Integrated Spectroscopy, College of Chemistry and Molecular Engineering, Peking University, Beijing, 100871, China
| | - Xing Tong
- Songshan Lake Materials Laboratory, Dongguan, 523808, China
| | - Zhiheng Xie
- Beijing National Laboratory for Molecular Sciences, Center for Integrated Spectroscopy, College of Chemistry and Molecular Engineering, Peking University, Beijing, 100871, China
| | - Siwei Huang
- Beijing National Laboratory for Molecular Sciences, Center for Integrated Spectroscopy, College of Chemistry and Molecular Engineering, Peking University, Beijing, 100871, China
| | - Yue-E Zhang
- Songshan Lake Materials Laboratory, Dongguan, 523808, China
- College of Physics, Liaoning University, Shenyang, 110036, China
| | - Hai-Bo Ke
- Songshan Lake Materials Laboratory, Dongguan, 523808, China.
| | - Wei-Hua Wang
- Songshan Lake Materials Laboratory, Dongguan, 523808, China
- Institute of Physics, Chinese Academy of Sciences, Beijing, 100190, China
| | - Jihan Zhou
- Beijing National Laboratory for Molecular Sciences, Center for Integrated Spectroscopy, College of Chemistry and Molecular Engineering, Peking University, Beijing, 100871, China.
| |
Collapse
|
13
|
Abbasi N, Chen K, Wong A, Bizheva K. Computational approach for correcting defocus and suppressing speckle noise in line-field optical coherence tomography images. BIOMEDICAL OPTICS EXPRESS 2024; 15:5491-5504. [PMID: 39296416 PMCID: PMC11407272 DOI: 10.1364/boe.530569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/20/2024] [Revised: 07/12/2024] [Accepted: 08/14/2024] [Indexed: 09/21/2024]
Abstract
The trade-off between transverse resolution and depth-of-focus (DOF) typical for optical coherence tomography (OCT) systems based on conventional optics, prevents "single-shot" acquisition of volumetric OCT images with sustained high transverse resolution over the entire imaging depth. Computational approaches for correcting defocus and higher order aberrations in OCT images developed in the past require highly stable phase data, which poses a significant technological challenge. Here, we present an alternative computational approach to sharpening OCT images and reducing speckle noise, based on intensity OCT data. The novel algorithm uses non-local priors to model correlated speckle noise within a maximum a posteriori framework to generate sharp and noise-free images. The performance of the algorithm was tested on images of plant tissue (cucumber) and in-vivo healthy human cornea, acquired with line-field spectral domain OCT (LF-SD-OCT) systems. The novel algorithm effectively suppressed speckle noise and sharpened or recovered morphological features in the OCT images for depths up to 13×DOF (depth-of-focus) relative to the focal plane.
Collapse
Affiliation(s)
- Nima Abbasi
- Department of Systems Design Engineering, University of Waterloo, Waterloo, Ontario, Canada
| | - Keyu Chen
- Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario, Canada
| | - Alexander Wong
- Department of Systems Design Engineering, University of Waterloo, Waterloo, Ontario, Canada
| | - Kostadinka Bizheva
- Department of Systems Design Engineering, University of Waterloo, Waterloo, Ontario, Canada
- Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario, Canada
- School of Optometry and Vision Sciences, University of Waterloo, Waterloo, Ontario, Canada
| |
Collapse
|
14
|
Shim J, Kang SH, Lee Y. Utility of block-matching and 3D filter for reproducibility of lung density and denoising in low-dose chest CT: A pilot study. Phys Med 2024; 124:103432. [PMID: 38996628 DOI: 10.1016/j.ejmp.2024.103432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Revised: 07/02/2024] [Accepted: 07/08/2024] [Indexed: 07/14/2024] Open
Abstract
PURPOSE This study aimed to acquire an image quality consistent with that of full-dose chest computed tomography (CT) when obtaining low-dose chest CT images and to analyze the effects of block-matching and 3D (BM3D) filters on lung density measurements and noise reduction in lung parenchyma. METHODS Using full-dose chest CT images, we evaluated lung density measurements and noise reduction in lung parenchyma images for low-dose chest CT. Three filters (median, Wiener, and the proposed BM3D) were applied to low-dose chest CT images for comparison and analysis with images from full-dose chest CT. To evaluate lung density measurements, we measured CT attenuation at the 15th percentile of the lung CT histogram. The coefficient of variation (COV) and contrast-to-noise ratio (CNR) were used to evaluate the noise level. RESULTS The 15th percentile of the lung CT histogram showed the smallest difference between full- and low-dose CT when applying the BM3D filter, and the highest difference between full- and low-dose CT without filters (full-dose = - 926.28 ± 0.32, BM3D = - 926.65 ± 0.32, and low-dose = - 959.43 ± 0.95) (p < 0.05). The COV was smallest when applying the BM3D filter, whereas the CNR was the highest (p < 0.05). CONCLUSIONS The results of the study prove that the BM3D filter can reduce image noise while increasing the reproducibility of the lung density, even for low-dose chest CT.
Collapse
Affiliation(s)
- Jina Shim
- Department of Diagnostic Radiology, Severance Hospital, Seoul, Republic of Korea
| | - Seong-Hyeon Kang
- Department of Radiological Science, Gachon University, Incheon, Republic of Korea.
| | - Youngjin Lee
- Department of Radiological Science, Gachon University, Incheon, Republic of Korea.
| |
Collapse
|
15
|
Laurenzis M, Bacher E. Fourier analysis in single photon imaging. OPTICS EXPRESS 2024; 32:26525-26536. [PMID: 39538517 DOI: 10.1364/oe.522742] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 06/29/2024] [Indexed: 11/16/2024]
Abstract
Single photon imaging has become an established sensing approach. Compared to intensity imaging, versatile advantages have been demonstrated, such as imaging with high sensitivity, at a high frame rate, and with a high dynamic range. In this paper, we investigate the Fourier analysis of single photon counting at a frame rate of approximately 100 kHz and a high spatial resolution of 512 px × 512 px. We observed signal modulation in raw data as well as in data converted to photon flux, but with the data processing, the signal's frequency response is affected by significant damping. Thus, analysis sensible to signal frequency should work on the raw single photon counting signal. Furthermore, imaging of magnitude and phase in the Fourier domain can visualize areas of certain signal modulation, and the gradient of the phase angle can reveal the direction of movements. Finally, we have applied our method to real world scenarios by analyzing unmanned aerial vehicle's motion in outdoor experiments.
Collapse
|
16
|
Hata S, Ihara S, Saito H, Murayama M. In-situ heating-and-electron tomography for materials research: from 3D (in-situ 2D) to 4D (in-situ 3D). Microscopy (Oxf) 2024; 73:133-144. [PMID: 38462986 PMCID: PMC11000667 DOI: 10.1093/jmicro/dfae008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 12/06/2023] [Accepted: 02/07/2024] [Indexed: 03/12/2024] Open
Abstract
In-situ observation has expanded the application of transmission electron microscopy (TEM) and has made a significant contribution to materials research and development for energy, biomedical, quantum, etc. Recent technological developments related to in-situ TEM have empowered the incorporation of three-dimensional observation, which was previously considered incompatible. In this review article, we take up heating as the most commonly used external stimulus for in-situ TEM observation and overview recent in-situ TEM studies. Then, we focus on the electron tomography (ET) and in-situ heating combined observation by introducing the authors' recent research as an example. Assuming that in-situ heating observation is expanded from two dimensions to three dimensions using a conventional TEM apparatus and a commercially available in-situ heating specimen holder, the following in-situ heating-and-ET observation procedure is proposed: (i) use a rapid heating-and-cooling function of a micro-electro-mechanical system holder; (ii) heat and cool the specimen intermittently and (iii) acquire a tilt-series dataset when the specimen heating is stopped. This procedure is not too technically challenging and can have a wide range of applications. Essential technical points for a successful 4D (space and time) observation will be discussed through reviewing the authors' example application.
Collapse
Affiliation(s)
- Satoshi Hata
- Faculty of Engineering Sciences, Kyushu University, 6-1 Kasugakoen, Kasuga, Fukuoka 816-8580, Japan
- The Ultramicroscopy Research Center, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka 819-0395, Japan
| | - Shiro Ihara
- Institute for Materials Chemistry and Engineering, Kyushu University, 6-1 Kasugakoen, Kasuga, Fukuoka 816-8580, Japan
| | - Hikaru Saito
- Institute for Materials Chemistry and Engineering, Kyushu University, 6-1 Kasugakoen, Kasuga, Fukuoka 816-8580, Japan
- Pan-Omics Data-Driven Research Innovation Center, Kyushu University, 6-1 Kasugakoen, Kasuga, Fukuoka 816-8580, Japan
| | - Mitsuhiro Murayama
- Institute for Materials Chemistry and Engineering, Kyushu University, 6-1 Kasugakoen, Kasuga, Fukuoka 816-8580, Japan
- Department of Materials Science and Engineering, Virginia Tech, 445 Old Turner St., Blacksburg, VA 24060, USA
- Reactor Materials and Mechanical Design Group, Energy and Environmental Directorate, Pacific Northwest National Laboratory, PO Box 999, Richland, WA 99352, USA
| |
Collapse
|
17
|
Bousse A, Kandarpa VSS, Rit S, Perelli A, Li M, Wang G, Zhou J, Wang G. Systematic Review on Learning-based Spectral CT. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2024; 8:113-137. [PMID: 38476981 PMCID: PMC10927029 DOI: 10.1109/trpms.2023.3314131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2024]
Abstract
Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.
Collapse
Affiliation(s)
- Alexandre Bousse
- LaTIM, Inserm UMR 1101, Université de Bretagne Occidentale, 29238 Brest, France
| | | | - Simon Rit
- Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Étienne, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69373, Lyon, France
| | - Alessandro Perelli
- Department of Biomedical Engineering, School of Science and Engineering, University of Dundee, DD1 4HN, UK
| | - Mengzhou Li
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA
| | - Guobao Wang
- Department of Radiology, University of California Davis Health, Sacramento, USA
| | - Jian Zhou
- CTIQ, Canon Medical Research USA, Inc., Vernon Hills, 60061, USA
| | - Ge Wang
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA
| |
Collapse
|
18
|
Piazza F, Sacco P, Marsich E, Baj G, Brun F, Asaro F, Grassi G, Grassi M, Donati I. Cell Activities on Viscoelastic Substrates Show an Elastic Energy Threshold and Correlate with the Linear Elastic Energy Loss in the Strain‐Softening Region. ADVANCED FUNCTIONAL MATERIALS 2023; 33. [DOI: 10.1002/adfm.202307224] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Indexed: 02/06/2025]
Abstract
Energy‐sensing in viscoelastic substrates has recently been shown to be an important regulator of cellular activities, modulating mechanical transmission and transduction processes. Here, this study fine‐tunes the elastic energy of viscoelastic hydrogels with different physical and chemical compositions and shows that this has an impact on cell response in 2D cell cultures. This study shows that there is a threshold value for elastic energy (≈0.15 J m−3) above which cell adhesion is impaired. When hydrogels leave the linear stress–strain range, they show softening (plastic) behavior typical of soft tissues. This study identifies a correlation between the theoretical linear elastic energy loss in the strain‐softening region and the number of cells adhering to the substrate. This also has implications for the formation of vinculin‐rich anchorage points and the ability of cells to remodel the substrate through traction forces. Overall, the results reported in this study support that the relationship between cell activities and energy‐sensing in viscoelastic substrates is an important aspect to consider in the development of reliable ex vivo models of human tissues that mimic both normal and pathological conditions.
Collapse
Affiliation(s)
- Francesco Piazza
- Department of Life Sciences University of Trieste Via Licio Giorgieri 5 I‐34127 Trieste Italy
| | - Pasquale Sacco
- Department of Life Sciences University of Trieste Via Licio Giorgieri 5 I‐34127 Trieste Italy
| | - Eleonora Marsich
- Department of Medicine Surgery and Health Sciences University of Trieste Piazza dell'Ospitale 1 I‐34129 Trieste Italy
| | - Gabriele Baj
- Department of Life Sciences University of Trieste Via Licio Giorgieri 5 I‐34127 Trieste Italy
| | - Francesco Brun
- Department of Engineering and Architecture University of Trieste Via A. Valerio 6/1 I‐34127 Trieste Italy
| | - Fioretta Asaro
- Department of Chemical and Pharmaceutical Sciences University of Trieste Via Licio Giorgieri 1 I‐34127 Trieste Italy
| | - Gabriele Grassi
- Department of Life Sciences University of Trieste Via Licio Giorgieri 5 I‐34127 Trieste Italy
| | - Mario Grassi
- Department of Engineering and Architecture University of Trieste Via A. Valerio 6/1 I‐34127 Trieste Italy
| | - Ivan Donati
- Department of Life Sciences University of Trieste Via Licio Giorgieri 5 I‐34127 Trieste Italy
| |
Collapse
|
19
|
Hellström M, Löfstedt T, Garpebring A. Denoising and uncertainty estimation in parameter mapping with approximate Bayesian deep image priors. Magn Reson Med 2023; 90:2557-2571. [PMID: 37582257 DOI: 10.1002/mrm.29823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Revised: 06/26/2023] [Accepted: 07/18/2023] [Indexed: 08/17/2023]
Abstract
PURPOSE To mitigate the problem of noisy parameter maps with high uncertainties by casting parameter mapping as a denoising task based on Deep Image Priors. METHODS We extend the concept of denoising with Deep Image Prior (DIP) into parameter mapping by treating the output of an image-generating network as a parametrization of tissue parameter maps. The method implicitly denoises the parameter mapping process by filtering low-level image features with an untrained convolutional neural network (CNN). Our implementation includes uncertainty estimation from Bernoulli approximate variational inference, implemented with MC dropout, which provides model uncertainty in each voxel of the denoised parameter maps. The method is modular, so the specifics of different applications (e.g., T1 mapping) separate into application-specific signal equation blocks. We evaluate the method on variable flip angle T1 mapping, multi-echo T2 mapping, and apparent diffusion coefficient mapping. RESULTS We found that deep image prior adapts successfully to several applications in parameter mapping. In all evaluations, the method produces noise-reduced parameter maps with decreased uncertainty compared to conventional methods. The downsides of the proposed method are the long computational time and the introduction of some bias from the denoising prior. CONCLUSION DIP successfully denoise the parameter mapping process and applies to several applications with limited hyperparameter tuning. Further, it is easy to implement since DIP methods do not use network training data. Although time-consuming, uncertainty information from MC dropout makes the method more robust and provides useful information when properly calibrated.
Collapse
Affiliation(s)
- Max Hellström
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| | - Tommy Löfstedt
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
- Department of Computing Science, Umeå University, Umeå, Sweden
| | | |
Collapse
|
20
|
Scattarella F, Diacono D, Monaco A, Amoroso N, Bellantuono L, Massaro G, Pepe FV, Tangaro S, Bellotti R, D'Angelo M. Deep learning approach for denoising low-SNR correlation plenoptic images. Sci Rep 2023; 13:19645. [PMID: 37950034 PMCID: PMC10638444 DOI: 10.1038/s41598-023-46765-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Accepted: 11/04/2023] [Indexed: 11/12/2023] Open
Abstract
Correlation Plenoptic Imaging (CPI) is a novel volumetric imaging technique that uses two sensors and the spatio-temporal correlations of light to detect both the spatial distribution and the direction of light. This novel approach to plenoptic imaging enables refocusing and 3D imaging with significant enhancement of both resolution and depth of field. However, CPI is generally slower than conventional approaches due to the need to acquire sufficient statistics for measuring correlations with an acceptable signal-to-noise ratio (SNR). We address this issue by implementing a Deep Learning application to improve image quality with undersampled frame statistics. We employ a set of experimental images reconstructed by a standard CPI architecture, at three different sampling ratios, and use it to feed a CNN model pre-trained through the transfer learning paradigm U-Net architecture with VGG-19 net for the encoding part. We find that our model reaches a Structural Similarity (SSIM) index value close to 1 both for the test sample (SSIM = [Formula: see text]) and in 5-fold cross validation (SSIM = [Formula: see text]); the results are also shown to outperform classic denoising methods, in particular for images with lower SNR. The proposed work represents the first application of Artificial Intelligence in the field of CPI and demonstrates its high potential: speeding-up the acquisition by a factor 20 over the fastest CPI so far demonstrated, enabling recording potentially 200 volumetric images per second. The presented results open the way to scanning-free real-time volumetric imaging at video rate, which is expected to achieve a substantial influence in various applications scenarios, from monitoring neuronal activity to machine vision and security.
Collapse
Affiliation(s)
- Francesco Scattarella
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| | - Domenico Diacono
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| | - Alfonso Monaco
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy.
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy.
| | - Nicola Amoroso
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
- Dipartimento di Farmacia - Scienze del Farmaco, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
| | - Loredana Bellantuono
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
- Dipartimento di Biomedicina Traslazionale e Neuroscienze (DiBraiN), Università degli Studi di Bari Aldo Moro, 70124, Bari, Italy
| | - Gianlorenzo Massaro
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| | - Francesco V Pepe
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| | - Sabina Tangaro
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
- Dipartimento di Scienze del Suolo, della Pianta e degli Alimenti, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
| | - Roberto Bellotti
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| | - Milena D'Angelo
- Dipartimento Interateneo di Fisica M. Merlin, Università degli Studi di Bari Aldo Moro, 70125, Bari, Italy
- Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Bari, 70125, Bari, Italy
| |
Collapse
|
21
|
Radke KL, Kamp B, Adriaenssens V, Stabinska J, Gallinnis P, Wittsack HJ, Antoch G, Müller-Lutz A. Deep Learning-Based Denoising of CEST MR Data: A Feasibility Study on Applying Synthetic Phantoms in Medical Imaging. Diagnostics (Basel) 2023; 13:3326. [PMID: 37958222 PMCID: PMC10650582 DOI: 10.3390/diagnostics13213326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 10/18/2023] [Accepted: 10/25/2023] [Indexed: 11/15/2023] Open
Abstract
Chemical Exchange Saturation Transfer (CEST) magnetic resonance imaging (MRI) provides a novel method for analyzing biomolecule concentrations in tissues without exogenous contrast agents. Despite its potential, achieving a high signal-to-noise ratio (SNR) is imperative for detecting small CEST effects. Traditional metrics such as Magnetization Transfer Ratio Asymmetry (MTRasym) and Lorentzian analyses are vulnerable to image noise, hampering their precision in quantitative concentration estimations. Recent noise-reduction algorithms like principal component analysis (PCA), nonlocal mean filtering (NLM), and block matching combined with 3D filtering (BM3D) have shown promise, as there is a burgeoning interest in the utilization of neural networks (NNs), particularly autoencoders, for imaging denoising. This study uses the Bloch-McConnell equations, which allow for the synthetic generation of CEST images and explores NNs efficacy in denoising these images. Using synthetically generated phantoms, autoencoders were created, and their performance was compared with traditional denoising methods using various datasets. The results underscored the superior performance of NNs, notably the ResUNet architectures, in noise identification and abatement compared to analytical approaches across a wide noise gamut. This superiority was particularly pronounced at elevated noise intensities in the in vitro data. Notably, the neural architectures significantly improved the PSNR values, achieving up to 35.0, while some traditional methods struggled, especially in low-noise reduction scenarios. However, the application to the in vivo data presented challenges due to varying noise profiles. This study accentuates the potential of NNs as robust denoising tools, but their translation to clinical settings warrants further investigation.
Collapse
Affiliation(s)
- Karl Ludger Radke
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Benedikt Kamp
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Vibhu Adriaenssens
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Julia Stabinska
- F.M. Kirby Research Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore, MD 21205, USA
- Division of MR Research, The Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA
| | - Patrik Gallinnis
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Hans-Jörg Wittsack
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Gerald Antoch
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| | - Anja Müller-Lutz
- Department of Diagnostic and Interventional Radiology, Medical Faculty, University Dusseldorf, 40225 Dusseldorf, Germany (G.A.); (A.M.-L.)
| |
Collapse
|
22
|
Simkó A, Ruiter S, Löfstedt T, Garpebring A, Nyholm T, Bylund M, Jonsson J. Improving MR image quality with a multi-task model, using convolutional losses. BMC Med Imaging 2023; 23:148. [PMID: 37784039 PMCID: PMC10544274 DOI: 10.1186/s12880-023-01109-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Accepted: 09/25/2023] [Indexed: 10/04/2023] Open
Abstract
PURPOSE During the acquisition of MRI data, patient-, sequence-, or hardware-related factors can introduce artefacts that degrade image quality. Four of the most significant tasks for improving MRI image quality have been bias field correction, super-resolution, motion-, and noise correction. Machine learning has achieved outstanding results in improving MR image quality for these tasks individually, yet multi-task methods are rarely explored. METHODS In this study, we developed a model to simultaneously correct for all four aforementioned artefacts using multi-task learning. Two different datasets were collected, one consisting of brain scans while the other pelvic scans, which were used to train separate models, implementing their corresponding artefact augmentations. Additionally, we explored a novel loss function that does not only aim to reconstruct the individual pixel values, but also the image gradients, to produce sharper, more realistic results. The difference between the evaluated methods was tested for significance using a Friedman test of equivalence followed by a Nemenyi post-hoc test. RESULTS Our proposed model generally outperformed other commonly-used correction methods for individual artefacts, consistently achieving equal or superior results in at least one of the evaluation metrics. For images with multiple simultaneous artefacts, we show that the performance of using a combination of models, trained to correct individual artefacts depends heavily on the order that they were applied. This is not an issue for our proposed multi-task model. The model trained using our novel convolutional loss function always outperformed the model trained with a mean squared error loss, when evaluated using Visual Information Fidelity, a quality metric connected to perceptual quality. CONCLUSION We trained two models for multi-task MRI artefact correction of brain, and pelvic scans. We used a novel loss function that significantly improves the image quality of the outputs over using mean squared error. The approach performs well on real world data, and it provides insight into which artefacts it detects and corrects for. Our proposed model and source code were made publicly available.
Collapse
Affiliation(s)
- Attila Simkó
- Department of Radiation Sciences, Umeå University, Umeå, Sweden.
| | - Simone Ruiter
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Tommy Löfstedt
- Department of Computing Science, Umeå University, Umeå, Sweden
| | | | - Tufve Nyholm
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| | - Mikael Bylund
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| | - Joakim Jonsson
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| |
Collapse
|
23
|
Marelli F, Liebling M. Efficient compressed sensing reconstruction for 3D fluorescence microscopy using OptoMechanical Modulation Tomography (OMMT) with a 1+2D regularization. OPTICS EXPRESS 2023; 31:31718-31733. [PMID: 37858990 DOI: 10.1364/oe.493611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 08/14/2023] [Indexed: 10/21/2023]
Abstract
OptoMechanical Modulation Tomography (OMMT) exploits compressed sensing to reconstruct high resolution microscopy volumes from fewer measurement images compared to exhaustive section sampling in conventional light sheet microscopy. Nevertheless, the volumetric reconstruction process is computationally expensive, making it impractically slow to use on large-size images, and prone to generating visual artefacts. Here, we propose a reconstruction approach that uses a 1+2D Total Variation (TV1+2) regularization that does not generate such artefacts and is amenable to efficient implementation using parallel computing. We evaluate our method for accuracy and scaleability on simulated and experimental data. Using a high quality, but computationally expensive, Plug-and-Play (PnP) method that uses the BM4D denoiser as a benchmark, we observe that our approach offers an advantageous trade-off between speed and accuracy.
Collapse
|
24
|
Aboumerhi K, Güemes A, Liu H, Tenore F, Etienne-Cummings R. Neuromorphic applications in medicine. J Neural Eng 2023; 20:041004. [PMID: 37531951 DOI: 10.1088/1741-2552/aceca3] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 08/02/2023] [Indexed: 08/04/2023]
Abstract
In recent years, there has been a growing demand for miniaturization, low power consumption, quick treatments, and non-invasive clinical strategies in the healthcare industry. To meet these demands, healthcare professionals are seeking new technological paradigms that can improve diagnostic accuracy while ensuring patient compliance. Neuromorphic engineering, which uses neural models in hardware and software to replicate brain-like behaviors, can help usher in a new era of medicine by delivering low power, low latency, small footprint, and high bandwidth solutions. This paper provides an overview of recent neuromorphic advancements in medicine, including medical imaging and cancer diagnosis, processing of biosignals for diagnosis, and biomedical interfaces, such as motor, cognitive, and perception prostheses. For each section, we provide examples of how brain-inspired models can successfully compete with conventional artificial intelligence algorithms, demonstrating the potential of neuromorphic engineering to meet demands and improve patient outcomes. Lastly, we discuss current struggles in fitting neuromorphic hardware with non-neuromorphic technologies and propose potential solutions for future bottlenecks in hardware compatibility.
Collapse
Affiliation(s)
- Khaled Aboumerhi
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, United States of America
| | - Amparo Güemes
- Electrical Engineering Division, Department of Engineering, University of Cambridge, 9 JJ Thomson Ave, Cambridge CB3 0FA, United Kingdom
| | - Hongtao Liu
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, United States of America
| | - Francesco Tenore
- Research and Exploratory Development Department, The Johns Hopkins University Applied Physics Laboratory, Laurel, MD, United States of America
| | - Ralph Etienne-Cummings
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, United States of America
| |
Collapse
|
25
|
Wirth-Singh A, Fröch JE, Han Z, Huang L, Mukherjee S, Zhou Z, Coppens Z, Böhringer KF, Majumdar A. Large field-of-view thermal imaging via all-silicon meta-optics. APPLIED OPTICS 2023; 62:5467-5474. [PMID: 37706864 DOI: 10.1364/ao.493555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 06/21/2023] [Indexed: 09/15/2023]
Abstract
A broad range of imaging and sensing technologies in the infrared require large field-of-view (FoV) operation. To achieve this, traditional refractive systems often employ multiple elements to compensate for aberrations, which leads to excess size, weight, and cost. For many applications, including night vision eye-wear, air-borne surveillance, and autonomous navigation for unmanned aerial vehicles, size and weight are highly constrained. Sub-wavelength diffractive optics, also known as meta-optics, can dramatically reduce the size, weight, and cost of these imaging systems, as meta-optics are significantly thinner and lighter than traditional refractive lenses. Here, we demonstrate 80° FoV thermal imaging in the long-wavelength infrared regime (8-12 µm) using an all-silicon meta-optic with an entrance aperture and lens focal length of 1 cm.
Collapse
|
26
|
Girod R, Lazaridis T, Gasteiger HA, Tileli V. Three-dimensional nanoimaging of fuel cell catalyst layers. Nat Catal 2023; 6:383-391. [PMID: 37252670 PMCID: PMC10212762 DOI: 10.1038/s41929-023-00947-y] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 03/14/2023] [Indexed: 05/31/2023]
Abstract
Catalyst layers in proton exchange membrane fuel cells consist of platinum-group-metal nanocatalysts supported on carbon aggregates, forming a porous structure through which an ionomer network percolates. The local structural character of these heterogeneous assemblies is directly linked to the mass-transport resistances and subsequent cell performance losses; its three-dimensional visualization is therefore of interest. Herein we implement deep-learning-aided cryogenic transmission electron tomography for image restoration, and we quantitatively investigate the full morphology of various catalyst layers at the local-reaction-site scale. The analysis enables computation of metrics such as the ionomer morphology, coverage and homogeneity, location of platinum on the carbon supports, and platinum accessibility to the ionomer network, with the results directly compared and validated with experimental measurements. We expect that our findings and methodology for evaluating catalyst layer architectures will contribute towards linking the morphology to transport properties and overall fuel cell performance.
Collapse
Affiliation(s)
- Robin Girod
- Institute of Materials, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Timon Lazaridis
- Chair of Technical Electrochemistry, Department of Chemistry and Catalysis Research Center, Technische Universität München, Garching, Germany
| | - Hubert A. Gasteiger
- Chair of Technical Electrochemistry, Department of Chemistry and Catalysis Research Center, Technische Universität München, Garching, Germany
| | - Vasiliki Tileli
- Institute of Materials, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
27
|
Karakis R, Gurkahraman K, Mitsis GD, Boudrias MH. DEEP LEARNING PREDICTION OF MOTOR PERFORMANCE IN STROKE INDIVIDUALS USING NEUROIMAGING DATA. J Biomed Inform 2023; 141:104357. [PMID: 37031755 DOI: 10.1016/j.jbi.2023.104357] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 02/24/2023] [Accepted: 04/01/2023] [Indexed: 04/11/2023]
Abstract
The degree of motor impairment and profile of recovery after stroke are difficult to predict for each individual. Measures obtained from clinical assessments, as well as neurophysiological and neuroimaging techniques have been used as potential biomarkers of motor recovery, with limited accuracy up to date. To address this, the present study aimed to develop a deep learning model based on structural brain images obtained from stroke participants and healthy volunteers. The following inputs were used in a multi-channel 3D convolutional neural network (CNN) model: fractional anisotropy, mean diffusivity, radial diffusivity, and axial diffusivity maps obtained from Diffusion Tensor Imaging (DTI) images, white and gray matter intensity values obtained from Magnetic Resonance Imaging, as well as demographic data (e.g., age, gender). Upper limb motor function was classified into "Poor" and "Good" categories. To assess the performance of the DL model, we compared it to more standard machine learning (ML) classifiers including k-nearest neighbor, support vector machines (SVM), Decision Trees, Random Forests, Ada Boosting, and Naïve Bayes, whereby the inputs of these classifiers were the features taken from the fully connected layer of the CNN model. The highest accuracy and area under the curve values were 0.92 and 0.92 for the 3D-CNN and 0.91 and 0.91 for the SVM, respectively. The multi-channel 3D-CNN with residual blocks and SVM supported by DL was more accurate than traditional ML methods to classify upper limb motor impairment in the stroke population. These results suggest that combining volumetric DTI maps and measures of white and gray matter integrity can improve the prediction of the degree of motor impairment after stroke. Identifying the potential of recovery early on after a stroke could promote the allocation of resources to optimize the functional independence of these individuals and their quality of life.
Collapse
Affiliation(s)
- Rukiye Karakis
- Department of Software Engineering, Faculty of Technology, Sivas Cumhuriyet University, Turkey
| | - Kali Gurkahraman
- Department of Computer Engineering, Faculty of Engineering, Sivas Cumhuriyet University, Turkey
| | - Georgios D Mitsis
- Department of Bioengineering, Faculty of Engineering, McGill University, Montreal, QC, Canada
| | - Marie-Hélène Boudrias
- School of Physical and Occupational Therapy, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada; BRAIN Laboratory, Jewish Rehabilitation Hospital, Site of Centre for Interdisciplinary Research of Greater Montreal (CRIR) and CISSS-Laval, QC, Canada.
| |
Collapse
|
28
|
Zhao Z, Li X. Image blending-based noise synthesis and attention-guided network for single image marine snow denoising. INT J MACH LEARN CYB 2023. [DOI: 10.1007/s13042-022-01756-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
29
|
Deep learning-based noise filtering toward millisecond order imaging by using scanning transmission electron microscopy. Sci Rep 2022; 12:13462. [PMID: 35931705 PMCID: PMC9356044 DOI: 10.1038/s41598-022-17360-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Accepted: 07/25/2022] [Indexed: 11/09/2022] Open
Abstract
Application of scanning transmission electron microscopy (STEM) to in situ observation will be essential in the current and emerging data-driven materials science by taking STEM's high affinity with various analytical options into account. As is well known, STEM's image acquisition time needs to be further shortened to capture a targeted phenomenon in real-time as STEM's current temporal resolution is far below the conventional TEM's. However, rapid image acquisition in the millisecond per frame or faster generally causes image distortion, poor electron signals, and unidirectional blurring, which are obstacles for realizing video-rate STEM observation. Here we show an image correction framework integrating deep learning (DL)-based denoising and image distortion correction schemes optimized for STEM rapid image acquisition. By comparing a series of distortion corrected rapid scan images with corresponding regular scan speed images, the trained DL network is shown to remove not only the statistical noise but also the unidirectional blurring. This result demonstrates that rapid as well as high-quality image acquisition by STEM without hardware modification can be established by the DL. The DL-based noise filter could be applied to in-situ observation, such as dislocation activities under external stimuli, with high spatio-temporal resolution.
Collapse
|
30
|
Marcos L, Quint F, Babyn P, Alirezaie J. Dilated Convolution ResNet with Boosting Attention Modules and Combined Loss Functions for LDCT Image Denoising. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:1548-1551. [PMID: 36086586 DOI: 10.1109/embc48229.2022.9870993] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
With the increasing concern regarding the radiation exposure of patients undergoing computed tomography (CT) scans, researchers have been using deep learning techniques to improve the quality of denoised low-dose CT (LDCT) images. In this paper, a cascaded dilated residual network (ResNet) with integrated attention modules, specifically spatial- and channel- attention modules, is proposed. This experiment demonstrated how these attention modules improved the denoised CT image by testing a simple ResNet with and without the modules. Further, an investigation regarding the effectiveness of per-pixel loss, perceptual loss via VGG16-Net, and structural dissimilarity loss functions is also covered through an ablation experiment. By knowing how these loss functions affect the output denoised images, a combination of the these loss function is then proposed which aims to prevent edge over-smoothing, enhance textural details and finally, preserve structural details on the denoised images. Finally, a bench testing was also done by comparing the visual and quantitative results of the proposed model with the state-of-the-art models such as block matching 3D (BM3D), patch-GAN and dilated convolution with edge detection layer (DRL-E-MP) for accuracy.
Collapse
|
31
|
Cabral D, Fradinho AC, Pereira T, Ramakrishnan MS, Bacci T, An D, Tenreiro S, Seabra MC, Balaratnasingam C, Freund KB. Macular Vascular Imaging and Connectivity Analysis Using High-Resolution Optical Coherence Tomography. Transl Vis Sci Technol 2022; 11:2. [PMID: 35648637 PMCID: PMC9172017 DOI: 10.1167/tvst.11.6.2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 05/06/2022] [Indexed: 11/24/2022] Open
Abstract
Purpose To characterize macular blood flow connectivity in vivo using high-resolution optical coherence tomography (HighRes OCT). Methods Cross-sectional, observational study. Dense (6-µm interscan distance) perifoveal HighRes OCT raster scans were performed on healthy participants. To mitigate the limitations of projection-resolved OCT-angiography, flow and structural data were used to observe the vascular structures of the superficial vascular complex (SVC) and the deep vascular complex. Vascular segmentation and rendering were performed using Imaris 9.5 software. Inflow and outflow patterns were classified according to vascular diameter and branching order from superficial arteries and veins, respectively. Results Eight eyes from eight participants were included in this analysis, from which 422 inflow and 459 outflow connections were characterized. Arteries had direct arteriolar connections to the SVC (78%) and to the intermediate capillary plexus (ICP, 22%). Deep capillary plexus (DCP) inflow derived from small-diameter vessels succeeding ICP arterioles. The most prevalent outflow pathways coursed through superficial draining venules (74%). DCP draining venules ordinarily merged with ICP draining venules and drained independently of superficial venules in 21% of cases. The morphology of DCP draining venules in structural HighRes OCT is distinct from other vessels crossing the inner nuclear layer and can be used to identify superficial veins. Conclusions Vascular connectivity analysis supports a hybrid circuitry of blood flow within the human parafoveal macula. Translational Relevance Characterization of parafoveal macular blood flow connectivity in vivo using a precise segmentation of HighRes OCT is consistent with ground-truth microscopy studies and shows a hybrid circuitry.
Collapse
Affiliation(s)
- Diogo Cabral
- Vitreous Retina Macula Consultants of New York, NY, USA
- CEDOC, NOVA Medical School I Faculdade de Ciências Médicas, Universidade NOVA de Lisboa, Lisbon, Portugal
| | - Ana C. Fradinho
- CEDOC, NOVA Medical School I Faculdade de Ciências Médicas, Universidade NOVA de Lisboa, Lisbon, Portugal
| | - Telmo Pereira
- CEDOC, NOVA Medical School I Faculdade de Ciências Médicas, Universidade NOVA de Lisboa, Lisbon, Portugal
| | | | - Tommaso Bacci
- Vitreous Retina Macula Consultants of New York, NY, USA
| | - Dong An
- Centre for Ophthalmology and Visual Science, University of Western Australia, Nedlands, Western Australia, Australia
- Lions Eye Institute, Nedlands, Western Australia, Australia
| | - Sandra Tenreiro
- CEDOC, NOVA Medical School I Faculdade de Ciências Médicas, Universidade NOVA de Lisboa, Lisbon, Portugal
| | - Miguel C. Seabra
- CEDOC, NOVA Medical School I Faculdade de Ciências Médicas, Universidade NOVA de Lisboa, Lisbon, Portugal
- UCL Institute of Ophthalmology, London, UK
| | - Chandrakumar Balaratnasingam
- Centre for Ophthalmology and Visual Science, University of Western Australia, Nedlands, Western Australia, Australia
- Lions Eye Institute, Nedlands, Western Australia, Australia
| | - K. Bailey Freund
- Vitreous Retina Macula Consultants of New York, NY, USA
- Department of Ophthalmology, NYU Grossman School of Medicine, New York, NY, USA
| |
Collapse
|
32
|
Raayai Ardakani M, Yu L, Kaeli DR, Fang Q. Framework for denoising Monte Carlo photon transport simulations using deep learning. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-220016SSR. [PMID: 35614533 PMCID: PMC9130925 DOI: 10.1117/1.jbo.27.8.083019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 04/14/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE The Monte Carlo (MC) method is widely used as the gold-standard for modeling light propagation inside turbid media, such as human tissues, but combating its inherent stochastic noise requires one to simulate a large number photons, resulting in high computational burdens. AIM We aim to develop an effective image denoising technique using deep learning (DL) to dramatically improve the low-photon MC simulation result quality, equivalently bringing further acceleration to the MC method. APPROACH We developed a cascade-network combining DnCNN with UNet, while extending a range of established image denoising neural-network architectures, including DnCNN, UNet, DRUNet, and deep residual-learning for denoising MC renderings (ResMCNet), in handling three-dimensional MC data and compared their performances against model-based denoising algorithms. We also developed a simple yet effective approach to creating synthetic datasets that can be used to train DL-based MC denoisers. RESULTS Overall, DL-based image denoising algorithms exhibit significantly higher image quality improvements over traditional model-based denoising algorithms. Among the tested DL denoisers, our cascade network yields a 14 to 19 dB improvement in signal-to-noise ratio, which is equivalent to simulating 25 × to 78 × more photons. Other DL-based methods yielded similar results, with our method performing noticeably better with low-photon inputs and ResMCNet along with DRUNet performing better with high-photon inputs. Our cascade network achieved the highest quality when denoising complex domains, including brain and mouse atlases. CONCLUSIONS Incorporating state-of-the-art DL denoising techniques can equivalently reduce the computation time of MC simulations by one to two orders of magnitude. Our open-source MC denoising codes and data can be freely accessed at http://mcx.space/.
Collapse
Affiliation(s)
- Matin Raayai Ardakani
- Northeastern University, Department of Electrical and Computer Engineering, Boston, Massachusetts, United States
| | - Leiming Yu
- Analogic Corporation, Peabody, Massachusetts, United States
| | - David R. Kaeli
- Northeastern University, Department of Electrical and Computer Engineering, Boston, Massachusetts, United States
| | - Qianqian Fang
- Northeastern University, Department of Bioengineering, Boston, Massachusetts, United States
| |
Collapse
|
33
|
Mäkinen Y, Marchesini S, Foi A. Ring artifact and Poisson noise attenuation via volumetric multiscale nonlocal collaborative filtering of spatially correlated noise. JOURNAL OF SYNCHROTRON RADIATION 2022; 29:829-842. [PMID: 35511015 PMCID: PMC9070695 DOI: 10.1107/s1600577522002739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 03/10/2022] [Indexed: 06/14/2023]
Abstract
X-ray micro-tomography systems often suffer from high levels of noise. In particular, severe ring artifacts are common in reconstructed images, caused by defects in the detector, calibration errors, and fluctuations producing streak noise in the raw sinogram data. Furthermore, the projections commonly contain high levels of Poissonian noise arising from the photon-counting detector. This work presents a 3-D multiscale framework for streak attenuation through a purposely designed collaborative filtering of correlated noise in volumetric data. A distinct multiscale denoising step for attenuation of the Poissonian noise is further proposed. By utilizing the volumetric structure of the projection data, the proposed fully automatic procedure offers improved feature preservation compared with 2-D denoising and avoids artifacts which arise from individual filtering of sinograms.
Collapse
Affiliation(s)
| | - Stefano Marchesini
- SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, CA 94025, USA
| | | |
Collapse
|
34
|
Liang X, Wang X, Lyu L, Han Y, Zheng J, Guo W, Guo J. Noise-immune image blur detection via sequency spectrum truncation. COMPLEX INTELL SYST 2022. [DOI: 10.1007/s40747-021-00592-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
AbstractBlur detection is aimed to differentiate the blurry and sharp regions from a given image. This task has attracted much attention in recent years due to its importance in computer vision with the integration of image processing and artificial intelligence. However, blur detection still suffers from problems such as the oversensitivity to image noise and the difficulty in cost–benefit balance. To deal with these issues, we propose an accurate and efficient blur detection method, which is concise in architecture and robust against noise. First, we develop a sequency spectrum-based blur metric to estimate the blurriness of each pixel by integrating a re-blur scheme and the Walsh transform. Meanwhile, to eliminate the noise interference, we propose an adaptive sequency spectrum truncation strategy by which we can obtain an accurate blur map even in noise-polluted cases. Finally, a multi-scale fusion segmentation framework is designed to extract the blur region based on the clustering-guided region growth. Experimental results on benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance and the best balance between cost and benefit. It offers an average F1 score of 0.887, MAE of 0.101, detecting time of 0.7 s, and training time of 0.5 s. Especially for noise-polluted blurry images, the proposed method achieves the F1 score of 0.887 and MAE of 0.101, which significantly surpasses other competitive approaches. Our method yields a cost–benefit advantage and noise immunity that has great application prospect in complex sensing environment.
Collapse
|
35
|
Zeng GL. Photon Starvation Artifact Reduction by Shift-Variant Processing. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2022; 10:13633-13649. [PMID: 35993039 PMCID: PMC9390879 DOI: 10.1109/access.2022.3142775] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The x-ray computed tomography (CT) images with low dose are noisy and may contain photon starvation artifacts. The artifacts are location and direction dependent. Therefore, the common shift-invariant denoising filters do not work well. The state-of-the-art methods to process the low-dose CT images are image reconstruction based; they require the raw projection data. In many situations, the raw CT projections are not accessible. This paper suggests a method to denoise the low-dose CT image using the pseudo projections generated by the application of a forward projector on the low-dose CT image. The feasibility of the proposed method is demonstrated by real clinical data.
Collapse
Affiliation(s)
- Gengsheng L Zeng
- Department of Computer Science, Utah Valley University, Orem, UT 84058, USA
- Department of Radiology and Imaging Sciences, University of Utah, Salt Lake City, UT 84108, USA
| |
Collapse
|
36
|
Chen B, Zhang Y, Chen H, Chen W, Pan B. A New Adaptive TV-Based BM3D Algorithm for Image Denoising. ARTIF INTELL 2022. [DOI: 10.1007/978-3-031-20500-2_28] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
|
37
|
Moummad I, Jaudet C, Lechervy A, Valable S, Raboutet C, Soilihi Z, Thariat J, Falzone N, Lacroix J, Batalla A, Corroyer-Dulmont A. The Impact of Resampling and Denoising Deep Learning Algorithms on Radiomics in Brain Metastases MRI. Cancers (Basel) 2021; 14:cancers14010036. [PMID: 35008198 PMCID: PMC8750741 DOI: 10.3390/cancers14010036] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Revised: 12/14/2021] [Accepted: 12/18/2021] [Indexed: 01/06/2023] Open
Abstract
Simple Summary Due to the central role of magnetic resonance Imaging (MRI) in the management of patients with cancer, waiting lists exceed clinically relevant delays. For this reason, many research groups and MRI manufacturers develop algorithms as resampling and denoising models to allow faster acquisition time without deterioration in image quality. Whereas these algorithms are available in all new MRI, it is not clear how they will impact image features as well as the validity of statistical model of radiomics which use deep images characteristics to predict treatment outcome. The aim of this study was to develop resampling and denoising deep learning (DL) models and evaluate their impact on radiomics from post-Gd-T1w-MRI brain images with brain metastases. We show that resampling and denoising DL models reconstruct low resolution and noised MRI images acquired quickly into high quality images. While fast acquisition loses most of the radiomic-features and invalidates predictive radiomic models, DL models restore these parameters. Abstract Background: Magnetic resonance imaging (MRI) is predominant in the therapeutic management of cancer patients, unfortunately, patients have to wait a long time to get an appointment for examination. Therefore, new MRI devices include deep-learning (DL) solutions to save acquisition time. However, the impact of these algorithms on intensity and texture parameters has been poorly studied. The aim of this study was to evaluate the impact of resampling and denoising DL models on radiomics. Methods: Resampling and denoising DL model was developed on 14,243 T1 brain images from 1.5T-MRI. Radiomics were extracted from 40 brain metastases from 11 patients (2049 images). A total of 104 texture features of DL images were compared to original images with paired t-test, Pearson correlation and concordance-correlation-coefficient (CCC). Results: When two times shorter image acquisition shows strong disparities with the originals concerning the radiomics, with significant differences and loss of correlation of 79.81% and 48.08%, respectively. Interestingly, DL models restore textures with 46.15% of unstable parameters and 25.96% of low CCC and without difference for the first-order intensity parameters. Conclusions: Resampling and denoising DL models reconstruct low resolution and noised MRI images acquired quickly into high quality images. While fast MRI acquisition loses most of the radiomic features, DL models restore these parameters.
Collapse
Affiliation(s)
- Ilyass Moummad
- Medical Physics Department, CLCC François Baclesse, 14000 Caen, France; (I.M.); (C.J.); (Z.S.); (A.B.)
| | - Cyril Jaudet
- Medical Physics Department, CLCC François Baclesse, 14000 Caen, France; (I.M.); (C.J.); (Z.S.); (A.B.)
| | - Alexis Lechervy
- UMR GREYC, Normandie University, UNICAEN, ENSICAEN, CNRS, 14000 Caen, France;
| | - Samuel Valable
- ISTCT/CERVOxy Group, Normandie University, UNICAEN, CEA, CNRS, 14000 Caen, France;
| | - Charlotte Raboutet
- Radiology Department, CLCC François Baclesse, 14000 Caen, France; (C.R.); (J.L.)
| | - Zamila Soilihi
- Medical Physics Department, CLCC François Baclesse, 14000 Caen, France; (I.M.); (C.J.); (Z.S.); (A.B.)
| | - Juliette Thariat
- Radiotherapy Department, CLCC François Baclesse, 14000 Caen, France;
| | - Nadia Falzone
- GenesisCare Theranostics, Building 1 & 11, The Mill, 41-43 Bourke Road, Alexandria, NSW 2015, Australia;
| | - Joëlle Lacroix
- Radiology Department, CLCC François Baclesse, 14000 Caen, France; (C.R.); (J.L.)
| | - Alain Batalla
- Medical Physics Department, CLCC François Baclesse, 14000 Caen, France; (I.M.); (C.J.); (Z.S.); (A.B.)
| | - Aurélien Corroyer-Dulmont
- Medical Physics Department, CLCC François Baclesse, 14000 Caen, France; (I.M.); (C.J.); (Z.S.); (A.B.)
- ISTCT/CERVOxy Group, Normandie University, UNICAEN, CEA, CNRS, 14000 Caen, France;
- Correspondence:
| |
Collapse
|
38
|
Marcos L, Alirezaie J, Babyn P. Low Dose CT Image Denoising Using Boosting Attention Fusion GAN with Perceptual Loss. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3407-3410. [PMID: 34891971 DOI: 10.1109/embc46164.2021.9630790] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Image denoising of Low-dose computed tomography (LDCT) images has continues to receive attention in the research community due to ongoing concerns about high-dose radiation exposure of patients for diagnosis. The use of low radiation CT image, however, could lead to inaccurate diagnosis due to the presence of noise. Deep learning techniques are being integrated into denoising methods to address this problem. In this paper, a General Adversarial Network (GAN) composed of boosting fusion of spatial and channel attention modules is proposed. These modules are embedded in the denoiser to address the limitations of other GAN-based denoising models that tend to only focus on the local processing and neglect the dependencies of creating feature maps with spatial- and channel- wise image characteristics. This study aims to preserve structural details of LDCT images by applying boosting attention modules, prevents edge over-smoothing by integrating perceptual loss via VGG16 pre-trained network, and finally, improves the computational efficiency by taking advantage of deep learning techniques and GPU parallel computation.
Collapse
|
39
|
Five-second STEM dislocation tomography for 300 nm thick specimen assisted by deep-learning-based noise filtering. Sci Rep 2021; 11:20720. [PMID: 34702955 PMCID: PMC8548491 DOI: 10.1038/s41598-021-99914-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 09/30/2021] [Indexed: 11/22/2022] Open
Abstract
Scanning transmission electron microscopy (STEM) is suitable for visualizing the inside of a relatively thick specimen than the conventional transmission electron microscopy, whose resolution is limited by the chromatic aberration of image forming lenses, and thus, the STEM mode has been employed frequently for computed electron tomography based three-dimensional (3D) structural characterization and combined with analytical methods such as annular dark field imaging or spectroscopies. However, the image quality of STEM is severely suffered by noise or artifacts especially when rapid imaging, in the order of millisecond per frame or faster, is pursued. Here we demonstrate a deep-learning-assisted rapid STEM tomography, which visualizes 3D dislocation arrangement only within five-second acquisition of all the tilt-series images even in a 300 nm thick steel specimen. The developed method offers a new platform for various in situ or operando 3D microanalyses in which dealing with relatively thick specimens or covering media like liquid cells are required.
Collapse
|
40
|
Bewes J, Doganay O, Chen M, McIntyre A, Gleeson F. Imaging Dynamic Expiration: Feasibility of MRI Spirometry Using Hyperpolarized Xenon Gas. Radiol Cardiothorac Imaging 2021; 3:e200571. [PMID: 34498002 DOI: 10.1148/ryct.2021200571] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 06/01/2021] [Accepted: 06/15/2021] [Indexed: 11/11/2022]
Abstract
Purpose To examine the feasibility of imaging-based spirometry using high-temporal-resolution projection MRI and hyperpolarized xenon 129 (129Xe) gas. Materials and Methods In this prospective exploratory study, five healthy participants (age range, 25-45 years; three men) underwent an MRI spirometry technique using inhaled hyperpolarized 129Xe and rapid two-dimensional projection MRI. Participants inhaled 129Xe, then performed a forced expiratory maneuver while in an MR imager. Images of the lungs during expiration were captured in time intervals as short as 250 msec. Volume-corrected images of the lungs at expiration commencement (0 second), 1 second after expiration, and 6 seconds after expiration were extracted to generate forced expiratory volume in 1 second (FEV1), forced vital capacity (FVC), and FEV1/FVC ratio pulmonary maps. For comparison, participants performed conventional spirometry in the sitting position using room air, in the supine position using room air, and in the supine position using a room air and 129Xe mixture. Paired t tests with Bonferroni corrections for multiple comparisons were used for statistical analyses. Results The mean MRI-derived FEV1/FVC value was lower in comparison with conventional spirometry (0.52 ± 0.03 vs 0.70 ± 0.05, P < .01), which may reflect selective 129Xe retention. A secondary finding of this study was that 1 L of inhaled 129Xe negatively impacted pulmonary function as measured by conventional spirometry (in supine position), which reduced measured FEV1 (2.70 ± 0.90 vs 3.04 ± 0.85, P < .01) and FEV1/FVC (0.70 ± 0.05 vs 0.79 ± 0.04, P < .01). Conclusion A forced expiratory maneuver was successfully imaged with hyperpolarized 129Xe and high-temporal-resolution MRI. Derivation of regional lung spirometric maps was feasible.Keywords: MR-Imaging, MR-Dynamic Contrast Enhanced, MR-Functional Imaging, Pulmonary, Thorax, Diaphragm, Lung, Pleura, Physics Supplemental material is available for this article. © RSNA, 2021.
Collapse
Affiliation(s)
- James Bewes
- Department of Radiology, The Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Old Road, Oxford OX3 7LE, England
| | - Ozkan Doganay
- Department of Radiology, The Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Old Road, Oxford OX3 7LE, England
| | - Mitchell Chen
- Department of Radiology, The Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Old Road, Oxford OX3 7LE, England
| | - Anthony McIntyre
- Department of Radiology, The Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Old Road, Oxford OX3 7LE, England
| | - Fergus Gleeson
- Department of Radiology, The Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Old Road, Oxford OX3 7LE, England
| |
Collapse
|
41
|
Mäkinen Y, Marchesini S, Foi A. Ring artifact reduction via multiscale nonlocal collaborative filtering of spatially correlated noise. JOURNAL OF SYNCHROTRON RADIATION 2021; 28:876-888. [PMID: 33949995 PMCID: PMC8127377 DOI: 10.1107/s1600577521001910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Accepted: 02/17/2021] [Indexed: 06/12/2023]
Abstract
X-ray micro-tomography systems often suffer severe ring artifacts in reconstructed images. These artifacts are caused by defects in the detector, calibration errors, and fluctuations producing streak noise in the raw sinogram data. In this work, these streaks are modeled in the sinogram domain as additive stationary correlated noise upon logarithmic transformation. Based on this model, a streak removal procedure is proposed where the Block-Matching and 3-D (BM3D) filtering algorithm is applied across multiple scales, achieving state-of-the-art performance in both real and simulated data. Specifically, the proposed fully automatic procedure allows for attenuation of streak noise and the corresponding ring artifacts without creating major distortions common to other streak removal algorithms.
Collapse
Affiliation(s)
| | - Stefano Marchesini
- SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, CA 94025, USA
| | | |
Collapse
|
42
|
Koonjoo N, Zhu B, Bagnall GC, Bhutto D, Rosen MS. Boosting the signal-to-noise of low-field MRI with deep learning image reconstruction. Sci Rep 2021; 11:8248. [PMID: 33859218 PMCID: PMC8050246 DOI: 10.1038/s41598-021-87482-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Accepted: 03/30/2021] [Indexed: 12/26/2022] Open
Abstract
Recent years have seen a resurgence of interest in inexpensive low magnetic field (< 0.3 T) MRI systems mainly due to advances in magnet, coil and gradient set designs. Most of these advances have focused on improving hardware and signal acquisition strategies, and far less on the use of advanced image reconstruction methods to improve attainable image quality at low field. We describe here the use of our end-to-end deep neural network approach (AUTOMAP) to improve the image quality of highly noise-corrupted low-field MRI data. We compare the performance of this approach to two additional state-of-the-art denoising pipelines. We find that AUTOMAP improves image reconstruction of data acquired on two very different low-field MRI systems: human brain data acquired at 6.5 mT, and plant root data acquired at 47 mT, demonstrating SNR gains above Fourier reconstruction by factors of 1.5- to 4.5-fold, and 3-fold, respectively. In these applications, AUTOMAP outperformed two different contemporary image-based denoising algorithms, and suppressed noise-like spike artifacts in the reconstructed images. The impact of domain-specific training corpora on the reconstruction performance is discussed. The AUTOMAP approach to image reconstruction will enable significant image quality improvements at low-field, especially in highly noise-corrupted environments.
Collapse
Affiliation(s)
- N Koonjoo
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, 02129, USA.
- Harvard Medical School, Boston, MA, 02115, USA.
| | - B Zhu
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, 02129, USA
- Harvard Medical School, Boston, MA, 02115, USA
| | - G Cody Bagnall
- Department of Biological and Agricultural Engineering, Texas A&M University, College Station, TX, 77843, USA
| | - D Bhutto
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, 02129, USA
- Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA
| | - M S Rosen
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, 02129, USA
- Harvard Medical School, Boston, MA, 02115, USA
- Department of Physics, Harvard University, Cambridge, MA, 02138, USA
| |
Collapse
|
43
|
Sano A, Nishio T, Masuda T, Karasawa K. Denoising PET images for proton therapy using a residual U-net. Biomed Phys Eng Express 2021; 7. [PMID: 33540390 DOI: 10.1088/2057-1976/abe33c] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Accepted: 02/04/2021] [Indexed: 11/11/2022]
Abstract
The use of proton therapy has the advantage of high dose concentration as it is possible to concentrate the dose on the tumor while suppressing damage to the surrounding normal organs. However, the range uncertainty significantly affects the actual dose distribution in the vicinity of the proton range, limiting the benefit of proton therapy for reducing the dose to normal organs. By measuring the annihilation gamma rays from the produced positron emitters, it is possible to obtain a proton induced positron emission tomography (pPET) image according to the irradiation region of the proton beam. Smoothing with a Gaussian filter is generally used to denoise PET images; however, this approach lowers the spatial resolution. Furthermore, other conventional smoothing processing methods may deteriorate the steep region of the pPET images. In this study, we proposed a denoising method based on a Residual U-Net for pPET images. We conducted the Monte Carlo simulation and irradiation experiment on a human phantom to obtain pPET data. The accuracy of the range estimation and the image similarity were evaluated for pPET images using the Residual U-Net, a Gaussian filter, a median filter, the block-matching and 3D-filtering (BM3D), and a total variation (TV) filter. Usage of the Residual U-Net yielded effective results corresponding to the range estimation; however, the results of peak-signal-to-noise ratio were identical to those for the Gaussian filter, median filter, BM3D, and TV filter. The proposed method can contribute to improving the accuracy of treatment verification and shortening the PET measurement time.
Collapse
Affiliation(s)
- Akira Sano
- Department of Medical Physics, Graduate School of Medicine, Tokyo Women's Medical University, 8-1, Kawadacho, Shinjuku-ku, Shinjuku-ku, Tokyo, 162-8666, JAPAN
| | - Teiji Nishio
- Department of Medical Physics, Graduate School of Medicine, Tokyo Women's Medical University, 8-1, Kawadacho, Shinjuku-ku, Shinjuku-ku, Tokyo, 162-8666, JAPAN
| | - Takamitsu Masuda
- Department of Medical Physics, Graduate School of Medicine, Tokyo Women's Medical University, 8-1, Kawadacho, Shinjuku-ku, Shinjuku-ku, Tokyo, 162-8666, JAPAN
| | - Kumiko Karasawa
- Department of Radiation Oncology, School of Medicine, Tokyo Women's Medical University, 8-1, Kawadacho, Shinjuku-ku, Shinjuku-ku, Tokyo, 162-8666, JAPAN
| |
Collapse
|
44
|
A Photogrammetry-Based Workflow for the Accurate 3D Construction and Visualization of Museums Assets. REMOTE SENSING 2021. [DOI: 10.3390/rs13030486] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Nowadays digital replicas of artefacts belonging to the Cultural Heritage (CH) are one of the most promising innovations for museums exhibitions, since they foster new forms of interaction with collections, at different scales. However, practical digitization is still a complex task dedicated to specialized operators. Due to these premises, this paper introduces a novel approach to support non-experts working in museums with robust, easy-to-use workflows based on low-cost widespread devices, aimed at the study, classification, preservation, communication and restoration of CH artefacts. The proposed methodology introduces an automated combination of acquisition, based on mobile equipment and visualization, based on Real-Time Rendering. After the description of devices used along the workflow, the paper focuses on image pre-processing and geometry processing techniques adopted to generate accurate 3D models from photographs. Assessment criteria for the developed process evaluation are illustrated. Tests of the methodology on some effective museum case studies are presented and discussed.
Collapse
|
45
|
Bhat SS, Poojar P, Padma CR, Ananth RK, Hanumantharaju MC, Geethanath S. Deep Learning-Based Denoising for High b-Value at 2000 s/mm2 Diffusion-Weighted Imaging. Crit Rev Biomed Eng 2021; 49:1-10. [PMID: 35993947 DOI: 10.1615/critrevbiomedeng.2022040279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Diffusion-weighted imaging (DWI) allows white matter quantification of the white matter tracts of the brain. However, at a high b-value (≥ 2000 s/mm2), DWI acquisition suffers from noise due to longer acquisition times obscuring white matter interpretation. DWI denoising techniques can be used to acquire high b-value DWI without increasing the number of signal averages. We used a residual learning-based convolutional neural network (DnCNN) to reduce noise in high b-value DWI based on the literature review. We applied the proposed denoising method on high b-value, retrospectively collected DWI data with multiple noise levels. Experimental results show an improved image quality after denoising in retrospective DWI (average PSNR before and after denoising: 27.63 ± 1.06 dB and 51.76 ± 1.95 dB, respectively). The prospective DWI included one and two signal averages for denoising. DWI with four signal averages was used as the reference. Representative images show high b-value prospective DW images denoised using the DnCNN. We demonstrated DnCNN for cases of multiple noise levels and signal averages. For the prospective study, the PSNR values for 1-NEX before and after denoising were 27.39 ± 3.75 dB and 27.68 ± 3.75 dB. For 2-NEX, the PSNR values before and after denoising were 27.51 ± 4.18 dB and 27.75 ± 4.05 dB.
Collapse
Affiliation(s)
- Seema S Bhat
- Department of Information Science and Engineering, Dayananda Sagar College of Engineering, Bengaluru, India
| | - Pavan Poojar
- Columbia University, New York, NY, USA; Department of Medical Electronics Engineering, Dayananda Sagar College of Engineering, Bengaluru, India
| | - Chennagiri Rajarao Padma
- Medical Imaging Research Center (MIRC), Department of Medical Electronics Engineering, Dayananda Sagar College of Engineering, Bengaluru, India
| | | | - M C Hanumantharaju
- Department of Electronics and Communication Engineering, BMS Institute of Technology Management, Bengaluru 560064, India
| | - Sairam Geethanath
- Medical Imaging Research Center (MIRC), Department of Medical Electronics, Dayananda Sagar College of Engineering, Bengaluru, India; Magnetic Resonance Research Center, Columbia University, New York, NY 10027
| |
Collapse
|