Dictionary learning technique enhances signal in LED-based photoacoustic imaging

Enhancement of structural and functional photoacoustic imaging based on a reference-inputted convolutional neural network

Photoacoustic microscopy has demonstrated outstanding performance in high-resolution functional imaging. However, in the process of photoacoustic imaging, the photoacoustic signals will be polluted by inevitable background noise. Besides, the image quality is compromised due to the biosafety limitation of the laser. The conventional approach to improving image quality, such as increasing laser pulse energy or multiple-times averaging, could result in more health risks and motion artifacts for high exposures to the laser. To overcome this challenge of biosafety and compromised image quality, we propose a reference-inputted convolutional neural network (Ri-Net). The network is trained using the photoacoustic signal and noise datasets from phantom experiments. Evaluation of the trained neural network demonstrates significant signal improvement. Human cuticle microvasculature imaging experiments are also conducted to further assess the performance and practicality of our network. The quantitative results show that we achieved a 2.6-fold improvement in image contrast and a 9.6 dB increase in signal-to-noise ratio. Finally, we apply our network, trained on single-wavelength data, to multi-wavelength functional imaging. The functional imaging of the mouse ear demonstrates the robustness of our method and the potential to capture the oxygen saturation of microvasculature. The Ri-Net enhances photoacoustic microscopy imaging, allowing for more efficient microcirculation assessments in a clinical setting.

Singular value decomposition with weighting matrix applied for optical-resolution photoacoustic microscopes

The prestige target selectivity and imaging depth of optical-resolution photoacoustic microscope (OR-PAM) have gained attentions to enable advanced intra-cellular visualizations. However, the broad-band nature of photoacoustic signals is prone to noise and artifacts caused by the inefficient light-to-pressure translation, resulting in poor image quality. The present study foresees application of singular value decomposition (SVD) to effectively extract the photoacoustic signals from these noise and artifacts. Although spatiotemporal SVD succeeded in ultrasound flow signal extraction, the conventional multi frame model is not suitable for data acquired with scanning OR-PAM due to the burden of accessing multiple frames. To utilize SVD on the OR-PAM, this study began with exploring SVD applied on multiple A-lines of photoacoustic signal instead of frames. Upon explorations, an obstacle of uncertain presence of unwanted singular vectors was observed. To tackle this, a data-driven weighting matrix was designed to extract relevant singular vectors based on the analyses of temporal-spatial singular vectors. Evaluation on the extraction capability by the SVD with the weighting matrix showed a superior signal quality with efficient computation against past studies. In summary, this study contributes to the field by providing exploration of SVD applied on A-line signals as well as its practical utilization to distinguish and recover photoacoustic signals from noise and artifact components.

Perspectives on endoscopic functional photoacoustic microscopy

Endoscopy, enabling high-resolution imaging of deep tissues and internal organs, plays an important role in basic research and clinical practice. Recent advances in photoacoustic microscopy (PAM), demonstrating excellent capabilities in high-resolution functional imaging, have sparked significant interest in its integration into the field of endoscopy. However, there are challenges in achieving functional PAM in the endoscopic setting. This Perspective article discusses current progress in the development of endoscopic PAM and the challenges related to functional measurements. Then, it points out potential directions to advance endoscopic PAM for functional imaging by leveraging fiber optics, microfabrication, optical engineering, and computational approaches. Finally, it highlights emerging opportunities for functional endoscopic PAM in basic and translational biomedicine.

4D spectral-spatial computational photoacoustic dermoscopy

Photoacoustic dermoscopy (PAD) is an emerging non-invasive imaging technology aids in the diagnosis of dermatological conditions by obtaining optical absorption information of skin tissues. Despite advances in PAD, it remains unclear how to obtain quantitative accuracy of the reconstructed PAD images according to the optical and acoustic properties of multilayered skin, the wavelength and distribution of excitation light, and the detection performance of ultrasound transducers. In this work, a computing method of four-dimensional (4D) spectral-spatial imaging for PAD is developed to enable quantitative analysis and optimization of structural and functional imaging of skin. This method takes the optical and acoustic properties of heterogeneous skin tissues into account, which can be used to correct the optical field of excitation light, detectable ultrasonic field, and provide accurate single-spectrum analysis or multi-spectral imaging solutions of PAD for multilayered skin tissues. A series of experiments were performed, and simulation datasets obtained from the computational model were used to train neural networks to further improve the imaging quality of the PAD system. All the results demonstrated the method could contribute to the development and optimization of clinical PADs by datasets with multiple variable parameters, and provide clinical predictability of photoacoustic (PA) data for human skin.

Adaptive coherent weighted averaging algorithm for enhancement of photoacoustic tomography images of brain

One common method to improve the low signal-to-noise ratio of the photoacoustic (PA) signal generated from weak absorbers or absorbers located in deep tissue is to acquire signal multiple times from the same region and perform averaging. However, pulse-to-pulse laser fluctuations together with differences in the beam profile of the pulses create undeterministic multiple scattering processes in the tissue. This phenomenon consequently induces a spatiotemporal displacement in the PA signal samples which in turn deteriorates the effectiveness of signal averaging. Here, we present an adaptive coherent weighted averaging algorithm to adjust the locations and values of PA signal samples for more efficient signal averaging. The proposed method is evaluated in a linear array-based PA imaging setup of ex vivo sheep brain.

An extremum-guided interpolation for sparsely sampled photoacoustic imaging

In photoacoustic (PA) reconstruction, spatial constraints or real-time system requirements often result to sparse PA sampling data. For sparse PA sensor data, the sparse spatial and dense temporal sampling often leads to poor signal continuity. To address the structural characteristics of sparse PA signals, a data interpolation algorithm based on extremum-guided interpolation is proposed. This algorithm is based on the continuity of the signal, and can complete the estimation of high sampling rate signals without complex mathematical calculations. PA signal data is interpolated and reconstructed, and the results are evaluated using image quality assessment methods. The simulation and experimental results show that the proposed method performs better than several typical algorithms, effectively restoring image details, suppressing the generation of artifacts and noise, and improving the quality of PA reconstruction under sparse sampling.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

An Investigation of Signal Preprocessing for Photoacoustic Tomography

Photoacoustic tomography (PAT) is increasingly being used for high-resolution biological imaging at depth. Signal-to-noise ratios and resolution are the main factors that determine image quality. Various reconstruction algorithms have been proposed and applied to reduce noise and enhance resolution, but the efficacy of signal preprocessing methods which also affect image quality, are seldom discussed. We, therefore, compared common preprocessing techniques, namely bandpass filters, wavelet denoising, empirical mode decomposition, and singular value decomposition. Each was compared with and without accounting for sensor directivity. The denoising performance was evaluated with the contrast-to-noise ratio (CNR), and the resolution was calculated as the full width at half maximum (FWHM) in both the lateral and axial directions. In the phantom experiment, counting in directivity was found to significantly reduce noise, outperforming other methods. Irrespective of directivity, the best performing methods for denoising were bandpass, unfiltered, SVD, wavelet, and EMD, in that order. Only bandpass filtering consistently yielded improvements. Significant improvements in the lateral resolution were observed using directivity in two out of three acquisitions. This study investigated the advantages and disadvantages of different preprocessing methods and may help to determine better practices in PAT reconstruction.

Clinical photoacoustic/ultrasound dual-modal imaging: Current status and future trends

Photoacoustic tomography (PAT) is an emerging biomedical imaging modality that combines optical and ultrasonic imaging, providing overlapping fields of view. This hybrid approach allows for a natural integration of PAT and ultrasound (US) imaging in a single platform. Due to the similarities in signal acquisition and processing, the combination of PAT and US imaging creates a new hybrid imaging for novel clinical applications. Over the recent years, particular attention is paid to the development of PAT/US dual-modal systems highlighting mutual benefits in clinical cases, with an aim of substantially improving the specificity and sensitivity for diagnosis of diseases. The demonstrated feasibility and accuracy in these efforts open an avenue of translating PAT/US imaging to practical clinical applications. In this review, the current PAT/US dual-modal imaging systems are discussed in detail, and their promising clinical applications are presented and compared systematically. Finally, this review describes the potential impacts of these combined systems in the coming future.

Sparse Coding-Enabled Low-Fluence Multi-Parametric Photoacoustic Microscopy

Uniquely capable of simultaneous imaging of the hemoglobin concentration, blood oxygenation, and flow speed at the microvascular level in vivo, multi-parametric photoacoustic microscopy (PAM) has shown considerable impact in biomedicine. However, the multi-parametric PAM acquisition requires dense sampling and thus a high laser pulse repetition rate (up to MHz), which sets a strict limit on the applicable pulse energy due to safety considerations. A similar limitation is shared by high-speed PAM, which also uses lasers with high pulse repetition rates. To achieve high quantitative accuracy besides good structural visualization at low levels of laser fluence in PAM, we have developed a new, sparse coding-based two-step denoising technique. In the setting of intravital brain imaging, we demonstrated that this unsupervised learning approach enabled the reduction of the laser fluence in PAM by 5 times without compromise of the image quality (structural similarity index measure or SSIM: >0.92) and the quantitative accuracy (errors: <4.9%). Offering a significant relaxation in the requirement of PAM on laser fluence while maintaining the quality of structural imaging and accuracy of quantitative measurements, this sparse coding-based approach is expected to facilitate the application and clinical translation of multi-parametric PAM and high-speed PAM, which have a tight photon budget due to either safety considerations or laser source limitations.

Photoacoustic-MR Image Registration Based on a Co-Sparse Analysis Model to Compensate for Brain Shift

Brain shift is an important obstacle to the application of image guidance during neurosurgical interventions. There has been a growing interest in intra-operative imaging to update the image-guided surgery systems. However, due to the innate limitations of the current imaging modalities, accurate brain shift compensation continues to be a challenging task. In this study, the application of intra-operative photoacoustic imaging and registration of the intra-operative photoacoustic with pre-operative MR images are proposed to compensate for brain deformation. Finding a satisfactory registration method is challenging due to the unpredictable nature of brain deformation. In this study, the co-sparse analysis model is proposed for photoacoustic-MR image registration, which can capture the interdependency of the two modalities. The proposed algorithm works based on the minimization of mapping transform via a pair of analysis operators that are learned by the alternating direction method of multipliers. The method was evaluated using an experimental phantom and ex vivo data obtained from a mouse brain. The results of the phantom data show about 63% improvement in target registration error in comparison with the commonly used normalized mutual information method. The results proved that intra-operative photoacoustic images could become a promising tool when the brain shift invalidates pre-operative MRI.

Modeling combined ultrasound and photoacoustic imaging: Simulations aiding device development and artificial intelligence

Combined ultrasound and photoacoustic (USPA) imaging has attracted several pre-clinical and clinical applications due to its ability to simultaneously display structural, functional, and molecular information of deep biological tissue in real time. However, the depth and wavelength dependent optical attenuation and the unknown optical and acoustic heterogeneities limit the USPA imaging performance in deep tissue regions. Novel instrumentation, image reconstruction, and artificial intelligence (AI) methods are currently being investigated to overcome these limitations and improve the USPA image quality. Effective implementation of these approaches requires a reliable USPA simulation tool capable of generating US based anatomical and PA based molecular contrasts of deep biological tissue. Here, we developed a hybrid USPA simulation platform by integrating finite element models of light (NIRFast) and ultrasound (k-Wave) propagations for co-simulation of B-mode US and PA images. The platform allows optimization of different design parameters for USPA devices, such as the aperture size and frequency of both light and ultrasound detector arrays. For designing tissue-realistic digital phantoms, a dictionary-based function has been added to k-Wave to generate various levels of ultrasound speckle contrast. The feasibility of modeling US imaging combined with optical fluence dependent multispectral PA imaging is demonstrated using homogeneous as well as heterogeneous tissue phantoms mimicking human organs (e.g., prostate and finger). In addition, we also demonstrate the potential of the simulation platform to generate large scale application-specific training and test datasets for AI enhanced USPA imaging. The complete USPA simulation codes together with the supplementary user guides have been posted to an open-source repository (https://github.com/KothapalliLabPSU/US-PA_simulation_codes).

Motion-compensated noninvasive periodontal health monitoring using handheld and motor-based photoacoustic-ultrasound imaging systems

Simultaneous visualization of the teeth and periodontium is of significant clinical interest for image-based monitoring of periodontal health. We recently reported the application of a dual-modality photoacoustic-ultrasound (PA-US) imaging system for resolving periodontal anatomy and periodontal pocket depths in humans. This work utilized a linear array transducer attached to a stepper motor to generate 3D images via maximum intensity projection. This prior work also used a medical head immobilizer to reduce artifacts during volume rendering caused by motion from the subject (e.g., breathing, minor head movements). However, this solution does not completely eliminate motion artifacts while also complicating the imaging procedure and causing patient discomfort. To address this issue, we report the implementation of an image registration technique to correctly align B-mode PA-US images and generate artifact-free 2D cross-sections. Application of the deshaking technique to PA phantoms revealed 80% similarity to the ground truth when shaking was intentionally applied during stepper motor scans. Images from handheld sweeps could also be deshaken using an LED PA-US scanner. In ex vivo porcine mandibles, pigmentation of the enamel was well-estimated within 0.1 mm error. The pocket depth measured in a healthy human subject was also in good agreement with our prior study. This report demonstrates that a modality-independent registration technique can be applied to clinically relevant PA-US scans of the periodontium to reduce operator burden of skill and subject discomfort while showing potential for handheld clinical periodontal imaging.

Photoacoustic Imaging of Human Vasculature Using LED versus Laser Illumination: A Comparison Study on Tissue Phantoms and In Vivo Humans

Vascular diseases are becoming an epidemic with an increasing aging population and increases in obesity and type II diabetes. Point-of-care (POC) diagnosis and monitoring of vascular diseases is an unmet medical need. Photoacoustic imaging (PAI) provides label-free multiparametric information of deep vasculature based on strong absorption of light photons by hemoglobin molecules. However, conventional PAI systems use bulky nanosecond lasers which hinders POC applications. Recently, light-emitting diodes (LEDs) have emerged as cost-effective and portable optical sources for the PAI of living subjects. However, state-of-art LED arrays carry significantly lower optical energy (<0.5 mJ/pulse) and high pulse repetition frequencies (PRFs) (4 KHz) compared to the high-power laser sources (100 mJ/pulse) with low PRFs of 10 Hz. Given these tradeoffs between portability, cost, optical energy and frame rate, this work systematically studies the deep tissue PAI performance of LED and laser illuminations to help select a suitable source for a given biomedical application. To draw a fair comparison, we developed a fiberoptic array that delivers laser illumination similar to the LED array and uses the same ultrasound transducer and data acquisition platform for PAI with these two illuminations. Several controlled studies on tissue phantoms demonstrated that portable LED arrays with high frame averaging show higher signal-to-noise ratios (SNRs) of up to 30 mm depth, and the high-energy laser source was found to be more effective for imaging depths greater than 30 mm at similar frame rates. Label-free in vivo imaging of human hand vasculature studies further confirmed that the vascular contrast from LED-PAI is similar to laser-PAI for up to 2 cm depths. Therefore, LED-PAI systems have strong potential to be a mobile health care technology for diagnosing vascular diseases such as peripheral arterial disease and stroke in POC and resource poor settings.

Oxygen Saturation Imaging Using LED-Based Photoacoustic System

Oxygen saturation imaging has potential in several preclinical and clinical applications. Dual-wavelength LED array-based photoacoustic oxygen saturation imaging can be an affordable solution in this case. For the translation of this technology, there is a need to improve its accuracy and validate it against ground truth methods. We propose a fluence compensated oxygen saturation imaging method, utilizing structural information from the ultrasound image, and prior knowledge of the optical properties of the tissue with a Monte-Carlo based light propagation model for the dual-wavelength LED array configuration. We then validate the proposed method with oximeter measurements in tissue-mimicking phantoms. Further, we demonstrate in vivo imaging on small animal and a human subject. We conclude that the proposed oxygen saturation imaging can be used to image tissue at a depth of 6–8 mm in both preclinical and clinical applications.

Resolution enhancement for flexible microscopic imaging based on dictionary learning

The idea of combining a flexible fiber bundle with the microscopic imaging system provides the possibility of the cross-scale detection of defects and textures on large-scale complex components. However, the pixelization artifacts caused by the inter-core spacing of the fibers degrade the image quality and make it difficult to identify the micro-features. A high-resolution reconstruction strategy is proposed based on dictionary learning. By training the high- and low-resolution image pairs after image registration, a coupled dictionary is obtained. Then high-quality images are obtained from the trained dictionary. Experimental results demonstrate that the pixelization artifacts can be effectively addressed, and the resolution of the reconstructed images can be promoted by 1.8 times.

Portable and Affordable Light Source-Based Photoacoustic Tomography

Photoacoustic imaging is a hybrid imaging modality that offers the advantages of optical (spectroscopic contrast) and ultrasound imaging (scalable spatial resolution and imaging depth). This promising modality has shown excellent potential in a wide range of preclinical and clinical imaging and sensing applications. Even though photoacoustic imaging technology has matured in research settings, its clinical translation is not happening at the expected pace. One of the main reasons for this is the requirement of bulky and expensive pulsed lasers for excitation. To accelerate the clinical translation of photoacoustic imaging and explore its potential in resource-limited settings, it is of paramount importance to develop portable and affordable light sources that can be used as the excitation light source. In this review, we focus on the following aspects: (1) the basic theory of photoacoustic imaging; (2) inexpensive light sources and different implementations; and (3) important preclinical and clinical applications, demonstrated using affordable light source-based photoacoustics. The main focus will be on laser diodes and light-emitting diodes as they have demonstrated promise in photoacoustic tomography-the key technological developments in these areas will be thoroughly reviewed. We believe that this review will be a useful opus for both the beginners and experts in the field of biomedical photoacoustic imaging.

Photoacoustic image improvement based on a combination of sparse coding and filtering

SIGNIFICANCE

Photoacoustic imaging (PAI) has been greatly developed in a broad range of diagnostic applications. The efficiency of light to sound conversion in PAI is limited by the ubiquitous noise arising from the tissue background, leading to a low signal-to-noise ratio (SNR), and thus a poor quality of images. Frame averaging has been widely used to reduce the noise; however, it compromises the temporal resolution of PAI.

AIM

We propose an approach for photoacoustic (PA) signal denoising based on a combination of low-pass filtering and sparse coding (LPFSC).

APPROACH

LPFSC method is based on the fact that PA signal can be modeled as the sum of low frequency and sparse components, which allows for the reduction of noise levels using a hybrid alternating direction method of multipliers in an optimization process.

RESULTS

LPFSC method was evaluated using in-silico and experimental phantoms. The results show a 26% improvement in the peak SNR of PA signal compared to the averaging method for in-silico data. On average, LPFSC method offers a 63% improvement in the image contrast-to-noise ratio and a 33% improvement in the structural similarity index compared to the averaging method for objects located at three different depths, ranging from 10 to 20 mm, in a porcine tissue phantom.

CONCLUSIONS

The proposed method is an effective tool for PA signal denoising, whereas it ultimately improves the quality of reconstructed images, especially at higher depths, without limiting the image acquisition speed.

Optimizing Irradiation Geometry in LED-Based Photoacoustic Imaging with 3D Printed Flexible and Modular Light Delivery System

Photoacoustic (PA) imaging–a technique combining the ability of optical imaging to probe functional properties of the tissue and deep structural imaging ability of ultrasound–has gained significant popularity in the past two decades for its utility in several biomedical applications. More recently, light-emitting diodes (LED) are being explored as an alternative to bulky and expensive laser systems used in PA imaging for their portability and low-cost. Due to the large beam divergence of LEDs compared to traditional laser beams, it is imperative to quantify the angular dependence of LED-based illumination and optimize its performance for imaging superficial or deep-seated lesions. A custom-built modular 3-D printed hinge system and tissue-mimicking phantoms with various absorption and scattering properties were used in this study to quantify the angular dependence of LED-based illumination. We also experimentally calculated the source divergence of the pulsed-LED arrays to be 58° ± 8°. Our results from point sources (pencil lead phantom) in non-scattering medium obey the cotangential relationship between the angle of irradiation and maximum PA intensity obtained at various imaging depths, as expected. Strong dependence on the angle of illumination at superficial depths (−5°/mm at 10 mm) was observed that becomes weaker at intermediate depths (−2.5°/mm at 20 mm) and negligible at deeper locations (−1.1°/mm at 30 mm). The results from the tissue-mimicking phantom in scattering media indicate that angles between 30–75° could be used for imaging lesions at various depths (12 mm–28 mm) where lower LED illumination angles (closer to being parallel to the imaging plane) are preferable for deep tissue imaging and superficial lesion imaging is possible with higher LED illumination angles (closer to being perpendicular to the imaging plane). Our results can serve as a priori knowledge for the future LED-based PA system designs employed for both preclinical and clinical applications.

High-quality photoacoustic image reconstruction based on deep convolutional neural network: towards intra-operative photoacoustic imaging

The use of intra-operative imaging system as an intervention solution to provide more accurate localization of complicated structures has become a necessity during the neurosurgery. However, due to the limitations of conventional imaging systems, high-quality real-time intra-operative imaging remains as a challenging problem. Meanwhile, photoacoustic imaging has appeared so promising to provide images of crucial structures such as blood vessels and microvasculature of tumors. To achieve high-quality photoacoustic images of vessels regarding the artifacts caused by the incomplete data, we proposed an approach based on the combination of time-reversal (TR) and deep learning methods. The proposed method applies a TR method in the first layer of the network which is followed by the convolutional neural network with weights adjusted to a set of simulated training data for the other layers to estimate artifact-free photoacoustic images. It was evaluated using a generated synthetic database of vessels. The mean of signal to noise ratio (SNR), peak SNR, structural similarity index, and edge preservation index for the test data were reached 14.6 dB, 35.3 dB, 0.97 and 0.90, respectively. As our results proved, by using the lower number of detectors and consequently the lower data acquisition time, our approach outperforms the TR algorithm in all criteria in a computational time compatible with clinical use.

Evaluation of multi-wavelengths LED-based photoacoustic imaging for maximum safe resection of glioma: a proof of concept study

PURPOSE

A real-time intra-operative imaging modality is required to update the navigation systems during neurosurgery, since precise localization and safe maximal resection of gliomas are of utmost clinical importance. Different intra-operative imaging modalities have been proposed to delineate the resection borders, each with advantages and disadvantages. This preliminary study was designed to simulate the photoacoustic imaging (PAI) to illustrate the brain tumor margin vessels for safe maximal resection of glioma.

METHODS

In this study, light emitting diode (LED)-based PAI was selected because of its lower cost, compact size and ease of use. We developed a simulation framework based on multi-wavelength LED-based PAI to further facilitate PAI during neurosurgery. This framework considers a multilayer model of the tumoral and normal brain tissue. The simulation of the optical fluence and absorption map in tissue at different depths was computed by Monte Carlo. Then, the propagation of initial photoacoustic pressure was simulated by using k-wave toolbox.

RESULTS

To evaluate the LED-based PAI, we used three evaluation criteria: signal-to-noise ratio (SNR), contrast ratio (CR) and full width of half maximum (FWHM). Results showed that by using proper wavelengths, the vessels were recovered with the same axial and lateral FWHM. Furthermore, by increasing the wavelength from 532 to 1064 nm, SNR and CR were increased in the deep region. The results showed that vessels with larger diameters at same wavelength have a higher CR with average improvement 28%.

CONCLUSION

Multi-wavelength LED-based PAI provides detailed images of the blood vessels which are crucial for detection of the residual glioma: The longer wavelengths like 1064 nm can be used for the deeper tumor margins, and the shorter wavelengths like 532 nm for tumor margins closer to the surface. LED-based PAI may be considered as a promising intra-operative imaging modality to delineate tumor margins.