Multimodal Imaging Spectroscopy of Tissue

Passive Q-switched mode-locked self-sweeping fiber laser

This Letter presents the first-ever, to the best of our knowledge, demonstration of a passive Q-switched mode-locked (QML) laser with wavelength self-sweeping. QML generation was observed in a figure-of-nine cavity Yb-doped fiber laser. A train of microsecond-long pulses was generated with each pulse consisting of a sequence of nanosecond-long pulses. The duration of the nanosecond-scale pulses was measured to be 3 ns, corresponding to the generation of ∼60 phase-locked longitudinal modes. At the same time, the spectral dynamics was self-induced wavelength sweeping with a sweeping range of up to 5 nm near 1075 nm. The results widen the understanding of the self-sweeping phenomenon and can be useful in applications requiring tunable pulsed radiation.

Trends and Challenges of the Modern Pathology Laboratory for Biopharmaceutical Research Excellence

Pathology, a fundamental discipline that bridges basic scientific discovery to the clinic, is integral to successful drug development. Intrinsically multimodal and multidimensional, anatomic pathology continues to be empowered by advancements in molecular and digital technologies enabling the spatial tissue detection of biomolecules such as genes, transcripts, and proteins. Over the past two decades, breakthroughs in spatial molecular biology technologies and advancements in automation and digitization of laboratory processes have enabled the implementation of higher throughput assays and the generation of extensive molecular data sets from tissue sections in biopharmaceutical research and development research units. It is our goal to provide readers with some rationale, advice, and ideas to help establish a modern molecular pathology laboratory to meet the emerging needs of biopharmaceutical research. This manuscript provides (1) a high-level overview of the current state and future vision for excellence in research pathology practice and (2) shared perspectives on how to optimally leverage the expertise of discovery, toxicologic, and translational pathologists to provide effective spatial, molecular, and digital pathology data to support modern drug discovery. It captures insights from the experiences, challenges, and solutions from pathology laboratories of various biopharmaceutical organizations, including their approaches to troubleshooting and adopting new technologies.

Multimodal Image Fusion Workflow Incorporating MALDI Imaging Mass Spectrometry and Microscopy for the Study of Small Pharmaceutical Compounds

Multimodal imaging analyses of dosed tissue samples can provide more comprehensive insights into the effects of a therapeutically active compound on a target tissue compared to single-modal imaging. For example, simultaneous spatial mapping of pharmaceutical compounds and endogenous macromolecule receptors is difficult to achieve in a single imaging experiment. Herein, we present a multimodal workflow combining imaging mass spectrometry with immunohistochemistry (IHC) fluorescence imaging and brightfield microscopy imaging. Imaging mass spectrometry enables direct mapping of pharmaceutical compounds and metabolites, IHC fluorescence imaging can visualize large proteins, and brightfield microscopy imaging provides tissue morphology information. Single-cell resolution images are generally difficult to acquire using imaging mass spectrometry but are readily acquired with IHC fluorescence and brightfield microscopy imaging. Spatial sharpening of mass spectrometry images would thus allow for higher fidelity coregistration with other higher-resolution microscopy images. Imaging mass spectrometry spatial resolution can be predicted to a finer value via a computational image fusion workflow, which models the relationship between the intensity values in the mass spectrometry image and the features of a high-spatial resolution microscopy image. As a proof of concept, our multimodal workflow was applied to brain tissue extracted from a Sprague-Dawley rat dosed with a kratom alkaloid, corynantheidine. Four candidate mathematical models, including linear regression, partial least-squares regression, random forest regression, and two-dimensional convolutional neural network (2-D CNN), were tested. The random forest and 2-D CNN models most accurately predicted the intensity values at each pixel as well as the overall patterns of the mass spectrometry images, while also providing the best spatial resolution enhancements. Herein, image fusion enabled predicted mass spectrometry images of corynantheidine, GABA, and glutamine to approximately 2.5 μm spatial resolutions, a significant improvement compared to the original images acquired at 25 μm spatial resolution. The predicted mass spectrometry images were then coregistered with an H&E image and IHC fluorescence image of the μ-opioid receptor to assess colocalization of corynantheidine with brain cells. Our study also provides insights into the different evaluation parameters to consider when utilizing image fusion for biological applications.

Small-Molecule Fluorescent Probes for Binding- and Activity-Based Sensing of Redox-Active Biological Metals

Although transition metals constitute less than 0.1% of the total mass within a human body, they have a substantial impact on fundamental biological processes across all kingdoms of life. Indeed, these nutrients play crucial roles in the physiological functions of enzymes, with the redox properties of many of these metals being essential to their activity. At the same time, imbalances in transition metal pools can be detrimental to health. Modern analytical techniques are helping to illuminate the workings of metal homeostasis at a molecular and atomic level, their spatial localization in real time, and the implications of metal dysregulation in disease pathogenesis. Fluorescence microscopy has proven to be one of the most promising non-invasive methods for studying metal pools in biological samples. The accuracy and sensitivity of bioimaging experiments are predominantly determined by the fluorescent metal-responsive sensor, highlighting the importance of rational probe design for such measurements. This review covers activity- and binding-based fluorescent metal sensors that have been applied to cellular studies. We focus on the essential redox-active metals: iron, copper, manganese, cobalt, chromium, and nickel. We aim to encourage further targeted efforts in developing innovative approaches to understanding the biological chemistry of redox-active metals.

A multi-modal image fusion workflow incorporating MALDI imaging mass spectrometry and microscopy for the study of small pharmaceutical compounds

Multi-modal imaging analyses of dosed tissue samples can provide more comprehensive insight into the effects of a therapeutically active compound on a target tissue compared to single-modal imaging. For example, simultaneous spatial mapping of pharmaceutical compounds and endogenous macromolecule receptors is difficult to achieve in a single imaging experiment. Herein, we present a multi-modal workflow combining imaging mass spectrometry with immunohistochemistry (IHC) fluorescence imaging and brightfield microscopy imaging. Imaging mass spectrometry enables direct mapping of pharmaceutical compounds and metabolites, IHC fluorescence imaging can visualize large proteins, and brightfield microscopy imaging provides tissue morphology information. Single-cell resolution images are generally difficult to acquire using imaging mass spectrometry, but are readily acquired with IHC fluorescence and brightfield microscopy imaging. Spatial sharpening of mass spectrometry images would thus allow for higher fidelity co-registration with higher resolution microscopy images. Imaging mass spectrometry spatial resolution can be predicted to a finer value via a computational image fusion workflow, which models the relationship between the intensity values in the mass spectrometry image and the features of a high spatial resolution microscopy image. As a proof of concept, our multi-modal workflow was applied to brain tissue extracted from a Sprague Dawley rat dosed with a kratom alkaloid, corynantheidine. Four candidate mathematical models including linear regression, partial least squares regression (PLS), random forest regression, and two-dimensional convolutional neural network (2-D CNN), were tested. The random forest and 2-D CNN models most accurately predicted the intensity values at each pixel as well as the overall patterns of the mass spectrometry images, while also providing the best spatial resolution enhancements. Herein, image fusion enabled predicted mass spectrometry images of corynantheidine, GABA, and glutamine to approximately 2.5 μm spatial resolutions, a significant improvement compared to the original images acquired at 25 μm spatial resolution. The predicted mass spectrometry images were then co-registered with an H&E image and IHC fluorescence image of the μ-opioid receptor to assess co-localization of corynantheidine with brain cells. Our study also provides insight into the different evaluation parameters to consider when utilizing image fusion for biological applications.

Feasibility studies of multimodal nonlinear endoscopy using multicore fiber bundles for remote scanning from tissue sections to bulk organs

Here, we report on the development and application of a compact multi-core fiber optical probe for multimodal non-linear imaging, combining the label-free modalities of Coherent Anti-Stokes Raman Scattering, Second Harmonic Generation, and Two-Photon Excited Fluorescence. Probes of this multi-core fiber design avoid moving and voltage-carrying parts at the distal end, thus providing promising improved compatibility with clinical requirements over competing implementations. The performance characteristics of the probe are established using thin cryo-sections and artificial targets before the applicability to clinically relevant samples is evaluated using ex vivo bulk human and porcine intestine tissues. After image reconstruction to counteract the data's inherently pixelated nature, the recorded images show high image quality and morpho-chemical conformity on the tissue level compared to multimodal non-linear images obtained with a laser-scanning microscope using a standard microscope objective. Furthermore, a simple yet effective reconstruction procedure is presented and demonstrated to yield satisfactory results. Finally, a clear pathway for further developments to facilitate a translation of the multimodal fiber probe into real-world clinical evaluation and application is outlined.

Comparison of denoising tools for the reconstruction of nonlinear multimodal images

Biophotonic multimodal imaging techniques provide deep insights into biological samples such as cells or tissues. However, the measurement time increases dramatically when high-resolution multimodal images (MM) are required. To address this challenge, mathematical methods can be used to shorten the acquisition time for such high-quality images. In this research, we compared standard methods, e.g., the median filter method and the phase retrieval method via the Gerchberg-Saxton algorithm with artificial intelligence (AI) based methods using MM images of head and neck tissues. The AI methods include two approaches: the first one is a transfer learning-based technique that uses the pre-trained network DnCNN. The second approach is the training of networks using augmented head and neck MM images. In this manner, we compared the Noise2Noise network, the MIRNet network, and our deep learning network namely incSRCNN, which is derived from the super-resolution convolutional neural network and inspired by the inception network. These methods reconstruct improved images using measured low-quality (LQ) images, which were measured in approximately 2 seconds. The evaluation was performed on artificial LQ images generated by degrading high-quality (HQ) images measured in 8 seconds using Poisson noise. The results showed the potential of using deep learning on these multimodal images to improve the data quality and reduce the acquisition time. Our proposed network has the advantage of having a simple architecture compared with similar-performing but highly parametrized networks DnCNN, MIRNet, and Noise2Noise.

Identification of tumor tissue in thin pathological samples via femtosecond laser-induced breakdown spectroscopy and machine learning

In the treatment of most newly discovered solid cancerous tumors, surgery remains the first treatment option. An important factor in the success of these operations is the precise identification of oncological safety margins to ensure the complete removal of the tumor without affecting much of the neighboring healthy tissue. Here we report on the possibility of applying femtosecond Laser-Induced Breakdown Spectroscopy (LIBS) combined with Machine Learning algorithms as an alternative discrimination technique to differentiate cancerous tissue. The emission spectra following the ablation on thin fixed liver and breast postoperative samples were recorded with high spatial resolution; adjacent stained sections served as a reference for tissue identification by classical pathological analysis. In a proof of principle test performed on liver tissue, Artificial Neural Networks and Random Forest algorithms were able to differentiate both healthy and tumor tissue with a very high Classification Accuracy of around 0.95. The ability to identify unknown tissue was performed on breast samples from different patients, also providing a high level of discrimination. Our results show that LIBS with femtosecond lasers is a technique with potential to be used in clinical applications for rapid identification of tissue type in the intraoperative surgical field.

Fluholoscopy-Compact and Simple Platform Combining Fluorescence and Holographic Microscopy

The combination of different imaging modalities into single imaging platforms has a strong potential in biomedical sciences as it permits the analysis of complementary properties of the target sample. Here, we report on an extremely simple, cost-effective, and compact microscope platform for achieving simultaneous fluorescence and quantitative phase imaging modes with the capability of working in a single snapshot. It is based on the use of a single illumination wavelength to both excite the sample's fluorescence and provide coherent illumination for phase imaging. After passing the microscope layout, the two imaging paths are separated using a bandpass filter, and the two imaging modes are simultaneously obtained using two digital cameras. We first present calibration and analysis of both fluorescence and phase imaging modalities working independently and, later on, experimental validation for the proposed common-path dual-mode imaging platform considering static (resolution test targets, fluorescent micro-beads, and water-suspended lab-made cultures) as well as dynamic (flowing fluorescent beads, human sperm cells, and live specimens from lab-made cultures) samples.

Empagliflozin Preserves Skeletal Muscle Function in a HFpEF Rat Model

Besides structural alterations in the myocardium, heart failure with preserved ejection fraction (HFpEF) is also associated with molecular and physiological alterations of the peripheral skeletal muscles (SKM) contributing to exercise intolerance often seen in HFpEF patients. Recently, the use of Sodium-Glucose-Transporter 2 inhibitors (SGLT2i) in clinical studies provided evidence for a significant reduction in the combined risk of cardiovascular death or hospitalization for HFpEF. The present study aimed to further elucidate the impact of Empagliflozin (Empa) on: (1) SKM function and metabolism and (2) mitochondrial function in an established HFpEF rat model. At the age of 24 weeks, obese ZSF1 rats were randomized either receiving standard care or Empa in the drinking water. ZSF1 lean animals served as healthy controls. After 8 weeks of treatment, echocardiography and SKM contractility were performed. Mitochondrial function was assessed in saponin skinned fibers and SKM tissue was snap frozen for molecular analyses. HFpEF was evident in the obese animals when compared to lean—increased E/é and preserved left ventricular ejection fraction. Empa treatment significantly improved E/é and resulted in improved SKM contractility with reduced intramuscular lipid content. Better mitochondrial function (mainly in complex IV) with only minor modulation of atrophy-related proteins was seen after Empa treatment. The results clearly documented a beneficial effect of Empa on SKM function in the present HFpEF model. These effects were accompanied by positive effects on mitochondrial function possibly modulating SKM function.

On 2D-FTIR-XRF microscopy - A step forward correlative tissue studies by infrared and hard X-ray radiation

Correlative Fourier Transform Infra-Red (FTIR) and hard X-Ray Fluorescence (XRF) microscopy studies of thin biological samples have recently evolved as complementary methods for biochemical fingerprinting of animal/human tissues. These are seen particularly useful for tracking the mechanisms of neurological diseases, i.e., in Alzheimer/Parkinson disease, in the brain where mishandling of trace metals (Fe, Cu, Zn) seems to be often associated with ongoing damage to molecular components via, among others, oxidative/reductive stress neurotoxicity. Despite substantial progress in state-of-the-art detection and data analysis methods, combined FTIR-XRF experiments have never benefited from correlation and co-localization analysis of molecular moieties and chemical elements, respectively. We here propose for the first time a completely novel data analysis pipeline, utilizing the idea of 2D correlation spectrometry for brain tissue analysis. In this paper, we utilized combined benchtop FTIR - synchrotron XRF mapping experiments on thin brain samples mounted on polypropylene membranes. By implementing our recently developed Multiple Linear Regression Multi-Reference (MLR-MR) algorithm, along with advanced image processing, artifact-free 2D FTIR-XRF spectra could be obtained by mitigating the impact of spectral artifacts, such as Etalon fringes and mild scattering Mie-like signatures, in the FTIR data. We demonstrated that the method is a powerful tool for co-localizing and correlating molecular arrangements and chemical elements (and vice versa) using visually attractive 2D correlograms. Moreover, the methods' applicability for fostering the identification of distinct (biological) materials, involving chemical elements and molecular arrangements, is also shown. Taken together, the 2D FTIR-XRF method opens up for new measures for in-situ investigating hidden complex biochemical correlations, and yet unraveled mechanisms in a biological sample. This step seems crucial for developing new strategies for facilitating the research on the interaction of metals/nonmetals with organic components. This is particularly important for enhancing our understanding of the diseases associated with metal/nonmetal mishandling.

Enhanced diffuse optical tomographic reconstruction using concurrent ultrasound information

Multimodal imaging is an active branch of research as it has the potential to improve common medical imaging techniques. Diffuse optical tomography (DOT) is an example of a low resolution, functional imaging modality that typically has very low resolution due to the ill-posedness of its underlying inverse problem. Combining the functional information of DOT with a high resolution structural imaging modality has been studied widely. In particular, the combination of DOT with ultrasound (US) could serve as a useful tool for clinicians for the formulation of accurate diagnosis of breast lesions. In this paper, we propose a novel method for US-guided DOT reconstruction using a portable time-domain measurement system. B-mode US imaging is used to retrieve morphological information on the probed tissues by means of a semi-automatical segmentation procedure based on active contour fitting. A two-dimensional to three-dimensional extrapolation procedure, based on the concept of distance transform, is then applied to generate a three-dimensional edge-weighting prior for the regularization of DOT. The reconstruction procedure has been tested on experimental data obtained on specifically designed dual-modality silicon phantoms. Results show a substantial quantification improvement upon the application of the implemented technique. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

Nanotechnology in Tumor Biomarker Detection: The Potential of Liganded Nanoclusters as Nonlinear Optical Contrast Agents for Molecular Diagnostics of Cancer

Cancer is one of the leading causes of premature death, and, as such, it can be prevented by developing strategies for early and accurate diagnosis. Cancer diagnostics has evolved from the macroscopic detection of malignant tissues to the fine analysis of tumor biomarkers using personalized medicine approaches. Recently, various nanomaterials have been introduced into the molecular diagnostics of cancer. This has resulted in a number of tumor biomarkers that have been detected in vitro and in vivo using nanodevices and corresponding imaging techniques. Atomically precise ligand-protected noble metal quantum nanoclusters represent an interesting class of nanomaterials with a great potential for the detection of tumor biomarkers. They are characterized by high biocompatibility, low toxicity, and suitability for controlled functionalization with moieties specifically recognizing tumor biomarkers. Their non-linear optical properties are of particular importance as they enable the visualization of nanocluster-labeled tumor biomarkers using non-linear optical techniques such as two-photon-excited fluorescence and second harmonic generation. This article reviews liganded nanoclusters among the different nanomaterials used for molecular cancer diagnosis and the relevance of this new class of nanomaterials as non-linear optical probe and contrast agents.

Coherent Raman scattering microscopy for chemical imaging of biological systems

Coherent Raman scattering (CRS) processes, including both the coherent anti-Stokes Raman scattering and stimulated Raman scattering, have been utilized in state-of-the-art microscopy platforms for chemical imaging of biological samples. The key advantage of CRS microscopy over fluorescence microscopy is label-free, which is an attractive characteristic for modern biological and medical sciences. Besides, CRS has other advantages such as higher selectivity to metabolites, no photobleaching, and narrow peak width. These features have brought fast-growing attention to CRS microscopy in biological research. In this review article, we will first briefly introduce the history of CRS microscopy, and then explain the theoretical background of the CRS processes in detail using the classical approach. Next, we will cover major instrumentation techniques of CRS microscopy. Finally, we will enumerate examples of recent applications of CRS imaging in biological and medical sciences.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Design of a Multimodal Imaging System and Its First Application to Distinguish Grey and White Matter of Brain Tissue. A Proof-of-Concept-Study

Multimodal imaging gains increasing popularity for biomedical applications. This article presents the design of a novel multimodal imaging system. The centerpiece is a light microscope operating in the incident and transmitted light mode. Additionally, Raman spectroscopy and VIS/NIR reflectance spectroscopy are adapted. The proof-of-concept is realized to distinguish between grey matter (GM) and white matter (WM) of normal mouse brain tissue. Besides Raman and VIS/NIR spectroscopy, the following optical microscopy techniques are applied in the incident light mode: brightfield, darkfield, and polarization microscopy. To complement the study, brightfield images of a hematoxylin and eosin (H&E) stained cryosection in the transmitted light mode are recorded using the same imaging system. Data acquisition based on polarization microscopy and Raman spectroscopy gives the best results regarding the tissue differentiation of the unstained section. In addition to the discrimination of GM and WM, both modalities are suited to highlight differences in the density of myelinated axons. For Raman spectroscopy, this is achieved by calculating the sum of two intensity peak ratios (I2857 + I2888)/I2930 in the high-wavenumber region. For an optimum combination of the modalities, it is recommended to apply the molecule-specific but time-consuming Raman spectroscopy to smaller regions of interest, which have previously been identified by the microscopic modes.

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Hematoxylin and Eosin (H&E) staining is the 'gold-standard' method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for 'real-time' disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author's best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.

Sacubitril/Valsartan Improves Diastolic Function But Not Skeletal Muscle Function in a Rat Model of HFpEF

The angiotensin receptor/neprilysin inhibitor Sacubitril/Valsartan (Sac/Val) has been shown to be beneficial in patients suffering from heart failure with reduced ejection fraction (HFrEF). However, the impact of Sac/Val in patients presenting with heart failure with preserved ejection fraction (HFpEF) is not yet clearly resolved. The present study aimed to reveal the influence of the drug on the functionality of the myocardium, the skeletal muscle, and the vasculature in a rat model of HFpEF. Female obese ZSF-1 rats received Sac/Val as a daily oral gavage for 12 weeks. Left ventricle (LV) function was assessed every four weeks using echocardiography. Prior to organ removal, invasive hemodynamic measurements were performed in both ventricles. Vascular function of the carotid artery and skeletal muscle function were monitored. Sac/Val treatment reduced E/é ratios, left ventricular end diastolic pressure (LVEDP) and myocardial stiffness as well as myocardial fibrosis and heart weight compared to the obese control group. Sac/Val slightly improved endothelial function in the carotid artery but had no impact on skeletal muscle function. Our results demonstrate striking effects of Sac/Val on the myocardial structure and function in a rat model of HFpEF. While vasodilation was slightly improved, functionality of the skeletal muscle remained unaffected.

Deep learning for 'artefact' removal in infrared spectroscopy

It has been well recognized that infrared spectra of microscopically heterogeneous media do not merely reflect the absorption of the sample but are influenced also by geometric factors and the wave nature of light causing scattering, reflection, interference, etc. These phenomena often occur simultaneously in complex samples like tissues and manifest themselves as intense baseline profiles, fringes, band distortion and band intensity changes in a measured IR spectrum. The information on the molecular level contained in IR spectra is thus entangled with the geometric structure of a sample and the optical model behind it, which largely hinders the data interpretation and in many cases renders the Beer-Lambert law invalid. It is required to recover the pure absorption (i.e., absorbance) of the sample from the measurement (i.e., apparent absorbance), that is, to remove the 'artefacts' caused merely by optical influences. To do so, we propose an artefact removal approach based on a deep convolutional neural network (CNN), specifically a 1-dimensional U-shape convolutional neural network (1D U-Net), and based our study on poly(methyl methacrylate) (PMMA) as materials. To start, a simulated dataset composed of apparent absorbance and absorbance pairs was generated according to the Mie-theory for PMMA spheres. After a data augmentation procedure, this dataset was utilized to train the 1D U-Net aiming to transform the input apparent absorbance into the corrected absorbance. The performance of the artefact removal was evaluated by the hit-quality-index (HQI) between the corrected and the true absorbance. Based on the prediction and the HQI of two experimental and one simulated independent testing datasets, we could demonstrate that the network was able to retrieve the absorbance very well, even in cases where the absorbance is completely overwhelmed by extremely large 'artefacts'. As the testing datasets bear different patterns of absorbance and 'artefacts' to the training data, the promising correction also indicated a good generalization performance of the 1D U-Net. Finally, the reliability and computational mechanism of the trained network were illustrated via two interpretation approaches including a direct visualization of layer-wise outputs as well as a saliency-based method.

Deep learning a boon for biophotonics?

This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state-of-the-art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real-time biophotonic decision-making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.

Non-invasive Imaging Techniques: From Histology to In Vivo Imaging : Chapter of Imaging in Oncology

In this chapter, we will introduce and review molecular-sensitive imaging techniques, which close the gap between ex vivo and in vivo analysis. In detail, we will introduce spontaneous Raman spectral imaging, coherent anti-Stokes Raman scattering (CARS), stimulated Raman scattering (SRS), second-harmonic generation (SHG) and third-harmonic generation (THG), two-photon excited fluorescence (TPEF), and fluorescence lifetime imaging (FLIM). After reviewing these imaging techniques, we shortly introduce chemometric methods and machine learning techniques, which are needed to use these imaging techniques in diagnostic applications.

A look into the use of Raman spectroscopy for brain and breast cancer diagnostics: linear and non-linear optics in cancer research as a gateway to tumor cell identity

Introduction: Currently, intensely developing of linear and non-linear optical methods for cancer detection provides a valuable tool to improve sensitivity and specificity. One of the main reasons for insufficient progress in cancer diagnostics is related to the fact that most cancer types are not only heterogeneous in their genetic composition but also reside in varying microenvironments and interact with different cell types. Until now, no technology has been fully proven for effective detecting of invasive cancer, which infiltrating the extracellular matrix.Areas covered: This review investigates the current status of Raman spectroscopy and Raman imaging for brain and breast cancer diagnostics. Moreover, the review provides a comprehensive overview of the applicability of atomic force microscopy (AFM), linear and non-linear optics in cancer research as a gateway to tumor cell identity.Expert commentary: A combination of linear and non-linear optics, particularly Raman-driven methods, has many additional advantages to identify alterations in cancer cells that are crucial for their proliferation and that distinguish them from normal cells.

Intravital microscopy: Imaging host-parasite interactions in lymphoid organs

Intravital microscopy allows imaging of biological phenomena within living animals, including host-parasite interactions. This has advanced our understanding of both, the function of lymphoid organs during parasitic infections, and the effect of parasites on such organs to allow their survival. In parasitic research, recent developments in this technique have been crucial for the direct study of host-parasite interactions within organs at depths, speeds and resolution previously difficult to achieve. Lymphoid organs have gained more attention as we start to understand their function during parasitic infections and the effect of parasites on them. In this review, we summarise technical and biological findings achieved by intravital microscopy with respect to the interaction of various parasites with host lymphoid organs, namely the bone marrow, thymus, lymph nodes, spleen and the mucosa-associated lymphoid tissue, and present a view into possible future applications.

Multimodal Nonlinear Microscopy for Therapy Monitoring of Cold Atmospheric Plasma Treatment

Here we report on a non-linear spectroscopic method for visualization of cold atmospheric plasma (CAP)-induced changes in tissue for reaching a new quality level of CAP application in medicine via online monitoring of wound or cancer treatment. A combination of coherent anti-Stokes Raman scattering (CARS), two-photon fluorescence lifetime imaging (2P-FLIM) and second harmonic generation (SHG) microscopy has been used for non-invasive and label-free detection of CAP-induced changes on human skin and mucosa samples. By correlation with histochemical staining, the observed local increase in fluorescence could be assigned to melanin. CARS and SHG prove the integrity of the tissue structure, visualize tissue morphology and composition. The influence of plasma effects by variation of plasma parameters e.g., duration of treatment, gas composition and plasma source has been evaluated. Overall quantitative spectroscopic markers could be identified for a direct monitoring of CAP-treated tissue areas, which is very important for translating CAPs into clinical routine.

Systematic study of the effect of ultrasound gel on the performances of time-domain diffuse optics and diffuse correlation spectroscopy

Recently, multimodal imaging has gained an increasing interest in medical applications thanks to the inherent combination of strengths of the different techniques. For example, diffuse optics is used to probe both the composition and the microstructure of highly diffusive media down to a depth of few centimeters, but its spatial resolution is intrinsically low. On the other hand, ultrasound imaging exhibits the higher spatial resolution of morphological imaging, but without providing solid constitutional information. Thus, the combination of diffuse optical imaging and ultrasound may improve the effectiveness of medical examinations, e.g. for screening or diagnosis of tumors. However, the presence of an ultrasound coupling gel between probe and tissue can impair diffuse optical measurements like diffuse optical spectroscopy and diffuse correlation spectroscopy, since it may provide a direct path for photons between source and detector. A systematic study on the effect of different ultrasound coupling fluids was performed on tissue-mimicking phantoms, confirming that a water-clear gel can produce detrimental effects on optical measurements when recovering absorption/reduced scattering coefficients from time-domain spectroscopy acquisitions as well as particle Brownian diffusion coefficient from diffuse correlation spectroscopy ones. On the other hand, we show the suitability for optical measurements of other types of diffusive fluids, also compatible with ultrasound imaging.

Double-Sided Anti-Reflection Nanostructures on Optical Convex Lenses for Imaging Applications

Anti-reflection coatings (ARCs) from the cornea nipple array of the moth-eye remarkably suppress the Fresnel reflection at the interface in broadband wavelength ranges. ARCs on flat glass have been studied to enhance the optical transmittance. However, little research on the implementation of ARCs on curved optical lenses, which are the core element in imaging devices, has been reported. Here, we report double-sided, bio-inspired ARCs on bi-convex lenses with high uniformity. We theoretically optimize the nanostructure geometry, such as the height, period, and morphology, since an anti-reflection property results from the gradually changed effective refractive index by the geometry of nanostructures. In an experiment, the transmittance of an ARCs lens increases up to 10% for a broadband spectrum without distortion in spot size and focal length. Moreover, we demonstrate ~30% improved transmittance of an imaging system composed of three bi-convex lenses, in series with double-sided ARCs (DARCs).

Optical coherence tomography follow-up of patients treated from periodontal disease

Optical coherence tomography (OCT) is one of the most important imaging modalities for biophotonics applications. In this work, an important step towards the clinical use of OCT in dental practice is reported, by following-up patients treated from periodontal disease (PD). A total of 147 vestibular dental sites from 14 patients diagnosed with PD were evaluated prior and after treatment, using a swept-source OCT and two periodontal probes (Florida probe and North Carolina) for comparison. The evaluation was performed at four stages: day 0, day 30, day 60 and day 90. Exceptionally one patient was evaluated 1-year after treatment. It was possible to visualize in the two-dimensional images the architectural components that compose the periodontal anatomy, and identify the improvements in biofilm and dental calculus upon treatment. In the follow-up after the treatment, it was observed in some cases decrease of the gingival thickness associated with extinction of gingival calculus. In some cases, the improvement of both depth of probing with the traditional probes and the evidence in the images of the region was emphasized. The study evidenced the ability of OCT in the identification of periodontal structures and alterations, being an important noninvasive complement or even alternative for periodontal probes for treatment follow-up. OCT system being used in a clinical environment. Above OCT image (left) prior treatment and (right) 30 days after treatment.

Label-free Imaging of Myocardial Remodeling in Atrial Fibrillation Using Nonlinear Optical Microscopy: A Feasibility Study

Atrial fibrillation, characterized by rapid disorganized electrical activation of myocardium, is caused by and accompanied by remodeling of myocardial tissue. We applied nonlinear optical microscopy (NLOM) to visualize typical myocardial features and atrial fibrillation effects in order to test anon-destructive imaging technology that in principle can be applied in vivo.Coherent anti-Stokes Raman scattering, endogenous two-photon excited fluorescence, and second harmonic generation were used to inspect unstained human atrial myocardium from three patients who underwent surgical Cox-MAZE procedure with amputation of left atrial appendage. Using NLOM techniques, we collected detailrich pictures of unstained tissue that enable comprehensive characterization of myocardial characteristics like myocyte structure, collagen and lipofuscin deposition, intercalating disc width, and fatty degradation. Development of in vivo application of the NLOM technique may represent a revolutionary approach in characterizing atrial fibrillation associated myocardial remodeling with important implications for therapy individualization and monitoring.

Fourier Transform Infrared (FT-IR) and Laser Ablation Inductively Coupled Plasma-Mass Spectrometry (LA-ICP-MS) Imaging of Cerebral Ischemia: Combined Analysis of Rat Brain Thin Cuts Toward Improved Tissue Classification

Microspectroscopic techniques are widely used to complement histological studies. Due to recent developments in the field of chemical imaging, combined chemical analysis has become attractive. This technique facilitates a deepened analysis compared to single techniques or side-by-side analysis. In this study, rat brains harvested one week after induction of photothrombotic stroke were investigated. Adjacent thin cuts from rats' brains were imaged using Fourier transform infrared (FT-IR) microspectroscopy and laser ablation inductively coupled plasma mass spectrometry (LA-ICP-MS). The LA-ICP-MS data were normalized using an internal standard (a thin gold layer). The acquired hyperspectral data cubes were fused and subjected to multivariate analysis. Brain regions affected by stroke as well as unaffected gray and white matter were identified and classified using a model based on either partial least squares discriminant analysis (PLS-DA) or random decision forest (RDF) algorithms. The RDF algorithm demonstrated the best results for classification. Improved classification was observed in the case of fused data in comparison to individual data sets (either FT-IR or LA-ICP-MS). Variable importance analysis demonstrated that both molecular and elemental content contribute to the improved RDF classification. Univariate spectral analysis identified biochemical properties of the assigned tissue types. Classification of multisensor hyperspectral data sets using an RDF algorithm allows access to a novel and in-depth understanding of biochemical processes and solid chemical allocation of different brain regions.

Depth-resolved multimodal imaging: Wavelength modulated spatially offset Raman spectroscopy with optical coherence tomography

A major challenge in biophotonics is multimodal imaging to obtain both morphological and molecular information at depth. We demonstrate a hybrid approach integrating optical coherence tomography (OCT) with wavelength modulated spatially offset Raman spectroscopy (WM-SORS). With depth colocalization obtained from the OCT, we can penetrate 1.2-mm deep into strong scattering media (lard) to acquire up to a 14-fold enhancement of a Raman signal from a hidden target (polystyrene) with a spatial offset. Our approach is capable of detecting both Raman and OCT signals for pharmaceutical particles embedded in turbid media and revealing the white matter at depth within a 0.6-mm thick brain tissue layer. This depth resolved label-free multimodal approach is a powerful route to analyze complex biomedical samples.

A guide to integrating immunohistochemistry and chemical imaging

A ‘how-to’ guide for designing chemical imaging experiments using antibodies and immunohistochemistry.

Elastic and inelastic light scattering spectroscopy and its possible use for label-free brain tumor typing

This paper presents an approach for label-free brain tumor tissue typing. For this application, our dual modality microspectroscopy system combines inelastic Raman scattering spectroscopy and Mie elastic light scattering spectroscopy. The system enables marker-free biomedical diagnostics and records both the chemical and morphologic changes of tissues on a cellular and subcellular level. The system setup is described and the suitability for measuring morphologic features is investigated. Graphical Abstract Bimodal approach for label-free brain tumor typing. Elastic and inelastic light scattering spectra are collected laterally resolved in one measurement setup. The spectra are investigated by multivariate data analysis for assigning the tissues to specific WHO grades according to their malignancy.

Optical parametric chirped pulse oscillation

A concept to flexibly adjust the spectral bandwidth of the output pulses of a fiber optical parametric oscillator is presented. By adjusting the chirp of the pump pulses appropriate to the chirp of the resonant pulses, the energy of the output pulses can be transferred into a user-defined spectral bandwidth. For this concept of optical parametric chirped pulse oscillation, we present numerical simulations of a parametric oscillator, which is able to convert pump pulses with a spectral bandwidth of 3.3 nm into output pulses with an adjustable spectral bandwidth between 9 and 0.05 nm. Combined with a wavelength tunability between 1200 and 1300 nm and pulse energies of up to 100 nJ, the concept should allow to adapt a single all-fiber parametric oscillator to a variety of applications, e.g., in multimodal nonlinear microscopy.

In vivo imaging of coral tissue and skeleton with optical coherence tomography

Application of optical coherence tomography (OCT) for in vivo imaging of tissue and skeleton structure of intact living corals enabled the non-invasive visualization of coral tissue layers (endoderm versus ectoderm), skeletal cavities and special structures such as mesenterial filaments and mucus release from intact living corals. Coral host chromatophores containing green fluorescent protein-like pigment granules appeared hyper-reflective to near-infrared radiation allowing for excellent optical contrast in OCT and a rapid characterization of chromatophore size, distribution and abundance. In vivo tissue plasticity could be quantified by the linear contraction velocity of coral tissues upon illumination resulting in dynamic changes in the live coral tissue surface area, which varied by a factor of 2 between the contracted and expanded state of a coral. Our study provides a novel view on the in vivo organization of coral tissue and skeleton and highlights the importance of microstructural dynamics for coral ecophysiology.

Developments in spontaneous and coherent Raman scattering microscopic imaging for biomedical applications

First, the potential role of Raman-based techniques in biomedicine is introduced. Second, an overview about the instrumentation for spontaneous and coherent Raman scattering microscopic imaging is given with a focus of recent developments. Third, imaging strategies are summarized including sequential registration with laser scanning microscopes, line imaging and global or wide-field imaging. Finally, examples of biomedical applications are presented in the context of single cells, laser tweezers, tissue sections, biopsies and whole animals.

Pseudo-HE images derived from CARS/TPEF/SHG multimodal imaging in combination with Raman-spectroscopy as a pathological screening tool

Background

Due to the steadily increasing number of cancer patients worldwide the early diagnosis and treatment of cancer is a major field of research. The diagnosis of cancer is mostly performed by an experienced pathologist via the visual inspection of histo-pathological stained tissue sections. To save valuable time, low quality cryosections are frequently analyzed with diagnostic accuracies that are below those of high quality embedded tissue sections. Thus, alternative means have to be found that enable for fast and accurate diagnosis as the basis of following clinical decision making.

Methods

In this contribution we will show that the combination of the three label-free non-linear imaging modalities CARS (coherent anti-Stokes Raman-scattering), TPEF (two-photon excited autofluorescence) and SHG (second harmonic generation) yields information that can be translated into computational hematoxylin and eosin (HE) images by multivariate statistics. Thereby, a computational HE stain is generated resulting in pseudo-HE overview images that allow for identification of suspicious regions. The latter are analyzed further by Raman-spectroscopy retrieving the tissue’s molecular fingerprint.

Results

The results suggest that the combination of non-linear multimodal imaging and Raman-spectroscopy possesses the potential as a precise and fast tool in routine histopathology.

Conclusions

As the key advantage, both optical methods are non-invasive enabling for further pathological investigations of the same tissue section, e.g. a direct comparison with the current pathological gold-standard.