Multimodal image reconstruction from tomographic diffraction microscopy data

Recent Advances and Current Trends in Transmission Tomographic Diffraction Microscopy

Optical microscopy techniques are among the most used methods in biomedical sample characterization. In their more advanced realization, optical microscopes demonstrate resolution down to the nanometric scale. These methods rely on the use of fluorescent sample labeling in order to break the diffraction limit. However, fluorescent molecules' phototoxicity or photobleaching is not always compatible with the investigated samples. To overcome this limitation, quantitative phase imaging techniques have been proposed. Among these, holographic imaging has demonstrated its ability to image living microscopic samples without staining. However, for a 3D assessment of samples, tomographic acquisitions are needed. Tomographic Diffraction Microscopy (TDM) combines holographic acquisitions with tomographic reconstructions. Relying on a 3D synthetic aperture process, TDM allows for 3D quantitative measurements of the complex refractive index of the investigated sample. Since its initial proposition by Emil Wolf in 1969, the concept of TDM has found a lot of applications and has become one of the hot topics in biomedical imaging. This review focuses on recent achievements in TDM development. Current trends and perspectives of the technique are also discussed.

Vectorial inverse scattering for dielectric tensor tomography: overcoming challenges of reconstruction of highly scattering birefringent samples

Many important microscopy samples, such as liquid crystals, biological tissue, or starches, are birefringent in nature. They scatter light differently depending on the polarization of the light and the orientation of the molecules. The complete characterization of a birefringent sample is a challenging task because its 3 × 3 dielectric tensor must be reconstructed at every three-dimensional position. Moreover, obtaining a birefringent tomogram is more arduous for thick samples, where multiple light scattering should also be considered. In this study, we developed a new dielectric tensor tomography algorithm that enables full characterization of highly scattering birefringent samples by solving the vectoral inverse scattering problem while accounting for multiple light scattering. We proposed a discrete image-processing theory to compute the error backpropagation of vectorially diffracting light. Finally, our theory was experimentally demonstrated using both synthetic and biologically birefringent samples.

3D differential interference contrast microscopy using polarisation-sensitive tomographic diffraction microscopy

Tomographic diffraction microscopy (TDM) is a generalisation of digital holographic microscopy (DHM), for which the illumination angle onto the sample is fully controlled, which has become a tool of choice for 3D, high-resolution imaging of unlabelled samples. TDM makes it possible to obtain the optical field in both amplitude and phase for each illumination angle. Proper information reallocation eventually allows for 3D reconstruction of the complex refractive index map. On the other hand, polarisation array sensors (PAS) paves new way for TDM, as vectorial information assessment about the investigated sample. In this contribution, we show an alternative use of this polarisation information based on the phase sensitive nature of TDM. Here, we demonstrated that TDM coupled with PAS can lead to a 3D differential interference contrast (DIC) microscope with almost no experimental configuration modification.

Regularization of dielectric tensor tomography

Dielectric tensor tomography reconstructs the three-dimensional dielectric tensors of microscopic objects and provides information about the crystalline structure orientations and principal refractive indices. Because dielectric tensor tomography is based on transmission measurement, it suffers from the missing cone problem, which causes poor axial resolution, underestimation of the refractive index, and halo artifacts. In this study, we study the application of total variation and positive semi-definiteness regularization to three-dimensional tensor distributions. In particular, we demonstrate the reduction of artifacts when applied to dielectric tensor tomography.

A Short Survey on Deep Learning for Multimodal Integration: Applications, Future Perspectives and Challenges

Deep learning has achieved state-of-the-art performances in several research applications nowadays: from computer vision to bioinformatics, from object detection to image generation. In the context of such newly developed deep-learning approaches, we can define the concept of multimodality. The objective of this research field is to implement methodologies which can use several modalities as input features to perform predictions. In this, there is a strong analogy with respect to what happens with human cognition, since we rely on several different senses to make decisions. In this article, we present a short survey on multimodal integration using deep-learning methods. In a first instance, we comprehensively review the concept of multimodality, describing it from a two-dimensional perspective. First, we provide, in fact, a taxonomical description of the multimodality concept. Secondly, we define the second multimodality dimension as the one describing the fusion approaches in multimodal deep learning. Eventually, we describe four applications of multimodal deep learning to the following fields of research: speech recognition, sentiment analysis, forensic applications and image processing.