Automatic Graph-Based Modeling of Brain Microvessels Captured With Two-Photon Microscopy

Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation

Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. However, achieving high pixel-wise accuracy, complete topology structure and robustness to various contrast variations are critical and challenging, and most existing methods focus only on achieving one or two of these aspects. In this paper, we present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach. Specifically, we compute a multiscale affinity field for each pixel, capturing its semantic relationships with neighboring pixels in the predicted mask image. This field represents the local geometry of vessel segments of different sizes, allowing us to learn spatial- and scale-aware adaptive weights to strengthen vessel features. We evaluate our AFN on four different types of vascular datasets: X-ray angiography coronary vessel dataset (XCAD), portal vein dataset (PV), digital subtraction angiography cerebrovascular vessel dataset (DSA) and retinal vessel dataset (DRIVE). Extensive experimental results demonstrate that our AFN outperforms the state-of-the-art methods in terms of both higher accuracy and topological metrics, while also being more robust to various contrast changes.

Cortical microvascular blood flow velocity mapping by combining dynamic light scattering optical coherence tomography and two-photon microscopy

Significance

The accurate large-scale mapping of cerebral microvascular blood flow velocity is crucial for a better understanding of cerebral blood flow (CBF) regulation. Although optical imaging techniques enable both high-resolution microvascular angiography and fast absolute CBF velocity measurements in the mouse cortex, they usually require different imaging techniques with independent system configurations to maximize their performances. Consequently, it is still a challenge to accurately combine functional and morphological measurements to co-register CBF speed distribution from hundreds of microvessels with high-resolution microvascular angiograms.

Aim

We propose a data acquisition and processing framework to co-register a large set of microvascular blood flow velocity measurements from dynamic light scattering optical coherence tomography (DLS-OCT) with the corresponding microvascular angiogram obtained using two-photon microscopy (2PM).

Approach

We used DLS-OCT to first rapidly acquire a large set of microvascular velocities through a sealed cranial window in mice and then to acquire high-resolution microvascular angiograms using 2PM. The acquired data were processed in three steps: (i) 2PM angiogram coregistration with the DLS-OCT angiogram, (ii) 2PM angiogram segmentation and graphing, and (iii) mapping of the CBF velocities to the graph representation of the 2PM angiogram.

Results

We implemented the developed framework on the three datasets acquired from the mice cortices to facilitate the coregistration of the large sets of DLS-OCT flow velocity measurements with 2PM angiograms. We retrieved the distributions of red blood cell velocities in arterioles, venules, and capillaries as a function of the branching order from precapillary arterioles and postcapillary venules from more than 1000 microvascular segments.

Conclusions

The proposed framework may serve as a useful tool for quantitative analysis of large microvascular datasets obtained by OCT and 2PM in studies involving normal brain functioning, progression of various diseases, and numerical modeling of the oxygen advection and diffusion in the realistic microvascular networks.

Toward next-generation endoscopes integrating biomimetic video systems, nonlinear optical microscopy, and deep learning

According to the World Health Organization, the proportion of the world's population over 60 years will approximately double by 2050. This progressive increase in the elderly population will lead to a dramatic growth of age-related diseases, resulting in tremendous pressure on the sustainability of healthcare systems globally. In this context, finding more efficient ways to address cancers, a set of diseases whose incidence is correlated with age, is of utmost importance. Prevention of cancers to decrease morbidity relies on the identification of precursor lesions before the onset of the disease, or at least diagnosis at an early stage. In this article, after briefly discussing some of the most prominent endoscopic approaches for gastric cancer diagnostics, we review relevant progress in three emerging technologies that have significant potential to play pivotal roles in next-generation endoscopy systems: biomimetic vision (with special focus on compound eye cameras), non-linear optical microscopies, and Deep Learning. Such systems are urgently needed to enhance the three major steps required for the successful diagnostics of gastrointestinal cancers: detection, characterization, and confirmation of suspicious lesions. In the final part, we discuss challenges that lie en route to translating these technologies to next-generation endoscopes that could enhance gastrointestinal imaging, and depict a possible configuration of a system capable of (i) biomimetic endoscopic vision enabling easier detection of lesions, (ii) label-free in vivo tissue characterization, and (iii) intelligently automated gastrointestinal cancer diagnostic.

Ultraliser: a framework for creating multiscale, high-fidelity and geometrically realistic 3D models for in silico neuroscience

Ultraliser is a neuroscience-specific software framework capable of creating accurate and biologically realistic 3D models of complex neuroscientific structures at intracellular (e.g. mitochondria and endoplasmic reticula), cellular (e.g. neurons and glia) and even multicellular scales of resolution (e.g. cerebral vasculature and minicolumns). Resulting models are exported as triangulated surface meshes and annotated volumes for multiple applications in in silico neuroscience, allowing scalable supercomputer simulations that can unravel intricate cellular structure-function relationships. Ultraliser implements a high-performance and unconditionally robust voxelization engine adapted to create optimized watertight surface meshes and annotated voxel grids from arbitrary non-watertight triangular soups, digitized morphological skeletons or binary volumetric masks. The framework represents a major leap forward in simulation-based neuroscience, making it possible to employ high-resolution 3D structural models for quantification of surface areas and volumes, which are of the utmost importance for cellular and system simulations. The power of Ultraliser is demonstrated with several use cases in which hundreds of models are created for potential application in diverse types of simulations. Ultraliser is publicly released under the GNU GPL3 license on GitHub (BlueBrain/Ultraliser).

SIGNIFICANCE

There is crystal clear evidence on the impact of cell shape on its signaling mechanisms. Structural models can therefore be insightful to realize the function; the more realistic the structure can be, the further we get insights into the function. Creating realistic structural models from existing ones is challenging, particularly when needed for detailed subcellular simulations. We present Ultraliser, a neuroscience-dedicated framework capable of building these structural models with realistic and detailed cellular geometries that can be used for simulations.

Multi-perspective label based deep learning framework for cerebral vasculature segmentation in whole-brain fluorescence images

The popularity of fluorescent labelling and mesoscopic optical imaging techniques enable the acquisition of whole mammalian brain vasculature images at capillary resolution. Segmentation of the cerebrovascular network is essential for analyzing the cerebrovascular structure and revealing the pathogenesis of brain diseases. Existing deep learning methods use a single type of annotated labels with the same pixel weight to train the neural network and segment vessels. Due to the variation in the shape, density and brightness of vessels in whole-brain fluorescence images, it is difficult for a neural network trained with a single type of label to segment all vessels accurately. To address this problem, we proposed a deep learning cerebral vasculature segmentation framework based on multi-perspective labels. First, the pixels in the central region of thick vessels and the skeleton region of vessels were extracted separately using morphological operations based on the binary annotated labels to generate two different labels. Then, we designed a three-stage 3D convolutional neural network containing three sub-networks, namely thick-vessel enhancement network, vessel skeleton enhancement network and multi-channel fusion segmentation network. The first two sub-networks were trained by the two labels generated in the previous step, respectively, and pre-segmented the vessels. The third sub-network was responsible for fusing the pre-segmented results to precisely segment the vessels. We validated our method on two mouse cerebral vascular datasets generated by different fluorescence imaging modalities. The results showed that our method outperforms the state-of-the-art methods, and the proposed method can be applied to segment the vasculature on large-scale volumes.

Challenges and opportunities of integrating imaging and mathematical modelling to interrogate biological processes

Advances in biological imaging have accelerated our understanding of human physiology in both health and disease. As these advances have developed, the opportunities gained by integrating with cutting-edge mathematical models have become apparent yet remain challenging. Combined imaging-modelling approaches provide unprecedented opportunity to correlate data on tissue architecture and function, across length and time scales, to better understand the mechanisms that underpin fundamental biology and also to inform clinical decisions. Here we discuss the opportunities and challenges of such approaches, providing literature examples across a range of organ systems. Given the breadth of the field we focus on the intersection of continuum modelling and in vivo imaging applied to the vasculature and blood flow, though our rationale and conclusions extend widely. We propose three key research pillars (image acquisition, image processing, mathematical modelling) and present their respective advances as well as future opportunity via better integration. Multidisciplinary efforts that develop imaging and modelling tools concurrently, and share them open-source with the research community, provide exciting opportunity for advancing these fields.

Segmentation-Less, Automated, Vascular Vectorization

Recent advances in two-photon fluorescence microscopy (2PM) have allowed large scale imaging and analysis of blood vessel networks in living mice. However, extracting network graphs and vector representations for the dense capillary bed remains a bottleneck in many applications. Vascular vectorization is algorithmically difficult because blood vessels have many shapes and sizes, the samples are often unevenly illuminated, and large image volumes are required to achieve good statistical power. State-of-the-art, three-dimensional, vascular vectorization approaches often require a segmented (binary) image, relying on manual or supervised-machine annotation. Therefore, voxel-by-voxel image segmentation is biased by the human annotator or trainer. Furthermore, segmented images oftentimes require remedial morphological filtering before skeletonization or vectorization. To address these limitations, we present a vectorization method to extract vascular objects directly from unsegmented images without the need for machine learning or training. The Segmentation-Less, Automated, Vascular Vectorization (SLAVV) source code in MATLAB is openly available on GitHub. This novel method uses simple models of vascular anatomy, efficient linear filtering, and vector extraction algorithms to remove the image segmentation requirement, replacing it with manual or automated vector classification. Semi-automated SLAVV is demonstrated on three in vivo 2PM image volumes of microvascular networks (capillaries, arterioles and venules) in the mouse cortex. Vectorization performance is proven robust to the choice of plasma- or endothelial-labeled contrast, and processing costs are shown to scale with input image volume. Fully-automated SLAVV performance is evaluated on simulated 2PM images of varying quality all based on the large (1.4×0.9×0.6 mm3 and 1.6×108 voxel) input image. Vascular statistics of interest (e.g. volume fraction, surface area density) calculated from automatically vectorized images show greater robustness to image quality than those calculated from intensity-thresholded images.

A simulation study investigating potential diffusion-based MRI signatures of microstrokes

Recent studies suggested that cerebrovascular micro-occlusions, i.e. microstokes, could lead to ischemic tissue infarctions and cognitive deficits. Due to their small size, identifying measurable biomarkers of these microvascular lesions remains a major challenge. This work aims to simulate potential MRI signatures combining arterial spin labeling (ASL) and multi-directional diffusion-weighted imaging (DWI). Driving our hypothesis are recent observations demonstrating a radial reorientation of microvasculature around the micro-infarction locus during recovery in mice. Synthetic capillary beds, randomly- and radially-oriented, and optical coherence tomography (OCT) angiograms, acquired in the barrel cortex of mice (n = 5) before and after inducing targeted photothrombosis, were analyzed. Computational vascular graphs combined with a 3D Monte-Carlo simulator were used to characterize the magnetic resonance (MR) response, encompassing the effects of magnetic field perturbations caused by deoxyhemoglobin, and the advection and diffusion of the nuclear spins. We quantified the minimal intravoxel signal loss ratio when applying multiple gradient directions, at varying sequence parameters with and without ASL. With ASL, our results demonstrate a significant difference (p < 0.05) between the signal-ratios computed at baseline and 3 weeks after photothrombosis. The statistical power further increased (p < 0.005) using angiograms measured at week 4. Without ASL, no reliable signal change was found. We found that higher ratios, and accordingly improved significance, were achieved at lower magnetic field strengths (e.g., B0 = 3T) and shorter echo time TE (< 16 ms). Our simulations suggest that microstrokes might be characterized through ASL-DWI sequence, providing necessary insights for posterior experimental validations, and ultimately, future translational trials.

It's clearly the heart! Optical transparency, cardiac tissue imaging, and computer modelling

Recent developments in clearing and microscopy enable 3D imaging with cellular resolution up to the whole organ level. These methods have been used extensively in neurobiology, but their uptake in other fields has been much more limited. Application of this approach to the human heart and effective use of the data acquired present challenges of scale and complexity. Four interlinked issues need to be addressed: 1) efficient clearing and labelling of heart tissue, 2) fast microscopic imaging of human-scale samples, 3) handling and processing of multi-terabyte 3D images, and 4) extraction of structural information in computationally tractable structure-based models of cardiac function. Preliminary studies show that each of these requirements can be achieved with the appropriate application and development of existing technologies.

A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy

Ultrasound Localization Microscopy (ULM) can resolve the microvascular bed down to a few micrometers. To achieve such performance, microbubble contrast agents must perfuse the entire microvascular network. Microbubbles are then located individually and tracked over time to sample individual vessels, typically over hundreds of thousands of images. To overcome the fundamental limit of diffraction and achieve a dense reconstruction of the network, low microbubble concentrations must be used, which leads to acquisitions lasting several minutes. Conventional processing pipelines are currently unable to deal with interference from multiple nearby microbubbles, further reducing achievable concentrations. This work overcomes this problem by proposing a Deep Learning approach to recover dense vascular networks from ultrasound acquisitions with high microbubble concentrations. A realistic mouse brain microvascular network, segmented from 2-photon microscopy, was used to train a three-dimensional convolutional neural network (CNN) based on a V-net architecture. Ultrasound data sets from multiple microbubbles flowing through the microvascular network were simulated and used as ground truth to train the 3D CNN to track microbubbles. The 3D-CNN approach was validated in silico using a subset of the data and in vivo in a rat brain. In silico, the CNN reconstructed vascular networks with higher precision (81%) than a conventional ULM framework (70%). In vivo, the CNN could resolve micro vessels as small as 10 μ m with an improvement in resolution when compared against a conventional approach.

Sparse channel sampling for ultrasound localization microscopy (SPARSE-ULM)

Ultrasound localization microscopy (ULM) has recently enabled the mapping of the cerebral vasculaturein vivowith a resolution ten times smaller than the wavelength used, down to ten microns. However, with frame rates up to 20000 frames per second, this method requires large amount of data to be acquired, transmitted, stored, and processed. The transfer rate is, as of today, one of the main limiting factors of this technology. Herein, we introduce a novel reconstruction framework to decrease this quantity of data to be acquired and the complexity of the required hardware by randomly subsampling the channels of a linear probe. Method performance evaluation as well as parameters optimization were conductedin silicousing the SIMUS simulation software in an anatomically realistic phantom and then compared toin vivoacquisitions in a rat brain after craniotomy. Results show that reducing the number of active elements deteriorates the signal-to-noise ratio and could lead to false microbubbles detections but has limited effect on localization accuracy. In simulation, the false positive rate on microbubble detection deteriorates from 3.7% for 128 channels in receive and 7 steered angles to 11% for 16 channels and 7 angles. The average localization accuracy ranges from 10.6μm and 9.93μm for 16 channels/3 angles and 128 channels/13 angles respectively. These results suggest that a compromise can be found between the number of channels and the quality of the reconstructed vascular network and demonstrate feasibility of performing ULM with a reduced number of channels in receive, paving the way for low-cost devices enabling high-resolution vascular mapping.

Anatomical Modeling of Brain Vasculature in Two-Photon Microscopy by Generalizable Deep Learning

Objective and Impact Statement. Segmentation of blood vessels from two-photon microscopy (2PM) angiograms of brains has important applications in hemodynamic analysis and disease diagnosis. Here, we develop a generalizable deep learning technique for accurate 2PM vascular segmentation of sizable regions in mouse brains acquired from multiple 2PM setups. The technique is computationally efficient, thus ideal for large-scale neurovascular analysis. Introduction. Vascular segmentation from 2PM angiograms is an important first step in hemodynamic modeling of brain vasculature. Existing segmentation methods based on deep learning either lack the ability to generalize to data from different imaging systems or are computationally infeasible for large-scale angiograms. In this work, we overcome both these limitations by a method that is generalizable to various imaging systems and is able to segment large-scale angiograms. Methods. We employ a computationally efficient deep learning framework with a loss function that incorporates a balanced binary-cross-entropy loss and total variation regularization on the network’s output. Its effectiveness is demonstrated on experimentally acquired in vivo angiograms from mouse brains of dimensions up to 808×808×702 μm. Results. To demonstrate the superior generalizability of our framework, we train on data from only one 2PM microscope and demonstrate high-quality segmentation on data from a different microscope without any network tuning. Overall, our method demonstrates 10× faster computation in terms of voxels-segmented-per-second and 3× larger depth compared to the state-of-the-art. Conclusion. Our work provides a generalizable and computationally efficient anatomical modeling framework for brain vasculature, which consists of deep learning-based vascular segmentation followed by graphing. It paves the way for future modeling and analysis of hemodynamic response at much greater scales that were inaccessible before.

Laplacian Flow Dynamics on Geometric Graphs for Anatomical Modeling of Cerebrovascular Networks

Generating computational anatomical models of cerebrovascular networks is vital for improving clinical practice and understanding brain oxygen transport. This is achieved by extracting graph-based representations based on pre-mapping of vascular structures. Recent graphing methods can provide smooth vessels trajectories and well-connected vascular topology. However, they require water-tight surface meshes as inputs. Furthermore, adding vessels radii information on their graph compartments restricts their alignment along vascular centerlines. Here, we propose a novel graphing scheme that works with relaxed input requirements and intrinsically captures vessel radii information. The proposed approach is based on deforming geometric graphs constructed within vascular boundaries. Under a laplacian optimization framework, we assign affinity weights on the initial geometry that drives its iterative contraction toward vessels centerlines. We present a mechanism to decimate graph structure at each run and a convergence criterion to stop the process. A refinement technique is then introduced to obtain final vascular models. Our implementation is available on https://github.com/Damseh/VascularGraph. We benchmarked our results with that obtained using other efficient and state-of-the-art graphing schemes, validating on both synthetic and real angiograms acquired with different imaging modalities. The experiments indicate that the proposed scheme produces the lowest geometric and topological error rates on various angiograms. Furthermore, it surpasses other techniques in providing representative models that capture all anatomical aspects of vascular structures.

Anatomical Modeling of Brain Vasculature in Two-Photon Microscopy by Generalizable Deep Learning

Objective and Impact Statement. Segmentation of blood vessels from two-photon microscopy (2PM) angiograms of brains has important applications in hemodynamic analysis and disease diagnosis. Here, we develop a generalizable deep learning technique for accurate 2PM vascular segmentation of sizable regions in mouse brains acquired from multiple 2PM setups. The technique is computationally efficient, thus ideal for large-scale neurovascular analysis. Introduction. Vascular segmentation from 2PM angiograms is an important first step in hemodynamic modeling of brain vasculature. Existing segmentation methods based on deep learning either lack the ability to generalize to data from different imaging systems or are computationally infeasible for large-scale angiograms. In this work, we overcome both these limitations by a method that is generalizable to various imaging systems and is able to segment large-scale angiograms. Methods. We employ a computationally efficient deep learning framework with a loss function that incorporates a balanced binary-cross-entropy loss and total variation regularization on the network's output. Its effectiveness is demonstrated on experimentally acquired in vivo angiograms from mouse brains of dimensions up to 808 × 808 × 702   μ m . Results. To demonstrate the superior generalizability of our framework, we train on data from only one 2PM microscope and demonstrate high-quality segmentation on data from a different microscope without any network tuning. Overall, our method demonstrates 10× faster computation in terms of voxels-segmented-per-second and 3× larger depth compared to the state-of-the-art. Conclusion. Our work provides a generalizable and computationally efficient anatomical modeling framework for brain vasculature, which consists of deep learning-based vascular segmentation followed by graphing. It paves the way for future modeling and analysis of hemodynamic response at much greater scales that were inaccessible before.

Interactive visualization and analysis of morphological skeletons of brain vasculature networks with VessMorphoVis

MOTIVATION

Accurate morphological models of brain vasculature are key to modeling and simulating cerebral blood flow in realistic vascular networks. This in silico approach is fundamental to revealing the principles of neurovascular coupling. Validating those vascular morphologies entails performing certain visual analysis tasks that cannot be accomplished with generic visualization frameworks. This limitation has a substantial impact on the accuracy of the vascular models employed in the simulation.

RESULTS

We present VessMorphoVis, an integrated suite of toolboxes for interactive visualization and analysis of vast brain vascular networks represented by morphological graphs segmented originally from imaging or microscopy stacks. Our workflow leverages the outstanding potentials of Blender, aiming to establish an integrated, extensible and domain-specific framework capable of interactive visualization, analysis, repair, high-fidelity meshing and high-quality rendering of vascular morphologies. Based on the initial feedback of the users, we anticipate that our framework will be an essential component in vascular modeling and simulation in the future, filling a gap that is at present largely unfulfilled.

AVAILABILITY AND IMPLEMENTATION

VessMorphoVis is freely available under the GNU public license on Github at https://github.com/BlueBrain/VessMorphoVis. The morphology analysis, visualization, meshing and rendering modules are implemented as an add-on for Blender 2.8 based on its Python API (application programming interface). The add-on functionality is made available to users through an intuitive graphical user interface, as well as through exhaustive configuration files calling the API via a feature-rich command line interface running Blender in background mode.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A Pilot Study Investigating Changes in Capillary Hemodynamics and Its Modulation by Exercise in the APP-PS1 Alzheimer Mouse Model

Dysfunction in neurovascular coupling that results in a mismatch between cerebral blood flow and neuronal activity has been suggested to play a key role in the pathogenesis of Alzheimer's disease (AD). Meanwhile, physical exercise is a powerful approach for maintaining cognitive health and could play a preventive role against the progression of AD. Given the fundamental role of capillaries in oxygen transport to tissue, our pilot study aimed to characterize changes in capillary hemodynamics with AD and AD supplemented by exercise. Exploiting two-photon microscopy, intrinsic signal optical imaging, and magnetic resonance imaging, we found hemodynamic alterations and lower vascular density with AD that were reversed by exercise. We further observed that capillary properties were branch order-dependent and that stimulation-evoked changes were attenuated with AD but increased by exercise. Our study provides novel indications into cerebral microcirculatory disturbances with AD and the modulating role of voluntary exercise on these alterations.