Identifying Naturalistic Movies from Human Brain Activity with High-Density Diffuse Optical Tomography

Modern neuroimaging modalities, particularly functional MRI (fMRI), can decode detailed human experiences. Thousands of viewed images can be identified or classified, and sentences can be reconstructed. Decoding paradigms often leverage encoding models that reduce the stimulus space into a smaller yet generalizable feature set. However, the neuroimaging devices used for detailed decoding are non-portable, like fMRI, or invasive, like electrocorticography, excluding application in naturalistic use. Wearable, non-invasive, but lower-resolution devices such as electroencephalography and functional near-infrared spectroscopy (fNIRS) have been limited to decoding between stimuli used during training. Herein we develop and evaluate model-based decoding with high-density diffuse optical tomography (HD-DOT), a higher-resolution expansion of fNIRS with demonstrated promise as a surrogate for fMRI. Using a motion energy model of visual content, we decoded the identities of novel movie clips outside the training set with accuracy far above chance for single-trial decoding. Decoding was robust to modulations of testing time window, different training and test imaging sessions, hemodynamic contrast, and optode array density. Our results suggest that HD-DOT can translate detailed decoding into naturalistic use.

Comprehensive workflow and its validation for simulating diffuse speckle statistics for optical blood flow measurements

Diffuse optical methods including speckle contrast optical spectroscopy and tomography (SCOS and SCOT), use speckle contrast (κ) to measure deep blood flow. In order to design practical systems, parameters such as signal-to-noise ratio (SNR) and the effects of limited sampling of statistical quantities, should be considered. To that end, we have developed a method for simulating speckle contrast signals including effects of detector noise. The method was validated experimentally, and the simulations were used to study the effects of physical and experimental parameters on the accuracy and precision of κ. These results revealed that systematic detector effects resulted in decreased accuracy and precision of κ in the regime of low detected signals. The method can provide guidelines for the design and usage of SCOS and/or SCOT instruments.

Anatomical Modeling and Optimization of Speckle Contrast Optical Tomography

Traditional methods for mapping cerebral blood flow (CBF), such as positron emission tomography and magnetic resonance imaging, offer only isolated snapshots of CBF due to scanner logistics. Speckle contrast optical tomography (SCOT) is a promising optical technique for mapping CBF. However, while SCOT has been established in mice, the method has not yet been demonstrated in humans - partly due to a lack of anatomical reconstruction methods and uncertainty over the optimal design parameters. Herein we develop SCOT reconstruction methods that leverage MRI-based anatomical head models and finite-element modeling of the SCOT forward problem (NIRFASTer). We then simulate SCOT for CBF perturbations to evaluate sensitivity of imaging performance to exposure time and SD-distances. We find image resolution comparable to intensity-based diffuse optical tomography at superficial cortical tissue depth (~1.5 cm). Localization errors can be reduced by including longer SD-measurements. With longer exposure times speckle contrast decreases, however, noise decreases faster, resulting in a net increase in SNR. Specifically, extending exposure time from 10μs to 10ms increased SCOT SNR by 1000X. Overall, our modeling methods provide anatomically-based image reconstructions that can be used to evaluate a broad range of tissue conditions, measurement parameters, and noise sources and inform SCOT system design.

Ultra-high density imaging arrays for diffuse optical tomography of human brain improve resolution, signal-to-noise, and information decoding

Functional magnetic resonance imaging (fMRI) has dramatically advanced non-invasive human brain mapping and decoding. Functional near-infrared spectroscopy (fNIRS) and high-density diffuse optical tomography (HD-DOT) non-invasively measure blood oxygen fluctuations related to brain activity, like fMRI, at the brain surface, using more-lightweight equipment that circumvents ergonomic and logistical limitations of fMRI. HD-DOT grids have smaller inter-optode spacing (∼13 mm) than sparse fNIRS (∼30 mm) and therefore provide higher image quality, with spatial resolution ∼1/2 that of fMRI. Herein, simulations indicated reducing inter-optode spacing to 6.5 mm would further improve image quality and noise-resolution tradeoff, with diminishing returns below 6.5 mm. We then constructed an ultra-high-density DOT system (6.5-mm spacing) with 140 dB dynamic range that imaged stimulus-evoked activations with 30-50% higher spatial resolution and repeatable multi-focal activity with excellent agreement with participant-matched fMRI. Further, this system decoded visual stimulus position with 19-35% lower error than previous HD-DOT, throughout occipital cortex.

Multi-mode fiber-based speckle contrast optical spectroscopy: analysis of speckle statistics

Speckle contrast optical spectroscopy/tomography (SCOS/T) provides a real-time, non-invasive, and cost-efficient optical imaging approach to mapping of cerebral blood flow. By measuring many speckles (n>>10), SCOS/T has an increased signal-to-noise ratio relative to diffuse correlation spectroscopy, which measures one or a few speckles. However, the current free-space SCOS/T designs are not ideal for large field-of-view imaging in humans because the curved head contour cannot be readily imaged with a single flat sensor and hair obstructs optical access. Herein, we evaluate the feasibility of using cost-efficient multi-mode fiber (MMF) bundles for use in SCOS/T systems. One challenge with speckle contrast measurements is the potential for confounding noise sources (e.g., shot noise, readout noise) which contribute to the standard deviation measure and corrupt the speckle contrast measure that is central to the SCOS/T systems. However, for true speckle measurements, the histogram of pixel intensities from light interference follows a non-Gaussian distribution, specifically a gamma distribution with non-zero skew, whereas most noise sources have pixel intensity distributions that are Gaussian. By evaluating speckle data from static and dynamic targets imaged through an MMF, we use histograms and statistical analysis of pixel histograms to evaluate whether the statistical properties of the speckles are retained. We show that flow-based speckle can be distinguished from static speckle and from sources of system noise through measures of skew in the pixel intensity histograms. Finally, we illustrate in humans that MMF bundles relay blood flow information.

Electrocardiographic textbooks based on template hearts warped using ultrasonic images

Advances in technology make the application of sophisticated approaches to assessing electrical condition of the heart practical. Estimates of cardiac electrical features inferred from body-surface electrocardiographic (ECG) maps are now routinely found in a clinical setting, but errors in those inverse solutions are especially sensitive to the accuracy of heart model geometry and placement within the torso. The use of a template heart model allows for accurate generation of individualized heart models and also permits effective comparison of inferred electrical features among multiple subjects. A collection of features mapped onto a common template forms a textbook of anatomically specific ECG variability. Our template warping process to individualize heart models based on a template heart uses ultrasonic images of the heart from a conventional, phased-array system. We chose ultrasound because it is nonionizing, less expensive, and more convenient than MR or CT imaging. To find the orientation and position in the torso model of each image, we calibrated the ultrasound probe by imaging a custom phantom consisting of multiple N-fiducials and computing a transformation between ultrasound coordinates and measurements of the torso surface. The template heart was warped using a mapping of corresponding landmarks identified on both the template and the ultrasonic images. Accuracy of the method is limited by patient movement, tracking error, and image analysis. We tested our approach on one normal control and one obese diabetic patient using the mixed-boundary-value inverse method and compared results from both on the template heart. We believe that our novel textbook approach using anatomically specific heart and torso models will facilitate the identification of electrophysiological biomarkers of cardiac dysfunction. Because the necessary data can be acquired and analyzed within about 30 min, this framework has the potential for becoming a routine clinical procedure.

3-D in vitro estimation of temperature using the change in backscattered ultrasonic energy

Temperature imaging with a non-invasive modality to monitor the heating of tumors during hyperthermia treatment is an attractive alternative to sparse invasive measurement. Previously, we predicted monotonic changes in backscattered energy (CBE) of ultrasound with temperature for certain sub-wavelength scatterers. We also measured CBE values similar to our predictions in bovine liver, turkey breast muscle, and pork rib muscle in 2-D in vitro studies and in nude mice during 2-D in vivo studies. To extend these studies to three dimensions, we compensated for motion and measured CBE in turkey breast muscle. 3-D data sets were assembled from images formed by a phased-array imager with a 7.5-MHz linear probe moved in 0.6-mm steps in elevation during uniform heating from 37 to 45 degrees C in 0.5 degrees C increments. We used cross-correlation as a similarity measure in RF signals to automatically track feature displacement as a function of temperature. Feature displacement was non-rigid. Envelopes of image regions, compensated for non-rigid motion, were found with the Hilbert transform then smoothed with a 3 x 3 running average filter before forming the backscattered energy at each pixel. CBE in 3-D motion-compensated images was nearly linear with an average sensitivity of 0.30 dB/ degrees C. 3-D estimation of temperature in separate tissue regions had errors with a maximum standard deviation of about 0.5 degrees C over 1-cm(3) volumes. Success of CBE temperature estimation based on 3-D non-rigid tracking and compensation for real and apparent motion of image features could serve as the foundation for the eventual generation of 3-D temperature maps in soft tissue in a non-invasive, convenient, and low-cost way in clinical hyperthermia.

Non-invasive estimation of hyperthermia temperatures with ultrasound

Ultrasound is an attractive modality for temperature monitoring because it is non-ionizing, convenient, inexpensive and has relatively simple signal processing requirements. This modality may be useful for temperature estimation if a temperature-dependent ultrasonic parameter can be identified, measured and calibrated. The most prominent methods for using ultrasound as a non-invasive thermometer exploit either (1) echo shifts due to changes in tissue thermal expansion and speed of sound (SOS), (2) variation in the attenuation coefficient or (3) change in backscattered energy from tissue inhomogeneities. The use of echo shifts has received the most attention in the last decade. By tracking scattering volumes and measuring the time shift of received echoes, investigators have been able to predict the temperature from a region of interest both theoretically and experimentally in phantoms, in isolated tissue regions in vitro and preliminary in vivo studies. A limitation of this method for general temperature monitoring is that prior knowledge of both SOS and thermal-expansion coefficients is necessary. Acoustic attenuation is dependent on temperature, but with significant changes occurring only at temperatures above 50 degrees C, which may lead to its use in thermal ablation therapies. Minimal change in attenuation, however, below this temperature range reduces its attractiveness for use in clinical hyperthermia. Models and measurements of the change in backscattered energy suggest that, over the clinical hyperthermia temperature range, changes in backscattered energy are dependent on the properties of individual scatterers or scattering regions. Calibration of the backscattered energy from different tissue regions is an important goal of this approach. All methods must be able to cope with motion of the image features on which temperature estimates are based. A crucial step in identifying a viable ultrasonic approach to temperature estimation is its performance during in vivo tests.

A simulation model for ultrasonic temperature imaging using change in backscattered energy

Ultrasound backscattered from tissue has previously been shown theoretically and experimentally to change predictably with temperature in the hyperthermia range, i.e., 37 degrees C to 45 degrees C, motivating use of the change in backscattered ultrasonic energy (CBE) for ultrasonic thermometry. Our earlier theoretical model predicts that CBE from an individual scatterer will be monotonic with temperature, with, e.g., positive change for lipid-based scatterers and negative for aqueous-based scatterers. Experimental results have previously confirmed the presence of these positive and negative changes in one-dimensional ultrasonic signals and in two-dimensional images acquired from in vitro bovine, porcine and turkey tissues. In order to investigate CBE for populations of scatterers, we have developed an ultrasonic image simulation model, including temperature dependence for individual scatterers based on predictions from our theoretical model. CBE computed from images simulated for populations of randomly distributed scatterers behaves similarly to experimental results, with monotonic variation for individual pixel measurements and for image regions. Effects on CBE of scatterer type and distribution, size of the image region and signal-to-noise ratio have been examined. This model also provides the basis for future work regarding significant issues relevant to temperature imaging based on ultrasonic CBE such as effects of motion on CBE, limitations of motion-compensation techniques and accuracy of temperature estimation, including tradeoffs between temperature accuracy and available spatial resolution.

Temperature dependence of ultrasonic backscattered energy in motion-compensated images

Noninvasive temperature imaging would enhance the ability to uniformly heat tumors at therapeutic levels. Ultrasound is an attractive modality for this purpose. Previously, we predicted monotonic changes in backscattered energy (CBE) of ultrasound with temperature for certain subwavelength scatterers. We also measured CBE values similar to our predictions in bovine liver, turkey breast, and pork muscle in one dimension (1-D). Those measurements were corrected manually for changes in the axial position of echo signals with temperature. To investigate the effect of temperature on CBE in 2-D, we imaged 1-cm thick samples of bovine liver, turkey breast, and pork muscle during heating in a water bath. Images were formed by a phased-array imager with a 7 MHz linear probe. Using radio frequency (RF) signals permitted the use of cross correlation as a similarity measure for automatic tracking of feature displacement as a function of temperature. Feature displacement across the specimen was nonuniform with typical total displacements of 0.5 mm in both axial and lateral directions. Apparent movement in eight image regions in each specimen was tracked from 37 to 50 degrees C in 0.5 degrees C steps. Envelopes of motion-compensated image regions were found then smoothed with a 3 x 3 running average filter before forming the backscattered energy at each pixel. Our measure of CBE compared means of both the positive and negative changes in the backscattered energy (BE) images. CBE was monotonic and differed by about 4 dB at 50 degrees C from its value at 37 degrees C. Relatively noise-free CBE curves from tissue volumes of less than 1 cm3 supports the use of CBE for temperature estimation.

Estimation of surface pose with a physically-based ultrasonic image model

State-of-the-art approaches to shape analysis in medical images use a variety of sophisticated models for object shape. We have developed an image model that permits the application of these approaches to ultrasonic images, with detailed methods for representing rough surfaces. Our physically-based, probabilistic image model incorporates the combined effects of the system point-spread function (PSF), the tissue microstructure, and the gross tissue shape. At each image pixel, the amplitude mean and variance are computed directly from the model, characterizing the combined influence of shape, microstructure, and system PSF. Calculation of the SNR0 is used to further classify each pixel as Rayleigh- or non-Rayleigh-distributed. This characterization was used here to generate a data likelihood representing any set of images of a given surface by a probability density conditioned on the surface pose, or rotation and translation. The utility of this likelihood was demonstrated by applying maximum likelihood estimation to infer the pose of a cadaveric vertebra from simulated images of its surface. Successful results were achieved using derivative-based optimization algorithms for a data set of only three images. With a quasi-Newton BFGS algorithm, error in 15 of 20 trials was less than 0.4 degrees in rotation and 0.2 mm in translation. Estimation was inaccurate in only 1 of 20 trials. These results illustrate the potential of a physically-based image model in a rigorous approach to image analysis and also serve as an example of quantitative assessment of the model via performance in a specific application.

Detection of the fingerprint of the electrophysiological abnormalities that increase vulnerability to life-threatening ventricular arrhythmias

Reduction of sudden death requires accurate identification of patients at risk for ventricular tachycardia (VT) and effective therapies. The Multicenter Unsustained Tachycardia Trial and Multicenter Automatic Defibrillator Implantation Trials demonstrate that the implantable cardioverter defibrillator impacts favorably on the incidence of VT in patients with myocardial infarction, underscoring the need to detect the electrophysiologic abnormalities required for the development of VT. Methods used for this purpose include: Holter monitoring, ejection fraction, signal-averaged ECG, heart rate variability, T-wave alternans, baroreflex sensitivity, and programmed stimulation. Performance of each method alone has demonstrated high-negative but low-positive predictive values. Recent studies confirm that their use in combination augments performance.A second approach for improving performance has been to reexamine how well each method detects the electrophysiological derangements that lead to VT. Our recent work has focused on the signal-averaged ECG. Judging from transmural maps of ventricular activation during VT and sinus rhythm obtained from patients, late potentials fail to detect completely signals from myocardium responsible for VT. To obviate this limitation we developed an approach based on inferred epicardial potentials in the frequency domain from 190-surface ECGs using individualized heart-torso models. Torso geometry and electrode positions are measured with a 3-armed digitizer. The location of cardiac structures is determined using echocardiography. The pericardial surface is approximated by a sphere that encloses the heart. Epicardial potentials are inferred using the boundary element method with zero-order Tikhonov regularization and the Composite Residual Smoothing Operator over the QRS complex. Studies are underway to determine if analysis of bioelectrical signals enveloping arrhythmogenic tissue improves identification of patients vulnerable to VT.

A physically based, probabilistic model for ultrasonic images incorporating shape, microstructure, and system characteristics

Recent successes with Bayesian methods for analysis of shape in medical images have motivated our development of a shape-based data likelihood for ultrasound, the foundation of which is a computationally feasible, pixel-based, probabilistic image model. Previous probabilistic models for ultrasound generally assume an analytic form, e.g., Rayleigh, Rician, K, Generalized K, etc., then attempt to fit data to the model. Assumptions and intensive computation inherent in such an approach make it unsuitable for our purposes. In the pursuit of a new model, we have described previously a physical model for image formation that incorporates the imaging system characteristics, the surface shape, and the surface microstructure. That physical model was validated via a visual comparison of simulated and actual images of a cadaveric vertebra. In this work, a random phasor sum representation of the physical model provides the basis for a probabilistic form. In contrast to existing probabilistic models, we compute the amplitude mean and variance directly from the physical model. These statistics can be displayed as images to characterize the tissue, but, more importantly, they permit the subsequent assignment of a suitable density function to each pixel for the purposes of constructing a data likelihood. The order of these steps, i.e., first computing the statistics and then assigning a density function, permits the inclusion of the local surface shape, the surface microstructure, and the system characteristics at every image pixel without violating the physical model. Currently, the value of the SNR0, the ratio of the mean to the standard deviation, is used to estimate whether a pixel is Rayleigh- or non-Rayleigh-distributed. This assessment forms the basis for a data likelihood constructed as a product of Rayleigh and Gaussian density functions describing the individual image pixels.

A discrete-scatterer model for ultrasonic images of rough surfaces

Automated analysis of ultrasonic images could be greatly improved with model-based Bayesian methods for image analysis. Such an approach would require an accurate probabilistic image model representing ultrasonic images in terms of the gross shape of underlying anatomical structure. Existing probabilistic models for ultrasonic image data do not adequately incorporate structure shape or system characteristics; thus, a substantially new approach is warranted. Toward that goal, we have developed models for the imaging system and rough surface with the following objectives: (1) accuracy in representation of basic image characteristics such as the texture and intensity, (2) a minimum of computational requirements, and (3) a form that is naturally extendable to an appropriate probabilistic image model. The imaging system was modeled as a linear system with a separable three-dimensional point-spread function with an envelope of Gaussian curves in each dimension. The rough surface was modeled as a collection of discrete scatterers placed on the continuum and parametrized by a surface roughness and scatterer concentration. Models were evaluated by a visual comparison of actual and simulated images of a cadaveric lumbar vertebra. The gross shape of the vertebral surface was estimated from computed tomography images of the vertebra, and simulated images were generated using the models and the gross surface shape. Actual images were registered with the surface and simulated images to within 2 mm. The similarity of the actual and simulated images was quite remarkable considering the simplicity of the models. Differences between the images were less than those between two simulated images separated by 0.4 mm or one-fifth the registration error. Further assessment of the models would require a statistical approach not yet available. The models do, however, provide the basis for the development of a computationally tractable probabilistic image model for image analysis. Such a model will provide the means for a statistical evaluation of the system and surface models.

Three-dimensional imaging with stereotactic ultrasonography

Stereotactic ultrasonography is a technique for determining the position and orientation of B-mode ultrasound images in a reference coordinate system. A technique for constructing three-dimensional (3D) image volumes has been developed that uses this new technology. Given several registered images, a 3D volume is constructed either by a "nearest-neighbor" or a "closest-points" interpolation approach. The resulting volume can be rendered using 3D rendering software. In addition, the voxels in the volume are at known positions allowing determination of position for structures in the volume. Results are shown for various test cases, and applicability to medical imaging applications and stereotactic neurosurgery is discussed.

Frameless stereotactic ultrasonography: method and applications

In stereotactic neurosurgery, computed tomography (CT) and magnetic resonance (MR) images are registered in a coordinate system defined with respect to the skull. By intraoperatively tracking the coordinate position of a surgical instrument, various displays can be formed which show the position of the instrument in the MR and/or CT images. However, the accuracy of this display varies because intracranial structures may shift or warp from their position prior to surgery. Ultrasonic imaging systems provide real-time images of the brain, but structures in these images are difficult to interpret because the images are based on ultrasonic echoes. A method has been developed for the real-time registration of these images. With this registration, software continuously updates a corresponding image constructed from the set of MR and/or CT images used for guidance. By developing this second view of the structures in the ultrasound image, the surgeon can easily interpret the ultrasound image, and it becomes possible to determine the extent of the intra-operative structure shift between the two images.