Semidefinite programming for model-based sensorless adaptive optics

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution - one that can be readily transferred between microscope modalities - has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a "black box", but provided physical insights on internal workings, which could influence future designs.

Adaptive optics in single objective inclined light sheet microscopy enables three-dimensional localization microscopy in adult Drosophila brains

Single-molecule localization microscopy (SMLM) enables the high-resolution visualization of organelle structures and the precise localization of individual proteins. However, the expected resolution is not achieved in tissue as the imaging conditions deteriorate. Sample-induced aberrations distort the point spread function (PSF), and high background fluorescence decreases the localization precision. Here, we synergistically combine sensorless adaptive optics (AO), in-situ 3D-PSF calibration, and a single-objective lens inclined light sheet microscope (SOLEIL), termed (AO-SOLEIL), to mitigate deep tissue-induced deteriorations. We apply AO-SOLEIL on several dSTORM samples including brains of adult Drosophila. We observed a 2x improvement in the estimated axial localization precision with respect to widefield without aberration correction while we used synergistic solution. AO-SOLEIL enhances the overall imaging resolution and further facilitates the visualization of sub-cellular structures in tissue.

The Lattice Geometry of Walsh-Function-Based Adaptive Optics

We show that there is an intrinsic link between the use of Walsh aberration modes in adaptive optics (AO) and the mathematics of lattices. The discrete and binary nature of these modes means that there are infinite combinations of Walsh mode coefficients that can optimally correct the same aberration. Finding such a correction is hence a poorly conditioned optimisation problem that can be difficult to solve. This can be mitigated by confining the AO correction space defined in Walsh mode coefficients to the fundamental Voronoi cell of a lattice. By restricting the correction space in this way, one can ensure there is only one set of Walsh coefficients that corresponds to the optimum correction aberration. This property is used to enable the design of efficient estimation algorithms to solve the inverse problem of finding correction aberrations from a sequence of measurements in a wavefront sensorless AO system. The benefit of this approach is illustrated using a neural-network-based estimator.

Self-calibrated general model-based wavefront sensorless adaptive optics for both point-like and extended objects

The deformable mirror (DM) in conventional model-based wavefront sensorless adaptive optics (WFSless AO) must be calibrated in advance by an additional WFS in order to precisely generate predetermined bias modes with known amplitudes. Although the WFS is unnecessary during correction, it will increase system complexity and may be unavailable in real applications. In this paper, the model-based WFSless AO algorithms, either for point-like or extended objects, are generalized to a unified form and the calibration problem comes down to the measurement of a Gram matrix. We proposed a novel self-calibration procedure to obtain the Gram matrix without using a WFS. The calibrated Gram matrix can be used directly for simultaneous correction if using the influence functions of DM as the bias modes, requiring N+1 images to correct N modes. Alternatively, orthogonal or gradient-orthogonal mirror modes obtained from the eigenvectors of the Gram matrix can be used as the modal basis to implement independent sequential correction that requires 2N images to correct N modes. Simulations and experiments have been done to verify the feasibility of proposed self-calibration and correction methods for both point-like and extended objects in a WFSless AO system.

General approach to precise deformable mirror control

We develop a simple and effective control method for accurate control of deformable mirrors (DMs). For a desired DM surface profile and using batches of observed surface profile data, the proposed method adaptively determines both a DM model (influence matrix) and control actions that produce the desired surface profile with good accuracy. In the first iteration, the developed method estimates a DM influence matrix by solving a multivariable least-squares problem. This matrix is then used to compute the control actions by solving a constrained least-squares problem. Then, the computed actions are randomly perturbed and applied to the DM to generate a new batch of surface profile data. The new data batch is used to estimate a new influence matrix that is then used to re-compute control actions. This procedure is repeated until convergence is achieved. The method is experimentally tested on a Boston Micromachines DM with 140 micro-electronic-mechanical-system actuators. Our experimental results show that the developed control approach can achieve accurate correction despite significant DM nonlinearities. Using only a few control iterations, the developed method is able to produce a surface profile root-mean-square error that varies from 5 - 30 [nm] for most of the tested Zernike wave-front modes without using direct feedback control. These results can additionally be improved by using larger data batches and more iterations or by combining the developed approach with feedback control. Finally, as we experimentally demonstrate, the developed method can be used to estimate a DM model that can effectively be used for a single-step open-loop DM control.

Fast dynamic correction algorithm for model-based wavefront sensorless adaptive optics in extended objects imaging

A major concern for wavefront sensorless adaptive optics (WFSless AO) is how to improve the algorithm's efficiency which is critical for dynamic aberration correction. For extended objects and dynamic aberration, a typical model-based WFSless AO algorithm is called "3N" which uses three image measurements to estimate each aberration mode and then corrects it immediately. The three images include an initial aberrated image and two biased images with deliberately introduced predetermined positive or negative modal aberrations. In this paper, an improved algorithm called "2N" that requires only one biased image is proposed. The reduction of one biased image is achieved by the estimation of a parameter that is considered unknown in the 3N algorithm. It is demonstrated that the 2N algorithm can achieve convergence with less image measurements and have better performance in dynamic correction.

Tunable fluidic lens with a dynamic high-order aberration control

Fluidic lenses based on electrowetting actuation are attractive for their wide focal tuning range, yet are limited by optical aberrations, either intrinsic to the lenses themselves or due to the optical imaging systems in which they are employed. However, the ability to control the meniscus shape that forms the lens refractive surface with a high degree of spatial accuracy will allow correction of and compensation for a wide range of these aberrations. We demonstrate here for what we believe, to the best of our knowledge, is the first time a tunable optofluidic lens controlled by 32 azimuthally placed electrodes for which most aberrations up to the fourth radial Zernike order may be corrected. Using both wavefront sensing and sensorless wavefront estimation techniques, it is shown that focal length tunability with a significant reduction in imaging aberrations and the ability to compensate for externally induced aberrations may be achieved using a single component.

Improved model-based wavefront sensorless adaptive optics for extended objects using N + 2 images

The original model-based wavefront sensorless adaptive optics (WFSless AO) for extended objects uses the low spatial frequency content of images as the metric function and employs 2N + 1 images to correct N Lukosz aberration modes. We propose an improved method that uses the same metric but requires only N + 2 images to correct N aberration modes. The N + 2 method can achieve comparable corrective accuracy but requiring much smaller number of images compared with the 2N + 1 method. The N + 2 method is not only more efficient, but also provides the flexibility of choosing arbitrary basis modes by involving the non-orthogonality between modes in a linear least-squares optimization process. It is demonstrated that the deformable mirror's influence functions modes (IFM), orthogonal mirror modes (OMM) and fitted Zernike modes (FZM) all can be used as modal basis for the N + 2 method to generate biased images. It is also proved that the N + 2 method is robust to image sampling rate and image noise, showing good prospects of applications in various imaging systems.

Wavefront sensorless adaptive optics OCT with the DONE algorithm for in vivo human retinal imaging [Invited]

In this report, which is an international collaboration of OCT, adaptive optics, and control research, we demonstrate the Data-based Online Nonlinear Extremum-seeker (DONE) algorithm to guide the image based optimization for wavefront sensorless adaptive optics (WFSL-AO) OCT for in vivo human retinal imaging. The ocular aberrations were corrected using a multi-actuator adaptive lens after linearization of the hysteresis in the piezoelectric actuators. The DONE algorithm succeeded in drastically improving image quality and the OCT signal intensity, up to a factor seven, while achieving a computational time of 1 ms per iteration, making it applicable for many high speed applications. We demonstrate the correction of five aberrations using 70 iterations of the DONE algorithm performed over 2.8 s of continuous volumetric OCT acquisition. Data acquired from an imaging phantom and in vivo from human research volunteers are presented.

Optical coherence tomography with a 2.8-mm beam diameter and sensorless defocus and astigmatism correction

An optical coherence tomography (OCT) system with a 2.8-mm beam diameter is presented. Sensorless defocus correction can be performed with a Badal optometer and astigmatism correction with a liquid crystal device. OCT B-scans were used in an image-based optimization algorithm for aberration correction. Defocus can be corrected from ? 4.3 ?? D to + 4.3 ?? D and vertical and oblique astigmatism from ? 2.5 ?? D to + 2.5 ?? D . A contrast gain of 6.9 times was measured after aberration correction. In comparison with a 1.3-mm beam diameter OCT system, this concept achieved a 3.7-dB gain in dynamic range on a model retina. Both systems were used to image the retina of a human subject. As the correction of the liquid crystal device can take more than 60 s, the subject’s spectacle prescription was adopted instead. This resulted in a 2.5 times smaller speckle size compared with the standard OCT system. The liquid crystal device for astigmatism correction does not need a high-voltage amplifier and can be operated at 5 V. The correction device is small ( 9 ?? mm × 30 ?? mm × 38 ?? mm ) and can easily be implemented in existing designs for OCT.

Model-based sensor-less wavefront aberration correction in optical coherence tomography

Several sensor-less wavefront aberration correction methods that correct nonlinear wavefront aberrations by maximizing the optical coherence tomography (OCT) signal are tested on an OCT setup. A conventional coordinate search method is compared to two model-based optimization methods. The first model-based method takes advantage of the well-known optimization algorithm (NEWUOA) and utilizes a quadratic model. The second model-based method (DONE) is new and utilizes a random multidimensional Fourier-basis expansion. The model-based algorithms achieve lower wavefront errors with up to ten times fewer measurements. Furthermore, the newly proposed DONE method outperforms the NEWUOA method significantly. The DONE algorithm is tested on OCT images and shows a significantly improved image quality.

Towards model-based adaptive optics optical coherence tomography

The transfer function for optical wavefront aberrations in single-mode fiber based optical coherence tomography is determined. The loss in measured OCT signal due to optical wavefront aberrations is quantified using Fresnel propagation and the calculation of overlap integrals. A distinction is made between a model for a mirror and a scattering medium model. The model predictions are validated with measurements on a mirror and a scattering medium obtained with an adaptive optics optical coherence tomography setup. Furthermore, a one-step defocus correction, based on a single A-scan measurement, is derived from the model and verified. Finally, the pseudo-convex structure of the optical coherence tomography transfer function is validated with the convergence of a hill climbing algorithm. The implications of this model for wavefront sensorless aberration correction are discussed.

Optimization-based wavefront sensorless adaptive optics for multiphoton microscopy

Optical aberrations have detrimental effects in multiphoton microscopy. These effects can be curtailed by implementing model-based wavefront sensorless adaptive optics, which only requires the addition of a wavefront shaping device, such as a deformable mirror (DM) to an existing microscope. The aberration correction is achieved by maximizing a suitable image quality metric. We implement a model-based aberration correction algorithm in a second-harmonic microscope. The tip, tilt, and defocus aberrations are removed from the basis functions used for the control of the DM, as these aberrations induce distortions in the acquired images. We compute the parameters of a quadratic polynomial that is used to model the image quality metric directly from experimental input-output measurements. Finally, we apply the aberration correction by maximizing the image quality metric using the least-squares estimate of the unknown aberration.

Snapshot coherence-gated direct wavefront sensing for multi-photon microscopy

Deep imaging in turbid media such as biological tissue is challenging due to scattering and optical aberrations. Adaptive optics has the potential to compensate the tissue aberrations. We present a wavefront sensing scheme for multi-photon scanning microscopes using the pulsed, near-infrared light reflected back from the sample utilising coherence gating and a confocal pinhole to isolate the light from a layer of interest. By interfering the back-reflected light with a tilted reference beam, we create a fringe pattern with a known spatial carrier frequency in an image of the back-aperture plane of the microscope objective. The wavefront aberrations distort this fringe pattern and thereby imprint themselves at the carrier frequency, which allows us to separate the aberrations in the Fourier domain from low spatial frequency noise. A Fourier analysis of the modulated fringes combined with a virtual Shack-Hartmann sensor for smoothing yields a modal representation of the wavefront suitable for correction. We show results with this method correcting both DM-induced and sample-induced aberrations in rat tail collagen fibres as well as a Hoechst-stained MCF-7 spheroid of cancer cells.

Iterative linear focal-plane wavefront correction

We propose an efficient approximation to the nonlinear phase diversity (PD) method for wavefront reconstruction and correction from intensity measurements with potential of being used in real-time applications. The new iterative linear phase diversity (ILPD) method assumes that the residual phase aberration is small and makes use of a first-order Taylor expansion of the point spread function (PSF), which allows for arbitrary (large) diversities in order to optimize the phase retrieval. For static disturbances, at each step, the residual phase aberration is estimated based on one defocused image by solving a linear least squares problem, and compensated for with a deformable mirror. Due to the fact that the linear approximation does not have to be updated with each correction step, the computational complexity of the method is reduced to that of a matrix-vector multiplication. The convergence of the ILPD correction steps has been investigated and numerically verified. The comparative study that we make demonstrates the improved performance in computational time with no decrease in accuracy with respect to existing methods that also linearize the PSF.

Iterative learning control of a membrane deformable mirror for optimal wavefront correction

We present an iterative learning control (ILC) algorithm for controlling the shape of a membrane deformable mirror (DM). We furthermore give a physical interpretation of the design parameters of the ILC algorithm. On the basis of this insight, we derive a simple tuning procedure for the ILC algorithm that, in practice, guarantees stable and fast convergence of the membrane to the desired shape. In order to demonstrate the performance of the algorithm, we have built an experimental setup that consists of a commercial membrane DM, a wavefront sensor, and a real-time controller. The experimental results show that, by using the ILC algorithm, we are able to achieve a relatively small error between the real and desired shape of the DM while at the same time we are able to control the saturation of the actuators. Moreover, we show that the ILC algorithm outperforms other control algorithms available in the literature.