Electrically tunable fluidic lens imaging system for laparoscopic fluorescence-guided surgery

First-in-man study of the PSMA Minibody IR800-IAB2M for molecularly targeted intraoperative fluorescence guidance during radical prostatectomy

PURPOSE

Prostate-specific membrane antigen (PSMA) is increasingly used to image prostate cancer in clinical practice. We sought to develop and test a humanised PSMA minibody IAB2M conjugated to the fluorophore IRDye 800CW-NHS ester in men undergoing robot-assisted laparoscopic radical prostatectomy (RARP) to image prostate cancer cells during surgery.

METHODS

The minibody was evaluated pre-clinically using PSMA positive/negative xenograft models, following which 23 men undergoing RARP between 2018 and 2020 received between 2.5 mg and 20 mg of IR800-IAB2M intravenously, at intervals between 24 h and 17 days prior to surgery. At every step of the procedure, the prostate, pelvic lymph node chains and extra-prostatic surrounding tissue were imaged with a dual Near-infrared (NIR) and white light optical platform for fluorescence in vivo and ex vivo. Histopathological evaluation of intraoperative and postoperative microscopic fluorescence imaging was undertaken for verification.

RESULTS

Twenty-three patients were evaluated to optimise both the dose of the reagent and the interval between injection and surgery and secure the best possible specificity of fluorescence images. Six cases are presented in detail as exemplars. Overall sensitivity and specificity in detecting non-lymph-node extra-prostatic cancer tissue were 100% and 65%, and 64% and 64% respectively for lymph node positivity. There were no side-effects associated with administration of the reagent.

CONCLUSION

Intraoperative imaging of prostate cancer tissue is feasible and safe using IR800-IAB2M. Further evaluation is underway to assess the benefit of using the technique in improving completion of surgical excision during RARP.

REGISTRATION

ISCRCTN10046036: https://www.isrctn.com/ISRCTN10046036 .

Tunable liquid lens for three-photon excitation microscopy

We demonstrate a novel electrowetting liquid combination using a room temperature ionic liquid (RTIL) and a nonpolar liquid, 1-phenyl-1-cyclohexene (PCH) suitable for focus-tunable 3-photon microscopy. We show that both liquids have over 90% transmission at 1300 nm over a 1.1 mm pathlength and an index of refraction contrast of 0.123. A lens using these liquids can be tuned from a contact angle of 133 to 48° with applied voltages of 0 and 60 V, respectively. Finally, a three-photon imaging system including an RTIL electrowetting lens was used to image a mouse brain slice. Axial scans taken with an electrowetting lens show excellent agreement with images acquired using a mechanically scanned objective.

Three-phase electrowetting liquid lens with deformable liquid iris

Inspired by the arrangement of iris and crystalline lens in human eyes, we propose a three-phase electrowetting liquid lens with a deformable liquid iris (TELL-DLI). The proposed electrowetting liquid lens has three-phase fluid: air, conductive liquid, and dyed insulating liquid. The insulating liquid is distributed on the inner wall of the chamber in a ring shape. By applying voltage, the contact angle is changed, so that the dyed insulating liquid contracts towards the center, which is similar to the contraction of iris and the function of crystalline lens muscle in human eyes. The variation range of focal length is from -451.9 mm to -107.9 mm. The variation range of the aperture is from 4.89 mm to 0.6 mm. Under the step voltage of 200 V, the TELL-DLI can be switched between the maximum aperture state and the zero aperture state, and the switching time is ∼150/200 ms. Because of the discrete electrodes, TELL-DLI can regionally control the shape and position of the iris, and switch between circle, ellipse, sector, and strip. The TELL-DLI has a wide application prospect in imaging systems, such as microscopic imaging system, and has the potential to be applied in the field of complex beam navigation.

Technologies for depth scanning in miniature optical imaging systems [Invited]

Biomedical optical imaging has found numerous clinical and research applications. For achieving 3D imaging, depth scanning presents the most significant challenge, particularly in miniature imaging devices. This paper reviews the state-of-art technologies for depth scanning in miniature optical imaging systems, which include two general approaches: 1) physically shifting part of or the entire imaging device to allow imaging at different depths and 2) optically changing the focus of the imaging optics. We mainly focus on the second group of methods, introducing a wide variety of tunable microlenses, covering the underlying physics, actuation mechanisms, and imaging performance. Representative applications in clinical and neuroscience research are briefly presented. Major challenges and future perspectives of depth/focus scanning technologies for biomedical optical imaging are also discussed.

A Monocular Variable Magnifications 3D Laparoscope System Using Double Liquid Lenses

During minimal invasive surgery (MIS), the laparoscope only provides a single viewpoint to the surgeon, leaving a lack of 3D perception. Many works have been proposed to obtain depth and 3D reconstruction by designing a new optical structure or by depending on the camera pose and image sequences. Most of these works modify the structure of the conventional laparoscopes and cannot provide 3D reconstruction of different magnification views. In this study, we propose a laparoscopic system based on double liquid lenses, which provide doctors with variable magnification rates, near observation, and real-time monocular 3D reconstruction. Our system composes of an optical structure that can obtain auto magnification change and autofocus without any physically moving element, and a deep learning network based on the Depth from Defocus (DFD) method, trained to suit inconsistent camera intrinsic situations and estimate depth from images of different focal lengths. The optical structure is portable and can be mounted on conventional laparoscopes. The depth estimation network estimates depth in real-time from monocular images of different focal lengths and magnification rates. Experiments show that our system provides a 0.68-1.44x zoom rate and can estimate depth from different magnification rates at 6fps. Monocular 3D reconstruction reaches at least 6mm accuracy. The system also provides a clear view even under 1mm close working distance. Ex-vivo experiments and implementation on clinical images prove that our system provides doctors with a magnified clear view of the lesion, as well as quick monocular depth perception during laparoscopy, which help surgeons get better detection and size diagnosis of the abdomen during laparoscope surgeries.

Reliability of tunable lenses: feedback sensors and the influence of temperature, orientation, and vibrations

We compare different aspects of the robustness to environmental conditions of two different types of piezo-actuated fluid-membrane lenses: a silicone membrane lens, where the piezo actuator indirectly deforms the flexible membrane through fluid displacement, and a glass membrane lens, where the piezo actuator directly deforms the stiff membrane. While both lenses operated reliably over the temperature range of 0°-75°C, there was a significant effect on their actuation characteristics, which can be well described through a simple model. The silicone lens in particular showed a variation in focal power of up to 0.1m^{-1 ∘}C^-1. We demonstrated that integrated pressure and temperature sensors can provide feedback for focal power, however, limited by the response time of the elastomers in the lenses, with polyurethane in the support structures of the glass membrane lens being more critical than the silicone. Studying the mechanical effects, the silicone membrane lens showed a gravity-induced coma and tilt, and a reduced imaging quality with the Strehl ratio decreasing from 0.89 to 0.31 at a vibration frequency of 100 Hz and an acceleration of 3g. The glass membrane lens was unaffected by gravity, and the Strehl ratio decreased from 0.92 to 0.73 at a vibration of 100 Hz, 3g. Overall, the stiffer glass membrane lens is more robust against environmental influences.

Experimental characterization, modelling and compensation of temperature effects in optotunable lenses

Most tunable lenses (TLs) are affected by deviations in optical power induced by external temperature changes or due to internal heating while in use. This study proposes: (1) An experimental characterization method to evaluate the magnitude of the optical power deviations due to internal temperature shifts; (2) three different mathematical models (experimental, polynomial, and optimized) to describe the response of the lens with temperature; (3) predictions of the internal temperature shifts while using the lens in time frames of minutes, seconds, and milliseconds and; (4) a real time optical power compensation tool based on the implementation of the models on a custom voltage electronic driver. The compensation methods were successfully applied to two TL samples in static and dynamic experiments and in hysteresis cycles. After 40 min at a static nominal power of 5 diopters (dpt), the internal temperature exponentially increased by 17 °C, producing an optical power deviation of 1.0 dpt (1.5 dpt when the lens cools down), representing a 20% distortion for heating and 30% for cooling. Modelling and compensation reduced the deviations to 0.2 dpt when heating (0.35 dpt when cooling) and the distortions to 4% and 7%. Similar levels of improvement were obtained in dynamic and hysteresis experiments. Compensation reduced temperature effects by more than 75%, representing a significant improvement in the performance of the lens.

Closed-loop experimental optimization of tunable lenses

Tunable lenses (TLs) are optical devices that can change their optical power in response to an electrical signal. In many applications, they are often pushed to or beyond their temporal limits. Fast periodic and/or abrupt variations of the optical power induce undesired distortions in their transient response and produce a decrease in their performance. A low-cost focimetry system, along with a custom closed-loop iterative optimization algorithm, was developed to (1) characterize a TL's response at high speed and (2) optimize their performance in realistic TL working conditions. A significant lens performance improvement was found in about 23 iterations with a decrease in the area under the error curve and an improved effective time. Applying the closed-loop optimization algorithm in a depth scanning experiment enhanced the image quality. Quantitatively, the image quality was evaluated using the structural similarity index metric that improves in individual frames, on average, from 0.345 to 0.895.

High stability liquid lens with optical path modulation function

In this paper, a high stability liquid lens with optical path modulation function is designed and fabricated. The liquid lens has an outer chamber and an inner chamber, and the inner chamber has a structure with three annular anchoring layers. This structure can limit the sliding of the three-phase contact line under electrowetting effect and anchor the position of contact angle with a limited distance. The feasibility of this structure is verified by simulation and practice. The zoom imaging, contact angle, focal length and response time of the liquid lens are analyzed. The structure with three annular anchoring layers provides six anchored precision optical path modulation gears, and the optical path difference can be changed by mechanical hydraulic control, up to 1.17 mm. Widespread applications of the proposed liquid lens are foreseeable such as microscopic imaging and a telescope system, etc.

Electrically tunable lenses - eliminating mechanical axial movements during high-speed 3D live imaging

The successful investigation of photosensitive and dynamic biological events, such as those in a proliferating tissue or a dividing cell, requires non-intervening high-speed imaging techniques. Electrically tunable lenses (ETLs) are liquid lenses possessing shape-changing capabilities that enable rapid axial shifts of the focal plane, in turn achieving acquisition speeds within the millisecond regime. These human-eye-inspired liquid lenses can enable fast focusing and have been applied in a variety of cell biology studies. Here, we review the history, opportunities and challenges underpinning the use of cost-effective high-speed ETLs. Although other, more expensive solutions for three-dimensional imaging in the millisecond regime are available, ETLs continue to be a powerful, yet inexpensive, contender for live-cell microscopy.

Optical zoom imaging systems using adaptive liquid lenses

An optical zoom imaging system that can vary the magnification factor without displacing the object and the image plane has been widely used. Nonetheless, conventional optical zoom imaging systems suffer from slow response, complicated configuration, vulnerability to misalignment during zoom operation, and are incompatible with miniaturized applications. This review article focuses on state-of-the-art research on novel optical zoom imaging systems that use adaptive liquid lenses. From the aspect of the configuration, according to the number of adaptive liquid lenses, we broadly divide the current optical zoom imaging systems using adaptive liquid lenses into two configurations: multiple adaptive liquid lenses, and a single adaptive liquid lens. The principles and configurations of these optical zoom imaging systems are introduced and represented. Three different working principles of the adaptive liquid lens (liquid crystal, polymer elastic membrane, and electrowetting effect) adopted in the optical zoom imaging systems are reviewed. Some representative applications of optical zoom imaging systems using adaptive liquid lenses are introduced. The opportunities and challenges of the optical zoom imaging systems using adaptive liquid lenses are also discussed. This review aims to provide a snapshot of the current state of this research field with the aim to attract more attention to put forward the development of the next-generation optical zoom imaging systems.

A review of the application of machine learning in molecular imaging

Molecular imaging (MI) is a science that uses imaging methods to reflect the changes of molecular level in living state and conduct qualitative and quantitative studies on its biological behaviors in imaging. Optical molecular imaging (OMI) and nuclear medical imaging are two key research fields of MI. OMI technology refers to the optical information generated by the imaging target (such as tumors) due to drug intervention and other reasons. By collecting the optical information, researchers can track the motion trajectory of the imaging target at the molecular level. Owing to its high specificity and sensitivity, OMI has been widely used in preclinical research and clinical surgery. Nuclear medical imaging mainly detects ionizing radiation emitted by radioactive substances. It can provide molecular information for early diagnosis, effective treatment and basic research of diseases, which has become one of the frontiers and hot topics in the field of medicine in the world today. Both OMI and nuclear medical imaging technology require a lot of data processing and analysis. In recent years, artificial intelligence technology, especially neural network-based machine learning (ML) technology, has been widely used in MI because of its powerful data processing capability. It provides a feasible strategy to deal with large and complex data for the requirement of MI. In this review, we will focus on the applications of ML methods in OMI and nuclear medical imaging.

Over 1000 nm Near-Infrared Multispectral Imaging System for Laparoscopic In Vivo Imaging

In this study, a laparoscopic imaging device and a light source able to select wavelengths by bandpass filters were developed to perform multispectral imaging (MSI) using over 1000 nm near-infrared (OTN-NIR) on regions under a laparoscope. Subsequently, MSI (wavelengths: 1000–1400 nm) was performed using the built device on nine live mice before and after tumor implantation. The normal and tumor pixels captured within the mice were used as teaching data sets, and the tumor-implanted mice data were classified using a neural network applied following a leave-one-out cross-validation procedure. The system provided a specificity of 89.5%, a sensitivity of 53.5%, and an accuracy of 87.8% for subcutaneous tumor discrimination. Aggregated true-positive (TP) pixels were confirmed in all tumor-implanted mice, which indicated that the laparoscopic OTN-NIR MSI could potentially be applied in vivo for classifying target lesions such as cancer in deep tissues.

High-speed coherent diffraction imaging by varying curvature of illumination with a focus tunable lens

A high-speed coherent diffraction imaging method is proposed by varying the curvature of illumination with a focus tunable lens. The imaging setup is free of conventional mechanical translation and takes only milliseconds to refocus by changing the electric signal applied on the lens. It is more compact and also an inexpensive alternative to coherent diffraction imaging with computerized translational stages. A detector that is kept at a fixed distance from the sample records diffraction patterns each time the spherical wavefront illuminations on the sample is changed with a control current. The complex wavefront of the object is then quantitatively recovered from the diffraction intensity measurements using an iterative phase retrieval algorithm. The feasibility of the proposed method is experimentally verified using various samples. Extremely short response time of the focus tunable lens makes the proposed method highly suitable for applications that requires high speed imaging.

Optical performance evaluation and chromatic aberration correction of a focus tunable lens used for 3D microscopy

Recently, the development of motion-free 3D microscopy utilizing focus tunable lenses (FTL) has been rapid. However, the downgrade of optical performance due to FTL and its gravity effect are rarely discussed in detail. Also, color dispersion is usually maintained purely depending on the FTL material without further correction. In this manuscript, we provide a quantitative evaluation of the impact of FTL on the optical performance of the microscope. The evaluation is based on both optical ray tracing simulations and lab experiments. In addition, we derive the first order conditions to correct axial color aberration of FTL through its entire power tuning range. Secondary spectrum correction is also possible and an apochromatic motion-free 3D microscope with 2 additional doublets is demonstrated. This study will serve a guidance in utilizing FTL as a motion-free element for 3D microscopy.

Tunable lenses: dynamic characterization and fine-tuned control for high-speed applications

Tunable lenses are becoming ubiquitous, in applications including microscopy, optical coherence tomography, computer vision, quality control, and presbyopic corrections. Many applications require an accurate control of the optical power of the lens in response to a time-dependent input waveform. We present a fast focimeter (3.8 KHz) to characterize the dynamic response of tunable lenses, which was demonstrated on different lens models. We found that the temporal response is repetitive and linear, which allowed the development of a robust compensation strategy based on the optimization of the input wave, using a linear time-invariant model. To our knowledge, this work presents the first procedure for a direct characterization of the transient response of tunable lenses and for compensation of their temporal distortions, and broadens the potential of tunable lenses also in high-speed applications.