The engineered eyeball, a tunable imaging system using soft-matter micro-optics

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

Development of a bio-inspired optical system that mimics accommodation and lighting regulation like the human eye

Bio-inspired optical systems have recently been developed using polarizers and liquid or rigid lenses. In this work, we propose a bio-inspired opto-mechatronic system that imitates the accommodation and regulation of light intensity as the human eye does. The system uses a polymeric lens as a cornea, an adjustable diaphragm as an iris, a tunable solid elastic lens as a crystalline lens, and a commercial sensor as a retina. We also present the development of the electronic control system to accommodate and regulate the amount of light that enters the system, for which two stepper motors, an Arduino control system, and light and movement sensors are used. The characterization of the system is presented together with the results obtained, where it can be seen that the system works in an acceptable range as the human eye does.

Parametric multiphysics study of focus-variable silicone lenses

By exploiting their inherent elasticity, focus-variable silicone lenses shift their focal length reversibly when deformed. Although biconcave and meniscus lenses contribute to optical systems just as well as biconvex lenses, studies primarily revolve around the latter. Thus, we aim to reveal the focal length shifting potential of all aforementioned lens types. Covering a wide parameter range of varying lens curvature radii, we present a coupled mechanical and optical simulation in which a lens deformation is applied. The results show significant differences in focal length shifting effectiveness for different lens types. Within the domains of specific lens types, trends in this effectiveness emerge for different combinations of curvature radii. Matching these radii when incorporating adaptive silicone lenses in optical systems may guide optics engineers toward more effective system designs through this study.

Design, Implementation, and Observer-based Output Control of a Super-coiled Polymer-Driven Two Degree-of-Freedom Robotic Eye

The prevalence of ineffective corrective surgeries for ocular motor disorders calls for a robotic eye platform in aiding ophthalmologists to better understand the biomechanisms of human eye movement. This letter presents the first hardware design and implementation of a 2-DOF robotic eye driven by super-coiled polymer (SCP) artificial muscles. While our previous work designed and simulated a deep deterministic policy gradient (DDPG) learning-based controller that requires full-state feedback of the SCP-driven robotic eye, measuring the temperature states of the slender SCPs is generally impractical for the ubiquitously aimed robot. To address this predicament, this letter proposes a reduced-order state observer to estimate the temperature of SCPs given the kinematic measurements. Combining the designed observer and the learning-based controller, the closed-loop output feedback control is implemented on the robotic eye prototype to examine its performance on three classical types of eye movements: visual fixation, saccadic pursuit, and smooth pursuit. The experimental results are presented which successfully validate the observer-based output control of the SCP-driven robotic eye.

Cuttlefish eye-inspired artificial vision for high-quality imaging under uneven illumination conditions

With the rise of mobile robotics, including self-driving automobiles and drones, developing artificial vision for high-contrast and high-acuity imaging in vertically uneven illumination conditions has become an important goal. In such situations, balancing uneven illumination, improving image contrast for facile object detection, and achieving high visual acuity in the main visual fields are key requirements. Meanwhile, in nature, cuttlefish (genus Sepia) have evolved an eye optimized for vertically uneven illumination conditions, which consists of a W-shaped pupil, a single spherical lens, and a curved retina with a high-density photoreceptor arrangement and polarized light sensitivity. Here, inspired by the cuttlefish eye, we report an artificial vision system consisting of a W-shaped pupil, a single ball lens, a surface-integrated flexible polarizer, and a cylindrical silicon photodiode array with a locally densified pixel arrangement. The W-shaped pupil integrated on the ball lens balances vertically uneven illumination, and the cylindrical silicon photodiode array integrated with the flexible polarizer enables high-contrast and high-acuity imaging.

Opto-Mechanical Eye Models, a Review on Human Vision Applications and Perspectives for Use in Industry

The purpose of this review is to aggregate technical information on existent optomechanical eye models (OME) described in the literature, for image quality assessment in different applications. Several physical eye models have been reviewed from peer-reviewed papers and patent applications. A typical eye model includes an artificial cornea, an intraocular lens or other lens to simulate the crystalline lens, an aperture as the pupil, and a posterior retinal surface, which may be connected to a light sensor. The interior of the eye model may be filled with a fluid to better emulate physiological conditions. The main focus of this review is the materials and physical characteristics used and the dimensional aspects of the main components including lenses, apertures, chambers, imaging sensors and filling medium. Various devices are described with their applications and technical details, which are systematically tabulated highlighting their main characteristics and applications. The models presented are detailed and discussed individually, and the features of different models are compared when applicable, highlighting strengths and limitations. In the end there is a brief discussion about the potential use of artificial eye models for industrial applications.

Fabrication of High Precision Silicon Spherical Microlens Arrays by Hot Embossing Process

In this paper, a high-precision, low-cost, batch processing nanoimprint method is proposed to process a spherical microlens array (MLA). The nanoimprint mold with high surface precision and low surface roughness was fabricated by single-point diamond turning. The anti-sticking treatment of the mold was carried out by perfluorooctyl phosphoric acid (PFOPA) liquid deposition. Through the orthogonal experiment of hot embossing with the treated mold and subsequent inductively coupled plasma (ICP) etching, the microstructure of MLA was transferred to the silicon substrate, with a root mean square error of 17.7 nm and a roughness of 12.1 nm Sa. The average fitted radius of the microlens array units is 406.145 µm, which is 1.54% different from the design radius.

Combined diffractive optical elements with adjustable optical properties controlled by a relative rotation: tutorial

A pair of adjacent transmissive diffractive optical elements (DOEs) forms a combined DOE with tunable optical properties, as, for example, a diffractive lens with an adjustable focal length. The optical properties are controlled by a relative movement of the two DOEs, such as a translation or a rotation around the optical axis. Here we discuss various implementations of this principle, such as tunable diffractive lenses, axicons, vortex plates, and aberration correction devices. We discuss the limits of the tuning range and of diffraction efficiency. Furthermore, it is demonstrated how chromatic aberrations can be suppressed by using multi-order DOEs.

Biomimetic models of the human eye, and their applications

Replicating the functionality of the human eye has been a challenge for more than a century, creating a great wealth of biomimetic and bioinspired devices, and providing ever improving models of the eye for myriad research purposes. As improvements in microelectronics have proceeded, individual components of the eye have been replicated, and models of the optical behaviour of the eye have improved. This review explores both work developed for improving medical components, with an ultimate aim of a fully functioning prosthetic eye, and work looking at improving existing devices through biomimetic means. It is hoped that this holistic approach to the subject will aid in the cross pollination of ideas between the two research foci. The review starts by summarising the reported measurements of optical parameters of various components of the eye. It then charts the development of individual bionic components. Particular focus is put on the development of bionic and biomimetic forms of the two main adaptive components of the eye, namely the lens and the iris, and the challenges faced in modelling the light sensitive retina. Work on each of these components is thoroughly reviewed, including an overview of the principles behind the many different approaches used to mimic the functionality, and discussion of the pros and cons of each approach. This is concluded by an overview of several reported models of the complete or semi-complete eye, including details of the components used and a summary of the models' functionality. Finally, some consideration is given to the direction of travel of this field of research, and which existing approaches are likely to bring us closer to the long term goal of a fully functional analogue of the eye.

Electrically Adaptive and Shape-Changeable Invertible Microlens

Existing soft actuators for adaptive microlenses suffer from high required input voltage, optical loss, liquid loss, and the need for assistant systems. In this study, we fabricate a polyvinyl chloride-based gel using a new synergistic plasticization method to achieve simultaneously a high optical transparency and an ultrasoft rubber-like elastic behavior with a large voltage-induced deformation under a weak electric field. By compressing the smooth gel between two sets of annular electrodes, a self-contained biconvex microlens is realized that is capable of considerable shape changes in the optical path. Each surface of the dual-curvature microlens can be independently adjusted to focus or scatter light to capture real or virtual images, yield variable focal lengths (+31.8 to -11.3 mm), and deform to various shapes to improve aberrations. In addition to simple fabrication, our microlens operates silently and consumes low power (0.52 mW), making it superior to existing microlenses.

Tunable electrochromic filter for in situ Fourier spatial frequency filtering

Spatial optical Fourier filtering is a widespread technique for in situ image or light field processing. However, conventional fixed absorbing patterns or mechanical irises only allow an inflexible, very restricted control. Thus, we present two electrochromic spatial filters with ring-shaped or directional segments, which can be individually addressed and continuously tuned in transmission resulting in up to 512 different filtering states. For realization of the electrochromic devices, we overcome technical obstacles to realize seamless, gap-free electrochromic segments. We describe this novel fabrication process and demonstrate the successful application in an optical Fourier transform set-up.

Soft Tunable Lenses Based on Zipping Electroactive Polymer Actuators

Compact and entirely soft optics with tunable and adaptive properties drive the development of life-like soft robotic systems. Yet, existing approaches are either slow, require rigid components, or use high operating voltages of several kilovolts. Here, soft focus-tunable lenses are introduced, which operate at practical voltages, cover a high range of adjustable focal lengths, and feature response times in the milliseconds range. The nature-inspired design comprises a liquid-filled elastomeric lens membrane, which is inflated by zipping electroactive polymers to tune the focal length. An analytic description of the tunable lens supports optimized designs and accurate prediction of the lens characteristics. Focal length changes between 22 and 550 mm (numerical aperture 0.14-0.005) within 260 ms, equal in performance to human eyes, are demonstrated for a lens with 3 mm aperture radius, while applying voltages below 500 V. The presented model, design rules, and fabrication methods address central challenges of soft electrostatic actuators and optical systems, and pave the way toward autonomous bio-inspired robots and machines.

Mini-Review on Bioinspired Superwetting Microlens Array and Compound Eye

Microlens arrays (MLAs) and MLA-based artificial compound eyes (ACEs) are the important miniaturized optical components in modern micro-optical systems. However, their optical performance will seriously decline once they are wetted by water droplets (such as fog, dew, and rain droplets) or are polluted by contaminations in a humid environment. In this mini-review, we summarize the research works related to the fabrication of superwetting MLAs and ACEs and show how to integrate superhydrophobic and superoleophobic microstructures with an MLA. The fabrication strategy can be split into two categories. One is the hybrid pattern composed of the MLA domain and the superwetting domain. Another is the direct formation of superwetting nanostructures on the surface of the microlenses. The superhydrophobicity or superoleophobicity endows the MLAs and ACEs with liquid repellence and self-cleaning function besides excellent optical performance. We believe that the superwetting MLAs and ACEs will have significant applications in various optical systems that are often used in the humid or liquid environment.

Changes in temperature inside an optomechanical model of the human eye during emulated transscleral cyclophotocoagulation

Currently, many diseases of the eye are treated by laser surgery. An understanding of light propagation and the heating of eye tissue during laser exposure is crucial to improving the outcome of these procedures. Here, we present the development of physical and computational models of the human eye by combining optical light propagation and thermal characteristics. For the physical model, all parts of the eye, including cornea, lens, ciliary body, sclera, aqueous and vitreous humors, and iris, were fabricated using a 3D printed holder and modified polydimethylsiloxane. We also present a computational model based on finite element analysis that allows for a direct comparison between the simulation and experimental measurements. These models provide an opportunity to directly assess the rise in temperature in all eye tissues. The simulated and physical models showed good agreement for the transmission of light at varying incident angles. The heating of optical components was investigated in the retina and the ciliary body during simulated laser surgery. Temperature increases of 45.3°C and 30.6°C in the retina and ciliary bodies, respectively, were found in the physical model after 1 minute of exposure to 186 mW of 850 nm laser radiation. This compared to 29.8°C and 33.9°C increases seen under the same conditions in the simulation model with human eye parameters and 48.1°C and 28.7°C for physical model parameters. These results and these models are very promising for further investigation of the impact of laser surgery.

Fabrication of concave microlenses on a diamond by a spin coating process

In this study, to fabricate diamond concave microlenses in a simple manner, an approach that combines a spin coating process with subsequent dry etching was demonstrated. First, photolithography was used to produce cylindrical holes in the photoresist layer on the diamond surface. Then, another photoresist was spin coated to fill the holes, and the concave structures with meniscus shapes were then obtained because of centrifugal force and interfacial tension. Finally, diamond concave microlenses were formed by transferring photoresist concave structures onto a diamond substrate using a dry etching technique. The fabricated diamond microlens exhibits a low surface roughness with nanometers as well as high-quality imaging and focusing performances, which is expected to have a wider range of potential applications under harsh and special conditions.

Biomimetic optics: liquid-based optical elements imitating the eye functionality

The optical systems mimicking the eye functions are of great importance in various applications including consumer electronics, medical equipment, machine vision systems and robotics. This optics offers advantages over traditional optical technologies such as the superior adaptation to changing conditions and the comprehensive range of functional characteristics at miniature sizes. This paper presents a review on the recent progress in the development of human eye-inspired optical systems. Liquid-based and elastomer-based tunable optical elements are discussed with the focus on the actuation mechanism, optical performance and the possibility of integration into artificial eye systems. This article is part of the theme issue 'Bioinspired materials and surfaces for green science and technology (part 3)'.

Bioinspired Zoom Compound Eyes Enable Variable-Focus Imaging

Natural compound eyes provide the inspiration for developing artificial optical devices that feature a large field of view (FOV). However, the imaging ability of artificial compound eyes is generally based on the large number of ommatidia. The lack of a tunable imaging mechanism significantly limits the practical applications of artificial compound eyes, for instance, distinguishing targets at different distances. Herein, we reported zoom compound eyes that enable variable-focus imaging by integrating a deformable poly(dimethylsiloxane) (PDMS) microlens array (MLA) with a microfluidic chamber. The thin and soft PDMS MLA was fabricated by soft lithography using a hard template prepared by a combined technology of femtosecond laser processing and wet etching. As compared with other mechanical machining strategies, our combined technology features high flexibility, efficiency, and uniformity, as well as designable processing capability, since the size, distribution, and arrangement of the ommatidia can be well controlled during femtosecond laser processing. By tuning the volume of water injected into the chamber, the PDMS MLA can deform from a planar structure to a hemispherical shape, evolving into a tunable compound eye of variable FOV up to 180°. More importantly, the tunable chamber can functionalize as the main zoom lens for tunable imaging, which endows the compound eye with the additional capability of distinguishing targets at different distances. Its focal length can be turned from 3.03 mm to infinity with an angular resolution of 3.86 × 10^-4 rad. This zoom compound eye combines the advantages of monocular eyes and compound eyes together, holding great promise for developing advanced micro-optical devices that enable large FOV and variable-focus imaging.

A handheld continuous-flow real-time fluorescence qPCR system with a PVC microreactor

The polymerase chain reaction (PCR) has unique advantages of sensitivity, specificity and rapidity in pathogen detection, which makes it at the forefront of academia and application in molecular biology diagnosis. In this study, we proposed a hand-held real-time fluorescence qPCR system, which can be used for the quantitative analysis of nucleic acid molecules. For the first time, we use a PVC microreactor which improved the transmittance of the microreactor and made it easy to collect the fluorescence signal. In order to make it portable, the system adopted a passive syringe for sample injection and integrated temperature control and detection with a lithium battery for power supply. What's more, the fluorescence signal was captured by using a smartphone through an external automatic robotic arm. This real-time qPCR system can detect genomic DNA of the H7N9 avian influenza over four orders of magnitude of concentration from 10⁷ to 10⁴ copies per μL. In addition, it was verified that the fluorescence images obtained by this system were clearer than those obtained by a traditional system (using a PTFE spatial PCR microreactor) with two typical dyes and a probe tested-EvaGreen, SYBR Green and FAM.

Hybrid driving variable-focus optofluidic lens

Conventional optofluidic lens usually has only one interface, which means that the zoom range is small, and the ability to correct aberrations is poor. In this paper, we propose a hybrid driving variable-focus optofluidic lens. It has one water-oil interface shifted by an applied voltage and one tunable Polydimethylsiloxane (PDMS) lens deformed by pumping liquid in or out of the cavity. The proposed lens combines the advantages of electrowetting lens and mechanical lens. Therefore, it can provide a large focal length tuning range with good image quality. The shortest positive and negative focal length are ∼6.02 mm and ∼-11.15 mm, respectively. The maximum resolution of our liquid lens can be reached 18 lp/mm. We also designed and fabricated a zoom system using the hybrid driving variable-focus optofluidic lens. In the experiment, the zoom range of the system is 14 mm∼30 mm and the zoom ratio is ∼2.14× without any mechanical moving parts. Its applications for zoom telescope system and zoom microscope and so on are foreseeable.

Error Analysis and Compensation in Images Stitching for the Mechanically Stitched CCD Aerial Cameras

Image misalignment during image stitching is a common issue for the mechanically stitched CCDs in the aerial cameras due to the relative image motion between CCDs. In this paper, we analyze the error in imaging stitching for the mechanically stitched CCDs and propose a compensation method based on position and orientation system (POS). First, the imaging relationship of overlapping pixels in mechanical stitching is analyzed. The imaging model of adjacent CCD is constructed according to the collinear equation. The effects of stitching staggered distance, carrier attitude change and flight speed on the image mosaic of overlapping areas are given. Monte Carlo algorithm is used to analyze the statistical value of image mosaic error in overlapping area under typical working conditions. Then, a geometric correction method based on POS recording of external orientation elements of adjacent CCD imaging is proposed. The overlapping area stitching error is reduced from 14.9 pixels to 0.4 pixels which meets the engineering requirements. The mechanical stitching mathematical model, analysis and correction method established in this paper have strong engineering and application significance for mechanical stitching of aerial cameras.

Optical modelling of a supplementary tunable air-spaced goggle lens for rodent eye imaging

Aberration variations severely degrade retinal imaging in small animal eyes. Previously, the approach of a goggle lens with a matching corneal index was proposed to overcome the on-axis resolution limit of static imaging systems, which allows the use of the full eye pupil. But this technique didn’t address the problem of the large power variation, and the ensuing aberration on and off-axis, when dealing with small animal eyes. In this study, we present the concept of a tunable goggle lens, designed to compensate individual ocular aberration for different rodent eye powers. Ray tracing evidences that lens-fitted goggles permit, not only to adjust individual eyes power, but also to surpass conventional adaptive correction technique over large viewing angle, provided a minimum use of two spaced liquids. We believe that the overlooked advantage of the 3D lens function is a seminal finding for further technological advancements in widefield retinal imaging.