A Time-to-Digital Converter for Low-Power Consumption Single Slope Analog-to-Digital Converters in a High-Speed CMOS Image Sensor

To reduce the power consumption of a TDC in high-speed applications, a TDC architecture applied to SS ADC is proposed to reduce redundant counting. This structure can remove the identical part between two rows of pixel signals in a CMOS image sensor by adjusting the start and stop signal of the TDC, which will reduce the number of flipping of D flip-flops in the TDC. This structure requires the simultaneous readout of two rows of pixels in the high-speed CMOS image sensor. In the 110 nm CMOS process, simulation results show that the designed 5-bit TDC achieves an effective number of bits (ENOB) at 4.72 bits and a figure-of-merit (FOM) at 104.7-162.3 fJ/step, with a power consumption ranging from 60 µW to 93 µW. Compared with traditional counting methods, the proposed TDC can reduce counting power consumption by 30%.

A Hybrid System for Defect Detection on Rail Lines through the Fusion of Object and Context Information

Defect detection on rail lines is essential for ensuring safe and efficient transportation. Current image analysis methods with deep neural networks (DNNs) for defect detection often focus on the defects themselves while ignoring the related context. In this work, we propose a fusion model that combines both a targeted defect search and a context analysis, which is seen as a multimodal fusion task. Our model performs rule-based decision-level fusion, merging the confidence scores of multiple individual models to classify rail-line defects. We call the model "hybrid" in the sense that it is composed of supervised learning components and rule-based fusion. We first propose an improvement to existing vision-based defect detection methods by incorporating a convolutional block attention module (CBAM) in the you only look once (YOLO) versions 5 (YOLOv5) and 8 (YOLOv8) architectures for the detection of defects and contextual image elements. This attention module is applied at different detection scales. The domain-knowledge rules are applied to fuse the detection results. Our method demonstrates improvements over baseline models in vision-based defect detection. The model is open for the integration of modalities other than an image, e.g., sound and accelerometer data.

High-resolution X-Ray imaging of small animal samples based on Commercial-Off-The-Shelf CMOS image sensors

 An automated system for acquiring microscopic-resolution radiographic images of biological samples was developed. Mass-produced, low-cost, and easily automated components were used, such as Commercial-Off-The-Self CMOS image sensors (CIS), stepper motors, and control boards based on Arduino and RaspberryPi. System configuration, imaging protocols, and Image processing (filtering and stitching) were defined to obtain high-resolution images and for successful computational image reconstruction. Radiographic images were obtained for animal samples including the widely used animal models zebrafish (Danio rerio) and the fruit-fly (Drosophila melanogaster), as well as other small animal samples. The use of phosphotungstic acid (PTA) as a contrast agent was also studied. Radiographic images with resolutions of up to (7±0.6)μm were obtained, making this system comparable to commercial ones. This work constitutes a starting point for the development of more complex systems such as X-ray attenuation micro-tomography systems based on low-cost off-the-shelf technology. It will also bring the possibility to expand the studies that can be carried out with small animal models at many institutions (mostly those working on tight budgets), particularly those on the effects of ionizing radiation and absorption of heavy metal contaminants in animal tissues.

A Feasibility Study on Extension of Measurement Distance in Vision Sensor Using Super-Resolution for Dynamic Response Measurement

The current civil infrastructure conditions can be assessed through the measurement of displacement using conventional contact-type sensors. To address the disadvantages of traditional sensors, vision-based sensor measurement systems have been derived in numerous studies and proven as an alternative to traditional sensors. Despite the benefits of the vision sensor, it is well known that the accuracy of the vision-based displacement measurement is largely dependent on the camera extrinsic or intrinsic parameters. In this study, the feasibility study of a deep learning-based single image super-resolution (SISR) technique in a vision-based sensor system is conducted to alleviate the low spatial resolution of image frames at long measurement distance ranges. Additionally, its robustness is evaluated using shaking table tests. As a result, it is confirmed that the SISR can reconstruct definite images of natural targets resulting in an extension of the measurement distance range. Additionally, it is determined that the SISR mitigates displacement measurement error in the vision sensor-based measurement system. Based on this fundamental study of SISR in the feature point-based measurement system, further analysis such as modal analysis, damage detection, and so forth should be continued in order to explore the functionality of SR images by applying low-resolution displacement measurement footage.

A Wafer-Scale Nanoporous 2D Active Pixel Image Sensor Matrix with High Uniformity, High Sensitivity, and Rapid Switching

2D transition-metal dichalcogenides (TMDs) have been successfully developed as novel ubiquitous optoelectronics owing to their excellent electrical and optical characteristics. However, active-matrix image sensors based on TMDs have limitations owing to the difficulty of fabricating large-area integrated circuitry and achieving high optical sensitivity. Herein, a large-area uniform, highly sensitive, and robust image sensor matrix with active pixels consisting of nanoporous molybdenum disulfide (MoS₂ ) phototransistors and indium-gallium-zinc oxide (IGZO) switching transistors is reported. Large-area uniform 4-inch wafer-scale bilayer MoS₂ films are synthesized by radio-frequency (RF) magnetron sputtering and sulfurization processes and patterned to be a nanoporous structure consisting of an array of periodic nanopores on the MoS₂ surface via block copolymer lithography. Edge exposure on the nanoporous bilayer MoS₂ induces the formation of subgap states, which promotes a photogating effect to obtain an exceptionally high photoresponsivity of 5.2 × 10⁴ A W^-1 . A 4-inch-wafer-scale image mapping is successively achieved using this active-matrix image sensor by controlling the device sensing and switching states. The high-performance active-matrix image sensor is state-of-the-art in 2D material-based integrated circuitry and pixel image sensor applications.

A Wide Bandgap Halide Perovskite Based Self-Powered Blue Photodetector with 84.9% of External Quantum Efficiency

A self-powered, color-filter-free blue photodetector (PD) based on halide perovskites is reported. A high external quantum efficiency (EQE) of 84.9%, which is the highest reported EQE in blue PDs, is achieved by engineering the A-site monovalent cations of wide-bandgap perovskites. The optimized composition of formamidinium (FA)/methylammonium (MA) increases the heat of formation, yielding a uniform and smooth film. The incorporation of Cs⁺ ions into the FA/MA composition suppresses the trap density and increases charge-carrier mobility, yielding the highest average EQE of 77.4%, responsivity of 0.280 A W^-1 , and detectivity of 5.08 × 10¹² Jones under blue light. Furthermore, Cs⁺ improves durability under repetitive operations and ambient atmosphere. The proposed device exhibits peak responsivity of 0.307 A W^-1 , which is higher than that of the commercial InGaN-based blue PD (0.289 A W^-1 ). This study will promote the development of next-generation image sensors with vertically stacked perovskite PDs.

Ultraviolet Photodetectors Based on Polymer Microwire Arrays toward Wearable Medical Devices

Polymer micro/nanoarchitectures have attracted intense interest for wearable medical applications due to their excellent mechanical flexibility, solution processability, and tunable optoelectronic properties. Based on polymer micro/nanostructures, high-performance ultraviolet (UV) photodetectors can not only functionalize the accurate image sensing but also sustain the biocomfortable flexible devices for real-time health monitoring. The main challenges are focused on the integration of medical wearable devices, which requires large-scale assembly of polymer micro/nanostructures with controlled morphology and strict alignment. Herein, we utilized a confined assembly system through the cautious regulation for the growth of high-quality polymer 1D arrays. UV photodetectors based on these polymer microwire arrays perform a high on/off ratio of 137 and responsivity of 19.1 mA W^-1. Polymer microarray photodetectors facilitate the scale-up fabrication of 14 × 18 multiplexed image sensors for highly accurate capturing the signals of Arabic numerals "1," "2," and "3." Flexible UV photodetectors based on these arrays present excellent flexibility and bending durability, maintaining 97% of their original on/off ratio after 4000 cycles with a 10 mm bending radius. UV photodetection signals were also collected from the attached flexible devices on the back skin of the mouse, demonstrating the great potential in wearable medical photodetection.

Probing the Efficacy of Large-Scale Nonporous IGZO for Visible-to-NIR Detection Capability: An Approach toward High-Performance Image Sensor Circuitry

The technological ability to detect a wide spectrum range of illuminated visible-to-NIR is substantially improved for an amorphous metal oxide semiconductor, indium gallium zinc oxide (IGZO), without employing an additional photoabsorber. The fundamentally tuned morphology via structural engineering results in the creation of nanopores throughout the entire thickness of ∼30 nm. See-through nanopores have edge functionalization with vacancies, which leads to a large density of substates near the conduction band minima and valence band maxima. The presence of nanoring edges with a high concentration of vacancies is investigated using chemical composition analysis. The process of creating a nonporous morphology is sophisticated and is demonstrated using a wafer-scale phototransistor array. The performance of the phototransistors is assessed in terms of photosensitivity (S) and photoresponsivity (R); both are of high magnitudes (S = 8.6 × 10⁴ at λ_ex = 638 nm and P_inc = 512 mW cm^2-; R = 120 A W^1- at P_inc = 2 mW cm^2- for the same λ_ex). Additionally, the 7 × 5 array of 35 phototransistors is effective in sensing and reproducing the input image by responding to selectively illuminated pixels.

Constructing a High-Quality t-Se/Si Interface Via In Situ Light Annealing of a-Se for a Self-Powered Image Sensor

The quality of the interface, e.g., the semiconductor-semiconductor or metal-semiconductor interface, is the main factor restricting the photodetection performance of a heterojunction. In this study, a high-quality Se/Si interface is constructed via in situ directional transformation of amorphous Se (a-Se) into crystalline Se (t-Se) on a Si substrate via light annealing. Benefitting from the high-quality interface and appropriate energy band between Si and Se, the t-Se/Si heterojunction exhibits an extremely high responsivity and detectivity of 583.33 mA W^-1 and 8.52 × 10¹² Jones at 760 nm, respectively. In addition, the device exhibits an ultrafast rise time of 183 µs and a decay time of 405 µs. Furthermore, an image sensor fabricated via local light annealing successfully recognizes patterns of "N," "P," and "U." This study provides valuable guidance for the construction of high-quality interfaces and the design of self-powered image sensors.

A Molecular-Switch-Embedded Organic Photodiode for Capturing Images against Strong Backlight

When the intensity of the incident light increases, the photocurrents of organic photodiodes (OPDs) exhibit relatively early saturation, due to which OPDs cannot easily detect objects against strong backlights, such as sunlight. In this study, this problem is addressed by introducing a light-intensity-dependent transition of the operation mode, such that the operation mode of the OPD autonomously changes to overcome early photocurrent saturation as the incident light intensity passes the threshold intensity. The photoactive layer is doped with a strategically designed and synthesized molecular switch, 1,2-bis-(2-methyl-5-(4-cyanobiphenyl)-3-thienyl)tetrafluorobenzene (DAB). The proposed OPD exhibits a typical OPD performance with an external quantum efficiency (EQE) of <100% and a photomultiplication behavior with an EQE of >100% under low-intensity and high-intensity light illuminations, respectively, thereby resulting in an extension of the photoresponse linearity to a light intensity of 434 mW cm^-2 . This unique and reversible transition of the operation mode can be explained by the unbalanced quantum yield of photocyclization/photocycloreversion of the molecular switch. The details of the operation mechanism are discussed in conjunction with various photophysical analyses. Furthermore, they establish a prototype image sensor with an array of molecular-switch-embedded OPD pixels to demonstrate their extremely high sensitivity against strong light illumination.

Edge Computing for Vision-Based, Urban-Insects Traps in the Context of Smart Cities

Our aim is to promote the widespread use of electronic insect traps that report captured pests to a human-controlled agency. This work reports on edge-computing as applied to camera-based insect traps. We present a low-cost device with high power autonomy and an adequate picture quality that reports an internal image of the trap to a server and counts the insects it contains based on quantized and embedded deep-learning models. The paper compares different aspects of performance of three different edge devices, namely ESP32, Raspberry Pi Model 4 (RPi), and Google Coral, running a deep learning framework (TensorFlow Lite). All edge devices were able to process images and report accuracy in counting exceeding 95%, but at different rates and power consumption. Our findings suggest that ESP32 appears to be the best choice in the context of this application according to our policy for low-cost devices.

High-throughput field crop phenotyping: current status and challenges

In contrast to the rapid advances made in plant genotyping, plant phenotyping is considered a bottleneck in plant science. This has promoted high-throughput plant phenotyping (HTP) studies, resulting in an exponential increase in phenotyping-related publications. The development of HTP was originally intended for use as indoor HTP technologies for model plant species under controlled environments. However, this subsequently shifted to HTP for use in crops in fields. Although HTP in fields is much more difficult to conduct due to unstable environmental conditions compared to HTP in controlled environments, recent advances in HTP technology have allowed these difficulties to be overcome, allowing for rapid, efficient, non-destructive, non-invasive, quantitative, repeatable, and objective phenotyping. Recent HTP developments have been accelerated by the advances in data analysis, sensors, and robot technologies, including machine learning, image analysis, three dimensional (3D) reconstruction, image sensors, laser sensors, environmental sensors, and drones, along with high-speed computational resources. This article provides an overview of recent HTP technologies, focusing mainly on canopy-based phenotypes of major crops, such as canopy height, canopy coverage, canopy biomass, and canopy stressed appearance, in addition to crop organ detection and counting in the fields. Current topics in field HTP are also presented, followed by a discussion on the low rates of adoption of HTP in practical breeding programs.

Nanophotonic Color Routing

Recent advances in low-dimensional materials and nanofabrication technologies have stimulated many breakthroughs in the field of nanophotonics such as metamaterials and plasmonics that provide efficient ways of light manipulation at a subwavelength scale. The representative structure-induced spectral engineering techniques have demonstrated superior design of freedom compared with natural materials such as pigment/dye. In particular, the emerging spectral routing scheme enables extraordinary light manipulation in both frequency-domain and spatial-domain with high-efficiency utilization of the full spectrum, which is critically important for various applications and may open up entirely new operating paradigms. In this review, a comparative introduction on the operating mechanisms of spectral routing and spectral filtering schemes is given and recent progress on various color nanorouters based on metasurfaces, plasmonics, dielectric antennas is reviewed with a focus on the potential application in high-resolution imaging. With a thorough analysis and discussion on the advanced properties and drawbacks of various techniques, this report is expected to provide an overview and vision for the future development and application of nanophotonic color (spectral) routing techniques.

Ultrasmall compact CMOS imaging system for bioluminescence reporter-based live gene expression analysis

SIGNIFICANCE

Gene expression analysis is an important fundamental area of biomedical research. However, live gene expression imaging has proven challenging due to constraints in conventional optical devices and fluorescent reporters.

AIM

Our aim is to develop smaller, more cost-effective, and versatile imaging capabilities compared with conventional devices. Bioluminescence reporter-based gene expression analysis was targeted due to its advantages over fluorescence-based imaging.

APPROACH

We created a small compact imaging system using micro-CMOS image sensors (μCIS). The μCIS model had an improved pixel design and a patterned absorption filter array to detect the low light intensity of bioluminescence.

RESULTS

The device demonstrated lower dark current, lower temporal noise, and higher sensitivity compared with previous designs. The filter array enabled us to subtract dark current drift and attain a clearer light signal. These improvements allowed us to measure bioluminescence reporter-based gene expression in living mammalian cells.

CONCLUSION

Using our μCIS system for bioluminescence imaging in the future, the device can be implanted in vivo for simultaneous gene expression imaging, behavioral analysis, and optogenetic modulation.

Ultra-High Performance Amorphous Ga2 O3 Photodetector Arrays for Solar-Blind Imaging

The growing demand for scalable solar-blind image sensors with remarkable photosensitive properties has stimulated the research on more advanced solar-blind photodetector (SBPD) arrays. In this work, the authors demonstrate ultrahigh-performance metal-semiconductor-metal (MSM) SBPDs based on amorphous (a-) Ga2 O3 via a post-annealing process. The post-annealed MSM a-Ga2 O3 SBPDs exhibit superhigh sensitivity of 733 A/W and high response speed of 18 ms, giving a high gain-bandwidth product over 104 at 5 V. The SBPDs also show ultrahigh photo-to-dark current ratio of 3.9 × 107 . Additionally, the PDs demonstrate super-high specific detectivity of 3.9 × 1016 Jones owing to the extremely low noise down to 3.5 fW Hz-1/2 , suggesting high signal-to-noise ratio. Underlying mechanism for such superior photoelectric properties is revealed by Kelvin probe force microscopy and first principles calculation. Furthermore, for the first time, a large-scale, high-uniformity 32 × 32 image sensor array based on the post-annealed a-Ga2 O3 SBPDs is fabricated. Clear image of target object with high contrast can be obtained thanks to the high sensitivity and uniformity of the array. These results demonstrate the feasibility and practicality of the Ga2 O3 PDs for applications in solar-blind imaging, environmental monitoring, artificial intelligence and machine vision.

CNN-Based Classifier as an Offline Trigger for the CREDO Experiment

Gamification is known to enhance users' participation in education and research projects that follow the citizen science paradigm. The Cosmic Ray Extremely Distributed Observatory (CREDO) experiment is designed for the large-scale study of various radiation forms that continuously reach the Earth from space, collectively known as cosmic rays. The CREDO Detector app relies on a network of involved users and is now working worldwide across phones and other CMOS sensor-equipped devices. To broaden the user base and activate current users, CREDO extensively uses the gamification solutions like the periodical Particle Hunters Competition. However, the adverse effect of gamification is that the number of artefacts, i.e., signals unrelated to cosmic ray detection or openly related to cheating, substantially increases. To tag the artefacts appearing in the CREDO database we propose the method based on machine learning. The approach involves training the Convolutional Neural Network (CNN) to recognise the morphological difference between signals and artefacts. As a result we obtain the CNN-based trigger which is able to mimic the signal vs. artefact assignments of human annotators as closely as possible. To enhance the method, the input image signal is adaptively thresholded and then transformed using Daubechies wavelets. In this exploratory study, we use wavelet transforms to amplify distinctive image features. As a result, we obtain a very good recognition ratio of almost 99% for both signal and artefacts. The proposed solution allows eliminating the manual supervision of the competition process.

Low-Noise Dual-Band Polarimetric Image Sensor Based on 1D Bi2 S3 Nanowire

With the increasing demand for detection accuracy and sensitivity, dual-band polarimetric image sensor has attracted considerable attention due to better object recognition by processing signals from diverse wavebands. However, the widespread use of polarimetric sensors is still limited by high noise, narrow photoresponse range, and low linearly dichroic ratio. Recently, the low-dimensional materials with intrinsic in-plane anisotropy structure exhibit the great potential to realize direct polarized photodetection. Here, strong anisotropy of 1D layered bismuth sulfide (Bi2 S3 ) is demonstrated experimentally and theoretically. The Bi2 S3 photodetector exhibits excellent device performance, which enables high photoresponsivity (32 A W-1 ), Ion /Ioff ratio (1.08 × 104 ), robust linearly dichroic ratio (1.9), and Hooge parameter (2.0 × 10-5 at 1 Hz) which refer to lower noise than most reported low-dimensional materials-based devices. Impressively, such Bi2 S3 nanowire exhibits a good broadband photoresponse, ranging from ultraviolet (360 nm) to short-wave infrared (1064 nm). Direct polarimetric imaging is implemented at the wavelengths of 532 and 808 nm. With these remarkable features, the 1D Bi2 S3 nanowires show great potential for direct dual-band polarimetric image sensors without using any external optical polarizer.

Image Sensors for Wave Monitoring in Shore Protection: Characterization through a Machine Learning Algorithm

Waves propagating on the water surface can be considered as propagating in a dispersive medium, where gravity and surface tension at the air–water interface act as restoring forces. The velocity at which energy is transported in water waves is defined by the group velocity. The paper reports the use of video-camera observations to study the impact of water waves on an urban shore. The video-monitoring system consists of two separate cameras equipped with progressive RGB CMOS sensors that allow 1080p HDTV video recording. The sensing system delivers video signals that are processed by a machine learning technique. The scope of the research is to identify features of water waves that cannot be normally observed. First, a conventional modelling was performed using data delivered by image sensors together with additional data such as temperature, and wind speed, measured with dedicated sensors. Stealth waves are detected, as are the inverting phenomena encompassed in waves. This latter phenomenon can be detected only through machine learning. This double approach allows us to prevent extreme events that can take place in offshore and onshore areas.

On Combining Convolutional Autoencoders and Support Vector Machines for Fault Detection in Industrial Textures

Defects in textured materials present a great variability, usually requiring ad-hoc solutions for each specific case. This research work proposes a solution that combines two machine learning-based approaches, convolutional autoencoders, CA; one class support vector machines, SVM. Both methods are trained using only defect free textured images for each type of analyzed texture, labeling the samples for the SVMs in an automatic way. This work is based on two image processing streams using image sensors: (1) the CA first processes the incoming image from the input to the output, producing a reconstructed image, from which a measurement of correct or defective image is obtained; (2) the second process uses the latent layer information as input to the SVM to produce a measurement of classification. Both measurements are effectively combined, making an additional research contribution. The results obtained achieve a percentage of success of 92% on average, outperforming results of previous works.

Recent Advances in Perovskite Photodetectors for Image Sensing

In recent years, metal halide perovskites have been widely investigated to fabricate photodetectors for image sensing due to the excellent photoelectric performance, tunable bandgap, and low-cost solution preparation process. In this review, a comprehensive overview of the recent advances in perovskite photodetectors for image sensing is provided. First, the key performance parameters and the basic device types of photodetectors are briefly introduced. Then, the recent developments of image sensors on the basis of different dimensional perovskite materials, including 0D, 1D, 2D, and 3D perovskite materials, are highlighted. Besides the device structures and photoelectric properties of perovskite image sensors, the preparation methods of perovskite photodetector arrays are also analyzed. Subsequently, the single-pixel imaging of perovskite photodetectors and the strategies to fabricate narrowband perovskite photodetectors for color discrimination are discussed. Finally, the potential challenges and possible solutions for the future development of perovskite image sensors are presented.

Arrayed MoS2 -In0.53 Ga0.47 As van der Waals Heterostructure for High-Speed and Broadband Detection from Visible to Shortwave-Infrared Light

A high-speed and broadband 5 × 5 photodetector array based on MoS₂ /In_0.53 Ga_0.47 As heterojunction is successfully demonstrated to take full advantage of the type-II band-aligned multilayer MoS₂ /In_0.53 Ga_0.47 As. The fabricated devices exhibit good uniformity in the Raman spectrum and clear rectifying characteristics. The fabricated MoS₂ /In_0.53 Ga_0.47 As photodetectors show good optical performances at a broad wavelength range showing high responsivities corresponding to the detectivity of ≈10¹⁰ Jones at -3 V for the incident broadband light from 400 to 1550 nm. A very fast photoresponse is also obtained with a small rise/fall time in the order of microseconds both for visible (638 nm) and shortwave infrared (1310 nm). Finally, the image scanning properties of MoS₂ /In_0.53 Ga_0.47 As devices are demonstrated for visible and infrared light, indicating that the suggested device is one of the promising options for future broadband imager, which can be integrated on the focal plane arrays (FPAs).

A New Smart CMOS Image Sensor with On-Chip Neuro-Fuzzy Bleeding Detection System for Wireless Capsule Endoscopy

Background:

In this paper, we have presented a new custom smart CMOS image sensor (CIS) for low power wireless capsule endoscopy.

Method:

The proposed new smart CIS includes a 256 × 256 current mode pixels array with a new on-chip adaptive neuro-fuzzy inference system that has been used to diagnosing bleeding images. We use a new pinned photodiode to realize the current mode of active pixels in the standard CMOS process. The proposed chip has been implemented in 0.18 μm CMOS 1P6M TSMC RF technology with a die area of 7 mm × 8 mm.

Results and Conclusion:

A built-in smart bleeding detection system on CIS leads to decrease in the RF transmitter power consumption near zero. The average power dissipation of the proposed smart CIS is 610 μW.

Experimental Comparison between Event and Global Shutter Cameras

We compare event-cameras with fast (global shutter) frame-cameras experimentally, asking: "What is the application domain, in which an event-camera surpasses a fast frame-camera?" Surprisingly, finding the answer has been difficult. Our methodology was to test event- and frame-cameras on generic computer vision tasks where event-camera advantages should manifest. We used two methods: (1) a controlled, cheap, and easily reproducible experiment (observing a marker on a rotating disk at varying speeds); (2) selecting one challenging practical ballistic experiment (observing a flying bullet having a ground truth provided by an ultra-high-speed expensive frame-camera). The experimental results include sampling/detection rates and position estimation errors as functions of illuminance and motion speed; and the minimum pixel latency of two commercial state-of-the-art event-cameras (ATIS, DVS240). Event-cameras respond more slowly to positive than to negative large and sudden contrast changes. They outperformed a frame-camera in bandwidth efficiency in all our experiments. Both camera types provide comparable position estimation accuracy. The better event-camera was limited by pixel latency when tracking small objects, resulting in motion blur effects. Sensor bandwidth limited the event-camera in object recognition. However, future generations of event-cameras might alleviate bandwidth limitations.

Dynamic Temperature Management of Near-Sensor Processing for Energy-Efficient High-Fidelity Imaging

Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Many researchers advocate pushing processing close to the sensor to substantially reduce data movement. However, continuous near-sensor processing raises sensor temperature, impairing imaging/vision fidelity. We characterize the thermal implications of using 3D stacked image sensors with near-sensor vision processing units. Our characterization reveals that near-sensor processing reduces system power but degrades image quality. For reasonable image fidelity, the sensor temperature needs to stay below a threshold, situationally determined by application needs. Fortunately, our characterization also identifies opportunities—unique to the needs of near-sensor processing—to regulate temperature based on dynamic visual task requirements and rapidly increase capture quality on demand. Based on our characterization, we propose and investigate two thermal management strategies—stop-capture-go and seasonal migration—for imaging-aware thermal management. For our evaluated tasks, our policies save up to 53% of system power with negligible performance impact and sustained image fidelity.

Automatic 360° Mono-Stereo Panorama Generation Using a Cost-Effective Multi-Camera System

In recent years, 360° videos have gained the attention of researchers due to their versatility and applications in real-world problems. Also, easy access to different visual sensor kits and easily deployable image acquisition devices have played a vital role in the growth of interest in this area by the research community. Recently, several 360° panorama generation systems have demonstrated reasonable quality generated panoramas. However, these systems are equipped with expensive image sensor networks where multiple cameras are mounted in a circular rig with specific overlapping gaps. In this paper, we propose an economical 360° panorama generation system that generates both mono and stereo panoramas. For mono panorama generation, we present a drone-mounted image acquisition sensor kit that consists of six cameras placed in a circular fashion with optimal overlapping gap. The hardware of our proposed image acquisition system is configured in such way that no user input is required to stitch multiple images. For stereo panorama generation, we propose a lightweight, cost-effective visual sensor kit that uses only three cameras to cover 360° of the surroundings. We also developed stitching software that generates both mono and stereo panoramas using a single image stitching pipeline where the panorama generated by our proposed system is automatically straightened without visible seams. Furthermore, we compared our proposed system with existing mono and stereo contents generation systems in both qualitative and quantitative perspectives, and the comparative measurements obtained verified the effectiveness of our system compared to existing mono and stereo generation systems.

A Microchannel-Confined Crystallization Strategy Enables Blade Coating of Perovskite Single Crystal Arrays for Device Integration

Perovskite single crystals (PSCs) possess superior optoelectronic properties compared to their corresponding polycrystalline films, but their applications of PSCs in high-performance, integrated devices are hindered by their heavy thickness and difficulty in scalable deposition. Here, a microchannel-confined crystallization (MCC) strategy to grow uniform and large-area PSC arrays for integrated device applications is reported. Benefiting from the confinement effect of the microchannels, solution flow dynamics is well controlled, and thus uniform deposition of PSC arrays with suitable thickness is achieved, meaning they are applicable for scale-up device applications. The resulting PSCs possess excellent optoelectronic properties in terms of a long carrier lifetime (175 ns) and an ultralow defect density (2 × 10⁹ cm^-3 ), which are comparable to the corresponding bulk crystals. The unique embedded structure of PSCs within the microchannels allows the construction of a high-integration image sensor. This work paves the way toward high-throughput growth of PSCs for integrated optoelectronic devices.

Modified Distance Transformation for ImageEnhancement in NIR Imaging of Finger Vein System

Most of the current image processing methods used in the near-infrared imaging of fingervascular system concentrate on the extraction of internal structures (veins). In this paper, we proposea novel approach which allows to enhance both internal and external features of a finger. The methodis based on the Distance Transformation and allows for selective extraction of physiological structuresfrom an observed finger. We evaluate the impact of its parameters on the effectiveness of the alreadyestablished processing pipeline used for biometric identification. The new method was comparedwith five state-of-the-art approaches to features extraction (position-gray-profile-curve-PGPGC,maximum curvature points in image profiles-MC, Niblack image adaptive thresholding-NAT,repeated dark line tracking-RDLT, and wide line detector-WD) on the GustoDB database of imagesobtained in a wide range of NIR wavelengths (730-950 nm). The results indicate a clear superiorityof the proposed approach over the remaining alternatives. The method managed to reach over 90%identification accuracy for all analyzed datasets.

A Blockchain-Based Secure Image Encryption Scheme for the Industrial Internet of Things

Smart cameras and image sensors are widely used in industrial processes, from the designing to the quality checking of the final product. Images generated by these sensors are at continuous risk of disclosure and privacy breach in the industrial Internet of Things (IIoT). Traditional solutions to secure sensitive data fade in IIoT environments because of the involvement of third parties. Blockchain technology is the modern-day solution for trust issues and eliminating or minimizing the role of the third party. In the context of the IIoT, we propose a permissioned private blockchain-based solution to secure the image while encrypting it. In this scheme, the cryptographic pixel values of an image are stored on the blockchain, ensuring the privacy and security of the image data. Based on the number of pixels change rate (NPCR), the unified averaged changed intensity (UACI), and information entropy analysis, we evaluate the strength of proposed image encryption algorithm ciphers with respect to differential attacks. We obtained entropy values near to an ideal value of 8, which is considered to be safe from brute force attack. Encrypted results show that the proposed scheme is highly effective for data leakage prevention and security.

Multilayered PdSe2/Perovskite Schottky Junction for Fast, Self-Powered, Polarization-Sensitive, Broadband Photodetectors, and Image Sensor Application

Group-10 transition metal dichalcogenides (TMDs) with distinct optical and tunable electrical properties have exhibited great potential for various optoelectronic applications. Herein, a self-powered photodetector is developed with broadband response ranging from deep ultraviolet to near-infrared by combining FA1- x Cs x PbI3 perovskite with PdSe2 layer, a newly discovered TMDs material. Optoelectronic characterization reveals that the as-assembled PdSe2/perovskite Schottky junction is sensitive to light illumination ranging from 200 to 1550 nm, with the highest sensitivity centered at ≈800 nm. The device also shows a large on/off ratio of ≈104, a high responsivity (R) of 313 mA W-1, a decent specific detectivity (D*) of ≈1013 Jones, and a rapid response speed of 3.5/4 µs. These figures of merit are comparable with or much better than most of the previously reported perovskite detectors. In addition, the PdSe2/perovskite device exhibits obvious sensitivity to polarized light, with a polarization sensitivity of 6.04. Finally, the PdSe2/perovskite detector can readily record five "P," "O," "L," "Y," and "U" images sequentially produced by 808 nm. These results suggest that the present PdSe2/perovskite Schottky junction photodetectors may be useful for assembly of optoelectronic system applications in near future.

A Highly Responsive Organic Image Sensor Based on a Two-Terminal Organic Photodetector with Photomultiplication

Highly responsive organic image sensors are crucial for medical imaging applications. To enhance the pixelwise photoresponse in an organic image sensor, the integration of an organic photodetector with amplifiers, or the use of a highly responsive organic photodetector without an additional amplifying component, is required. The use of vertically stacked, two-terminal organic photodetectors with photomultiplication is a promising approach for highly responsive organic image sensors owing to their simple two-terminal structure and intrinsically large responsivity. However, there are no demonstrations of an imaging sensor array using organic photomultiplication photodetectors. The main obstacle to a sensor array is the weak-light sensitivity, which is limited by a relatively large dark current. Herein, a highly responsive organic image sensor based on monolithic, vertically stacked two-terminal pixels is presented. This is achieved using pixels of a vertically stacked diode-type organic photodetector with photomultiplication. Furthermore, applying an optimized injection electrode and additionally stacked rectifying layers, this two-terminal device simultaneously demonstrates a high responsivity (>40 A W^-1 ), low dark current, and high rectification under illumination. An organic image sensor based on this device with an extremely simple architecture exhibits a high pixel photoresponse, demonstrating a weak-light imaging capability even at 1 µW cm^-2 .

Molecular Engineering of a Donor-Acceptor Polymer To Realize Single Band Absorption toward a Red-Selective Thin-Film Organic Photodiode

Herein, we explore the strategy of realizing a red-selective thin-film organic photodiode (OPD) by synthesizing a new copolymer with a highly selective red-absorption feature. PCZ-Th-DPP, with phenanthrocarbazole (PCZ) and diketopyrrolopyrrole (DPP) as donor and acceptor units, respectively, was strategically designed/synthesized based on a time-dependent density functional theory calculation, which predicted the significant suppression of the band II absorption of PCZ-Th-DPP due to the extremely efficient intramolecular charge transfer. We demonstrate that the synthesized PCZ-Th-DPP exhibits not only a high absorption coefficient within the red-selective band I region, as theoretically predicted, but also a preferential face-on intermolecular structure in the thin-film state, which is beneficial for vertical charge extraction as an outcome of a glancing incidence X-ray diffraction study. By employing PCZ-Th-DPP as a photoactive layer of Schottky OPD, to fully match its absorption characteristic to the spectral response of the red-selective OPD, we demonstrate a genuine red-selective specific detectivity in the order of 10¹² Jones while maintaining a thin active layer thickness of ∼300 nm. This work demonstrates the possibility of realizing a full color image sensor with a synthetic approach to the constituting active layers without optical manipulation.

Plasmonic Color Filter Array with High Color Purity for CMOS Image Sensors

We demonstrate the multiband color filtering of a standard RGB color and a complementary CMY color by a plasmonic color filter, composed of concentric corrugated metallic thin film rings. The surface plasmon resonance is excited by the periodic corrugation, and the coupled light is transmitted through the central subwavelength aperture. Color selectivity is achieved not only in the visible but also in the near-infrared (NIR) region. Therefore, simultaneous imaging with visible and NIR can be realized by the integration of plasmonic color filters with sensors. We investigate the angle of incidence dependence of the transmission color selectivity and the color purity of the fabricated plasmonic color filter array.

Analysis and Design of a CMOS Ultra-High-Speed Burst Mode Imager with In-Situ Storage Topology Featuring In-Pixel CDS Amplification

This paper presents an in-situ storage topology for ultra-high-speed burst mode imagers, enabling low noise operation while keeping a high frame depth. The proposed pixel architecture contains a 4T pinned photodiode, a correlated double sampling (CDS) amplification stage, and an in-situ memory bank. Focusing on the sampling noise, the system level trade-off of the proposed pixel architecture is discussed, showing its advantages on the noise, power, and scaling capability. Integrated with an AC coupling CDS stage, the amplification is obtained by exploiting the strong capacitance to the voltage relation of a single NMOS transistor. A comprehensive noise model is developed for optimizing the trade-off between the area and noise. As a proof-of-concept, a prototype imager with a 30 µm pixel pitch was fabricated in a CMOS 130 nm technology. A 108-cell memory bank is implemented allowing dense layout and parallel readout. Two types of CDS amplification stages were investigated. Despite the limited memory capacitance of 10 fF/cell, the photon transfer curves of both pixel types were measured over different operation speeds up to 20 Mfps showing a noise performance of 8.4 e⁻.

A Bezel-Less Tetrahedral Image Sensor Formed by Solvent-Assisted Plasticization and Transformation of an Acrylonitrile Butadiene Styrene Framework

A method for transforming planar electronic devices into 3D structures under mechanically mild and stable conditions is demonstrated. This strategy involves diffusion control of acetone as a plasticizer into a spatially designed acrylonitrile butadiene styrene (ABS) framework to both laminate membrane-type electronic devices and transform them into a desired 3D shape. Optical, mechanical, and electrical analysis reveals that the plasticized region serves as a damper and even reflows to release the stress of fragile elements, for example, an Au interconnect electrode in this study, below the ultimate stress point. This method also gives considerable freedom in aligning electronic devices not only in the neutral mechanical plane of the ABS framework, which is the general approach in flexible electronics, but also to the top surface, without inducing electrical failure. Finally, to develop a prototype omnidirectional optical system with minimal aberrations, this method is used to produce a bezel-less tetrahedral image sensor.

Application of an RGBD augmented C-arm for minimally invasive scoliosis surgery assistance

Minimally invasive surgeries (MISs) are gaining popularity as alternatives to conventional open surgeries. In thoracoscopic scoliosis MIS, fluoroscopy is used to guide pedicle screw placement and to visualise the effect of the intervention on the spine curvature. However, cosmetic external appearance is the most important concern for patients, while correction of the spine and achieving coronal and sagittal trunk balance are the top priorities for surgeons. The authors present the feasibility study of the first intra-operative assistive system for scoliosis surgery composed of a single RGBD camera affixed on a C-arm which allows visualising in real time the surgery effects on the patient trunk surface in the transverse plane. They perform three feasibility experiments from simulated data based on scoliotic patients to live acquisition from non-scoliotic mannequin and person, all showing that the proposed system accuracy is comparable with scoliotic surface reconstruction state of art.

Hand-eye calibration using a target registration error model

Surgical cameras are prevalent in modern operating theatres and are often used as a surrogate for direct vision. Visualisation techniques (e.g. image fusion) made possible by tracking the camera require accurate hand-eye calibration between the camera and the tracking system. The authors introduce the concept of 'guided hand-eye calibration', where calibration measurements are facilitated by a target registration error (TRE) model. They formulate hand-eye calibration as a registration problem between homologous point-line pairs. For each measurement, the position of a monochromatic ball-tip stylus (a point) and its projection onto the image (a line) is recorded, and the TRE of the resulting calibration is predicted using a TRE model. The TRE model is then used to guide the placement of the calibration tool, so that the subsequent measurement minimises the predicted TRE. Assessing TRE after each measurement produces accurate calibration using a minimal number of measurements. As a proof of principle, they evaluated guided calibration using a webcam and an endoscopic camera. Their endoscopic camera results suggest that millimetre TRE is achievable when at least 15 measurements are acquired with the tracker sensor ∼80 cm away on the laparoscope handle for a target ∼20 cm away from the camera.

ZnO Quantum Dot Decorated Zn2SnO4 Nanowire Heterojunction Photodetectors with Drastic Performance Enhancement and Flexible Ultraviolet Image Sensors

Here we report the fabrication of high-performance ultraviolet photodetectors based on a heterojunction device structure in which ZnO quantum dots were used to decorate Zn₂SnO₄ nanowires. Systematic investigations have shown their ultrahigh light-to-dark current ratio (up to 6.8 × 10⁴), specific detectivity (up to 9.0 × 10¹⁷ Jones), photoconductive gain (up to 1.1 × 10⁷), fast response, and excellent stability. Compared with a pristine Zn₂SnO₄ nanowire, a quantum dot decorated nanowire demonstrated about 10 times higher photocurrent and responsivity. Device physics modeling showed that their high performance originates from the rational energy band engineering, which allows efficient separation of electron-hole pairs at the interfaces between ZnO quantum dots and a Zn₂SnO₄ nanowire. As a result of band engineering, holes migrate to ZnO quantum dots, which increases electron concentration and lifetime in the nanowire conduction channel, leading to significantly improved photoresponse. The enhancement mechanism found in this work can also be used to guide the design of high-performance photodetectors based on other nanomaterials. Furthermore, flexible ultraviolet photodetectors were fabricated and integrated into a 10 × 10 device array, which constitutes a high-performance flexible ultraviolet image sensor. These intriguing results suggest that the band alignment engineering on nanowires can be rationally achieved using compound semiconductor quantum dots. This can lead to largely improved device performance. Particularly for ZnO quantum dot decorated Zn₂SnO₄ nanowires, these decorated nanowires may find broad applications in future flexible and wearable electronics.

3D Arrays of 1024-Pixel Image Sensors based on Lead Halide Perovskite Nanowires

Large-scale and highly ordered 3D perov-skite nanowire (NW) arrays are achieved in nanoengineering templates by a unique vapor-solid-solid reaction process. The excellent material properties, in conjunction with the high integration density of the NW arrays, make them promising for 3D integrated nanoelectronics/optoelectronics. Image sensors with 1024 pixels are assembled and characterized to demonstrate the technological potency.

Nanophotonic Image Sensors

The increasing miniaturization and resolution of image sensors bring challenges to conventional optical elements such as spectral filters and polarizers, the properties of which are determined mainly by the materials used, including dye polymers. Recent developments in spectral filtering and optical manipulating techniques based on nanophotonics have opened up the possibility of an alternative method to control light spectrally and spatially. By integrating these technologies into image sensors, it will become possible to achieve high compactness, improved process compatibility, robust stability and tunable functionality. In this Review, recent representative achievements on nanophotonic image sensors are presented and analyzed including image sensors with nanophotonic color filters and polarizers, metamaterial-based THz image sensors, filter-free nanowire image sensors and nanostructured-based multispectral image sensors. This novel combination of cutting edge photonics research and well-developed commercial products may not only lead to an important application of nanophotonics but also offer great potential for next generation image sensors beyond Moore's Law expectations.

Organic Photodiodes: The Future of Full Color Detection and Image Sensing

Major growth in the image sensor market is largely as a result of the expansion of digital imaging into cameras, whether stand-alone or integrated within smart cellular phones or automotive vehicles. Applications in biomedicine, education, environmental monitoring, optical communications, pharmaceutics and machine vision are also driving the development of imaging technologies. Organic photodiodes (OPDs) are now being investigated for existing imaging technologies, as their properties make them interesting candidates for these applications. OPDs offer cheaper processing methods, devices that are light, flexible and compatible with large (or small) areas, and the ability to tune the photophysical and optoelectronic properties - both at a material and device level. Although the concept of OPDs has been around for some time, it is only relatively recently that significant progress has been made, with their performance now reaching the point that they are beginning to rival their inorganic counterparts in a number of performance criteria including the linear dynamic range, detectivity, and color selectivity. This review covers the progress made in the OPD field, describing their development as well as the challenges and opportunities.

Noise Reduction Techniques and Scaling Effects towards Photon Counting CMOS Image Sensors

This paper presents an overview of the read noise in CMOS image sensors (CISs) based on four-transistors (4T) pixels, column-level amplification and correlated multiple sampling. Starting from the input-referred noise analytical formula, process level optimizations, device choices and circuit techniques at the pixel and column level of the readout chain are derived and discussed. The noise reduction techniques that can be implemented at the column and pixel level are verified by transient noise simulations, measurement and results from recently-published low noise CIS. We show how recently-reported process refinement, leading to the reduction of the sense node capacitance, can be combined with an optimal in-pixel source follower design to reach a sub-0.3erms- read noise at room temperature. This paper also discusses the impact of technology scaling on the CIS read noise. It shows how designers can take advantage of scaling and how the Metal-Oxide-Semiconductor (MOS) transistor gate leakage tunneling current appears as a challenging limitation. For this purpose, both simulation results of the gate leakage current and 1/f noise data reported from different foundries and technology nodes are used.

Aligned Single-Crystalline Perovskite Microwire Arrays for High-Performance Flexible Image Sensors with Long-Term Stability

A simple, low-cost blade-coating method is developed for the large-area fabrication of single-crystalline aligned CH3NH3PbI3 microwire (MW) arrays. The solution-coating method is applicable to flexible substrates, enabling the fabrication of MW-array-based photodetectors with excellent long-term stability, flexibility, and bending durability. Integrated devices from such photodetectors demonstrate high performance for high-resolution, flexible image sensors.

High-Performance Motion Estimation for Image Sensors with Video Compression

It is important to reduce the time cost of video compression for image sensors in video sensor network. Motion estimation (ME) is the most time-consuming part in video compression. Previous work on ME exploited intra-frame data reuse in a reference frame to improve the time efficiency but neglected inter-frame data reuse. We propose a novel inter-frame data reuse scheme which can exploit both intra-frame and inter-frame data reuse for ME in video compression (VC-ME). Pixels of reconstructed frames are kept on-chip until they are used by the next current frame to avoid off-chip memory access. On-chip buffers with smart schedules of data access are designed to perform the new data reuse scheme. Three levels of the proposed inter-frame data reuse scheme are presented and analyzed. They give different choices with tradeoff between off-chip bandwidth requirement and on-chip memory size. All three levels have better data reuse efficiency than their intra-frame counterparts, so off-chip memory traffic is reduced effectively. Comparing the new inter-frame data reuse scheme with the traditional intra-frame data reuse scheme, the memory traffic can be reduced by 50% for VC-ME.

Experimental quantification of the tactile spatial responsivity of human cornea

We present the first experimental quantification of the tactile spatial responsivity of the cornea and we teach a subject to recognize spatial tactile shapes that are stimulated on their cornea.

Smartphone-based analysis of biochemical tests for health monitoring support at home

In the context of home-based healthcare monitoring systems, it is desirable that the results obtained from biochemical tests - tests of various body fluids such as blood and urine - are objective and automatically generated to reduce the number of man-made errors. The authors present the StripTest reader - an innovative smartphone-based interpreter of biochemical tests based on paper-based strip colour using image processing techniques. The working principles of the reader include image acquisition of the colour strip pads using the camera phone, analysing the images within the phone and comparing them with reference colours provided by the manufacturer to obtain the test result. The detection of kidney damage was used as a scenario to illustrate the application of, and test, the StripTest reader. An extensive evaluation using laboratory and human urine samples demonstrates the reader's accuracy and precision of detection, indicating the successful development of a cheap, mobile and smart reader for home-monitoring of kidney functioning, which can facilitate the early detection of health problems and a timely treatment intervention.

A novel method to increase LinLog CMOS sensors' performance in high dynamic range scenarios

Images from high dynamic range (HDR) scenes must be obtained with minimum loss of information. For this purpose it is necessary to take full advantage of the quantification levels provided by the CCD/CMOS image sensor. LinLog CMOS sensors satisfy the above demand by offering an adjustable response curve that combines linear and logarithmic responses. This paper presents a novel method to quickly adjust the parameters that control the response curve of a LinLog CMOS image sensor. We propose to use an Adaptive Proportional-Integral-Derivative controller to adjust the exposure time of the sensor, together with control algorithms based on the saturation level and the entropy of the images. With this method the sensor’s maximum dynamic range (120 dB) can be used to acquire good quality images from HDR scenes with fast, automatic adaptation to scene conditions. Adaptation to a new scene is rapid, with a sensor response adjustment of less than eight frames when working in real time video mode. At least 67% of the scene entropy can be retained with this method.

A new approach to visual-based sensory system for navigation into orange groves

One of the most important parts of an autonomous robot is to establish the path by which it should navigate in order to successfully achieve its goals. In the case of agricultural robotics, a procedure that determines this desired path can be useful. In this paper, a new virtual sensor is introduced in order to classify the elements of an orange grove. This proposed sensor will be based on a color CCD camera with auto iris lens which is in charge of doing the captures of the real environment and an ensemble of neural networks which processes the capture and differentiates each element of the image. Then, the Hough’s transform and other operations will be applied in order to extract the desired path from the classification performed by the virtual sensory system. With this approach, the robotic system can correct its deviation with respect to the desired path. The results show that the sensory system properly classifies the elements of the grove and can set trajectory of the robot.

Implantable CMOS Biomedical Devices

The results of recent research on our implantable CMOS biomedical devices are reviewed. Topics include retinal prosthesis devices and deep-brain implantation devices for small animals. Fundamental device structures and characteristics as well as in vivo experiments are presented.