Plenoptic Cameras in Surgical Robotics: Calibration, Registration, and Evaluation

Measurement Technologies of Light Field Camera: An Overview

Visual measurement methods are extensively used in various fields, such as aerospace, biomedicine, agricultural production, and social life, owing to their advantages of high speed, high accuracy, and non-contact. However, traditional camera-based measurement systems, relying on the pinhole imaging model, face challenges in achieving three-dimensional measurements using a single camera by one shot. Moreover, traditional visual systems struggle to meet the requirements of high precision, efficiency, and compact size simultaneously. With the development of light field theory, the light field camera has garnered significant attention as a novel measurement method. Due to its special structure, the light field camera enables high-precision three-dimensional measurements with a single camera through only one shot. This paper presents a comprehensive overview of light field camera measurement technologies, including the imaging principles, calibration methods, reconstruction algorithms, and measurement applications. Additionally, we explored future research directions and the potential application prospects of the light field camera.

Image Feature Information Extraction for Interest Point Detection: A Comprehensive Review

Interest point detection is one of the most fundamental and critical problems in computer vision and image processing. In this paper, we carry out a comprehensive review on image feature information (IFI) extraction techniques for interest point detection. To systematically introduce how the existing interest point detection methods extract IFI from an input image, we propose a taxonomy of the IFI extraction techniques for interest point detection. According to this taxonomy, we discuss different types of IFI extraction techniques for interest point detection. Furthermore, we identify the main unresolved issues related to the existing IFI extraction techniques for interest point detection and any interest point detection methods that have not been discussed before. The existing popular datasets and evaluation standards are provided and the performances for fifteen state-of-the-art approaches are evaluated and discussed. Moreover, future research directions on IFI extraction techniques for interest point detection are elaborated.

Hand-eye calibration for an unfocused light-field camera

A calibration framework is established for an unfocused light-field camera and a robotic arm. With Gaussian optics and light-field imaging principles, the mapping relationship between a point light source and its corresponding plenoptic disc feature is established, and the intrinsic and extrinsic parameters of the unfocused light-field camera are calculated through nonlinear optimization. Transformation matrices for eye-to-hand and eye-in-hand configurations are subsequently solved and are validated by applying them to an industrial light-field camera-robotic arm system. With the proposed calibration method, 3D reconstruction for calibration board in different poses is demonstrated and calibration uncertainty is discussed in detail.

Monitoring Wound Health through Bandages with Passive LC Resonant Sensors

This paper details a passive, inductor-capacitor (LC) resonant sensor embedded in a commercial dressing for low-cost, contact-free monitoring of a wound; this would enable tracking of the healing process while keeping the site closed and sterile. Spiral LC resonators were fabricated from flexible, copper-coated polyimide and interrogated using external reader antennas connected to a two-port vector network analyzer; the forward transmission scattering parameter (S₂₁) magnitude was collected, and the resonant frequency (MHz) and the peak-to-peak amplitude of the resonant feature were identified. These increase during the healing process as the permittivity and conductivity of the tissue change. The sensor was first tested on gelatin-based tissue-mimicking phantoms that simulate layers of muscle, blood, fat, and skin at varying phases of wound healing. Finite element modeling was also used to verify the empirical results based on the expected variations in dielectric properties of the tissue. The performance of the resonant sensors for in vivo applications was investigated by conducting animal studies using canine patients that presented with a natural wound as well as a controlled cohort of rat models with surgically administered wounds. Finally, transfer functions are presented that relate the resonant frequency to wound size using an exponential model (R² = 0.58-0.96). The next steps in sensor design and fabrication as well as the reading platform to achieve the goal of a universal calibration curve are then discussed.

Biocompatible Near-Infrared Three-Dimensional Tracking System

A fundamental challenge in soft-tissue surgery is that target tissue moves and deforms, becomes occluded by blood or other tissue, and is difficult to differentiate from surrounding tissue. We developed small biocompatible near-infrared fluorescent (NIRF) markers with a novel fused plenoptic and NIR camera tracking system, enabling three-dimensional tracking of tools and target tissue while overcoming blood and tissue occlusion in the uncontrolled, rapidly changing surgical environment. In this work, we present the tracking system and marker design and compare tracking accuracies to standard optical tracking methods using robotic experiments. At speeds of 1 mm/s, we observe tracking accuracies of 1.61 mm, degrading only to 1.71 mm when the markers are covered in blood and tissue.