A tactile enhancement instrument for minimally invasive surgery

Perceptual Space of Algorithms for Three-to-One Dimensional Reduction of Realistic Vibrations

Haptics researchers often endeavor to deliver realistic vibrotactile feedback through broad-bandwidth actuators; however, these actuators typically generate only single-axis vibrations, not 3D vibrations like those that occur in natural tool-mediated interactions. Several three-to-one (321) dimensional reduction algorithms have thus been developed to combine 3D vibrations into 1D vibrations. Surprisingly, the perceptual quality of 321-converted vibrations has never been comprehensively compared to rendering of the original 3D signals. In this study, we develop a multi-dimensional vibration rendering system using a magnetic levitation haptic interface. We verify the system's ability to generate realistic 3D vibrations recorded in both tapping and dragging interactions with four surfaces. We then conduct a study with 15 participants to measure the perceived dissimilarities between five 321 algorithms (SAZ, SUM, VM, DFT, PCA) and the original recordings. The resulting perceptual space is investigated with multiple regression and Procrustes analysis to unveil the relationship between the physical and perceptual properties of 321-converted vibrations. Surprisingly, we found that participants perceptually discriminated the original 3D vibrations from all tested 1D versions. Overall, our results indicate that spectral, temporal, and directional attributes may all contribute to the perceived similarities of vibration signals.

Vibrotactile Feedback System From the Fingertip to the Temples for Perceptual Enhancement of Contracture Palpation

Contractures are generally assessed by a physician or physical therapist through palpation. However, contracture palpation requires skill and experience. The frictional vibration, which has a pulse-like vibration due to sliding disturbances around the affected area during palpation, is important in assessing the degree of contracture progression. This paper aims to enhance the perceptual sensitivity of frictional vibration for contracture palpation using a vibrotactile feedback system. We previously proposed an evaluation system for palpation with a wearable skin vibration sensor that detects skin-propagated vibration, allowing touch with a bare fingertip. In this paper, we propose the vibrotactile feedback system that presents the tactile information of the fingertip detected by the wearable tactile sensor to the temples with a vibrotactile display. A stimulator that gives vibrations similar to those during the palpation, which include pulse-like vibration and small vibration, was assembled. Then, psychophysical experiments on the vibrotactile feedback system were conducted using this stimulator. The results showed that the detection sensitivity of the pulse-like vibration was significantly enhanced with the feedback.

A Pneumatic Tactile Ring for Instantaneous Sensory Feedback in Laparoscopic Tumor Localization

We aim to achieve intraoperative localization of an early-stage gastric tumor that cannot be visually detected during laparoscopic surgery. In this study, we developed and evaluated a pneumatic tactile ring, which is a clinically applicable tactile device to provide instantaneous feedback from a tactile sensor directly manipulated by a surgeon. It was designed to be worn on the finger of the manipulating hand and to present pressure to the finger pad. It is lightweight, cost-effective, disposable, and sterilizable. We also developed a compact pneumatic drive unit to control the pressure and investigated its fundamental performance. The bandwidth of the pressure control was at least 1.3 Hz with a controllable range of up to 79.7 kPa. Moreover, a psychophysical experiment was performed to obtain the Weber ratio of the pressure and evaluate the effectiveness of the instantaneous tactile feedback of the sensor output through the tactile ring. The Weber ratio was 0.40 at the reference pressure of 22.7 kPa. The provided tactile feedback significantly reduced the absolute localization error and increased participants' confidence in their answers. It was shown that the tactile feedback through the ring is effective in laparoscopic tumor localization.

Endomicroscopy for Computer and Robot Assisted Intervention

Endomicroscopy is a new technique that allows human tissue to be characterized in vivo and in situ, circumventing the need for conventional biopsy and histology. Despite increased application and growing research interests in this area, the clinical application of endomicroscopy, however, is limited by difficulties in ergonomic control, consistent probe-tissue contact, large area surveillance, and retargeting. Recently, advances in high-speed imaging, mosaicing, and robotics have aimed to address these difficulties. The development of robot-assisted devices in particular has shown great promises in extending the clinical potential of endomicroscopy. Issues related to miniaturization, adaptation to tissue deformation, control stability, force and position compensation, cost, and sterility are being pursued by both research and commercial communities. In this review, recent clinical and technical developments in different aspects of computer and robotic assisted endomicroscopy interventions including instrumentation, multiscale integration, and high-speed imaging techniques are presented. We further address emerging trends and new research opportunities toward more widespread clinical acceptance of robotically assisted endomicroscopy technologies.

The brain uses extrasomatic information to estimate limb displacement

A fundamental problem faced by the brain is to estimate whether a touched object is rigidly attached to a ground reference or is movable. A simple solution to this problem would be for the brain to test whether pushing on the object with a limb is accompanied by limb displacement. The mere act of pushing excites large populations of mechanoreceptors, generating a sensory response that is only weakly sensitive to limb displacement if the movements are small, and thus can hardly be used to determine the mobility of the object. In the mechanical world, displacement or deformation of objects frequently co-occurs with microscopic fluctuations associated with the frictional sliding of surfaces in contact or with micro-failures inside an object. In this study,we provide compelling evidence that the brain relies on these microscopic mechanical events to estimate the displacement of the limb in contact with an object, and hence the mobility of the touched object. We show that when pressing with a finger on a stiff surface, fluctuations that resemble the mechanical response of granular solids provoke a sensation of limb displacement. Our findings suggest that when acting on an external object, prior knowledge about the sensory consequences of interacting with the object contributes to proprioception.

Psychophysical evaluation of haptic perception under augmentation by a handheld device

OBJECTIVE

This study investigated the effectiveness of force augmentation in haptic perception tasks.

BACKGROUND

Considerable engineering effort has been devoted to developing force augmented reality (AR) systems to assist users in delicate procedures like microsurgery. In contrast, far less has been done to characterize the behavioral outcomes of these systems, and no research has systematically examined the impact of sensory and perceptual processes on force augmentation effectiveness.

METHOD

Using a handheld force magnifier as an exemplar haptic AR, we conducted three experiments to characterize its utility in the perception of force and stiffness. Experiments 1 and 2 measured, respectively, the user's ability to detect and differentiate weak force (<0.5 N) with or without the assistance of the device and compared it to direct perception. Experiment 3 examined the perception of stiffness through the force augmentation.

RESULTS

The user's ability to detect and differentiate small forces was significantly improved by augmentation at both threshold and suprathreshold levels. The augmentation also enhanced stiffness perception. However, although perception of augmented forces matches that of the physical equivalent for weak forces, it falls off with increasing intensity.

CONCLUSION

The loss in the effectiveness reflects the nature of sensory and perceptual processing. Such perceptual limitations should be taken into consideration in the design and development of haptic AR systems to maximize utility.

APPLICATION

The findings provide useful information for building effective haptic AR systems, particularly for use in microsurgery.

Magnetic resonance imaging-compatible tactile sensing device based on a piezoelectric array

Minimally invasive surgery is a widely used medical technique, one of the drawbacks of which is the loss of direct sense of touch during the operation. Palpation is the use of fingertips to explore and make fast assessments of tissue morphology. Although technologies are developed to equip minimally invasive surgery tools with haptic feedback capabilities, the majority focus on tissue stiffness profiling and tool-tissue interaction force measurement. For greatly increased diagnostic capability, a magnetic resonance imaging-compatible tactile sensor design is proposed, which allows minimally invasive surgery to be performed under image guidance, combining the strong capability of magnetic resonance imaging soft tissue and intuitive palpation. The sensing unit is based on a piezoelectric sensor methodology, which conforms to the stringent mechanical and electrical design requirements imposed by the magnetic resonance environment The sensor mechanical design and the device integration to a 0.2 Tesla open magnetic resonance imaging scanner are described, together with the device's magnetic resonance compatibility testing. Its design limitations and potential future improvements are also discussed. A tactile sensing unit based on a piezoelectric sensor principle is proposed, which is designed for magnetic resonance imaging guided interventions.

Creating Realistic Virtual Textures from Contact Acceleration Data

Modern haptic interfaces are adept at conveying the large-scale shape of virtual objects, but they often provide unrealistic or no feedback when it comes to the microscopic details of surface texture. Direct texture-rendering challenges the state of the art in haptics because it requires a finely detailed model of the surface's properties, real-time dynamic simulation of complex interactions, and high-bandwidth haptic output to enable the user to feel the resulting contacts. This paper presents a new, fully realized solution for creating realistic virtual textures. Our system employs a sensorized handheld tool to capture the feel of a given texture, recording three-dimensional tool acceleration, tool position, and contact force over time. We reduce the three-dimensional acceleration signals to a perceptually equivalent one-dimensional signal, and then we use linear predictive coding to distill this raw haptic information into a database of frequency-domain texture models. Finally, we render these texture models in real time on a Wacom tablet using a stylus augmented with small voice coil actuators. The resulting virtual textures provide a compelling simulation of contact with the real surfaces, which we verify through a human subject study.

Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment

Virtual reality (VR) technology is being used with increasing frequency as a training medium for motor rehabilitation. However, before addressing training effectiveness in virtual environments (VEs), it is necessary to identify if movements made in such environments are kinematically similar to those made in physical environments (PEs) and the effect of provision of haptic feedback on these movement patterns. These questions are important since reach-to-grasp movements may be inaccurate when visual or haptic feedback is altered or absent. Our goal was to compare kinematics of reaching and grasping movements to three objects performed in an immersive three-dimensional (3D) VE with haptic feedback (cyberglove/grasp system) viewed through a head-mounted display to those made in an equivalent physical environment (PE). We also compared movements in PE made with and without wearing the cyberglove/grasp haptic feedback system. Ten healthy subjects (8 women, 62.1±8.8years) reached and grasped objects requiring 3 different grasp types (can, diameter 65.6mm, cylindrical grasp; screwdriver, diameter 31.6mm, power grasp; pen, diameter 7.5mm, precision grasp) in PE and visually similar virtual objects in VE. Temporal and spatial arm and trunk kinematics were analyzed. Movements were slower and grip apertures were wider when wearing the glove in both the PE and the VE compared to movements made in the PE without the glove. When wearing the glove, subjects used similar reaching trajectories in both environments, preserved the coordination between reaching and grasping and scaled grip aperture to object size for the larger object (cylindrical grasp). However, in VE compared to PE, movements were slower and had longer deceleration times, elbow extension was greater when reaching to the smallest object and apertures were wider for the power and precision grip tasks. Overall, the differences in spatial and temporal kinematics of movements between environments were greater than those due only to wearing the cyberglove/grasp system. Differences in movement kinematics due to the viewing environment were likely due to a lack of prior experience with the virtual environment, an uncertainty of object location and the restricted field-of-view when wearing the head-mounted display. The results can be used to inform the design and disposition of objects within 3D VEs for the study of the control of prehension and for upper limb rehabilitation.

Measurement-based modeling of contact forces and textures for haptic rendering

Haptic texture represents the fine-grained attributes of an object's surface and is related to physical characteristics such as roughness and stiffness. We introduce an interactive and mobile scanning system for the acquisition and synthesis of haptic textures that consists of a visually tracked handheld touch probe. The most novel aspect of our work is an estimation method for the contact stiffness of an object based solely on the acceleration and forces measured during stroking of its surface with the handheld probe. We establish an experimental relationship between the estimated stiffness and the contact stiffness observed during compression. We also measure the height-displacement profile of an object's surface enabling us to generate haptic textures. We show an example of mapping the textures on to a coarse surface mesh obtained with an image-based technique, but the textures may also be combined with coarse surface meshes obtained by manual modeling.

Review on aspects of artificial tactile feedback in laparoscopic surgery

Since direct manual tissue palpation is not possible in laparoscopic surgery, feedback information on tactile tissue properties is considerably diminished. Restoring part of the surgeon's sense of touch through devices capable of providing artificial tactile feedback (ATF) is an active field of applied research and development. Despite more than two decades of research, technical development of such devices is still basic, and pre-clinical as well as clinical experience is limited. This article provides an overview of the technological aspects of ATF in laparoscopic surgery, gives background information on principles of human perception of related feedback information, and reviews current research attempts in the field of ATF systems in laparoscopic surgery, broken down into three main system components: tactile sensor, display, and data processing. Tactile sensors have been developed to measure tissue compliance, reveal hidden structures or foreign bodies in tissue through measurement of pressure distribution, and to identify and locate arteries by detecting their pulsation. Furthermore, different solutions for presenting tactile data to the surgeon have been developed. Visual and auditory displays are easy to implement into the operating room equipment, while tactile displays still suffer from difficulties concerning their performance and requirements for clinical usability. The role of the data processing system as the linking component in an artificial tactile feedback system has been identified as crucial for effectiveness of the system and easy reception of tactile data by the surgeon. The investigations on theoretical and technological foundations of ATF have led to an extensive database in recent years. An application-driven development approach will likely be a driving factor in the future evolution of this field.

Improving Tactile Sensation in Minimally Invasive Pediatric Surgery

PURPOSE

The aim of this study was to investigate factors that impact tactile sensation during minimally invasive pediatric surgery.

METHODS

Three different 3-mm Maryland laparoscopic instruments were tested with and without the resistance of a trocar (Ethicon 3-mm): Jarit (24-cm shaft, 113 g), Storz (30-cm shaft, 62 g), and an ultra-light prototype (24-cm shaft, 5 g). Experiments were conducted in a custom-designed laparoscopic simulator that directs instruments at fixed angles toward a central target. Surgeons were instructed to insert the instruments into the simulator and make contact with the target with as little force as possible. Instantaneous pressure measurements on the target were measured and recorded every 0.0001 seconds. The differences between impact pressures were compared with a paired, two-tailed, Student's t test.

RESULTS

Twenty-seven (27) surgeons participated in the study. The ultra-light prototype had significantly lower impact pressures than the Storz instrument at all angles both with a trocar (P < 0.05) and without a trocar (P < 0.001). The ultra-light prototype had significantly lower impact pressures than the Jarit instrument at all angles in the absence of a trocar (P < 0.001), but with a trocar in place the only significant difference was at 5 degrees (P < 0.001). The presence of the trocar on the ultra-light prototype had a negative impact on tactile sensation that was statistically significant (P < 0.01).

CONCLUSIONS

The presence of a trocar negatively impacted the surgeon's tactile sensation. Decreasing instrument mass by 10- to 20 fold did make a statistically significant improvement in tactile sensation.

Haptic discrimination of two-dimensional angles: influence of exploratory strategy

The aim of this study was to define the relative contribution of self-generated cutaneous and proprioceptive feedback to haptic shape discrimination by systematically constraining the exploratory strategy. Subjects (n = 23) explored pairs of two-dimensional (2-D) angles (standard angle, 90 degrees; comparison angles, 91 degrees -103 degrees) placed at arm's length from the subject, and identified the larger angle of each pair. The exploratory strategies included a reference condition, dynamic scan of the index finger over the entire object [combined cutaneous and proprioceptive (shoulder) feedback], and modified conditions, static touch of the intersection of the two bars that formed the angle using the index finger (cutaneous feedback) and dynamic scans of the object using a hand-held tool (proprioceptive feedback, shoulder). Discrimination thresholds (75% correct) were very similar for dynamic and static touch with the index finger. Thresholds varied as a function of the static contact duration (<1 s, 7.2 degrees +/- 0.6 degrees; approximately 3 s, 4.2 degrees +/- 0.5 degrees), but were not different from the reference condition (6.0 degrees +/- 0.9 degrees). The higher threshold with short static touch likely reflects movement-related gating of self-generated tactile inputs. Together, the results suggested that cutaneous feedback alone may be sufficient to explain 2-D angle discrimination, because the added proprioceptive feedback did not improve performance. Also, threshold did not vary with the number of dynamic scans (one or two), suggesting that the critical information was gathered on the first pass over the angle. In contrast, when the angles were explored with the tool, the threshold increased relative to the corresponding reference condition from the same session (tool, 9.6 degrees +/- 0.9 degrees; dynamic scan with the finger, 6.2 degrees +/- 1.0 degrees). Thus, performance was poorer with proprioceptive feedback alone, suggesting that cutaneous feedback was relatively more important for 2-D haptic angle discrimination in the present experiment.