Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications

Uneven Terrain Recognition Using Neuromorphic Haptic Feedback

Recent years have witnessed relevant advancements in the quality of life of persons with lower limb amputations thanks to the technological developments in prosthetics. However, prostheses that provide information about the foot-ground interaction, and in particular about terrain irregularities, are still missing on the market. The lack of tactile feedback from the foot sole might lead subjects to step on uneven terrains, causing an increase in the risk of falling. To address this issue, a biomimetic vibrotactile feedback system that conveys information about gait and terrain features sensed by a dedicated insole has been assessed with intact subjects. After having shortly experienced both even and uneven terrains, the recruited subjects discriminated them with an accuracy of 87.5%, solely relying on the replay of the vibrotactile feedback. With the objective of exploring the human decoding mechanism of the feedback startegy, a KNN classifier was trained to recognize the uneven terrains. The outcome suggested that the subjects achieved such performance with a temporal dynamics of 45 ms. This work is a leap forward to assist lower-limb amputees to appreciate the floor conditions while walking, adapt their gait and promote a more confident use of their artificial limb.

An Overview of Wearable Haptic Technologies and Their Performance in Virtual Object Exploration

We often interact with our environment through manual handling of objects and exploration of their properties. Object properties (OP), such as texture, stiffness, size, shape, temperature, weight, and orientation provide necessary information to successfully perform interactions. The human haptic perception system plays a key role in this. As virtual reality (VR) has been a growing field of interest with many applications, adding haptic feedback to virtual experiences is another step towards more realistic virtual interactions. However, integrating haptics in a realistic manner, requires complex technological solutions and actual user-testing in virtual environments (VEs) for verification. This review provides a comprehensive overview of recent wearable haptic devices (HDs) categorized by the OP exploration for which they have been verified in a VE. We found 13 studies which specifically addressed user-testing of wearable HDs in healthy subjects. We map and discuss the different technological solutions for different OP exploration which are useful for the design of future haptic object interactions in VR, and provide future recommendations.

Modeling the basic behaviors of Anesthesia Training in Relation to Puncture and Penetration Feedback

Failure rates in spinal anesthesia are generally low in experienced hands. However, studies report a failure rate variation of 1% to 17% in this procedure. The aim of this study is to bring the main characteristics of in vivo procedure to the virtual reality simulated environment. The first step is to model the behavior of tissue layers being punctured by a needle to then make its inclusion in medical training possible. The simulation proposed here is implemented using a Phantom Omni haptic device. Every crucial sensation of the method mentioned here was assessed by a dozen volunteers who participated in two experiments designed to validate the modeled response. Each user answered six questions (three for each experiment). Good results were achieved in certain essential aspects of the process, such as identifying the number of layers, the most rigid layer to puncture, and the most resistant layers to pass through. These results indicated that it is possible to represent many typical behaviors through virtual needle insertion in spinal anesthesia with the correct use of haptic properties.Clinical relevance- The idea is to create a spinal anesthesia simulator that could work as a complementary step in training new anesthetists. The use of a simulator avoids introducing the first puncture haptic sensation directly in patients.

Perception of Time-Discrete Haptic Feedback on the Waist is Invariant With Gait Events

The effectiveness of haptic feedback devices highly depends on the perception of tactile stimuli, which differs across body parts and can be affected by movement. In this study, a novel wearable sensory feedback apparatus made of a pair of pressure-sensitive insoles and a belt equipped with vibrotactile units is presented; the device provides time-discrete vibrations around the waist, synchronized with biomechanically-relevant gait events during walking. Experiments with fifteen healthy volunteers were carried out to investigate users' tactile perception on the waist. Stimuli of different intensities were provided at twelve locations, each time synchronously with one pre-defined gait event (i.e. heel strike, flat foot or toe off), following a pseudo-random stimulation sequence. Reaction time, detection rate and localization accuracy were analyzed as functions of the stimulation level and site and the effect of gait events on perception was investigated. Results revealed that above-threshold stimuli (i.e. vibrations characterized by acceleration amplitudes of 1.92g and 2.13g and frequencies of 100 Hz and 150 Hz, respectively) can be effectively perceived in all the sites and successfully localized when the intertactor spacing is set to 10 cm. Moreover, it was found that perception of time-discrete vibrations was not affected by phase-related gating mechanisms, suggesting that the waist could be considered as a preferred body region for delivering haptic feedback during walking.

A Machine-Learning-Based Approach to Solve Both Contact Location and Force in Soft Material Tactile Sensors

This study addresses a design and calibration methodology based on numerical finite element method (FEM) modeling for the development of a soft tactile sensor able to simultaneously solve the magnitude and the application location of a normal load exerted onto its surface. The sensor entails the integration of a Bragg grating fiber optic sensor in a Dragon Skin 10 polymer brick (110 mm length, 24 mm width). The soft polymer mediates the transmission of the applied load to the buried fiber Bragg gratings (FBGs), and we also investigated the effect of sensor thickness on receptive field and sensitivity, both with the developed model and experimentally. Force-controlled indentations of the sensor (up to 2.5 N) were carried out through a cylindrical probe applied along the direction of the optical fiber (over an ∼90 mm span in length). A finite element model of the sensor was built and experimentally validated for 1 and 6 mm thicknesses of the soft polymeric encapsulation material, considering that the latter thickness resulted from numerical simulations as leading to optimal cross talk and sensitivity, given the chosen soft material. The FEM model was also used to train a neural network so as to obtain the inverse sensor function. Using four FBG transducers embedded in the 6-mm-thick soft polymer, the proposed machine learning approach managed to accurately detect both load magnitude (R = 0.97) and location (R = 0.99) over the whole experimental range. The proposed system could be used for developing tactile sensors that can be effectively used for a broad range of applications.

A Modular Transradial Bypass Socket for Surface Myoelectric Prosthetic Control in Non-Amputees

Bypass sockets allow researchers to perform tests of prosthetic systems from the prosthetic user's perspective. We designed a modular upper-limb bypass socket with 3D-printed components that can be easily modified for use with a variety of terminal devices. Our bypass socket preserves access to forearm musculature and the hand, which are necessary for surface electromyography and to provide substituted sensory feedback. Our bypass socket allows a sufficient range of motion to complete tasks in the frontal working area, as measured on non-amputee participants. We examined the performance of non-amputee participants using the bypass socket on the original and modified Box and Block Tests. Participants moved 11.3 ± 2.7 and 11.7 ± 2.4 blocks in the original and modified Box and Block Tests (mean ± SD), respectively, within the range of reported scores using amputee participants. Range of motion for users wearing the bypass socket meets or exceeds most reported range of motion requirements for activities of daily living. The bypass socket was originally designed with a freely rotating wrist; we found that adding elastic resistance to user wrist rotation while wearing the bypass socket had no significant effect on motor decode performance. We have open-sourced the design files and an assembly manual for the bypass socket. We anticipate that the bypass socket will be a useful tool to evaluate and develop sensorized myoelectric prosthesis technology.

A Mechatronic Platform for Computer Aided Detection of Nodules in Anatomopathological Analyses via Stiffness and Ultrasound Measurements

This study presents a platform for ex-vivo detection of cancer nodules, addressing automation of medical diagnoses in surgery and associated histological analyses. The proposed approach takes advantage of the property of cancer to alter the mechanical and acoustical properties of tissues, because of changes in stiffness and density. A force sensor and an ultrasound probe were combined to detect such alterations during force-regulated indentations. To explore the specimens, regardless of their orientation and shape, a scanned area of the test sample was defined using shape recognition applying optical background subtraction to the images captured by a camera. The motorized platform was validated using seven phantom tissues, simulating the mechanical and acoustical properties of ex-vivo diseased tissues, including stiffer nodules that can be encountered in pathological conditions during histological analyses. Results demonstrated the platform’s ability to automatically explore and identify the inclusions in the phantom. Overall, the system was able to correctly identify up to 90.3% of the inclusions by means of stiffness in combination with ultrasound measurements, paving pathways towards robotic palpation during intraoperative examinations.

Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence †

Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions.

Tactile Display Design for Flight Envelope Protection and Situational Awareness

Spatial disorientation and visual channel saturation are defined as critical situations encountered by military pilots. Such subjects are interesting research areas likely to create innovative systems able to surmount obstacles of this kind. The integration of new stimulation techniques (sensory substitute, adjunct for visual and audio feedback) may be considered to make the visual channel better. This contribution may help in integrating tactile stimulation to improve or substitute the visual channel. It may also help to better interpret the spatial disorientation awareness signals and the vestibulo-ocular response limitations. The innovation of the proposed approach translates in: (i) the development of the PI-Inverse dynamics controller to provide a time delay reduction of the low cost tactile actuator, and thus, high-performance tactile system; (ii) an approach based on fuzzy logic controller (FLC) is being used in order to translate the turn rate angle, the flight path climb angle and the warning messages into tactile signal features instead of a conventional approach based on direct coding of the pitch and bank angles; and (iii) the consideration of the flight envelope. The fuzzy set translation of flight parameters into tactile signals is also a pragmatic and useful way to design the system.

Less Data Same Information for Event-Based Sensors: A Bioinspired Filtering and Data Reduction Algorithm

Sensors provide data which need to be processed after acquisition to remove noise and extract relevant information. When the sensor is a network node and acquired data are to be transmitted to other nodes (e.g., through Ethernet), the amount of generated data from multiple nodes can overload the communication channel. The reduction of generated data implies the possibility of lower hardware requirements and less power consumption for the hardware devices. This work proposes a filtering algorithm (LDSI—Less Data Same Information) which reduces the generated data from event-based sensors without loss of relevant information. It is a bioinspired filter, i.e., event data are processed using a structure resembling biological neuronal information processing. The filter is fully configurable, from a “transparent mode” to a very restrictive mode. Based on an analysis of configuration parameters, three main configurations are given: weak, medium and restrictive. Using data from a DVS event camera, results for a similarity detection algorithm show that event data can be reduced up to 30% while maintaining the same similarity index when compared to unfiltered data. Data reduction can reach 85% with a penalty of 15% in similarity index compared to the original data. An object tracking algorithm was also used to compare results of the proposed filter with other existing filter. The LDSI filter provides less error (4.86 ± 1.87) when compared to the background activity filter (5.01 ± 1.93). The algorithm was tested under a PC using pre-recorded datasets, and its FPGA implementation was also carried out. A Xilinx Virtex6 FPGA received data from a 128 × 128 DVS camera, applied the LDSI algorithm, created a AER dataflow and sent the data to the PC for data analysis and visualization. The FPGA could run at 177 MHz clock speed with a low resource usage (671 LUT and 40 Block RAM for the whole system), showing real time operation capabilities and very low resource usage. The results show that, using an adequate filter parameter tuning, the relevant information from the scene is kept while fewer events are generated (i.e., fewer generated data).