Structuring of tactile sensory information for category formation in robotics palpation

Vibro-acoustic sensing of tissue-instrument-interactions allows a differentiation of biological tissue in computerised palpation

BACKGROUND

The shift towards minimally invasive surgery is associated with a significant reduction of tactile information available to the surgeon, with compensation strategies ranging from vision-based techniques to the integration of sensing concepts into surgical instruments. Tactile information is vital for palpation tasks such as the differentiation of tissues or the characterisation of surfaces. This work investigates a new sensing approach to derive palpation-related information from vibration signals originating from instrument-tissue-interactions.

METHODS

We conducted a feasibility study to differentiate three non-animal and three animal tissue specimens based on palpation of the surface. A sensor configuration was mounted at the proximal end of a standard instrument opposite the tissue-interaction point. Vibro-acoustic signals of 1680 palpation events were acquired, and the time-varying spectrum was computed using Continuous-Wavelet-Transformation. For validation, nine spectral energy-related features were calculated for a subsequent classification using linear Support Vector Machine and k-Nearest-Neighbor.

RESULTS

Indicators derived from the vibration signal are highly stable in a set of palpations belonging to the same tissue specimen, regardless of the palpating subject. Differences in the surface texture of the tissue specimens reflect in those indicators and can serve as a basis for differentiation. The classification following a supervised learning approach shows an accuracy of >93.8% for the three-tissue classification tasks and decreases to 78.8% for a combination of all six tissues.

CONCLUSIONS

Simple features derived from the vibro-acoustic signals facilitate the differentiation between biological tissues, showing the potential of the presented approach to provide information related to the interacting tissue. The results encourage further investigation of a yet little-exploited source of information in minimally invasive surgery.

Vibro-Acoustic Sensing of Instrument Interactions as a Potential Source of Texture-Related Information in Robotic Palpation

The direct tactile assessment of surface textures during palpation is an essential component of open surgery that is impeded in minimally invasive and robot-assisted surgery. When indirectly palpating with a surgical instrument, the structural vibrations from this interaction contain tactile information that can be extracted and analysed. This study investigates the influence of the parameters contact angle α and velocity v→ on the vibro-acoustic signals from this indirect palpation. A 7-DOF robotic arm, a standard surgical instrument, and a vibration measurement system were used to palpate three different materials with varying α and v→. The signals were processed based on continuous wavelet transformation. They showed material-specific signatures in the time-frequency domain that retained their general characteristic for varying α and v→. Energy-related and statistical features were extracted, and supervised classification was performed, where the testing data comprised only signals acquired with different palpation parameters than for training data. The classifiers support vector machine and k-nearest neighbours provided 99.67% and 96.00% accuracy for the differentiation of the materials. The results indicate the robustness of the features against variations in the palpation parameters. This is a prerequisite for an application in minimally invasive surgery but needs to be confirmed in realistic experiments with biological tissues.

ESPRESS.0: Eustachian Tube-Inspired Tactile Sensor Exploiting Pneumatics for Range Extension and SenSitivity Tuning

Optimising the sensitivity of a tactile sensor to a specific range of stimuli magnitude usually compromises the sensor's widespread usage. This paper presents a novel soft tactile sensor capable of dynamically tuning its stiffness for enhanced sensitivity across a range of applied forces, taking inspiration from the Eustachian tube in the mammalian ear. The sensor exploits an adjustable pneumatic back pressure to control the effective stiffness of its 20 mm diameter elastomer interface. An internally translocated fluid is coupled to the membrane and optically tracked to measure physical interactions at the interface. The sensor can be actuated by pneumatic pressure to dynamically adjust its stiffness. It is demonstrated to detect forces as small as 0.012 N, and to be sensitive to a difference of 0.006 N in the force range of 35 to 40 N. The sensor is demonstrated to be capable of detecting tactile cues on the surface of objects in the sub-millimetre scale. It is able to adapt its compliance to increase its ability for distinguishing between stimuli with similar stiffnesses (0.181 N/mm difference) over a large range (0.1 to 1.1 N/mm) from only a 0.6 mm deep palpation. The sensor is intended to interact comfortably with skin, and the feasibility of its use in palpating tissue in search of hard inclusions is demonstrated by locating and estimating the size of a synthetic hard node embedded 20 mm deep in a soft silicone sample. The results suggest that the sensor is a good candidate for tactile tasks involving unpredictable or unknown stimuli.

A Polymeric Piezoelectric Tactile Sensor Fabricated by 3D Printing and Laser Micromachining for Hardness Differentiation during Palpation

Tactile sensors are important bionic microelectromechanical systems that are used to implement an artificial sense of touch for medical electronics. Compared with the natural sense of touch, this artificial sense of touch provides more quantitative information, augmenting the objective aspects of several medical operations, such as palpation-based diagnosis. Tactile sensors can be effectively used for hardness differentiation during the palpation process. Since palpation requires direct physical contact with patients, medical safety concerns are alleviated if the sensors used can be made disposable. In this respect, the low-cost, rapid fabrication of tactile sensors based on polymers is a possible alternative. The present work uses the 3D printing of elastic resins and the laser micromachining of piezoelectric polymeric films to make a low-cost tactile sensor for hardness differentiation through palpation. The fabricated tactile sensor has a sensitivity of 1.52 V/mm to mechanical deformation at the vertical direction, a sensitivity of 11.72 mV/HA in sensing material hardness with a pressing depth of 500 µm for palpation, and a validated capability to detect rigid objects buried in a soft tissue phantom. Its performance is comparable with existing piezoelectric tactile sensors for similar applications. In addition, the tactile sensor has the additional advantage of providing a simpler microfabrication process.

Variable Stiffness Object Recognition with a CNN-Bayes Classifier on a Soft Gripper

Soft grippers significantly widen the palpation capabilities of robots, ranging from soft to hard materials without the assistance of cameras. From a medical perspective, the detection of size and shape of hard inclusions concealed within soft three-dimensional (3D) objects is meaningful for the early detection of cancer through palpation. This article proposes a framework for variable-stiffness object recognition using tactile information collected by force sensitive resistors on a three-finger soft gripper. A 15 × 50 spatiotemporal tactile image is generated for each 3D palpation process and then fed into a convolutional neural network (CNN) for object identification. The training set consists of tactile images generated from different grasping orientations. We developed our own CNN architecture, named SoftTactNet, and compared its performance with several state-of-the-art CNNs on the image dataset produced by our experiments. The results show that our proposed method excels in distinguishing 3D shapes and sizes of objects enclosed by a thick soft foam. The average recognition rate is significantly improved using a Naive Bayes classifier, reaching a 97% recognition accuracy. The detection of shapes and sizes of hard objects underneath soft tissues is extremely important for breast and testicular cancer early detection, a field where Soft Robots can shine with inexpensive and ubiquitous devices.

How the Environment Shapes Tactile Sensing: Understanding the Relationship Between Tactile Filters and Surrounding Environment

The mechanical properties of a sensor strongly affect its tactile sensing capabilities. By exploiting tactile filters, mechanical structures between the sensing unit and the environment, it is possible to tune the interaction dynamics with the surrounding environment. But how can we design a good tactile filter? Previously, the role of filters’ geometry and stiffness on the quality of the tactile data has been the subject of several studies, both implementing static filters and adaptable filters. State-of-the-art works on online adaptive stiffness highlight a crucial role of the filters’ mechanical behavior in the structure of the recorded tactile data. However, the relationship between the filter’s and the environment’s characteristics is still largely unknown. We want to show the effect of the environment’s mechanical properties on the structure of the acquired tactile data and the performance of a classification task while testing a wide range of static tactile filters. Moreover, we fabricated the filters using four materials commonly exploited in soft robotics, to merge the gap between tactile sensing and robotic applications. We collected data from the interaction with a standard set of twelve objects of different materials, shapes, and textures, and we analyzed the effect of the filter’s material on the structure of such data and the performance of nine common machine learning classifiers, both considering the overall test set and the three individual subsets made by all objects of the same material. We showed that depending on the material of the test objects, there is a drastic change in the performance of the four tested filters, and that the filter that matches the mechanical properties of the environment always outperforms the others.

Action Augmentation of Tactile Perception for Soft-Body Palpation

Medical palpation is a diagnostic technique in which physicians use the sense of touch to manipulate the soft human tissue. This can be done to enable the diagnosis of possibly life-threatening conditions, such as cancer. Palpation is still poorly understood because of the complex interaction dynamics between the practitioners' hands and the soft human body. To understand this complex of soft body interactions, we explore robotic palpation for the purpose of diagnosing the presence of abnormal inclusions, or tumors. Using a Bayesian framework for training and classification, we show that the exploration of soft bodies requires complex, multi-axis, palpation trajectories. We also find that this probabilistic approach is capable of rapidly searching the large action space of the robot. This work progresses “robotic” palpation, and it provides frameworks for understanding and exploiting soft body interactions.

On the Efficiency of Haptic Based Object Identification: Determining Where to Grasp to Get the Most Distinguishing Information

Haptic perception is one of the key modalities in obtaining physical information of objects and in object identification. Most existing literature focused on improving the accuracy of identification algorithms with less attention paid to the efficiency. This work aims to investigate the efficiency of haptic object identification to reduce the number of grasps required to correctly identify an object out of a given object set. Thus, in a case where multiple grasps are required to characterise an object, the proposed algorithm seeks to determine where the next grasp should be on the object to obtain the most amount of distinguishing information. As such, the paper proposes the construction of the object description that preserves the association of the spatial information and the haptic information on the object. A clustering technique is employed both to construct the description of the object in a data set and for the identification process. An information gain (IG) based method is then employed to determine which pose would yield the most distinguishing information among the remaining possible candidates in the object set to improve the efficiency of the identification process. This proposed algorithm is validated experimentally. A Reflex TakkTile robotic hand with integrated joint displacement and tactile sensors is used to perform both the data collection for the dataset and the object identification procedure. The proposed IG approach was found to require a significantly lower number of grasps to identify the objects compared to a baseline approach where the decision was made by random choice of grasps.

Online Morphological Adaptation for Tactile Sensing Augmentation

Sensor morphology and structure has the ability to significantly aid and improve tactile sensing capabilities, through mechanisms such as improved sensitivity or morphological computation. However, different tactile tasks require different morphologies posing a challenge as to how to best design sensors, and also how to enable sensor morphology to be varied. We introduce a jamming filter which, when placed over a tactile sensor, allows the filter to be shaped and molded online, thus varying the sensor structure. We demonstrate how this is beneficial for sensory tasks analyzing how the change in sensor structure varies the information that is gained using the sensor. Moreover, we show that appropriate morphology can significantly influence discrimination, and observe how the selection of an appropriate filter can increase the object classification accuracy when using standard classifiers by up to 28%.