Palpation force modulation strategies to identify hard regions in soft tissue organs

Sensor-Based Discovery of Search and Palpation Modes in the Clinical Breast Examination

PURPOSE

Successful implementation of precision education systems requires widespread adoption and seamless integration of new technologies with unique data streams that facilitate real-time performance feedback. This paper explores the use of sensor technology to quantify hands-on clinical skills. The goal is to shorten the learning curve through objective and actionable feedback.

METHOD

A sensor-enabled clinical breast examination (CBE) simulator was used to capture force and video data from practicing clinicians (N = 152). Force-by-time markers from the sensor data and a machine learning algorithm were used to parse physicians' CBE performance into periods of search and palpation and then these were used to investigate distinguishing characteristics of successful versus unsuccessful attempts to identify masses in CBEs.

RESULTS

Mastery performance from successful physicians showed stable levels of speed and force across the entire CBE and a 15% increase in force when in palpation mode compared with search mode. Unsuccessful physicians failed to search with sufficient force to detect deep masses ( F [5,146] = 4.24, P = .001). While similar proportions of male and female physicians reached the highest performance level, males used more force as noted by higher palpation to search force ratios ( t [63] = 2.52, P = .014).

CONCLUSIONS

Sensor technology can serve as a useful pathway to assess hands-on clinical skills and provide data-driven feedback. When using a sensor-enabled simulator, the authors found specific haptic approaches that were associated with successful CBE outcomes. Given this study's findings, continued exploration of sensor technology in support of precision education for hands-on clinical skills is warranted.

Multi-silicone bilateral soft physical twin as an alternative to traditional user interfaces for remote palpation: a comparative study

Teleoperated medical technologies are a fundamental part of the healthcare system. From telemedicine to remote surgery, they allow remote diagnosis and treatment. However, the absence of any interface able to effectively reproduce the sense of touch and interaction with the patient prevents the implementation of teleoperated systems for primary care examinations, such as palpation. In this paper, we propose the first reported case of a soft robotic bilateral physical twin for remote palpation. By creating an entirely soft interface that can be used both to control the robot and receive feedback, the proposed device allows the user to achieve remote palpation by simply palpating the soft physical twin. This is achieved through a compact design showcasing 9 pneumatic chambers and exploiting multi-silicone casting to minimize cross-noise and allow teleoperation. A comparative study has been run against a traditional setup, and both the control and feedback of the physical twin are carefully analyzed. Despite distributed tactile feedback not achieving the same performance as the visual map, the soft control and visual feedback combination showcases a 5.1% higher accuracy. Moreover, the bilateral soft physical twin results always in a less invasive procedure, with 41% lower mechanical work exchanged with the remote phantom.

Vocal pain expression augmentation for a robopatient

Abdominal palpation is one of the basic but important physical examination methods used by physicians. Visual, auditory, and haptic feedback from the patients are known to be the main sources of feedback they use in the diagnosis. However, learning to interpret this feedback and making accurate diagnosis require several years of training. Many abdominal palpation training simulators have been proposed to date, but very limited attempts have been reported in integrating vocal pain expressions into physical abdominal palpation simulators. Here, we present a vocal pain expression augmentation for a robopatient. The proposed robopatient is capable of providing real-time facial and vocal pain expressions based on the exerted palpation force and position on the abdominal phantom of the robopatient. A pilot study is conducted to test the proposed system, and we show the potential of integrating vocal pain expressions to the robopatient. The platform has also been tested by two clinical experts with prior experience in abdominal palpation. Their evaluations on functionality and suggestions for improvements are presented. We highlight the advantages of the proposed robopatient with real-time vocal and facial pain expressions as a controllable simulator platform for abdominal palpation training studies. Finally, we discuss the limitations of the proposed approach and suggest several future directions for improvements.

ESPRESS.0: Eustachian Tube-Inspired Tactile Sensor Exploiting Pneumatics for Range Extension and SenSitivity Tuning

Optimising the sensitivity of a tactile sensor to a specific range of stimuli magnitude usually compromises the sensor's widespread usage. This paper presents a novel soft tactile sensor capable of dynamically tuning its stiffness for enhanced sensitivity across a range of applied forces, taking inspiration from the Eustachian tube in the mammalian ear. The sensor exploits an adjustable pneumatic back pressure to control the effective stiffness of its 20 mm diameter elastomer interface. An internally translocated fluid is coupled to the membrane and optically tracked to measure physical interactions at the interface. The sensor can be actuated by pneumatic pressure to dynamically adjust its stiffness. It is demonstrated to detect forces as small as 0.012 N, and to be sensitive to a difference of 0.006 N in the force range of 35 to 40 N. The sensor is demonstrated to be capable of detecting tactile cues on the surface of objects in the sub-millimetre scale. It is able to adapt its compliance to increase its ability for distinguishing between stimuli with similar stiffnesses (0.181 N/mm difference) over a large range (0.1 to 1.1 N/mm) from only a 0.6 mm deep palpation. The sensor is intended to interact comfortably with skin, and the feasibility of its use in palpating tissue in search of hard inclusions is demonstrated by locating and estimating the size of a synthetic hard node embedded 20 mm deep in a soft silicone sample. The results suggest that the sensor is a good candidate for tactile tasks involving unpredictable or unknown stimuli.

Human Stabilization of Delay-Induced Instability of Haptic Rendering in a Stiffness Discrimination Task

Towards developing a coupled stability theory for haptic systems, we study the interaction of operators with time-delayed force feedback. In this work, we analyzed and validated experimentally the stability boundaries of an uncoupled system - without considering the human. We then designed an experiment in which the participants used a haptic device to interact with virtual elastic force fields in a stiffness discrimination task. We compared the performance and kinematics of users in uncoupled-unstable and uncoupled-stable conditions and characterized the stabilizing contribution of the users. We found that the users were able to perform the task regardless of the uncoupled-stability conditions. In addition, in uncoupled-unstable conditions, users maintained movement characteristics that were important for exploratory mediation, such as depth and duration of the movement, whereas other characteristics were not preserved. The results were reproduced in a simulation of the human controller that combined an inverse model and an optimal feedback controller. Adequate performance under the uncoupled-unstable yet coupled-stable conditions supports the potential benefit of designing for coupled stability of haptic systems. This could lead to the use of less conservative controllers than state-of-the-art solutions in haptic and teleoperation systems, and advance the fidelity of haptic feedback.

Force-Sensorless Identification and Classification of Tissue Biomechanical Parameters for Robot-Assisted Palpation

The implementation of robotic systems for minimally invasive surgery and medical procedures is an active topic of research in recent years. One of the most common procedures is the palpation of soft tissues to identify their mechanical characteristics. In particular, it is very useful to identify the tissue's stiffness or equivalently its elasticity coefficient. However, this identification relies on the existence of a force sensor or a tactile sensor mounted at the tip of the robot, as well as on measuring the robot velocity. For some applications it would be desirable to identify the biomechanical characteristics of soft tissues without the need for a force/tactile nor velocity sensors. An estimation of such quantities can be obtained by a model-based state observer for which the inputs are only the robot joint positions and its commanded joint torques. The estimated velocities and forces can then be employed for closed-loop force control, force reflection, and mechanical parameters estimation. In this work, a closed-loop force control is proposed based on the estimated contact forces to avoid any tissue damage. Then, the information from the estimated forces and velocities is used in a least squares estimator of the mechanical parameters. Moreover, the estimated biomechanical parameters are employed in a Bayesian classifier to provide further help for the physician to make a diagnosis. We have found that a combination of the parameters of both linear and nonlinear viscoelastic models provide better classification results: 0% misclassifications against 50% when using a linear model, and 3.12% when using only a nonlinear model, for the case in which the samples have very similar mechanical properties.

Simulator-Based Metrics for Quantifying Vascular Palpation Skill for Cannulation

Palpation is essential for accurate diagnosis and treatment in many clinical examinations and procedures. Specifically, vascular palpation is used to diagnose cardiovascular health issues and identify anatomical landmarks in the peripheral vascular system. However, little attention has been given to quantifying what comprises skilled vascular palpation; therefore, this study aims to objectively quantify the differences between high performer (HP), mid performer (MP), and low performer (LP) behavior towards understanding vascular palpation skills. Eleven HPs, twenty-five MPs, and ten LPs completed sixteen trials on our simulator under various conditions. There were four fistulas, two skin thicknesses, and two motor vibration intensities. Finger force and location data were recorded for each trial on the simulator. We examined three types of palpation metrics: time, force, and location. All three types of metrics demonstrated statistically significant differences between HP and LP palpation behavior. Therefore, these metrics could be used for structured and standardized palpation skills training in the future, potentially improving patient outcomes.

A reliability study of a novel visual ischemic palpation scale in an experimental setting

BACKGROUND

Manual palpation is an important part of the clinical examination and generally it has low reliability. The aim of this study was to assess the reliability of a novel method for discriminating 3 different levels of palpation force.

METHODS

This reliability study included 96 healthy physiotherapists and physiotherapy students, who have been taught a new palpation graduated procedure called Visual Ischemic Palpatory Scale (VIPS), aimed to classify the applied pressure based on the finger's ischemia. Force was recorded by a force measurement system putting sensor over a rigid surface. To study the characteristic of VIPS the analysis of variance (ANOVA), Spearman rank correlation coefficient, Intraclass Correlation Coefficient (ICC), Standard Error of Measurements (SEM), and Minimal Detectable Change (MDC) were calculated.

RESULTS

Three distinct degrees were found with distinct forces expression: 1st degree 76.04 g (95% CI 65.86-86.22), 2nd degree 307.87 g (95% CI 263.29-352.44) and 3rd degree 1319.48 g (CI 1204.73-1434.23). Male participants significantly recorded a greater force than females. Good to excellent reliability across degrees were found (0.89 [95% CI: 0.82-0.97]), and final agreement found that more than 65.6% of sample recorded a force in the cut-offs identified. SEM values became bigger as the recorded force increased and MDC were equal to 48.94 g, 188.73 g, and 379.24 g for 1st, 2nd, and 3rd degree, respectively.

CONCLUSIONS

VIPS would appear to have three distinct degrees, sex dependent, with specific force expression for each degree and a good to excellent intra-rater reliability, but a poor agreement between raters.

A Novel Method for Surface Exploration by 6-DOF Encountered-Type Haptic Display Towards Virtual Palpation

Surface exploration in virtual reality has a large potential to enrich the user's experience. It could for example be used to train and simulate medical palpation. During palpation, users tap, indent, and rub the surface of a sample to estimate the underlying properties. However, up to now there is no good approach to render such intricate interaction realistically. This paper introduces 6 degrees of freedom (DoF) encountered-type haptic display technology for simulating surface exploration tasks. Among the different phases of exploration, this article focuses on the 'in-contact sliding' phase. Two novel control approaches to render sliding over a virtual surface are elaborated. A first rendering method generates lateral frictional forces as the finger slides over the surface. A second method adjusts the inclination of the end-effector to render tissue properties. With both methods a stiff nodule embedded in a soft tissue was prepared. User experiments were carried out to find proper parameter and intensity ranges and to confirm the feasibility of the new rendering schemes. Participants indicated that both rendering schemes felt realistic. Compared to earlier work, where only the vertical stiffness was altered, lower thresholds to detect and localise embedded virtual nodules were found. Users also made fewer errors in detecting nodule edges. Furthermore, the method that used end-effector inclination allowed faster discovery of the nodule's edges. It is expected that approaches that combine both rendering methods could provide an even more realistic feel.

A tactile sensor using the acoustic reflection principle for assessing the contact force component in laparoscopic tumor localization

PURPOSE

Localization of an early stage gastric tumor is easily performed in conventional open surgery, whereas it is a difficult procedure in minimally invasive surgery (MIS). A tactile sensor could allow precise resection of the tumor in laparoscopic surgery. The safety of medical tools should be ensured in MIS. Moreover, boundary conditions such as a double-ended beam without a supporting rigid base during tissue palpation were hardly considered. Thus, we suppose that it is informative to assess the normal force and shear force for practical tumor detection considering the boundary condition.

METHODS

In this study, a tactile sensor with normal and shear force measurement functions using the acoustic reflection principle was developed for gastric tumor detection in MIS. The developed tactile sensor was tested using an artificial phantom of the stomach without a supporting rigid base to evaluate the force response of the sensor in intraoperative tumor localization.

RESULTS

The developed sensor is safe for human tissue and can be sterilized. The experimental results show that the developed tactile sensor has the capability to measure normal and shear forces. In the gastric tumor detection test, the shear force of the sensor was more stable and highly responsive to the tumor position than the normal force, which is greatly affected by the bending of the tissue during the operation.

CONCLUSIONS

A two-axis tactile sensor using the acoustic reflection principle was assembled for tissue palpation in MIS. The results showed that the developed sensor is suitable for tumor detection, indicating that the shear force information of the developed sensor is more useful in MIS for early stage gastric tumor localization.

Mechanical characterization of porcine liver properties for computational simulation of indentation on cancerous tissue

An accurate characterization of soft biological tissue properties is essential for a realistic simulation of surgical procedures. Unconfined uniaxial compression tests with specimens affixed to the fixtures are often performed to characterize the stress-stretch curves of soft biological tissues, with which the material parameters can be obtained. However, the constrained boundary condition causes non-uniform deformation during the uniaxial test, posing challenges for accurate measurement of tissue deformation. In this study, we measured the deformation locally at the middle of liver specimens and obtained the corresponding stress-stretch curves. Since the effect of the constrained boundary condition on the local deformation of specimen is minimized, the stress-stretch curves are thus more realistic. Subsequently, we fitted the experimental stress-stretch curves with several constitutive models and found that the first-order Ogden hyperelastic material model was most suitable for characterizing the mechanical properties of porcine liver tissues. To further verify the characterized material properties, we carried out indentation tests on porcine liver specimens and compared the experimental data with computational results by using finite element simulations. A good agreement was achieved. Finally, we constructed computational models of liver tissue with a tumor and investigated the effect of the tumor on the mechanical response of the tissue under indentation. The computational results revealed that the liver specimen with tumor shows a stiffer response if the distance between the tumor and the indenter is small.

Development of a microsurgery-assisted robot for high-precision thread traction and tension control, and confirmation of its applicability

Background

Microsurgery requires high skills for suturing using fragile threads, often within narrow surgical fields. Precise tension is required for good healing and to avoid the risk of thread breakage.

Methods

To meet the demands, we developed a novel assist robot utilizing high‐precision sensorless haptic technology. The robot adopts a cable‐driven mechanism to maintain a distance from the surgical area and enhances compatibility with surgical equipment such as microscopes. The robot performance was verified through in vitro and in vivo experiments using a rat model.

Results

The realization of precise tension control was confirmed in both experiments. In particular, in the in vivo experiments, the developed robot succeeded to produce a knot with an accurate tension of 0.66% error.

Conclusions

The developed robot can realize to control traction force precisely. This technology might open up the window for a full assist robot for microsurgery with haptic feeling.

Conditioned haptic perception for 3D localization of nodules in soft tissue palpation with a variable stiffness probe

This paper provides a solution for fast haptic information gain during soft tissue palpation using a Variable Lever Mechanism (VLM) probe. More specifically, we investigate the impact of stiffness variation of the probe to condition likelihood functions of the kinesthetic force and tactile sensors measurements during a palpation task for two sweeping directions. Using knowledge obtained from past probing trials or Finite Element (FE) simulations, we implemented this likelihood conditioning in an autonomous palpation control strategy. Based on a recursive Bayesian inferencing framework, this new control strategy adapts the sweeping direction and the stiffness of the probe to detect abnormal stiff inclusions in soft tissues. This original control strategy for compliant palpation probes shows a sub-millimeter accuracy for the 3D localization of the nodules in a soft tissue phantom as well as a 100% reliability detecting the existence of nodules in a soft phantom.

Haptic Intracorporeal Palpation Using a Cable-Driven Parallel Robot: A User Study

OBJECTIVE

Intraoperative palpation is a surgical gesture jeopardized by the lack of haptic feedback which affects robotic minimally invasive surgery. Restoring the force reflection in teleoperated systems may improve both surgeons' performance and procedures' outcome.

METHODS

A force-based sensing approach was developed, based on a cable-driven parallel manipulator with anticipated seamless and low-cost integration capabilities in teleoperated robotic surgery. No force sensor on the end-effector is used, but tissue probing forces are estimated from measured cable tensions. A user study involving surgical trainees (n = 22) was conducted to experimentally evaluate the platform in two palpation-based test-cases on silicone phantoms. Two modalities were compared: visual feedback alone and both visual + haptic feedbacks available at the master site.

RESULTS

Surgical trainees' preference for the modality providing both visual and haptic feedback is corroborated by both quantitative and qualitative metrics. Hard nodules detection sensitivity improves (94.35 ± 9.1% vs 76.09 ± 19.15% for visual feedback alone), while also exerting smaller forces (4.13 ± 1.02 N vs 4.82 ± 0.81 N for visual feedback alone) on the phantom tissues. At the same time, the subjective perceived workload decreases.

CONCLUSION

Tissue-probe contact forces are estimated in a low cost and unique way, without the need of force sensors on the end-effector. Haptics demonstrated an improvement in the tumor detection rate, a reduction of the probing forces, and a decrease in the perceived workload for the trainees.

SIGNIFICANCE

Relevant benefits are demonstrated from the usage of combined cable-driven parallel manipulators and haptics during robotic minimally invasive procedures. The translation of robotic intraoperative palpation to clinical practice could improve the detection and dissection of cancer nodules.

Structuring of tactile sensory information for category formation in robotics palpation

This paper proposes a framework to investigate the influence of physical interactions to sensory information, during robotic palpation. We embed a capacitive tactile sensor on a robotic arm to probe a soft phantom and detect and classify hard inclusions within it. A combination of PCA and K-Means clustering is used to: first, reduce the dimensionality of the spatiotemporal data obtained through the probing of each area in the phantom; second categorize the re-encoded data into a given number of categories. Results show that appropriate probing interactions can be useful in compensating for the quality of the data, or lack thereof. Finally, we test the proposed framework on a palpation scenario where a Support Vector Machine classifier is trained to discriminate amongst different types of hard inclusions. We show the proposed framework is capable of predicting the best-performing motion strategy, as well as the relative classification performance of the SVM classifier, solely based on unsupervised cluster analysis methods.

The Stiffness Comparison Test: A pilot study to determine inter-individual differences in palpatory skill related to gender, age, and occupation-related experience

BACKGROUND

Manual palpation is a core skill in physical examination. Assessing elastic properties such as tissue stiffness has the potential for being an important diagnostics tool in the detection of cancer and other diseases.

OBJECTIVE

The study describes the newly developed Stiffness Comparison Test (SCT). The aim of our study was to test the SCT as a tool to detect interindividual differences in palpation skill related to gender, age and occupational experience.

METHODS

We used eight pairs of polyuterhane gel pads with the stiffness difference decreasing from the first to the last pair. Test subjects were asked to palpate each pair and determine stiffness differences.

PARTICIPANTS

We recruited 25 osteopaths, 48 other manual therapists and 50 participants from other non-manual professions.

RESULTS

As hypothesized there was no significant difference in SCT performance between the sexes (t(121) = 0.288, p = .774). To investigate if an age-related decline would have an effect on palpation skill, we carried out a linear regression. As hypothesized, the model did not predict any significant associations (F(1, 121) = 2.733, b = -0.149, p = .101, R² = 0.022). To compare the effect of occupational groups on SCT performance a one-way ANOVA was conducted. There were no statistically significant differences between group means (F(2, 120) = 0.598, p = .552).

CONCLUSIONS

The SCT can be used as simple and affordable tool for assessment, teaching and training in all disciplines of manual medicine. Further refinements of the tool are suggested to advance its discrimination power.

Development of Stiffness Measurement Program Using Color Mapping in Shear Wave Elastography

Shear wave elastography with ultrasound is a noninvasive method used for measuring stiffness in the human body. Shear wave elastography can be used for accurately and quantitatively measuring stiffness. However, its disadvantage is that the stiffness value can vary significantly because the region of interest (ROI) setting depends on the diagnostic operator. In this study, a stiffness measurement program using color mapping in shear wave elastography was developed to address the above-mentioned disadvantage. Color map and color ratios were obtained and evaluated for major lower limb muscles (i.e., biceps femoris, medial gastrocnemius, rectus femoris, and tibialis anterior) at active voluntary contraction. According to the result, when the developed program was used, a small standard deviation compared to the conventional stiffness measurement method, such as kilopascal or meter per second unit using ROIs, was measured in all cases. In conclusion, our results demonstrate that the stiffness measurement method using our program is expected to improve reliability in shear wave elastography ultrasound imaging.

Development and assessment of a stiffness display system for minimally invasive surgery based on smart magneto-rheological elastomers

In the present study, a solution to address the clinical need for stiffness display during manual and robotic minimally invasive surgery was postulated, developed, and assessed. To this end, a magneto-rheological elastomer-based stiffness display, MiTouch, was designed, developed, and analyzed. The mechanical properties of the MRE and system parameters were identified experimentally, based on which the force-field-stiffness response surface of the smart MRE was characterized. Based on the response surface, a stiffness controller was designed and verified for a set of performance requirements. A heartbeat simulation experiment showed the capability of the system for replicating desired tactile forces through stiffness control. Also, the system successfully attained an arbitrarily selected stiffness (4 N/mm) and maintained it within a bounded range (4.07 ± 0.41 N/mm). A comparison of the system performance with current literature validated its applicability for the proposed medical application.

Visuo-Haptic Discrimination of Viscoelastic Materials

In our daily lives, we interact with different types of deformable materials. Regarding their mechanical behavior, some of those materials lie in a range that is between purely elastic and purely viscous. This range of mechanical behavior is described as viscoelasticity. In certain types of haptic interactions, such as assessment of ripeness of fruit, firmness of cheese, and consistency of organ tissue, we rely heavily on our haptic perception of viscoelastic materials. The relationship between the mechanical behavior of viscoelastic materials and our perception of them has been investigated in the field of psychorheology. However, our knowledge on how we perceive viscoelastic materials is still quite limited though some research work has already been done on purely elastic and purely viscous materials. History- and frequency-dependent behavior of viscoelastic materials result in a complex time-dependent response, which requires relatively more sophisticated models to investigate their behavior than those of purely elastic and viscous materials. In this study, we model viscoelasticity using a "springpot" (i.e., fractional-order derivative element) and express its behavior in the frequency domain using two physical parameters-"magnitude" and "phase" of complex stiffness. In the frequency domain, we are able to devise signal detection experiments where we can investigate the perception of viscoelastic materials using the perceptual terms of "firmness" and "bounciness," corresponding to the physical parameters of "magnitude" and "phase." The results of our experiments show that the just-noticeable difference (JND) for bounciness increases linearly with increasing "phase," following Weber's law, while the JND for firmness is surprisingly independent of the level of "phase."

Compensating Uncertainties in Force Sensing for Robotic-Assisted Palpation

Force sensing in robotic-assisted minimally invasive surgery (RMIS) is crucial for performing dedicated surgical procedures, such as bilateral teleoperation and palpation. Due to the bio-compatibility and sterilization requirements, a specially designed surgical tool/shaft is normally attached to the sensor while contacting the organ targets. Through this design, the measured force from the sensor usually contains uncertainties, such as noise, inertial force etc., and thus cannot reflect the actual interaction force with the tissue environment. Motivated to provide the authentic contact force between a robotic tool and soft tissue, we proposed a data-driven force compensation scheme without intricate modeling to reduce the effects of force measurement uncertainties. In this paper, a neural-network-based approach is utilized to automatically model the inertial force subject to noise during the robotic palpation procedure, then the exact contact force can be obtained through the force compensation method which cancels the noise and inertial force. Following this approach, the genuine interaction force during the palpation task can be achieved furthermore to improve the appraisal of the tumor surrounded by the soft tissue. Experiments are conducted with robotic-assisted palpation tasks on a silicone-based soft tissue phantom and the results verify the effectiveness of the suggested method.

Haptic feedback in the da Vinci Research Kit (dVRK): A user study based on grasping, palpation, and incision tasks

BACKGROUND

It was suggested that the lack of haptic feedback, formerly considered a limitation for the da Vinci robotic system, does not affect robotic surgeons because of training and compensation based on visual feedback. However, conclusive studies are still missing, and the interest in force reflection is rising again.

METHODS

We integrated a seven-DoF master into the da Vinci Research Kit. We designed tissue grasping, palpation, and incision tasks with robotic surgeons, to be performed by three groups of users (expert surgeons, medical residents, and nonsurgeons, five users/group), either with or without haptic feedback. Task-specific quantitative metrics and a questionnaire were used for assessment.

RESULTS

Force reflection made a statistically significant difference for both palpation (improved inclusion detection rate) and incision (decreased tissue damage).

CONCLUSIONS

Haptic feedback can improve key surgical outcomes for tasks requiring a pronounced cognitive burden for the surgeon, to be possibly negotiated with longer completion times.

Modelling the structure of object-independent human affordances of approaching to grasp for robotic hands

Grasp affordances in robotics represent different ways to grasp an object involving a variety of factors from vision to hand control. A model of grasp affordances that is able to scale across different objects, features and domains is needed to provide robots with advanced manipulation skills. The existing frameworks, however, can be difficult to extend towards a more general and domain independent approach. This work is the first step towards a modular implementation of grasp affordances that can be separated into two stages: approach to grasp and grasp execution. In this study, human experiments of approaching to grasp are analysed, and object-independent patterns of motion are defined and modelled analytically from the data. Human subjects performed a specific action (hammering) using objects of different geometry, size and weight. Motion capture data relating the hand-object approach distance was used for the analysis. The results showed that approach to grasp can be structured in four distinct phases that are best represented by non-linear models, independent from the objects being handled. This suggests that approaching to grasp patterns are following an intentionally planned control strategy, rather than implementing a reactive execution.

DNN-Based Assistant in Laparoscopic Computer-Aided Palpation

Tactile sensory input of surgeons is severely limited in minimally invasive surgery, therefore manual palpation cannot be performed for intraoperative tumor detection. Computer-aided palpation, in which tactile information is acquired by devices and relayed to the surgeon, is one solution for overcoming this limitation. An important design factor is the method by which the acquired information is fed back to the surgeon. However, currently there is no systematic method for achieving this aim, and it is possible that a badly implemented feedback mechanism could adversely affect the performance of the surgeon. In this study, we propose an assistance algorithm for intraoperative tumor detection in laparoscopic surgery. Our scenario is that the surgeon manipulates a sensor probe, makes a decision based on the feedback provided from the sensor, while simultaneously, the algorithm analyzes the time series of the sensor output. Thus, the algorithm assists the surgeon in making decisions by providing independent detection results. A deep neural network model with three hidden layers was used to analyze the sensor output. We propose methods to input the time series of the sensor output to the model for real-time analysis, and to determine the criterion for detection by the model. This study aims to validate the feasibility of the algorithm by using data acquired in our previous psychophysical experiment. There, novice participants were asked to detect a phantom of an early-stage gastric tumor through visual feedback from the tactile sensor. In addition to the analysis of the accuracy, signal detection theory was employed to assess the potential detection performance of the model. The detection performance was compared with that of human participants. We conducted two types of validation, and found that the detection performance of the model was not significantly different from that of the human participants if the data from a known user was included in the model construction. The result supports the feasibility of the proposed algorithm for detection assistance in computer-aided palpation.

Design of a Wearable Fingertip Haptic Device for Remote Palpation: Characterisation and Interface with a Virtual Environment

This paper presents the development of a wearable Fingertip Haptic Device (FHD) that can provide cutaneous feedback via a Variable Compliance Platform (VCP). The FHD includes an inertial measurement unit, which tracks the motion of the user’s finger while its haptic functionality relies on two parameters: pressure in the VCP and its linear displacement towards the fingertip. The combination of these two features results in various conditions of the FHD, which emulate the remote object or surface stiffness properties. Such a device can be used in tele-operation, including virtual reality applications, where rendering the level of stiffness of different physical or virtual materials could provide a more realistic haptic perception to the user. The FHD stiffness representation is characterised in terms of resulting pressure and force applied to the fingertip created through the relationship of the two functional parameters – pressure and displacement of the VCP. The FHD was tested in a series of user studies to assess its potential to create a user perception of the object’s variable stiffness. The viability of the FHD as a haptic device has been further confirmed by interfacing the users with a virtual environment. The developed virtual environment task required the users to follow a virtual path, identify objects of different hardness on the path and navigate away from “no-go” zones. The task was performed with and without the use of the variable compliance on the FHD. The results showed improved performance with the presence of the variable compliance provided by the FHD in all assessed categories and particularly in the ability to identify correctly between objects of different hardness.