Using deep learning for pruning region detection and plant organ segmentation in dormant spur-pruned grapevines

Even though mechanization has dramatically decreased labor requirements, vineyard management costs are still affected by selective operations such as winter pruning. Robotic solutions are becoming more common in agriculture, however, few studies have focused on grapevines. This work aims at fine-tuning and testing two different deep neural networks for: (i) detecting pruning regions (PRs), and (ii) performing organ segmentation of spur-pruned dormant grapevines. The Faster R-CNN network was fine-tuned using 1215 RGB images collected in different vineyards and annotated through bounding boxes. The network was tested on 232 RGB images, PRs were categorized by wood type (W), orientation (Or) and visibility (V), and performance metrics were calculated. PR detection was dramatically affected by visibility. Highest detection was associated with visible intermediate complex spurs in Merlot (0.97), while most represented coplanar simple spurs allowed a 74% detection rate. The Mask R-CNN network was trained for grapevine organs (GOs) segmentation by using 119 RGB images annotated by distinguishing 5 classes (cordon, arm, spur, cane and node). The network was tested on 60 RGB images of light pruned (LP), shoot-thinned (ST) and unthinned control (C) grapevines. Nodes were the best segmented GOs (0.88) and general recall was higher for ST (0.85) compared to C (0.80) confirming the role of canopy management in improving performances of hi-tech solutions based on artificial intelligence. The two fine-tuned and tested networks are part of a larger control framework that is under development for autonomous winter pruning of grapevines.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11119-023-10006-y.

Study of Patients Self-Training Influence on Peripheral Neuropathies Diseases Diagnosis through D.I.T.A Device

In our daily life, the sight and the sense of touch play a fundamental role in objects recognitions. This process is helped by the experience: if a subject has already seen or already touched an object in the past, he will recognize it more easily in the future. Following this assumption, the authors of this paper wanted to investigate if the experience can influence the results of a clinical examination where the subject has an active role. The attention was focused on the peripheral neuropathies diagnosis since they require an accurate assessment of several parameters including the tactile sensitivity trend. In other words, if the tests encompass an active role of the subjects, one of the main uncertainties is the self-training that influences the subject responses. This work focuses on the study of this self-training using the D.I.T.A device (Dynamic Investigation Test-rig on hAptics). Results clearly show a fundamental role of priming during "haptic modality": expert subjects, previously experienced with the tests, demonstrated better recognition of the encountered stimuli, compared to novices. Moreover, the results show that the maximum difference between the two groups of subjects is in the first part of the test. An ANOVA analysis was carried out to demonstrate that also the errors between the pins-arrays are affected by the priming.

Development and validation of a tactile sensitivity scale for peripheral neuropathy screening

One of the difficult of the hands peripheral neuropathy screening uncertainty is that the current diagnosis is based not on assessments obtained by accurate and repeatable devices, but mostly on the clinical examination. So, in this paper the authors present a tactile pins-array scale determined with well-defined parameters assessed by non-invasive DITA device (Dynamic Investigation Test-rig on hAptics). This high resolution scale permits to screen the gradual tactile sensory deficit of patients affected by neuropathic diseases. The work has started with an experiment on healthy subjects penalizing their bare finger tactile sensitivity with five different pins-arrays. So, a pins-array scale divided in six levels (grouped in three ranges: low, uncertain and normal tactile sensitivity) was created. The scale was validated with a pilot study on six subjects affected by neuropathic disease. Results show an important role of the scale, supporting the clinical screening and reducing the uncertainty.

A High Performance Tactile Feedback Display and Its Integration in Teleoperation

This paper presents the development of a compact tactile display and its integration in teleoperation. The system's operation is based on the display of surface shape to an area of the fingertip through a 4 × 4 array of tactors moving perpendicularly to the skin surface. The tactors are spring loaded and are actuated remotely by dc motors through a flexible tendon transmission. This novel implementation of conventional actuation principles achieves a compact design with superior performance compared to devices of a similar footprint, demonstrating an excellent combination of tactor spatiotemporal resolution, force, and amplitude. The display's ergonomic design and high performance make it suitable for integration on haptic devices for tactile feedback in VR and in Teleoperation. This paper presents the design, control, and performance of the tactile display and of the transmission system. It also demonstrates its integration on an Omega7 force feedback device for the teleoperation of an LWR KUKA manipulator. An experiment is presented where users teleoperated the stylus of the robot in a 3D contour following task with and without tactile feedback. In this experiment, force feedback from the slave is fused with model-based local tactile feedback. Subjects' performances indicate an improvement in teleoperation when both tactile and force feedback are present.

Integrating Haptics with Augmented Reality in a Femoral Palpation and Needle Insertion Training Simulation

This paper presents a virtual environment for training femoral palpation and needle insertion, the opening steps of many interventional radiology procedures. A novel augmented reality simulation called PalpSim has been developed that allows the trainees to feel a virtual patient using their own hands. The palpation step requires both force and tactile feedback. For the palpation haptics effect, two off-the-shelf force feedback devices have been linked together to provide a hybrid device that gives five degrees of force feedback. This is combined with a custom built hydraulic interface to provide a pulse like tactile effect. The needle interface is based on a modified PHANTOM Omni end effector that allows a real interventional radiology needle to be mounted and used during simulation. While using the virtual environment, the haptics hardware is masked from view using chroma-key techniques. The trainee sees a computer generated patient and needle, and interacts using their own hands. This simulation provides a high level of face validity and is one of the first medical simulation devices to integrate haptics with augmented reality.

Proportional mechanical ventilation through PWM driven on/off solenoid valve

Proportional strategies for artificial ventilation are the most recent form of synchronized partial ventilatory assistance and intra-breath control techniques available in clinical practice. Currently, the majority of commercial ventilators allowing proportional ventilation uses proportional valves to generate the flow rate pattern. This paper proposes on-off solenoid valves for proportional ventilation given their small size, low cost and short switching time, useful for supplying high frequency ventilation. A new system based on a novel fast switching driver circuit combined with on/off solenoid valve is developed. The average short response time typical of onoff solenoid valves was further reduced through the driving circuit for the implementation of PWM control. Experimental trials were conducted for identifying the dynamic response of the PWM driven on/off valve and for verifying its effectiveness in generating variable-shaped ventilatory flow rate patterns. The system was able to smoothly follow the reference flow rate patterns also changing in time intervals as short as 20 ms, achieving a flow rate resolution up to 1 L/min and repeatability in the order of 0.5 L/min. Preliminary results showed the feasibility of developing a stand alone portable device able to generate both proportional and high frequency ventilation by only using on-off solenoid valves.

Assessment of the assistive performance of an ankle exerciser using electromyographic signals

This paper presents the design of an admittance-based assistive controller and preliminary experimental results for a high performance parallel robot used for ankle rehabilitation. The goal of this work was to design a suitable control algorithm for diagnosis, training and rehabilitation of the ankle in presence of musculoskeletal injuries. An admittance control technique is used to perform patient-active exercises with and without motion assistance. Electromyographic (EMG) signals are used to evaluate patient's effort during training/exercising. The results indicate the great potential of the rehabilitation device as a tool to fasten and improve the ankle therapies outcome.

Biomimetic actuators in prosthetic and rehabilitation applications

Where humans and mechanical systems operate in close proximity there is a need to provide drive systems that combine the positive attributes of conventional actuator design with a 'softer' safer interaction capacity. This is achieved by natural muscle, and engineering emulation of this functionality could have a significant benefits in many areas, but particularly the medical domain. This work will study the use of compliance regulated and controlled pairs of antagonistic pneumatic Muscle Actuators (pMAs) in two medical scenarios; i) The construction of dexterous prosthetic hands having a high power and low mass potential, ii) The construction of a power assist device that can be used to augment the strength of those suffering from degenerative muscle wasting diseases.