Tracking and evaluating motion skills in laparoscopy with inertial sensors

BACKGROUND

Analysis of surgical instrument motion is applicable in surgical skill assessment and monitoring of the learning progress in laparoscopy. Current commercial instrument tracking technology (optical or electromagnetic) has specific limitations and is expensive. Therefore, in this study, we apply inexpensive, off-the-shelf inertial sensors to track laparoscopic instruments in a training scenario.

METHODS

We calibrated two laparoscopic instruments to the inertial sensor and investigated its accuracy on a 3D-printed phantom. In a user study during a one-week laparoscopy training course with medical students and physicians, we then documented and compared the training effect in laparoscopic tasks on a commercially available laparoscopy trainer (Laparo Analytic, Laparo Medical Simulators, Wilcza, Poland) and the newly developed tracking setup.

RESULTS

Eighteen participants (twelve medical students and six physicians) participated in the study. The student subgroup showed significantly poorer results for the count of swings (CS) and count of rotations (CR) at the beginning of the training compared to the physician subgroup (p = 0.012 and p = 0.042). After training, the student subgroup showed significant improvements in the rotatory angle sum, CS, and CR (p = 0.025, p = 0.004 and p = 0.024). After training, there were no significant differences between medical students and physicians. There was a strong correlation between the measured learning success (LS) from the data of our inertial measurement unit system (LS_IMU) and the Laparo Analytic (LS_Lap) (Pearson's r = 0.79).

CONCLUSION

In the current study, we observed a good and valid performance of inertial measurement units as a possible tool for instrument tracking and surgical skill assessment. Moreover, we conclude that the sensor can meaningfully examine the learning progress of medical students in an ex-vivo setting.

Recent Advances in Tracking Devices for Biomedical Ultrasound Imaging Applications

With the rapid advancement of tracking technologies, the applications of tracking systems in ultrasound imaging have expanded across a wide range of fields. In this review article, we discuss the basic tracking principles, system components, performance analyses, as well as the main sources of error for popular tracking technologies that are utilized in ultrasound imaging. In light of the growing demand for object tracking, this article explores both the potential and challenges associated with different tracking technologies applied to various ultrasound imaging applications, including freehand 3D ultrasound imaging, ultrasound image fusion, ultrasound-guided intervention and treatment. Recent development in tracking technology has led to increased accuracy and intuitiveness of ultrasound imaging and navigation with less reliance on operator skills, thereby benefiting the medical diagnosis and treatment. Although commercially available tracking systems are capable of achieving sub-millimeter resolution for positional tracking and sub-degree resolution for orientational tracking, such systems are subject to a number of disadvantages, including high costs and time-consuming calibration procedures. While some emerging tracking technologies are still in the research stage, their potentials have been demonstrated in terms of the compactness, light weight, and easy integration with existing standard or portable ultrasound machines.

SCADE: Simultaneous Sensor Calibration and Deformation Estimation of FBG-Equipped Unmodeled Continuum Manipulators

In this article, we present a novel stochastic algorithm called simultaneous sensor calibration and deformation estimation (SCADE) to address the problem of modeling deformation behavior of a generic continuum manipulator (CM) in free and obstructed environments. In SCADE, using a novel mathematical formulation, we introduce a priori model-independent filtering algorithm to fuse the continuous and inaccurate measurements of an embedded sensor (e.g., magnetic or piezoelectric sensors) with an intermittent but accurate data of an external imaging system (e.g., optical trackers or cameras). The main motivation of this article is the crucial need of obtaining an accurate shape/position estimation of a CM utilized in a surgical intervention. In these robotic procedures, the CM is typically equipped with an embedded sensing unit (ESU) while an external imaging modality (e.g., ultrasound or a fluoroscopy machine) is also available in the surgical site. The results of two different set of prior experiments in free and obstructed environments were used to evaluate the efficacy of SCADE algorithm. The experiments were performed with a CM specifically designed for orthopaedic interventions equipped with an inaccurate Fiber Bragg Grating (FBG) ESU and overhead camera. The results demonstrated the successful performance of the SCADE algorithm in simultaneous estimation of unknown deformation behavior of the utilized unmodeled CM together with realizing the time-varying drift of the poor-calibrated FBG sensing unit. Moreover, the results showed the phenomenal out-performance of the SCADE algorithm in estimation of the CM's tip position as compared to FBG-based position estimations.

Knee arthroscopic navigation using virtual-vision rendering and self-positioning technology

PURPOSE

Knee arthroscopy suffers from a lack of depth information and easy occlusion of the visual field. To solve these limitations, we propose an arthroscopic navigation system based on self-positioning technology, with the guidance of virtual-vision views. This system can work without any external tracking devices or added markers, thus increasing the working range and improving the robustness of the rotating operation.

METHODS

The fly-through view and global positioning view for surgical guidance are rendered through virtual-vision rendering in real time. The fly-through view provides surgeons with navigating the arthroscope in the internal anatomical structures using a virtual camera perspective. The global positioning view shows the posture of the arthroscope relative to the preoperative model in a transparent manner. The posture of the arthroscope is estimated from the fusion of visual and inertial data based on the visual-inertial stereo slam. A flexible calibration method that transforms the posture of the arthroscope in the physical world into the virtual-vision rendering framework is proposed for the arthroscopic navigation system with self-positioning information.

RESULTS

Quantitative experiments for evaluating self-positioning accuracy were performed. For translation, the acquired mean error was 0.41 ± 0.28 mm; for rotation, it was 0.11° ± 0.07°. The tracking range of the proposed system was approximately 1.4 times that of the traditional external optical tracking system for the rotating operation. Simulated surgical operations were performed on the phantom. The fly-through and global positing views were paired with original arthroscopic images for intuitive surgical guidance.

CONCLUSION

The proposed system provides surgeons with both fly-through and global positioning views without a dependence on the traditional external tracking systems for surgical guidance. The feasibility and robustness of the system are evaluated, and it shows promise for medical applications.

Real-time endoscopic image orientation correction system using an accelerometer and gyrosensor

The discrepancy between spatial orientations of an endoscopic image and a physician's working environment can make it difficult to interpret endoscopic images. In this study, we developed and evaluated a device that corrects the endoscopic image orientation using an accelerometer and gyrosensor. The acceleration of gravity and angular velocity were retrieved from the accelerometer and gyrosensor attached to the handle of the endoscope. The rotational angle of the endoscope handle was calculated using a Kalman filter with transmission delay compensation. Technical evaluation of the orientation correction system was performed using a camera by comparing the optical rotational angle from the captured image with the rotational angle calculated from the sensor outputs. For the clinical utility test, fifteen anesthesiology residents performed a video endoscopic examination of an airway model with and without using the orientation correction system. The participants reported numbers written on papers placed at the left main, right main, and right upper bronchi of the airway model. The correctness and the total time it took participants to report the numbers were recorded. During the technical evaluation, errors in the calculated rotational angle were less than 5 degrees. In the clinical utility test, there was a significant time reduction when using the orientation correction system compared with not using the system (median, 52 vs. 76 seconds; P = .012). In this study, we developed a real-time endoscopic image orientation correction system, which significantly improved physician performance during a video endoscopic exam.

Computed tomography (CT)-compatible remote center of motion needle steering robot: Fusing CT images and electromagnetic sensor data

Lung cancer is the most common cause of cancer-related death, and early detection can reduce the mortality rate. Patients with lung nodules greater than 10 mm usually undergo a computed tomography (CT)-guided biopsy. However, aligning the needle with the target is difficult and the needle tends to deflect from a straight path. In this work, we present a CT-compatible robotic system, which can both position the needle at the puncture point and also insert and rotate the needle. The robot has a remote-center-of-motion arm which is achieved through a parallel mechanism. A new needle steering scheme is also developed where CT images are fused with electromagnetic (EM) sensor data using an unscented Kalman filter. The data fusion allows us to steer the needle using the real-time EM tracker data. The robot design and the steering scheme are validated using three experimental cases. Experimental Case I and II evaluate the accuracy and CT-compatibility of the robot arm, respectively. In experimental Case III, the needle is steered towards 5 real targets embedded in an anthropomorphic gelatin phantom of the thorax. The mean targeting error for the 5 experiments is 1.78 ± 0.70 mm. The proposed robotic system is shown to be CT-compatible with low targeting error. Small nodule size and large needle diameter are two risk factors that can lead to complications in lung biopsy. Our results suggest that nodules larger than 5 mm in diameter can be targeted using our method which may result in lower complication rate.

Simultaneous Electromagnetic Tracking and Calibration for Dynamic Field Distortion Compensation

Electromagnetic (EM) tracking systems are highly susceptible to field distortion. The interference can cause measurement errors up to a few centimeters in clinical environments, which limits the reliability of these systems. Unless corrected for, this measurement error imperils the success of clinical procedures. It is therefore fundamental to dynamically calibrate EM tracking systems and compensate for measurement error caused by field distorting objects commonly present in clinical environments. We propose to combine a motion model with observations of redundant EM sensors and compensate for field distortions in real time. We employ a simultaneous localization and mapping technique to accurately estimate the pose of the tracked instrument while creating the field distortion map. We conducted experiments with six degrees-of-freedom motions in the presence of field distorting objects in research and clinical environments. We applied our approach to improve the EM tracking accuracy and compared our results to a conventional sensor fusion technique. Using our approach, the maximum tracking error was reduced by 67% for position measurements and by 64% for orientation measurements. Currently, clinical applications of EM trackers are hampered by the adverse distortion effects. Our approach introduces a novel method for dynamic field distortion compensation, independent from preoperative calibrations or external tracking devices, and enables reliable EM navigation for potential applications.

Sensor fusion methods for reducing false alarms in heart rate monitoring

Automatic patient monitoring is an essential resource in hospitals for good health care management. While alarms caused by abnormal physiological conditions are important for the delivery of fast treatment, they can be also a source of unnecessary noise because of false alarms caused by electromagnetic interference or motion artifacts. One significant source of false alarms is related to heart rate, which is triggered when the heart rhythm of the patient is too fast or too slow. In this work, the fusion of different physiological sensors is explored in order to create a robust heart rate estimation. A set of algorithms using heart rate variability index, Bayesian inference, neural networks, fuzzy logic and majority voting is proposed to fuse the information from the electrocardiogram, arterial blood pressure and photoplethysmogram. Three kinds of information are extracted from each source, namely, heart rate variability, the heart rate difference between sensors and the spectral analysis of low and high noise of each sensor. This information is used as input to the algorithms. Twenty recordings selected from the MIMIC database were used to validate the system. The results showed that neural networks fusion had the best false alarm reduction of 92.5 %, while the Bayesian technique had a reduction of 84.3 %, fuzzy logic 80.6 %, majority voter 72.5 % and the heart rate variability index 67.5 %. Therefore, the proposed algorithms showed good performance and could be useful in bedside monitors.

A cost-effective surgical navigation solution for periacetabular osteotomy (PAO) surgery

PURPOSE

To evaluate a low-cost, inertial sensor-based surgical navigation solution for periacetabular osteotomy (PAO) surgery without the line-of-sight impediment.

METHODS

Two commercial inertial measurement units (IMU, Xsens Technologies, The Netherlands), are attached to a patient's pelvis and to the acetabular fragment, respectively. Registration of the patient with a pre-operatively acquired computer model is done by recording the orientation of the patient's anterior pelvic plane (APP) using one IMU. A custom-designed device is used to record the orientation of the APP in the reference coordinate system of the IMU. After registration, the two sensors are mounted to the patient's pelvis and acetabular fragment, respectively. Once the initial position is recorded, the orientation is measured and displayed on a computer screen. A patient-specific computer model generated from a pre-operatively acquired computed tomography scan is used to visualize the updated orientation of the acetabular fragment.

RESULTS

Experiments with plastic bones (eight hip joints) performed in an operating room comparing a previously developed optical navigation system with our inertial-based navigation system showed no statistically significant difference on the measurement of acetabular component reorientation. In all eight hip joints the mean absolute difference was below four degrees.

CONCLUSION

Using two commercially available inertial measurement units we show that it is possible to accurately measure the orientation (inclination and anteversion) of the acetabular fragment during PAO surgery and therefore to successfully eliminate the line-of-sight impediment that optical navigation systems have.

Simultaneous localization and calibration for electromagnetic tracking systems

BACKGROUND

In clinical environments, field distortion can cause significant electromagnetic tracking errors. Therefore, dynamic calibration of electromagnetic tracking systems is essential to compensate for measurement errors.

METHODS

It is proposed to integrate the motion model of the tracked instrument with redundant EM sensor observations and to apply a simultaneous localization and mapping algorithm in order to accurately estimate the pose of the instrument and create a map of the field distortion in real-time. Experiments were conducted in the presence of ferromagnetic and electrically-conductive field distorting objects and results compared with those of a conventional sensor fusion approach.

RESULTS

The proposed method reduced the tracking error from 3.94±1.61 mm to 1.82±0.62 mm in the presence of steel, and from 0.31±0.22 mm to 0.11±0.14 mm in the presence of aluminum.

CONCLUSIONS

With reduced tracking error and independence from external tracking devices or pre-operative calibrations, the approach is promising for reliable EM navigation in various clinical procedures. Copyright © 2015 John Wiley & Sons, Ltd.

Vision-based endoscope tracking for 3D ultrasound image-guided surgical navigation

This work introduces a self-contained framework for endoscopic camera tracking by combining 3D ultrasonography with endoscopy. The approach can be readily incorporated into surgical workflows without installing external tracking devices. By fusing the ultrasound-constructed scene geometry with endoscopic vision, this integrated approach addresses issues related to initialization, scale ambiguity, and interest point inadequacy that may be faced by conventional vision-based approaches when applied to fetoscopic procedures. Vision-based pose estimations were demonstrated by phantom and ex vivo monkey placenta imaging. The potential contribution of this method may extend beyond fetoscopic procedures to include general augmented reality applications in minimally invasive procedures.

An adaptive 6-DOF tracking method by hybrid sensing for ultrasonic endoscopes

In this paper, a novel hybrid sensing method for tracking an ultrasonic endoscope within the gastrointestinal (GI) track is presented, and the prototype of the tracking system is also developed. We implement 6-DOF localization by sensing integration and information fusion. On the hardware level, a tri-axis gyroscope and accelerometer, and a magnetic angular rate and gravity (MARG) sensor array are attached at the end of endoscopes, and three symmetric cylindrical coils are placed around patients' abdomens. On the algorithm level, an adaptive fast quaternion convergence (AFQC) algorithm is introduced to determine the orientation by fusing inertial/magnetic measurements, in which the effects of magnetic disturbance and acceleration are estimated to gain an adaptive convergence output. A simplified electro-magnetic tracking (SEMT) algorithm for dimensional position is also implemented, which can easily integrate the AFQC's results and magnetic measurements. Subsequently, the average position error is under 0.3 cm by reasonable setting, and the average orientation error is 1° without noise. If magnetic disturbance or acceleration exists, the average orientation error can be controlled to less than 3.5°.

Needle deflection estimation: prostate brachytherapy phantom experiments

PURPOSE

The performance of a fusion-based needle deflection estimation method was experimentally evaluated using prostate brachytherapy phantoms. The accuracy of the needle deflection estimation was determined. The robustness of the approach with variations in needle insertion speed and soft tissue biomechanical properties was investigated.

METHODS

A needle deflection estimation method was developed to determine the amount of needle bending during insertion into deformable tissue by combining a kinematic deflection model with measurements taken from two electromagnetic trackers placed at the tip and the base of the needle. Experimental verification of this method for use in prostate brachytherapy needle insertion procedures was performed. A total of 21 beveled tip, 18 ga, 200 mm needles were manually inserted at various speeds through a template and toward different targets distributed within 3 soft tissue mimicking polyvinyl chloride prostate phantoms of varying stiffness. The tracked positions of both the needle tip and base were recorded, and Kalman filters were applied to fuse the sensory information. The estimation results were validated using ground truth obtained from fluoroscopy images.

RESULTS

The manual insertion speed ranged from 8 to 34 mm/s, needle deflection ranged from 5 to 8 mm at an insertion depth of 76 mm, and the elastic modulus of the soft tissue ranged from 50 to 150 kPa. The accuracy and robustness of the estimation method were verified within these ranges. When compared to purely model-based estimation, we observed a reduction in needle tip position estimation error by [Formula: see text] % (mean [Formula: see text] SD) and the cumulative deflection error by [Formula: see text] %.

CONCLUSIONS

Fusion of electromagnetic sensors demonstrated significant improvement in estimating needle deflection compared to model-based methods. The method has potential clinical applicability in the guidance of needle placement medical interventions, particularly prostate brachytherapy.

Quaternionic attitude estimation for robotic and human motion tracking using sequential Monte Carlo methods with von Mises-Fisher and nonuniform densities simulations

In recent years, wireless positioning and tracking devices based on semiconductor micro electro-mechanical system (MEMS) sensors have successfully integrated into the consumer electronics market. Information from the sensors is processed by an attitude estimation program. Many of these algorithms were developed primarily for aeronautical applications. The parameters affecting the accuracy and stability of the system vary with the intended application. The performance of these algorithms occasionally destabilize during human motion tracking activities, which does not satisfy the reliability and high accuracy demand in biomedical application. A previous study accessed the feasibility of using semiconductor based inertial measurement units (IMUs) for human motion tracking. IMU hardware has been redesigned and an attitude estimation algorithm using sequential Monte Carlo (SMC) methods, or particle filter, for quaternions was developed. The method presented in this paper uses von Mises-Fisher and a nonuniform simulation to provide density estimation of the rotation group SO(3). Synthetic signal simulation, robotics applications, and human applications have been investigated.

ROBOTICS IN NATURAL ORIFICE TRANSLUMINAL ENDOSCOPIC SURGERY

Natural orifice translumenal endoscopic surgery (NOTES) is the latest surgery paradigm in which the abdominal cavity is accessed via the body's natural orifice, e.g., vagina, mouth, etc. Compared with traditional laparoscopic surgery, NOTES completely eliminates the skin incision and therefore benefits the patients in several aspects such as less post-operative pain, shorter recovery period, fewer complications, etc. Due to the unique characteristics of NOTES, instruments for traditional laparoscopic surgery are not suitable for NOTES and hence novel hardware design is necessary for facilitating system development. This paper gives an overview of the state of the arts in the development of surgical instruments for NOTES, particularly with a focus on the promising robotic endoscopes.

Using analytical redundancy to increase safety of a synergistic manually guided instrument for craniotomy

In this paper, two methods for bridging line of sight interruptions occurring during the use of the synergistically operated semiautomatic trepanation system (STS) are presented. In the STS, position information is acquired using an optical tracking system with the disadvantage of possible line-of-sight interruptions. Their compensation is crucial, as a real-time control system automatically adjusts the cutting depth of the instrument on the basis of position information and a-priori data. The surgeon is only responsible for guiding the instrument along the resection line. Hence, availability of position information is crucial for depth control, set point generation, and thus for patient safety. In favour of enhancing reliability of position and orientation acquisition, two approaches were developed which are intended to estimate the position during line of sight interruptions on the basis of a-priori system information and process parameters. To assure patient's safety during this procedure, several parameters of the system (e.g. cutting radius, skull gradient) are used in order to estimate the possible cutting error while the redundant system is activated. These two algorithms and the online risk assessment were implemented, and afterwards evaluated. The evaluation was performed using a skull phantom, and yielded promising results.