A Robust Tri-Electromagnet-Based 6-DoF Pose Tracking System Using an Error-State Kalman Filter

Magnetic pose tracking is a non-contact, accurate, and occlusion-free method that has been increasingly employed to track intra-corporeal medical devices such as endoscopes in computer-assisted medical interventions. In magnetic pose-tracking systems, a nonlinear estimation algorithm is needed to recover the pose information from magnetic measurements. In existing pose estimation algorithms such as the extended Kalman filter (EKF), the 3-DoF orientation in the S3 manifold is normally parametrized as unit quaternions and simply treated as a vector in the Euclidean space, which causes a violation of the unity constraint of quaternions and reduces pose tracking accuracy. In this paper, a pose estimation algorithm based on the error-state Kalman filter (ESKF) is proposed to improve the accuracy and robustness of electromagnetic tracking systems. The proposed system consists of three electromagnetic coils for magnetic field generation and a tri-axial magnetic sensor attached to the target object for field measurement. A strategy of sequential coil excitation is developed to separate the magnetic fields from different coils and reject magnetic disturbances. Simulation and experiments are conducted to evaluate the pose tracking performance of the proposed ESKF algorithm, which is also compared with standard EKF and constrained EKF. It is shown that the ESKF can effectively maintain the quaternion unity and thus achieve a better tracking accuracy, i.e., a Euclidean position error of 2.23 mm and an average orientation angle error of 0.45°. The disturbance rejection performance of the electromagnetic tracking system is also experimentally validated.

Posture Dynamic Modeling and Stability Analysis of a Magnetic Driven Dual-Spin Spherical Capsule Robot

In order to realize the intervention operation in the unstructured and ample environments such as stomach and colon, a dual-spin spherical capsule robot (DSCR) driven by pure magnetic torque generated by the universal rotating magnetic field (URMF) is proposed. The coupled magnetic torque, the viscoelastic friction torque, and the gravity torque were analyzed. Furthermore, the posture dynamic model describing the electric-magnetic-mechanical-liquid coupling dynamic behavior of the DSCR in the gastrointestinal (GI) tract was established. This model is a second-order periodic variable coefficient dynamics equation, which should be regarded as an extension of the Lagrange case for the dual-spin body system under the fixed-point motion, since the external torques were applied. Based on the Floquet-Lyapunov theory, the stability domain of the DSCR for the asymptotically stable motion and periodic motion were obtained by investigating the influence of the angular velocity of the URMF, the magnetic induction intensity, and the centroid deviation. Research results show that the DSCR can realize three kinds of motion, which are asymptotically stable motion, periodic motion, and chaotic motion, according to the distribution of the system characteristic multipliers. Moreover, the posture stability of the DSCR can be improved by increasing the angular velocity of the URMF and reducing the magnetic induction intensity.

Magnetic Position System Design Method Applied to Three-Axis Joystick Motion Tracking

This manuscript discusses the difficulties with magnetic position and orientation (MPO) system design and proposes a general method for finding optimal layouts. The formalism introduces a system quality measure through state separation and reduces the question “How to design an MPO system?” to a global optimization problem. The latter is then solved by combining differential evolution algorithms with magnet shape variation based on analytical computations of the field. The proposed formalism is then applied to study possible realizations of continuous three-axis joystick motion tracking, realized with just a single magnet and a single 3D magnetic field sensor. The computations show that this is possible when a specific design condition is fulfilled and that large state separations as high as 1mT/∘ can be achieved under realistic conditions. Finally, a comparison to state-of-the-art design methods is drawn, computation accuracy is reviewed critically, and an experimental validation is presented.

Robotic Localization Based on Planar Cable Robot and Hall Sensor Array Applied to Magnetic Capsule Endoscope

Recently an active locomotive capsule endoscope (CE) for diagnosis and treatment in the digestive system has been widely studied. However, real-time localization to achieve precise feedback control and record suspicious positioning in the intestine is still challenging owing to the limitation of capsule size, relatively large diagnostic volume, and compatibility of other devices in clinical site. To address this issue, we present a novel robotic localization sensing methodology based on the kinematics of a planar cable driven parallel robot (CDPR) and measurements of the quasistatic magnetic field of a Hall effect sensor (HES) array. The arrangement of HES and the Levenberg-Marquardt (LM) algorithm are applied to estimate the position of the permanent magnet (PM) in the CE, and the planar CDPR is incorporated to follow the PM in the CE. By tracking control of the planar CDPR, the position of PM in any arbitrary position can be obtained through robot forward kinematics with respect to the global coordinates at the bedside. The experimental results show that the root mean square error (RMSE) for the estimated position value of PM was less than 1.13 mm in the X, Y, and Z directions and less than 1.14° in the θ and φ orientation, where the sensing space could be extended to ±70 mm for the given 34 × 34 mm² HES array and the average moving distance in the Z-direction is 40 ± 2.42 mm. The proposed method of the robotic sensing with HES and CDPR may advance the sensing space expansion technology by utilizing the provided single sensor module of limited sensible volume.

A Novel Approach for Automatic State Detection of A Magnetically Actuated Capsule

In recent years, the Simultaneous Magnetic Actuation and Localization (SMAL) technology has been developed to accelerate and locate the wireless capsule endoscopy (WCE) in the intestine. In this paper, we propose a novel approach to detect the state of the capsule for improving the localization results. By creating a function to fit the relationship between the theoretical values of the actuating magnetic field and the measurement results, we present an algorithm for automatic estimation of the capsule state according to the fitting parameters. Experiment results on phantoms demonstrate the feasibility of the proposed method for detecting different states of the capsule during magnetic actuation.

Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation

Early diagnosis of colorectal cancer significantly improves survival. However, over half of cases are diagnosed late due to demand exceeding the capacity for colonoscopy - the "gold standard" for screening. Colonoscopy is limited by the outdated design of conventional endoscopes, associated with high complexity of use, cost and pain. Magnetic endoscopes represent a promising alternative, overcoming drawbacks of pain and cost, but struggle to reach the translational stage as magnetic manipulation is complex and unintuitive. In this work, we use machine vision to develop intelligent and autonomous control of a magnetic endoscope, for the first time enabling non-expert users to effectively perform magnetic colonoscopy in-vivo. We combine the use of robotics, computer vision and advanced control to offer an intuitive and effective endoscopic system. Moreover, we define the characteristics required to achieve autonomy in robotic endoscopy. The paradigm described here can be adopted in a variety of applications where navigation in unstructured environments is required, such as catheters, pancreatic endoscopy, bronchoscopy, and gastroscopy. This work brings alternative endoscopic technologies closer to the translational stage, increasing availability of early-stage cancer treatments.

Simultaneous Independent Translational and Rotational Feedback Motion Control System for a Cylindrical Magnet using Planar Arrays of Magnetic Sensors and Cylindrical Coils

This letter describes an electromagnetic feedback control system for rigid-body motion control of a magnet. Its novel features are that sensing and actuation using magnetometer sensors and actuator coils operate simultaneously, and magnetic field models from the controlled magnet and each of the actuator coil currents are used together to calculate the 3D position and orientation of the magnet to control motion simultaneously and independently in multiple degrees of freedom including planar translation and two in rotation, leaving rotation about the cylindrical axis of magnetization uncontrolled. The system configuration and the localization and actuation methods are presented with experimental results of magnet localization with constant and varying coil currents, and during feedback control of trajectory following motion of the magnet in multiple directions on a planar surface and with controlled changes in orientation. The intended application of the system is for motion control of magnetic endoscope capsules and other miniature medical devices inside the human body.

Sensitivity Ellipsoids for Force Control of Magnetic Robots with Localization Uncertainty

The navigation of magnetic medical robots typically relies on localizing an actuated, intracorporeal, ferromagnetic body and back-computing a necessary field and gradient that would result in a desired wrench on the device. Uncertainty in this localization degrades the precision of force transmission. Reducing applied force uncertainty may enhance tasks such as in-vivo navigation of miniature robots, actuation of magnetically guided catheters, tissue palpation, as well as simply ensuring a bound on forces applied on sensitive tissue. In this paper, we analyzed the effects of localization noise on force uncertainty by using sensitivity ellipsoids of the magnetic force Jacobian and introduced an algorithm for uncertainty reduction. We validated the algorithm in both a simulation study and in a physical experiment. In simulation, we observed reductions in estimated force uncertainty by factors of up to 2.8 and 3.1 when using one and two actuating magnets, respectively. On a physical platform, we demonstrated a force uncertainty reduction by a factor of up to 2.5 as measured using an external sensor. Being the first consideration of force uncertainty resulting from noisy localization, this work provides a strategy for investigators to minimize uncertainty in magnetic force transmission.

Enhanced Real-Time Pose Estimation for Closed-Loop Robotic Manipulation of Magnetically Actuated Capsule Endoscopes

Pose estimation methods for robotically guided magnetic actuation of capsule endoscopes have recently enabled trajectory following and automation of repetitive endoscopic maneuvers. However, these methods face significant challenges in their path to clinical adoption including the presence of regions of magnetic field singularity, where the accuracy of the system degrades, and the need for accurate initialization of the capsule's pose. In particular, the singularity problem exists for any pose estimation method that utilizes a single source of magnetic field if the method does not rely on the motion of the magnet to obtain multiple measurements from different vantage points. We analyze the workspace of such pose estimation methods with the use of the point-dipole magnetic field model and show that singular regions exist in areas where the capsule is nominally located during magnetic actuation. Since the dipole model can approximate most magnetic field sources, the problem discussed herein pertains to a wider set of pose estimation techniques. We then propose a novel hybrid approach employing static and time-varying magnetic field sources and show that this system has no regions of singularity. The proposed system was experimentally validated for accuracy, workspace size, update rate and performance in regions of magnetic singularity. The system performed as well or better than prior pose estimation methods without requiring accurate initialization and was robust to magnetic singularity. Experimental demonstration of closed-loop control of a tethered magnetic device utilizing the developed pose estimation technique is provided to ascertain its suitability for robotically guided capsule endoscopy. Hence, advances in closed-loop control and intelligent automation of magnetically actuated capsule endoscopes can be further pursued toward clinical realization by employing this pose estimation system.

Magnetically guided capsule endoscopy

Wireless capsule endoscopy (WCE) is a powerful tool for medical screening and diagnosis, where a small capsule is swallowed and moved by means of natural peristalsis and gravity through the human gastrointestinal (GI) tract. The camera-integrated capsule allows for visualization of the small intestine, a region which was previously inaccessible to classical flexible endoscopy. As a diagnostic tool, it allows to localize the sources of bleedings in the middle part of the gastrointestinal tract and to identify diseases, such as inflammatory bowel disease (Crohn's disease), polyposis syndrome, and tumors. The screening and diagnostic efficacy of the WCE, especially in the stomach region, is hampered by a variety of technical challenges like the lack of active capsular position and orientation control. Therapeutic functionality is absent in most commercial capsules, due to constraints in capsular volume and energy storage. The possibility of using body-exogenous magnetic fields to guide, orient, power, and operate the capsule and its mechanisms has led to increasing research in Magnetically Guided Capsule Endoscopy (MGCE). This work shortly reviews the history and state-of-art in WCE technology. It highlights the magnetic technologies for advancing diagnostic and therapeutic functionalities of WCE. Not restricting itself to the GI tract, the review further investigates the technological developments in magnetically guided microrobots that can navigate through the various air- and fluid-filled lumina and cavities in the body for minimally invasive medicine.