The i2Snake Robotic Platform for Endoscopic Surgery

Evaluation of current and emerging endoluminal robotic platforms using the IDEAL framework

BACKGROUND

Robotic-assisted endoluminal systems are rapidly evolving within the field of minimally invasive surgery. The IDEAL framework (Idea, Development, Exploration, Assessment, and Surveillance) can be used to evaluate novel technologies. This review provides a summary of current and emerging endoluminal systems using the IDEAL framework.

METHODS

A scoping review was conducted to include all existing and developing robotic-assisted endoluminal systems. Data was collected via virtual interviews, questionnaires, biomedical databases, company websites, and peer-reviewed articles. Key metrics were reported, enabling the assignment of each system to an IDEAL stage.

RESULTS

The review identified 17 distinct systems from 16 companies. Nine systems received regulatory approval in their respective countries. Our evaluation showed that two systems were at the pre-IDEAL Stage 0. Seven systems were in the Idea stage (Stage 1), six systems were in the Development stage (Stage 2) and two systems completed Stage 3. No system underwent long-term study evaluation (Stage 4).

CONCLUSIONS

There is a gap in long-term clinical data of robotic-assisted endoluminal systems, indicated by the absence of systems at Stage 4. Collaborative efforts amongst the medical community, regulatory bodies, and industry specialists are vital to ensure the delivery of evidence-based medicine in the discipline of robotics.

Endoluminal robotics

The search for less-invasive approaches led to the birth of endoluminal surgery 40 years ago. The limited dexterity of flexible endoscopic systems generated an interest in the development of robotic platforms to encourage the adoption of surgical-like, bimanual techniques involving manipulation of tissues, thus favoring organ preservation. Nevertheless, the road ahead for endoluminal surgery is still at the beginning.

Real-Time Shape Estimation of Hyper-Redundant Flexible Manipulator Using Coiled Fiber Sensors

The shape of flexible endoscopic surgical robot should be obtained to increase control accuracy and prevent unwanted tissue damage. To estimate the shape of flexible manipulator, space efficiency, cost-effectiveness, system complexity, and ease of calibration should be considered to integrate sensors into the manipulator. In this article, we propose a real-time method to estimate the shape of a hyper-redundant manipulator having embedded coiled fiber sensors. The main advantage of this method is guaranteeing shape recognition even when the manipulator is subjected to an external load. The fiber sensors are highly flexible, compact, and inexpensive, as well as they can functionally measure both compressive and tensile strain of hyper-redundant manipulator. The sensor design was optimized to achieve durability and sensitivity. The numbers of sensor and the placements were determined by the analysis of the kinematics and moment distribution of the manipulator. The accuracy of shape estimation was validated experimentally under both free-loading and loading conditions. The proposed method achieved real-time estimating capability with a mean maximum error of each joint position smaller than 3.54% in free-loading condition and 5.47% in loading condition.

Neural Simulation of Actions for Serpentine Robots

The neural or mental simulation of actions is a powerful tool for allowing cognitive agents to develop Prospection Capabilities that are crucial for learning and memorizing key aspects of challenging skills. In previous studies, we developed an approach based on the animation of the redundant human body schema, based on the Passive Motion Paradigm (PMP). In this paper, we show that this approach can be easily extended to hyper-redundant serpentine robots as well as to hybrid configurations where the serpentine robot is functionally integrated with a traditional skeletal infrastructure. A simulation model is analyzed in detail, showing that it incorporates spatio-temporal features discovered in the biomechanical studies of biological hydrostats, such as the elephant trunk or octopus tentacles. It is proposed that such a generative internal model could be the basis for a cognitive architecture appropriate for serpentine robots, independent of the underlying design and control technologies. Although robotic hydrostats have received a lot of attention in recent decades, the great majority of research activities have been focused on the actuation/sensorial/material technologies that can support the design of hyper-redundant soft/serpentine robots, as well as the related control methodologies. The cognitive level of analysis has been limited to motion planning, without addressing synergy formation and mental time travel. This is what this paper is focused on.

Snake Robot with Motion Based on Shape Memory Alloy Spring-Shaped Actuators

This study presents the design and evaluation of a prototype snake-like robot that possesses an actuation system based on shape memory alloys (SMAs). The device is constructed based on a modular structure of links connected by two degrees of freedom links utilizing Cardan joints, where each degree of freedom is actuated by an agonist-antagonist mechanism using the SMA spring-shaped actuators to generate motion, which can be easily replaced once they reach a degradation point. The methodology for programming the spring shape into the SMA material is described in this work, as well as the instrumentation required for the monitoring and control of the actuators. A simplified design is presented to describe the way in which the motion is performed and the technical difficulties faced in manufacturing. Based on this information, the way in which the design is adapted to generate a feasible robotic system is described, and a mathematical model for the robot is developed to implement an independent joint controller. The feasibility of the implementation of the SMA actuators regarding the motion of the links is verified for the case of a joint, and the change in the shape of the snake robot is verified through the implementation of a set of tracking references based on a central pattern generator. The generated tracking results confirm the feasibility of the proposed mechanism in terms of performing snake gaits, as well as highlighting some of the drawbacks that should be considered in further studies.

A Framework for the Evaluation of Human Machine Interfaces of Robot-Assisted Colonoscopy

The Human Machine Interface (HMI) of intraluminal robots has a crucial impact on the clinician's performance. It increases or decreases the difficulty of the tasks, and is connected to the users' physical and mental stress.

OBJECTIVE

This article presents a framework to compare and evaluate different HMIs for robotic colonoscopy, with the objective of identifying the optimal HMI that minimises the clinician's effort and maximises the clinical outcomes.

METHODS

The framework comprises a 1) a virtual simulator (clinically validated), 2) wearable sensors measuring the cognitive load, 3) a data collection unit of metrics correlated to the clinical performance, and 4) questionnaires exploring the users' impressions and perceived stress. The framework was tested with 42 clinicians investigating the optimal device for tele-operated control of robotic colonoscopes. Two control devices were selected and compared: a haptic serial-kinematic device and a standard videogame joypad.

RESULTS

The haptic device was preferred by the endoscopists, but the joypad enabled better clinical performance and reduced cognitive and physical load.

CONCLUSION

The framework can be used to evaluate different aspects of a HMI, both hardware and software, and determine the optimal HMI that can reduce the burden on clinicians while improving the clinical outcome.

SIGNIFICANCE

The findings of this study, and of future studies performed with this framework, can inform the design and development of HMIs for intraluminal robots, leading to improved clinical performance, reduced physical and mental stress for clinicians, and ultimately better patient outcomes.

Bioinspired handheld time-share driven robot with expandable DoFs

Handheld robots offer accessible solutions with a short learning curve to enhance operator capabilities. However, their controllable degree-of-freedoms are limited due to scarce space for actuators. Inspired by muscle movements stimulated by nerves, we report a handheld time-share driven robot. It comprises several motion modules, all powered by a single motor. Shape memory alloy (SMA) wires, acting as "nerves", connect to motion modules, enabling the selection of the activated module. The robot contains a 202-gram motor base and a 0.8 cm diameter manipulator comprised of sequentially linked bending modules (BM). The manipulator can be tailored in length and integrated with various instruments in situ, facilitating non-invasive access and high-dexterous operation at remote surgical sites. The applicability was demonstrated in clinical scenarios, where a surgeon held the robot to conduct transluminal experiments on a human stomach model and an ex vivo porcine stomach. The time-share driven mechanism offers a pragmatic approach to build a multi-degree-of-freedom robot for broader applications.

A novel miniature flexible robotic system for endoscopic mucosal dissection: an animal experimental study

Endoscopic submucosal dissection (ESD) is a standard treatment for early gastrointestinal cancer due to its higher rate of en-bloc resection and lower recurrence rate. However, the technical challenges lead to long learning curve and high risks of adverse events. A gastrointestinal flexible robotic-tool system (GIFTS) was proposed to reduce the difficulty and shorten the learning curve of novices. This is an animal study to evaluate the feasibility of GIFTS in ESD. The GIFTS provides a total of 13 degrees of freedom within 10 mm in diameter and variable stiffness function to achieve endoscopic intervention and submucosal dissection with the cooperation of two flexible robotic instruments. One esophageal and four colorectal ESDs in five porcine models were performed. In all five ESD procedures, the GIFTS was successfully intubated and submucosal dissection was completed without perforation or significant bleeding, and there was no system fault. The mean operative time was 99 min, and the mean size of the specimen was 151 mm2. The fifth experiment showed significantly better results than the first one. In vivo animal experiments confirmed the feasibility of GIFTS in performing ESD. The control of GIFTS is friendly to inexperienced beginners, which will help reduce the technical challenges of ESD and shorten the learning curve of endoscopists.

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

Microneedle puncture is a standard minimally invasive treatment and surgical method, which is widely used in extracting blood, tissues, and their secretions for pathological examination, needle-puncture-directed drug therapy, local anaesthesia, microwave ablation needle therapy, radiotherapy, and other procedures. The use of robots for microneedle puncture has become a worldwide research hotspot, and medical imaging navigation technology plays an essential role in preoperative robotic puncture path planning, intraoperative assisted puncture, and surgical efficacy detection. This paper introduces medical imaging technology and minimally invasive puncture robots, reviews the current status of research on the application of medical imaging navigation technology in minimally invasive puncture robots, and points out its future development trends and challenges.

Advanced Endoscopy for Thoracic Surgeons

The thoracic surgeon, well versed in advanced endoscopy, has an array of therapeutic options for foregut pathologic conditions. Peroral endoscopic myotomy (POEM) offers a less-invasive means to treat achalasia, and the authors' preferred approach is described in this article. They also describe variations of POEM, such as G-POEM, Z-POEM, and D-POEM. In addition, endoscopic stenting, endoluminal vacuum therapy, endoscopic internal drainage, and endoscopic suturing/clipping are discussed and can be valuable tools for esophageal leaks and perforations. Endoscopic procedures are advancing rapidly, and thoracic surgeons must maintain at the forefront of these technologies.

Robotic endoscope with double-balloon and double-bend tube for colonoscopy

The insertion of conventional colonoscopes can sometimes cause patients to experience pain during the procedure owing to the stretching of the mesentery. In this study, a prototype of a robotic colonoscope with a double-balloon and double-bend tube based on the conventional double-balloon endoscope was developed to simplify insertion and prevent the overstretching of the colon. Both the outer and inner tubes were confirmed to be free from interference from wires and sheaths. Additionally, all functions such as tip bending, inflation and deflation of the balloons, and actuator-driven pulling and pushing of the inner tube were operated properly. During the insertion test, the device could be reached the cecum of a colon model in approximately 442 s when operated by a non-medical operator. In addition, the device did not overstretch the colon model, thereby suggesting that the insertion mechanism can follow the shape of the colon model. As a result, the developed mechanism has the potential to navigate through a highly-bent colon without overstretching.

Navigation with minimal occupation volume for teleoperated snake-like surgical robots: MOVE

Master-Slave control is a common mode of operation for surgical robots as it ensures that surgeons are always in control and responsible for the procedure. Most teleoperated surgical systems use low degree-of-freedom (DOF) instruments, thus facilitating direct mapping of manipulator position to the instrument pose and tip location (tip-to-tip mapping). However, with the introduction of continuum and snake-like robots with much higher DOF supported by their inherent redundant architecture for navigating through curved anatomical pathways, there is a need for developing effective kinematic methods that can actuate all the joints in a controlled fashion. This paper introduces the concept of navigation with Minimal Occupation VolumE (MOVE), a teleoperation method that extends the concept of follow-the-leader navigation. It defines the path taken by the head while using all the available space surrounding the robot constrained by individual joint limits. The method was developed for the i ² Snake robot and validated with detailed simulation and control experiments. The results validate key performance indices such as path following, body weights, path weights, fault tolerance and conservative motion. The MOVE solver can run in real-time on a standard computer at frequencies greater than 1 kHz.

User interfaces for actuated scope maneuvering in surgical systems: a scoping review

BACKGROUND

A variety of human computer interfaces are used by robotic surgical systems to control and actuate camera scopes during minimally invasive surgery. The purpose of this review is to examine the different user interfaces used in both commercial systems and research prototypes.

METHODS

A comprehensive scoping review of scientific literature was conducted using PubMed and IEEE Xplore databases to identify user interfaces used in commercial products and research prototypes of robotic surgical systems and robotic scope holders. Papers related to actuated scopes with human-computer interfaces were included. Several aspects of user interfaces for scope manipulation in commercial and research systems were reviewed.

RESULTS

Scope assistance was classified into robotic surgical systems (for multiple port, single port, and natural orifice) and robotic scope holders (for rigid, articulated, and flexible endoscopes). Benefits and drawbacks of control by different user interfaces such as foot, hand, voice, head, eye, and tool tracking were outlined. In the review, it was observed that hand control, with its familiarity and intuitiveness, is the most used interface in commercially available systems. Control by foot, head tracking, and tool tracking are increasingly used to address limitations, such as interruptions to surgical workflow, caused by using a hand interface.

CONCLUSION

Integrating a combination of different user interfaces for scope manipulation may provide maximum benefit for the surgeons. However, smooth transition between interfaces might pose a challenge while combining controls.

Continuum Robots: From Conventional to Customized Performance Indicators

Continuum robots have often been compared with rigid-link designs through conventional performance metrics (e.g., precision and Jacobian-based indicators). However, these metrics were developed to suit rigid-link robots and are tuned to capture specific facets of performance, in which continuum robots do not excel. Furthermore, conventional metrics either fail to capture the key advantages of continuum designs, such as their capability to operate in complex environments thanks to their slender shape and flexibility, or see them as detrimental (e.g., compliance). Previous work has rarely addressed this issue, and never in a systematic way. Therefore, this paper discusses the facets of a continuum robot performance that cannot be characterized by existing indicator and aims at defining a tailored framework of geometrical specifications and kinetostatic indicators. The proposed framework combines the geometric requirements dictated by the target environment and a methodology to obtain bioinspired reference metrics from a biological equivalent of the continuum robot (e.g., a snake, a tentacle, or a trunk). A numerical example is then reported for a swimming snake robot use case.

2.5-mm articulated endoluminal forceps using a compliant mechanism

PURPOSE

Gastrointestinal cancer can be treated using a flexible endoscope through a natural orifice. However, treatment instruments with limited degrees of freedom (DOFs) require a highly skilled operator. Articulated devices useful for endoluminal procedures, such as endoscopic submucosal dissection and biopsy, have been developed. These devices enable dexterous operation in a narrow lumen; however, they suffer from limitations such as large size and high cost. To overcome these limitations, we developed a 2.5-mm articulated forceps that can be inserted into a standard endoscope channel based on a compliant mechanism.

METHODS

The compliant mechanism allows the device to be compact and affordable, which is possible due to its monolithic structure. The proposed mechanism consists of two segments, 1-DOF grasping and 2-DOF bending, that are actuated by tendon-sheath mechanisms. A prototype was designed based on finite element analysis results.

RESULTS

To confirm the effectiveness of the proposed mechanism, we fabricated the prototype using a 3D printer. A series of mechanical performance tests on the prototype revealed that it achieved the following specifications: (1) DOF: 1-DOF grasping + 2-DOF bending, (2) outer diameter: 2.5 mm, (3) length of the bending segment: 30 mm, and (4) range of motion: [Formula: see text] to [Formula: see text] (grasping) and [Formula: see text] to [Formula: see text] (bending). Finally, we performed a tissue manipulation test on an excised porcine colon and found that a piece of mucous membrane tissue was successfully resected using an electric knife while being lifted with the developed forceps.

CONCLUSION

The results of the evaluation experiment demonstrated a positive feasibility of the proposed mechanism, which has a simpler structure compared to those of other conventional mechanisms; furthermore, it is potentially more cost-effective and is disposable. The mechanical design, prototype implementation, and evaluations are reported in this paper.

Robotics in therapeutic endoscopy (with video)

Since its inception, endoscopy has evolved from a solely diagnostic procedure to an expanding therapeutic field within gastroenterology. The incorporation of robotics in gastroenterology initially addressed shortcomings of flexible endoscopes in natural orifice transluminal endoscopy. Developing therapeutic endoscopic robotic platforms now offer operators improved ergonomics, visualization, dexterity, precision, and control and the possibility of increasing proficiency and standardization of complex endoscopic procedures including endoscopic submucosal dissection, endoscopic full-thickness resection, and endoscopic suturing. The following review discusses the history, potential applications, and tools currently available and in development for robotics in therapeutic endoscopy.

Continuum Robots for Medical Interventions

Continuum robots are not constructed with discrete joints but, instead, change shape and position their tip by flexing along their entire length. Their narrow curvilinear shape makes them well suited to passing through body lumens, natural orifices, or small surgical incisions to perform minimally invasive procedures. Modeling and controlling these robots are, however, substantially more complex than traditional robots comprised of rigid links connected by discrete joints. Furthermore, there are many approaches to achieving robot flexure. Each presents its own design and modeling challenges, and to date, each has been pursued largely independently of the others. This article attempts to provide a unified summary of the state of the art of continuum robot architectures with respect to design for specific clinical applications. It also describes a unifying framework for modeling and controlling these systems while additionally explaining the elements unique to each architecture. The major research accomplishments are described for each topic and directions for the future progress needed to achieve widespread clinical use are identified.

Sigmoidal Auxiliary Tendon-Driven Mechanism Reinforcing Structural Stiffness of Hyper-Redundant Manipulator for Endoscopic Surgery

The overtube of an endoscopic surgery robot is fixed when performing tasks, unlike those of commercial endoscopes, and this overtube should have high structural stiffness after reaching the target lesion so that sufficient tension can be applied to the lesion tissue with the surgical tool and there are fewer changes in the field of view of the endoscopic camera from this reaction force. Various methods have been proposed to reinforce the structural stiffnesses of hyper-redundant manipulators. However, the safety, rapid response, space efficiency, and cost-effectiveness of these methods should be considered for use in actual clinical environments, such as the gastrointestinal tract. This study proposed a method to minimize the positional changes of the overtube end tip due to external forces using only auxiliary tendons in the optimized path without additional mechanical structures. Overall, the proposed method involved moving the overtube to the target lesion through the main driving tendon and applying tension to the auxiliary tendons to reinforce the structural stiffness. The complete system was analyzed in terms of energy, and the sigmoidal auxiliary tendons were verified to effectively reinforce the structural stiffness of the overtube consisting of rolling joints. In addition, the design guidelines of the overtube for actual endoscopic surgery were proposed considering hollowness, retroflexion, and high structural stiffness. The positional changes due to external forces were confirmed to be reduced by 60% over the entire workspace.

A Novel Collision-Free Homotopy Path Planning for Planar Robotic Arms

Achieving the smart motion of any autonomous or semi-autonomous robot requires an efficient algorithm to determine a feasible collision-free path. In this paper, a novel collision-free path homotopy-based path-planning algorithm applied to planar robotic arms is presented. The algorithm utilizes homotopy continuation methods (HCMs) to solve the non-linear algebraic equations system (NAES) that models the robot’s workspace. The method was validated with three case studies with robotic arms in different configurations. For the first case, a robot arm with three links must enter a narrow corridor with two obstacles. For the second case, a six-link robot arm with a gripper is required to take an object inside a narrow corridor with two obstacles. For the third case, a twenty-link arm must take an object inside a maze-like environment. These case studies validated, by simulation, the versatility and capacity of the proposed path-planning algorithm. The results show that the CPU time is dozens of milliseconds with a memory consumption less than 4.5 kB for the first two cases. For the third case, the CPU time is around 2.7 s and the memory consumption around 18 kB. Finally, the method’s performance was further validated using the industrial robot arm CRS CataLyst-5 by Thermo Electron.

Snake Robots for Surgical Applications: A Review

Although substantial advancements have been achieved in robot-assisted surgery, the blueprint to existing snake robotics predominantly focuses on the preliminary structural design, control, and human–robot interfaces, with features which have not been particularly explored in the literature. This paper aims to conduct a review of planning and operation concepts of hyper-redundant serpentine robots for surgical use, as well as any future challenges and solutions for better manipulation. Current researchers in the field of the manufacture and navigation of snake robots have faced issues, such as a low dexterity of the end-effectors around delicate organs, state estimation and the lack of depth perception on two-dimensional screens. A wide range of robots have been analysed, such as the i²Snake robot, inspiring the use of force and position feedback, visual servoing and augmented reality (AR). We present the types of actuation methods, robot kinematics, dynamics, sensing, and prospects of AR integration in snake robots, whilst addressing their shortcomings to facilitate the surgeon’s task. For a smoother gait control, validation and optimization algorithms such as deep learning databases are examined to mitigate redundancy in module linkage backlash and accidental self-collision. In essence, we aim to provide an outlook on robot configurations during motion by enhancing their material compositions within anatomical biocompatibility standards.

First in-human trial and prospective case series of an articulated laparoscopic camera system in minimally invasive surgery in gynecology: an IDEAL stage 1 and 2a study

Objectives

Precision Robotics’ Sirius Robotic Flexible Endoscopic System is a new, fully integrated, compact three-dimensional laparoscopic camera system with a disposable single-use flexible tip that can change its viewing direction. This IDEAL Stage 1 and 2a study assessed its safety, reliability and potential efficacy particularly for single incision laparoscopic surgery and vaginal natural orifice transluminal endoscopic surgery.

Design

Prospective single-institution, single-surgeon study.

Setting

The study was conducted in a multispecialty hospital.

Participants

Women aged 18–70 years scheduled for gynecological laparoscopic surgery were invited to participate. An information sheet and consent was available for the women and an informed consent was obtained. Thirteen participants completed this study.

Interventions

The laparoscopic procedures were done in the usual manner. The only difference was the Sirius System was used in place of the conventional laparoscope. All other procedures and instruments remained the same.

Main outcome measures

Primary outcome was the proportion of women who successfully completed the intended procedure using the Sirius System without conversion to another camera system, camera users and surgeon’s view and experience, and iterations and modifications to the system. Secondary outcomes were the incidence of intraoperative and postoperative complications during the first 6 weeks following surgery, and duration of surgery.

Results

85% (11/13) of women had their procedure completed successfully using the Sirius System. Two women required immediate conversion to the conventional laparoscope due to technical issues. There were no intraoperative complications. Users agreed that the improved field of view was beneficial for laparoscopic surgery. Iterative improvements were made in the imaging quality, user interface and manufacturing quality.

Conclusions

Sirius System has early indications for safety and efficacy for intermediate and major minimally invasive laparoscopic procedures in gynecology. Further studies are needed to confirm it can replace a conventional laparoscope in the surgical workflow.

Trial registration number

NCT05048407.

Stress Dispersion Design in Continuum Compliant Structure toward Multi-DOF Endoluminal Forceps

Gastrointestinal cancer, when detected early, is treated by accessing the lesion through the natural orifice using flexible endoscopes. However, the limited degree-of-freedom (DOF) of conventional treatment devices and the narrow surgical view through the endoscope demand advanced techniques. In contrast, multi-DOF forceps systems are an excellent alternative; however, these systems often involve high fabrication costs because they require a large number of micro-parts. To solve this problem, we designed compact multi-DOF endoluminal forceps with a monolithic structure comprising compliant hinges. To allow an efficient stress dispersion at the base end when the hinge bends, we proposed a novel design method to obtain the hinge parameters using the beam of uniform strength theory. This method does not involve a high computational cost. The results show that the improved design with a variable hinge thickness can reduce the maximum bending stress, dispersing the stress in a larger area than that of the previous design considering a constant thickness of the hinge. Moreover, the experiments conducted in a prototype confirm that the radius of the curvature was significantly improved. The proposed method could aid in designing other continuum robots relying on compliant hinges.

A Three-Limb Teleoperated Robotic System with Foot Control for Flexible Endoscopic Surgery

Flexible endoscopy requires a lot of skill to manipulate both the endoscope and the associated instruments. In most robotic flexible endoscopic systems, the endoscope and instruments are controlled separately by two operators, which may result in communication errors and inefficient operation. Our solution is to enable the surgeon to control both the endoscope and the instruments. Here, we present a novel tele-operation robotic endoscopic system commanded by one operator using the continuous and simultaneous movements of their two hands and one foot. This 13-degree-of-freedom (DoF) system integrates a foot-controlled robotic flexible endoscope and two hand-controlled robotic endoscopic instruments, a robotic grasper and a robotic cauterizing hook. A dedicated foot-interface transfers the natural foot movements to the 4-DoF movements of the endoscope while two other commercial hand interfaces map the movements of the two hands to the two instruments individually. An ex-vivo experiment was carried out by six subjects without surgical experience, where the simultaneous control with foot and hands was compared with a sequential clutch-based hand control. The participants could successfully teleoperate the endoscope and the two instruments to cut the tissues at scattered target areas in a porcine stomach. Foot control yielded 43.7% faster task completion and required less mental effort as compared to the clutch-based hand control scheme, which proves the concept of three-limb tele-operation surgery and the developed flexible endoscopic system.

Hard frame detection for the automated clipping of surgical nasal endoscopic video

PURPOSE

The automated clipping of surgical nasal endoscopic video is a challenging task because there are many hard frames that have indiscriminative visual features which lead to misclassification. Prior works mainly aim to classify these hard frames along with other frames, and it would seriously affect the performance of classification.

METHODS

We propose a hard frame detection method using a convolutional LSTM network (called HFD-ConvLSTM) to remove invalid video frames automatically. Firstly, a new separator based on the coarse-grained classifier is defined to remove the invalid frames. Meanwhile, the hard frames are detected via measuring the blurring score of a video frame. Then, the squeeze-and-excitation is used to select the informative spatial-temporal features of endoscopic videos and further classify the video frames with a fine-grained ConvLSTM learning from the reconstructed training set with hard frames.

RESULTS

We justify the proposed solution through extensive experiments using 12 surgical videos (duration:8501 s). The experiments are performed on both hard frame detection and video frame classification. Nearly 88.3% fuzzy frames can be detected and the classification accuracy is boosted to 95.2%. HFD-ConvLSTM achieves superior performance compared to other methods.

CONCLUSION

HFD-ConvLSTM provides a new paradigm for video clipping by breaking the complex clipping problem into smaller, more easily managed 2-classification problems. Our investigation reveals that the hard framed detection based on blurring score calculation is effective for nasal endoscopic video clipping.

easyEndo robotic endoscopy system: Development and usability test in a randomized controlled trial with novices and physicians

BACKGROUND

Some difficulties are common when using endoscopes. Steering is not intuitive, the endoscope weight is a physical burden to physicians and communication problems often occur between operators.

METHOD

To overcome these, we developed a robotic endoscopy system and conducted a usability test to compare conventional and robotic manipulation. Nine novices and eighteen physicians participated with the physicians being divided into intermediate and expert groups. The participants performed endoscope insertion into a simulator (physicians) or lesion marking on a testbed (novices) and simulate biopsies.

RESULT

Novices completed the tasks faster and with a lower workload when using robotic manipulation, whereas the experts showed the opposite trend. Still, the intermediates showed no significant difference as trials proceeded. Nevertheless, the learning curve analysis showed that the learning rate in all groups is greater for robotic manipulation (21.02% on average) than for conventional manipulation (13.75%) and predicted that physicians can reach manual performance.

CONCLUSION

The proposed robotic endoscopy system may allow solo-manipulation using one controller and may be more intuitive and convenient to use than conventional manipulation.

Positioning and stiffening of an articulated/continuum manipulator for implant delivery in minimally invasive surgery

BACKGROUND

Hollow, bendable manipulators can advance implant delivery in minimally invasive surgery, by circumventing the drawbacks of straight-line delivery and fostering single-port approaches. Variable stiffness manipulators are sought to be safe and effective.

METHODS

We designed and experimentally assessed a cable-driven articulated/continuum manipulator, devised for cardiac valve delivery. Positioning and stiffening were teleoperated, based on cable shortening. Stiffening was parameterized by using the leading tension (LT, ie, tension of the cables driving bending). We assessed positioning (repeatability/reversibility along eight/two bending directions) and stiffening (eight bent configurations).

RESULTS

We achieved good repeatability and reversibility (mean errors <1% and 1.5%, respectively, of the workspace characteristic length). Stiffening was effective (up to 9-fold increase, depending on pose). Stiffening was linearly correlated (R² = 0.92) with LT for all the considered configurations.

CONCLUSION

We accurately positioned and effectively stiffened the manipulator in several bent configurations. The proposed stiffness modulation strategy can be extended to other manipulators.

Fluidic Fabric Muscle Sheets for Wearable and Soft Robotics

Conformable robotic systems are attractive for applications in which they may actuate structures with large surface areas, provide forces through wearable garments, or enable autonomous robotic systems. We present a new family of soft actuators that we refer to as Fluidic Fabric Muscle Sheets (FFMS). They are composite fabric structures that integrate fluidic transmissions based on arrays of elastic tubes. These sheet-like actuators can strain, squeeze, bend, and conform to hard or soft objects of arbitrary shapes or sizes, including the human body. We show how to design and fabricate FFMS actuators via facile apparel engineering methods, including computerized sewing techniques that determine the stress and strain distributions that can be generated. We present a simple mathematical model that proves effective for predicting their performance. FFMS can operate at frequencies of 5 Hz or more, achieve engineering strains exceeding 100%, and exert forces >115 times their weight. They can be safely used in intimate contact with the human body even when delivering stresses exceeding 10⁶ Pascals. We demonstrate their versatility for actuating a variety of bodies or structures, and in configurations that perform multiaxis actuation, including bending and shape change. As we also show, FFMS can be used to exert forces on body tissues for wearable and biomedical applications. We demonstrate several potential use cases, including a miniature steerable robot, a glove for grasp assistance, garments for applying compression to the extremities, and devices for actuating small body regions or tissues via localized skin stretch.

The evolution of lower gastrointestinal endoscopy: where are we now?

Lower gastrointestinal endoscopy has evolved over time, fulfilling a widening diagnostic and therapeutic remit. As our understanding of colorectal cancer and its prevention has improved, endoscopy has progressed with improved diagnostic technologies and advancing endoscopic therapies. Despite this, the fundamental design of the endoscope has remained similar since its inception. This review presents the important role lower gastrointestinal endoscopy serves in the prevention of colorectal cancer and the desirable characteristics of the endoscope that would enhance this. A brief history of the endoscope is presented. Current and future robotic endoscopic platforms, which may fulfil these desirable characteristics, are discussed. The incorporation of new technologies from allied scientific disciplines will help the endoscope fulfil its maximum potential in preventing the increasing global burden of colorectal cancer. There are a number of endoscopic platforms under development, which show significant promise.

Effect of backlash hysteresis of surgical tool bending joints on task performance in teleoperated flexible endoscopic robot

BACKGROUND

The tendon-sheath mechanism provides flexibility but degrades the task performance of the flexible endoscopic robot because of the inherent backlash hysteresis problem. Previous studies have only focused on reducing backlash hysteresis. The goal of this study is to identify the backlash hysteresis criteria of surgical tool bending joints to maintain efficient surgical performance.

METHODS

A test platform for a surgical tool has been developed that has initial backlash hysteresis under 5° and can adjust the backlash hysteresis intentionally. Performance variation has been investigated in three bench-top endoscopic tasks in which various backlash hysteresis conditions were intentionally adjusted.

RESULTS

A clear drop-off in task performance has been observed when the backlash hysteresis of the bending joints was greater than 10° regardless of the type of task and link length.

CONCLUSIONS

The backlash hysteresis of surgical tool bending joints should be reduced to at least 10° to maintain efficient performance in robotic endoscopic surgery.

Combining Differential Kinematics and Optical Flow for Automatic Labeling of Continuum Robots in Minimally Invasive Surgery

The segmentation of continuum robots in medical images can be of interest for analyzing surgical procedures or for controlling them. However, the automatic segmentation of continuous and flexible shapes is not an easy task. On one hand conventional approaches are not adapted to the specificities of these instruments, such as imprecise kinematic models, and on the other hand techniques based on deep-learning showed interesting capabilities but need many manually labeled images. In this article we propose a novel approach for segmenting continuum robots on endoscopic images, which requires no prior on the instrument visual appearance and no manual annotation of images. The method relies on the use of the combination of kinematic models and differential kinematic models of the robot and the analysis of optical flow in the images. A cost function aggregating information from the acquired image, from optical flow and from robot encoders is optimized using particle swarm optimization and provides estimated parameters of the pose of the continuum instrument and a mask defining the instrument in the image. In addition a temporal consistency is assessed in order to improve stochastic optimization and reject outliers. The proposed approach has been tested for the robotic instruments of a flexible endoscopy platform both for benchtop acquisitions and an in vivo video. The results show the ability of the technique to correctly segment the instruments without a prior, and in challenging conditions. The obtained segmentation can be used for several applications, for instance for providing automatic labels for machine learning techniques.

Functional Organ Preservation Surgery in Head and Neck Cancer: Transoral Robotic Surgery and Beyond

In recent years, interest in functional organ preservation surgery (FOPS) in the treatment of head and neck cancer has increased dramatically as clinicians seek to minimize the adverse effects of treatment while maximizing survival and quality of life. In this context, the use of transoral robotic surgery (TORS) is becoming increasingly common. TORS is a relatively new and rapidly-evolving technique, with a growing range of treatment indications. A wide range of novel, flexible surgical robots are now in development and their commercialization is expected to significantly expand the current indications for TORS. In the present review, we discuss the current and future role of this organ-preserving modality as the central element in the multimodal treatment of head and neck cancer.