Image-guided surgery: preliminary feasibility studies of frameless stereotactic liver surgery

Evaluation of the accuracy of an augmented reality-based tumor-targeting guide for breast-conserving surgery

BACKGROUND AND OBJECTIVES

Although magnetic resonance imaging (MRI) is commonly used for breast tumor detection, significant challenges remain in determining and presenting the three-dimensional (3D) morphology of tumors to guide breast-conserving surgery. To address this challenge, we have developed the augmented reality-breast surgery guide (AR-BSG) and compared its performance with that of a traditional 3D-printed breast surgical guide (3DP-BSG).

METHODS

Based on the MRI results of a breast cancer patient, a breast phantom made of skin, body, and tumor was fabricated through 3D printing and silicone-casting. AR-BSG and 3DP-BSG were executed using surgical plans based on the breast phantom's computed tomography scan images. Three operators independently inserted a catheter into the phantom using each guide. Their targeting accuracy was then evaluated using Bland-Altman analysis with limits of agreement (LoA). Differences between the users of each guide were evaluated using the intraclass correlation coefficient (ICC).

RESULTS

The entry and end point errors associated with AR-BSG were -0.34±0.68 mm (LoA: -1.71-1.01 mm) and 0.81±1.88 mm (LoA: -4.60-3.00 mm), respectively, whereas 3DP-BSG was associated with entry and end point errors of -0.28±0.70 mm (LoA: -1.69-1.11 mm) and -0.62±1.24 mm (LoA: -3.00-1.80 mm), respectively. The AR-BSG's entry and end point ICC values were 0.99 and 0.97, respectively, whereas 3DP-BSG was associated with entry and end point ICC values of 0.99 and 0.99, respectively.

CONCLUSIONS

AR-BSG can consistently and accurately localize tumor margins for surgeons without inferior guiding accuracy AR-BSG can consistently and accurately localize tumor margins for surgeons without inferior guiding accuracy compared to 3DP-BSG. Additionally, when compared with 3DP-BSG, AR-BSG can offer better spatial perception and visualization, lower costs, and a shorter setup time.

Intraoperative liver deformation and organ motion caused by ventilation, laparotomy, and pneumoperitoneum in a porcine model for image-guided liver surgery

BACKGROUND

Image-guidance promises to make complex situations in liver interventions safer. Clinical success is limited by intraoperative organ motion due to ventilation and surgical manipulation. The aim was to assess influence of different ventilatory and operative states on liver motion in an experimental model.

METHODS

Liver motion due to ventilation (expiration, middle, and full inspiration) and operative state (native, laparotomy, and pneumoperitoneum) was assessed in a live porcine model (n = 10). Computed tomography (CT)-scans were taken for each pig for each possible combination of factors. Liver motion was measured by the vectors between predefined landmarks along the hepatic vein tree between CT scans after image segmentation.

RESULTS

Liver position changed significantly with ventilation. Peripheral regions of the liver showed significantly higher motion (maximal Euclidean motion 17.9 ± 2.7 mm) than central regions (maximal Euclidean motion 12.6 ± 2.1 mm, p < 0.001) across all operative states. The total average motion measured 11.6 ± 0.7 mm (p < 0.001). Between the operative states, the position of the liver changed the most from native state to pneumoperitoneum (14.6 ± 0.9 mm, p < 0.001). From native state to laparotomy comparatively, the displacement averaged 9.8 ± 1.2 mm (p < 0.001). With pneumoperitoneum, the breath-dependent liver motion was significantly reduced when compared to other modalities. Liver motion due to ventilation was 7.7 ± 0.6 mm during pneumoperitoneum, 13.9 ± 1.1 mm with laparotomy, and 13.5 ± 1.4 mm in the native state (p < 0.001 in all cases).

CONCLUSIONS

Ventilation and application of pneumoperitoneum caused significant changes in liver position. Liver motion was reduced but clearly measurable during pneumoperitoneum. Intraoperative guidance/navigation systems should therefore account for ventilation and intraoperative changes of liver position and peripheral deformation.

Defining digital surgery: a SAGES white paper

BACKGROUND

Digital surgery is a new paradigm within the surgical innovation space that is rapidly advancing and encompasses multiple areas.

METHODS

This white paper from the SAGES Digital Surgery Working Group outlines the scope of digital surgery, defines key terms, and analyzes the challenges and opportunities surrounding this disruptive technology.

RESULTS

In its simplest form, digital surgery inserts a computer interface between surgeon and patient. We divide the digital surgery space into the following elements: advanced visualization, enhanced instrumentation, data capture, data analytics with artificial intelligence/machine learning, connectivity via telepresence, and robotic surgical platforms. We will define each area, describe specific terminology, review current advances as well as discuss limitations and opportunities for future growth.

CONCLUSION

Digital Surgery will continue to evolve and has great potential to bring value to all levels of the healthcare system. The surgical community has an essential role in understanding, developing, and guiding this emerging field.

Clinical use of augmented reality, mixed reality, three-dimensional-navigation and artificial intelligence in liver surgery

A precise knowledge of intra-parenchymal vascular and biliary architecture and the location of lesions in relation to the complex anatomy is indispensable to perform liver surgery. Therefore, virtual three-dimensional (3D)-reconstruction models from computed tomography/magnetic resonance imaging scans of the liver might be helpful for visualization. Augmented reality, mixed reality and 3D-navigation could transfer such 3D-image data directly into the operation theater to support the surgeon. This review examines the literature about the clinical and intraoperative use of these image guidance techniques in liver surgery and provides the reader with the opportunity to learn about these techniques. Augmented reality and mixed reality have been shown to be feasible for the use in open and minimally invasive liver surgery. 3D-navigation facilitated targeting of intraparenchymal lesions. The existing data is limited to small cohorts and description about technical details e.g., accordance between the virtual 3D-model and the real liver anatomy. Randomized controlled trials regarding clinical data or oncological outcome are not available. Up to now there is no intraoperative application of artificial intelligence in liver surgery. The usability of all these sophisticated image guidance tools has still not reached the grade of immersion which would be necessary for a widespread use in the daily surgical routine. Although there are many challenges, augmented reality, mixed reality, 3D-navigation and artificial intelligence are emerging fields in hepato-biliary surgery.

CBCT-based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi-rigid organ approximation and electromagnetic tracking of the liver

Purpose

The surgical navigation system that provides guidance throughout the surgery can facilitate safer and more radical liver resections, but such a system should also be able to handle organ motion. This work investigates the accuracy of intraoperative surgical guidance during open liver resection, with a semi‐rigid organ approximation and electromagnetic tracking of the target area.

Methods

The suggested navigation technique incorporates a preoperative 3D liver model based on diagnostic 4D MRI scan, intraoperative contrast‐enhanced CBCT imaging and electromagnetic (EM) tracking of the liver surface, as well as surgical instruments, by means of six degrees‐of‐freedom micro‐EM sensors.

Results

The system was evaluated during surgeries with 35 patients and resulted in an accurate and intuitive real‐time visualization of liver anatomy and tumor's location, confirmed by intraoperative checks on visible anatomical landmarks. Based on accuracy measurements verified by intraoperative CBCT, the system’s average accuracy was 4.0 ± 3.0 mm, while the total surgical delay due to navigation stayed below 20 min.

Conclusions

The electromagnetic navigation system for open liver surgery developed in this work allows for accurate localization of liver lesions and critical anatomical structures surrounding the resection area, even when the liver was manipulated. However, further clinically integrating the method requires shortening the guidance‐related surgical delay, which can be achieved by shifting to faster intraoperative imaging like ultrasound. Our approach is adaptable to navigation on other mobile and deformable organs, and therefore may benefit various clinical applications.

[Navigated liver surgery : Current state and importance in the future]

The preoperative computer-assisted resection planning is the basis for every navigation. Thanks to modern algorithms, the prerequisites have been created to carry out a virtual resection planning and a risk analysis. Thus, individual segment resections can be precisely planned in any conceivable combination. The transfer of planning information and resection suggestions to the operating theater is still problematic. The so-called stereotactic liver navigation supports the exact intraoperative implementation of the planned resection strategy and provides the surgeon with real-time three-dimensional information on resection margins and critical structures during the resection. This is made possible by a surgical navigation system that measures the position of surgical instruments and then presents them together with the preoperative surgical planning data. Although surgical navigation systems have been indispensable in neurosurgery and spinal surgery for many years, these procedures have not yet become established as standard in liver surgery. This is mainly due to the technical challenge of navigating a moving organ. As the liver is constantly moving and deforming during surgery due to respiration and surgical manipulation, the surgical navigation system must be able to measure these alterations in order to adapt the preoperative navigation data to the current situation. Despite these advances, further developments are required until navigated liver resection enters clinical routine; however, it is already clear that laparoscopic liver surgery and robotic surgery will benefit most from navigation technology.

Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery

PURPOSE

A key component of computer- assisted surgery systems is the accurate and robust registration of preoperative planning data with intraoperative sensor data. In laparoscopic surgery, this image-based registration remains challenging due to soft tissue deformations. This paper presents a novel approach for biomechanical soft tissue registration of preoperative CT data with stereo endoscopic image data.

METHODS

The proposed method consists of two registrations steps. First, we use a 3D surface mosaic from partial surfaces reconstructed from stereo endoscopic images to initially align the biomechanical model with the intraoperative position and shape of the organ. After this initialization, the biomechanical model is projected onto newly captured surfaces, resulting in displacement boundary conditions, which in turn are used to update the biomechanical model.

RESULTS

The method is evaluated in silico, using a human liver model, and in vivo, using porcine data. The quantitative in silico data shows a stable behaviour of the biomechanical model and root-mean-square deviation of volume vertices of under 3 mm with adjusted biomechanical parameters.

CONCLUSION

This work contributes a fully automatic featureless non-rigid registration approach. The results of the in silico and in vivo experiments suggest that our method is able to handle dynamic deformations during surgery. Additional experiments, especially regarding human tissue behaviour, are an important next step towards clinical applications.

Three-Dimensional Liver Surgery Simulation: Computer-Assisted Surgical Planning with Three-Dimensional Simulation Software and Three-Dimensional Printing<sup/>

To perform accurate hepatectomy without injury, it is necessary to understand the anatomical relationship among the branches of Glisson's sheath, hepatic veins, and tumor. In Japan, three-dimensional (3D) preoperative simulation for liver surgery is becoming increasingly common, and liver 3D modeling and 3D hepatectomy simulation by 3D analysis software for liver surgery have been covered by universal healthcare insurance since 2012. Herein, we review the history of virtual hepatectomy using computer-assisted surgery (CAS) and our research to date, and we discuss the future prospects of CAS. We have used the SYNAPSE VINCENT medical imaging system (Fujifilm Medical, Tokyo, Japan) for 3D visualization and virtual resection of the liver since 2010. We developed a novel fusion imaging technique combining 3D computed tomography (CT) with magnetic resonance imaging (MRI). The fusion image enables us to easily visualize anatomic relationships among the hepatic arteries, portal veins, bile duct, and tumor in the hepatic hilum. In 2013, we developed an original software, called Liversim, which enables real-time deformation of the liver using physical simulation, and a randomized control trial has recently been conducted to evaluate the use of Liversim and SYNAPSE VINCENT for preoperative simulation and planning. Furthermore, we developed a novel hollow 3D-printed liver model whose surface is covered with frames. This model is useful for safe liver resection, has better visibility, and the production cost is reduced to one-third of a previous model. Preoperative simulation and navigation with CAS in liver resection are expected to help planning and conducting a surgery and surgical education. Thus, a novel CAS system will contribute to not only the performance of reliable hepatectomy but also to surgical education.

Laparoscopic image-based navigation for microwave ablation of liver tumors-A multi-center study

BACKGROUND

Stereotactic navigation technology has been proposed to augment accuracy in targeting intrahepatic lesions for local ablation therapy. This retrospective study evaluated accuracy, efficacy, and safety when using laparoscopic image-guided microwave ablation (LIMA) for malignant liver tumors.

METHODS

All patients treated for malignant liver lesions using LIMA at two European centers between 2013 and 2015 were included for analysis. A landmark-based registration technique was applied for intraoperative tumor localization and positioning of ablation probes. Intraoperative efficiency of the procedure was measured as number of registration attempts and time needed to achieve sufficient registration accuracy. Technical accuracy was assessed as Fiducial Registration Error (FRE). Outcome at 90 days including mortality, postoperative morbidity, rates of incomplete ablations, and early intrahepatic recurrences were reported.

RESULTS

In 34 months, 54 interventions were performed comprising a total of 346 lesions (median lesions per patient 3 (1-25)). Eleven patients had concomitant laparoscopic resections of the liver or the colorectal primary tumor. Median time for registration was 4:38 min (0:26-19:34). Average FRE was 8.1 ± 2.8 mm. Follow-up at 90 days showed one death, 24% grade I/II, and 4% grade IIIa complications. Median length of hospital stay was 2 days (1-11). Early local recurrence was 9% per lesion and 32% per patient. Of these, 63% were successfully re-ablated within 6 months.

CONCLUSIONS

LIMA does not interfere with the intraoperative workflow and results in low complication and early local recurrence rates, even when simultaneously targeting multiple lesions. LIMA may represent a valid therapy option for patients with extensive hepatic disease within a multimodal treatment approach.

Stereotactic navigation for TAMIS-TME

Stereotactic navigation allows for real-time, image-guided surgery, thus providing an augmented working environment for the operator. This technique can be applied to complex minimally invasive surgery for fixed anatomic targets. Transanal minimally invasive surgery represents a new approach to rectal cancer surgery that is technically demanding and introduces the potential for procedure-specific morbidity. Feasibility of stereotactic navigation for TAMIS-TME has been demonstrated, and this could theoretically translate into improved resection quality by improving the surgeon's spatial awareness. The future of minimally invasive surgery as it relates to augmented reality and image-guided surgery is discussed.

Intraoperative image-guided navigation system: development and applicability in 65 patients undergoing liver surgery

BACKGROUND

Image-guided systems have recently been introduced for their application in liver surgery. We aimed to identify and propose suitable indications for image-guided navigation systems in the domain of open oncologic liver surgery and, more specifically, in the setting of liver resection with and without microwave ablation.

METHOD

Retrospective analysis was conducted in patients undergoing liver resection with and without microwave ablation using an intraoperative image-guided stereotactic system during three stages of technological development (accuracy: 8.4 ± 4.4 mm in phase I and 8.4 ± 6.5 mm in phase II versus 4.5 ± 3.6 mm in phase III). It was evaluated, in which indications image-guided surgery was used according to the different stages of technical development.

RESULTS

Between 2009 and 2013, 65 patients underwent image-guided surgical treatment, resection alone (n = 38), ablation alone (n = 11), or a combination thereof (n = 16). With increasing accuracy of the system, image guidance was progressively used for atypical resections and combined microwave ablation and resection instead of formal liver resection (p < 0.0001).

CONCLUSION

Clinical application of image guidance is feasible, while its efficacy is subject to accuracy. The concept of image guidance has been shown to be increasingly efficient for selected indications in liver surgery. While accuracy of available technology is increasing pertaining to technological advancements, more and more previously untreatable scenarios such as multiple small, bilobar lesions and so-called vanishing lesions come within reach.

A clinically applicable laser-based image-guided system for laparoscopic liver procedures

PURPOSE

Laser range scanners (LRS) allow performing a surface scan without physical contact with the organ, yielding higher registration accuracy for image-guided surgery (IGS) systems. However, the use of LRS-based registration in laparoscopic liver surgery is still limited because current solutions are composed of expensive and bulky equipment which can hardly be integrated in a surgical scenario.

METHODS

In this work, we present a novel LRS-based IGS system for laparoscopic liver procedures. A triangulation process is formulated to compute the 3D coordinates of laser points by using the existing IGS system tracking devices. This allows the use of a compact and cost-effective LRS and therefore facilitates the integration into the laparoscopic setup. The 3D laser points are then reconstructed into a surface to register to the preoperative liver model using a multi-level registration process.

RESULTS

Experimental results show that the proposed system provides submillimeter scanning precision and accuracy comparable to those reported in the literature. Further quantitative analysis shows that the proposed system is able to achieve a patient-to-image registration accuracy, described as target registration error, of [Formula: see text].

CONCLUSIONS

We believe that the presented approach will lead to a faster integration of LRS-based registration techniques in the surgical environment. Further studies will focus on optimizing scanning time and on the respiratory motion compensation.

Image-guided real-time navigation for transanal total mesorectal excision: a pilot study

BACKGROUND

Frameless stereotaxy for real-time, image-guided surgery has been most utilized for neurological and orthopedic surgery. Recently, our center has reported the application of real-time navigation for transanal total mesorectal excision.

METHODS

During a 5-month period (June 2013-October 2013), three male patients underwent transanal minimally invasive surgery for total mesorectal excision with image-guided real-time navigation during the transanal portion of the operation. This was completed using a frameless stereotactic navigational system as shown in a demonstration video. Male patients with anterior, locally advanced rectal cancer were selected for enrollment into the pilot study.

RESULTS

Three male patients (mean age 69) underwent transanal total mesorectal excision with stereotactic navigation during a 5-month study period. Mean operative time was 402 min, and there were no intra-operative complications recorded. The mean distance from anal verge of the tumor was 6.3 cm (range 4-8 cm). The navigational accuracy was computed to be ±3.69 mm (range ±3.20 to ±4.02 mm). The average navigation setup time was 47 min, not including scan time. The surgical specimens were found to have completely intact mesorectal envelopes (Quirke 3) in all cases. All margins, including radial and distal margins, were negative. Mean postoperative length of stay was 5 days. At a median of 18-month follow-up, there was no evidence of locoregional recurrence or distant metastatic disease.

CONCLUSION

This is the first pilot series to report the use of frameless stereotactic navigation for TAMIS-TME. Stereotactic navigation for transanal total mesorectal excision is shown to be feasible, and may aid in providing colorectal surgeons with the ability to better perform safe, high-quality surgery in select cases.

Navigation surgery using an augmented reality for pancreatectomy

AIM

The aim of this study was to evaluate the utility of navigation surgery using augmented reality technology (AR-based NS) for pancreatectomy.

METHODS

The 3D reconstructed images from CT were created by segmentation. The initial registration was performed by using the optical location sensor. The reconstructed images were superimposed onto the real organs in the monitor display. Of the 19 patients who had undergone hepatobiliary and pancreatic surgery using AR-based NS, the accuracy, visualization ability, and utility of our system were assessed in five cases with pancreatectomy.

RESULTS

The position of each organ in the surface-rendering image corresponded almost to that of the actual organ. Reference to the display image allowed for safe dissection while preserving the adjacent vessels or organs. The locations of the lesions and resection line on the targeted organ were overlaid on the operating field. The initial mean registration error was improved to approximately 5 mm by our refinements. However, several problems such as registration accuracy, portability and cost still remain.

CONCLUSION

AR-based NS contributed to accurate and effective surgical resection in pancreatectomy. The pancreas appears to be a suitable organ for further investigations. This technology is promising to improve surgical quality, training, and education.

Clinical application of navigation surgery using augmented reality in the abdominal field

This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.

Liver deformation in an animal model due to pneumoperitoneum assessed by a vessel-based deformable registration

PURPOSE

Surgical navigation based on preoperative images partly overcomes some of the drawbacks of minimally invasive interventions - reduction of free sight, lack of dexterity and tactile feedback. The usefulness of preoperative images is limited in laparoscopic liver surgery, as the liver shifts due to respiration, induction of pneumoperitoneum and surgical manipulation. In this study, we evaluated the shift and deformation in an animal liver caused by respiration and pneumopertioneum using intraoperative cone beam CT.

MATERIAL AND METHODS

3D cone beam CT scans were acquired with arterial contrast. The centerlines of the segmented vessels were extracted from the images taken at different respiration and pressure settings. A non-rigid registration method was used to measure the shift and deformation. The mean Euclidean distance between the annotated landmarks was used for evaluation.

RESULTS

A shift and deformation of 44.6 mm on average was introduced due to the combined effect of respiration and pneumoperitoneum. On average 91% of the deformations caused by the respiration and pneumoperitoneum were recovered.

CONCLUSION

The results can contribute to the use of intraoperative imaging to correct for anatomic shift so that preoperative data can be used with greater confidence and accuracy during guidance of laparoscopic liver procedures.

Utility of augmented reality system in hepatobiliary surgery

BACKGROUND/PURPOSE

The aim of this study was to evaluate the utility of an image display system for augmented reality in hepatobiliary surgery under laparotomy.

METHODS

An overlay display of organs, vessels, or tumor was obtained using a video see-through system as a display system developed at our institute. Registration between visceral organs and the surface-rendering image reconstructed by preoperative computed tomography (CT) was carried out with an optical location sensor. Using this system, we performed laparotomy for a patient with benign biliary stricture, a patient with gallbladder carcinoma, and a patient with hepatocellular carcinoma.

RESULTS

The operative procedures performed consisted of choledochojejunostomy, right hepatectomy, and microwave coagulation therapy. All the operations were carried out safely using images of the site of tumor, preserved organs, and resection aspect overlaid onto the operation field images observed on the monitors. The position of each organ in the overlaid image closely corresponded with that of the actual organ. Intraoperative information generated from this system provided us with useful navigation. However, several problems such as registration error and lack of depth knowledge were noted.

CONCLUSION

The image display system appeared to be useful in performing hepatobiliary surgery under laparotomy. Further improvement of the system with individualized function for each operation will be essential, with feedback from clinical trials in the future.

Virtual liver resection: computer-assisted operation planning using a three-dimensional liver representation

In liver surgery, understanding the complicated liver structures and a detailed evaluation of the functional liver remnant volume are essential to perform safe surgical procedures. Recent advances in imaging technology have enabled operation planning using three-dimensional (3D) image-processing software. Virtual liver resection systems provide (1) 3D imaging of liver structures, (2) detailed volumetric analyses based on portal perfusion, and (3) quantitative estimates of the venous drainage area, enabling the investigation of uncharted fields that cannot be examined using a conventional two-dimensional modality. The next step in computer-assisted liver surgery is the application of a virtual hepatectomy to real-time operations. However, the need for a precise alignment between the preoperative imaging data and the intraoperative situation remains to be adequately addressed, since the liver is subject to deformation and respiratory movements during the surgical procedures. We expect that the practical application of a navigation system for transferring the preoperative planning to real-time operations could make liver surgery safer and more standardized in the near future.

Evolution of image-guided liver surgery: transition from open to laparoscopic procedures

INTRODUCTION

Indications for liver surgery to treat primary and secondary hepatic malignancies are broadening. Utilizing data from B-mode or 2-D intraoperative ultrasound, it is often challenging to replicate the findings from preoperative CT or MRI scans. Additional data from more recently developed image-guidance technology, which registers preoperative axial imaging to a 3-D real-time model, may be used to improve operative planning, locate difficult to find hepatic tumors, and guide ablations.

METHODS

Laparoscopic liver procedures are often more challenging than their open counterparts. Image-guidance technology can assist in overcoming some of the obstacles to minimally invasive liver procedures by enhancing ultrasound findings and ablation guidance. This manuscript describes a protocol that evaluated an open image-guidance system, and a subsequent protocol that directly compared, for validation, a laparoscopic with an open image-guidance system. Both protocols were limited to ablations within the liver.

DISCUSSION

The laparoscopic image-guidance system successfully creates a 3-D model at both 7 and 14 mm Hg that is similar to the open 3-D model. Ultimately, improving intraoperative image guidance can help expand the ability to perform both laparoscopic and open liver surgeries.

Image-Guided Abdominal Surgery and Therapy Delivery

Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.

Organ surface deformation measurement and analysis in open hepatic surgery: method and preliminary results from 12 clinical cases

The incidence of soft tissue deformation has been well documented in neurosurgical procedures and is known to compromise the spatial accuracy of image-guided surgery systems.Within the context of image-guided liver surgery (IGLS), no detailed method to study and analyze the observed organ shape change between preoperative imaging and the intra-operative presentation has been developed. Contrary to the studies of deformation in neurosurgical procedures, the majority of deformation in IGLS is imposed prior to resection and due to laparotomy and mobilization. As such, methods of analyzing the organ shape change must be developed to use the intra-operative data (e.g. laser range scan (LRS) surfaces) acquired with the organ in its fully deformed shape. To achieve this end we use a signed closest point distance deformation metric computed after rigid alignment of the intra-operative LRS data with organ surfaces generated from the preoperative tomograms. The rigid alignment between the intra-operative LRS surfaces and pre-operative image data was computed with a feature weighted surface registration algorithm. In order to compare the deformation metrics across patients, an inter-patient non-rigid registration of the pre-operative CT images was performed. Given the inter-patient liver registrations, an analysis was performed to determine the potential similarities in the distribution of measured deformation between patients for which similar procedures had been performed. The results of the deformation measurement and analysis indicates the potential for soft tissue deformation to compromise surgical guidance information and suggests a similarity in imposed deformation among similar procedure types.

Novel 3-D laparoscopic magnetic ultrasound image guidance for lesion targeting

OBJECTIVES

Accurate laparoscopic liver lesion targeting for biopsy or ablation depends on the ability to merge laparoscopic and ultrasound images with proprioceptive instrument positioning, a skill that can be acquired only through extensive experience. The aim of this study was to determine whether using magnetic positional tracking to provide three-dimensional, real-time guidance improves accuracy during laparoscopic needle placement.

METHODS

Magnetic sensors were embedded into a needle and laparoscopic ultrasound transducer. These sensors interrupted the magnetic fields produced by an electromagnetic field generator, allowing for real-time, 3-D guidance on a stereoscopic monitor. Targets measuring 5 mm were embedded 3-5 cm deep in agar and placed inside a laparoscopic trainer box. Two novices (a college student and an intern) and two experts (hepatopancreatobiliary surgeons) targeted the lesions out of the ultrasound plane using either traditional or 3-D guidance.

RESULTS

Each subject targeted 22 lesions, 11 with traditional and 11 with the novel guidance (n= 88). Hit rates of 32% (14/44) and 100% (44/44) were observed with the traditional approach and the 3-D magnetic guidance approach, respectively. The novices were essentially unable to hit the targets using the traditional approach, but did not miss using the novel system. The hit rate of experts improved from 59% (13/22) to 100% (22/22) (P < 0.0001).

CONCLUSIONS

The novel magnetic 3-D laparoscopic ultrasound guidance results in perfect targeting of 5-mm lesions, even by surgical novices.

A navigation system for open liver surgery: design, workflow and first clinical applications

BACKGROUND

The surgical treatment of liver tumours relies on precise localization of the lesions and detailed knowledge of the patient-specific vascular and biliary anatomy. Detailed three-dimensional (3D) anatomical information facilitates complete tumour removal while preserving a sufficient amount of functional liver tissue.

METHODS

We present an easy to use, clinically applicable navigation system for efficient visualization and tool guidance during liver surgery. Accurate instrument guidance within 3D planning models was achieved with a fast registration procedure, assuming a locally rigid and temporarily static scenario. After deformations occurring during the procedure, efficient means for registration updates are provided. Special focus was given to workflow integration and the minimization of overhead time. The navigation system was validated with nine clinical cases.

RESULTS

Navigated surgical interventions were performed with a median time overhead of 16.5 min. The navigation technology had a median accuracy of 6.3 mm, improving anatomical orientation and the detection of structures at risk.

CONCLUSIONS

Successful application of the navigation technology to open liver surgery was achieved by minimizing the procedural complexity and optimizing integration within the existing surgical environment. The assumption of locally rigid patient registration was validated, and clinical evaluation shows clear benefits for the surgeon.

Upgrade of an optical navigation system with a permanent electromagnetic position control: a first step towards "navigated control" for liver surgery

INTRODUCTION

The main problems of navigation in liver surgery are organ movement and deformation. With a combination of direct optical and indirect electromagnetic tracking technology, visualisation and positional control of surgical instruments within three-dimensional ultrasound data and registration of organ movements can be realised simultaneously.

METHODS

Surgical instruments for liver resection were localised with an infrared-based navigation system (Polaris). Movements of the organ itself were registered using an electromagnetic navigation system (Aurora). The combination of these two navigation techniques and a new surgical navigation procedure focussed on a circumscribed critical dissection area were applied for the first time in liver resections.

RESULTS

This new technique was effectively implemented. The position of the surgical instrument was localised continuously. Repeated position control with observation of the navigation screen was not necessary. During surgical resection, a sonic warning signal was activated when the surgical instrument entered a "no touch" area--an area of reduced safety margin.

CONCLUSION

Optical tracking of surgical instruments and simultaneous electromagnetic registration of organ position is feasible in liver resection.

Laparoscopic liver resection for cancer

The field of laparoscopic liver resection surgery has rapidly evolved, with more than 1000 cases now reported. Laparoscopic hepatic resection was initially described for small, peripheral, benign lesions. Experienced teams are now performing laparoscopic anatomic resections for cancer. Operative times improved with experience. When compared with open cases, blood loss was less in most laparoscopic series, but was the main indication for conversion to an open procedure. Patients undergoing laparoscopic resection had shorter length of hospital stay and quicker recovery. Perioperative complications were comparable between the two approaches. Importantly, basic oncologic principles were maintained in the laparoscopic liver resections. The purpose of this review is to summarize the data available on outcomes for laparoscopic hepatic resection for cancer. This includes primary hepatocellular carcinoma, as well as metastatic colorectal cancer to the liver. The evidence to date suggests that laparoscopic results are comparable with the open approach in cancer patients.

Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation

A successful surface-based image-to-physical space registration in image-guided liver surgery (IGLS) is critical to provide reliable guidance information to surgeons and pertinent surface displacement data for use in deformation correction algorithms. The current protocol used to perform the image-to-physical space registration involves an initial pose estimation provided by a point based registration of anatomical landmarks identifiable in both the preoperative tomograms and the intraoperative presentation. The surface based registration is then performed via a traditional iterative closest point (ICP) algorithm between the preoperative liver surface, segmented from the tomographic image set, and an intraoperatively acquired point cloud of the liver surface provided by a laser range scanner. Using this more conventional method, the registration accuracy can be compromised by poor initial pose estimation as well as tissue deformation due to the laparotomy and liver mobilization performed prior to tumor resection. In order to increase the robustness of the current surface-based registration method used in IGLS, we propose the incorporation of salient anatomical features, identifiable in both the preoperative image sets and intraoperative liver surface data, to aid in the initial pose estimation and play a more significant role in the surface-based registration via a novel weighting scheme. Examples of such salient anatomical features are the falciform groove region as well as the inferior ridge of the liver surface. In order to validate the proposed weighted patch registration method, the alignment results provided by the proposed algorithm using both single and multiple patch regions were compared with the traditional ICP method using six clinical datasets. Robustness studies were also performed using both phantom and clinical data to compare the resulting registrations provided by the proposed algorithm and the traditional method under conditions of varying initial pose. The results provided by the robustness trials and clinical registration comparisons suggest that the proposed weighted patch registration algorithm provides a more robust method with which to perform the image-to-physical space registration in IGLS. Furthermore, the implementation of the proposed algorithm during surgical procedures does not impose significant increases in computation or data acquisition times.

Navigation in laparoscopy--prototype research platform for improved image-guided surgery

The manipulation of the surgical field in laparoscopic surgery, through small incisions with rigid instruments, reduces free sight, dexterity, and tactile feedback. To help overcome some of these drawbacks, we present a prototype research and development platform, CustusX, for navigation in minimally invasive therapy. The system can also be used for planning and follow-up studies. With this platform we can import and display a range of medical images, also real-time data such as ultrasound and X-ray, during surgery. Tracked surgical tools, such as pointers, video laparoscopes, graspers, and various probes, allow surgeons to interactively control the display of medical images during the procedure. This paper introduces navigation technologies and methods for laparoscopic therapy, and presents our software and hardware research platform. Furthermore, we illustrate the use of the system with examples from two pilots performed during laparoscopic therapy. We also present new developments that are currently being integrated into the system for future use in the operating room. Our initial results from pilot studies using this technology with preoperative images and guidance in the retroperitoneum during laparoscopy are promising. Finally, we shortly describe an ongoing multicenter study using this surgical navigation system platform.

Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection

In recent years, an increasing number of liver tumor indications were treated by minimally invasive laparoscopic resection. Besides the restricted view, two major intraoperative issues in laparoscopic liver resection are the optimal planning of ports as well as the enhanced visualization of (hidden) vessels, which supply the tumorous liver segment and thus need to be divided (e.g., clipped) prior to the resection. We propose an intuitive and precise method to plan the placement of ports. Preoperatively, self-adhesive fiducials are affixed to the patient's skin and a computed tomography (CT) data set is acquired while contrasting the liver vessels. Immediately prior to the intervention, the laparoscope is moved around these fiducials, which are automatically reconstructed to register the patient to its preoperative imaging data set. This enables the simulation of a camera flight through the patient's interior along the laparoscope's or instruments' axes to easily validate potential ports. Intraoperatively, surgeons need to update their surgical planning based on actual patient data after organ deformations mainly caused by application of carbon dioxide pneumoperitoneum. Therefore, preoperative imaging data can hardly be used. Instead, we propose to use an optically tracked mobile C-arm providing cone-beam CT imaging capability intraoperatively. After patient positioning, port placement, and carbon dioxide insufflation, the liver vessels are contrasted and a 3-D volume is reconstructed during patient exhalation. Without any further need for patient registration, the reconstructed volume can be directly augmented on the live laparoscope video, since prior calibration enables both the volume and the laparoscope to be positioned and oriented in the tracking coordinate frame. The augmentation provides the surgeon with advanced visual aid for the localization of veins, arteries, and bile ducts to be divided or sealed.

Laparoscopic surface scanning and subsurface targeting: Implications for image-guided laparoscopic liver surgery

Segmental liver resection and locoregional ablative therapies are dependent upon accurate tumor localization to ensure safety as well as acceptable oncologic results. Because of the liver's limited external landmarks and complex internal anatomy, such tumor localization poses a technical challenge. Image guided therapies (IGT) address this problem by mapping the real-time, intraoperative position of surgical instruments onto preoperative tomographic imaging through a process called registration. Accuracy is critical to IGT and is a function of: 1) the registration technique, 2) the tissue characteristics, and 3) imaging techniques. The purpose of this study is to validate a novel method of registration using an endoscopic Laser Range Scanner (eLRS) and demonstrate its applicability to laparoscopic liver surgery. Six radiopaque targets were inserted into an ex-vivo bovine liver and a computed tomography (CT) scan was obtained. Using the eLRS, the liver surface was scanned and a surface-based registration was constructed to predict the position of the intraparenchymal targets. The target registration error (TRE) achieved using our surface-based registration was 2.4 +/- 1.0 mm. A comparable TRE using traditional fiducial-based registration was 2.6 +/- 1.7 mm. Compared to traditional fiducial-based registration, laparoscopic surface scanning is able to predict the location of intraparenchymal liver targets with similar accuracy and rate of data acquisition.

Concepts and preliminary data toward the realization of image-guided liver surgery

Image-guided surgery provides navigational assistance to the surgeon by displaying the surgical probe position on a set of preoperative tomograms in real time. In this study, the feasibility of implementing image-guided surgery concepts into liver surgery was examined during eight hepatic resection procedures. Preoperative tomographic image data were acquired and processed. Accompanying intraoperative data on liver shape and position were obtained through optically tracked probes and laser range scanning technology. The preoperative and intraoperative representations of the liver surface were aligned using the iterative closest point surface matching algorithm. Surface registrations resulted in mean residual errors from 2 to 6 mm, with errors of target surface regions being below a stated goal of 1 cm. Issues affecting registration accuracy include liver motion due to respiration, the quality of the intraoperative surface data, and intraoperative organ deformation. Respiratory motion was quantified during the procedures as cyclical, primarily along the cranial-caudal direction. The resulting registrations were more robust and accurate when using laser range scanning to rapidly acquire thousands of points on the liver surface and when capturing unique geometric regions on the liver surface, such as the inferior edge. Finally, finite element models recovered much of the observed intraoperative deformation, further decreasing errors in the registration. Image-guided liver surgery has shown the potential to provide surgeons with important navigation aids that could increase the accuracy of targeting lesions and the number of patients eligible for surgical resection.

Image-guidance for surgical procedures

Contemporary imaging modalities can now provide the surgeon with high quality three- and four-dimensional images depicting not only normal anatomy and pathology, but also vascularity and function. A key component of image-guided surgery (IGS) is the ability to register multi-modal pre-operative images to each other and to the patient. The other important component of IGS is the ability to track instruments in real time during the procedure and to display them as part of a realistic model of the operative volume. Stereoscopic, virtual- and augmented-reality techniques have been implemented to enhance the visualization and guidance process. For the most part, IGS relies on the assumption that the pre-operatively acquired images used to guide the surgery accurately represent the morphology of the tissue during the procedure. This assumption may not necessarily be valid, and so intra-operative real-time imaging using interventional MRI, ultrasound, video and electrophysiological recordings are often employed to ameliorate this situation. Although IGS is now in extensive routine clinical use in neurosurgery and is gaining ground in other surgical disciplines, there remain many drawbacks that must be overcome before it can be employed in more general minimally-invasive procedures. This review overviews the roots of IGS in neurosurgery, provides examples of its use outside the brain, discusses the infrastructure required for successful implementation of IGS approaches and outlines the challenges that must be overcome for IGS to advance further.

Application of soft tissue modelling to image-guided surgery

The deformation of soft tissue compromises the accuracy of image-guided surgery based on preoperative images, and restricts its applicability to surgery on or near bony structures. One way to overcome these limitations is to combine biomechanical models with sparse intraoperative data, in order to realistically warp the preoperative image to match the surgical situation. We detail the process of biomechanical modelling in the context of image-guided surgery. We focus in particular on the finite element method, which is shown to be a promising approach, and review the constitutive relationships which have been suggested for representing tissue during surgery. Appropriate intraoperative measurements are required to constrain the deformation, and we discuss the potential of the modalities which have been applied to this task. This technology is on the verge of transition into clinical practice, where it promises to increase the guidance accuracy and facilitate less invasive interventions. We describe here how soft tissue modelling techniques have been applied to image-guided surgery applications.

Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery

In this paper, we present a novel method for intra-operative registration directly from monocular endoscopic images. This technique has the potential to provide a more accurate surface registration at the surgical site than existing methods. It can operate autonomously from as few as two images and can be particularly useful in revision cases where surgical landmarks may be absent. A by-product of video registration is an estimate of the local surface structure of the anatomy, thus providing the opportunity to dynamically update anatomical models as the surgery progresses. Our approach is based on a previously presented method [Burschka, D., Hager, G.D., 2004. V-GPS (SLAM):--Vision-based inertial system for mobile robots. In: Proceedings of ICRA, 409-415] for reconstruction of a scaled 3D model of the environment from unknown camera motion. We use this scaled reconstruction as input to a PCA-based algorithm that registers the reconstructed data to the CT data and recovers the scale and pose parameters of the camera in the coordinate frame of the CT scan. The result is used in an ICP registration step to refine the registration estimates. The details of our approach and the experimental results with a phantom of a human skull and a head of a pig cadaver are presented in this paper.

Ultrasound-to-computer-tomography registration for image-guided laparoscopic liver surgery

BACKGROUND

The application of image-guided surgery (IGS) to laparoscopic liver resection and ablation is currently limited, but it would assist in intraoperative decision making regarding oncologic margins, ablation probe placement, and ablation tracking.

METHODS

Eight spherical surface targets on a liver phantom were imaged with an optically tracked laparoscopic ultrasound (US) probe. Ten US images of each target were registered to computer tomography (CT) images of the phantoms and then mapped to the CT scans. Accuracy of the registration was assessed by comparing the distance between the predicted target location and the position obtained directly from CT.

RESULTS

The average localization error was 5.3 mm. The errors resulted primarily from inaccurate US probe tracking but were otherwise insensitive to the variability that arises from manually identifying targets in US and CT images.

CONCLUSIONS

The results obtained for US-to-CT registration in a phantom model suggest that further investigations into its clinical use are warranted and that other IGS technologies could be applied to laparoscopic liver surgery as well.

Tissue deformation and shape models in image-guided interventions: a discussion paper

This paper promotes the concept of active models in image-guided interventions. We outline the limitations of the rigid body assumption in image-guided interventions and describe how intraoperative imaging provides a rich source of information on spatial location of anatomical structures and therapy devices, allowing a preoperative plan to be updated during an intervention. Soft tissue deformation and variation from an atlas to a particular individual can both be determined using non-rigid registration. Established methods using free-form deformations have a very large number of degrees of freedom. Three examples of deformable models--motion models, biomechanical models and statistical shape models--are used to illustrate how prior information can be used to restrict the number of degrees of freedom of the registration algorithm and thus provide active models for image-guided interventions. We provide preliminary results from applications for each type of model.

Laparoscopic navigation pointer for three-dimensional image-guided surgery

BACKGROUND

The main drawback with the laparoscopic approach is that the surgeon is unable to palpate vessels, tumors, and organs during surgery. Furthermore, the laparoscope provides only surface view of organs. There is a need for more advanced visualizations that can enhance the view to include information below the surface of the organs for planning of the procedure and for control and guidance during treatment.

METHODS

We propose three-dimensional (3D) navigation technology based on preoperatively acquired magnetic resonance or computed tomography data used in combination with a laparoscopic navigation pointer (LNP). The LNP has an attached position tracker that allows the surgeon to control the display of images interactively before and during surgery. This study evaluated the patient registration accuracy, the feasibility of image-based navigation and, qualitatively, the navigation precision in the retroperitoneum during laparoscopic surgery.

RESULTS

This technology was used during the treatment of six patients (involving adrenalectomies and a neuroma protruding into the pelvis). An average patient registration accuracy of 6.90 mm was achieved. The precision during navigation in the retroperitoneum was, in some cases, better than the patient registration accuracy suggested. The technology helped the surgeons to understand better the anatomy and to locate blood vessels.

CONCLUSIONS

In the reported cases, the LNP was a useful tool for image guidance in laparoscopic surgery, both for planning the surgical approach in detail and for guidance. The authors believe that adominal 3D image guidance using an LNP has a large potential for improving laparoscopic surgery, especially when vessels and anatomic relations may be difficult to identify using only a laparoscope. Accordingly, they believe this new technology could increase safety and make it easier for the surgeon to perform successful laparoscopic surgery.

Design and implementation of a PC-based image-guided surgical system

In interactive, image-guided surgery, current physical space position in the operating room is displayed on various sets of medical images used for surgical navigation. We have developed a PC-based surgical guidance system (ORION) which synchronously displays surgical position on up to four image sets and updates them in real time. There are three essential components which must be developed for this system: (1) accurately tracked instruments; (2) accurate registration techniques to map physical space to image space; and (3) methods to display and update the image sets on a computer monitor. For each of these components, we have developed a set of dynamic link libraries in MS Visual C++ 6.0 supporting various hardware tools and software techniques. Surgical instruments are tracked in physical space using an active optical tracking system. Several of the different registration algorithms were developed with a library of robust math kernel functions, and the accuracy of all registration techniques was thoroughly investigated. Our display was developed using the Win32 API for windows management and tomographic visualization, a frame grabber for live video capture, and OpenGL for visualization of surface renderings. We have begun to use this current implementation of our system for several surgical procedures, including open and minimally invasive liver surgery.

Computer-aided hepatic tumour ablation: requirements and preliminary results

Surgical resection of hepatic tumours is not always possible, since it depends on different factors, among which their location inside the liver functional segments. Alternative techniques consist in local use of chemical or physical agents to destroy the tumour. Radio frequency and cryosurgical ablations are examples of such alternative techniques that may be performed percutaneously. This requires a precise localisation of the tumour placement during ablation. Computer-assisted surgery tools may be used in conjunction with these new ablation techniques to improve the therapeutic efficiency, whilst they benefit from minimal invasiveness. This paper introduces the principles of a system for computer-assisted hepatic tumour ablation and describes preliminary experiments focusing on data registration evaluation. To keep close to conventional protocols, we consider registration of pre-operative CT or MRI data to intra-operative echographic data.

Registration of physical space to laparoscopic image space for use in minimally invasive hepatic surgery

While laparoscopes are used for numerous minimally invasive (MI) procedures, MI liver resection and ablative surgery is infrequently performed. The paucity of cases is due to the restriction of the field of view by the laparoscope and the difficulty in determining tumor location and margins under video guidance. By merging MI surgery with interactive, image-guided surgery (IIGS), we hope to overcome localization difficulties present in laparoscopic liver procedures. One key component of any IIGS system is the development of accurate registration techniques to map image space to physical or patient space. This manuscript focuses on the accuracy and analysis of the direct linear transformation (DLT) method to register physical space with laparoscopic image space on both distorted and distortion-corrected video images. Experiments were conducted on a liver-sized plastic phantom affixed with 20 markers at various depths. After localizing the points in both physical and laparoscopic image space, registration accuracy was assessed for different combinations and numbers of control points (n) to determine the quantity necessary to develop a robust registration matrix. For n = 11, average target registration error (TRE) was 0.70 +/- 0.20 mm. We also studied the effects of distortion correction on registration accuracy. For the particular distortion correction method and laparoscope used in our experiments, there was no statistical significance between physical to image registration error for distorted and corrected images. In cases where a minimum number of control points (n = 6) are acquired, the DLT is often not stable and the mathematical process can lead to high TRE values. Mathematical filters developed through the analysis of the DLT were used to prospectively eliminate outlier cases where the TRE was high. For n = 6, prefilter average TRE was 17.4 +/- 153 mm for all trials; when the filters were applied, average TRE decreased to 1.64 +/- 1.10 mm for the remaining trials.