Real-time wireless tumor tracking during breast conserving surgery

3D magnetic seed localization for augmented reality in surgery

PURPOSE

For tumor resection, surgeons need to localize the tumor. For this purpose, a magnetic seed can be inserted into the tumor by a radiologist and, during surgery, a magnetic detection probe informs the distance to the seed for localization. In this case, the surgeon still needs to mentally reconstruct the position of the tumor from the probe's information. The purpose of this study is to develop and assess a method for 3D localization and visualization of the seed, facilitating the localization of the tumor.

METHODS

We propose a method for 3D localization of the magnetic seed by extending the magnetic detection probe with a tracking-based localization. We attach a position sensor (QR-code or optical marker) to the probe in order to track its 3D pose (respectively, using a head-mounted display with a camera or optical tracker). Following an acquisition protocol, the 3D probe tip and seed position are subsequently obtained by solving a system of equations based on the distances and the 3D probe poses.

RESULTS

The method was evaluated with an optical tracking system. An experimental setup using QR-code tracking (resp. using an optical marker) achieves an average of 1.6 mm (resp. 0.8 mm) 3D distance between the localized seed and the ground truth. Using a breast phantom setup, the average 3D distance is 4.7 mm with a QR-code and 2.1 mm with an optical marker.

CONCLUSION

Tracking the magnetic detection probe allows 3D localization of a magnetic seed, which opens doors for augmented reality target visualization during surgery. Such an approach should enhance the perception of the localized region of interest during the intervention, especially for breast tumor resection where magnetic seeds can already be used in the protocol.

An improved camera model for oblique-viewing laparoscopes: high reprojection accuracy independent of telescope rotation

Objective. Oblique-viewing laparoscopes are popular in laparoscopic surgeries where the target anatomy is located in narrow areas. Their viewing direction can be shifted by telescope rotation without changing the laparoscope pose. This rotation also changes laparoscope camera parameters that are estimated by camera calibration to be able to reproject an anatomical model onto the laparoscopic view, creating augmented reality (AR). The aim of this study was to develop a camera model that accounts for these changes, achieving high reprojection accuracy for any telescope rotation.Approach. Camera parameters were acquired by calibrations encompassing a wide telescope rotation range. For those parameters showing periodic changes upon rotation, interpolation models were created and used to establish an updatable camera model. With this model, corner points of a tracked checkerboard were reprojected onto the checkerboard laparoscopic images, at random rotation angles. Root-mean-square reprojection errors (RMSEs) were calculated between the reprojected and imaged corner points.Main results. Reprojection RMSEs were low and approximately independent on telescope rotation angle, over a wide rotation range of 320°. The mean reprojection RMSE was 2.8±0.7 pixels for a conventional laparoscope and 3.6±0.7 pixels for a chip-on-the-tip (COTT) laparoscope, corresponding to 0.3±0.1 mm and 0.4±0.1 mm in world coordinates respectively. Worst-case reprojection errors were about 9 pixels (0.8 mm) for both laparoscopes.Significance. The camera model developed in this study improves on existing models for oblique-viewing laparoscopes because it provides high reprojection accuracy independent of the telescope rotation angle and is applicable for conventional and chip-on-a-tip oblique-viewing laparoscopes. The work presented here is an important step towards creating accurate AR in image-guided interventions where oblique-viewing laparoscopes are used while simultaneously providing the surgeon the flexibility to rotate the telescope to any desired rotation angle.Acronyms. CC: camera coordinates; CCToolbox: camera calibration toolbox; COTT: chip-on-the-tip; CS: camera sensor; DD: decentering distortion; FL: focal length; OTS: optical tracking system; PP: principal point; RD: radial distortion; SI: supplementary information;tHE:hand-eye translation component.

Computational Imaging to Compensate for Soft-Tissue Deformations in Image-Guided Breast Conserving Surgery

OBJECTIVE

During breast conserving surgery (BCS), magnetic resonance (MR) images aligned to accurately display intraoperative lesion locations can offer improved understanding of tumor extent and position relative to breast anatomy. Unfortunately, even under consistent supine conditions, soft tissue deformation compromises image-to-physical alignment and results in positional errors.

METHODS

A finite element inverse modeling technique has been developed to nonrigidly register preoperative supine MR imaging data to the surgical scene for improved localization accuracy during surgery. Registration is driven using sparse data compatible with acquisition during BCS, including corresponding surface fiducials, sparse chest wall contours, and the intra-fiducial skin surface. Deformation predictions were evaluated at surface fiducial locations and subsurface tissue features that were expertly identified and tracked. Among n = 7 different human subjects, an average of 22 ± 3 distributed subsurface targets were analyzed in each breast volume.

RESULTS

The average target registration error (TRE) decreased significantly when comparing rigid registration to this nonrigid approach (10.4 ± 2.3 mm vs 6.3 ± 1.4 mm TRE, respectively). When including a single subsurface feature as additional input data, the TRE significantly improved further (4.2 ± 1.0 mm TRE), and in a region of interest within 15 mm of a mock biopsy clip TRE was 3.9 ± 0.9 mm.

CONCLUSION

These results demonstrate accurate breast deformation estimates based on sparse-data-driven model predictions.

SIGNIFICANCE

The data suggest that a computational imaging approach can account for image-to-surgery shape changes to enhance surgical guidance during BCS.

Bridging 3D Slicer and ROS2 for Image-Guided Robotic Interventions

Developing image-guided robotic systems requires access to flexible, open-source software. For image guidance, the open-source medical imaging platform 3D Slicer is one of the most adopted tools that can be used for research and prototyping. Similarly, for robotics, the open-source middleware suite robot operating system (ROS) is the standard development framework. In the past, there have been several "ad hoc" attempts made to bridge both tools; however, they are all reliant on middleware and custom interfaces. Additionally, none of these attempts have been successful in bridging access to the full suite of tools provided by ROS or 3D Slicer. Therefore, in this paper, we present the SlicerROS2 module, which was designed for the direct use of ROS2 packages and libraries within 3D Slicer. The module was developed to enable real-time visualization of robots, accommodate different robot configurations, and facilitate data transfer in both directions (between ROS and Slicer). We demonstrate the system on multiple robots with different configurations, evaluate the system performance and discuss an image-guided robotic intervention that can be prototyped with this module. This module can serve as a starting point for clinical system development that reduces the need for custom interfaces and time-intensive platform setup.

Real-Time Wireless Tumor Tracking in Navigated Liver Resections: An Ex Vivo Feasibility Study

BACKGROUND

Surgical navigation systems generally require intraoperative steps, such as intraoperative imaging and registration, to link the system to the patient anatomy. Because this hampers surgical workflow, we developed a plug-and-play wireless navigation system that does not require any intraoperative steps. In this ex vivo study on human hepatectomy specimens, the feasibility was assessed of using this navigation system to accurately resect a planned volume with small margins to the lesion.

METHODS

For ten hepatectomy specimens, a planning CT was acquired in which a virtual spherical lesion with 5 mm margin was delineated, inside the healthy parenchyma. Using two implanted trackers, the real-time position of this planned resection volume was visualized on a screen, relative to the used tracked pointer. Experienced liver surgeons were asked to accurately resect the nonpalpable planned volume, fully relying on the navigation screen. Resected and planned volumes were compared using CT.

RESULTS

The surgeons resected the planned volume while cutting along its border with a mean accuracy of - 0.1 ± 2.4 mm and resected 98 ± 12% of the planned volume. Nine out of ten resections were radical and one case showed a cut of 0.8 mm into the lesion. The sessions took approximately 10 min each, and no considerable technical issues were encountered.

CONCLUSIONS

This ex vivo liver study showed that it is feasible to accurately resect virtual hepatic lesions with small planned margins using our novel navigation system, which is promising for clinical applications where nonpalpable hepatic metastases have to be resected with small resection margins.

CBCT-based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi-rigid organ approximation and electromagnetic tracking of the liver

Purpose

The surgical navigation system that provides guidance throughout the surgery can facilitate safer and more radical liver resections, but such a system should also be able to handle organ motion. This work investigates the accuracy of intraoperative surgical guidance during open liver resection, with a semi‐rigid organ approximation and electromagnetic tracking of the target area.

Methods

The suggested navigation technique incorporates a preoperative 3D liver model based on diagnostic 4D MRI scan, intraoperative contrast‐enhanced CBCT imaging and electromagnetic (EM) tracking of the liver surface, as well as surgical instruments, by means of six degrees‐of‐freedom micro‐EM sensors.

Results

The system was evaluated during surgeries with 35 patients and resulted in an accurate and intuitive real‐time visualization of liver anatomy and tumor's location, confirmed by intraoperative checks on visible anatomical landmarks. Based on accuracy measurements verified by intraoperative CBCT, the system’s average accuracy was 4.0 ± 3.0 mm, while the total surgical delay due to navigation stayed below 20 min.

Conclusions

The electromagnetic navigation system for open liver surgery developed in this work allows for accurate localization of liver lesions and critical anatomical structures surrounding the resection area, even when the liver was manipulated. However, further clinically integrating the method requires shortening the guidance‐related surgical delay, which can be achieved by shifting to faster intraoperative imaging like ultrasound. Our approach is adaptable to navigation on other mobile and deformable organs, and therefore may benefit various clinical applications.

Accurate surgical navigation with real-time tumor tracking in cancer surgery

In the past decades, image-guided surgery has evolved rapidly. In procedures with a relatively fixed target area, like neurosurgery and orthopedics, this has led to improved patient outcomes. In cancer surgery, intraoperative guidance could be of great benefit to secure radical resection margins since residual disease is associated with local recurrence and poor survival. However, most tumor lesions are mobile with a constantly changing position. Here, we present an innovative technique for real-time tumor tracking in cancer surgery. In this study, we evaluated the feasibility of real-time tumor tracking during rectal cancer surgery. The application of real-time tumor tracking using an intraoperative navigation system is feasible and safe with a high median target registration accuracy of 3 mm. This technique allows oncological surgeons to obtain real-time accurate information on tumor location, as well as critical anatomical information. This study demonstrates that real-time tumor tracking is feasible and could potentially decrease positive resection margins and improve patient outcome.

Accuracy assessment of target tracking using two 5-degrees-of-freedom wireless transponders

Purpose

Surgical navigation systems are generally only applied for targets in rigid areas. For non-rigid areas, real-time tumor tracking can be included to compensate for anatomical changes. The only clinically cleared system using a wireless electromagnetic tracking technique is the Calypso^® System (Varian Medical Systems Inc., USA), designed for radiotherapy. It is limited to tracking maximally three wireless 5-degrees-of-freedom (DOF) transponders, all used for tumor tracking. For surgical navigation, a surgical tool has to be tracked as well. In this study, we evaluated whether accurate 6DOF tumor tracking is possible using only two 5DOF transponders, leaving one transponder to track a tool.

Methods

Two methods were defined to derive 6DOF information out of two 5DOF transponders. The first method uses the vector information of both transponders (TTV), and the second method combines the vector information of one transponder with the distance vector between the transponders (OTV). The accuracy of tracking a rotating object was assessed for each method mimicking clinically relevant and worst-case configurations. Accuracy was compared to using all three transponders to derive 6DOF (Default method). An optical tracking system was used as a reference for accuracy.

Results

The TTV method performed best and was as accurate as the Default method for almost all transponder configurations (median errors < 0.5°, 95% confidence interval < 3°). Only when the angle between the transponders was less than 2°, the TTV method was inaccurate and the OTV method may be preferred. The accuracy of both methods was independent of the angle of rotation, and only the OTV method was sensitive to the plane of rotation.

Conclusion

These results indicate that accurate 6DOF tumor tracking is possible using only two 5DOF transponders. This encourages further development of a wireless EM surgical navigation approach using a readily available clinical system.

[Modalities and advantages of image guided radiation therapy of breast cancer in adjuvant setting]

In adjuvant setting, breast cancer radiotherapy volumes include whole mammary gland or chest wall, and when indicated, nodal area such as axilla, supraclavicular, and internal mammary chain. An accurate patients positioning is required due to some geometric complexity of target volumes closed to organs at risk as heart and lung. Image guided radiation therapy allows such accuracy. Here we propose a review on image guided radiotherapy for breast cancer.

Accuracy assessment of wireless transponder tracking in the operating room environment

PURPOSE

To evaluate the applicability of the Calypso^® wireless transponder tracking system (Varian Medical Systems Inc., USA) for real-time tumor motion tracking during surgical procedures on tumors in non-rigid target areas. An accuracy assessment was performed for an extended electromagnetic field of view (FoV) of 27.5 × 27.5 × 22.5 cm (which included the standard FoV of 14 × 14 × 19 cm) in which 5DOF wireless Beacon^® transponders can be tracked.

METHODS

Using a custom-made measurement setup, we assessed single transponder relative accuracy, absolute accuracy and jitter throughout the extended FoV at 1440 locations interspaced with 2.5 cm in each orthogonal direction. The NDI Polaris Spectra optical tracking system (OTS) was used as a reference. Measurements were taken in a room without surrounding distorting factors and repeated in an operating room (OR). In the OR, the influence of a carbon fiber and regular stainless steel OR tabletop was investigated.

RESULTS

The calibration of the OTS and transponder system resulted in an average root-mean-square error (RMSE) vector of 0.03 cm. For both the standard and extended FoV, all accuracy measures were dependent on transponder to tracking array (TA) distances and the absolute accuracy was also dependent on TA to OR tabletop distances. This latter influence was reproducible, and after calibrating this, the residual error was below 0.1 cm RMSE within the entire standard FoV. Within the extended FoV, this residual RMSE did not exceed 0.1 cm for transponder to TA distances up to 25 cm.

CONCLUSION

This study shows that transponder tracking is promising for accurate tumor tracking in the operating room. This applies when using the standard FoV, but also when using the extended FoV up to 25 cm above the TA, substantially increasing flexibility.