A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery

Background: Three-dimensional reconstructions of state-of-the-art high-resolution imaging are progressively being used more for preprocedural assessment in thoracic surgery. It is a promising tool that aims to improve patient-specific treatment planning, for example, for minimally invasive or robotic-assisted lung resections. Increasingly available mixed-reality hardware based on video pass-through technology enables the projection of image data as a hologram onto the patient. We describe the novel method of real-time 3D surgical planning in a mixed-reality setting by presenting three representative cases utilizing volume rendering. Materials: A mixed-reality system was set up using a high-performance workstation running a video pass-through-based head-mounted display. Image data from computer tomography were imported and volume-rendered in real-time to be customized through live editing. The image-based hologram was projected onto the patient, highlighting the regions of interest. Results: Three oncological cases were selected to explore the potentials of the mixed-reality system. Two of them presented large tumor masses in the thoracic cavity, while a third case presented an unclear lesion of the chest wall. We aligned real-time rendered 3D holographic image data onto the patient allowing us to investigate the relationship between anatomical structures and their respective body position. Conclusions: The exploration of holographic overlay has proven to be promising in improving preprocedural surgical planning, particularly for complex oncological tasks in the thoracic surgical field. Further studies on outcome-related surgical planning and navigation should therefore be conducted. Ongoing technological progress of extended reality hardware and intelligent software features will most likely enhance applicability and the range of use in surgical fields within the near future.

[Robotic Assisted Proctocolectomy with Ileal Pouch-anal Anastomosis in Familial Adenomatous Polyposis - a Video Vignette]

Das kolorektale Karzinom (KRK) ist eines der häufigsten Malignome, bei dem in den letzten Jahren eine signifikante Zunahme hereditärer Fälle beobachtet werden konnte. Die zweithäufigste Ursache für ein hereditäres KRK ist die familiäre adenomatöse Polyposis, eine obligate Präkanzerose. Sinnvollster Therapieansatz ist eine prophylaktische laparoskopische Proktokolektomie mit Ileumpouch-analer Anastomose (IPAA) im jungen Erwachsenenalter. Mit der zunehmenden Etablierung robotischer Verfahren stellt sich die Frage, ob die Vorteile der robotischen Operationen, wie bspw. vereinfachtes Operieren und bessere Visualisierung in engen Räumen, insbesondere bei der prophylaktischen Proktokolektomie, sinnvoll sein kann. Problematisch ist allerdings die Notwendigkeit, in allen 4 Quadranten des Abdomens operieren zu müssen, was bei robotischen Eingriffen ein limitierender Faktor sein kann. Ziel dieser Arbeit war es daher, die Machbarkeit der robotisch assistierten Proktokolektomie mit IPAA zu demonstrieren und Tipps für eine Anwendung in der klinischen Praxis zu geben.

Collaborative Virtual Reality Real-Time 3D Image Editing for Chest Wall Resections and Reconstruction Planning

The integration of extended reality (XR) technologies into health care procedures presents transformative opportunities, particularly in surgical processes. This study delves into the utilization of virtual reality (VR) for preoperative planning related to chest wall resections in thoracic surgery. Leveraging the capabilities of 3-dimensional (3D) imaging, real-time visualization, and collaborative VR environments, surgeons gain enhanced anatomical insights and can develop predictive surgical strategies. Two clinical cases highlighted the effectiveness of this approach, showcasing the potential for personalized and intricate surgical planning. The setup provides an immersive, dynamic representation of real patient data, enabling collaboration among teams from separate locations. While VR offers enhanced interactive and visualization capabilities, preliminary evidence suggests it may support more refined preoperative strategies, potentially influence postoperative outcomes, and optimize resource management. However, its comparative advantage over traditional methods needs further empirical validation. Emphasizing the potential of XR, this exploration suggests its broad implications in thoracic surgery, especially when dealing with complex cases requiring multidisciplinary collaboration in the immersive virtual space, often referred to as the metaverse. This innovative approach necessitates further examination, marking a shift toward future surgical preparations. In this article, we sought to demonstrate the technique of an immersive real-time volume-rendered collaborative VR-planning tool using exemplary case studies in chest wall surgery.

[Extended Reality (XR) - Applications in Thoracic Surgery]

Extended reality (XR) includes the sub-terms of virtual reality (VR), augmented reality (AR) and mixed reality (MR) and describes interactive and immersive technologies that replace the real world with digital elements or seamlessly extend it with such approaches. XR thus offers a very wide range of possible applications in medicine. In surgery, and thoracic surgery in particular, XR technologies can be harnessed for treatment planning, navigation, training, and patient information. Such applications are increasingly being tested and need to be evaluated. We provide an overview of the status quo of technical development, current surgical applications of XR, and look into the future of the medical XR landscape with integration of artificial intelligence (AI).

Machine Learning for Decision-Support in Acute Abdominal Pain - Proof of Concept and Central Considerations

Acute abdominal pain is a common presenting symptom in the emergency department and represents heterogeneous causes and diagnoses. There is often a decision to be made regarding emergency surgical care. Machine learning (ML) could be used here as a decision-support and relieve the time and personnel resource shortage.Patients with acute abdominal pain presenting to the Department of Surgery at Bonn University Hospital in 2020 and 2021 were retrospectively analyzed. Clinical parameters as well as laboratory values were used as predictors. After randomly splitting into a training and test data set (ratio 80 to 20), three ML algorithms were comparatively trained and validated. The entire procedure was repeated 20 times.A total of 1357 patients were identified and included in the analysis, with one in five (n = 276, 20.3%) requiring emergency abdominal surgery within 24 hours. Patients operated on were more likely to be male (p = 0.026), older (p = 0.006), had more gastrointestinal symptoms (nausea: p < 0.001, vomiting p < 0.001) as well as a more recent onset of pain (p < 0.001). Tenderness (p < 0.001) and guarding (p < 0.001) were more common in surgically treated patients and blood analyses showed increased inflammation levels (white blood cell count: p < 0.001, CRP: p < 0.001) and onset of organ dysfunction (creatinine: p < 0.014, quick p < 0.001). Of the three trained algorithms, the tree-based methods (h2o random forest and cforest) showed the best performance. The algorithms classified patients, i.e., predicted surgery, with a median AUC ROC of 0.81 and 0.79 and AUC PRC of 0.56 in test sets.A proof-of-concept was achieved with the development of an ML model for predicting timely surgical therapy for acute abdomen. The ML algorithm can be a valuable tool in decision-making. Especially in the context of heavily used medical resources, the algorithm can help to use these scarce resources more effectively. Technological progress, especially regarding artificial intelligence, increasingly enables evidence-based approaches in surgery but requires a strictly interdisciplinary approach. In the future, the use and handling of ML should be integrated into surgical training.

[Robotics in thoracic surgery]

The increasing diffusion of the robotic-assisted technique in thoracic surgery (RATS) in Germany was initially delayed in comparison with other countries. Therefore, there is a large potential to implement the volume of the surgical procedures performed by RATS.The RATS-technique has many positive aspects. For example, the angulated instruments allow a full wristed dexterity like the human hand, but with a greater range of motion. The surgical Robot has a tremor filter and replicates perfectly the surgeon's movements. Furthermore, the 3D-scope enables an image magnification up to 10 times compared to the normal thoracoscopes. The RATS has also some disadvantages. For example, the operating surgeon sits far away from the patient and is not sterile while performing surgery. This is an important factor in in case of emergency situations, like major bleeding, which often require a conversion to thoracotomy.All robotic systems are built after the same master-slave technology, that allows the operating surgeon to have full control of the master system. The slave system consists of mechanical actuators that respond to the master system's inputs, so the surgical robot will translate every single movement of the surgeon at the console.The main surgical indications for RATS are: mediastinal tumors, diaphragm plication and anatomical lung resection like segment resections, lobectomies or sleeve resections.In the future, the implementation of virtual and augmented reality is expected in the training but also in the planning of RATS-operations.