Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation

Automated Operative Phase and Step Recognition in Vestibular Schwannoma Surgery: Development and Preclinical Evaluation of a Deep Learning Neural Network (IDEAL Stage 0)

BACKGROUND AND OBJECTIVES

Machine learning (ML) in surgical video analysis offers promising prospects for training and decision support in surgery. The past decade has seen key advances in ML-based operative workflow analysis, though existing applications mostly feature shorter surgeries (<2 hours) with limited scene changes. The aim of this study was to develop and evaluate a ML model capable of automated operative workflow recognition for retrosigmoid vestibular schwannoma (VS) resection. In doing so, this project furthers previous research by applying workflow prediction platforms to lengthy (median >5 hours duration), data-heavy surgeries, using VS resection as an exemplar.

METHODS

A video dataset of 21 microscopic retrosigmoid VS resections was collected at a single institution over 3 years and underwent workflow annotation according to a previously agreed expert consensus (Approach, Excision, and Closure phases; and Debulking or Dissection steps within the Excision phase). Annotations were used to train a ML model consisting of a convolutional neural network and a recurrent neural network. 5-fold cross-validation was used, and performance metrics (accuracy, precision, recall, F1 score) were assessed for phase and step prediction.

RESULTS

Median operative video time was 5 hours 18 minutes (IQR 3 hours 21 minutes-6 hours 1 minute). The "Tumor Excision" phase accounted for the majority of each case (median 4 hours 23 minutes), whereas "Approach and Exposure" (28 minutes) and "Closure" (17 minutes) comprised shorter phases. The ML model accurately predicted operative phases (accuracy 81%, weighted F1 0.83) and dichotomized steps (accuracy 86%, weighted F1 0.86).

CONCLUSION

This study demonstrates that our ML model can accurately predict the surgical phases and intraphase steps in retrosigmoid VS resection. This demonstrates the successful application of ML in operative workflow recognition on low-volume, lengthy, data-heavy surgical videos. Despite this, there remains room for improvement in individual step classification. Future applications of ML in low-volume high-complexity operations should prioritize collaborative video sharing to overcome barriers to clinical translation.

Vision techniques for anatomical structures in laparoscopic surgery: a comprehensive review

Laparoscopic surgery is the method of choice for numerous surgical procedures, while it confronts a lot of challenges. Computer vision exerts a vital role in addressing these challenges and has become a research hotspot, especially in the classification, segmentation, and target detection of abdominal anatomical structures. This study presents a comprehensive review of the last decade of research in this area. At first, a categorized overview of the core subtasks is presented regarding their relevance and applicability to real-world medical scenarios. Second, the dataset used in the experimental validation is statistically analyzed. Subsequently, the technical approaches and trends of classification, segmentation, and target detection tasks are explored in detail, highlighting their advantages, limitations, and practical implications. Additionally, evaluation methods for the three types of tasks are discussed. Finally, gaps in current research are identified. Meanwhile, the great potential for development in this area is emphasized.

Remote robotic surgery: implementing a technology 20 years in the making

Remote robotic surgery represents a transformative and unique approach to surgical care, offering the potential to expand access to healthcare to underserved areas, improve patient outcomes, and enhance surgical technologies. However, it is not without significant challenges, including technical limitations, ethical concerns, and financial implications. These hurdles must be carefully addressed to ensure safe and equitable integration into mainstream healthcare. Following initial excitement over 20 years ago regarding its potential as a remote tool in arenas, such as combat zones, its mainstream adoption faced challenges in terms of technological infrastructure, regulatory compliance, and ethical considerations. With the widespread adoption of robotic surgery and improvements in both the technology and the communications infrastructure, the potential for remote robotic telesurgery is experiencing a resurgence. While surgery in austere environments is intriguing, this article aims to explore the roadmap for potentially integrating remote robotic surgery into mainstream healthcare, as well as the feasibility of remote robotic surgery in the current clinical climate.

Systematic review on the use of artificial intelligence to identify anatomical structures during laparoscopic cholecystectomy: a tool towards the future

PURPOSE

Bile duct injury (BDI) during laparoscopic cholecystectomy (LC) is a dreaded complication. Artificial intelligence (AI) has recently been introduced in surgery. This systematic review aims to investigate whether AI can guide surgeons in identifying anatomical structures to facilitate safer dissection during LC.

METHODS

Following PROSPERO registration CRD-42023478754, a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-compliant systematic search of MEDLINE (via PubMed), EMBASE, and Web of Science databases was conducted.

RESULTS

Out of 2304 articles identified, twenty-five were included in the analysis. The mean average precision for biliary structures detection reported in the included studies reaches 98%. The mean intersection over union ranges from 0.5 to 0.7, and the mean Dice/F1 spatial correlation index was greater than 0.7/1. AI system provided a change in the annotations in 27% of the cases, and 70% of these shifts were considered safer changes. The contribution to preventing BDI was reported at 3.65/4.

CONCLUSIONS

Although studies on the use of AI during LC are few and very heterogeneous, AI has the potential to identify anatomical structures, thereby guiding surgeons towards safer LC procedures.

Development of an artificial intelligence system to indicate intraoperative findings of scarring in laparoscopic cholecystectomy for cholecystitis

BACKGROUND

The surgical difficulty of laparoscopic cholecystectomy (LC) for acute cholecystitis (AC) and the risk of bile duct injury (BDI) depend on the degree of fibrosis and scarring caused by inflammation; therefore, understanding these intraoperative findings is crucial to preventing BDI. Scarring makes it particularly difficult to perform safely and increases the BDI risk. This study aimed to develop an artificial intelligence (AI) system to indicate intraoperative findings of scarring in LC for AC.

MATERIALS AND METHODS

An AI system was developed to detect scarred areas using an algorithm for semantic segmentation based on deep learning. The training dataset consisted of 2025 images extracted from LC videos of 21 cases with AC. External evaluation committees (EEC) evaluated the AI system on 20 cases of untrained data from other centers. EECs evaluated the accuracy in identifying the scarred area and the usefulness of the AI system, which were assessed based on annotation and a 5-point Likert-scale questionnaire.

RESULTS

The average DICE coefficient for scarred areas between AI detection and EEC annotation was 0.612. The EEC's average detection accuracy on the Likert scale was 3.98 ± 0.76. AI systems were rated as relatively useful for both clinical and educational applications.

CONCLUSION

We developed an AI system to detect scarred areas in LC for AC. Since scarring increases the surgical difficulty, this AI system has the potential to reduce BDI.

Concept Graph Neural Networks for Surgical Video Understanding

Analysis of relations between objects and comprehension of abstract concepts in the surgical video is important in AI-augmented surgery. However, building models that integrate our knowledge and understanding of surgery remains a challenging endeavor. In this paper, we propose a novel way to integrate conceptual knowledge into temporal analysis tasks using temporal concept graph networks. In the proposed networks, a knowledge graph is incorporated into the temporal video analysis of surgical notions, learning the meaning of concepts and relations as they apply to the data. We demonstrate results in surgical video data for tasks such as verification of the critical view of safety, estimation of the Parkland grading scale as well as recognizing instrument-action-tissue triplets. The results show that our method improves the recognition and detection of complex benchmarks as well as enables other analytic applications of interest.

Development of a liver graft assessment expert machine-learning system: when the artificial intelligence helps liver transplant surgeons

Background

The complex process of liver graft assessment is one point for improvement in liver transplantation. The main objective of this study is to develop a tool that supports the surgeon who is responsible for liver donation in the decision-making process whether to accept a graft or not using the initial variables available to it.

Material and method

Liver graft samples candidate for liver transplantation after donor brain death were studied. All of them were evaluated "in situ" for transplantation, and those discarded after the "in situ" evaluation were considered as no transplantable liver grafts, while those grafts transplanted after "in situ" evaluation were considered as transplantable liver grafts. First, a single-center, retrospective and cohort study identifying the risk factors associated with the no transplantable group was performed. Then, a prediction model decision support system based on machine learning, and using a tree ensemble boosting classifier that is capable of helping to decide whether to accept or decline a donor liver graft, was developed.

Results

A total of 350 liver grafts that were evaluated for liver transplantation were studied. Steatosis was the most frequent reason for classifying grafts as no transplantable, and the main risk factors identified in the univariant study were age, dyslipidemia, personal medical history, personal surgical history, bilirubinemia, and the result of previous liver ultrasound (p < 0.05). When studying the developed model, we observe that the best performance reordering in terms of accuracy corresponds to 76.29% with an area under the curve of 0.79. Furthermore, the model provides a classification together with a confidence index of reliability, for most cases in our data, with the probability of success in the prediction being above 0.85.

Conclusion

The tool presented in this study obtains a high accuracy in predicting whether a liver graft will be transplanted or deemed non-transplantable based on the initial variables assigned to it. The inherent capacity for improvement in the system causes the rate of correct predictions to increase as new data are entered. Therefore, we believe it is a tool that can help optimize the graft pool for liver transplantation.

SurgSmart: an artificial intelligent system for quality control in laparoscopic cholecystectomy: an observational study

BACKGROUND

The rate of bile duct injury in laparoscopic cholecystectomy (LC) continues to be high due to low critical view of safety (CVS) achievement and the absence of an effective quality control system. The development of an intelligent system enables the automatic quality control of LC surgery and, eventually, the mitigation of bile duct injury. This study aims to develop an intelligent surgical quality control system for LC and using the system to evaluate LC videos and investigate factors associated with CVS achievement.

MATERIALS AND METHODS

SurgSmart, an intelligent system capable of recognizing surgical phases, disease severity, critical division action, and CVS automatically, was developed using training datasets. SurgSmart was also applied in another multicenter dataset to validate its application and investigate factors associated with CVS achievement.

RESULTS

SurgSmart performed well in all models, with the critical division action model achieving the highest overall accuracy (98.49%), followed by the disease severity model (95.45%) and surgical phases model (88.61%). CVSI, CVSII, and CVSIII had an accuracy of 80.64, 97.62, and 78.87%, respectively. CVS was achieved in 4.33% in the system application dataset. In addition, the analysis indicated that surgeons at a higher hospital level had a higher CVS achievement rate. However, there was still considerable variation in CVS achievement among surgeons in the same hospital.

CONCLUSIONS

SurgSmart, the surgical quality control system, performed admirably in our study. In addition, the system's initial application demonstrated its broad potential for use in surgical quality control.

Retrospective validation of parkland grading scale in a Latin-American high-volume center

BACKGROUND

Increased complication rates following laparoscopic cholecystectomies have been described, likely related to surgical difficulty, anatomical variations, and gallbladder inflammation severity. Parkland Grading Scale (PGS) stratifies the severity of intraoperative findings to predict operative difficulty and complications. This study aims to validate PGS as a postoperative-outcome predictive tool, comparing its performance with Tokyo Guidelines Grading System (TGGS).

METHODS

This is a single-center retrospective cohort study where PGS and TGGS performances were evaluated regarding intraoperative and postoperative outcomes. Both univariate and bivariate analyses were performed on each severity grading scale using STATA-SE 16.0 software. Additionally, we proposed a Logistic Regression Model for each scale. Their association with outcomes was compared between both scales by their Receiver Operating Characteristic Curve.

RESULTS

400 Patients were included. Grade 1 predominance was observed for both PGS and TGGS (47.36% and 25.3%, respectively). A positive association was observed between higher PGS grades and inpatient postoperative care, length of stay, ICU care, and antibiotic requirement. Based on the area under the ROC curve, better performance was observed for PGS over TGGS in the evaluated outcomes.

CONCLUSION

PGS performed better than TGGS as a predictive tool for inpatient postoperative care, length of stay, ICU, and antibiotic requirement, especially in severe cases.

Validation of an artificial intelligence platform for the guidance of safe laparoscopic cholecystectomy

BACKGROUND

Many surgical adverse events, such as bile duct injuries during laparoscopic cholecystectomy (LC), occur due to errors in visual perception and judgment. Artificial intelligence (AI) can potentially improve the quality and safety of surgery, such as through real-time intraoperative decision support. GoNoGoNet is a novel AI model capable of identifying safe ("Go") and dangerous ("No-Go") zones of dissection on surgical videos of LC. Yet, it is unknown how GoNoGoNet performs in comparison to expert surgeons. This study aims to evaluate the GoNoGoNet's ability to identify Go and No-Go zones compared to an external panel of expert surgeons.

METHODS

A panel of high-volume surgeons from the SAGES Safe Cholecystectomy Task Force was recruited to draw free-hand annotations on frames of prospectively collected videos of LC to identify the Go and No-Go zones. Expert consensus on the location of Go and No-Go zones was established using Visual Concordance Test pixel agreement. Identification of Go and No-Go zones by GoNoGoNet was compared to expert-derived consensus using mean F1 Dice Score, and pixel accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).

RESULTS

A total of 47 frames from 25 LC videos, procured from 3 countries and 9 surgeons, were annotated simultaneously by an expert panel of 6 surgeons and GoNoGoNet. Mean (± standard deviation) F1 Dice score were 0.58 (0.22) and 0.80 (0.12) for Go and No-Go zones, respectively. Mean (± standard deviation) accuracy, sensitivity, specificity, PPV and NPV for the Go zones were 0.92 (0.05), 0.52 (0.24), 0.97 (0.03), 0.70 (0.21), and 0.94 (0.04) respectively. For No-Go zones, these metrics were 0.92 (0.05), 0.80 (0.17), 0.95 (0.04), 0.84 (0.13) and 0.95 (0.05), respectively.

CONCLUSIONS

AI can be used to identify safe and dangerous zones of dissection within the surgical field, with high specificity/PPV for Go zones and high sensitivity/NPV for No-Go zones. Overall, model prediction was better for No-Go zones compared to Go zones. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

Computer vision in surgery: from potential to clinical value

Hundreds of millions of operations are performed worldwide each year, and the rising uptake in minimally invasive surgery has enabled fiber optic cameras and robots to become both important tools to conduct surgery and sensors from which to capture information about surgery. Computer vision (CV), the application of algorithms to analyze and interpret visual data, has become a critical technology through which to study the intraoperative phase of care with the goals of augmenting surgeons' decision-making processes, supporting safer surgery, and expanding access to surgical care. While much work has been performed on potential use cases, there are currently no CV tools widely used for diagnostic or therapeutic applications in surgery. Using laparoscopic cholecystectomy as an example, we reviewed current CV techniques that have been applied to minimally invasive surgery and their clinical applications. Finally, we discuss the challenges and obstacles that remain to be overcome for broader implementation and adoption of CV in surgery.