Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos

A systematic review of annotation for surgical process model analysis in minimally invasive surgery based on video

BACKGROUND

Annotated data are foundational to applications of supervised machine learning. However, there seems to be a lack of common language used in the field of surgical data science. The aim of this study is to review the process of annotation and semantics used in the creation of SPM for minimally invasive surgery videos.

METHODS

For this systematic review, we reviewed articles indexed in the MEDLINE database from January 2000 until March 2022. We selected articles using surgical video annotations to describe a surgical process model in the field of minimally invasive surgery. We excluded studies focusing on instrument detection or recognition of anatomical areas only. The risk of bias was evaluated with the Newcastle Ottawa Quality assessment tool. Data from the studies were visually presented in table using the SPIDER tool.

RESULTS

Of the 2806 articles identified, 34 were selected for review. Twenty-two were in the field of digestive surgery, six in ophthalmologic surgery only, one in neurosurgery, three in gynecologic surgery, and two in mixed fields. Thirty-one studies (88.2%) were dedicated to phase, step, or action recognition and mainly relied on a very simple formalization (29, 85.2%). Clinical information in the datasets was lacking for studies using available public datasets. The process of annotation for surgical process model was lacking and poorly described, and description of the surgical procedures was highly variable between studies.

CONCLUSION

Surgical video annotation lacks a rigorous and reproducible framework. This leads to difficulties in sharing videos between institutions and hospitals because of the different languages used. There is a need to develop and use common ontology to improve libraries of annotated surgical videos.

Surgesture: a novel instrument based on surgical actions for objective skill assessment

BACKGROUND

Due to varied surgical skills and the lack of an efficient rating system, we developed Surgesture based on elementary functional surgical gestures performed by surgeons, which could serve as objective metrics to evaluate surgical performance in laparoscopic cholecystectomy (LC).

METHODS

We defined 14 LC basic Surgestures. Four surgeons annotated Surgestures among LC videos performed by experts and novices. The counts, durations, average action time, and dissection/exposure ratio (D/E ratio) of LC Surgestures were compared. The phase of mobilizing hepatocystic triangle (MHT) was extracted for skill assessment by three professors using a modified Global Operative Assessment of Laparoscopic Skills (mGOALS).

RESULTS

The novice operation time was significantly longer than the expert operation time (58.12 ± 19.23 min vs. 26.66 ± 8.00 min, P < 0.001), particularly during MHT phase. Novices had significantly more Surgestures than experts in both hands (P < 0.05). The left hand and inefficient Surgesture of novices were dramatically more than those of experts (P < 0.05). The experts demonstrated a significantly higher D/E ratio of duration than novices (0.79 ± 0.37 vs. 2.84 ± 1.98, P < 0.001). The counts and time pattern map of LC Surgestures during MHT demonstrated that novices tended to complete LC with more types of Surgestures and spent more time exposing the surgical scene. The performance metrics of LC Surgesture had significant but weak associations with each aspect of mGOALS.

CONCLUSION

The newly constructed Surgestures could serve as accessible and quantifiable metrics for demonstrating the operative pattern and distinguishing surgeons with various skills. The association between Surgestures and Global Rating Scale laid the foundation for establishing a bridge to automated objective surgical skill evaluation.

Development of an International Standardized Curriculum for Laparoscopic Sleeve Gastrectomy Teaching Utilizing Modified Delphi Methodology

BACKGROUND

The performance of laparoscopic sleeve gastrectomy has increased markedly to become the single-most performed bariatric surgical procedure globally. To date, a means of standardized trainee teaching has not been developed. The aim of this study was to design a laparoscopic curriculum for trainees of bariatric surgery utilizing modified Delphi consensus methodology.

METHODS

A panel of surgeons was assembled to devise an academic framework of technical, non-technical and cognitive skills utilized in the performance of laparoscopic sleeve gastrectomy. The panel invited 18 bariatric surgeons experienced in laparoscopic gastrectomy from 11 countries to rate the items for inclusion in the curriculum to a predefined level of agreement.

RESULTS

A consensus of experts was achieved for 24 of the 30 proposed elements for inclusion within the first round of the curriculum Delphi panel. All components pertaining to anatomical knowledge, peri-operative considerations and non-technical items were accepted. A second round further examined six statements, of which three were accepted. Agreement of the panel was reached for 27 of the cognitive, technical and non-technical components after two rounds. Three statements found no consensus.

CONCLUSIONS

Utilizing modified Delphi methodology, a curriculum outlining the most important components of teaching the procedure of laparoscopic sleeve gastrectomy, has been determined by a consensus of international experts in bariatric surgery. The curriculum is suggested as a standard in proficiency-based training of this procedure. It forms a generic template which facilitates individual jurisdictions to perform content validation, adapting the curriculum to local requirements in teaching the next generation of bariatric surgeons.

A Scoping Review of Artificial Intelligence and Machine Learning in Bariatric and Metabolic Surgery: Current Status and Future Perspectives

Artificial intelligence (AI) is a revolution in data analysis with emerging roles in various specialties and with various applications. The objective of this scoping review was to retrieve current literature on the fields of AI that have been applied to metabolic bariatric surgery (MBS) and to investigate potential applications of AI as a decision-making tool of the bariatric surgeon. Initial search yielded 3260 studies published from January 2000 until March 2021. After screening, 49 unique articles were included in the final analysis. Studies were grouped into categories, and the frequency of appearing algorithms, dataset types, and metrics were documented. The heterogeneity of current studies showed that meticulous validation, strict reporting systems, and reliable benchmarking are mandatory for ensuring the clinical validity of future research.

Explaining a model predicting quality of surgical practice: a first presentation to and review by clinical experts

PURPOSE

Surgical Data Science (SDS) is an emerging research domain offering data-driven answers to challenges encountered by clinicians during training and practice. We previously developed a framework to assess quality of practice based on two aspects: exposure of the surgical scene (ESS) and the surgeon's profile of practice (SPP). Here, we wished to investigate the clinical relevance of the parameters learned by this model by (1) interpreting these parameters and identifying associated representative video samples and (2) presenting this information to surgeons in the form of a video-enhanced questionnaire. To our knowledge, this is the first approach in the field of SDS for laparoscopy linking the choices made by a machine learning model predicting surgical quality to clinical expertise.

METHOD

Spatial features and quality of practice scores extracted from labeled and segmented frames in 30 laparoscopic videos were used to predict the ESS and the SPP. The relationships between the inputs and outputs of the model were then analyzed and translated into meaningful sentences (statements, e.g., "To optimize the ESS, it is very important to correctly handle the spleen"). Representative video clips illustrating these statements were semi-automatically identified. Eleven statements and video clips were used in a survey presented to six experienced digestive surgeons to gather their opinions on the algorithmic analyses.

RESULTS

All but one of the surgeons agreed with the proposed questionnaire overall. On average, surgeons agreed with 7/11 statements.

CONCLUSION

This proof-of-concept study provides preliminary validation of our model which has a high potential for use to analyze and understand surgical practices.

Patch-based classification of gallbladder wall vascularity from laparoscopic images using deep learning

PURPOSE

In this study, we propose a deep learning approach for assessment of gallbladder (GB) wall vascularity from images of laparoscopic cholecystectomy (LC). Difficulty in the visualization of GB wall vessels may be the result of fatty infiltration or increased thickening of the GB wall, potentially as a result of cholecystitis or other diseases.

METHODS

The dataset included 800 patches and 181 region outlines of the GB wall extracted from 53 operations of the Cholec80 video collection. The GB regions and patches were annotated by two expert surgeons using two labeling schemes: 3 classes (low, medium and high vascularity) and 2 classes (low vs. high). Two convolutional neural network (CNN) architectures were investigated. Preprocessing (vessel enhancement) and post-processing (late fusion of CNN output) techniques were applied.

RESULTS

The best model yielded accuracy 94.48% and 83.77% for patch classification into 2 and 3 classes, respectively. For the GB wall regions, the best model yielded accuracy 91.16% (2 classes) and 80.66% (3 classes). The inter-observer agreement was 91.71% (2 classes) and 78.45% (3 classes). Late fusion analysis allowed the computation of spatial probability maps, which provided a visual representation of the probability for each vascularity class across the GB wall region.

CONCLUSIONS

This study is the first significant step forward to assess the vascularity of the GB wall from intraoperative images based on computer vision and deep learning techniques. The classification performance of the CNNs was comparable to the agreement of two expert surgeons. The approach may be used for various applications such as for classification of LC operations and context-aware assistance in surgical education and practice.