Hybrid Spatiotemporal Contrastive Representation Learning for Content-Based Surgical Video Retrieval

Artificial intelligence-assisted phase recognition and skill assessment in laparoscopic surgery: a systematic review

With the widespread adoption of minimally invasive surgery, laparoscopic surgery has been an essential component of modern surgical procedures. As key technologies, laparoscopic phase recognition and skill evaluation aim to identify different stages of the surgical process and assess surgeons' operational skills using automated methods. This, in turn, can improve the quality of surgery and the skill of surgeons. This review summarizes the progress of research in laparoscopic surgery, phase recognition, and skill evaluation. At first, the importance of laparoscopic surgery is introduced, clarifying the relationship between phase recognition, skill evaluation, and other surgical tasks. The publicly available surgical datasets for laparoscopic phase recognition tasks are then detailed. The review highlights the research methods that have exhibited superior performance in these public datasets and identifies common characteristics of these high-performing methods. Based on the insights obtained, the commonly used phase recognition research and surgical skill evaluation methods and models in this field are summarized. In addition, this study briefly outlines the standards and methods for evaluating laparoscopic surgical skills. Finally, an analysis of the difficulties researchers face and potential future development directions is presented. Moreover, this paper aims to provide valuable references for researchers, promoting further advancements in this domain.

Epileptic Seizure Classification with Patient-level and Video-level Contrastive Pretraining

Accurate classification of epileptic seizure types through seizure semiology analysis demands significant clinical expertise. While previous studies have employed various action recognition modules, the scarcity of labeled clinical videos has hindered the deployment of larger models. In this study, we explore unlabeled data to pretrain a transformer-based model with contrastive loss, taking advantage of the information that circumvents the need for additional annotation from medical professionals. We maximize the similarity between embeddings from the same patient and video while minimizing those from different patients and videos. Subsequently, a classification head was finetuned to distinguishing temporal lobe epilepsy (TLE) and extratemporal lobe epilepsy (exTLE), achieving a 5-fold accuracy of 0.93 and an F1 score of 0.88 on the video level (N = 57). Our results outperformed other state-of-the-art seizure classification models, demonstrating the efficacy of our approach. This suggests potential applications in clinical practice, where unlabeled data could serve as a valuable aid in improving seizure classification accuracy and patient care.

Semantic-Preserving Surgical Video Retrieval With Phase and Behavior Coordinated Hashing

Medical professionals rely on surgical video retrieval to discover relevant content within large numbers of videos for surgical education and knowledge transfer. However, the existing retrieval techniques often fail to obtain user-expected results since they ignore valuable semantics in surgical videos. The incorporation of rich semantics into video retrieval is challenging in terms of the hierarchical relationship modeling and coordination between coarse- and fine-grained semantics. To address these issues, this paper proposes a novel semantic-preserving surgical video retrieval (SPSVR) framework, which incorporates surgical phase and behavior semantics using a dual-level hashing module to capture their hierarchical relationship. This module preserves the semantics in binary hash codes by transforming the phase and behavior similarities into high- and low-level similarities in a shared Hamming space. The binary codes are optimized by performing a reconstruction task, a high-level similarity preservation task, and a low-level similarity preservation task, using a coordinated optimization strategy for efficient learning. A self-supervised learning scheme is adopted to capture behavior semantics from video clips so that the indexing of behaviors is unencumbered by fine-grained annotation and recognition. Experiments on four surgical video datasets for two different disciplines demonstrate the robust performance of the proposed framework. In addition, the results of the clinical validation experiments indicate the ability of the proposed method to retrieve the results expected by surgeons. The code can be found at https://github.com/trigger26/SPSVR.