Vision-Based Mouth Motion Analysis in Epilepsy: A 3D Perspective

NeuroKinect4K: A Novel 4K RGB-D-IR Video System with 3D Scene Reconstruction for Enhanced Epileptic Seizure Semiology Monitoring

Epileptic seizures are clearly characterized by their displayed behavior, the semiology, which is used in diagnosis and classification as a base for therapy. This article presents a novel 4K 3D video recording and reviewing system for epilepsy monitoring, introducing a novel perspective and allowing continuous recording and review of 3D videos in the epilepsy monitoring unit (EMU), providing significantly more detail than the current clinical systems, which can lead to the recognition of more Movements of Interest (MOIs) and may reduce inter-rater variability. To put the system to an initial test in clinical practice the article presents three real-world examples of subtle MOIs, that could only be appreciated on the 4K-video, but not on the VGA-video, recorded as part of the clinical routine. In conclusion, a 4K-RGB recording, 3D cropping, and 3D video playing system was developed, implemented, and tested for realworld clinical scenarios, considering the specific requirements of clinical monitoring in EMUs. The new data acquisition setup can support clinical diagnosis, which may lead to new insights in the field of epilepsy and the development of AI approaches in the future.

Computer vision for automated seizure detection and classification: A systematic review

Computer vision (CV) shows increasing promise as an efficient, low-cost tool for video seizure detection and classification. Here, we provide an overview of the fundamental concepts needed to understand CV and summarize the structure and performance of various model architectures used in video seizure analysis. We conduct a systematic literature review of the PubMed, Embase, and Web of Science databases from January 1, 2000 to September 15, 2023, to identify the strengths and limitations of CV seizure analysis methods and discuss the utility of these models when applied to different clinical seizure phenotypes. Reviews, nonhuman studies, and those with insufficient or poor quality data are excluded from the review. Of the 1942 records identified, 45 meet inclusion criteria and are analyzed. We conclude that the field has shown tremendous growth over the past 2 decades, leading to several model architectures with impressive accuracy and efficiency. The rapid and scalable detection offered by CV models holds the potential to reduce sudden unexpected death in epilepsy and help alleviate resource limitations in epilepsy monitoring units. However, a lack of standardized, thorough validation measures and concerns about patient privacy remain important obstacles for widespread acceptance and adoption. Investigation into the performance of models across varied datasets from clinical and nonclinical environments is an essential area for further research.

A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis

Disease diagnosis represents a critical and arduous endeavor within the medical field. Artificial intelligence (AI) techniques, spanning from machine learning and deep learning to large model paradigms, stand poised to significantly augment physicians in rendering more evidence-based decisions, thus presenting a pioneering solution for clinical practice. Traditionally, the amalgamation of diverse medical data modalities (e.g., image, text, speech, genetic data, physiological signals) is imperative to facilitate a comprehensive disease analysis, a topic of burgeoning interest among both researchers and clinicians in recent times. Hence, there exists a pressing need to synthesize the latest strides in multi-modal data and AI technologies in the realm of medical diagnosis. In this paper, we narrow our focus to five specific disorders (Alzheimer's disease, breast cancer, depression, heart disease, epilepsy), elucidating advanced endeavors in their diagnosis and treatment through the lens of artificial intelligence. Our survey not only delineates detailed diagnostic methodologies across varying modalities but also underscores commonly utilized public datasets, the intricacies of feature engineering, prevalent classification models, and envisaged challenges for future endeavors. In essence, our research endeavors to contribute to the advancement of diagnostic methodologies, furnishing invaluable insights for clinical decision making.

Novel 3D video action recognition deep learning approach for near real time epileptic seizure classification

Seizure semiology is a well-established method to classify epileptic seizure types, but requires a significant amount of resources as long-term Video-EEG monitoring needs to be visually analyzed. Therefore, computer vision based diagnosis support tools are a promising approach. In this article, we utilize infrared (IR) and depth (3D) videos to show the feasibility of a 24/7 novel object and action recognition based deep learning (DL) monitoring system to differentiate between epileptic seizures in frontal lobe epilepsy (FLE), temporal lobe epilepsy (TLE) and non-epileptic events. Based on the largest 3Dvideo-EEG database in the world (115 seizures/+680,000 video-frames/427GB), we achieved a promising cross-subject validation f1-score of 0.833±0.061 for the 2 class (FLE vs. TLE) and 0.763 ± 0.083 for the 3 class (FLE vs. TLE vs. non-epileptic) case, from 2 s samples, with an automated semi-specialized depth (Acc.95.65%) and Mask R-CNN (Acc.96.52%) based cropping pipeline to pre-process the videos, enabling a near-real-time seizure type detection and classification tool. Our results demonstrate the feasibility of our novel DL approach to support 24/7 epilepsy monitoring, outperforming all previously published methods.