Extraction of motion strength and motor activity signals from video recordings of neonatal seizures

Computer vision for automated seizure detection and classification: A systematic review

Computer vision (CV) shows increasing promise as an efficient, low-cost tool for video seizure detection and classification. Here, we provide an overview of the fundamental concepts needed to understand CV and summarize the structure and performance of various model architectures used in video seizure analysis. We conduct a systematic literature review of the PubMed, Embase, and Web of Science databases from January 1, 2000 to September 15, 2023, to identify the strengths and limitations of CV seizure analysis methods and discuss the utility of these models when applied to different clinical seizure phenotypes. Reviews, nonhuman studies, and those with insufficient or poor quality data are excluded from the review. Of the 1942 records identified, 45 meet inclusion criteria and are analyzed. We conclude that the field has shown tremendous growth over the past 2 decades, leading to several model architectures with impressive accuracy and efficiency. The rapid and scalable detection offered by CV models holds the potential to reduce sudden unexpected death in epilepsy and help alleviate resource limitations in epilepsy monitoring units. However, a lack of standardized, thorough validation measures and concerns about patient privacy remain important obstacles for widespread acceptance and adoption. Investigation into the performance of models across varied datasets from clinical and nonclinical environments is an essential area for further research.

Acute symptomatic seizures in newborns: a narrative review

Acute symptomatic seizures are the main sign of neurological dysfunction in newborns. This is linked to the unique characteristics of the neonatal brain, making it hyperexcitable compared to older ages, and to the common occurrence of some forms of acquired brain injury, namely hypoxic-ischemic encephalopathy. In this narrative review we will provide an overview of neonatal seizures definition, their main underlying etiologies, diagnostic work-up and differential diagnoses, and will discuss about therapeutic options and prognostic outlook. The latest publications from the ILAE Task Force on Neonatal Seizures will be presented and discussed. Of note, they highlight the current lack of robust evidence in this field of clinical neurology. We will also report on specificities pertaining to low-and-middle income countries in terms of incidence, main etiologies and diagnosis. The possibilities offered by telemedicine and automated seizures detection will also be summarized in order to provide a framework for future directions in seizures diagnosis and management with a global perspective. Many challenges and opportunities for improving identification, monitoring and treatment of acute symptomatic seizures in newborns exist. All current caveats potentially represent different lines of research with the aim to provide better care and reach a deeper understanding of this important topic of neonatal neurology.

Heart Rate Variability Analysis for Seizure Detection in Neonatal Intensive Care Units

In Neonatal Intensive Care Units (NICUs), the early detection of neonatal seizures is of utmost importance for a timely clinical intervention. Over the years, several neonatal seizure detection systems were proposed to detect neonatal seizures automatically and speed up seizure diagnosis, most based on the EEG signal analysis. Recently, research has focused on other possible seizure markers, such as electrocardiography (ECG). This work proposes an ECG-based NSD system to investigate the usefulness of heart rate variability (HRV) analysis to detect neonatal seizures in the NICUs. HRV analysis is performed considering time-domain, frequency-domain, entropy and multiscale entropy features. The performance is evaluated on a dataset of ECG signals from 51 full-term babies, 29 seizure-free. The proposed system gives results comparable to those reported in the literature: Area Under the Receiver Operating Characteristic Curve = 62%, Sensitivity = 47%, Specificity = 67%. Moreover, the system’s performance is evaluated in a real clinical environment, inevitably affected by several artefacts. To the best of our knowledge, our study proposes for the first time a multi-feature ECG-based NSD system that also offers a comparative analysis between babies suffering from seizures and seizure-free ones.

Video and audio processing in paediatrics: a review

OBJECTIVE

Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU).

APPROACH

For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts-full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization.

MAIN RESULTS

The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed.

SIGNIFICANCE

Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.

Unobtrusive Monitoring of Neonatal Brain Temperature Using a Zero-Heat-Flux Sensor Matrix

The temperature of preterm neonates must be maintained within a narrow window to ensure their survival. Continuously measuring their core temperature provides an optimal means of monitoring their thermoregulation and their response to environmental changes. However, existing methods of measuring core temperature can be very obtrusive, such as rectal probes, or inaccurate/lagging, such as skin temperature sensors and spot-checks using tympanic temperature sensors. This study investigates an unobtrusive method of measuring brain temperature continuously using an embedded zero-heat-flux (ZHF) sensor matrix placed under the head of the neonate. The measured temperature profile is used to segment areas of motion and incorrect positioning, where the neonate's head is not above the sensors. We compare our measurements during low motion/stable periods to esophageal temperatures for 12 preterm neonates, measured for an average of 5 h per neonate. The method we propose shows good correlation with the reference temperature for most of the neonates. The unobtrusive embedding of the matrix in the neonate's environment poses no harm or disturbance to the care work-flow, while measuring core temperature. To address the effect of motion on the ZHF measurements in the current embodiment, we recommend a more ergonomic embedding ensuring the sensors are continuously placed under the neonate's head.

Non-intrusive head movement analysis of videotaped seizures of epileptic origin

In this work we propose a non-intrusive video analytic system for patient's body parts movement analysis in Epilepsy Monitoring Unit. The system utilizes skin color modeling, head/face pose template matching and face detection to analyze and quantify the head movements. Epileptic patients' heads are analyzed holistically to infer seizure and normal random movements. The patient does not require to wear any special clothing, markers or sensors, hence it is totally non-intrusive. The user initializes the person-specific skin color and selects few face/head poses in the initial few frames. The system then tracks the head/face and extracts spatio-temporal features. Support vector machines are then used on these features to classify seizure-like movements from normal random movements. Experiments are performed on numerous long hour video sequences captured in an Epilepsy Monitoring Unit at a local hospital. The results demonstrate the feasibility of the proposed system in pediatric epilepsy monitoring and seizure detection.

Vision-based motion detection, analysis and recognition of epileptic seizures--a systematic review

The analysis of human motion from video has been the object of interest for many application areas, these including surveillance, control, biomedical analysis, video annotation etc. This paper addresses the advances within this topic in relation to epilepsy, a domain where human motion is with no doubt one of the most important elements of a patient's clinical image. It describes recent achievements in vision-based detection, analysis and recognition of human motion in epilepsy for marker-based and marker-free systems. An overview of motion-characterizing features extracted so far is presented separately. The objective is to gain existing knowledge in this field and set the route marks for the future development of an integrated decision support system for epilepsy diagnosis and disease management based on automated video analysis. This review revealed that the quantification of motion patterns of selected epileptic seizures has been studied thoroughly while the recognition of seizures is currently in its beginnings, but however feasible. Moreover, only a limited set of seizure types have been analyzed so far, indicating that a holistic approach addressing all epileptic syndromes is still missing.

Combination of EEG and ECG for improved automatic neonatal seizure detection

OBJECTIVE

Neonatal seizures are the most common central nervous system disorder in newborn infants. A system that could automatically detect the presence of seizures in neonates would be a significant advance facilitating timely medical intervention.

METHODS

A novel method is proposed for the robust detection of neonatal seizures through the combination of simultaneously-recorded electroencephalogram (EEG) and electrocardiogram (ECG). A patient-specific and a patient-independent system are considered, employing statistical classifier models.

RESULTS

Results for the signals combined are compared to results for each signal individually. For the patient-specific system, 617 of 633 (97.52%) expert-labelled seizures were correctly detected with a false detection rate of 13.18%. For the patient-independent system, 516 of 633 (81.44%) expert-labelled seizures were correctly detected with a false detection rate of 28.57%.

CONCLUSIONS

A novel algorithm for neonatal seizure detection is proposed. The combination of an ECG-based classifier system with a novel multi-channel EEG-based classifier system has led to improved seizure detection performance. The algorithm was evaluated using a large data-set containing ECG and multi-channel EEG of realistic duration and quality.

SIGNIFICANCE

Analysis of simultaneously-recorded EEG and ECG represents a new approach in seizure detection research and the detection performance of the proposed system is a significant improvement on previous reported results for automated neonatal seizure detection.

Electrocardiogram Based Neonatal Seizure Detection

A method for the detection of seizures in the newborn using the electrocardiogram (ECG) signal is presented. Using a database of eight recordings, a method was developed for automatically annotating each 1-min epoch as "nonseizure" or "seizure". The system uses a linear discriminant classifier to process 41 heartbeat timing interval features. Performance assessment of the method showed that on a patient-specific basis an average accuracy of 70.5% was achieved in detecting seizures with associated sensitivity of 62.2% and specificity of 71.8%. On a patient-independent basis the average accuracy was 68.3% with sensitivity of 54.6% and specificity of 77.3%. Shifting the decision threshold for the patient-independent classifier allowed an increase in sensitivity to 78.4% at the expense of decreased specificity (51.6%), leading to increased false detections. The results of our ECG-based method are comparable with those reported for EEG-based neonatal seizure detection systems and offer the benefit of an easier acquisition methodology for seizure detection.

Discrete optical flow estimation methods and their application in the extraction of motion strength signals from video recordings of neonatal seizures

This work presents a methodology for the development of regularized optical flow computation methods for video. The proposed methodology is based on a discrete formulation of the optical flow problem. The optical flow computation methods produced by the proposed methodology are utilized to extract temporal motion strength signals from video recordings of neonatal seizures.

Quantifying motion in video recordings of neonatal seizures by feature trackers based on predictive block matching

This work introduces predictive block matching, a method developed to track motion in video by exploiting the advantages of block motion estimation and adaptive block matching. The proposed method relies on a pure translation motion model to estimate the displacement of a block between two successive video frames before initiating the search for the best match of the block tracked throughout the frame sequence. The search for the best match relies on adaptive block matching, which employs an update strategy based on Kalman filtering to account for the changing appearance of the block. Predictive block matching was used to extract motor activity signals from video recordings of neonatal seizures.

Automated Detection of Videotaped Neonatal Seizures Based on Motion Tracking Methods

This study was carried out during the second phase of the project "Video Technologies for Neonatal Seizures" and aimed at the development of a seizure detection system by training neural networks, using quantitative motion information extracted by motion tracking methods from short video segments of infants monitored for seizures. The motion of the infants' body parts was quantified by temporal motion trajectory signals extracted from video recordings by robust motion trackers, based on block motion models. These motion trackers were developed to autonomously adjust to illumination and contrast changes that may occur during the video frame sequence. The computational tools and procedures developed for automated seizure detection were evaluated on short video segments selected and labeled by physicians from a set of 240 video recordings of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments). This evaluation provided the basis for selecting the most effective strategy for training neural networks to detect neonatal seizures as well as the decision scheme used for interpreting the responses of the trained neural networks. The best neural networks exhibited sensitivity and specificity above 90%. The best among the motion trackers developed in this study produced quantitative features that constitute a reliable basis for detecting myoclonic and focal clonic neonatal seizures. The performance targets of the second phase of the project may be achieved by combining the quantitative features described in this paper with those obtained by analyzing motion strength signals produced by motion segmentation methods.

Automated detection of videotaped neonatal seizures based on motion segmentation methods

OBJECTIVE

This study was aimed at the development of a seizure detection system by training neural networks using quantitative motion information extracted by motion segmentation methods from short video recordings of infants monitored for seizures.

METHODS

The motion of the infants' body parts was quantified by temporal motion strength signals extracted from video recordings by motion segmentation methods based on optical flow computation. The area of each frame occupied by the infants' moving body parts was segmented by direct thresholding, by clustering of the pixel velocities, and by clustering the motion parameters obtained by fitting an affine model to the pixel velocities. The computational tools and procedures developed for automated seizure detection were tested and evaluated on 240 short video segments selected and labeled by physicians from a set of video recordings of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments).

RESULTS

The experimental study described in this paper provided the basis for selecting the most effective strategy for training neural networks to detect neonatal seizures as well as the decision scheme used for interpreting the responses of the trained neural networks. Depending on the decision scheme used for interpreting the responses of the trained neural networks, the best neural networks exhibited sensitivity above 90% or specificity above 90%.

CONCLUSIONS

The best among the motion segmentation methods developed in this study produced quantitative features that constitute a reliable basis for detecting myoclonic and focal clonic neonatal seizures. The performance targets of this phase of the project may be achieved by combining the quantitative features described in this paper with those obtained by analyzing motion trajectory signals produced by motion tracking methods.

SIGNIFICANCE

A video system based upon automated analysis potentially offers a number of advantages. Infants who are at risk for seizures could be monitored continuously using relatively inexpensive and non-invasive video techniques that supplement direct observation by nursery personnel. This would represent a major advance in seizure surveillance and offers the possibility for earlier identification of potential neurological problems and subsequent intervention.

Automated Detection of Videotaped Neonatal Seizures of Epileptic Origin

PURPOSE

This study aimed at the development of a seizure-detection system by training neural networks with quantitative motion information extracted from short video segments of neonatal seizures of the myoclonic and focal clonic types and random infant movements.

METHODS

The motion of the infants' body parts was quantified by temporal motion-strength signals extracted from video segments by motion-segmentation methods based on optical flow computation. The area of each frame occupied by the infants' moving body parts was segmented by clustering the motion parameters obtained by fitting an affine model to the pixel velocities. The motion of the infants' body parts also was quantified by temporal motion-trajectory signals extracted from video recordings by robust motion trackers based on block-motion models. These motion trackers were developed to adjust autonomously to illumination and contrast changes that may occur during the video-frame sequence. Video segments were represented by quantitative features obtained by analyzing motion-strength and motion-trajectory signals in both the time and frequency domains. Seizure recognition was performed by conventional feed-forward neural networks, quantum neural networks, and cosine radial basis function neural networks, which were trained to detect neonatal seizures of the myoclonic and focal clonic types and to distinguish them from random infant movements.

RESULTS

The computational tools and procedures developed for automated seizure detection were evaluated on a set of 240 video segments of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments). Regardless of the decision scheme used for interpreting the responses of the trained neural networks, all the neural network models exhibited sensitivity and specificity>90%. For one of the decision schemes proposed for interpreting the responses of the trained neural networks, the majority of the trained neural-network models exhibited sensitivity>90% and specificity>95%. In particular, cosine radial basis function neural networks achieved the performance targets of this phase of the project (i.e., sensitivity>95% and specificity>95%).

CONCLUSIONS

The best among the motion segmentation and tracking methods developed in this study produced quantitative features that constitute a reliable basis for detecting neonatal seizures. The performance targets of this phase of the project were achieved by combining the quantitative features obtained by analyzing motion-strength signals with those produced by analyzing motion-trajectory signals. The computational procedures and tools developed in this study to perform off-line analysis of short video segments will be used in the next phase of this project, which involves the integration of these procedures and tools into a system that can process and analyze long video recordings of infants monitored for seizures in real time.

Quantifying motion in video recordings of neonatal seizures by regularized optical flow methods

This paper presents the development of regularized optical flow computation methods and an evaluation of their performance in the extraction of quantitative motion information from video recordings of neonatal seizures. A general formulation of optical flow computation is presented and a mathematical framework for the development of practical tools for computing optical flow is outlined. In addition, this paper proposes an alternative formulation of the optical flow problem that relies on a discrete approximation of a family of quadratic functionals. These regularized optical flow computation methods are used to extract motion strength signals from video recordings of neonatal seizures.

Quantifying Motion in Video Recordings of Neonatal Seizures by Robust Motion Trackers Based on Block Motion Models

This paper introduces a methodology for the development of robust motion trackers for video based on block motion models. According to this methodology, the motion of a site between two successive frames is estimated by minimizing an error function defined in terms of the intensities at these frames. The proposed methodology is used to develop robust motion trackers that rely on fractional block motion models. The motion trackers developed in this paper are utilized to extract motor activity signals from video recordings of neonatal seizures. The experimental results reveal that the proposed motion trackers are more accurate and reliable than existing motion tracking methods relying on pure translation and affine block motion models.

Computerized Motion Analysis of Videotaped Neonatal Seizures of Epileptic Origin

PURPOSE

The main objective of this research is the development of automated video processing and analysis procedures aimed at the recognition and characterization of the types of neonatal seizures. The long-term goal of this research is the integration of these computational procedures into the development of a stand-alone automated system that could be used as a supplement in the neonatal intensive care unit (NICU) to provide 24-h per day noninvasive monitoring of infants at risk for seizures.

METHODS

We developed and evaluated a variety of computational tools and procedures that may be used to carry out the three essential tasks involved in the development of a seizure recognition and characterization system: the extraction of quantitative motion information from video recordings of neonatal seizures in the form of motion-strength and motor-activity signals, the selection of quantitative features that convey some unique behavioral characteristics of neonatal seizures, and the training of artificial neural networks to distinguish neonatal seizures from random infant behaviors and to differentiate between myoclonic and focal clonic seizures.

RESULTS

The methods were tested on a set of 240 video recordings of 43 patients exhibiting myoclonic seizures (80 cases), focal clonic seizures (80 cases), and random infant movements (80 cases). The outcome of the experiments verified that optical- flow methods are promising computational tools for quantifying neonatal seizures from video recordings in the form of motion-strength signals. The experimental results also verified that the robust motion trackers developed in this study outperformed considerably the motion trackers based on predictive block matching in terms of both reliability and accuracy. The quantitative features selected from motion-strength and motor-activity signals constitute a satisfactory representation of neonatal seizures and random infant movements and seem to be complementary. Such features lead to trained neural networks that exhibit performance levels exceeding the initial goals of this study, the sensitivity goal being >or=80% and the specificity goal being >or=90%.

CONCLUSIONS

The outcome of this experimental study provides strong evidence that it is feasible to develop an automated system for the recognition and characterization of the types of neonatal seizures based on video recordings. This will be accomplished by enhancing the accuracy and improving the reliability of the computational tools and methods developed during the course of the study outlined here.

Automated Extraction of Temporal Motor Activity Signals From Video Recordings of Neonatal Seizures Based on Adaptive Block Matching

This paper presents an automated procedure developed to extract quantitative information from video recordings of neonatal seizures in the form of motor activity signals. This procedure relies on optical flow computation to select anatomical sites located on the infants' body parts. Motor activity signals are extracted by tracking selected anatomical sites during the seizure using adaptive block matching. A block of pixels is tracked throughout a sequence of frames by searching for the most similar block of pixels in subsequent frames; this search is facilitated by employing various update strategies to account for the changing appearance of the block. The proposed procedure is used to extract temporal motor activity signals from video recordings of neonatal seizures and other events not associated with seizures.

Improving the Accuracy and Reliability of Motion Tracking Methods Used for Extracting Temporal Motor Activity Signals From Video Recordings of Neonatal Seizures

This paper presents an approach for improving the accuracy and reliability of motion tracking methods developed for video based on block motion models. This approach estimates the displacement of a block of pixels between two successive frames by minimizing an error function defined in terms of the pixel intensities at these frames. The minimization problem is made analytically tractable by approximating the error function using a second-order Taylor expansion. The improved reliability of the proposed method is illustrated by its application in the extraction of temporal motor activity signals from video recordings of neonatal seizures.

Extraction of motion strength and motor activity signals from video recordings of neonatal seizures

This paper presents two methods developed to extract quantitative information from video recordings of neonatal seizures in the form of temporal motion strength and motor activity signals. Motion strength signals are extracted by measuring the area of the body parts that move during the seizure and the relative speed of motion using a combination of spatiotemporal subband decomposition of video, nonlinear filtering, and segmentation. Motor activity signals are extracted by tracking selected anatomical sites during the seizure using a modified version of a feature-tracking procedure developed for video, known as the Kanade-Lucas-Tomasi (KLT) algorithm. The experiments indicate that the temporal signals produced by the proposed methods provide the basis for differentiating myoclonic from focal clonic seizures and distinguishing these types of neonatal seizures from normal infant behaviors.