Monitoring infants by automatic video processing: A unified approach to motion analysis

Reimagining apnea monitoring in the neonatal ICU

PURPOSE OF REVIEW

This review outlines the prevalence and complications of apneas and intermittent hypoxemic events in preterm infants, examines current monitoring limitations in neonatal ICUs (NICUs), and explores emerging technologies addressing these challenges.

RECENT FINDINGS

New evidence from the Prematurity-Related Ventilatory Control (Pre-Vent) study, which analyzed cardiorespiratory data from 717 extremely preterm infants, exposes the varying frequency, duration, and severity of apneas, intermittent hypoxemia, bradycardias, and periodic breathing during hospitalization, and highlights the negative impact of intermittent hypoxemia on pulmonary outcomes at discharge. Although traditional monitoring methods cannot differentiate between apnea types and quantify their burden, recent advancements in sensor technologies and data integration hold promise for improving real-time detection and evaluation of apneas in the NICU. Notably, small wearable mechano-acoustic sensors could improve apnea monitoring through continuous detection of airflow and respiratory efforts. Additionally, integrating bedside physiological data with modalities such as near-infrared spectroscopy, diaphragmatic activity, and electrical impedance tomography could help predict adverse outcomes by monitoring regional oxygen saturation and lung function in relation to apneas.

SUMMARY

Enhancing our understanding of neonatal apneas and overcoming the current limitations in apnea monitoring through advanced sensor technologies and data integration could lead to more personalized management and improved outcomes for preterm infants.

Veni, Vidi, Vici-When Is Home Video Seizure Monitoring Helpful?

Seizure detection is vital for managing epilepsy as seizures can lead to injury and even death, in addition to impacting quality of life. Prompt detection of seizures and intervention can help prevent injury and improve outcomes for individuals with epilepsy. Wearable sensors show promising results for automated detection of certain seizures, but they have limitations such as patient tolerance, impracticality for newborns, and the need for recharging. Non-contact video and audio-based technologies have become available, but a comprehensive literature review on these methods is lacking. This scoping literature review provides an overview of video and audio-based seizure detection, highlighting their potential benefits and challenges. It encompasses a thorough search and evaluation of relevant articles, summarizing methods and performances of these systems. The primary aim of this review is to examine and analyze existing research to identify patterns and gaps and establish a foundation for future advancements. We screened 7 databases using a set of standardized search criteria to minimize any potential missed articles. Four thousand four hundred eighty-seven deduplicated abstracts were screened and narrowed down to 34 studies that varied in design, algorithm methods, types of seizures detected, and performance metrics. Seizure detection sensitivity ranged from 100% to 0%, with optical flow analysis showing the highest sensitivity. The specificity of all included articles ranged from 97.7% to 60%. While limited studies reported accuracy, the highest reported was 100% using Radon Transform based technique on Dual Tree Complex Wavelet coefficients. Video and audio-based tools offer novel, noncontact approaches for detecting and monitoring seizures. Available studies are limited in sample sizes, dataset diversity, and standardized evaluation protocols, impacting the generalizability of results. Future research focusing on larger-scale investigations with diverse datasets, standardized evaluation protocols, and consistent reporting metrics is needed.

Neonatal apnea and hypopnea prediction in infants with Robin sequence with neural additive models for time series

Neonatal apneas and hypopneas present a serious risk for healthy infant development. Treating these adverse events requires frequent manual stimulation by skilled personnel, which can lead to alarm fatigue. This study aims to develop and validate an interpretable model that can predict apneas and hypopneas. Automatically predicting these adverse events before they occur would enable the use of methods for automatic intervention. We propose a neural additive model to predict individual occurrences of neonatal apnea and hypopnea and apply it to a physiological dataset from infants with Robin sequence at risk of upper airway obstruction. The dataset will be made publicly available together with this study. Our proposed model allows the prediction of individual apneas and hypopneas, achieving an average AuROC of 0.80 when discriminating segments of polysomnography recordings starting 15 seconds before the onset of apneas and hypopneas from control segments. Its additive nature makes the model inherently interpretable, which allowed insights into how important a given signal modality is for prediction and which patterns in the signal are discriminative. For our problem of predicting apneas and hypopneas in infants with Robin sequence, prior irregularities in breathing-related modalities as well as decreases in SpO2 levels were especially discriminative. Our prediction model presents a step towards an automatic prediction of neonatal apneas and hypopneas in infants at risk for upper airway obstruction. Together with the publicly released dataset, it has the potential to facilitate the development and application of methods for automatic intervention in clinical practice.

Characteristics of motion signal profiles of tonic-clonic, tonic, hyperkinetic, and motor seizures extracted from nocturnal video recordings

OBJECTIVE

In this study, characteristics of signal profiles formed by motion, oscillation, and sound signals were analyzed to evaluate generalizability and variability in a single patient setting (intra-patient variability) and between patients (inter-patient variability). As a secondary objective, the effect of brivaracetam intervention on signal profiles was explored.

METHODS

Patient data included 13 hyperkinetic seizures, 65 tonic seizures, 13 tonic-clonic seizures, and 138 motor seizures from 11 patients. All patients underwent an 8-week monitoring, and after a 3-week baseline, brivaracetam was initiated. Motion, oscillation, and sound features extracted from the video were used to form signal profiles. Variance of signals was calculated, and combined median and quartile visualizations were used to visualize the results. Similarly, the effect of intervention was visualized.

RESULTS

Hyperkinetic motion signals showed a rapid increase in motion and sound signals without oscillations and achieved low intra-patient variance. Tonic component created a recognizable peak in motion signal typical for tonic and tonic-clonic seizures. For tonic seizures, inter-patient variance was low. Motor signal profiles were varying, and they did not form a generalizable signal profile. Visually recognizable changes were observed in the signal profiles of two patients.

SIGNIFICANCE

Video-based motion signal analysis enabled the extraction of motion features characteristic for different motor seizure types which might be useful in further development of this system. Tonic component formed a recognizable seizure signature in the motion signal. Hyperkinetic and motor seizures may have not only significantly different motion signal amplitude but also overlapping signal profile characteristics which might hamper their automatic differentiation. Motion signals might be useful in the assessment of movement intensity changes to evaluate the treatment effect. Further research is needed to test generalizability and to increase reliability of the results.

Adapting Action Recognition Neural Networks for Automated Infantile Spasm Detection

Infantile spasms are a severe epileptic syndrome characterized by short muscular contractions lasting from 0.5 to 2 seconds. They are often misdiagnosed due to their atypical presentation, and treatment is frequently delayed, leading to stagnation or regression in psychomotor development and significant cognitive and motor sequelae. One promising approach to addressing this issue is the use of markerless computer vision techniques. In this paper, we introduce a novel approach for recognizing infantile spasms based exclusively on video data. We utilize an expanded 3D neural network pre-trained on an extensive human action recognition dataset called Kinetics. By employing this model, we extract features from short segments of varying sizes sampled from seizure videos, which allows us to effectively capture the spatio-temporal characteristics of infantile spasms. We then apply multiple classifiers to perform binary classification on these extracted features. The best system achieved an average area under the ROC curve of 0.813±0.058 for a 3-second window.

A nomogram for predicting neonatal apnea: a retrospective analysis based on the MIMIC database

Introduction

The objective of this study is to develop a model based on indicators in the routine examination of neonates to effectively predict neonatal apnea.

Methods

We retrospectively analysed 8024 newborns from the MIMIC IV database, building logistic regression models and decision tree models. The performance of the model is examined by decision curves, calibration curves and ROC curves. Variables were screened by stepwise logistic regression analysis and LASSO regression.

Results

A total of 7 indicators were ultimately included in the model: gestational age, birth weight, ethnicity, gender, monocytes, lymphocytes and acetaminophen. The mean AUC (the area under the ROC curve) of the 5-fold cross-validation of the logistic regression model in the training set and the AUC in the validation set are 0.879 and 0.865, respectively. The mean AUC (the area under the ROC curve) of the 5-fold cross-validation of the decision tree model in the training set and the AUC in the validation set are 0.861 and 0.850, respectively. The calibration and decision curves in the two cohorts also demonstrated satisfactory predictive performance of the model. However, the logistic regression model performs relatively well.

Discussion

Our results proved that blood indicators were valuable and effective predictors of neonatal apnea, which could provide effective predictive information for medical staff.

Computer vision for automated seizure detection and classification: A systematic review

Computer vision (CV) shows increasing promise as an efficient, low-cost tool for video seizure detection and classification. Here, we provide an overview of the fundamental concepts needed to understand CV and summarize the structure and performance of various model architectures used in video seizure analysis. We conduct a systematic literature review of the PubMed, Embase, and Web of Science databases from January 1, 2000 to September 15, 2023, to identify the strengths and limitations of CV seizure analysis methods and discuss the utility of these models when applied to different clinical seizure phenotypes. Reviews, nonhuman studies, and those with insufficient or poor quality data are excluded from the review. Of the 1942 records identified, 45 meet inclusion criteria and are analyzed. We conclude that the field has shown tremendous growth over the past 2 decades, leading to several model architectures with impressive accuracy and efficiency. The rapid and scalable detection offered by CV models holds the potential to reduce sudden unexpected death in epilepsy and help alleviate resource limitations in epilepsy monitoring units. However, a lack of standardized, thorough validation measures and concerns about patient privacy remain important obstacles for widespread acceptance and adoption. Investigation into the performance of models across varied datasets from clinical and nonclinical environments is an essential area for further research.

Automated analysis and detection of epileptic seizures in video recordings using artificial intelligence

Introduction

Automated seizure detection promises to aid in the prevention of SUDEP and improve the quality of care by assisting in epilepsy diagnosis and treatment adjustment.

Methods

In this phase 2 exploratory study, the performance of a contactless, marker-free, video-based motor seizure detection system is assessed, considering video recordings of patients (age 0-80 years), in terms of sensitivity, specificity, and Receiver Operating Characteristic (ROC) curves, with respect to video-electroencephalographic monitoring (VEM) as the medical gold standard. Detection performances of five categories of motor epileptic seizures (tonic-clonic, hyperkinetic, tonic, unclassified motor, automatisms) and psychogenic non-epileptic seizures (PNES) with a motor behavioral component lasting for >10 s were assessed independently at different detection thresholds (rather than as a categorical classification problem). A total of 230 patients were recruited in the study, of which 334 in-scope (>10 s) motor seizures (out of 1,114 total seizures) were identified by VEM reported from 81 patients. We analyzed both daytime and nocturnal recordings. The control threshold was evaluated at a range of values to compare the sensitivity (n = 81 subjects with seizures) and false detection rate (FDR) (n = all 230 subjects).

Results

At optimal thresholds, the performance of seizure groups in terms of sensitivity (CI) and FDR/h (CI): tonic-clonic- 95.2% (82.4, 100%); 0.09 (0.077, 0.103), hyperkinetic- 92.9% (68.5, 98.7%); 0.64 (0.59, 0.69), tonic- 78.3% (64.4, 87.7%); 5.87 (5.51, 6.23), automatism- 86.7% (73.5, 97.7%); 3.34 (3.12, 3.58), unclassified motor seizures- 78% (65.4, 90.4%); 4.81 (4.50, 5.14), and PNES- 97.7% (97.7, 100%); 1.73 (1.61, 1.86). A generic threshold recommended for all motor seizures under study asserted 88% sensitivity and 6.48 FDR/h.

Discussion

These results indicate an achievable performance for major motor seizure detection that is clinically applicable for use as a seizure screening solution in diagnostic workflows.

Acute symptomatic seizures in newborns: a narrative review

Acute symptomatic seizures are the main sign of neurological dysfunction in newborns. This is linked to the unique characteristics of the neonatal brain, making it hyperexcitable compared to older ages, and to the common occurrence of some forms of acquired brain injury, namely hypoxic-ischemic encephalopathy. In this narrative review we will provide an overview of neonatal seizures definition, their main underlying etiologies, diagnostic work-up and differential diagnoses, and will discuss about therapeutic options and prognostic outlook. The latest publications from the ILAE Task Force on Neonatal Seizures will be presented and discussed. Of note, they highlight the current lack of robust evidence in this field of clinical neurology. We will also report on specificities pertaining to low-and-middle income countries in terms of incidence, main etiologies and diagnosis. The possibilities offered by telemedicine and automated seizures detection will also be summarized in order to provide a framework for future directions in seizures diagnosis and management with a global perspective. Many challenges and opportunities for improving identification, monitoring and treatment of acute symptomatic seizures in newborns exist. All current caveats potentially represent different lines of research with the aim to provide better care and reach a deeper understanding of this important topic of neonatal neurology.

Multimodality Video Acquisition System for the Assessment of Vital Distress in Children

In children, vital distress events, particularly respiratory, go unrecognized. To develop a standard model for automated assessment of vital distress in children, we aimed to construct a prospective high-quality video database for critically ill children in a pediatric intensive care unit (PICU) setting. The videos were acquired automatically through a secure web application with an application programming interface (API). The purpose of this article is to describe the data acquisition process from each PICU room to the research electronic database. Using an Azure Kinect DK and a Flir Lepton 3.5 LWIR attached to a Jetson Xavier NX board and the network architecture of our PICU, we have implemented an ongoing high-fidelity prospectively collected video database for research, monitoring, and diagnostic purposes. This infrastructure offers the opportunity to develop algorithms (including computational models) to quantify vital distress in order to evaluate vital distress events. More than 290 RGB, thermographic, and point cloud videos of each 30 s have been recorded in the database. Each recording is linked to the patient's numerical phenotype, i.e., the electronic medical health record and high-resolution medical database of our research center. The ultimate goal is to develop and validate algorithms to detect vital distress in real time, both for inpatient care and outpatient management.

An Evaluation of Non-Contact Photoplethysmography-Based Methods for Remote Respiratory Rate Estimation

The respiration rate (RR) is one of the physiological signals deserving monitoring for assessing human health and emotional states. However, traditional devices, such as the respiration belt to be worn around the chest, are not always a feasible solution (e.g., telemedicine, device discomfort). Recently, novel approaches have been proposed aiming at estimating RR in a less invasive yet reliable way, requiring the acquisition and processing of contact or remote Photoplethysmography (contact reference and remote-PPG, respectively). The aim of this paper is to address the lack of systematic evaluation of proposed methods on publicly available datasets, which currently impedes a fair comparison among them. In particular, we evaluate two prominent families of PPG processing methods estimating Respiratory Induced Variations (RIVs): the first encompasses methods based on the direct extraction of morphological features concerning the RR; and the second group includes methods modeling respiratory artifacts adopting, in the most promising cases, single-channel blind source separation. Extensive experiments have been carried out on the public BP4D+ dataset, showing that the morphological estimation of RIVs is more reliable than those produced by a single-channel blind source separation method (both in contact and remote testing phases), as well as in comparison with a representative state-of-the-art Deep Learning-based approach for remote respiratory information estimation.

Methods for Detecting Abnormal Ventilation in Children - the Case Study of 13-Years old Pitt-Hopkins Girl

We present contactless technology measuring abnormal ventilation and compare it with polysomnography (PSG). A 13-years old girl with Pitt-Hopkins syndrome presented hyperpnoea periods with apneic spells. The PSG was conducted simultaneously with Emfit movement sensor (Emfit, Finland) and video camera with depth sensor (NEL, Finland). The respiratory efforts from PSG, Emfit sensor, and NEL were compared. In addition, we measured daytime breathing with tracheal microphone (PneaVox,France). The aim was to deepen the knowledge of daytime hyperpnoea periods and ensure that no upper airway obstruction was present during sleep. The signs of upper airway obstruction were not detected despite of minor sleep time. Monitoring respiratory effort with PSG is demanding in all patient groups. The used unobtrusive methods were capable to reveal breathing frequency and hyperpnoea periods. Every day diagnostics need technology like this for monitoring vital signs at hospital wards and at home from subjects with disabilities and co-operation difficulties.

Heart Rate Variability Analysis for Seizure Detection in Neonatal Intensive Care Units

In Neonatal Intensive Care Units (NICUs), the early detection of neonatal seizures is of utmost importance for a timely clinical intervention. Over the years, several neonatal seizure detection systems were proposed to detect neonatal seizures automatically and speed up seizure diagnosis, most based on the EEG signal analysis. Recently, research has focused on other possible seizure markers, such as electrocardiography (ECG). This work proposes an ECG-based NSD system to investigate the usefulness of heart rate variability (HRV) analysis to detect neonatal seizures in the NICUs. HRV analysis is performed considering time-domain, frequency-domain, entropy and multiscale entropy features. The performance is evaluated on a dataset of ECG signals from 51 full-term babies, 29 seizure-free. The proposed system gives results comparable to those reported in the literature: Area Under the Receiver Operating Characteristic Curve = 62%, Sensitivity = 47%, Specificity = 67%. Moreover, the system’s performance is evaluated in a real clinical environment, inevitably affected by several artefacts. To the best of our knowledge, our study proposes for the first time a multi-feature ECG-based NSD system that also offers a comparative analysis between babies suffering from seizures and seizure-free ones.

Video-Based Automatic Baby Motion Analysis for Early Neurological Disorder Diagnosis: State of the Art and Future Directions

Neurodevelopmental disorders (NDD) are impairments of the growth and development of the brain and/or central nervous system. In the light of clinical findings on early diagnosis of NDD and prompted by recent advances in hardware and software technologies, several researchers tried to introduce automatic systems to analyse the baby's movement, even in cribs. Traditional technologies for automatic baby motion analysis leverage contact sensors. Alternatively, remotely acquired video data (e.g., RGB or depth) can be used, with or without active/passive markers positioned on the body. Markerless approaches are easier to set up and maintain (without any human intervention) and they work well on non-collaborative users, making them the most suitable technologies for clinical applications involving children. On the other hand, they require complex computational strategies for extracting knowledge from data, and then, they strongly depend on advances in computer vision and machine learning, which are among the most expanding areas of research. As a consequence, also markerless video-based analysis of movements in children for NDD has been rapidly expanding but, to the best of our knowledge, there is not yet a survey paper providing a broad overview of how recent scientific developments impacted it. This paper tries to fill this gap and it lists specifically designed data acquisition tools and publicly available datasets as well. Besides, it gives a glimpse of the most promising techniques in computer vision, machine learning and pattern recognition which could be profitably exploited for children motion analysis in videos.

Bionic for Training: Smart Framework Design for Multisensor Mechatronic Platform Validation

Home monitoring supports the continuous improvement of the therapy by sharing data with healthcare professionals. It is required when life-threatening events can still occur after hospital discharge such as neonatal apnea. However, multiple sources of external noise could affect data quality and/or increase the misdetection rate. In this study, we developed a mechatronic platform for sensor characterizations and a framework to manage data in the context of neonatal apnea. The platform can simulate the movement of the abdomen in different plausible newborn positions by merging data acquired simultaneously from three-axis accelerometers and infrared sensors. We simulated nine apnea conditions combining three different linear displacements and body postures in the presence of self-generated external noise, showing how it is possible to reduce errors near to zero in phenomena detection. Finally, the development of a smart 8Ws-based software and a customizable mobile application were proposed to facilitate data management and interpretation, classifying the alerts to guarantee the correct information sharing without specialized skills.

RGB-D scene analysis in the NICU

Continuity of care is achieved in the neonatal intensive care unit (NICU) through careful documentation of all events of clinical significance, including clinical interventions and routine care events (e.g., feeding, diaper change, weighing, etc.). As a step towards automating this documentation process, we propose a scene recognition algorithm that can automatically identify key features in a single image of the patient environment, paired with a rule-based sentence generator to caption the scene. Color and depth video were obtained from 29 newborn patients from the Children's Hospital of Eastern Ontario (CHEO) using an Intel RealSense SR300 RGB-D camera and manual bedside event annotation. Image processing techniques are implemented to classify two lighting conditions: brightness level and phototherapy. A deep neural network is developed for three image classification tasks: on-going intervention, bed occupancy, and patient coverage. Transfer learning is leveraged in the feature extraction layers, such that weights learned from a generic data-rich task are applied to the clinical domain where data collection is complex and costly. Different depth fusion techniques are implemented and compared among classification tasks, where the depth and color data are fused as an RGB-D image (image fusion) or separately at various layers in the network (network fusion). Promising results were obtained with >84% sensitivity and >73% F1 measure across all context variables despite the large class imbalance. RGBD-based models are shown to outperform RGB models on most tasks. In general, a 4-channel image fusion and network fusion at the 11th layer of the VGG-16 architecture were preferred. Ultimately, achieving complete scene understanding through multimodal computer vision could form the basis for a semi-automated charting system to assist clinical staff.

Predicting apneic events in preterm infants using cardio-respiratory and movement features

BACKGROUND AND OBJECTIVE

Preterm neonates are prone to episodes of apnea, bradycardia and hypoxia (ABH) that can lead to neurological morbidities or even death. There is broad interest in developing methods for real-time prediction of ABH events to inform interventions that prevent or reduce their incidence and severity. Using advances in machine learning methods, this study develops an algorithm to predict ABH events.

METHODS

Following previous studies showing that respiratory instabilities are closely associated with bouts of movement, we present a modeling framework that can predict ABH events using both movement and cardio-respiratory features derived from routine clinical recordings. In 10 preterm infants, movement onsets and durations were estimated with a wavelet-based algorithm that quantified artifactual distortions of the photoplethysmogram signal. For prediction, cardio-respiratory features were created from time-delayed correlations of inter-beat and inter-breath intervals with past values; movement features were derived from time-delayed correlations with inter-breath intervals. Gaussian Mixture Models and Logistic Regression were used to develop predictive models of apneic events. Performance of the models was evaluated with ROC curves.

RESULTS

Performance of the prediction framework (mean AUC) was 0.77 ± 0.04 for 66 ABH events on training data from 7 infants. When grouped by the severity of the associated bradycardia during the ABH event, the framework was able to predict 83% and 75% of the most severe episodes in the 7-infant training set and 3-infant test set, respectively. Notably, inclusion of movement features significantly improved the predictions compared with modeling with only cardio-respiratory signals.

CONCLUSIONS

Our findings suggest that recordings of movement provide important information for predicting ABH events in preterm infants, and can inform preemptive interventions designed to reduce the incidence and severity of ABH events.

Video-based neonatal state assessment method for timing of procedures

BACKGROUND

Procedures should be performed when an infant is most receptive to disruptions in order to reduce the stress on the infant. However, frequent direct observations place a heavy burden on medical staff. There is therefore a need for a method for quantitatively and automatically evaluating the neonatal state.

METHODS

Ten infants in our hospital were enrolled in this study. The states of the infants were assessed by medical staff using the Brazelton Neonatal Behavioral Assessment Scale and were recorded on video at the same time. The recorded states were reclassified as activity levels, a new state classification method that includes middle activity, which is the appropriate time for a procedure. Using image analysis, motions of the infant were quantified as two indices: activity and pause time. Activity and pause time were compared for each activity level. The cutoff values of the indices were calculated, and the sensitivity and specificity of the middle activity were calculated.

RESULTS

There was a significant difference between all groups of activity level (P < 0.01). The maximum sensitivity and specificity of middle activity were 71.7% and 51.2%, respectively.

CONCLUSIONS

The neonatal state of infants can be quantitatively and automatically evaluated using video cameras, and the activity level can be used to determine an appropriate time for procedures in infants. This will reduce the burden on medical staff and lead to less stressful procedures for infants.

Detection of Breathing Movements of Preterm Neonates by Recording Their Abdominal Movements with a Time-of-Flight Camera

In order to deliver an aerosolized drug in a breath-triggered manner, the initiation of the patient’s inspiration needs to be detected. The best-known systems monitoring breathing patterns are based on flow sensors. However, due to their large dead space volume, flow sensors are not advisable for monitoring the breathing of (preterm) neonates. Newly-developed respiratory sensors, especially when contact-based (invasive), can be tested on (preterm) neonates only with great effort due to clinical and ethical hurdles. Therefore, a physiological model is highly desirable to validate these sensors. For developing such a system, abdominal movement data of (preterm) neonates are required. We recorded time sequences of five preterm neonates’ abdominal movements with a time-of-flight camera and successfully extracted various breathing patterns and respiratory parameters. Several characteristic breathing patterns, such as forced breathing, sighing, apnea and crying, were identified from the movement data. Respiratory parameters, such as duration of inspiration and expiration, as well as respiratory rate and breathing movement over time, were also extracted. This work demonstrated that respiratory parameters of preterm neonates can be determined without contact. Therefore, such a system can be used for breathing detection to provide a trigger signal for breath-triggered drug release systems. Furthermore, based on the recorded data, a physiological abdominal movement model of preterm neonates can now be developed.

Voxyvi: A system for long-term audio and video acquisitions in neonatal intensive care units

BACKGROUND

In the European Union, 300,000 newborn babies are born prematurely every year. Their care is ensured in Neonatal Intensive Care Units (NICU) where vital signs are constantly monitored. In addition, other descriptors such as motion, facial and vocal activities have been shown to be essential to assess neurobehavioral development.

AIM

In the scope of the European project Digi-NewB, we aimed to develop and evaluate a new audio-video device designed to non-invasively acquire multi-modal data (audio, video and thermal images), while fitting the wide variety of bedding environment in NICU.

METHODS

Firstly, a multimodal system and associated software and guidelines to collect data in neonatal intensive care unit were proposed. Secondly, methods for post-evaluation of the acquisition phase were developed, including the study of clinician feedback and a qualitative analysis of the data.

RESULTS

The deployment of 19 acquisition devices in six French hospitals allowed to record more than 500 newborns of different gestational and postmenstrual ages. After the acquisition phase, clinical feedback was mostly positive. In addition, quality of more than 300 recordings was inspected and showed that 77% of the data is exploitable. In depth, the percentage of sole presence of the newborn was estimated at 62% within recordings.

CONCLUSIONS

This study demonstrates that audio-video acquisitions are feasible on a large scale in real life in NICU. The experience also allowed us to make a clear observation of the requirements and challenges that will have to be overcome in order to set up audio-video monitoring methods.

Machine Learning-Based Automatic Classification of Video Recorded Neonatal Manipulations and Associated Physiological Parameters: A Feasibility Study

Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO₂)) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates <32 weeks’ gestation, diaper changes were associated with significant changes in HR and SpO₂, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.

Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.

Automated video-based detection of nocturnal motor seizures in children

Seizure detection devices can improve epilepsy care, but wearables are not always tolerated. We previously demonstrated good performance of a real‐time video‐based algorithm for detection of nocturnal convulsive seizures in adults with learning disabilities. The algorithm calculates the relative frequency content based on the group velocity reconstruction from video‐sequence optical flow. We aim to validate the video algorithm on nocturnal motor seizures in a pediatric population. We retrospectively analyzed the algorithm performance on a database including 1661 full recorded nights of 22 children (age = 3‐17 years) with refractory epilepsy at home or in a residential care setting. The algorithm detected 118 of 125 convulsions (median sensitivity per participant = 100%, overall sensitivity = 94%, 95% confidence interval = 61%‐100%) and identified all 135 hyperkinetic seizures. Most children had no false alarms; 81 false alarms occurred in six children (median false alarm rate [FAR] per participant per night = 0 [range = 0‐0.47], overall FAR = 0.05 per night). Most false alarms (62%) were behavior‐related (eg, awake and playing in bed). Our noncontact detection algorithm reliably detects nocturnal epileptic events with only a limited number of false alarms and is suitable for real‐time use.

The yield of long-term electrocardiographic recordings in refractory focal epilepsy

OBJECTIVE

To determine the incidence of clinically relevant arrhythmias in refractory focal epilepsy and to assess the potential of postictal arrhythmias as risk markers for sudden unexpected death in epilepsy (SUDEP).

METHODS

We recruited people with refractory focal epilepsy without signs of ictal asystole and who had at least one focal seizure per month and implanted a loop recorder with 2-year follow-up. The devices automatically record arrhythmias. Subjects and caregivers were instructed to make additional peri-ictal recordings. Clinically relevant arrhythmias were defined as asystole ≥ 6 seconds; atrial fibrillation < 55 beats per minute (bpm), or > 200 bpm and duration > 30 seconds; persistent sinus bradycardia < 40 bpm while awake; and second- or third-degree atrioventricular block and ventricular tachycardia/fibrillation. We performed 12-lead electrocardiography (ECG) and tilt table testing to identify non-seizure-related causes of asystole.

RESULTS

We included 49 people and accumulated 1060 months of monitoring. A total of 16 474 seizures were reported, of which 4679 were captured on ECG. No clinically relevant arrhythmias were identified. Three people had a total of 18 short-lasting (<6 seconds) periods of asystole, resulting in an incidence of 2.91 events per 1000 patient-months. None of these coincided with a reported seizure; one was explained by micturition syncope. Other non-clinically relevant arrhythmias included paroxysmal atrial fibrillation (n = 2), supraventricular tachycardia (n = 1), and sinus tachycardia with a right bundle branch block configuration (n = 1).

SIGNIFICANCE

We found no clinically relevant arrhythmias in people with refractory focal epilepsy during long-term follow-up. The absence of postictal arrhythmias does not support the use of loop recorders in people at high SUDEP risk.

Video and audio processing in paediatrics: a review

OBJECTIVE

Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU).

APPROACH

For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts-full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization.

MAIN RESULTS

The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed.

SIGNIFICANCE

Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.