Combining hidden Markov models for comparing the dynamics of multiple sleep electroencephalograms

Combining mixed effects hidden Markov models with latent alternating recurrent event processes to model diurnal active-rest cycles

Data collected from wearable devices can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating processes, between the state of active use and the state of being idle. Markov chains and alternating recurrent event models are commonly used to model state transitions in cases such as these, and the incorporation of random effects can be used to introduce diurnal effects. While state labels can be derived prior to modeling dynamics, this approach omits informative regression covariates that can influence state memberships. We instead propose an alternating recurrent event proportional hazards (PH) regression to model the transitions between latent states. We propose an expectation-maximization algorithm for imputing latent state labels and estimating parameters. We show that our E-step simplifies to the hidden Markov model (HMM) forward-backward algorithm, allowing us to recover an HMM with logistic regression transition probabilities. In addition, we show that PH modeling of discrete-time transitions implicitly penalizes the logistic regression likelihood and results in shrinkage estimators for the relative risk. This new estimator favors an extended stay in a state and is useful for modeling diurnal rhythms. We derive asymptotic distributions for our parameter estimates and compare our approach against competing methods through simulation as well as in a digital phenotyping study that followed smartphone use in a cohort of adolescents with mood disorders.

Infinite hidden Markov models for multiple multivariate time series with missing data

Exposure to air pollution is associated with increased morbidity and mortality. Recent technological advancements permit the collection of time-resolved personal exposure data. Such data are often incomplete with missing observations and exposures below the limit of detection, which limit their use in health effects studies. In this paper, we develop an infinite hidden Markov model for multiple asynchronous multivariate time series with missing data. Our model is designed to include covariates that can inform transitions among hidden states. We implement beam sampling, a combination of slice sampling and dynamic programming, to sample the hidden states, and a Bayesian multiple imputation algorithm to impute missing data. In simulation studies, our model excels in estimating hidden states and state-specific means and imputing observations that are missing at random or below the limit of detection. We validate our imputation approach on data from the Fort Collins Commuter Study. We show that the estimated hidden states improve imputations for data that are missing at random compared to existing approaches. In a case study of the Fort Collins Commuter Study, we describe the inferential gains obtained from our model including improved imputation of missing data and the ability to identify shared patterns in activity and exposure among repeated sampling days for individuals and among distinct individuals.

Measuring Daily Activity Rhythms in Young Adults at Risk of Affective Instability Using Passively Collected Smartphone Data: Observational Study

BACKGROUND

Irregularities in circadian rhythms have been associated with adverse health outcomes. The regularity of rhythms can be quantified using passively collected smartphone data to provide clinically relevant biomarkers of routine.

OBJECTIVE

This study aims to develop a metric to quantify the regularity of activity rhythms and explore the relationship between routine and mood, as well as demographic covariates, in an outpatient psychiatric cohort.

METHODS

Passively sensed smartphone data from a cohort of 38 young adults from the Penn or Children's Hospital of Philadelphia Lifespan Brain Institute and Outpatient Psychiatry Clinic at the University of Pennsylvania were fitted with 2-state continuous-time hidden Markov models representing active and resting states. The regularity of routine was modeled as the hour-of-the-day random effects on the probability of state transition (ie, the association between the hour-of-the-day and state membership). A regularity score, Activity Rhythm Metric, was calculated from the continuous-time hidden Markov models and regressed on clinical and demographic covariates.

RESULTS

Regular activity rhythms were associated with longer sleep durations (P=.009), older age (P=.001), and mood (P=.049).

CONCLUSIONS

Passively sensed Activity Rhythm Metrics are an alternative to existing metrics but do not require burdensome survey-based assessments. Low-burden, passively sensed metrics based on smartphone data are promising and scalable alternatives to traditional measurements.

Identifying the Recurrence of Sleep Apnea Using A Harmonic Hidden Markov Model

We propose to model time-varying periodic and oscillatory processes by means of a hidden Markov model where the states are defined through the spectral properties of a periodic regime. The number of states is unknown along with the relevant periodicities, the role and number of which may vary across states. We address this inference problem by a Bayesian nonparametric hidden Markov model assuming a sticky hierarchical Dirichlet process for the switching dynamics between different states while the periodicities characterizing each state are explored by means of a trans-dimensional Markov chain Monte Carlo sampling step. We develop the full Bayesian inference algorithm and illustrate the use of our proposed methodology for different simulation studies as well as an application related to respiratory research which focuses on the detection of apnea instances in human breathing traces.

A primer on coupled state-switching models for multiple interacting time series

State-switching models such as hidden Markov models or Markov-switching regression models are routinely applied to analyse sequences of observations that are driven by underlying non-observable states. Coupled state-switching models extend these approaches to address the case of multiple observation sequences whose underlying state variables interact. In this article, we provide an overview of the modelling techniques related to coupling in state-switching models, thereby forming a rich and flexible statistical framework particularly useful for modelling correlated time series. Simulation experiments demonstrate the relevance of being able to account for an asynchronous evolution as well as interactions between the underlying latent processes. The models are further illustrated using two case studies related to (a) interactions between a dolphin mother and her calf as inferred from movement data and (b) electronic health record data collected on 696 patients within an intensive care unit.

Wearable Device Heart Rate and Activity Data in an Unsupervised Approach to Personalized Sleep Monitoring: Algorithm Validation

Background

The proliferation of wearable devices that collect activity and heart rate data has facilitated new ways to measure sleeping and waking durations unobtrusively and longitudinally. Most existing sleep/wake identification algorithms are based on activity only and are trained on expensive and laboriously annotated polysomnography (PSG). Heart rate can also be reflective of sleep/wake transitions, which has motivated its investigation herein in an unsupervised algorithm. Moreover, it is necessary to develop a personalized approach to deal with interindividual variance in sleep/wake patterns.

Objective

We aimed to develop an unsupervised personalized sleep/wake identification algorithm using multifaceted data to explore the benefits of incorporating both heart rate and activity level in these types of algorithms and to compare this approach’s output with that of an existing commercial wearable device’s algorithms.

Methods

In this study, a total of 14 community-dwelling older adults wore wearable devices (Fitbit Alta; Fitbit Inc) 24 hours a day and 7 days a week over period of 3 months during which their heart rate and activity data were collected. After preprocessing the data, a model was developed to distinguish sleep/wake states based on each individual’s data. We proposed the use of hidden Markov models and compared different modeling schemes. With the best model selected, sleep/wake patterns were characterized by estimated parameters in hidden Markov models, and sleep/wake states were identified.

Results

When applying our proposed algorithm on a daily basis, we found there were significant differences in estimated parameters between weekday models and weekend models for some participants.

Conclusions

Our unsupervised approach can be effectively implemented based on an individual’s multifaceted sleep-related data from a commercial wearable device. A personalized model is shown to be necessary given the interindividual variability in estimated parameters.

Hidden Markov models for monitoring circadian rhythmicity in telemetric activity data

Wearable computing devices allow collection of densely sampled real-time information on movement enabling researchers and medical experts to obtain objective and non-obtrusive records of actual activity of a subject in the real world over many days. Our interest here is motivated by the use of activity data for evaluating and monitoring the circadian rhythmicity of subjects for research in chronobiology and chronotherapeutic healthcare. In order to translate the information from such high-volume data arising we propose the use of a Markov modelling approach which (i) naturally captures the notable square wave form observed in activity data along with heterogeneous ultradian variances over the circadian cycle of human activity, (ii) thresholds activity into different states in a probabilistic way while respecting time dependence and (iii) gives rise to circadian rhythm parameter estimates, based on probabilities of transitions between rest and activity, that are interpretable and of interest to circadian research.

Accuracy of Fitbit Wristbands in Measuring Sleep Stage Transitions and the Effect of User-Specific Factors

BACKGROUND

It has become possible for the new generation of consumer wristbands to classify sleep stages based on multisensory data. Several studies have validated the accuracy of one of the latest models, that is, Fitbit Charge 2, in measuring polysomnographic parameters, including total sleep time, wake time, sleep efficiency (SE), and the ratio of each sleep stage. Nevertheless, its accuracy in measuring sleep stage transitions remains unknown.

OBJECTIVE

This study aimed to examine the accuracy of Fitbit Charge 2 in measuring transition probabilities among wake, light sleep, deep sleep, and rapid eye movement (REM) sleep under free-living conditions. The secondary goal was to investigate the effect of user-specific factors, including demographic information and sleep pattern on measurement accuracy.

METHODS

A Fitbit Charge 2 and a medical device were used concurrently to measure a whole night's sleep in participants' homes. Sleep stage transition probabilities were derived from sleep hypnograms. Measurement errors were obtained by comparing the data obtained by Fitbit with those obtained by the medical device. Paired 2-tailed t test and Bland-Altman plots were used to examine the agreement of Fitbit to the medical device. Wilcoxon signed-rank test was performed to investigate the effect of user-specific factors.

RESULTS

Sleep data were collected from 23 participants. Sleep stage transition probabilities measured by Fitbit Charge 2 significantly deviated from those measured by the medical device, except for the transition probability from deep sleep to wake, from light sleep to REM sleep, and the probability of staying in REM sleep. Bland-Altman plots demonstrated that systematic bias ranged from 0% to 60%. Fitbit had the tendency of overestimating the probability of staying in a sleep stage while underestimating the probability of transiting to another stage. SE>90% (P=.047) was associated with significant increase in measurement error. Pittsburgh sleep quality index (PSQI)<5 and wake after sleep onset (WASO)<30 min could be associated to significantly decreased or increased errors, depending on the outcome sleep metrics.

CONCLUSIONS

Our analysis shows that Fitbit Charge 2 underestimated sleep stage transition dynamics compared with the medical device. Device accuracy may be significantly affected by perceived sleep quality (PSQI), WASO, and SE.

Modelling reassurances of clinicians with hidden Markov models

BACKGROUND

A key element in the interaction between clinicians and patients with cancer is reassurance giving. Learning about the stochastic nature of reassurances as well as making inferential statements about the influence of covariates such as patient response and time spent on previous reassurances are of particular importance.

METHODS

We fit Hidden Markov Models (HMMs) to reassurance type from multiple time series of clinicians' reassurances, decoded from audio files of review consultations between patients with breast cancer and their therapeutic radiographer. Assuming a latent state process driving the observations process, HMMs naturally accommodate serial dependence in the data. Extensions to the baseline model such as including covariates as well as allowing for fixed effects for the different clinicians are straightforward to implement.

RESULTS

We found that clinicians undergo different states, in which they are more or less inclined to provide a particular type of reassurance. The states are very persistent, however switches occasionally occur. The lengthier the previous reassurance, the more likely the clinician is to stay in the current state.

CONCLUSIONS

HMMs prove to be a valuable tool and provide important insights for practitioners.

TRIAL REGISTRATION

Trial Registration number: ClinicalTrials.gov: NCT02599506. Prospectively registered on 11th March 2015.

Multivariate generalized hidden Markov regression models with random covariates: Physical exercise in an elderly population

A time-varying latent variable model is proposed to jointly analyze multivariate mixed-support longitudinal data. The proposal can be viewed as an extension of hidden Markov regression models with fixed covariates (HMRMFCs), which is the state of the art for modelling longitudinal data, with a special focus on the underlying clustering structure. HMRMFCs are inadequate for applications in which a clustering structure can be identified in the distribution of the covariates, as the clustering is independent from the covariates distribution. Here, hidden Markov regression models with random covariates are introduced by explicitly specifying state-specific distributions for the covariates, with the aim of improving the recovering of the clusters in the data with respect to a fixed covariates paradigm. The hidden Markov regression models with random covariates class is defined focusing on the exponential family, in a generalized linear model framework. Model identifiability conditions are sketched, an expectation-maximization algorithm is outlined for parameter estimation, and various implementation and operational issues are discussed. Properties of the estimators of the regression coefficients, as well as of the hidden path parameters, are evaluated through simulation experiments and compared with those of HMRMFCs. The method is applied to physical activity data.

Unsupervised Estimation of Mouse Sleep Scores and Dynamics Using a Graphical Model of Electrophysiological Measurements

The proportion, number of bouts, and mean bout duration of different vigilance states (Wake, NREM, REM) are useful indices of dynamics in experimental sleep research. These metrics are estimated by first scoring state, sometimes using an algorithm, based on electrophysiological measurements such as the electroencephalogram (EEG) and electromyogram (EMG), and computing their values from the score sequence. Isolated errors in the scores can lead to large discrepancies in the estimated sleep metrics. But most algorithms score sleep by classifying the state from EEG/EMG features independently in each time epoch without considering the dynamics across epochs, which could provide contextual information. The objective here is to improve estimation of sleep metrics by fitting a probabilistic dynamical model to mouse EEG/EMG data and then predicting the metrics from the model parameters. Hidden Markov models (HMMs) with multivariate Gaussian observations and Markov state transitions were fitted to unlabeled 24-h EEG/EMG feature time series from 20 mice to model transitions between the latent vigilance states; a similar model with unbiased transition probabilities served as a reference. Sleep metrics predicted from the HMM parameters did not deviate significantly from manual estimates except for rapid eye movement sleep (REM) ([Formula: see text]; Wilcoxon signed-rank test). Changes in value from Light to Dark conditions correlated well with manually estimated differences (Spearman’s rho 0.43–0.84) except for REM. HMMs also scored vigilance state with over 90% accuracy. HMMs of EEG/EMG features can therefore characterize sleep dynamics from EEG/EMG measurements, a prerequisite for characterizing the effects of perturbation in sleep monitoring and control applications.

Development of the National Healthy Sleep Awareness Project Sleep Health Surveillance Questions

OBJECTIVES

For the first time ever, as emphasized by inclusion in the Healthy People 2020 goals, sleep health is an emphasis of national health aims. The National Healthy Sleep Awareness Project (NHSAP) was tasked to propose questions for inclusion in the next Behavioral Risk Factor Surveillance System (BRFSS), a survey that includes a number of questions that target behaviors thought to impact health, as a means to measure community sleep health. The total number of questions could not exceed five, and had to include an assessment of the risk for obstructive sleep apnea (OSA).

METHODS

An appointed workgroup met via teleconference and face-to-face venues to develop an inventory of published survey questions being used to identify sleep health, to develop a framework on which to analyze the strengths and weaknesses of current survey questions concerning sleep, and to develop recommendations for sleep health and disease surveillance questions going forward.

RESULTS

The recommendation was to focus on certain existing BRFSS questions pertaining to sleep duration, quality, satisfaction, daytime alertness, and to add to these other BRFSS existing questions to make a modified STOP-BANG questionnaire (minus the N for neck circumference) to assess for risk of OSA.

CONCLUSIONS

Sleep health is an important dimension of health that has previously received less attention in national health surveys. We believe that 5 questions recommended for the upcoming BRFSS question banks will assist as important measures of sleep health, and may help to evaluate the effectiveness of interventions to improve sleep health in our nation.

Quasi-supervised scoring of human sleep in polysomnograms using augmented input variables

The limitations of manual sleep scoring make computerized methods highly desirable. Scoring errors can arise from human rater uncertainty or inter-rater variability. Sleep scoring algorithms either come as supervised classifiers that need scored samples of each state to be trained, or as unsupervised classifiers that use heuristics or structural clues in unscored data to define states. We propose a quasi-supervised classifier that models observations in an unsupervised manner but mimics a human rater wherever training scores are available. EEG, EMG, and EOG features were extracted in 30s epochs from human-scored polysomnograms recorded from 42 healthy human subjects (18-79 years) and archived in an anonymized, publicly accessible database. Hypnograms were modified so that: 1. Some states are scored but not others; 2. Samples of all states are scored but not for transitional epochs; and 3. Two raters with 67% agreement are simulated. A framework for quasi-supervised classification was devised in which unsupervised statistical models-specifically Gaussian mixtures and hidden Markov models--are estimated from unlabeled training data, but the training samples are augmented with variables whose values depend on available scores. Classifiers were fitted to signal features incorporating partial scores, and used to predict scores for complete recordings. Performance was assessed using Cohen's Κ statistic. The quasi-supervised classifier performed significantly better than an unsupervised model and sometimes as well as a completely supervised model despite receiving only partial scores. The quasi-supervised algorithm addresses the need for classifiers that mimic scoring patterns of human raters while compensating for their limitations.

Nonparametric inference in hidden Markov models using P-splines

Hidden Markov models (HMMs) are flexible time series models in which the distribution of the observations depends on unobserved serially correlated states. The state-dependent distributions in HMMs are usually taken from some class of parametrically specified distributions. The choice of this class can be difficult, and an unfortunate choice can have serious consequences for example on state estimates, and more generally on the resulting model complexity and interpretation. We demonstrate these practical issues in a real data application concerned with vertical speeds of a diving beaked whale, where we demonstrate that parametric approaches can easily lead to overly complex state processes, impeding meaningful biological inference. In contrast, for the dive data, HMMs with nonparametrically estimated state-dependent distributions are much more parsimonious in terms of the number of states and easier to interpret, while fitting the data equally well. Our nonparametric estimation approach is based on the idea of representing the densities of the state-dependent distributions as linear combinations of a large number of standardized B-spline basis functions, imposing a penalty term on non-smoothness in order to maintain a good balance between goodness-of-fit and smoothness.

Latent time-varying factors in longitudinal analysis: a linear mixed hidden Markov model for heart rates

Longitudinal data are often segmented by unobserved time-varying factors, which introduce latent heterogeneity at the observation level, in addition to heterogeneity across subjects. We account for this latent structure by a linear mixed hidden Markov model. It integrates subject-specific random effects and Markovian sequences of time-varying effects in the linear predictor. We propose an expectationŰ-maximization algorithm for maximum likelihood estimation, based on data augmentation. It reduces to the iterative maximization of the expected value of a complete likelihood function, derived from an augmented dataset with case weights, alternated with weights updating. In a case study of the Survey on Stress Aging and Health in Russia, the model is exploited to estimate the influence of the observed covariates under unobserved time-varying factors, which affect the cardiovascular activity of each subject during the observation period.