Classification Using Censored Functional Data

Shape Analysis of Functional Data With Elastic Partial Matching

Elastic Riemannian metrics have been used successfully for statistical treatments of functional and curve shape data. However, this usage suffers from a significant restriction: the function boundaries are assumed to be fixed and matched. In practice, functional data often comes with unmatched boundaries. It happens, for example, in dynamical systems with variable evolution rates, such as COVID-19 infection rate curves associated with different geographical regions. Here, we develop a Riemannian framework that allows for partial matching, comparing, and clustering of functions with phase variability and uncertain boundaries. We extend past work by (1) Defining a new diffeomorphism group G over the positive reals that is the semidirect product of a time-warping group and a time-scaling group; (2) Introducing a metric that is invariant to the action of G; (3) Imposing a Riemannian Lie group structure on G to allow for an efficient gradient-based optimization for elastic partial matching; and (4) Presenting a modification that, while losing the metric property, allows one to control the amount of boundary disparity in the registration. We illustrate this framework by registering and clustering shapes of COVID-19 rate curves, identifying basic patterns, minimizing mismatch errors, and reducing variability within clusters compared to previous methods.

Depth-based reconstruction method for incomplete functional data

The problem of estimating missing fragments of curves from a functional sample has been widely considered in the literature. However, most reconstruction methods rely on estimating the covariance matrix or the components of its eigendecomposition, which may be difficult. In particular, the estimation accuracy might be affected by the complexity of the covariance function, the noise of the discrete observations, and the poor availability of complete discrete functional data. We introduce a non-parametric alternative based on depth measures for partially observed functional data. Our simulations point out that the benchmark methods perform better when the data come from one population, curves are smooth, and there is a large proportion of complete data. However, our approach is superior when considering more complex covariance structures, non-smooth curves, and when the proportion of complete functions is scarce. Moreover, even in the most severe case of having all the functions incomplete, our method provides good estimates; meanwhile, the competitors are unable. The methodology is illustrated with two real data sets: the Spanish daily temperatures observed in different weather stations and the age-specific mortality by prefectures in Japan. They highlight the interpretability potential of the depth-based method.

Bayesian Framework for Simultaneous Registration and Estimation of Noisy, Sparse and Fragmented Functional Data

In many applications, smooth processes generate data that is recorded under a variety of observational regimes, including dense sampling and sparse or fragmented observations that are often contaminated with error. The statistical goal of registering and estimating the individual underlying functions from discrete observations has thus far been mainly approached sequentially without formal uncertainty propagation, or in an application-specific manner by pooling information across subjects. We propose a unified Bayesian framework for simultaneous registration and estimation, which is flexible enough to accommodate inference on individual functions under general observational regimes. Our ability to do this relies on the specification of strongly informative prior models over the amplitude component of function variability using two strategies: a data-driven approach that defines an empirical basis for the amplitude subspace based on training data, and a shape-restricted approach when the relative location and number of extrema is well-understood. The proposed methods build on the elastic functional data analysis framework to separately model amplitude and phase variability inherent in functional data. We emphasize the importance of uncertainty quantification and visualization of these two components as they provide complementary information about the estimated functions. We validate the proposed framework using multiple simulation studies and real applications.

A Bayesian multiple imputation approach to bivariate functional data with missing components

Existing missing data methods for functional data mainly focus on reconstructing missing measurements along a single function-a univariate functional data setting. Motivated by a renal study, we focus on a bivariate functional data setting, where each sampling unit is a collection of two distinct component functions, one of which may be missing. Specifically, we propose a Bayesian multiple imputation approach based on a bivariate functional latent factor model that exploits the joint changing patterns of the component functions to allow accurate and stable imputation of one component given the other. We further extend the framework to address multilevel bivariate functional data with missing components by modeling and exploiting inter-component and intra-subject correlations. We develop a Gibbs sampling algorithm that simultaneously generates multiple imputations of missing component functions and posterior samples of model parameters. For multilevel bivariate functional data, a partially collapsed Gibbs sampler is implemented to improve computational efficiency. Our simulation study demonstrates that our methods outperform other competing methods for imputing missing components of bivariate functional data under various designs and missingness rates. The motivating renal study aims to investigate the distribution and pharmacokinetic properties of baseline and post-furosemide renogram curves that provide further insights into the underlying mechanism of renal obstruction, with post-furosemide renogram curves missing for some subjects. We apply the proposed methods to impute missing post-furosemide renogram curves and obtain more refined insights.

Censored functional data for incomplete follow-up studies

Functional data analysis plays an increasingly important role in medical research because patients are followed over time. Thus, the measurements of a particular biomarker for each patient are often registered as curves. Hence, it is of interest to estimate the mean function under certain conditions as an average of the observed functional data over a given period. However, this is often difficult as this type of follow-up studies are confronted with the challenge of some individuals dropping-out before study completion. Therefore, for these individuals, only a partial functional observation is available. In this study, we propose an estimator for the functional mean when the functions may be censored from the right, and thus, only partly observed. Unlike sparse functional data, the censored curves are observed until some (random) time and this censoring time may depend on the trajectory of the functional observations. Our approach is model-free and fully nonparametric, although the proposed methods can also be incorporated into regression models. The use of the functional structure of the data distinguishes our approach from the longitudinal data approaches. In addition, in this study, we propose a bootstrap-based confidence band for the mean function, examine the estimation of the covariance function, and apply our new approach to functional principal component analysis. Employing an extensive simulation study, we demonstrate that our method outperforms the only two existing approaches. Furthermore, we apply our new estimator to a real data example on lung growth, measured by changes in pulmonary function for girls in the United States.

Basis expansions for functional snippets

Estimation of mean and covariance functions is fundamental for functional data analysis. While this topic has been studied extensively in the literature, a key assumption is that there are enough data in the domain of interest to estimate both the mean and covariance functions. We investigate mean and covariance estimation for functional snippets in which observations from a subject are available only in an interval of length strictly, and often much, shorter than the length of the whole interval of interest. For such a sampling plan, no data is available for direct estimation of the off-diagonal region of the covariance function. We tackle this challenge via a basis representation of the covariance function. The proposed estimator enjoys a convergence rate that is adaptive to the smoothness of the underlying covariance function, and has superior finite-sample performance in simulation studies.

Robust probabilistic classification applicable to irregularly sampled functional data

A robust probabilistic classifier for functional data is developed to predict class membership based on functional input measurements and to provide a reliable probability estimates for class membership. The method combines a Bayes classifier and semi-parametric mixed effects model with robust tuning parameter to make the method robust to outlying curves, and to improve the accuracy of the risk or uncertainty estimates, which is crucial in medical diagnostic applications. The approach applies to functional data with varying ranges and irregular sampling without making parametric assumptions on the within-curve covariance. Simulation studies evaluate the proposed method and competitors in terms of sensitivity to heavy tailed functional distributions and outlying curves. Classification performance is evaluated by both error rate and logloss, the latter of which imposes heavier penalties on highly confident errors than on less confident errors. Runtime experiments on the R implementation indicate that the proposed method scales well computationally. Illustrative applications include data from quantitative ultrasound analysis and phoneme recognition.

Bayesian additive adaptive basis tensor product models for modeling high dimensional surfaces: an application to high-throughput toxicity testing

Many modern datasets are sampled with error from complex high-dimensional surfaces. Methods such as tensor product splines or Gaussian processes are effective and well suited for characterizing a surface in two or three dimensions, but they may suffer from difficulties when representing higher dimensional surfaces. Motivated by high throughput toxicity testing where observed dose-response curves are cross sections of a surface defined by a chemical's structural properties, a model is developed to characterize this surface to predict untested chemicals' dose-responses. This manuscript proposes a novel approach that models the multidimensional surface as a sum of learned basis functions formed as the tensor product of lower dimensional functions, which are themselves representable by a basis expansion learned from the data. The model is described and a Gibbs sampling algorithm is proposed. The approach is investigated in a simulation study and through data taken from the US EPA's ToxCast high throughput toxicity testing platform.