Longitudinal variable selection by cross-validation in the case of many covariates

Mix-method toolbox for monitoring greenhouse gas production and microbiome responses to soil amendments

In this study, we adopt an interdisciplinary approach, integrating agronomic field experiments with soil chemistry, molecular biology techniques, and statistics to investigate the impact of organic residue amendments, such as vinasse (a by-product of sugarcane ethanol production), on soil microbiome and greenhouse gas (GHG) production. The research investigates the effects of distinct disturbances, including organic residue application alone or combined with inorganic N fertilizer on the environment. The methods assess soil microbiome dynamics (composition and function), GHG emissions, and plant productivity. Detailed steps for field experimental setup, soil sampling, soil chemical analyses, determination of bacterial and fungal community diversity, quantification of genes related to nitrification and denitrification pathways, measurement and analysis of gas fluxes (N2O, CH4, and CO2), and determination of plant productivity are provided. The outcomes of the methods are detailed in our publications (Lourenço et al., 2018a; Lourenço et al., 2018b; Lourenço et al., 2019; Lourenço et al., 2020). Additionally, the statistical methods and scripts used for analyzing large datasets are outlined. The aim is to assist researchers by addressing common challenges in large-scale field experiments, offering practical recommendations to avoid common pitfalls, and proposing potential analyses, thereby encouraging collaboration among diverse research groups.•Interdisciplinary methods and scientific questions allow for exploring broader interconnected environmental problems.•The proposed method can serve as a model and protocol for evaluating the impact of soil amendments on soil microbiome, GHG emissions, and plant productivity, promoting more sustainable management practices.•Time-series data can offer detailed insights into specific ecosystems, particularly concerning soil microbiota (taxonomy and functions).

Feature selection for high-dimensional temporal data

BACKGROUND

Feature selection is commonly employed for identifying collectively-predictive biomarkers and biosignatures; it facilitates the construction of small statistical models that are easier to verify, visualize, and comprehend while providing insight to the human expert. In this work we extend established constrained-based, feature-selection methods to high-dimensional "omics" temporal data, where the number of measurements is orders of magnitude larger than the sample size. The extension required the development of conditional independence tests for temporal and/or static variables conditioned on a set of temporal variables.

RESULTS

The algorithm is able to return multiple, equivalent solution subsets of variables, scale to tens of thousands of features, and outperform or be on par with existing methods depending on the analysis task specifics.

CONCLUSIONS

The use of this algorithm is suggested for variable selection with high-dimensional temporal data.

Stochastic variable selection strategies for zero-inflated models

When count data exhibit excess zero, that is more zero counts than a simpler parametric distribution can model, the zero-inflated Poisson (ZIP) or zero-inflated negative binomial (ZINB) models are often used. Variable selection for these models is even more challenging than for other regression situations because the availability of p covariates implies 4 p possible models. We adapt to zero-inflated models an approach for variable selection that avoids the screening of all possible models. This approach is based on a stochastic search through the space of all possible models, which generates a chain of interesting models. As an additional novelty, we propose three ways of extracting information from this rich chain and we compare them in two simulation studies, where we also contrast our approach with regularization (penalized) techniques available in the literature. The analysis of a typical dataset that has motivated our research is also presented, before concluding with some recommendations.

A multiphase non-linear mixed effects model: An application to spirometry after lung transplantation

In medical sciences, we often encounter longitudinal temporal relationships that are non-linear in nature. The influence of risk factors may also change across longitudinal follow-up. A system of multiphase non-linear mixed effects model is presented to model temporal patterns of longitudinal continuous measurements, with temporal decomposition to identify the phases and risk factors within each phase. Application of this model is illustrated using spirometry data after lung transplantation using readily available statistical software. This application illustrates the usefulness of our flexible model when dealing with complex non-linear patterns and time-varying coefficients.

A Predictive Based Regression Algorithm for Gene Network Selection

Gene selection has become a common task in most gene expression studies. The objective of such research is often to identify the smallest possible set of genes that can still achieve good predictive performance. To do so, many of the recently proposed classification methods require some form of dimension-reduction of the problem which finally provide a single model as an output and, in most cases, rely on the likelihood function in order to achieve variable selection. We propose a new prediction-based objective function that can be tailored to the requirements of practitioners and can be used to assess and interpret a given problem. Based on cross-validation techniques and the idea of importance sampling, our proposal scans low-dimensional models under the assumption of sparsity and, for each of them, estimates their objective function to assess their predictive power in order to select. Two applications on cancer data sets and a simulation study show that the proposal compares favorably with competing alternatives such as, for example, Elastic Net and Support Vector Machine. Indeed, the proposed method not only selects smaller models for better, or at least comparable, classification errors but also provides a set of selected models instead of a single one, allowing to construct a network of possible models for a target prediction accuracy level.

Ultrahigh dimensional time course feature selection

Statistical challenges arise from modern biomedical studies that produce time course genomic data with ultrahigh dimensions. In a renal cancer study that motivated this paper, the pharmacokinetic measures of a tumor suppressor (CCI-779) and expression levels of 12,625 genes were measured for each of 33 patients at 8 and 16 weeks after the start of treatments, with the goal of identifying predictive gene transcripts and the interactions with time in peripheral blood mononuclear cells for pharmacokinetics over the time course. The resulting data set defies analysis even with regularized regression. Although some remedies have been proposed for both linear and generalized linear models, there are virtually no solutions in the time course setting. As such, a novel GEE-based screening procedure is proposed, which only pertains to the specifications of the first two marginal moments and a working correlation structure. Different from existing methods that either fit separate marginal models or compute pairwise correlation measures, the new procedure merely involves making a single evaluation of estimating functions and thus is extremely computationally efficient. The new method is robust against the mis-specification of correlation structures and enjoys theoretical readiness, which is further verified via Monte Carlo simulations. The procedure is applied to analyze the aforementioned renal cancer study and identify gene transcripts and possible time-interactions that are relevant to CCI-779 metabolism in peripheral blood.

Dimensionality reduction for knowledge discovery in medical claims database: application to antidepressant medication utilization study

Data mining, through its capacity to discover knowledge embedded in large databases to improve organizational decision-making, has the potential to contribute to efficiencies and cost savings in the increasingly costly healthcare industry. One important aspect of the methods of mining medical databases includes reducing dimensionality through feature selection. Traditionally feature selection is accomplished through stepwise regression, which tends to produce an unnecessarily high number of "significant" variables. This paper applies a filter-based feature selection method using inconsistency rate measure and discretization, to a medical claims database to predict the adequacy of duration of antidepressant medication utilization. Compared to traditional stepwise logistic regression, which selected seven variables from a total of nine potential explanatory variables to characterize patients with inadequate antidepressant medication utilization, the filter-based method selected two variables (age and number of claims) to achieve a similar prediction accuracy. This comparison suggests it may be feasible and efficient to apply the filter-based feature selection method to reduce the dimensionality of healthcare databases.