Collaborative targeted maximum likelihood estimation for variable importance measure: Illustration for functional outcome prediction in mild traumatic brain injuries

Application of targeted maximum likelihood estimation in public health and epidemiological studies: a systematic review

PURPOSE

The targeted maximum likelihood estimation (TMLE) statistical data analysis framework integrates machine learning, statistical theory, and statistical inference to provide a least biased, efficient, and robust strategy for estimation and inference of a variety of statistical and causal parameters. We describe and evaluate the epidemiological applications that have benefited from recent methodological developments.

METHODS

We conducted a systematic literature review in PubMed for articles that applied any form of TMLE in observational studies. We summarized the epidemiological discipline, geographical location, expertize of the authors, and TMLE methods over time. We used the Roadmap of Targeted Learning and Causal Inference to extract key methodological aspects of the publications. We showcase the contributions to the literature of these TMLE results.

RESULTS

Of the 89 publications included, 33% originated from the University of California at Berkeley, where the framework was first developed by Professor Mark van der Laan. By 2022, 59% of the publications originated from outside the United States and explored up to seven different epidemiological disciplines in 2021-2022. Double-robustness, bias reduction, and model misspecification were the main motivations that drew researchers toward the TMLE framework. Through time, a wide variety of methodological, tutorial, and software-specific articles were cited, owing to the constant growth of methodological developments around TMLE.

CONCLUSIONS

There is a clear dissemination trend of the TMLE framework to various epidemiological disciplines and to increasing numbers of geographical areas. The availability of R packages, publication of tutorial papers, and involvement of methodological experts in applied publications have contributed to an exponential increase in the number of studies that understood the benefits and adoption of TMLE.

Causal inference in case of near-violation of positivity: comparison of methods

In causal studies, the near-violation of the positivity may occur by chance, because of sample-to-sample fluctuation despite the theoretical veracity of the positivity assumption in the population. It may mostly happen when the exposure prevalence is low or when the sample size is small. We aimed to compare the robustness of g-computation (GC), inverse probability weighting (IPW), truncated IPW, targeted maximum likelihood estimation (TMLE), and truncated TMLE in this situation, using simulations and one real application. We also tested different extrapolation situations for the sub-group with a positivity violation. The results illustrated that the near-violation of the positivity impacted all methods. We demonstrated the robustness of GC and TMLE-based methods. Truncation helped in limiting the bias in near-violation situations, but at the cost of bias in normal conditions. The application illustrated the variability of the results between the methods and the importance of choosing the most appropriate one. In conclusion, compared to propensity score-based methods, methods based on outcome regression should be preferred when suspecting near-violation of the positivity assumption.

Evaluating the robustness of targeted maximum likelihood estimators via realistic simulations in nutrition intervention trials

Several recently developed methods have the potential to harness machine learning in the pursuit of target quantities inspired by causal inference, including inverse weighting, doubly robust estimating equations and substitution estimators like targeted maximum likelihood estimation. There are even more recent augmentations of these procedures that can increase robustness, by adding a layer of cross-validation (cross-validated targeted maximum likelihood estimation and double machine learning, as applied to substitution and estimating equation approaches, respectively). While these methods have been evaluated individually on simulated and experimental data sets, a comprehensive analysis of their performance across real data based simulations have yet to be conducted. In this work, we benchmark multiple widely used methods for estimation of the average treatment effect using ten different nutrition intervention studies data. A nonparametric regression method, undersmoothed highly adaptive lasso, is used to generate the simulated distribution which preserves important features from the observed data and reproduces a set of true target parameters. For each simulated data, we apply the methods above to estimate the average treatment effects as well as their standard errors and resulting confidence intervals. Based on the analytic results, a general recommendation is put forth for use of the cross-validated variants of both substitution and estimating equation estimators. We conclude that the additional layer of cross-validation helps in avoiding unintentional over-fitting of nuisance parameter functionals and leads to more robust inferences.

Prognostic Value of a Combination of Circulating Biomarkers in Critically Ill Patients with Traumatic Brain Injury: Results from the European CREACTIVE Study

Individualized patient care is essential to reduce the global burden of traumatic brain injury (TBI). This pilot study focused on TBI patients admitted to intensive care units (ICUs) and aimed at identifying patterns of circulating biomarkers associated with the disability level at 6 months from injury, measured by the extended Glasgow Outcome Scale (GOS-E). The concentration of 107 biomarkers, including proteins related to inflammation, innate immunity, TBI, and central nervous system, were quantified in blood samples collected on ICU admission from 80 patients. Patients were randomly selected among those prospectively enrolled in the Collaborative Research on Acute Traumatic Brain Injury in Intensive Care Medicine in Europe (CREACTIVE) observational study. Six biomarkers were selected to be associated with indicators of primary or secondary brain injury: three glial proteins (glial cell-derived neurotrophic factor, glial fibrillary acidic protein, and S100 calcium-binding protein B) and three cytokines (stem cell factor, fibroblast growth factor [FGF] 23 and FGF19). The subjects were grouped into three clusters according to the expression of these proteins. The distribution of the 6-month GOS-E was significantly different across clusters (p < 0.001). In two clusters, the number of 6-month deaths or vegetative states was significantly lower than expected, as calculated according to a customization of the corticosteroid randomization after significant head injury (CRASH) scores (observed/expected [O/E] events = 0.00, 95% confidence interval [CI]: 0.00-0.90 and 0.00, 95% CI: 0.00-0.94). In one cluster, less-than-expected unfavorable outcomes (O/E = 0.50, 95% CI: 0.05-0.95) and more-than-expected good recoveries (O/E = 1.55, 95% CI: 1.05-2.06) were observed. The improved prognostic accuracy of the pattern of these six circulating biomarkers at ICU admission upon established clinical parameters and computed tomography results needs validation in larger, independent cohorts. Nonetheless, the results of this pilot study are promising and will prompt further research in personalized medicine for TBI patients.

Performance of propensity score matching to estimate causal effects in small samples

Propensity score (PS) matching is a very popular causal estimator usually used to estimate the average treatment effect on the treated (ATT) from observational data. However, opting for this estimator may raise some efficiency issues when the sample size is limited. Therefore, we aimed to evaluate the performance of propensity score matching in this context. We started with a motivating example based on a cohort of 66 children with sickle cell anemia who received either allogeneic bone-marrow transplant or chronic transfusion. We found substantial differences in the ATT estimate according to the model selected for propensity score estimation and subsequent matching. Then, we assessed the performance of the different propensity score matching methods and post-matching analyses to estimate the ATT using a simulation study. Although all selected propensity score matching methods were based of previous recommendations, we found important discrepancies in the estimation of treatment effect between them, underlining the importance of thorough sensitivity analyses when using propensity score matching in the context of small sample sizes.

Pre-injury Comorbidities Are Associated With Functional Impairment and Post-concussive Symptoms at 3- and 6-Months After Mild Traumatic Brain Injury: A TRACK-TBI Study

Introduction: Over 70% of traumatic brain injuries (TBI) are classified as mild (mTBI), which present heterogeneously. Associations between pre-injury comorbidities and outcomes are not well-understood, and understanding their status as risk factors may improve mTBI management and prognostication. Methods: mTBI subjects (GCS 13-15) from TRACK-TBI Pilot completing 3- and 6-month functional [Glasgow Outcome Scale-Extended (GOSE)] and post-concussive outcomes [Acute Concussion Evaluation (ACE) physical/cognitive/sleep/emotional subdomains] were extracted. Pre-injury comorbidities >10% incidence were included in regressions for functional disability (GOSE ≤ 6) and post-concussive symptoms by subdomain. Odds ratios (OR) and mean differences (B) were reported. Significance was assessed at p < 0.0083 (Bonferroni correction). Results: In 260 subjects sustaining blunt mTBI, mean age was 44.0-years and 70.4% were male. Baseline comorbidities >10% incidence included psychiatric-30.0%, cardiac (hypertension)-23.8%, cardiac (structural/valvular/ischemic)-20.4%, gastrointestinal-15.8%, pulmonary-15.0%, and headache/migraine-11.5%. At 3- and 6-months separately, 30.8% had GOSE ≤ 6. At 3-months, psychiatric (GOSE ≤ 6: OR = 2.75, 95% CI [1.44-5.27]; ACE-physical: B = 1.06 [0.38-1.73]; ACE-cognitive: B = 0.72 [0.26-1.17]; ACE-sleep: B = 0.46 [0.17-0.75]; ACE-emotional: B = 0.64 [0.25-1.03]), headache/migraine (GOSE ≤ 6: OR = 4.10 [1.67-10.07]; ACE-sleep: B = 0.57 [0.15-1.00]; ACE-emotional: B = 0.92 [0.35-1.49]), and gastrointestinal history (ACE-physical: B = 1.25 [0.41-2.10]) were multivariable predictors of worse outcomes. At 6-months, psychiatric (GOSE ≤ 6: OR = 2.57 [1.38-4.77]; ACE-physical: B = 1.38 [0.68-2.09]; ACE-cognitive: B = 0.74 [0.28-1.20]; ACE-sleep: B = 0.51 [0.20-0.83]; ACE-emotional: B = 0.93 [0.53-1.33]), and headache/migraine history (ACE-physical: B = 1.81 [0.79-2.84]) predicted worse outcomes. Conclusions: Pre-injury psychiatric and pre-injury headache/migraine symptoms are risk factors for worse functional and post-concussive outcomes at 3- and 6-months post-mTBI. mTBI patients presenting to acute care should be evaluated for psychiatric and headache/migraine history, with lower thresholds for providing TBI education/resources, surveillance, and follow-up/referrals. Clinical Trial Registration: www.ClinicalTrials.gov, identifier NCT01565551.

Big data and targeted machine learning in action to assist medical decision in the ICU

Historically, personalised medicine has been synonymous with pharmacogenomics and oncology. We argue for a new framework for personalised medicine analytics that capitalises on more detailed patient-level data and leverages recent advances in causal inference and machine learning tailored towards decision support applicable to critically ill patients. We discuss how advances in data technology and statistics are providing new opportunities for asking more targeted questions regarding patient treatment, and how this can be applied in the intensive care unit to better predict patient-centred outcomes, help in the discovery of new treatment regimens associated with improved outcomes, and ultimately how these rules can be learned in real-time for the patient.

Targeted maximum likelihood estimation for a binary treatment: A tutorial

When estimating the average effect of a binary treatment (or exposure) on an outcome, methods that incorporate propensity scores, the G-formula, or targeted maximum likelihood estimation (TMLE) are preferred over naïve regression approaches, which are biased under misspecification of a parametric outcome model. In contrast propensity score methods require the correct specification of an exposure model. Double-robust methods only require correct specification of either the outcome or the exposure model. Targeted maximum likelihood estimation is a semiparametric double-robust method that improves the chances of correct model specification by allowing for flexible estimation using (nonparametric) machine-learning methods. It therefore requires weaker assumptions than its competitors. We provide a step-by-step guided implementation of TMLE and illustrate it in a realistic scenario based on cancer epidemiology where assumptions about correct model specification and positivity (ie, when a study participant had 0 probability of receiving the treatment) are nearly violated. This article provides a concise and reproducible educational introduction to TMLE for a binary outcome and exposure. The reader should gain sufficient understanding of TMLE from this introductory tutorial to be able to apply the method in practice. Extensive R-code is provided in easy-to-read boxes throughout the article for replicability. Stata users will find a testing implementation of TMLE and additional material in the Appendix S1 and at the following GitHub repository: https://github.com/migariane/SIM-TMLE-tutorial.