Architecture dependent transport behavior of iron (0) entrapped biodegradable polymeric particles for groundwater remediation

Polylactic acid based spherical particles with three architectural variations (Isotropic (P1), Semi porous (P2), and Janus (P3)) were employed to encapsulate zero valent iron nanoparticles (ZVINPs), and their performance was extensively evaluated in our previous studies. However, little was known about their transportability through saturated porous media of varying grain size kept under varying ionic strength. In this particular study, we aimed to investigate the architectural effect of polymeric particles (P1-P3) on their mobility through the sand column of varying grain size in presence of mono, di, and tri-valent ions of varying concentrations (25-200 mM (millimoles)). As per column breakthrough experiments (BTCs) using various types of sands, amphiphilic Janus type (P3) particles exhibited the maximum transportability among all the tested particles, irrespective of the nature of the sand. Owing to the narrower travel path, sands with lower porosity (31%) delayed the plateau by shifting it to a higher pore volume with a minimum retention of iron (C/Co: 0.94 for P3) in the column. The impact of mono (Na+, K+), di (Ca2+, Mg2+), and trivalent (Al3+) ions on their transportability was progressively increased from P3 to P1, especially at higher ionic concentrations (200 mM), with P3 being the most mobile particles (C/Co:0.54 for Al3+). Among all the ions, Al3+ exhibited maximum hindrance to their mobility through the sand column. This could be due to their strong charge screening effect coupled with cation bridging complex formation with moving particles. Experimental results obtained from BTCs were found to be well-fitted with a theoretical model based on advection-dispersion equation, showing minimum retention for P3 particles. Overall, it can be inferred that encapsulation of ZVINPs inside Janus particles (P3) with a right balance of amphiphilicity and highly negative surface charge would be required to achieve considerable transportability through sand aquifers to target contaminants in polluted groundwater existing under harsh conditions (high ionic concentrations).

Systematically missing data in causally interpretable meta-analysis

Causally interpretable meta-analysis combines information from a collection of randomized controlled trials to estimate treatment effects in a target population in which experimentation may not be possible but from which covariate information can be obtained. In such analyses, a key practical challenge is the presence of systematically missing data when some trials have collected data on one or more baseline covariates, but other trials have not, such that the covariate information is missing for all participants in the latter. In this article, we provide identification results for potential (counterfactual) outcome means and average treatment effects in the target population when covariate data are systematically missing from some of the trials in the meta-analysis. We propose three estimators for the average treatment effect in the target population, examine their asymptotic properties, and show that they have good finite-sample performance in simulation studies. We use the estimators to analyze data from two large lung cancer screening trials and target population data from the National Health and Nutrition Examination Survey (NHANES). To accommodate the complex survey design of the NHANES, we modify the methods to incorporate survey sampling weights and allow for clustering.

Population Neuroscience: Understanding Concepts of Generalizability and Transportability and Their Application to Improving the Public's Health

In population neuroscience, samples are not often selected with equal or known probability from an underlying population of interest; in other words, samples are not often formally representative of a specified underlying population. This chapter provides an overview of an epidemiological approach to considering the implications of selective participation on the value of our results for population health. We discuss definitions of generalizability and transportability, given the growing recognition that generalizability and transportability are central for interpreting data that are aiming to be population-based. We provide evidence that differences in the prevalence of effect measure modifiers between a study sample and a target population will lead to a lack of generalizability and transportability. We provide an example of an association between a poly-genetic risk score and depression, showing how an internally valid association can differ based on the prevalence of effect measure modifiers. We show that when estimating associations, inferences from a study sample to a population can depend on clearly defining a target population. Given that representative sampling from explicitly defined target populations may not be feasible or realistic in many situations, especially given the sample sizes needed for statistical power for many exposures of interest (and especially when interactions are being tested), researchers should be well versed in tools available to enhance the interpretability of samples regarding target populations.

Addressing the longitudinal components of surgical treatments

Despite their pre- and postoperative components, surgical treatment strategies have typically been cast as point interventions in causal inference research. When longitudinal perioperative components affect outcomes of interest, leaving them unspecified or failing to measure adherence to them complicates the interpretation of effect estimates. Inspired by two recent landmark trials that assessed the risk of stroke or death after transcatheter aortic valve replacement (TAVR) compared with surgical aortic valve replacement (SAVR), the PARTNER 3 trial and the Evolut Low Risk trial, we discuss possible ways that different postoperative therapies in treatment arms and incomplete adherence to those therapies can impact the interpretation of intention-to-treat effect estimates in surgical trials. We argue that surgical treatments are not necessarily point interventions, and make recommendations for improving the design and analysis of trials involving surgical interventions. Central to these recommendations is the need for investigators to specify and report adherence to longitudinal perioperative treatment components.

Transportability of two heart failure trials to a disease registry using individual patient data

OBJECTIVES

Randomized controlled trials are the gold-standard for determining therapeutic efficacy, but are often unrepresentative of real-world settings. Statistical transportation methods (hereafter transportation) can partially account for these differences, improving trial applicability without breaking randomization. We transported treatment effects from two heart failure (HF) trials to a HF registry.

STUDY DESIGN AND SETTING

Individual-patient-level data from two trials (Carvedilol or Metoprolol European Trial (COMET), comparing carvedilol and metoprolol, and digitalis investigation group trial (DIG), comparing digoxin and placebo) and a Scottish HF registry were obtained. The primary end point for both trials was all-cause mortality; composite outcomes were all-cause mortality or hospitalization for COMET and HF-related death or hospitalization for DIG. We performed transportation using regression-based and inverse odds of sampling weights (IOSW) approaches.

RESULTS

Registry patients were older, had poorer renal function and received higher-doses of loop-diuretics than trial participants. For each trial, point estimates were similar for the original and IOSW (e.g., DIG composite outcome: OR 0.75 (0.69, 0.82) vs. 0.73 (0.64, 0.83)). Treatment effect estimates were also similar when examining high-risk (0.64 (0.46, 0.89)) and low-risk registry patients (0.73 (0.61, 0.86)). Similar results were obtained using regression-based transportation.

CONCLUSION

Regression-based or IOSW approaches can be used to transport trial effect estimates to patients administrative/registry data, with only moderate reductions in precision.

Assessing the transportability of clinical prediction models for cognitive impairment using causal models

BACKGROUND

Machine learning models promise to support diagnostic predictions, but may not perform well in new settings. Selecting the best model for a new setting without available data is challenging. We aimed to investigate the transportability by calibration and discrimination of prediction models for cognitive impairment in simulated external settings with different distributions of demographic and clinical characteristics.

METHODS

We mapped and quantified relationships between variables associated with cognitive impairment using causal graphs, structural equation models, and data from the ADNI study. These estimates were then used to generate datasets and evaluate prediction models with different sets of predictors. We measured transportability to external settings under guided interventions on age, APOE ε4, and tau-protein, using performance differences between internal and external settings measured by calibration metrics and area under the receiver operating curve (AUC).

RESULTS

Calibration differences indicated that models predicting with causes of the outcome were more transportable than those predicting with consequences. AUC differences indicated inconsistent trends of transportability between the different external settings. Models predicting with consequences tended to show higher AUC in the external settings compared to internal settings, while models predicting with parents or all variables showed similar AUC.

CONCLUSIONS

We demonstrated with a practical prediction task example that predicting with causes of the outcome results in better transportability compared to anti-causal predictions when considering calibration differences. We conclude that calibration performance is crucial when assessing model transportability to external settings.

Extending prediction models for use in a new target population with failure time outcomes

Prediction models are often built and evaluated using data from a population that differs from the target population where model-derived predictions are intended to be used in. In this article, we present methods for evaluating model performance in the target population when some observations are right censored. The methods assume that outcome and covariate data are available from a source population used for model development and covariates, but no outcome data, are available from the target population. We evaluate the finite sample performance of the proposed estimators using simulations and apply the methods to transport a prediction model built using data from a lung cancer screening trial to a nationally representative population of participants eligible for lung cancer screening.

Encapsulation of zero valent iron nanoparticles in biodegradable amphiphilic janus particles for groundwater remediation

Reactive Zero Valent Iron (ZVI) nanoparticles have been widely explored for in situ ground water remediation to degrade both non-aqueous phase liquid (NAPL) and water-soluble contaminants. However, they usually suffer from rapid oxidation and severe agglomerations restricting their delivery at NAPL/water interface. Aim of this study was to encapsulate the ZVI nanoparticles (50 nm) in amphiphilic bicompartmental Janus particles (711 ± 11 nm) fabricated by EHDC (electrohydrodynamic co-jetting). The dual compartments were composed of PLA (polylactic acid) and a blend of PLA, PE (poly (hexamethylene 2,3-O-isopropylidenetartarate) and PAG (photo acid generator). Upon UV irradiation, PAG releases acid to unmask hydroxyl groups present in PE to make only PE compartment hydrophilic. The entrapped ZVI nanoparticles (20 w/w%; ∼99 % encapsulation efficiency) were observed to degrade both hydrophilic (methyl orange dye) and hydrophobic (trichloro ethylene) contaminants. UV treated Janus particles provided stable dispersion (dispersed up to 3 weeks in water), prolonged reactivity (∼24 days in contaminated water), and recyclability (recyclable up to 9 times) as compared to non-treated ones. In addition, the amphiphilic Janus particles demonstrated high transportability (>95%) through porous media (sand column) with very low attachment efficiency (0.07), making them a promising candidate to target contaminants at NAPL/water interface prevailed in groundwater.

Data integration: exploiting ratios of parameter estimates from a reduced external model

We consider the situation of estimating the parameters in a generalized linear prediction model, from an internal dataset, where the outcome variable [Formula: see text] is binary and there are two sets of covariates, [Formula: see text] and [Formula: see text]. We have information from an external study that provides parameter estimates for a generalized linear model of [Formula: see text] on [Formula: see text]. We propose a method that makes limited assumptions about the similarity of the distributions in the two study populations. The method involves orthogonalizing the [Formula: see text] variables and then borrowing information about the ratio of the coefficients from the external model. The method is justified based on a new result relating the parameters in a generalized linear model to the parameters in a generalized linear model with omitted covariates. The method is applicable if the regression coefficients in the [Formula: see text] given [Formula: see text] model are similar in the two populations, up to an unknown scalar constant. This type of transportability between populations is something that can be checked from the available data. The asymptotic variance of the proposed method is derived. The method is evaluated in a simulation study and shown to gain efficiency compared to simple analysis of the internal dataset, and is robust compared to an alternative method of incorporating external information.

Estimating the impact of stimulant use on initiation of buprenorphine and extended-release naltrexone in two clinical trials and real-world populations

BACKGROUND

Co-use of stimulants and opioids is rapidly increasing. Randomized clinical trials (RCTs) have established the efficacy of medications for opioid use disorder (MOUD), but stimulant use may decrease the likelihood of initiating MOUD treatment. Furthermore, trial participants may not represent "real-world" populations who would benefit from treatment.

METHODS

We conducted a two-stage analysis. First, associations between stimulant use (time-varying urine drug screens for cocaine, methamphetamine, or amphetamines) and initiation of buprenorphine or extended-release naltrexone (XR-NTX) were estimated across two RCTs (CTN-0051 X:BOT and CTN-0067 CHOICES) using adjusted Cox regression models. Second, results were generalized to three target populations who would benefit from MOUD: Housed adults identifying the need for OUD treatment, as characterized by the National Survey on Drug Use and Health (NSDUH); adults entering OUD treatment, as characterized by Treatment Episodes Dataset (TEDS); and adults living in rural regions of the U.S. with high rates of injection drug use, as characterized by the Rural Opioids Initiative (ROI). Generalizability analyses adjusted for differences in demographic characteristics, substance use, housing status, and depression between RCT and target populations using inverse probability of selection weighting.

RESULTS

Analyses included 673 clinical trial participants, 139 NSDUH respondents (weighted to represent 661,650 people), 71,751 TEDS treatment episodes, and 1,933 ROI participants. The majority were aged 30-49 years, male, and non-Hispanic White. In RCTs, stimulant use reduced the likelihood of MOUD initiation by 32% (adjusted HR [aHR] = 0.68, 95% CI 0.49-0.94, p = 0.019). Stimulant use associations were slightly attenuated and non-significant among housed adults needing treatment (25% reduction, aHR = 0.75, 0.48-1.18, p = 0.215) and adults entering OUD treatment (28% reduction, aHR = 0.72, 0.51-1.01, p = 0.061). The association was more pronounced, but still non-significant among rural people injecting drugs (39% reduction, aHR = 0.61, 0.35-1.06, p = 0.081). Stimulant use had a larger negative impact on XR-NTX initiation compared to buprenorphine, especially in the rural population (76% reduction, aHR = 0.24, 0.08-0.69, p = 0.008).

CONCLUSIONS

Stimulant use is a barrier to buprenorphine or XR-NTX initiation in clinical trials and real-world populations that would benefit from OUD treatment. Interventions to address stimulant use among patients with OUD are urgently needed, especially among rural people injecting drugs, who already suffer from limited access to MOUD.

Regression-based estimation of heterogeneous treatment effects when extending inferences from a randomized trial to a target population

Most work on extending (generalizing or transporting) inferences from a randomized trial to a target population has focused on estimating average treatment effects (i.e., averaged over the target population's covariate distribution). Yet, in the presence of strong effect modification by baseline covariates, the average treatment effect in the target population may be less relevant for guiding treatment decisions. Instead, the conditional average treatment effect (CATE) as a function of key effect modifiers may be a more useful estimand. Recent work on estimating target population CATEs using baseline covariate, treatment, and outcome data from the trial and covariate data from the target population only allows for the examination of heterogeneity over distinct subgroups. We describe flexible pseudo-outcome regression modeling methods for estimating target population CATEs conditional on discrete or continuous baseline covariates when the trial is embedded in a sample from the target population (i.e., in nested trial designs). We construct pointwise confidence intervals for the CATE at a specific value of the effect modifiers and uniform confidence bands for the CATE function. Last, we illustrate the methods using data from the Coronary Artery Surgery Study (CASS) to estimate CATEs given history of myocardial infarction and baseline ejection fraction value in the target population of all trial-eligible patients with stable ischemic heart disease.

External validation of existing dementia prediction models on observational health data

BACKGROUND

Many dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records.

METHODS

We identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters' Dementia Risk Score, Mehta's RxDx-Dementia Risk Index, and Nori's ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models.

RESULTS

We reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67-0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69-0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62-0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard.

CONCLUSION

We observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

Transfer learning of individualized treatment rules from experimental to real-world data

Individualized treatment effect lies at the heart of precision medicine. Interpretable individualized treatment rules (ITRs) are desirable for clinicians or policymakers due to their intuitive appeal and transparency. The gold-standard approach to estimating the ITRs is randomized experiments, where subjects are randomized to different treatment groups and the confounding bias is minimized to the extent possible. However, experimental studies are limited in external validity because of their selection restrictions, and therefore the underlying study population is not representative of the target real-world population. Conventional learning methods of optimal interpretable ITRs for a target population based only on experimental data are biased. On the other hand, real-world data (RWD) are becoming popular and provide a representative sample of the target population. To learn the generalizable optimal interpretable ITRs, we propose an integrative transfer learning method based on weighting schemes to calibrate the covariate distribution of the experiment to that of the RWD. Theoretically, we establish the risk consistency for the proposed ITR estimator. Empirically, we evaluate the finite-sample performance of the transfer learner through simulations and apply it to a real data application of a job training program.

Prognostic models for COVID-19 needed updating to warrant transportability over time and space

BACKGROUND

Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time.

METHODS

We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit.

RESULTS

Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data.

CONCLUSIONS

NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic.

Experimental study on in situ remediation of Cr(VI) contaminated groundwater by sulfidated micron zero valent iron stabilized with xanthan gum

Micron zero valent iron (mZVI) was an underground remediation material, which had great application potential to replace nano zero valent iron (nZVI) from the perspective of economic and health benefits. However, mZVI was highly prone to gravitational settling, which limited its wide application for in situ remediation of contaminated groundwater. This paper was devoted to develop an efficient and economical groundwater remediation material based on mZVI, which should possess excellent stability, reactivity, and transportability. Thereby xanthan gum (XG) stabilized and Na₂S₂O₄ sulfidated mZVI (XG-S-mZVI) was synthesized and characterized with SEM, XRD, XPS, and FTIR techniques. In terms of stability, the adsorbed XG and the dispersed XG worked together to resist the sedimentation of S-mZVI. In terms of reactivity, sulfidation enhanced the electron transfer rate and electron selectivity of XG-S-mZVI, thereby improved the reactivity of XG-S-mZVI. The hexavalent chromium (Cr(VI)) removal rate constant by XG-S-mZVI was determined to be 832.4 times than bare mZVI. In terms of transportability, the transportability of XG-S-mZVI was greatly improved (~80 cm in coarse sand and ~50 cm in medium sand). Straining was the main mechanism of XG-S-mZVI retention in porous media. XG-S-mZVI in situ reactive zone (XG-S-mZVI-IRZ) was only suitable to the media with a grain size larger than 0.25 mm. This study could provide theoretical support and guidance for the implementation of IRZ technology based on mZVI.

Predicting counterfactual risks under hypothetical treatment strategies: an application to HIV

The accuracy of a prediction algorithm depends on contextual factors that may vary across deployment settings. To address this inherent limitation of prediction, we propose an approach to counterfactual prediction based on the g-formula to predict risk across populations that differ in their distribution of treatment strategies. We apply this to predict 5-year risk of mortality among persons receiving care for HIV in the U.S. Veterans Health Administration under different hypothetical treatment strategies. First, we implement a conventional approach to develop a prediction algorithm in the observed data and show how the algorithm may fail when transported to new populations with different treatment strategies. Second, we generate counterfactual data under different treatment strategies and use it to assess the robustness of the original algorithm's performance to these differences and to develop counterfactual prediction algorithms. We discuss how estimating counterfactual risks under a particular treatment strategy is more challenging than conventional prediction as it requires the same data, methods, and unverifiable assumptions as causal inference. However, this may be required when the alternative assumption of constant treatment patterns across deployment settings is unlikely to hold and new data is not yet available to retrain the algorithm.

Mortality prediction in intensive care units including premorbid functional status improved performance and internal validity

OBJECTIVE

Prognostic models are key for benchmarking intensive care units (ICUs). They require up-to-date predictors and should report transportability properties for reliable predictions. We developed and validated an in-hospital mortality risk prediction model to facilitate benchmarking, quality assurance, and health economics evaluation.

STUDY DESIGN AND SETTING

We retrieved data from the database of an international (Finland, Estonia, Switzerland) multicenter ICU cohort study from 2015 to 2017. We used a hierarchical logistic regression model that included age, a modified Simplified Acute Physiology Score-II, admission type, premorbid functional status, and diagnosis as grouping variable. We used pooled and meta-analytic cross-validation approaches to assess temporal and geographical transportability.

RESULTS

We included 61,224 patients treated in the ICU (hospital mortality 10.6%). The developed prediction model had an area under the receiver operating characteristic curve 0.886, 95% confidence interval (CI) 0.882-0.890; a calibration slope 1.01, 95% CI (0.99-1.03); a mean calibration -0.004, 95% CI (-0.035 to 0.027). Although the model showed very good internal validity and geographic discrimination transportability, we found substantial heterogeneity of performance measures between ICUs (I-squared: 53.4-84.7%).

CONCLUSION

A novel framework evaluating the performance of our prediction model provided key information to judge the validity of our model and its adaptation for future use.

Target validity: Bringing treatment of external validity in line with internal validity

Purpose of Review

"Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity.

Recent findings

External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available.

Summary

The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.

The findings of a surgical hip fracture trial were generalizable to the UK national hip fracture database

OBJECTIVE

To estimate the generalizability of treatment effects observed in a randomized trial of hip fracture surgery implants to a broader population of people undergoing hip surgery in the United Kingdom.

STUDY DESIGN AND SETTING

In 2018, the WHiTE-3 trial (n = 958) demonstrated that a modular hemiarthroplasty implant conferred no additional benefit over the traditional monoblock implant for quality of life and length of hospital stay. We compared and weighted the trial sample against two target populations: WHiTE-cohort (n = 2,457) and UK-National Hip Fracture Database (NHFD, n = 190,894), and re-estimate expected treatment effects for the target populations.

RESULTS

Despite differences in baseline characteristics of the trial sample and target populations, the re-estimated treatment effects were comparable. For quality of life, the differences between the trial estimate and WHiTE-cohort and NHFD estimates were 0.01 points on the EuroQol (EQ5D). For length of stay, the difference between the trial estimate and WHiTE-cohort was 0.50 days; and the difference between the trial estimate and NHFD estimate was -0.47 days.

CONCLUSION

This generalizability analysis of the WHiTE-3 trial found that the inferences from the trial can be generalized to a wider population of individuals in the UK NHFD and the WHiTE-cohort who met the inclusion criteria for WHiTE-3.

Generalizing experimental results by leveraging knowledge of mechanisms

We show how experimental results can be generalized across diverse populations by leveraging knowledge of local mechanisms that produce the outcome of interest, only some of which may differ in the target domain. We use structural causal models and a refined version of selection diagrams to represent such knowledge, and to decide whether it entails the invariance of probabilities of causation across populations, which then enables generalization. We further provide: (i) bounds for the target effect when some of these conditions are violated; (ii) new identification results for probabilities of causation and the transported causal effect when trials from multiple source domains are available; as well as (iii) a Bayesian approach for estimating the transported causal effect from finite samples. We illustrate these methods both with simulated data and with a real example that transports the effects of Vitamin A supplementation on childhood mortality across different regions.

Directed acyclic graphs and causal thinking in clinical risk prediction modeling

BACKGROUND

In epidemiology, causal inference and prediction modeling methodologies have been historically distinct. Directed Acyclic Graphs (DAGs) are used to model a priori causal assumptions and inform variable selection strategies for causal questions. Although tools originally designed for prediction are finding applications in causal inference, the counterpart has remained largely unexplored. The aim of this theoretical and simulation-based study is to assess the potential benefit of using DAGs in clinical risk prediction modeling.

METHODS

We explore how incorporating knowledge about the underlying causal structure can provide insights about the transportability of diagnostic clinical risk prediction models to different settings. We further probe whether causal knowledge can be used to improve predictor selection in clinical risk prediction models.

RESULTS

A single-predictor model in the causal direction is likely to have better transportability than one in the anticausal direction in some scenarios. We empirically show that the Markov Blanket, the set of variables including the parents, children, and parents of the children of the outcome node in a DAG, is the optimal set of predictors for that outcome.

CONCLUSIONS

Our findings provide a theoretical basis for the intuition that a diagnostic clinical risk prediction model including causes as predictors is likely to be more transportable. Furthermore, using DAGs to identify Markov Blanket variables may be a useful, efficient strategy to select predictors in clinical risk prediction models if strong knowledge of the underlying causal structure exists or can be learned.

Feasibility and evaluation of a large-scale external validation approach for patient-level prediction in an international data network: validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation

Background

To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets.

Methods

Five previously published prognostic models (ATRIA, CHADS₂, CHADS₂VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites.

Results

The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57–0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation.

Conclusion

This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

Need for Cognition, Transportability, and Engagement with Interactive Narratives

Objective: Games for heath can take the form of interactive narratives, or stories in which readers have the option to make decisions about the direction of the plot. Individual differences may affect the extent to which individuals become engaged in such narratives. Materials and Methods: In two studies, we randomly assigned participants to read either a traditional linear narrative or an interactive version of the same narrative. We examined the influence of need for cognition (NFC) and transportability (the extent to which individuals tend to become immersed in narratives) on transportation, character identification, and perceived realism. Results: Transportability led to higher perceptions of realism in the interactive narrative in Study 1, but this effect was not replicated in Study 2. In Study 1, higher NFC led to greater identification in the interactive narrative; in Study 2, higher NFC led to greater transportation into the interactive narrative. Conclusion: Greater willingness to exert mental effort may lead to greater immersion in interactive narratives.

The transportability of Memory Specificity Training (MeST): adapting an intervention derived from experimental psychology to routine clinical practices

Background

Accumulating evidence shows that a cognitive factor associated with a worsening of depressive symptoms amongst people with and without diagnoses of depression – reduced Autobiographical Memory (rAMS) – can be ameliorated by a group cognitive training protocol referred to as Memory Specificity Training (MeST). When transporting interventions such as MeST from research to routine clinical practices (RCPs), modifications are inevitable, with potentially a decrease in effectiveness, so called voltage drop. We examined the transportability of MeST to RCPs as an add-on to treatment as usual with depressed in- and out- patients.

Methods

We examined whether 1) MeST was adaptable to local needs of RCPs by implementing MeST in a joint decision-making process in seven Belgian RCPs 2) without losing its effect on rAMS. The effectiveness of MeST was measured by pre- and post- intervention measurements of memory specificity.

Results

Adaptations were made to the MeST protocol to optimize the fit with RCPs. Local needs of RCPs were met by dismantling MeST into different subparts. By dismantling it in this way, we were able to address several challenges raised by clinicians. In particular, multidisciplinary teams could divide the workload across different team members and, for the open version of MeST, the intervention could be offered continuously with tailored dosing per patient. Both closed and open versions of MeST, with or without peripheral components, and delivered by health professionals with different backgrounds, resulted in a significant increase in memory specificity for depressed in- and out- patients in RCPs.

Conclusions

MeST is shown to be a transportable and adaptable add-on intervention which effectively maintains its core mechanism when delivered in RCPs.

Trial registration

ISRCTN registry, IDISRCTN10144349, registered on January 22, 2019. Retrospectively registered.

Electronic supplementary material

The online version of this article (10.1186/s40359-019-0279-y) contains supplementary material, which is available to authorized users.

Participation weighting based on sociodemographic register data improved external validity in a population-based cohort study

OBJECTIVE

To investigate whether inverse probability of participation weighting (IPPW) using register data on sociodemographic and disease history variables can improve external validity in a cohort study with selective participation.

STUDY DESIGN AND SETTING

We fitted various IPPW models by logistic regression using register data for the participants (n = 1,111) and nonparticipants (n = 1,132) of a Swedish cohort study. For each of six diagnostic groups, we then estimated (1) weighted disease prevalence proportions and (2) weighted cross-sectional associations (odds ratios) between sociodemographic variables and disease prevalence. Using register data on the remaining individuals of the entire study population of men and women aged 50-64 years (n = 22,259), we addressed how the choice of variables used for IPPW influenced estimation errors.

RESULTS

Disease prevalence proportions were generally underestimated in the absence of IPPW but became markedly closer to population values after IPPW using sociodemographic variables. We found limited evidence of selective participation bias in association estimates, but IPPW improved external validity when bias was present.

CONCLUSIONS

IPPW using sociodemographic register data can improve the external validity of disease prevalence estimates in cohort studies with selective participation. The performance of IPPW for association estimates merits further investigations in longitudinal settings and larger cohorts.

Transportability of imagery-enhanced CBT for social anxiety disorder

Pilot and open trials suggest that imagery-enhanced group cognitive behaviour therapy (CBT) is highly effective for social anxiety disorder (SAD). However, before being considered reliable and generalisable, the effects of the intervention need to be replicated by clinicians in a setting that is independent of the protocol developers. The current study compared outcomes from clients with a principal diagnosis of SAD at the Australian clinic where the protocol was developed (n = 123) to those from an independent Canadian clinic (n = 46) to investigate whether the large effects would generalise. Trainee clinicians from the independent clinic ran the groups using the treatment protocol without any input from its developers. The treatment involved 12 2-h group sessions plus a one-month follow-up. Treatment retention was comparable across both clinics (74% vs. 78%, ≥9/12 sessions) and the between-site effect size was very small and non-significant on the primary outcome (social interaction anxiety, d = 0.09, p = .752). Within-group effect sizes were very large in both settings (ds = 2.05 vs. 2.19), and a substantial minority (41%-44%) achieved clinically significant improvement at follow-up. Replication of treatment effects within an independent clinic and with trainee clinicians increases confidence that outcomes are generalisable.

Transportable data from non-target arthropod field studies for the environmental risk assessment of genetically modified maize expressing an insecticidal double-stranded RNA

As part of an environmental risk assessment, the potential impact of genetically modified (GM) maize MON 87411 on non-target arthropods (NTAs) was evaluated in the field. MON 87411 confers resistance to corn rootworm (CRW; Diabrotica spp.) by expressing an insecticidal double-stranded RNA (dsRNA) transcript and the Cry3Bb1 protein and tolerance to the herbicide glyphosate by producing the CP4 EPSPS protein. Field trials were conducted at 14 sites providing high geographic and environmental diversity within maize production areas from three geographic regions including the U.S., Argentina, and Brazil. MON 87411, the conventional control, and four commercial conventional reference hybrids were evaluated for NTA abundance and damage. Twenty arthropod taxa met minimum abundance criteria for valid statistical analysis. Nine of these taxa occurred in at least two of the three regions and in at least four sites across regions. These nine taxa included: aphid, predatory earwig, lacewing, ladybird beetle, leafhopper, minute pirate bug, parasitic wasp, sap beetle, and spider. In addition to wide regional distribution, these taxa encompass the ecological functions of herbivores, predators and parasitoids in maize agro-ecosystems. Thus, the nine arthropods may serve as representative taxa of maize agro-ecosystems, and thereby support that analysis of relevant data generated in one region can be transportable for the risk assessment of the same or similar GM crop products in another region. Across the 20 taxa analyzed, no statistically significant differences in abundance were detected between MON 87411 and the conventional control for 123 of the 128 individual-site comparisons (96.1%). For the nine widely distributed taxa, no statistically significant differences in abundance were detected between MON 87411 and the conventional control. Furthermore, no statistically significant differences were detected between MON 87411 and the conventional control for 53 out of 56 individual-site comparisons (94.6 %) of NTA pest damage to the crop. In each case where a significant difference was observed in arthropod abundance or damage, the mean value for MON 87411 was within the reference range and/or the difference was not consistently observed across collection methods and/or sites. Thus, the differences were not representative of an adverse effect unfamiliar to maize and/or were not indicative of a consistent plant response associated with the GM traits. Results from this study support a conclusion of no adverse environmental impact of MON 87411 on NTAs compared to conventional maize and demonstrate the utility of relevant transportable data across regions for the ERA of GM crops.

A new framework to enhance the interpretation of external validation studies of clinical prediction models

OBJECTIVES

It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models.

STUDY DESIGN AND SETTING

We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting.

RESULTS

We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings.

CONCLUSION

The proposed framework enhances the interpretation of findings at external validation of prediction models.

Polymers influencing transportability profile of drug

Drug release from various polymers is generally governed by the type of polymer/s incorporated in the formulation and mechanism of drug release from polymer/s. A single polymer may show one or more mechanisms of drug release out of which one mechanism is majorly followed for drug release. Some of the common mechanisms of drug release from polymers were, diffusion, swelling, matrix release, leaching of drug, etc. Mechanism or rate of drug release from a polymer or a combination of polymers can be predicted by using different computational methods or models. These models were capable of predicting drug release from its dosage form in advance without actual formulation and testing of drug release from dosage form. Quantitative structure-property relationship (QSPR) is an important tool used in the prediction of various physicochemical properties of actives as well as inactives. Since last several decades QSPR has been applied in new drug development for reducing the total number of drugs to be synthesized, as it involves a selection of the most desirable compound of interest. This technique was also applied in predicting in vivo performance of drug/s for various parameters. QSPR serves as a predictive tool to correlate structural descriptors of molecules with biological as well as physicochemical properties. Several researchers have contributed at different extents in this area to modify various properties of pharmaceuticals. The present review is focused on a study of different polymers that influence the transportability profiles of drugs along with the application of QSPR either to study different properties of polymers that regulate drug release or in predicting drug transportability from different polymer systems used in formulations.