Treatment Effect Heterogeneity

Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity

BACKGROUND

Treatment variation from observational data has been used to estimate patient-specific treatment effects. Causal Forest Algorithms (CFAs) developed for this task have unknown properties when treatment effect heterogeneity from unmeasured patient factors influences treatment choice - essential heterogeneity.

METHODS

We simulated eleven populations with identical treatment effect distributions based on patient factors. The populations varied in the extent that treatment effect heterogeneity influenced treatment choice. We used the generalized random forest application (CFA-GRF) to estimate patient-specific treatment effects for each population. Average differences between true and estimated effects for patient subsets were evaluated.

RESULTS

CFA-GRF performed well across the population when treatment effect heterogeneity did not influence treatment choice. Under essential heterogeneity, however, CFA-GRF yielded treatment effect estimates that reflected true treatment effects only for treated patients and were on average greater than true treatment effects for untreated patients.

CONCLUSIONS

Patient-specific estimates produced by CFAs are sensitive to why patients in real-world practice make different treatment choices. Researchers using CFAs should develop conceptual frameworks of treatment choice prior to estimation to guide estimate interpretation ex post.

Causal complexity in human research: On the shared challenges of behavior genetics, medical genetics, and environmentally oriented social science

We received 23 spirited commentaries on our target article from across the disciplines of philosophy, economics, evolutionary genetics, molecular biology, criminology, epidemiology, and law. We organize our reply around three overarching questions: (1) What is a cause? (2) How are randomized controlled trials (RCTs) and within-family genome-wide association studies (GWASs) alike and unalike? (3) Is behavior genetics a qualitatively different enterprise? Throughout our discussion of these questions, we advocate for the idea that behavior genetics shares many of the same pitfalls and promises as environmentally oriented research, medical genetics, and other arenas of the social and behavioral sciences.