The Revolution Will Be Hard to Evaluate: How Co-Occurring Policy Changes Affect Research on the Health Effects of Social Policies

The long-term health effects of welfare reform

Childhood poverty is associated with poor health in adulthood, but social policies may buffer poverty's long-term harmful effects by providing cash and in-kind resources. This study assessed the long-term health effects of welfare reform, which added work requirements, sanctions for noncompliance, lifetime limits, and family caps to welfare programming in the US in the mid-1990s. Research has found mixed evidence on the short-term health effects of welfare reform, but fewer studies have documented its impacts on health across the life course. This study exploited temporal and state variation in the implementation of welfare reform to examine the association between exposure to welfare reform as a child on self-rated health, psychological distress, and chronic conditions in adulthood using data from the Panel Study of Income Dynamics. We found that welfare reform exposure in early childhood, from conception to age five, was associated with more psychological distress (b = 0.41, 95%CI 0.08, 0.74) and more chronic conditions (b = 0.21, 95%CI 0.10, 0.33) in adulthood. We also found that welfare reform exposure in later childhood, from ages six to 18, was associated with lower likelihood of reporting poor health (b = -0.03, 95%CI -0.05, -0.01), less psychological distress (b = -0.33, 95%CI -0.54, -0.11), and fewer chronic conditions (b = -0.18, 95%CI -0.25, -0.10), which was especially true for non-Hispanic Black and Hispanic individuals compared to non-Hispanic White individuals. Our findings suggest that the difference in outcomes based on age of exposure is related to the ease with which parents fulfill welfare requirements, though more research is needed on possible mechanisms.

The epidemiologic case for urban health: conceptualizing and measuring the magnitude of challenges and potential benefits

We discuss how epidemiology has been and can continue to be used to advance understanding of the links between urban areas and health informed by an existing urban-health conceptual framework. This framework considers urban areas as contexts for health, determinants of health and modifiers of health pathways, and part of a complex system that affects health. We highlight opportunities for descriptive epidemiology to inform the context of urban health, for example, by characterizing the social and physical environments that give rise to health and the actions that change those conditions. We then describe inferential tools for evaluating the impact of group-level actions (e.g., interventions, policies) on urban health, providing some examples, and describing assumptions and challenges. Finally, we discuss opportunities and challenges of applying systems thinking and methods to advance urban health. While different conceptual frames lead to different insights, each perspective demonstrates that urban health is a major and growing challenge. The effectiveness of urban health knowledge, action, and policy as the world continues to urbanize can be informed by applying and expanding upon research and surveillance methods described here.

How do restrictions on opioid prescribing, harm reduction, and treatment coverage policies relate to opioid overdose deaths in the United States in 2013-2020? An application of a new state opioid policy scale

BACKGROUND

Identifying the most effective state laws and provisions to reduce opioid overdose deaths remains critical.

METHODS

Using expert ratings of opioid laws, we developed annual state scores for three domains: opioid prescribing restrictions, harm reduction, and Medicaid treatment coverage. We modeled associations of state opioid policy domain scores with opioid-involved overdose death counts in 3133 counties, and among racial/ethnic subgroups in 1485 counties (2013-2020). We modeled a second set of domain scores based solely on experts' highest 20 ranked provisions to compare with the all-provisions model.

RESULTS

From 2013 to 2020, moving from non- to full enactment of harm reduction domain laws (i.e., 0 to 1 in domain score) was associated with reduced county-level relative risk (RR) of opioid overdose death in the subsequent year (adjusted RR = 0.84, 95 % credible interval (CrI): 0.77, 0.92). Moving from non- to full enactment of opioid prescribing restrictions and Medicaid treatment coverage domains was associated with higher overdose in 2013-2016 (aRR 1.69 (1.35, 2.11) and aRR 1.20 (1.11, 1.29) respectively); both shifted to the null in 2017-2020. Effect sizes and direction were similar across racial/ethnic groups. Results for experts' highest 20 ranked provisions did not differ from the all-provision model.

CONCLUSIONS

More robust state harm reduction policy scores were associated with reduced overdose risk, adjusting for other policy domains. Harmful associations with opioid prescribing restrictions in 2013-2016 may reflect early unintended consequences of these laws. Medicaid coverage domain findings did not align with experts' perceptions, though data limitations precluded inclusion of several highly ranked Medicaid policies.

The association of state policies and opioid analgesic amount dispensed from retail pharmacies

IMPORTANCE

States have implemented multiple policies likely to influence opioid prescribing; few national general population studies examine those policies' effects on per-capita opioid morphine milligram equivalents (MME) dispensed.

OBJECTIVE

To examine state policies' effects on opioids per-capita MMEs dispensed at retail pharmacies.

DESIGN

A longitudinal study of associations between MME per capita and implementation of policy interventions at different times across states.

SETTING

United States, 2006-2018 PATIENTS: Data representing approximately 90 % of prescriptions filled at retail pharmacies in the United States.

INTERVENTIONS

States implementing (1) Pain management clinic laws, (2) operational Prescription Drug Monitoring Programs (PDMP), (3) mandatory PDMP use, (4) required continuing medical education for opioid prescribers, (5) medical cannabis dispensary laws, and (6) initial prescription duration limit laws.

MAIN OUTCOMES AND MEASUREMENTS

Monthly county-level opioid MME per-capita.

RESULTS

Pain management clinic policies' effects were modest, not significantly associated with MME per-capita in the year following implementation, but negatively associated in subsequent years. Operational PDMP policies were negatively associated with MME per-capita in all five years following implementation. We found no evidence of significant effects of mandatory PDMP requirements, mandatory continuing medical education policies, or medical cannabis dispensary policies in any of the five years following policy implementation. Initial prescription duration limits were associated with increased per-capita MME dispensed in years 3-5 following implementation.

CONCLUSIONS AND RELEVANCE

Several state policies are associated with reductions in the total amount of opioid analgesic dispensed. Additional research should examine the policies' effects on clinical outcomes in both general and high-risk populations.

Duration of exposure to eviction moratoria in the early COVID-19 pandemic and perinatal outcomes: A population-level analysis of US births conceived in March-May 2020

OBJECTIVE

To estimate associations between the length of state-level eviction moratoria enacted in March and April 2020 in the United States and perinatal outcomes.

METHODS

We used data from natality files, 2020-2021 to identify individuals with Medicaid or no insurance who conceived in March-May 2020. The exposure was the number of months exposed to a moratorium (0 (referent, no state-level moratoria), 1-2, 3-4, 5 or more). Outcomes included preterm birth (PTB, < 37 weeks gestation), very preterm birth (VPTB, < 32 weeks gestation), low birthweight (LBW, < 2500 g), very low birthweight (VLBW, < 1500 g), primary cesarean, or maternal morbidity. We estimated risk ratios (RRs) using log-binomial regression, including individual, county, and state-level confounders. We conducted several sensitivity analyses to rule out residual state-level confounding including a negative control analysis of 2019 conceptions and difference-in-difference analysis.

RESULTS

We included 375,821 births. Following adjustment, having a moratorium in place for 5 or more months was associated with slightly reduced risk of PTB (RR: 0.95, 95 % CI: 0.88, 1.02), VPTB (RR: 0.90, 95 % CI: 0.8-1.01), LBW (RR: 0.95, 95 % CI: 0.9-1.01), and VLBW (RR: 0.91, 95 % CI: 0.81-1.02) compared to states without a moratorium. There was no association with cesarean or maternal morbidity. Sensitivity analyses showed that all or most of the observed associations may be explained by residual state-level confounding.

CONCLUSIONS

State-level eviction moratoria were associated with improved birth outcomes, yet it is likely that all or most of the observed association is due to other policy actions or characteristics of enacting states.

Methods Used to Evaluate the Health Effects of Social Policies: A Systematic Review

Purpose of review

Despite the growth of research on social policies and health in recent years, few studies have systematically summarized the methodological approaches used in this growing literature. This review characterizes the range of and trends in analytic methods used in studies of the health effects of US social policies published in leading health journals during January 2014-July 2024.

Recent findings

Among the 117 studies reviewed, confounder-control approaches were the most commonly used method to assess health effects of social policies. Quasi-experimental methods were also frequently used, especially difference-in-differences designs. Heterogeneous subgroup effects were consistently assessed.

Summary

Although there was frequent use of quasi-experimental designs that meet standards for rigorous evidence used to inform policymaking, many opportunities for improvement remain. We suggest improvements to data infrastructure, highlight less frequently studied policies as fruitful future research opportunities, and encourage researchers to implement quasi-experimental approaches best suited to identify causal estimates.

Strengthening tobacco control research: key factors impacting policy outcomes and health equity

In this policy brief, we explore several potential drivers of heterogeneity in policy outcomes that can be examined in tobacco control policy evaluations, expanding the evidence base to contribute to continued, equitable progress in reducing tobacco-related health outcomes. We discuss these factors in the context of a hypothetical evaluation of the impact of smoke-free laws on current smoking and quit attempts in the Tobacco Nation. Despite a similar policy environment within the Tobacco Nation, there is variation in the strength of smoke-free law coverage across states. This commentary considers how policy design and other contextual factors, including co-occurring policies, and differential impacts across subgroups, may influence policy-attributable outcomes across time and space.

Target Trial Emulation for Evaluating Health Policy

Target trial emulation is an approach to designing rigorous nonexperimental studies by "emulating" key features of a clinical trial. Most commonly used outside of policy contexts, this approach is also valuable for policy evaluation as policies typically are not randomly assigned. In this article, we discuss the application of the target trial emulation framework in a policy evaluation context. The policy trial emulation framework includes 7 components: the units and eligibility criteria, definitions of the exposure and comparison conditions, assignment mechanism, baseline ("time zero") and follow-up, outcomes, causal estimand, and statistical analysis and assumptions. Policy evaluations that emulate a randomized trial across these dimensions can yield estimates of the causal effects of the policy on outcomes. Using the policy trial emulation framework to conduct and report on research design and methods supports transparent assessment of threats to causal inference in nonexperimental studies intended to assess the effect of a health policy on clinical or population health outcomes.

A Longitudinal Multivariable Analysis: State Policies and Opioid Dispensing in Medicare Beneficiaries Undergoing Surgery

BACKGROUND

States have implemented policies to decrease clinically unnecessary opioid prescribing, but few studies have examined how state policies affect opioid dispensing rate trends for surgical patients.

OBJECTIVE

To examine trends in the perioperative opioid dispensing rates for fee-for-service Medicare beneficiaries and the effects of select state policies.

DESIGN AND PARTICIPANTS

A retrospective cohort study using 2006 to 2018 Medicare claims data for individuals undergoing surgical procedures for which opioid analgesic treatment is common.

EXPOSURES

State policies mandating prescription drug monitoring program (PDMP; PDMP policies) use, initial opioid prescription duration limit (duration limit policies), and mandated continuing medical education (CME; CME pain policies) on pain management.

MAIN MEASURES

Opioid dispensing rates, days' supply, and the daily morphine milligram equivalent dose (MMED).

KEY RESULTS

The percentage of Medicare beneficiaries dispensed opioids in the perioperative period increased from 2007 to 2018; MMED and days' supply decreased over the same period, with significant variation by age, sex, and race. None of the three state policies affected the likelihood of Medicare beneficiaries being dispensed perioperative opioids. However, CME pain policies and duration limit policies were associated with decreased days' supply and decreased MMED in the several years following implementation, respectively.

CONCLUSION

While we observed a slight increase in the rate of Medicare beneficiaries dispensed opioids perioperatively and a substantial decrease in MMED and days' supply for those receiving opioids, state policies examined had relatively modest effects on the main measures. Our findings suggest that these state policies may have a limited impact on opioid dispensing for a patient population that is commonly dispensed opioid analgesics to help control surgical pain, and as a result may have little direct effect on clinical outcomes for this population. Changes in opioid dispensing for this population may be the result of broader societal trends than such state policies.

Building A Comprehensive, Longitudinal Dataset to Advance Research on the Efficacy of State-Level Anti-bullying Legislation: 1999 to 2017

Bullying is one of the most common forms of youth violence and is associated with myriad adverse consequences over the life course. There has been increasing interest in examining whether anti-bullying legislation is effective in preventing bullying victimization and its negative effects. However, a lack of data structures that comprehensively and longitudinally assess anti-bullying legislation and its provisions has hampered this effort. We provide 18 years of data (1999-2017) on anti-bullying legislation and amendments across 50 U.S. states and the District of Columbia, which we are making publicly available at LawAtlas.org. This article describes how the legal content analysis was conducted, provides information on the reliability of the coding, and details provisions of the legislation that were coded, such as funding provisions and enumerated groups (a total of 122 individual codes are provided). Over 90% of states had at least one amendment to their legislation during this 18-year period (range: 0-22; Mean = 6.1), highlighting both the evolving content of anti-bullying statutes and the importance of tracking these changes with longitudinal data. Additionally, we offer illustrative examples of the kinds of research questions that might be pursued with these new data. For instance, using survival analyses, we show that a variety of state characteristics (e.g., political leaning of state legislatures) predict time to adoption of key provisions of anti-bullying legislation (e.g., the comprehensiveness of legal provisions). Finally, we end with a discussion of how the dataset might be used in future research on the efficacy of anti-bullying legislation.

Policies and Indicators to Operationalize Structural Racism for Quantitative Health Research

The field of structural racism is developing rapidly, with researchers moving from commentaries and review papers toward empirical research that aims to quantify the impact of exposure to structural racism on racial and ethnic inequities in health. Despite the increasing recognition of the impact of structural racism on health outcomes, its measurement in the empirical literature has focused on a limited set of area-based measures. This paper proposes two ways to expand the measurement of structural racism in public health research. First, by situating indicators of structural racism within the broader context of discriminatory policies that gave rise to racial and ethnic inequities, and second, by broadening the range of area-level indicators of structural racism beyond what has been used in the extant literature. The paper provides practical tools, including an extensive list of discriminatory federal policies, a list of common and novel indicators of structural racism across multiple domains, and an applied example showing how to connect policies and indicators of structural racism. These tools will aid researchers in developing a robust empirical evidence base to examine the role of structural racism in long-standing health inequities and advance quantitative research on the impact of structural racism on health inequities.

Reporting and Description of Research Methodology in Studies Estimating Effects of Firearm Policies

BACKGROUND

Evidence about which firearm policies work, to what extent, and for whom is hotly debated, perhaps partly because variation in research methodology has produced mixed and inconclusive effect estimates. We conducted a scoping review of firearm policy research in the health sciences in the United States, focusing on methodological considerations for causal inference.

METHODS

We identified original, empirical articles indexed in PubMed from 1 January 2000 to 1 September 2021 that examined any of 18 prespecified firearm policies. We extracted key study components, including policy type(s) examined, policy operationalization, outcomes, study setting and population, study approach and design, causal language, and whether and how authors acknowledged potential sources of bias.

RESULTS

We screened 7733 articles and included 124. A plurality of studies used a legislative score as their primary exposure (n = 39; 32%) and did not examine change in policies over time (n = 47; 38%). Most examined firearm homicide (n = 51; 41%) or firearm suicide (n = 40; 32%) as outcomes. One-third adjusted for other firearm policies (n = 41; 33%). Three studies (2%) explicitly mentioned that their goal was to estimate causal effects, but over half used language implying causality (n = 72; 58%). Most acknowledged causal identification assumptions of temporality (n = 91; 73%) and exchangeability (n = 111; 90%); other assumptions were less often acknowledged. One-third of studies included bias analyses (n = 42; 34%).

CONCLUSIONS

We identified a range of methodologic approaches in firearm policy research in the health sciences. Acknowledging the imitations of data availability and quality, we identify opportunities to improve causal inferences about and reporting on the effects of firearm policies on population health.

Punitive legal responses to prenatal drug use in the United States: A survey of state policies and systematic review of their public health impacts

BACKGROUND

Punitive legal responses to prenatal drug use may be associated with unintended adverse health consequences. However, in a rapidly shifting policy climate, current information has not been summarized. We conducted a survey of U.S. state policies that utilize criminal or civil legal system penalties to address prenatal drug use. We then systematically identified empirical studies evaluating these policies and summarized their potential public health impacts.

METHODS

Using existing databases and original statutory research, we surveyed current U.S. state-level prenatal drug use policies authorizing explicit criminalization, involuntary commitment, civil child abuse substantiation, and parental rights termination. Next, we systematically identified quantitative associations between these policies and health outcomes, restricting to U.S.-based peer-reviewed research, published January 2000-December 2022. Results described study characteristics and synthesized the evidence on health-related harms and benefits associated with punitive policies. Validity threats were described narratively.

RESULTS

By 2022, two states had adopted policies explicitly authorizing criminal prosecution, and five states allowed pregnancy-specific and drug use-related involuntary civil commitment. Prenatal drug use was grounds for substantiating civil child abuse and terminating parental rights in 22 and five states, respectively. Of the 16 review-identified articles, most evaluated associations between punitive policies generally (k = 12), or civil child abuse policies specifically (k = 2), and multiple outcomes, including drug treatment utilization (k = 6), maltreatment reporting and foster care entry (k = 5), neonatal drug withdrawal syndrome (NDWS, k = 4) and other pregnancy and birth-related outcomes (k = 3). Most included studies reported null associations or suggested increases in adverse outcome following punitive policy adoption.

CONCLUSIONS

Nearly half of U.S. states have adopted policies that respond to prenatal drug use with legal system penalties. While additional research is needed to clarify whether such approaches engender overt health harms, current evidence indicates that punitive policies are not associated with public health benefits, and therefore constitute ineffective policy.

Structural stigma and LGBTQ+ health: a narrative review of quantitative studies

Health disparities related to sexual orientation and gender identity exist across multiple outcomes. Scholarship has begun to evaluate whether structural stigma-ie, societal-level conditions, cultural norms, and institutional policies that constrain opportunities, resources, and wellbeing-contributes to health burdens among LGBTQ+ individuals. We conducted a comprehensive review of quantitative studies examining this hypothesis. We found 98 articles that linked objective (ie, non-self-reported) measures of structural stigma to mental (n=57), behavioural (ie, substance use; n=27; HIV/AIDS or sexually transmitted infection; n=20), and physical (n=20) health outcomes. There was generally consistent evidence that structural stigma increases risk of poor health among LGBTQ+ individuals. Several methodological strengths were identified, including the use of multiple measures (eg, laws or policies [59%, 58 of 98]), designs (eg, quasi-experiments [21%, 21 of 98]), and samples (eg, probability-based [56%, 55 of 98]). However, important gaps exist. Just over half of studies included area-level covariates or non-LGBTQ+ comparison groups, which are necessary to address alternative explanations for the observed associations. Additionally, while studies (n=90) have begun to identify candidate mechanisms, only nine (10%) formally tested mediation. We offer suggestions for future research to advance this literature, which has implications not only for the identification of structural determinants of LGBTQ+ health but also for the development of public health interventions that reduce LGBTQ+ health disparities.

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-level Policies

BACKGROUND

Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined.

METHODS

We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships).

RESULTS

Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice.

CONCLUSIONS

In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

A statewide study of disparities in local policies and tobacco, vape, and cannabis retail environments

The current study: (1) assesses sociodemographic disparities in local policies related to tobacco and cannabis retail, and (2) examines the cross-sectional association between policy strength and retailer densities of tobacco, e-cigarette (vape), and cannabis retailers within California cities and county unincorporated areas (N = 539). We combined (a) American Community Survey data (2019 5-year estimates), (b) 2018 tobacco, vape, and cannabis retailer locations from a commercial data provider, (c) 2017 tobacco and vape retail environment policy data from American Lung Association, and (d) 2018 cannabis policy data from California Cannabis Local Laws Database. Conditional autoregressive models examined policy strength associations with sociodemographic composition and retailer density in California jurisdictions. Jurisdictions with larger percentages of Black and foreign-born residents had stronger tobacco and vape policies. For cannabis policy, only income had a small, significant positive association with policy strength. Contrary to hypothesis, tobacco/vape policies were not significantly associated with retailer density, but cannabis policy strength was associated with lower cannabis retailer density (relative rate = 0.58, 95% Uncertainty Interval 0.47-0.70)-this effect was completely driven by storefront bans. Thus, storefront cannabis bans were the only policy studied that was associated with lower cannabis retailer density. Further research is needed to understand policies and disparities in retail environments for tobacco, vape, and cannabis, including data on the prospective association between policy implementation and subsequent retailer density, and the role of enforcement.

Legislating Inequity: Structural Racism In Groups Of State Laws And Associations With Premature Mortality Rates

Most evaluations of health equity policy have focused on the effects of individual laws. However, multiple laws' combined effects better reflect the crosscutting nature of structurally racist legal regimes. To measure the combined effects of multiple laws, we used latent class analysis, a method for detecting unobserved "subgroups" in a population, to identify clusters of US states based on thirteen structural racism-related legal domains in 2013. We identified three classes of states: one with predominantly harmful laws ([Formula: see text]), another with predominantly protective laws ([Formula: see text]), and a third with a mix of both ([Formula: see text]). Premature mortality rates overall-defined as deaths before age seventy-five per 100,000 population-were highest in states with predominantly harmful laws, which included eighteen states with past Jim Crow laws. This study offers a new method for measuring structural racism on the basis of how groups of laws are associated with premature mortality rates.

A quantitative assessment of the frequency and magnitude of heterogeneous treatment effects in studies of the health effects of social policies

Substantial heterogeneity in effects of social policies on health across subgroups may be common, but has not been systematically characterized. Using a sample of 55 contemporary studies on health effects of social policies, we recorded how often heterogeneous treatment effects (HTEs) were assessed, for what subgroups (e.g., male, female), and the subgroup-specific effect estimates expressed as Standardized Mean Differences (SMDs). For each study, outcome, and dimension (e.g., gender), we fit a random-effects meta-analysis. We characterized the magnitude of heterogeneity in policy effects using the standard deviation of the subgroup-specific effect estimates (τ). Among the 44% of studies reporting subgroup-specific estimates, policy effects were generally small (<0.1 SMDs) with mixed impacts on health (67% beneficial) and disparities (50% implied narrowing of disparities). Across study-outcome-dimensions, 54% indicated any heterogeneity in effects, and 20% had τ > 0.1 SMDs. For 26% of study-outcome-dimensions, the magnitude of τ indicated that effects of opposite signs were plausible across subgroups. Heterogeneity was more common in policy effects not specified a priori. Our findings suggest social policies commonly have heterogeneous effects on health of different populations; these HTEs may substantially impact disparities. Studies of social policies and health should routinely evaluate HTEs.

US State Policy Contexts and Population Health

Policy Points This Perspective connects the dots between the polarization in US states' policy contexts and the divergence in population health across states. Key interlocking forces that fueled this polarization are the political investments of wealthy individuals and organizations and the nationalization of US political parties. Key policy priorities for the next decade include ensuring all Americans have opportunities for economic security, deterring behaviors that kill or injure hundreds of thousands of Americans each year, and protecting voting rights and democratic functioning.

Income support policies and firearm violence prevention: A scoping review

Firearm violence is a major threat to global public health and safety. Several individual, family, peer, community, and societal risk and protective factors determine or modify the risk of firearm violence. Specifically, there is a strong relationship between poverty, income inequality, and firearm violence; as such, interventions that influence upstream determinants of health by providing income support may hold much promise in affecting multiple domains of risk that are on the causal pathway to firearm violence. Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews, we conducted a scoping review to examine the current state of evidence on the relationship between income support policies and risk of firearm violence. We searched 8 databases related to health and social sciences from inception through March 30, 2022, and placed no time, language, setting, or other publication restrictions on our search, as long as the study was quantitative or mixed-methods and addressed firearm violence specifically, rather than violence more broadly, as an outcome in relation to income support policies. We found 4 studies; of those, 3 were conducted in the United States and 1 in Brazil. All 4 found associations of policy-relevant magnitude between income support policies and reductions in risk of inter-personal firearm violence. We propose future opportunities to enhance the substantive scope and methodologic rigor of this field of research and inform policy and practice for greater impact.

When Effects Cannot be Estimated: Redefining Estimands to Understand the Effects of Naloxone Access Laws

Violations of the positivity assumption (also called the common support condition) challenge health policy research and can result in significant bias, large variance, and invalid inference. We define positivity in the single- and multiple-timepoint (i.e., longitudinal) health policy evaluation setting, and discuss real-world threats to positivity. We show empirical evidence of the practical positivity violations that can result when attempting to estimate the effects of health policies (in this case, Naloxone Access Laws). In such scenarios, an alternative is to estimate the effect of a shift in law enactment (e.g., the effect if enactment had been delayed by some number of years). Such an effect corresponds to what is called a modified treatment policy, and dramatically weakens the required positivity assumption, thereby offering a means to estimate policy effects even in scenarios with serious positivity problems. We apply the approach to define and estimate the longitudinal effects of Naloxone Access Laws on opioid overdose rates.

Changes in Racial/Ethnic Disparities in Opioid-Related Outcomes in Urban Areas during the COVID-19 Pandemic: A Rapid Review of the Literature

Opioid use disorders (OUDs) are increasingly common among minoritized populations, who have historically experienced limited access to healthcare, a situation that may have worsened during the COVID-19 pandemic. Using a structured keyword search in Pubmed, we reviewed the literature to synthesize the evidence on changes in racial/ethnic disparities in OUD-related outcomes in urban areas during the COVID-19 pandemic in the US. Nine articles were included in the final analysis. Six found increases in OUD-related outcomes during the pandemic, with four showing a widening of disparities. Results also point to the worsening of opioid outcomes among Black and Latinx individuals related to shelter-in-place or stay-at-home orders. Studies examining the use of telehealth and access to OUD treatment showed that minoritized groups have benefited from telehealth programs. The limited number of studies in a small number of jurisdictions indicate a gap in research examining the intersection between COVID-19 and OUD-related outcomes with a focus on disparities. More research is needed to understand the impact of the COVID-19 pandemic and related policies on OUD outcomes among racial/ethnic minoritized groups, including examining the impact of service disruptions on vulnerable groups with OUD.

Methodological considerations for estimating policy effects in the context of co-occurring policies

Understanding how best to estimate state-level policy effects is important, and several unanswered questions remain, particularly about the ability of statistical models to disentangle the effects of concurrently enacted policies. In practice, many policy evaluation studies do not attempt to control for effects of co-occurring policies, and this issue has not received extensive attention in the methodological literature to date. In this study, we utilized Monte Carlo simulations to assess the impact of co-occurring policies on the performance of commonly-used statistical models in state policy evaluations. Simulation conditions varied effect sizes of the co-occurring policies and length of time between policy enactment dates, among other factors. Outcome data (annual state-specific opioid mortality rate per 100,000) were obtained from 1999 to 2016 National Vital Statistics System (NVSS) Multiple Cause of Death mortality files, thus yielding longitudinal annual state-level data over 18 years from 50 states. When co-occurring policies are ignored (i.e., omitted from the analytic model), our results demonstrated that high relative bias (> 82%) arises, particularly when policies are enacted in rapid succession. Moreover, as expected, controlling for all co-occurring policies will effectively mitigate the threat of confounding bias; however, effect estimates may be relatively imprecise (i.e., larger variance) when policies are enacted in near succession. Our findings highlight several key methodological issues regarding co-occurring policies in the context of opioid-policy research yet also generalize more broadly to evaluation of other state-level policies, such as policies related to firearms or COVID-19, showcasing the need to think critically about co-occurring policies that are likely to influence the outcome when specifying analytic models.

Heterogeneous treatment effects in social policy studies: An assessment of contemporary articles in the health and social sciences

PURPOSE

. Social policies are important determinants of population health but may have varying effects on subgroups of people. Evaluating heterogeneous treatment effects (HTEs) of social policies is critical to determine how social policies will affect health inequities. Methods for evaluating HTEs are not standardized. Little is known about how often and by what methods HTEs are assessed in social policy and health research.

METHODS

. A sample of 55 articles from 2019 on the health effects of social policies were evaluated for frequency of reporting HTEs; for what subgroupings HTEs were reported; frequency of a priori specification of intent to assess HTEs; and methods used for assessing HTEs.

RESULTS

. 24 (44%) studies described some form of HTE assessment, including by age, gender, education, race/ethnicity, and/or geography. Among studies assessing HTEs, 63% specified HTE assessment a priori, and most (71%) used descriptive methods such as stratification; 21% used statistical tests (e.g., interaction terms in a regression); and no studies used data-driven algorithms.

CONCLUSIONS

. Although understanding HTEs could enhance policy and practice-based efforts to reduce inequities, it is not routine research practice. Increased evaluation of HTEs across relevant subgroups is needed.

Characterizing the Landscape of Safety Net Programs and Policies in California during the COVID-19 Pandemic

The COVID-19 pandemic prompted rapid and innovative policymaking around the world at the national, regional, and local levels. There has been limited work to systematically document and characterize new and expanded local U.S. pandemic-era policies, which is imperative to better understand the policy variation and resulting health impacts during this unprecedented time. California, the most populous U.S. state, provides a case example of a particularly active policy response. The aim of this Brief Report is to summarize the creation and potential areas of application of a newly created publicly available California- and US-based COVID-19 policy database. We generated an extensive list of California and US policies that were modified or created in response to the COVID-19 pandemic. From July-November 2021, we searched current and historical California and federal government websites, press releases, social media, and news sources and recorded detailed information on these policies, including coverage dates, eligibility criteria, and benefit amounts. This comprehensive dataset includes 39 public health, economic, housing, and safety net programs and policies implemented at both federal and state levels and provides details of the complex and multifaceted policy landscape in California from March 2020 to November 2021. Our database is publicly available. Future investigators can leverage the information systematically recorded in this database to rigorously assess the short- and long-term effects of these policies, which will in turn inform future preparedness response plans in California and beyond.

What to Do When Everything Happens at Once: Analytic Approaches to Estimate the Health Effects of Co-Occurring Social Policies

Social policies have great potential to improve population health and reduce health disparities. Increasingly, those doing empirical research have sought to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence that must be handled analytically for valid inferences. Although this is a substantial methodological challenge for researchers aiming to isolate social policy effects, only in a limited number of studies have researchers systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated 7 analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We used an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only in 17 (31%) did authors report checking for any co-occurring policies, although in 36 studies (67%), at least 1 approach was used that helps address policy co-occurrence. The most common approaches were adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen studies on the health effects of social policies.