Fatal overdose: Predicting to prevent

A flexible framework for local-level estimation of the effective reproductive number in geographic regions with sparse data

BACKGROUND

Our research focuses on local-level estimation of the effective reproductive number, which describes the transmissibility of an infectious disease and represents the average number of individuals one infectious person infects at a given time. The ability to accurately estimate the infectious disease reproductive number in geographically granular regions is critical for disaster planning and resource allocation. However, not all regions have sufficient infectious disease outcome data; this lack of data presents a significant challenge for accurate estimation.

METHODS

To overcome this challenge, we propose a two-step approach that incorporates existing [Formula: see text] estimation procedures (EpiEstim, EpiFilter, EpiNow2) using data from geographic regions with sufficient data (step 1), into a covariate-adjusted Bayesian Integrated Nested Laplace Approximation (INLA) spatial model to predict [Formula: see text] in regions with sparse or missing data (step 2). Our flexible framework effectively allows us to implement any existing estimation procedure for [Formula: see text] in regions with coarse or entirely missing data. We perform external validation and a simulation study to evaluate the proposed method and assess its predictive performance.

RESULTS

We applied our method to estimate [Formula: see text]using data from South Carolina (SC) counties and ZIP codes during the first COVID-19 wave ('Wave 1', June 16, 2020 - August 31, 2020) and the second wave ('Wave 2', December 16, 2020 - March 02, 2021). Among the three methods used in the first step, EpiNow2 yielded the highest accuracy of [Formula: see text] prediction in the regions with entirely missing data. Median county-level percentage agreement (PA) was 90.9% (Interquartile Range, IQR: 89.9-92.0%) and 92.5% (IQR: 91.6-93.4%) for Wave 1 and 2, respectively. Median zip code-level PA was 95.2% (IQR: 94.4-95.7%) and 96.5% (IQR: 95.8-97.1%) for Wave 1 and 2, respectively. Using EpiEstim, EpiFilter, and an ensemble-based approach yielded median PA ranging from 81.9 to 90.0%, 87.2-92.1%, and 88.4-90.9%, respectively, across both waves and geographic granularities.

CONCLUSION

These findings demonstrate that the proposed methodology is a useful tool for small-area estimation of [Formula: see text], as our flexible framework yields high prediction accuracy for regions with coarse or missing data.

Is fentanyl in everything? Examining the unexpected occurrence of illicit opioids in British Columbia's drug supply

BACKGROUND

Illicit opioids, including fentanyl, are linked to unprecedented levels of overdose in Canada and elsewhere. The risks associated with illicit opioids can include high potency, unpredictable concentration and the unexpected presence in other drugs. Within this context, we examine drug checking data to better understand the presence of illicit opioids such as fentanyl in other drugs and possible ways to interpret these results.

METHODS

Three years (2021-2023) of data (18,474 samples) from Substance Drug Checking in British Columbia, Canada were examined to investigate the risks associated with the detection of opioids in other drugs such as cocaine and methamphetamine, as well as in other drug categories. Samples were tested by paper spray mass spectrometry (PS-MS), fentanyl test strips and Fourier-Transform infrared spectroscopy (FTIR). We examine the 8889 samples not expected to include fentanyl to confirm; if the expected drug was detected, if unexpected opioids were detected, and when the unexpected opioids are in trace concentration.

RESULTS

Unexpected opioids were rarely detected (2%) in other drugs (189 of 8889 samples) with most (61.4%) detected at trace concentration levels. Unexpected opioids are far more likely to be found in samples that did not contain the expected drug than in samples that were confirmed to contain the expected drug. The least common scenario (below 1%) were substances that included the expected drug plus unexpected opioid above trace concentration. These findings raise questions on how to interpret and communicate the detection of fentanyl and related opioids in other drugs. We present three potential interpretations: (1) mistaken and misrepresented samples where the expected drug was never detected, (2) cross contamination when opioids were at trace concentration levels, or (3) adulteration as the least frequent scenario where opioids were detected above trace concentrations in combination with the expected drug.

CONCLUSIONS

In a region where fentanyl is associated with extreme rates of overdose, it remains rare to find such opioids in other drugs. However, the risk of fentanyl in other drugs remains an ongoing threat that warrants responses by individuals and public health. We provide possible interpretations to inform such responses. Our data raises questions on how to interpret and communicate the detection of fentanyl and other opioids in other drugs.

A Flexible Framework for Local-Level Estimation of the Effective Reproductive Number in Geographic Regions with Sparse Data

Background

Our research focuses on local-level estimation of the effective reproductive number, which describes the transmissibility of an infectious disease and represents the average number of individuals one infectious person infects at a given time. The ability to accurately estimate the infectious disease reproductive number in geographically granular regions is critical for disaster planning and resource allocation. However, not all regions have sufficient infectious disease outcome data; this lack of data presents a significant challenge for accurate estimation.

Methods

To overcome this challenge, we propose a two-step approach that incorporates existingR t estimation procedures (EpiEstim, EpiFilter, EpiNow2) using data from geographic regions with sufficient data (step 1), into a covariate-adjusted Bayesian Integrated Nested Laplace Approximation (INLA) spatial model to predictR t in regions with sparse or missing data (step 2). Our flexible framework effectively allows us to implement any existing estimation procedure forR t in regions with coarse or entirely missing data. We perform external validation and a simulation study to evaluate the proposed method and assess its predictive performance.

Results

We applied our method to estimateR t using data from South Carolina (SC) counties and ZIP codes during the first COVID-19 wave ('Wave 1', June 16, 2020 - August 31, 2020) and the second wave ('Wave 2', December 16, 2020 - March 02, 2021). Among the three methods used in the first step, EpiNow2 yielded the highest accuracy ofR t prediction in the regions with entirely missing data. Median county-level percentage agreement (PA) was 90.9% (Interquartile Range, IQR: 89.9-92.0%) and 92.5% (IQR: 91.6-93.4%) for Wave 1 and 2, respectively. Median zip code-level PA was 95.2% (IQR: 94.4-95.7%) and 96.5% (IQR: 95.8-97.1%) for Wave 1 and 2, respectively. Using EpiEstim, EpiFilter, and an ensemble-based approach yielded median PA ranging from 81.9%-90.0%, 87.2%-92.1%, and 88.4%-90.9%, respectively, across both waves and geographic granularities.

Conclusion

These findings demonstrate that the proposed methodology is a useful tool for small-area estimation ofR t , as our flexible framework yields high prediction accuracy for regions with coarse or missing data.

"Sometimes I'm interested in seeing a fuller story to tell with numbers" Implementing a forecasting dashboard for harm reduction and overdose prevention: a qualitative assessment

OBJECTIVES

The escalating overdose crisis in the United States points to the urgent need for new and novel data tools. Overdose data tools are growing in popularity but still face timely delays in surveillance data availability, lack of completeness, and wide variability in quality by region. As such, we need innovative tools to identify and prioritize emerging and high-need areas. Forecasting offers one such solution. Machine learning methods leverage numerous datasets that could be used to predict future vulnerability to overdose at the regional, town, and even neighborhood levels. This study aimed to understand the multi-level factors affecting the early stages of implementation for an overdose forecasting dashboard. This dashboard was developed with and for statewide harm reduction providers to increase data-driven response and resource distribution at the neighborhood level.

METHODS

As part of PROVIDENT (Preventing OVerdose using Information and Data from the EnvironmeNT), a randomized, statewide community trial, we conducted an implementation study where we facilitated three focus groups with harm reduction organizations enrolled in the larger trial. Focus group participants held titles such as peer outreach workers, case managers, and program coordinators/managers. We employed the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework to guide our analysis. This framework offers a multi-level, four-phase analysis unique to implementation within a human services environment to assess the exploration and preparation phases that influenced the early launch of the intervention.

RESULTS

Multiple themes centering on organizational culture and resources emerged, including limited staff capacity for new interventions and repeated exposure to stress and trauma, which could limit intervention uptake. Community-level themes included the burden of data collection for program funding and statewide efforts to build stronger networks for data collection and dashboarding and data-driven resource allocation.

DISCUSSION

Using an implementation framework within the larger study allowed us to identify multi-level and contextual factors affecting the early implementation of a forecasting dashboard within the PROVIDENT community trial. Additional investments to build organizational and community capacity may be required to create the optimal implementation setting and integration of forecasting tools.

Overdose risk environment for people who use drugs in New Jersey: Imagining possible points of intervention for harm reduction practitioners

Background

The Risk Environment Framework is widely utilized theoretical framework for understanding the landscape of harm for people who use drugs (PWUD). This study sought to understand factors contributing to risk of overdose for PWUD in New Jersey. Understanding these factors can lead to improved policy interventions, programmatic targets, and a shared understanding that overdose risk is impacted by larger societal forces influencing PWUD.

Methods

Using a community based participatory design model, this study conducted 30 semi-structured, in-depth interviews with PWUD and naloxone distributors in New Brunswick and Newark, New Jersey from February to November of 2022. Thematic analysis was performed using a collaborative analytical approach.

Results

Risk factors for overdose fell into all four categories of Rhodes's Risk Environment Framework - physical, social, economic, and policy. Many factors overlapped in multiple categories, and most factors had elements existing at both the macro and micro levels.

Conclusions

Interventions supporting PWUD should see overdose risk as an environmental, structural consideration, and be constructed to address comprehensive risks, rather than directing themselves exclusively at the individual level. Factors contributing to risk at the macro level included systemic and institutional concerns and stigma toward PWUD. At the micro level, mental health, substance use behaviors, treatment and recovery, and trauma were cited as potential risk factors.

A deep learning analysis for dual healthcare system users and risk of opioid use disorder

The opioid crisis has disproportionately affected U.S. veterans, leading the Veterans Health Administration to implement opioid prescribing guidelines. Veterans who receive care from both VA and non-VA providers-known as dual-system users-have an increased risk of Opioid Use Disorder (OUD). The interaction between dual-system use and demographic and clinical factors, however, has not been previously explored. We conducted a retrospective study of 856,299 patient instances from the Washington DC and Baltimore VA Medical Centers (2012-2019), using a deep neural network (DNN) and explainable Artificial Intelligence to examine the impact of dual-system use on OUD and how demographic and clinical factors interact with it. Of the cohort, 146,688(17%) had OUD, determined through Natural Language Processing of clinical notes and ICD-9/10 diagnoses. The DNN model, with a 78% area under the curve, confirmed that dual-system use is a risk factor for OUD, along with prior opioid use or other substance use. Interestingly, a history of other drug use interacted negatively with dual-system use regarding OUD risk. In contrast, older age was associated with a lower risk of OUD but interacted positively with dual-system use. These findings suggest that within the dual-system users, patients with certain risk profiles warrant special attention.

Drug Overdose Deaths in Mexican-Heritage Arizonans: An Examination of Mortality Rates, Demographics, Drugs Involved, and Place of Death

OBJECTIVE

This study examined drug overdose deaths in Mexican-heritage Arizonans, with the goal of informing tailored overdose prevention programs for this community.

METHODS

We analyzed death certificate data (from the Arizona Department of Health Services) for drug overdose deaths among Arizona residents from 2018-2022. We compared deaths in US-born and foreign-born Mexican-heritage Arizonans and, as a frame of reference, Non-Hispanic (NH) White Arizonans. We compared demographics, circumstances of death, and mortality rates, using descriptive statistics, multinomial logistic regression models, and age-standardized mortality rates and ratios.

RESULTS

The age-standardized drug overdose mortality rate (per 100,000) was lower in the overall Mexican-heritage population (28.0) than in the NH White population (35.9). Nonetheless, the rate in the US-born Mexican-heritage male subgroup (59.5) was higher than in US-born NH White males (49.9) or any other subgroup examined. Synthetic opioids such as fentanyl were involved in higher proportions of deaths among US-born (64.6%) and foreign-born (65.1%) Mexican-heritage Arizonans than among NH White Arizonans (48.5%). In multinomial regression models, the risk of a medical place of death, relative to death at home, was significantly higher in the foreign-born (adjusted Relative Risk Ratio [aRRR] 1.82; 95% Confidence Interval [CI], 1.38-2.42) and US-born (aRRR 1.85; 95% CI, 1.62-2.11) Mexican-heritage groups than the NH White group, adjusting for age, sex, marital status, county of residence, overdose intent, and drugs involved.

CONCLUSIONS

Findings highlight disparate rates of overdose mortality in US-born Mexican-heritage Arizona men, also underscoring racial/ethnic/nativity-based differences in overdose circumstances and decedent characteristics.

Substance Use Epidemiology as a Foundation for Prevention

In the 50 years since its establishment, the National Institute on Drug Abuse has made significant investment and strides toward improving individual and public health. Epidemiology serves as the foundation for understanding the how many, why, how, where, and who of drug use and its consequences, and effective epidemiology research and training are geared toward actionable findings that can inform real-world responses. Epidemiologic findings enhance clinicians' ability to provide ongoing care by incorporating information about the patterns and outcomes of drug use that their patients may experience. The goal of this article is to provide a context for epidemiology of substance use as a foundation for prevention, with examples of how epidemiology can provide targets for prevention, and to set the stage for addressing the importance of prevention in clinical settings.

Identification of factors associated with opioid-related and hepatitis C virus-related hospitalisations at the ZIP code area level in the USA: an ecological and modelling study

BACKGROUND

Opioid overdose and related diseases remain a growing public health crisis in the USA. Identifying sociostructural and other contextual factors associated with adverse health outcomes is needed to improve prediction models to inform policy and interventions. We aimed to identify high-risk communities for targeted delivery of screening and prevention interventions for opioid use disorder and hepatitis C virus (HCV).

METHODS

In this ecological and modelling study, we fit mixed-effects negative binomial regression models to identify factors associated with, and predict, opioid-related and HCV-related hospitalisations for ZIP code tabulation areas (ZCTAs) in South Carolina, USA. All individuals aged 18 years or older living in South Carolina from Jan 1, 2016, to Dec 31, 2021, were included. Data on opioid-related and HCV-related hospitalisations, as well as data on additional individual-level variables, were collected from medical claims records, which were obtained from the South Carolina Revenue and Fiscal Affairs Office. Demographic and socioeconomic variables were obtained from the United States Census Bureau (American Community Survey, 2021) with additional structural health-care barrier data obtained from South Carolina's Center for Rural and Primary Health Care, and the American Hospital Directory.

FINDINGS

Between Jan 1, 2016, and Dec 31, 2021, 41 691 individuals were hospitalised for opioid misuse and 26 860 were hospitalised for HCV. There were a median of 80 (IQR 24-213) opioid-related hospitalisations and 61 (21-196) HCV-related hospitalisations per ZCTA. A standard deviation increase in ZCTA-level uninsured rate (relative risk 1·24 [95% CI 1·17-1·31]), poverty rate (1·24 [1·17-1·31]), mortality (1·18 [1·12-1·25]), and social vulnerability index (1·17 [1·10-1·24]) was significantly associated with increased combined opioid-related and HCV-related hospitalisation rates. A standard deviation increase in ZCTA-level income (0·79 [0·75-0·84]) and unemployment rate (0·87 [0·82-0·93]) was significantly associated with decreased combined opioid-related and HCV-related hospitalisations. Using 2016-20 hospitalisations as training data, our models predicted ZCTA-level opioid-related hospitalisations in 2021 with a median of 80·4% (IQR 66·8-91·1) accuracy and HCV-related hospitalisations in 2021 with a median of 75·2% (61·2-87·7) accuracy. Several underserved high-risk ZCTAs were identified for delivery of targeted interventions.

INTERPRETATION

Our results suggest that individuals from economically disadvantaged and medically under-resourced communities are more likely to have an opioid-related or HCV-related hospitalisation. In conjunction with hospitalisation forecasts, our results could be used to identify and prioritise high-risk, underserved communities for delivery of field-level interventions.

FUNDING

South Carolina Center for Rural and Primary Healthcare, National Institute on Drug Abuse, and National Library of Medicine.

Beyond a spec: assessing heterogeneity in the unregulated opioid supply

BACKGROUND

Drug checking services aim to provide compositional information for the illicit drug supply and are being employed in public health responses to extreme rates of overdose associated with fentanyl within street opioids. The technologies used within these services range from basic qualitative tests, such as immunoassay test strips, to comprehensive quantitative analyses, such as mass spectrometry. In general, there is concern that heterogeneity of a drug mixture adds significant uncertainty when using drug checking results based on a small subsamples. The presence of hot spots of active drug components in this context is often termed the 'chocolate chip cookie effect'. Establishing the limitations of the service are essential for interpretation of the results.

METHODS

This study assesses the consequence of drug heterogeneity and sampling of consumer level opioid purchased in Victoria, British Columbia ( n = 21 , 50-100 mg each) on quantitative fentanyl results determined from testing with paper spray mass spectrometry.

RESULTS

Using descriptive statistics, such as relative standard deviation and interquartile range, the results demonstrate varied distributions of fentanyl concentrations within a single drug batch. However, the presence of hot spots, defined as outliers, were relatively rare.

CONCLUSIONS

This study found that the variability in fentanyl concentration from drug heterogeneity and sampling is greater than that attributed to the analytical technique. On a practical level, this provides data to help guide communication of limitations of drug checking services, supporting the aim of trust and transparency between services and people who use drugs. However, if drug checking services continue to be restricted from fully engaging with the reality of manufacturing, buying, selling, mixing and dosing practices, the accuracy, usefulness, and impact will always be limited.

Forecasting drug-overdose mortality by age in the United States at the national and county levels

The drug-overdose crisis in the United States continues to intensify. Fatalities have increased 5-fold since 1999 reaching a record high of 108,000 deaths in 2021. The epidemic has unfolded through distinct waves of different drug types, uniquely impacting various age, gender, race, and ethnic groups in specific geographical areas. One major challenge in designing interventions and efficiently delivering treatment is forecasting age-specific overdose patterns at the local level. To address this need, we develop a forecasting method that assimilates observational data obtained from the CDC WONDER database with an age-structured model of addiction and overdose mortality. We apply our method nationwide and to three select areas: Los Angeles County, Cook County, and the five boroughs of New York City, providing forecasts of drug-overdose mortality and estimates of relevant epidemiological quantities, such as mortality and age-specific addiction rates.

Changing dynamics of drug overdoses in the United Kingdom: An attempt to replicate the Jalal et al. findings of steady exponential growth

BACKGROUND

Jalal et al. discovered that between 1979 and 2020 total rates and counts of fatal drug overdoses in the United States exhibited exponential growth at a very steady rate even though deaths from individual drugs did not. That is a startling result because it means that the different drugs are in effect "taking turns", with one growing faster just as another drug's death rate growth ebbs. That raises the question of whether this steadiness in the all-drug death rates is in some sense just a coincidence peculiar to the United States or whether it might reflect some more general phenomenon and so manifest in other countries.

METHODS

We fit the same model used by Jalal et al. to data on drug-related death rates for the countries of the United Kingdom.

RESULTS

The main finding is largely a failure to replicate the United States result. Simple graphical display of the trends and a number of statistical measures show that the growth in the United Kingdom was not only slower than in the United States, it was also less steady, with the exception of Northern Ireland.

CONCLUSIONS

Steady exponential growth in the all-drugs mortality rate may be a phenomenon specific to certain contexts. It remains an open question whether the explanation of steady exponential growth in the United States and Northern Ireland relates to demand and supply mechanisms, to social and political conditions, or to coincidence.

Drug checking in the fentanyl era: Utilization and interest among people who inject drugs in San Diego, California

BACKGROUND

In North America, overdose rates have steeply risen over the past five years, largely due to the ubiquity of illicitly manufactured fentanyls in the drug supply. Drug checking services (DCS) represent a promising harm reduction strategy and characterizing experiences of use and interest among people who inject drugs (PWID) is a priority.

METHODS

Between February-October 2022, PWID participating in a cohort study in San Diego, CA and Tijuana, Mexico completed structured surveys including questions about DCS, socio-demographics and substance use behaviors. We used Poisson regression to assess factors associated with lifetime DCS use and characterized experiences with DCS and interest in free access to DCS.

RESULTS

Of 426 PWID, 72% were male, 59% Latinx, 79% were experiencing homelessness and 56% ever experienced a nonfatal overdose. One third had heard of DCS, of whom 57% had ever used them. Among the latter, most (98%) reported using fentanyl test strips (FTS) the last time they used DCS; 66% did so less than once per month. In the last six months, respondents used FTS to check methamphetamine (48%), heroin (30%) or fentanyl (29%). Relative to White/non-Latinx PWID, those who were non-White/Latinx were significantly less likely to have used DCS [adjusted risk ratio (aRR): 0.22; 95% CI: 0.10, 0.47), as were PWID experiencing homelessness (aRR:0.45; 95% CI: 0.28, 0.72). However, a significant interaction indicated that non-White/Latinx syringe service program (SSP) clients were more likely to have used DCS than non-SSP clients (aRR: 2.79; CI: 1.09, 7.2). Among all PWID, 44% expressed interest in free access to FTS, while 84% (of 196 PWID) expressed interest in advanced spectrometry DCS to identify and quantify multiple substances.

CONCLUSIONS

Our findings highlight low rates of DCS awareness and utilization, inequities by race/ethnicity and housing situation, high interest in advanced spectrometry DCS versus FTS, and the potential role of SSPs in improving access to DCS, especially among racial/ethnic minorities.

Early warnings and slow deaths: A sociology of outbreak and overdose

In this paper, we offer a sociological analysis of early warning and outbreak in the field of drug policy, focusing on opioid overdose. We trace how 'outbreak' is enacted as a rupturing event which enables rapid reflex responses of precautionary control, based largely on short-term and proximal early warning indicators. We make the case for an alternative view of early warning and outbreak. We argue that practices of detection and projection that help to materialise drug-related outbreaks are too focused on the proximal and short-term. Engaging with epidemiological and sociological work investigating epidemics of opioid overdose, we show how the short-termism and rapid reflex response of outbreak fails to appreciate the slow violent pasts of epidemics indicative of an ongoing need and care for structural and societal change. Accordingly, we gather together ideas of 'slow emergency' (Ben Anderson), 'slow death' (Lauren Berlant) and 'slow violence' (Rob Nixon), to re-assemble outbreaks in 'long view'. This locates opioid overdose in long-term attritional processes of deindustrialisation, pharmaceuticalisation, and other forms of structural violence, including the criminalisation and problematisation of people who use drugs. Outbreaks evolve in relation to their slow violent pasts. To ignore this can perpetuate harm. Attending to the social conditions that create the possibilities for outbreak invites early warning that goes 'beyond outbreak' and 'beyond epidemic' as generally configured.

Small Area Forecasting of Opioid-Related Mortality: Bayesian Spatiotemporal Dynamic Modeling Approach

BACKGROUND

Opioid-related overdose mortality has remained at crisis levels across the United States, increasing 5-fold and worsened during the COVID-19 pandemic. The ability to provide forecasts of opioid-related mortality at granular geographical and temporal scales may help guide preemptive public health responses. Current forecasting models focus on prediction on a large geographical scale, such as states or counties, lacking the spatial granularity that local public health officials desire to guide policy decisions and resource allocation.

OBJECTIVE

The overarching objective of our study was to develop Bayesian spatiotemporal dynamic models to predict opioid-related mortality counts and rates at temporally and geographically granular scales (ie, ZIP Code Tabulation Areas [ZCTAs]) for Massachusetts.

METHODS

We obtained decedent data from the Massachusetts Registry of Vital Records and Statistics for 2005 through 2019. We developed Bayesian spatiotemporal dynamic models to predict opioid-related mortality across Massachusetts' 537 ZCTAs. We evaluated the prediction performance of our models using the one-year ahead approach. We investigated the potential improvement of prediction accuracy by incorporating ZCTA-level demographic and socioeconomic determinants. We identified ZCTAs with the highest predicted opioid-related mortality in terms of rates and counts and stratified them by rural and urban areas.

RESULTS

Bayesian dynamic models with the full spatial and temporal dependency performed best. Inclusion of the ZCTA-level demographic and socioeconomic variables as predictors improved the prediction accuracy, but only in the model that did not account for the neighborhood-level spatial dependency of the ZCTAs. Predictions were better for urban areas than for rural areas, which were more sparsely populated. Using the best performing model and the Massachusetts opioid-related mortality data from 2005 through 2019, our models suggested a stabilizing pattern in opioid-related overdose mortality in 2020 and 2021 if there were no disruptive changes to the trends observed for 2005-2019.

CONCLUSIONS

Our Bayesian spatiotemporal models focused on opioid-related overdose mortality data facilitated prediction approaches that can inform preemptive public health decision-making and resource allocation. While sparse data from rural and less populated locales typically pose special challenges in small area predictions, our dynamic Bayesian models, which maximized information borrowing across geographic areas and time points, were used to provide more accurate predictions for small areas. Such approaches can be replicated in other jurisdictions and at varying temporal and geographical levels. We encourage the formation of a modeling consortium for fatal opioid-related overdose predictions, where different modeling techniques could be ensembled to inform public health policy.