Stochastic optimization for vaccine and testing kit allocation for the COVID-19 pandemic

Reinforcement learning for healthcare operations management: methodological framework, recent developments, and future research directions

With the advancement in computing power and data science techniques, reinforcement learning (RL) has emerged as a powerful tool for decision-making problems in complex systems. In recent years, the research on RL for healthcare operations has grown rapidly. Especially during the COVID-19 pandemic, RL has played a critical role in optimizing decisions with greater degrees of uncertainty. RL for healthcare applications has been an exciting topic across multiple disciplines, including operations research, operations management, healthcare systems engineering, and data science. This review paper first provides a tutorial on the overall framework of RL, including its key components, training models, and approximators. Then, we present the recent advances of RL in the domain of healthcare operations management (HOM) and analyze the current trends. Our paper concludes by presenting existing challenges and future directions for RL in HOM.

Chance-constrained stochastic optimal control of epidemic models: A fourth moment method-based reformulation

This work proposes a methodology for the reformulation of chance-constrained stochastic optimal control problems that ensures reliable uncertainty management of epidemic outbreaks. Specifically, the chance constraints are reformulated in terms of the first four moments of the stochastic state variables through the so-called fourth moment method for reliability. Moreover, a spectral technique is employed to obtain surrogate models of the stochastic state variables, which enables the efficient computation of the required statistics. The practical implementation of the proposed approach is demonstrated via the optimal control of two different stochastic mathematical models of the COVID-19 transmission. The numerical experiments confirm that, unlike those reformulations based on the Chebyshev-Cantelli's inequality, the proposed method does not exhibit the undesired outcomes that are typically observed when a higher precision is required for the risk level associated to the given chance constraints.

Optimization Modeling for Pandemic Vaccine Supply Chain Management: A Review and Future Research Opportunities

During various stages of the COVID-19 pandemic, countries implemented diverse vaccine management approaches, influenced by variations in infrastructure and socio-economic conditions. This article provides a comprehensive overview of optimization models developed by the research community throughout the COVID-19 era, aimed at enhancing vaccine distribution and establishing a standardized framework for future pandemic preparedness. These models address critical issues such as site selection, inventory management, allocation strategies, distribution logistics, and route optimization encountered during the COVID-19 crisis. A unified framework is employed to describe the models, emphasizing their integration with epidemiological models to facilitate a holistic understanding. This article also summarizes evolving nature of literature, relevant research gaps, and authors' perspectives for model selection. Finally, future research scopes are detailed both in the context of modeling and solutions approaches.

Analyzing economic effect on mRNA vaccine inventory management with redistribution policy

This study focused on the challenges posed by the fluctuating demand for COVID-19 vaccines, considering factors such as side effects, religious objections, and absenteeism, which result in the accumulation of excess vaccines. Recognizing the resulting social, economic, and environmental issues, this study investigated the application of a lateral transshipment policy for the management of the inventory of short-term vaccines, considering related unpredictabilities. A discrete event simulation built on foundational principles derived from a mixed-integer linear programming model was employed to explore the dynamics of mRNA-based vaccine distribution among two hospitals based on lateral transshipment and reordering policies. Through the simulation of various scenarios over periods of 1-30 days, transshipment based on the availability policy is employed to determine the quantity of vaccines to be transshipped, constrained to vial amounts, and the (s, S) inventory system for reordering. The results of this study underscore the efficacy of lateral transshipment, particularly in situations where demand discrepancies exist between hospitals, thereby revealing its superiority over non-transshipment strategies within 7 days.

Disease X epidemic control using a stochastic model and a deterministic approximation: Performance comparison with and without parameter uncertainties

BACKGROUND

The spread of infectious diseases can be modeled using deterministic or stochastic models. A deterministic approximation of a stochastic model can be appropriate under some conditions, but is unable to capture the discrete nature of populations. We look into the choice of a model from the perspective of decision making.

METHOD

We consider an emerging disease (Disease X) in a closed population modeled by a stochastic SIR model or its deterministic approximation. The objective of the decision maker is to minimize the cumulative number of symptomatic infected-days over the course of the epidemic by picking a vaccination policy. We consider four decision making scenarios: based on the stochastic model or the deterministic model, and with or without parameter uncertainty. We also consider different sample sizes for uncertain parameter draws and stochastic model runs. We estimate the average performance of decision making in each scenario and for each sample size.

RESULTS

The model used for decision making has an influence on the picked policies. The best achievable performance is obtained with the stochastic model, knowing parameter values, and for a large sample size. For small sample sizes, the deterministic model can outperform the stochastic model due to stochastic effects. Resolving uncertainties may bring more benefit than switching to the stochastic model in our example.

CONCLUSION

This article illustrates the interplay between the choice of a type of model, parameter uncertainties, and sample sizes. It points to issues to be considered when optimizing a stochastic model.

Covid19Vaxplorer: A free, online, user-friendly COVID-19 vaccine allocation comparison tool

There are many COVID-19 vaccines currently available, however, Low- and middle-income countries (LMIC) still have large proportions of their populations unvaccinated. Decision-makers must decide how to effectively allocate available vaccines (e.g. boosters or primary series vaccination, which age groups to target) but LMIC often lack the resources to undergo quantitative analyses of vaccine allocation, resulting in ad-hoc policies. We developed Covid19Vaxplorer (https://covid19vaxplorer.fredhutch.org/), a free, user-friendly online tool that simulates region-specific COVID-19 epidemics in conjunction with vaccination with the purpose of providing public health officials worldwide with a tool for vaccine allocation planning and comparison. We developed an age-structured mathematical model of SARS-CoV-2 transmission and COVID-19 vaccination. The model considers vaccination with up to three different vaccine products, primary series and boosters. We simulated partial immunity derived from waning of natural infection and vaccination. The model is embedded in an online tool, Covid19Vaxplorer that was optimized for its ease of use. By prompting users to fill information through several windows to input local parameters (e.g. cumulative and current prevalence), epidemiological parameters (e.g basic reproduction number, current social distancing interventions), vaccine parameters (e.g. vaccine efficacy, duration of immunity) and vaccine allocation (both by age groups and by vaccination status). Covid19Vaxplorer connects the user to the mathematical model and simulates, in real time, region-specific epidemics. The tool then produces key outcomes including expected numbers of deaths, hospitalizations and cases, with the possibility of simulating several scenarios of vaccine allocation at once for a side-by-side comparison. We provide two usage examples of Covid19Vaxplorer for vaccine allocation in Haiti and Afghanistan, which had as of Spring 2023, 2% and 33% of their populations vaccinated, and show that for these particular examples, using available vaccine as primary series vaccinations prevents more deaths than using them as boosters.

Fair-split distribution of multi-dose vaccines with prioritized age groups and dynamic demand: The case study of COVID-19

The emergence of the SARS-CoV-2 virus and new viral variations with higher transmission and mortality rates have highlighted the urgency to accelerate vaccination to mitigate the morbidity and mortality of the COVID-19 pandemic. For this purpose, this paper formulates a new multi-vaccine, multi-depot location-inventory-routing problem for vaccine distribution. The proposed model addresses a wide variety of vaccination concerns: prioritizing age groups, fair distribution, multi-dose injection, dynamic demand, etc. To solve large-size instances of the model, we employ a Benders decomposition algorithm with a number of acceleration techniques. To monitor the dynamic demand of vaccines, we propose a new adjusted susceptible-infectious-recovered (SIR) epidemiological model, where infected individuals are tested and quarantined. The solution to the optimal control problem dynamically allocates the vaccine demand to reach the endemic equilibrium point. Finally, to illustrate the applicability and performance of the proposed model and solution approach, the paper reports extensive numerical experiments on a real case study of the vaccination campaign in France. The computational results show that the proposed Benders decomposition algorithm is 12 times faster, and its solutions are, on average, 16% better in terms of quality than the Gurobi solver under a limited CPU time. In terms of vaccination strategies, our results suggest that delaying the recommended time interval between doses of injection by a factor of 1.5 reduces the unmet demand up to 50%. Furthermore, we observed that the mortality is a convex function of fairness and an appropriate level of fairness should be adapted through the vaccination.

Risk-based allocation of COVID-19 personal protective equipment under supply shortages

The COVID-19 outbreak put healthcare systems across the globe under immense pressure to meet the unprecedented demand for critical supplies and personal protective equipment (PPE). The traditional cost-effective supply chain paradigm failed to respond to the increased demand, putting healthcare workers (HCW) at a much higher infection risk relative to the general population. Recognizing PPE shortages and high infection risk for HCWs, the World Health Organization (WHO) recommends allocations based on ethical principles. In this paper, we model the infection risk for HCWs as a function of usage and use it as the basis for distribution planning that balances government procurement decisions, hospitals' PPE usage policies, and WHO ethical allocation guidelines. We propose an infection risk model that integrates PPE allocation decisions with disease progression estimates to quantify infection risk among HCWs. The proposed risk function is used to derive closed-form allocation decisions under WHO ethical guidelines in both deterministic and stochastic settings. The modelling is then extended to dynamic distribution planning. Although nonlinear, we reformulate the resulting model to make it solvable using off-the-shelf software. The risk function successfully accounts for virus prevalence in space and in time and leads to allocations that are sensitive to the differences between regions. Comparative analysis shows that the allocation policies lead to significantly different levels of infection risk, especially under high virus prevalence. The best-outcome allocation policy that aims to minimize the total infected cases outperforms other policies under this objective and that of minimizing the maximum number of infections per period.

Behavioral Analytics for Myopic Agents

Many multi-agent systems have a single coordinator providing incentives to a large number of agents. Two challenges faced by the coordinator are a finite budget from which to allocate incentives, and an initial lack of knowledge about the utility function of the agents. Here, we present a behavioral analytics approach for solving the coordinator's problem when the agents make decisions by maximizing utility functions that depend on prior system states, inputs, and other parameters that are initially unknown. Our behavioral analytics framework involves three steps: first, we develop a model that describes the decision-making process of an agent; second, we use data to estimate the model parameters for each agent and predict their future decisions; and third, we use these predictions to optimize a set of incentives that will be provided to each agent. The framework and approaches we propose in this paper can then adapt incentives as new information is collected. Furthermore, we prove that the incentives computed by this approach are asymptotically optimal with respect to a loss function that describes the coordinator's objective. We optimize incentives with a decomposition scheme, where each sub-problem solves the coordinator's problem for a single agent, and the master problem is a pure integer program. We conclude with a simulation study to evaluate the effectiveness of our approach for designing a personalized weight loss program. The results show that our approach maintains efficacy of the program while reducing its costs by up to 60%, while adaptive heuristics provide substantially less savings.

Optimal timing of non-pharmaceutical interventions during an epidemic

In response to the recent outbreak of the SARS-CoV-2 virus governments have aimed to reduce the virus's spread through, inter alia, non-pharmaceutical intervention. We address the question when such measures should be implemented and, once implemented, when to remove them. These issues are viewed through a real-options lens and we develop an SIRD-like continuous-time Markov chain model to analyze a sequence of options: the option to intervene and introduce measures and, after intervention has started, the option to remove these. Measures can be imposed multiple times. We implement our model using estimates from empirical studies and, under fairly general assumptions, our main conclusions are that: (1) measures should be put in place not long after the first infections occur; (2) if the epidemic is discovered when there are many infected individuals already, then it is optimal never to introduce measures; (3) once the decision to introduce measures has been taken, these should stay in place until the number of susceptible or infected members of the population is close to zero; (4) it is never optimal to introduce a tier system to phase-in measures but it is optimal to use a tier system to phase-out measures; (5) a more infectious variant may reduce the duration of measures being in place; (6) the risk of infections being brought in by travelers should be curbed even when no other measures are in place. These results are robust to several variations of our base-case model.

Designing an optimization model for the vaccine supply chain during the COVID-19 pandemic

The COVID-19 pandemic has affected people's lives worldwide. Among various strategies being applied to addressing such a global crisis, public vaccination has been arguably the most appropriate approach to control a pandemic. However, vaccine supply chain and management have become a new challenge for governments. In this study, a solution for the vaccine supply chain is presented to address the hurdles in the public vaccination program according to the concerns of the government and the organizations involved. For this purpose, a robust bi-level optimization model is proposed. At the upper level, the risk of mortality due to the untimely supply of the vaccine and the risk of inequality in the distribution of the vaccine is considered. All costs related to the vaccine supply chain are considered at the lower level, including the vaccine supply, allocation of candidate centers for vaccine injection, cost of maintenance and injection, transportation cost, and penalty cost due to the vaccine shortage. In addition, the uncertainty of demand for vaccines is considered with multiple scenarios of different demand levels. Numerical experiments are conducted based on the vaccine supply chain in Kermanshah, Iran, and the results show that the proposed model significantly reduces the risk of mortality and inequality in the distribution of vaccines as well as the total cost, which leads to managerial insights for better coordination of the vaccination network during the COVID-19 pandemic.

On the drivers of drug shortages: empirical evidence from Germany

Purpose

Disruptions and shortages of drugs have become severe problems in recent years, which has triggered strong media and public interest in the topic. However, little is known about the factors that can be associated with the increased frequency of shortages. In this paper, the authors analyze the drivers of drug shortages using empirical data for Germany, the fourth largest pharmaceutical market.

Design/methodology/approach

The authors use a dataset provided by the German Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte [BfArM]) with 425 reported shortages for drug substances (DSs) in the 24-month period between May 2017 and April 2019 and enrich the data with information from additional sources. Using logistic and negative binomial regression models, the authors analyze the impact of (1) market characteristics, (2) drug substance characteristics and (3) regulatory characteristics on the likelihood of a shortage.

Findings

The authors find that factors like market concentration, patent situation, manufacturing processes or dosage form are significantly associated with the odds of a shortage. The authors discuss the implications of these findings to reduce the frequency and severity of shortages.

Originality/value

The authors contribute to the empirical research on drug shortages by analyzing the impact of market characteristics, DS characteristics and regulatory characteristics on the reported shortages. The authors’ analysis provides a starting point for better prioritizing efforts to strengthen drug supply as it is currently intensely discussed healthcare authorities.

Propagation dynamics in an SIRS model with general incidence functions

This paper studies the initial value problems and traveling wave solutions in an SIRS model with general incidence functions. Linearizing the infected equation at the disease free steady state, we can define a threshold if the corresponding basic reproduction ratio in kinetic system is larger than the unit. When the initial condition for the infected is compactly supported, we prove that the threshold is the spreading speed for three unknown functions. At the same time, this threshold is the minimal wave speed for traveling wave solutions modeling the disease spreading process. If the corresponding basic reproduction ratio in kinetic system is smaller than the unit, then we confirm the extinction of the infected and the nonexistence of nonconstant traveling waves.

A new vaccine supply chain network under COVID-19 conditions considering system dynamic: Artificial intelligence algorithms

With the discovery of the COVID-19 vaccine, what has always been worrying the decision-makers is related to the distribution management, the vaccination centers' location, and the inventory control of all types of vaccines. As the COVID-19 vaccine is highly demanded, planning for its fair distribution is a must. University is one of the most densely populated areas in a city, so it is critical to vaccinate university students so that the spread of this virus is curbed. As a result, in the present study, a new stochastic multi-objective, multi-period, and multi-commodity simulation-optimization model has been developed for the COVID-19 vaccine's production, distribution, location, allocation, and inventory control decisions. In this study, the proposed supply chain network includes four echelons of manufacturers, hospitals, vaccination centers, and volunteer vaccine students. Vaccine manufacturers send the vaccines to the vaccination centers and hospitals after production. The students with a history of special diseases such as heart disease, corticosteroids, blood clots, etc. are vaccinated in hospitals because of accessing more medical care, and the rest of the students are vaccinated in the vaccination centers. Then, a system dynamic structure of the prevalence of COVID -19 in universities is developed and the vaccine demand is estimated using simulation, in which the demand enters the mathematical model as a given stochastic parameter. Thus, the model pursues some goals, namely, to minimize supply chain costs, maximize student desirability for vaccination, and maximize justice in vaccine distribution. To solve the proposed model, Variable Neighborhood Search (VNS) and Whale Optimization Algorithm (WOA) algorithms are used. In terms of novelties, the most important novelties in the simulation model are considering the virtual education and exerted quarantine effect on estimating the number of the vaccines. In terms of the mathematical model, one of the remarkable contributions is paying attention to social distancing while receiving the injection and the possibility of the injection during working and non-working hours, and regarding the novelties in the solution methodology, a new heuristic method based on a meta-heuristic algorithm called Modified WOA with VNS (MVWOA) is developed. In terms of the performance metrics and the CPU time, the MOWOA is discovered with a superior performance than other given algorithms. Moreover, regarding the data, a case study related to the COVID-19 pandemic period in Tehran/Iran is provided to validate the proposed algorithm. The outcomes indicate that with the demand increase, the costs increase sharply while the vaccination desirability for students decreases with a slight slope.

Protection against SARS-CoV-2 BA.4 and BA.5 subvariants via vaccination and natural infection: A modeling study

With continuing emergence of new SARS-CoV-2 variants, understanding the proportion of the population protected against infection is crucial for public health risk assessment and decision-making and so that the general public can take preventive measures. We aimed to estimate the protection against symptomatic illness caused by SARS-CoV-2 Omicron variants BA.4 and BA.5 elicited by vaccination against and natural infection with other SARS-CoV-2 Omicron subvariants. We used a logistic model to define the protection rate against symptomatic infection caused by BA.1 and BA.2 as a function of neutralizing antibody titer values. Applying the quantified relationships to BA.4 and BA.5 using two different methods, the estimated protection rate against BA.4 and BA.5 was 11.3% (95% confidence interval [CI]: 0.01-25.4) (method 1) and 12.9% (95% CI: 8.8-18.0) (method 2) at 6 months after a second dose of BNT162b2 vaccine, 44.3% (95% CI: 20.0-59.3) (method 1) and 47.3% (95% CI: 34.1-60.6) (method 2) at 2 weeks after a third BNT162b2 dose, and 52.3% (95% CI: 25.1-69.2) (method 1) and 54.9% (95% CI: 37.6-71.4) (method 2) during the convalescent phase after infection with BA.1 and BA.2, respectively. Our study indicates that the protection rate against BA.4 and BA.5 are significantly lower compared with those against previous variants and may lead to substantial morbidity, and overall estimates were consistent with empirical reports. Our simple yet practical models enable prompt assessment of public health impacts posed by new SARS-CoV-2 variants using small sample-size neutralization titer data to support public health decisions in urgent situations.

Introduction to the special issue on the role of operational research in future epidemics/ pandemics

In this special issue, 23 research papers are published focusing on COVID-19 and operational research solution techniques. First, we detail the process from advertising the call for papers to the point where the best papers are accepted. Then, we provide a summary of each paper focusing on applications, solution techniques and insights for practitioners and policy makers. To provide a holistic view for readers, we have clustered the papers into different groups: transmission, propagation and forecasting, non-pharmaceutical intervention, healthcare network configuration, healthcare resource allocation, hospital operations, vaccine and testing kits, and production and manufacturing. Then, we introduce other possible subjects that can be considered for future research.

Optimal control of the spatial allocation of COVID-19 vaccines: Italy as a case study

While campaigns of vaccination against SARS-CoV-2 are underway across the world, communities face the challenge of a fair and effective distribution of a limited supply of doses. Current vaccine allocation strategies are based on criteria such as age or risk. In the light of strong spatial heterogeneities in disease history and transmission, we explore spatial allocation strategies as a complement to existing approaches. Given the practical constraints and complex epidemiological dynamics, designing effective vaccination strategies at a country scale is an intricate task. We propose a novel optimal control framework to derive the best possible vaccine allocation for given disease transmission projections and constraints on vaccine supply and distribution logistics. As a proof-of-concept, we couple our framework with an existing spatially explicit compartmental COVID-19 model tailored to the Italian geographic and epidemiological context. We optimize the vaccine allocation on scenarios of unfolding disease transmission across the 107 provinces of Italy, from January to April 2021. For each scenario, the optimal solution significantly outperforms alternative strategies that prioritize provinces based on incidence, population distribution, or prevalence of susceptibles. Our results suggest that the complex interplay between the mobility network and the spatial heterogeneities implies highly non-trivial prioritization strategies for effective vaccination campaigns. Our work demonstrates the potential of optimal control for complex and heterogeneous epidemiological landscapes at country, and possibly global, scales.