A Framework for Developing the Structure of Public Health Economic Models

The conceptualisation of cardiometabolic disease policy model in the UK

BACKGROUND

Decision models are increasingly used to inform policy-making processes, and there is a need to improve their credibility. The estimation of health and economic outcomes generated from decision models is influenced by the development process itself. This paper aims to present the conceptual model development process of cardiometabolic disease (CMD) policy models in the UK setting.

METHODS

This conceptual model followed the International Society of Pharmacoeconomics and Outcomes Research-Society of Medical Decision Making (ISPOR-SMDM) Modelling Good Research Practices Task Force-2.

RESULTS

First, for the conceptualisation of the problem, the CMD disease staging, progression and current clinical guidelines were summarised, followed by a systematic review of published policy models. We critically appraised policy models such as cardiovascular disease and type 2 diabetes. Key messages from the review emphasised the importance of understanding various determinants influencing model development, including risk factors, model structure, models' parameters, data utilisation, economic perspective, equality/equity consideration, transparency and validation process. Second, as a sequential process, is model conceptualisation, to determine which modelling types and their attributes best represent the defined problem. Expert opinions, including a clinician and experienced modellers, provided input on the state transition model to ensure the structure is clinically relevant. From this stage, the consideration and agreement to establish a disease state in a state transition model was discussed.

CONCLUSION

This conceptual model serves as a basis for representing the systematic process for structuring a CMD policy model to enhance its transparency and credibility.

A Framework for Assessing the Impact of Outbreak Response Immunization Programs

The impact of outbreak response immunization (ORI) can be estimated by comparing observed outcomes to modelled counterfactual scenarios without ORI, but the most appropriate metrics depend on stakeholder needs and data availability. This study developed a framework for using mathematical models to assess the impact of ORI for vaccine-preventable diseases. Framework development involved (1) the assessment of impact metrics based on stakeholder interviews and literature reviews determining data availability and capacity to capture as model outcomes; (2) mapping investment in ORI elements to model parameters to define scenarios; (3) developing a system for engaging stakeholders and formulating model questions, performing analyses, and interpreting results; and (4) example applications for different settings and pathogens. The metrics identified as most useful were health impacts, economic impacts, and the risk of severe outbreaks. Scenario categories included investment in the response scale, response speed, and vaccine targeting. The framework defines four phases: (1) problem framing and data sourcing (identification of stakeholder needs, metrics, and scenarios); (2) model choice; (3) model implementation; and (4) interpretation and communication. The use of the framework is demonstrated by application to two outbreaks, measles in Papua New Guinea and Ebola in the Democratic Republic of the Congo. The framework is a systematic way to engage with stakeholders and ensure that an analysis is fit for purpose, makes the best use of available data, and uses suitable modelling methodology.

A Guide to an Iterative Approach to Model-Based Decision Making in Health and Medicine: An Iterative Decision-Making Framework

Decision makers frequently face decisions about optimal resource allocation. A model-based economic evaluation can be used to guide decision makers in their choices by systematically evaluating the magnitude of expected health effects and costs of decision options and by making trade-offs explicit. We provide a guide to an iterative approach to the medical decision-making process by following a coherent framework, and outline the overarching iterative steps of model-based decision making. We systematized the framework by performing three steps. First, we compiled the existing guidelines provided by the ISPOR-SMDM Modeling Good Research Practices Task Force, and the ISPOR Value of Information Task Force. Second, we identified other previous work related to frameworks and guidelines for model-based decision analyses through a literature search in PubMed. Third, we assessed the role of the evidence and iterative process in decision making and formalized key steps in a model-based decision-making framework. We provide guidance on an iterative approach to medical decision making by applying the compiled iterative model-based decision-making framework. The framework formally combines the decision problem conceptualization (Part I), the model conceptualization and development (Part II), and the process of model-based decision analysis (Part III). Following the overarching steps of the framework ensures compliance to the principles of evidence-based medicine and regular updates of the evidence, given that value of information analysis represents an essential component of model-based decision analysis in the framework. Following the provided guide and the steps outlined in the framework can help inform various health care decisions, and therefore it has the potential to improve decision making.

Cross-model validation of public health microsimulation models; comparing two models on estimated effects of a weight management intervention

BACKGROUND

Health economic modelling indicates that referral to a behavioural weight management programme is cost saving and generates QALY gains compared with a brief intervention. The aim of this study was to conduct a cross-model validation comparing outcomes from this cost-effectiveness analysis to those of a comparator model, to understand how differences in model structure contribute to outcomes.

METHODS

The outcomes produced by two models, the School for Public Health Research diabetes prevention (SPHR) and Health Checks (HC) models, were compared for three weight-management programme strategies; Weight Watchers (WW) for 12 weeks, WW for 52 weeks, and a brief intervention, and a simulated no intervention scenario. Model inputs were standardised, and iterative adjustments were made to each model to identify drivers of differences in key outcomes.

RESULTS

The total QALYs estimated by the HC model were higher in all treatment groups than those estimated by the SPHR model, and there was a large difference in incremental QALYs between the models. SPHR simulated greater QALY gains for 12-week WW and 52-week WW relative to the Brief Intervention. Comparisons across socioeconomic groups found a stronger socioeconomic gradient in the SPHR model. Removing the impact of treatment on HbA1c from the SPHR model, running both models only with the conditions that the models have in common and, to a lesser extent, changing the data used to estimate risk factor trajectories, resulted in more consistent model outcomes.

CONCLUSIONS

The key driver of difference between the models was the inclusion of extra evidence-based detail in SPHR on the impacts of treatments on HbA1c. The conclusions were less sensitive to the dataset used to inform the risk factor trajectories. These findings strengthen the original cost-effectiveness analyses of the weight management interventions and provide an increased understanding of what is structurally important in the models.

Incorporating frailty to address the key challenges to geriatric economic evaluation

BACKGROUND

The multidimensional and dynamically complex process of ageing presents key challenges to economic evaluation of geriatric interventions, including: (1) accounting for indirect, long-term effects of a geriatric shock such as a fall; (2) incorporating a wide range of societal, non-health outcomes such as informal caregiver burden; and (3) accounting for heterogeneity within the demographic group. Measures of frailty aim to capture the multidimensional and syndromic nature of geriatric health. Using a case study of community-based falls prevention, this article explores how incorporating a multivariate frailty index in a decision model can help address the above key challenges.

METHODS

A conceptual structure of the relationship between geriatric shocks and frailty was developed. This included three key associations involving frailty: (A) the shock-frailty feedback loop; (B) the secondary effects of shock via frailty; and (C) association between frailty and intervention access. A case study of economic modelling of community-based falls prevention for older persons aged 60 + was used to show how parameterising these associations contributed to addressing the above three challenges. The English Longitudinal Study of Ageing (ELSA) was the main data source for parameterisation. A new 52-item multivariate frailty index was generated from ELSA. The main statistical methods were multivariate logistic and linear regressions. Estimated regression coefficients were inputted into a discrete individual simulation with annual cycles to calculate the continuous variable value or probability of binary event given individuals' characteristics.

RESULTS

All three conceptual associations, in their parameterised forms, contributed to addressing challenge (1). Specifically, by worsening the frailty progression, falls incidence in the model increased the risk of falling in subsequent cycles and indirectly impacted the trajectories and levels of EQ-5D-3 L, mortality risk, and comorbidity care costs. Intervention access was positively associated with frailty such that the greater access to falls prevention by frailer individuals dampened the falls-frailty feedback loop. Association (B) concerning the secondary effects of falls via frailty was central to addressing challenge (2). Using this association, the model was able to estimate how falls prevention generated via its impact on frailty paid and unpaid productivity gains, out-of-pocket care expenditure reduction, and informal caregiving cost reduction. For challenge (3), frailty captured the variations within demographic groups of key model outcomes including EQ-5D-3 L, QALY, and all-cause care costs. Frailty itself was shown to have a social gradient such that it mediated socially inequitable distributions of frailty-associated outcomes.

CONCLUSION

The frailty-based conceptual structure and parameterisation methods significantly improved upon the methods previously employed by falls prevention models to address the key challenges for geriatric economic evaluation. The conceptual structure is applicable to other geriatric and non-geriatric intervention areas and should inform the data selection and statistical methods to parameterise structurally valid economic models of geriatric interventions.

The long-term effectiveness and cost-effectiveness of public health interventions; how can we model behavior? A review

The effectiveness and cost of a public health intervention is dependent on complex human behaviors, yet health economic models typically make simplified assumptions about behavior, based on little theory or evidence. This paper reviews existing methods across disciplines for incorporating behavior within simulation models, to explore what methods could be used within health economic models and to highlight areas for further research. This may lead to better-informed model predictions. The most promising methods identified which could be used to improve modeling of the causal pathways of behavior-change interventions include econometric analyses, structural equation models, data mining and agent-based modeling; the latter of which has the advantage of being able to incorporate the non-linear, dynamic influences on behavior, including social and spatial networks. Twenty-two studies were identified which quantify behavioral theories within simulation models. These studies highlight the importance of combining individual decision making and interactions with the environment and demonstrate the importance of social norms in determining behavior. However, there are many theoretical and practical limitations of quantifying behavioral theory. Further research is needed about the use of agent-based models for health economic modeling, and the potential use of behavior maintenance theories and data mining.

Guidance on the use of complex systems models for economic evaluations of public health interventions

To help health economic modelers respond to demands for greater use of complex systems models in public health. To propose identifiable features of such models and support researchers to plan public health modeling projects using these models. A working group of experts in complex systems modeling and economic evaluation was brought together to develop and jointly write guidance for the use of complex systems models for health economic analysis. The content of workshops was informed by a scoping review. A public health complex systems model for economic evaluation is defined as a quantitative, dynamic, non-linear model that incorporates feedback and interactions among model elements, in order to capture emergent outcomes and estimate health, economic and potentially other consequences to inform public policies. The guidance covers: when complex systems modeling is needed; principles for designing a complex systems model; and how to choose an appropriate modeling technique. This paper provides a definition to identify and characterize complex systems models for economic evaluations and proposes guidance on key aspects of the process for health economics analysis. This document will support the development of complex systems models, with impact on public health systems policy and decision making.

Stakeholder Engagement in the Development of Public Health Economic Models: An Application to Modelling of Minimum Unit Pricing of Alcohol in South Africa

BACKGROUND

Health economic models aim to provide decision makers with information that is contextually relevant, understandable and credible. This requires ongoing engagement throughout the research project between the modeller and end-users.

OBJECTIVES

We aim to reflect on how a public health economic model of minimum unit pricing of alcohol in South Africa benefited from, and was shaped by, stakeholders. We outline how engagement activities were used during the development, validation and communication phases of the research with input gathered at each stage to inform future priorities.

METHODS

A stakeholder mapping exercise was completed to identify stakeholders with the required knowledge, for example academics with expertise in modelling alcohol harm in South Africa, members of civil society organisations with lived experience of informal alcohol outlets, and policy professionals working at the forefront of alcohol policy development in South Africa. The stakeholder engagement consisted of four phases: developing a detailed understanding of the local policy context; co-producing model focus and structure; scrutinising model development and communication planning; and communicating research evidence to end-users. The first phase utilised 12 individual semi-structured interviews. Phases two to four centred around face-to-face workshops (two online) with both individual and group-based exercises employed to achieve required outputs.

RESULTS

Phase one provided key learning on policy context and initiated working relationships. Phases two to four provided a conceptualisation of the problem of alcohol harm in South Africa and the choice of policy to model. Stakeholders chose population subgroups of interest and advised on both economic and health outcomes. They provided input on critical assumptions, data sources, priorities for future work, and communication strategies. The final workshop provided a platform to communicate the results of the model to a largely policy audience. These activities led to the production of highly contextualised research methods and findings that were able to be communicated widely beyond academia.

CONCLUSIONS

Our programme of stakeholder engagement was fully integrated into the research programme. It resulted in a number of benefits including creating positive working relationships, guiding modelling decisions, tailoring the research to the context, and providing ongoing opportunities for communication.

Economic model of community-based falls prevention: seeking methodological solutions in evaluating the efficiency and equity of UK guideline recommendations

Background

Falls significantly harm geriatric health and impose substantial costs on care systems and wider society. Decision modelling can inform the commissioning of falls prevention but face methodological challenges, including: (1) capturing non-health outcomes and societal intervention costs; (2) considering heterogeneity and dynamic complexity; (3) considering theories of human behaviour and implementation; and (4) considering issues of equity. This study seeks methodological solutions in developing a credible economic model of community-based falls prevention for older persons (aged 60 +) to inform local falls prevention commissioning as recommended by UK guidelines.

Methods

A framework for conceptualising public health economic models was followed. Conceptualisation was conducted in Sheffield as a representative local health economy. Model parameterisation used publicly available data including English Longitudinal Study of Ageing and UK-based falls prevention trials. Key methodological developments in operationalising a discrete individual simulation model included: (1) incorporating societal outcomes including productivity, informal caregiving cost, and private care expenditure; (2) parameterising dynamic falls-frailty feedback loop whereby falls influence long-term outcomes via frailty progression; (3) incorporating three parallel prevention pathways with unique eligibility and implementation conditions; and (4) assessing equity impacts through distributional cost-effectiveness analysis (DCEA) and individual-level lifetime outcomes (e.g., number reaching ‘fair innings’). Guideline-recommended strategy (RC) was compared against usual care (UC). Probabilistic sensitivity, subgroup, and scenario analyses were conducted.

Results

RC had 93.4% probability of being cost-effective versus UC at cost-effectiveness threshold of £20,000 per QALY gained under 40-year societal cost-utility analysis. It increased productivity and reduced private expenditure and informal caregiving cost, but productivity gain and private expenditure reduction were outstripped by increases in intervention time opportunity costs and co-payments, respectively. RC reduced inequality delineated by socioeconomic status quartile. Gains in individual-level lifetime outcomes were small. Younger geriatric age groups can cross-subsidise their older peers for whom RC is cost-ineffective. Removing the falls-frailty feedback made RC no longer efficient or equitable versus UC.

Conclusion

Methodological advances addressed several key challenges associated with falls prevention modelling. RC appears cost-effective and equitable versus UC. However, further analyses should confirm whether RC is optimal versus other potential strategies and investigate feasibility issues including capacity implications.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-023-03916-z.

Collaborating with Patient Partners to Model Clinical Care Pathways in Major Depressive Disorder: The Benefits of Mixing Evidence and Lived Experience

BACKGROUND

Partnering with patients can enrich the design and development of models of clinical care pathways, yet the practice is not commonplace. Guidelines or "best practices" for patient involvement in modeling are scarce.

OBJECTIVES

In this paper, we outline the steps we took to form an effective partnership with patients to design a robust microsimulation Markov model of major depressive disorder care pathways in British Columbia, Canada, with the aim of encouraging other teams to partner with patients in healthcare modeling endeavors.

METHODS

We describe three unique phases of our collaborative process: uncertainty, mapping, and structured collaboration. We then explore the unique contributions the patient partners made, not only to the model itself, but to our process. Key perspectives are shared from both the modeler and the patient partners in their own words.

RESULTS

The patient partners made distinct contributions by challenging and verifying modeling assumptions, noting limitations of the model, and suggesting areas for future research. Both the patient partners and the modelers saw great value in the partnership and agreed that the model was strengthened by the diversity of the team.

CONCLUSIONS

We present our learning and key recommendations for future modeling teams in the absence of tested frameworks. We encourage more widespread adoption of patient involvement in modeling and the development of guidelines for such work to increase the democracy of scientific decision making.

Systematic review and critical methodological appraisal of community-based falls prevention economic models

BACKGROUND

Falls impose significant health and economic burdens on community-dwelling older persons. Decision modelling can inform commissioning of alternative falls prevention strategies. Several methodological challenges arise when modelling public health interventions including community-based falls prevention. This study aims to conduct a systematic review (SR) to: systematically identify community-based falls prevention economic models; synthesise and critically appraise how the models handled key methodological challenges associated with public health modelling; and suggest areas for further methodological research.

METHODS

The SR followed the 2021 PRISMA reporting guideline and covered the period 2003-2020 and 12 academic databases and grey literature. The extracted methodological features of included models were synthesised by their relevance to the following challenges: (1) capturing non-health outcomes and societal intervention costs; (2) considering heterogeneity and dynamic complexity; (3) considering theories of human behaviour and implementation; and (4) considering equity issues. The critical appraisal assessed the prevalence of each feature across models, then appraised the methods used to incorporate the feature. The methodological strengths and limitations stated by the modellers were used as indicators of desirable modelling practice and scope for improvement, respectively. The methods were also compared against those suggested in the broader empirical and methodological literature. Areas of further methodological research were suggested based on appraisal results.

RESULTS

46 models were identified. Comprehensive incorporation of non-health outcomes and societal intervention costs was infrequent. The assessments of heterogeneity and dynamic complexity were limited; subgroup delineation was confined primarily to demographics and binary disease/physical status. Few models incorporated heterogeneity in intervention implementation level, efficacy and cost. Few dynamic variables other than age and falls history were incorporated to characterise the trajectories of falls risk and general health/frailty. Intervention sustainability was frequently based on assumptions; few models estimated the economic/health returns from improved implementation. Seven models incorporated ethnicity- and severity-based subgroups but did not estimate the equity-efficiency trade-offs. Sixteen methodological research suggestions were made.

CONCLUSION

Existing community-based falls prevention models contain methodological limitations spanning four challenge areas relevant for public health modelling. There is scope for further methodological research to inform the development of falls prevention and other public health models.

LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS (LOCOMOTION): protocol for a mixed-methods study in the UK

INTRODUCTION

Long COVID, a new condition whose origins and natural history are not yet fully established, currently affects 1.5 million people in the UK. Most do not have access to specialist long COVID services. We seek to optimise long COVID care both within and outside specialist clinics, including improving access, reducing inequalities, helping self-management and providing guidance and decision support for primary care. We aim to establish a 'gold standard' of care by systematically analysing current practices, iteratively improving pathways and systems of care.

METHODS AND ANALYSIS

This mixed-methods, multisite study is informed by the principles of applied health services research, quality improvement, co-design, outcome measurement and learning health systems. It was developed in close partnership with patients (whose stated priorities are prompt clinical assessment; evidence-based advice and treatment and help with returning to work and other roles) and with front-line clinicians. Workstreams and tasks to optimise assessment, treatment and monitoring are based in three contrasting settings: workstream 1 (qualitative research, up to 100 participants), specialist management in 10 long COVID clinics across the UK, via a quality improvement collaborative, experience-based co-design and targeted efforts to reduce inequalities of access, return to work and peer support; workstream 2 (quantitative research, up to 5000 participants), patient self-management at home, technology-supported monitoring and validation of condition-specific outcome measures and workstream 3 (quantitative research, up to 5000 participants), generalist management in primary care, harnessing electronic record data to study population phenotypes and develop evidence-based decision support, referral pathways and analysis of costs. Study governance includes an active patient advisory group.

ETHICS AND DISSEMINATION

LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS study is sponsored by the University of Leeds and approved by Yorkshire & The Humber-Bradford Leeds Research Ethics Committee (ref: 21/YH/0276). Participants will provide informed consent. Dissemination plans include academic and lay publications, and partnerships with national and regional policymakers.

TRIAL REGISTRATION NUMBER

NCT05057260, ISRCTN15022307.

Development of a conceptual model for evaluating new non-curative and curative therapies for sickle cell disease

BACKGROUND

Sickle cell disease (SCD) is a clinically heterogeneous disease with many acute and chronic complications driven by ongoing vaso-occlusion and hemolysis. It causes a disproportionate burden on Black and Hispanic communities. Our objective was to follow the SMDM/ISPOR Task Force recommendations for good practices and create a conceptual model of the progression of SCD under current clinical practice to inform cost-effectiveness analyses (CEA) of promising curative therapies in the pipeline over a lifetime horizon.

METHODS

We used consultations with experts, providers, and patients to identify acute events and chronic conditions in the conceptual model. We compared our model structure to previous CEA models of interventions for SCD, assessed the prevalence of the identified disease attributes in Medicaid and Medicare claims databases, and identified relevant outcomes following the 2nd Panel in CEA. We determined an appropriate modeling technique and relevant data sources for parameterizing the model.

RESULTS

The conceptual model structure included four dimensions of disease: chronic pain, acute events, chronic conditions, and treatment complications, spanning 26 disease attributes with significant impacts on health-related quality of life and resource. We modeled chronic pain separately to reflect its importance to patients and interaction with all other disease attributes. We identified additional data sources for health state utilities and non-medical costs and benefits of SCD. We will use a microsimulation model with age- and sex-specific transitions between health states predicted by patient demographic characteristics and disease history.

CONCLUSION

Developing the model structure through an explicit process of model conceptualization can increase the transparency and accuracy of results. We will populate the conceptual model with the data sources described and evaluate the cost-effectiveness of curative therapies.

Economic evaluation of community-based falls prevention interventions for older populations: a systematic methodological overview of systematic reviews

Background

Falls impose significant health and economic burdens on older people. The volume of falls prevention economic evaluations has increased, the findings from which have been synthesised by systematic reviews (SRs). Such SRs can inform commissioning and design of future evaluations; however, their findings can be misleading and incomplete, dependent on their pre-specified criteria. This study aims to conduct a systematic overview (SO) to: (1) systematically identify SRs of community-based falls prevention economic evaluations; (2) describe the methodology and findings of SRs; (3) critically appraise the methodology of SRs; and (4) suggest commissioning recommendations based on SO findings.

Methods

The SO followed the PRISMA guideline and the Cochrane guideline on SO, covering 12 databases and grey literature for the period 2003–2020. Eligible studies were SRs with 50% or more included studies that were economic evaluations of community-based falls prevention (against any comparator) for older persons (aged 60 +) or high-risk individuals aged 50–59. Identified SRs’ aims, search strategies and results, extracted data fields, quality assessment methods/results, and commissioning and research recommendations were synthesised. The comprehensiveness of previous SRs’ data synthesis was judged against criteria drawn from literature on falls prevention/public health economic evaluation. Outcomes of general population, lifetime decision models were re-analysed to inform commissioning recommendations. The SO protocol is registered in the Prospective Register of Systematic Reviews (CRD42021234379).

Results

Seven SRs were identified, which extracted 8 to 33 data fields from 44 economic evaluations. Four economic evaluation methodological/reporting quality checklists were used; three SRs narratively synthesised methodological features to varying extent and focus. SRs generally did not appraise decision modelling features, including methods for characterising dynamic complexity of falls risk and intervention need. Their commissioning recommendations were based mainly on cost-per-unit ratios (e.g., incremental cost-effectiveness ratios) and neglected aggregate impact. There is model-based evidence of multifactorial and environmental interventions, home assessment and modification and Tai Chi being cost-effective but also the risk that they exacerbate social inequities of health.

Conclusions

Current SRs of falls prevention economic evaluations do not holistically inform commissioning and evaluation. Accounting for broader decisional factors and methodological nuances of economic evaluations, particularly decision models, is needed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07764-2.

Economic models of community-based falls prevention: a systematic review with subsequent commissioning and methodological recommendations

BACKGROUND

Falls impose significant health and economic burdens among older populations, making their prevention a priority. Health economic models can inform whether the falls prevention intervention represents a cost-effective use of resources and/or meet additional objectives such as reducing social inequities of health. This study aims to conduct a systematic review (SR) of community-based falls prevention economic models to: (i) systematically identify such models; (ii) synthesise and critically appraise modelling methods/results; and (iii) formulate methodological and commissioning recommendations.

METHODS

The SR followed PRISMA 2021 guideline, covering the period 2003-2020, 12 academic databases and grey literature. A study was included if it: targeted community-dwelling persons aged 60 and over and/or aged 50-59 at high falls risk; evaluated intervention(s) designed to reduce falls or fall-related injuries; against any comparator(s); reported outcomes of economic evaluation; used decision modelling; and had English full text. Extracted data fields were grouped by: (A) model and evaluation overview; (B) falls epidemiology features; (C) falls prevention intervention features; and (D) evaluation methods and outcomes. A checklist for falls prevention economic evaluations was used to assess reporting/methodological quality. Extracted fields were narratively synthesised and critically appraised to inform methodological and commissioning recommendations. The SR protocol is registered in the Prospective Register of Systematic Reviews (CRD42021232147).

RESULTS

Forty-six models were identified. The most prevalent issue according to the checklist was non-incorporation of all-cause care costs. Based on general population, lifetime models conducting cost-utility analyses, seven interventions produced favourable ICERs relative to no intervention under the cost-effectiveness threshold of US$41,900 (£30,000) per QALY gained; of these, results for (1) combined multifactorial and environmental intervention, (2) physical activity promotion for women, and (3) targeted vitamin D supplementation were from validated models. Decision-makers should explore the transferability and reaches of interventions in their local settings. There was some evidence that exercise and home modification exacerbate existing social inequities of health. Sixteen methodological recommendations were formulated.

CONCLUSION

There is significant methodological heterogeneity across falls prevention models. This SR's appraisals of modelling methods should facilitate the conceptualisation of future falls prevention models. Its synthesis of evaluation outcomes, though limited to published evidence, could inform commissioning.

Four Aspects Affecting Health Economic Decision Models and Their Validation

Health care decision makers in many jurisdictions use cost-effectiveness analysis based on health economic decision models for policy decisions regarding coverage and price negotiation for medicines and medical devices. While validation of health economic decision models has always been considered important, many reviews of model-based cost-effectiveness studies report limitations regarding their validation. The current opinion paper discusses four aspects of current health economic decision modeling with relevance for future directions in model validation: increased use of complex models, international cooperation, open-source modeling, and stakeholder involvement. First, new, more complex clinical study designs and treatment strategies may require relatively complex model structures and/or input data analyses. Simultaneously, more widespread technical knowledge along with wider data availability have led to a broader range of model types. This puts extra requirements on model validation and transparency. Second, increased international cooperation of policy makers and, in particular, health technology assessment (HTA) authorities in performing model assessments is discussed in relation to the repeated use of health economic models (multi-use disease models). We argue such coordinated efforts may benefit model validity. Third, open-source modeling is discussed as one possible answer to increased transparency requirements. Finally, involvement of all relevant stakeholders throughout the whole decision process is an ongoing development that necessarily also includes health economic modeling. We argue this implies that model validity should be considered in a broader perspective, with more focus on conceptual modeling, model transparency, accuracy requirements, and choice of relevant model outcomes than previously.

Methodological issues in economic evaluations of disease prevention and health promotion: an overview of systematic and scoping reviews

Background

We aimed to provide a comprehensive overview of methodological challenges in economic evaluations of disease prevention and health promotion (DPHP)-measures.

Methods

We conducted an overview of reviews searching MEDLINE, EMBASE, NHS Economic Evaluation Database, Database of Promoting Health Effectiveness Reviews, Cochrane Database of Systematic Reviews (CDSR) and Database of Promoting Health Effectiveness Reviews (DOPHER) (from their inception to October 2021). We included both systematic and scoping reviews of economic evaluations in DPHP addressing following methodological aspects: (i) attribution of effects, (ii) outcomes, (iii) inter-sectoral (accruing to non-health sectors of society) costs and consequences and (iv) equity. Data were extracted according to the associated sub-criteria of the four methodological aspects including study design economic evaluation (e.g. model-based), type/scope of the outcomes (e.g. outcomes beyond health), perspective, cost categories related to non-health sectors of society, and consideration of equity (method of inclusion). Two reviewers independently screened all citations, full-text articles, and extracted data. A narrative synthesis without a meta-analysis or other statistical synthesis methods was conducted.

Results

The reviewing process resulted in ten systematic and one scoping review summarizing 494 health economic evaluations. A lifelong time horizon was adopted in about 23% of DPHP evaluations, while 64% of trial-based evaluations had a time horizon up to 2 years. Preference-based outcomes (36%) and non-health outcomes (8%) were only applied in a minority of studies. Although the inclusion of inter-sectoral costs (i.e. costs accruing to non-health sectors of society) has increased in recent years, these were often neglected (between 6 and 23% depending on the cost category). Consideration to equity was barely given in economic evaluations, and only addressed in six of the eleven reviews.

Conclusions

Economic evaluations of DPHP measures give only little attention to the specific methodological challenges related to this area. For future economic DPHP evaluations a tool with structured guidance should be developed.

This overview of reviews was not registered and a published protocol does not exist.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-12174-w.

Value judgments in a COVID-19 vaccination model: A case study in the need for public involvement in health-oriented modelling

Scientific modelling is a value-laden process: the decisions involved can seldom be made using 'scientific' criteria alone, but rather draw on social and ethical values. In this paper, we draw on a body of philosophical literature to analyze a COVID-19 vaccination model, presenting a case study of social and ethical value judgments in health-oriented modelling. This case study urges us to make value judgments in health-oriented models explicit and interpretable by non-experts and to invite public involvement in making them.

Qualitative research to inform economic modelling: a case study in older people's views on implementing the NICE falls prevention guideline

Background

High prevalence of falls among older persons makes falls prevention a public health priority. Yet community-based falls prevention face complexity in implementation and any commissioning strategy should be subject to economic evaluation to ensure cost-effective use of healthcare resources. The study aims to capture the views of older people on implementing the National Institute for Health and Care Excellence (NICE) guideline on community-based falls prevention and explore how the qualitative data can be used to inform commissioning strategies and conceptual modelling of falls prevention economic evaluation in the local area of Sheffield.

Methods

Focus group and interview participants (n = 27) were recruited from Sheffield, England, and comprised falls prevention service users and eligible non-users of varying falls risks. Topics concerned key components of the NICE-recommended falls prevention pathway, including falls risk screening, multifactorial risk assessment and treatment uptake and adherence. Views on other topics concerning falls prevention were also invited. Framework analysis was applied for data analysis, involving data familiarisation, identifying themes, indexing, charting and mapping and interpretation. The qualitative data were mapped to three frameworks: (1) facilitators and barriers to implementing the NICE-recommended pathway and contextual factors; (2) intervention-related causal mechanisms for formulating commissioning strategies spanning context, priority setting, need, supply and demand; and (3) methodological and evaluative challenges for public health economic modelling.

Results

Two cross-component factors were identified: health motives of older persons; and professional competence. Participants highlighted the need for intersectoral approaches and prioritising the vulnerable groups. The local commissioning strategy should consider the socioeconomic, linguistic, geographical, legal and cultural contexts, priority setting challenges, supply-side mechanisms spanning provider, organisation, funding and policy (including intersectoral) and health and non-health demand motives. Methodological and evaluative challenges identified included: incorporating non-health outcomes and societal intervention costs; considering dynamic complexity; considering social determinants of health; and conducting equity analyses.

Conclusions

Holistic qualitative research can inform how commissioned falls prevention pathways can be feasible and effective. Qualitative data can inform commissioning strategies and conceptual modelling for economic evaluations of falls prevention and other geriatric interventions. This would improve the structural validity of quantitative models used to inform geriatric public health policies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07056-1.

Developing logic models to inform public health policy outcome evaluation: an example from tobacco control

BACKGROUND

The evaluation of large-scale public health policy interventions often relies on observational designs where attributing causality is challenging. Logic models-visual representations of an intervention's anticipated causal pathway-facilitate the analysis of the most relevant outcomes. We aimed to develop a set of logic models that could be widely used in tobacco policy evaluation.

METHODS

We developed an overarching logic model that reflected the broad categories of outcomes that would be expected following the implementation of tobacco control policies. We subsequently reviewed policy documents to identify the outcomes expected to result from the implementation of each policy and conducted a literature review of existing evaluations to identify further outcomes. The models were revised according to feedbacks from a range of stakeholders.

RESULTS

The final models represented expected causal pathways for each policy. The models included short-term outcomes (such as policy awareness, compliance and social cognitive outcomes), intermediate outcomes (such as changes in smoking behaviour) and long-term outcomes (such as mortality, morbidity and health service usage).

CONCLUSIONS

The use of logic models enables transparent and theory-based planning of evaluation analyses and should be encouraged in the evaluation of tobacco control policy, as well as other areas of public health.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Stakeholder engagement in economic evaluation: Protocol for using the nominal group technique to elicit patient, healthcare provider, and health system stakeholder input in the development of an early economic evaluation model of chimeric antigen receptor T-cell therapy

INTRODUCTION

Chimeric antigen receptor T-cell (CAR-T) therapy is a class of immunotherapy. An economic evaluation conducted at an early stage of development of CAR-T therapy for treatment of adult relapsed or refractory acute lymphoblastic leukaemia could provide insight into factors contributing to the cost of treatment, the potential clinical benefits, and what the health system can afford. Traditionally, stakeholders are engaged in certain parts of health technology assessment processes, such as in the identification and selection of technologies, formulation of recommendations, and implementation of recommendations; however, little is known about processes for stakeholder engagement during the conduct of the assessment. This is especially the case for economic evaluations. Stakeholders, such as clinicians, policy-makers, patients, and their support networks, have insight into factors that can enhance the validity of an economic evaluation model. This research outlines a specific methodology for stakeholder engagement and represents an avenue to enhance health economic evaluations and support the use of these models to inform decision making for resource allocation. This protocol may inform a tailored framework for stakeholder engagement processes in future economic evaluation model development.

METHODS AND ANALYSIS

We will involve clinicians, healthcare researchers, payers, and policy-makers, as well as patients and their support networks in the conduct and verification of an early economic evaluation of a novel health technology to incorporate stakeholder-generated knowledge. Three stakeholder-specific focus groups will be conducted using an online adaptation of the nominal group technique to elicit considerations from each. This study will use CAR-T therapy for adults with relapsed or refractory B-cell acute lymphoblastic leukaemia as a basis for investigating broader stakeholder engagement processes.

ETHICS AND DISSEMINATION

This study received ethics approval from the Ottawa Hospital Research Institute Research Ethics Board (REB 20200320-01HT) and the results will be shared via conference presentations, peer-reviewed publications, and ongoing stakeholder engagement.

Effects of minimum unit pricing for alcohol in South Africa across different drinker groups and wealth quintiles: a modelling study

OBJECTIVES

To quantify the potential impact of minimum unit pricing (MUP) for alcohol on alcohol consumption, spending and health in South Africa. We provide these estimates disaggregated by different drinker groups and wealth quintiles.

DESIGN

We developed an epidemiological policy appraisal model to estimate the effects of MUP across sex, drinker groups (moderate, occasional binge, heavy) and wealth quintiles. Stakeholder interviews and workshops informed model development and ensured policy relevance.

SETTING

South African drinking population aged 15+.

PARTICIPANTS

The population (aged 15+) of South Africa in 2018 stratified by drinking group and wealth quintiles, with a model time horizon of 20 years.

MAIN OUTCOME MEASURES

Change in standard drinks (SDs) (12 g of ethanol) consumed, weekly spend on alcohol, annual number of cases and deaths for five alcohol-related health conditions (HIV, intentional injury, road injury, liver cirrhosis and breast cancer), reported by drinker groups and wealth quintile.

RESULTS

We estimate an MUP of R10 per SD would lead to an immediate reduction in consumption of 4.40% (-0.93 SD/week) and an increase in spend of 18.09%. The absolute reduction is greatest for heavy drinkers (-1.48 SD/week), followed by occasional binge drinkers (-0.41 SD/week) and moderate drinkers (-0.40 SD/week). Over 20 years, we estimate 20 585 fewer deaths and 9 00 332 cases averted across the five health-modelled harms.Poorer drinkers would see greater impacts from the policy (consumption: -7.75% in the poorest quintile, -3.19% in richest quintile). Among the heavy drinkers, 85% of the cases averted and 86% of the lives saved accrue to the bottom three wealth quintiles.

CONCLUSIONS

We estimate that MUP would reduce alcohol consumption in South Africa, improving health outcomes while raising retail and tax revenue. Consumption and harm reductions would be greater in poorer groups.

Jointly Modelling Economics and Epidemiology to Support Public Policy Decisions for the COVID-19 Response: A Review of UK Studies

COVID-19 in the UK has had a profound impact on population health and other socially important outcomes, including on education and the economy. Although a range of evidence has guided policy, epidemiological models have been central. It is less clear whether models to support decision making have sought to integrate COVID-19 epidemiology with a consideration of broader health, wellbeing and economic implications. We report on a rapid review of studies seeking to integrate epidemiological and economic modelling to assess the impacts of alternative policies. Overall, our results suggest that few studies have explored broader impacts of different COVID-19 policies in the UK. Three studies looked only at health, capturing impacts on individuals with and without COVID-19, with various methods used to model the latter. Four models considered health and wider impacts on individuals' economic outcomes, such as wages. However, these models made no attempt to consider the dynamic impacts on economic outcomes of others and the wider economy. The most complex analyses sought to link epidemiological and dynamic economic models. Studies compared a wide range of policies, although most were defined in general terms with minimal consideration of their granular specifications. There was minimal exploration of uncertainty, with no consideration in half the studies. Selecting appropriate models to inform decisions requires careful thought of factors relevant to the decision options under consideration such as the outcomes of interest, sectors likely to be impacted and causal pathways. In summary, better linking epidemiological and economic modelling would help to inform COVID-19 policy.

Putting Stakeholder Engagement at the Center of Health Economic Modeling for Health Technology Assessment in the United States

While evidence generated from health economic (HE) models is being used more commonly in health technology assessment (HTA) in the US, it is not consistently adopted by different stakeholder groups or across therapeutic areas. We hypothesize that actively engaging with multiple stakeholder groups throughout the model development process may result in models more widely considered by decision makers. To test this hypothesis, the Innovation and Value Initiative has launched a modeling effort to build an open-source HE model focusing on the disease state 'major depressive disorder'. A 20-member advisory group has been formed with representatives from patients, employers, clinicians, innovators, payers, and researchers to guide the model development process. While this effort is still in the early stages, the ongoing stakeholder engagement effort has yielded valuable insights that inform the model design. We have also identified several challenges to implementing this new approach. Our early findings suggest that the stakeholder engagement approach to HE model development has the potential to improve HTA in the US.

Developing a disability inclusive model for low vision service

The current practice for low vision management in India exclusively focuses on clinical aspects without much of the rehabilitation components. While making all efforts to improve independent living skills, daily living activities, and quality of life as a whole for people living visual disabilities, vision rehabilitation is an indispensable component. There is no single appropriate low vision and rehabilitation model implementable at health care institutions in the country to cover these fundamental aspects of a visually impaired individual. We did a literature review to know the existing practices of low vision and various disability models. The purpose of the review is to discern any pitfalls and shortcomings in managing visually disabled in India and to underpin the credibility and feasibility as well as suitability of the developed model. The review was done using search key terms low vision, current practices, visual disability, disability models, vision rehabilitation, and service delivery. Therefore, the article discusses the development of an inclusive low vision management model name as “Clinico-Social Model”, which we consider the most appropriate for the best management of people with vision loss. The primary aim of this model is to provide both clinical and vision rehabilitation components of management for people with visual disabilities. Such an approach is likely to have the potential to improve the quality of life of people with vision loss and can provide practical guide to eye care managers across India. Given the specific context in the current practices of low vision in India, it is desirable to design a similar model to care for the visually disabled.

Evaluations of public health interventions produced by health technology assessment agencies: A mapping review and analysis by type and evidence content

OBJECTIVES

Health technology assessments (HTAs) have been suggested as a strategy to bridge the evidence-to-policy gap in public health. It is unclear to what extent HTAs have been prepared to assist decisions to implement public health interventions (PHIs). We aimed to describe the experience of HTA agencies by mapping, classifying, and analyzing the evidence content of HTAs of PHIs.

METHODS

We systematically searched databases of 35 HTA agencies from 18 countries for evaluations of PHIs between 2008-2018. Interventions were classified using the International Classification of Health Interventions and the evidence content analysed with the INAHTA Product-Type-mark checklist.

RESULTS

Only 1010 (9%) of HTAs were on PHIs. 500 (50%) publications targeted Body Systems and Functions, 302 (30%) Health-related Behaviours, 137 (14%) the Environment and 44 (4%) Activities and Participation Domains. Out of 734 publications perused, few met the criteria of full-HTAs (71;10%) or mini-HTAs (110;15%). Most were rapid reviews (420;57%). 72% of all reports came from only 6 countries.

CONCLUSION

HTAs on PHIs were uncommon relative to clinical interventions. HTAs on population-based PHIs were less comprehensive in quality and rigor of the evidence. Countries with more resources and mature HTA-systems had done the most evaluations. Exploring the experiences of forerunners could help overcome barriers to evaluations of PHIs and exploit the full potential of HTAs to promote evidence-based public health.

Conceptualising changes to tobacco and alcohol policy as affecting a single interlinked system

BACKGROUND

To support a move towards a coordinated non-communicable disease approach in public health policy, it is important to conceptualise changes to policy on tobacco and alcohol as affecting a single interlinked system. For health economic models to effectively inform policy, the first step in their development should be to develop a conceptual understanding of the system complexity that is likely to affect the outcomes of policy change. Our aim in this study was to support the development and interpretation of health economic models of the effects of changes to tobacco and alcohol policies by developing a conceptual understanding of the main components and mechanisms in the system that links policy change to outcomes.

METHODS

Our study was based on a workshop from which we captured data on participant discussions on the joint tobacco-alcohol policy system. To inform these discussions, we prepared with a literature review and a survey of participants. Participants were academics and policy professionals who work in the United Kingdom. Data were analysed thematically to produce a description of the main components and mechanisms within the system.

RESULTS

Of the people invited, 24 completed the survey (18 academic, 6 policy); 21 attended the workshop (16 academic, 5 policy). Our analysis identified eleven mechanisms through which individuals might modify the effects of a policy change, which include mechanisms that might lead to linked effects of policy change on tobacco and alcohol consumption. We identified ten mechanisms by which the tobacco and alcohol industries might modify the effects of policy changes, grouped into two categories: Reducing policy effectiveness; Enacting counter-measures. Finally, we identified eighteen research questions that indicate potential avenues for further work to understand the potential outcomes of policy change.

CONCLUSIONS

Model development should carefully consider the ways in which individuals and the tobacco and alcohol industries might modify the effects of policy change, and the extent to which this results in an unequal societal distribution of outcomes. Modelled evidence should then be interpreted in the light of the conceptual understanding of the system that the modelling necessarily simplifies in order to predict the outcomes of policy change.

Modelling in economic evaluations of medicines

Economic evaluation in health (also known as pharmacoeconomic in case of medicines) identifies, measures, and values costs and outcomes of alternative healthcare technologies, and can be performed alongside controlled clinical trials, but analytical modelling is usually used. Decision trees and Markov models are the two most common approaches used in economic evaluation. The biggest advantages of a decision tree are clarity, simplicity, and straightforwardness. On the other hand, the main advantage of the Markov model is its ability to incorporate complex events into the simulation, which is practically impossible to do with a decision tree. Reimbursement policy in Serbia mandatorily incorporates economic evaluations to promote availability and accessibility of the prescription medicines. To show current pharmacoeconomic value of a medicine, budget impact analysis and the cost-effectiveness analysis should be included. The latter should be conducted using appropriate modelling techniques. However, since no official methodological guidelines about the modelling and economic analysis exist, the submissions by marketing authorization holders vary greatly. The future of pharmacoeconomic modelling depends on the research area of interest, with new frameworks and approaches being developed.

Modeling Obesity in Norway (The MOON Study): A Decision-Analytic Approach-Prevalence, Costs, and Years of Life Lost

BACKGROUND

Limited knowledge exists on the expected long-term effects and cost-effectiveness of initiatives aiming to reduce the burden of obesity.

AIM

To develop a Norwegian obesity-focused disease-simulation model: the MOON model.

MATERIAL AND METHODS

We developed a Markov model and simulated a Norwegian birth cohort's movement between the health states "normal weight,""overweight,""obese 1,""obese 2," and "dead" using a lifetime perspective. Model input was estimated using longitudinal data from health surveys and real-world data (RWD) from local and national registers (N = 99,348). The model is deterministic and probabilistic and stratified by gender. Model validity was assessed by estimating the cohort's expected prevalence, health care costs, and mortality related to overweight and obesity.

RESULTS

Throughout the cohort's life, the prevalence of overweight increased steadily and stabilized at 45% at 45 y of age. The number of obese 1 and 2 individuals peaked at age 75 y, when 44% of women and 35% of men were obese. The incremental costs per person associated with obesity was highest in older ages and, when accumulated over the lifetime, higher among women (€12,118, €9,495-€15,047) than men (€6,646, €5,252-€10,900). On average, obesity shortened the life expectancy of women/men in the whole cohort by 1.31/1.08 y. The life expectancy for normal-weight women/men at age 30 was 83.31/80.31. The life expectancy was reduced by 1.05/0.65 y if the individual was overweight, obese (2.87/2.71 y), or obese 2 (4.06/4.83 y).

CONCLUSION

The high expected prevalence of obesity in the future will lead to substantial health care costs and large losses in life-years. This underscores the need to implement interventions to reduce the burden of obesity; the MOON model will enable economic evaluations for a wide range of interventions.

Quantitative Evidence Synthesis Methods for the Assessment of the Effectiveness of Treatment Sequences for Clinical and Economic Decision Making: A Review and Taxonomy of Simplifying Assumptions

Sequential use of alternative treatments for chronic conditions represents a complex intervention pathway; previous treatment and patient characteristics affect both the choice and effectiveness of subsequent treatments. This paper critically explores the methods for quantitative evidence synthesis of the effectiveness of sequential treatment options within a health technology assessment (HTA) or similar process. It covers methods for developing summary estimates of clinical effectiveness or the clinical inputs for the cost-effectiveness assessment and can encompass any disease condition. A comprehensive review of current approaches is presented, which considers meta-analytic methods for assessing the clinical effectiveness of treatment sequences and decision-analytic modelling approaches used to evaluate the effectiveness of treatment sequences. Estimating the effectiveness of a sequence of treatments is not straightforward or trivial and is severely hampered by the limitations of the evidence base. Randomised controlled trials (RCTs) of sequences were often absent or very limited. In the absence of sufficient RCTs of whole sequences, there is no single best way to evaluate treatment sequences; however, some approaches could be re-used or adapted, sharing ideas across different disease conditions. Each has advantages and disadvantages, and is influenced by the evidence available, extent of treatment sequences (number of treatment lines or permutations), and complexity of the decision problem. Due to the scarcity of data, modelling studies applied simplifying assumptions to data on discrete treatments. A taxonomy for all possible assumptions was developed, providing a unique resource to aid the critique of existing decision-analytic models.

The Impact of Including Costs and Outcomes of Dementia in a Health Economic Model to Evaluate Lifestyle Interventions to Prevent Diabetes and Cardiovascular Disease

Objectives

Economic evaluations of lifestyle interventions, which aim to prevent diabetes/cardiovascular disease (CVD), have not included dementia. Lifestyle interventions decrease dementia risk and extend life expectancy, leading to competing effects on health care costs. We aim to demonstrate the feasibility of including dementia in a public health cost-effectiveness analysis and quantify the overall impacts accounting for these competing effects.

Methods

The School for Public Health Research (SPHR) diabetes prevention model describes individuals’ risk of type 2 diabetes, microvascular outcomes, CVD, congestive heart failure, cancer, osteoarthritis, depression, and mortality in England. In version 3.1, we adapted the model to include dementia using published data from primary care databases, health surveys, and trials of dementia to describe dementia incidence, diagnosis, and disease progression. We estimate the impact of dementia on lifetime costs and quality-adjusted life years (QALYs) gained of the National Health Service diabetes prevention program (NHS DPP) from an NHS/personal social services perspective with 3 scenarios: 1) no dementia, 2) dementia only, and 3) reduced dementia risk. Subgroup, parameter, and probabilistic sensitivity analyses were conducted.

Results

The lifetime cost savings of the NHS DPP per patient were £145 in the no-dementia scenario, £121 in the dementia-only scenario, and £167 in the reduced dementia risk scenario. The QALY gains increased by 0.0006 in dementia only and 0.0134 in reduced dementia risk. Dementia did not alter the recommendation that the NHS/DPP is cost-effective.

Conclusions

Including dementia into a model of lifestyle interventions was feasible but did not change policy recommendations or modify health economic outcomes. The impact on health economic outcomes was largest where a direct impact on dementia incidence was assumed, particularly in elderly populations.

How methodological frameworks are being developed: evidence from a scoping review

Background

Although the benefits of using methodological frameworks are increasingly recognised, to date, there is no formal definition of what constitutes a ‘methodological framework’, nor is there any published guidance on how to develop one. For the purposes of this study we have defined a methodological framework as a structured guide to completing a process or procedure. This study’s aims are to: (a) map the existing landscape on the use of methodological frameworks; (b) identify approaches used for the development of methodological frameworks and terminology used; and (c) provide suggestions for developing future methodological frameworks. We took a broad view and did not limit our study to methodological frameworks in research and academia.

Methods

A scoping review was conducted, drawing on Arksey and O’Malley’s methods and more recent guidance. We systematically searched two major electronic databases (MEDLINE and Web of Science), as well as grey literature sources and the reference lists and citations of all relevant papers. Study characteristics and approaches used for development of methodological frameworks were extracted from included studies. Descriptive analysis was conducted.

Results

We included a total of 30 studies, representing a wide range of subject areas. The most commonly reported approach for developing a methodological framework was ‘Based on existing methods and guidelines’ (66.7%), followed by ‘Refined and validated’ (33.3%), ‘Experience and expertise’ (30.0%), ‘Literature review’ (26.7%), ‘Data synthesis and amalgamation’ (23.3%), ‘Data extraction’ (10.0%), ‘Iteratively developed’ (6.7%) and ‘Lab work results’ (3.3%). There was no consistent use of terminology; diverse terms for methodological framework were used across and, interchangeably, within studies.

Conclusions

Although no formal guidance exists on how to develop a methodological framework, this scoping review found an overall consensus in approaches used, which can be broadly divided into three phases: (a) identifying data to inform the methodological framework; (b) developing the methodological framework; and (c) validating, testing and refining the methodological framework. Based on these phases, we provide suggestions to facilitate the development of future methodological frameworks.

Comparative analyses of published cost effectiveness models highlight critical considerations which are useful to inform development of new models

Background: Comparative analyses of published cost effectiveness models provide useful insights into critical issues to inform the development of new cost effectiveness models in the same disease area.Objective: The purpose of this study was to describe a comparative analysis of cost-effectiveness models and highlight the importance of such work in informing development of new models. This research uses genotypic antiretroviral resistance testing after first line treatment failure for Human Immunodeficiency Virus (HIV) as an example.Method: A literature search was performed, and published cost effectiveness models were selected according to predetermined eligibility criteria. A comprehensive comparative analysis was undertaken for all aspects of the models.Results: Five published models were compared, and several critical issues were identified for consideration when developing a new model. These include the comparator, time horizon and scope of the model. In addition, the composite effect of drug resistance prevalence, antiretroviral therapy efficacy, test performance and the proportion of patients switching to second-line ART potentially have a measurable effect on model results. When considering CD4 count and viral load, dichotomizing patients according to higher cost and lower quality of life (AIDS) versus lower cost and higher quality of life (non-AIDS) status will potentially capture differences between resistance testing and other strategies, which could be confirmed by cross-validation/convergent validation. A quality adjusted life year is an essential outcome which should be explicitly explored in probabilistic sensitivity analysis, where possible.Conclusions: Using an example of GART for HIV, this study demonstrates comparative analysis of previously published cost effectiveness models yields critical information which can be used to inform the structure and specifications of new models.

Cost-effectiveness of intensive interventions compared to standard care in individuals with type 2 diabetes: A systematic review and critical appraisal of decision-analytic models

AIMS

The objective of this systematic review is to identify and assess the quality of published decision-analytic models evaluating the long-term cost-effectiveness of target-driven intensive interventions for single and multifactorial risk factor control compared to standard care in people with type 2 diabetes.

METHODS

We searched the electronic databases MEDLINE, the National Health Service Economic Evaluation Database, Web of Science and the Cochrane Library from inception to October 31, 2019. Articles were eligible for inclusion if the studies had used a decision-analytic model evaluating both the long-term costs and benefits associated with intensive interventions for risk factor control compared to standard care in people with type 2 diabetes. Data were extracted using a standardised form, while quality was assessed using the decision-analytic model-specific Philips-criteria.

RESULTS

Overall, nine articles (11 models) were identified, four models evaluated intensive glycaemic control, three evaluated intensive blood pressure control, two evaluated intensive lipid control, and two evaluated intensive multifactorial interventions. Six reported using discrete-time simulations modelling approach, whereas five reported using a Markov modelling framework. The majority, seven studies, reported that the intensive interventions were dominant or cost-effective, given the assumptions and analytical perspective taken. The methodological and reporting quality of the studies was generally weak, with only four studies fulfilling more than 50% of their applicable Philips-criteria.

CONCLUSIONS

This is the first systematic review of decision-analytic models of target-driven intensive interventions for single and multifactorial risk factor control in individuals with type 2 diabetes. Identified shortcomings are lack of transparency in data identification and evidence synthesis as well as for the selection of the modelling approaches. Future models should aim to include greater evaluation of the quality of the data sources used and the assessment of uncertainty in the model.

Scientific Development of HTA—A Proposal by the Health Technology Assessment International Scientific Development and Capacity Building Committee

Objectives

To report from the Scientific Development and Capacity Building Committee of Health Technology Assessment International (HTAi) on activities that are being undertaken within HTAi regarding the promotion of scientific rigor in the field of health technology assessment (HTA).

Methods

Retrieval of definitions of HTA that the SDCB committee considered reflective of the current practice of HTA, followed by a narrative synthesis of the core components of HTA.

Results

Several definitions of HTA have been provided, all sharing the notion that HTA is the formal, systematic, and transparent inquiry into the meaning and value, broadly defined, of health technologies, when used in specific patient populations.

Many frameworks and tools have been developed for assessing the quality of specific tasks that may be conducted in the context of HTA. Collating such frameworks and tools is likely to be helpful in developing standards and in providing guidance as to how the scientific quality of HTA may be secured. Two current trends in HTA were noted: a stronger health systems focus, and the need to involve stakeholders throughout the HTA process. A wider systems’ perspective requires that plausible alternative scenarios are being developed, and wide consultation of various stakeholders is a prerequisite to the development of such scenarios with data from various sources.

Conclusions

Current trends in HTA will lead to different demands on the HTA expert. The task of this emerging policy professional would be not just to provide technical information for problem-solving, but also to combine it with a new function of facilitating public deliberation and learning.

PRIMEtime CE: a multistate life table model for estimating the cost-effectiveness of interventions affecting diet and physical activity

BACKGROUND

Non-communicable diseases are the leading cause of death in England, and poor diet and physical inactivity are two of the principle behavioural risk factors. In the context of increasingly constrained financial resources, decision makers in England need to be able to compare the potential costs and health outcomes of different public health policies aimed at improving these risk factors in order to know where to invest so that they can maximise population health. This paper describes PRIMEtime CE, a multistate life table cost-effectiveness model that can directly compare interventions affecting multiple disease outcomes.

METHODS

The multistate life table model, PRIMEtime Cost Effectiveness (PRIMEtime CE), is developed from the Preventable Risk Integrated ModEl (PRIME) and the PRIMEtime model. PRIMEtime CE uses routinely available data to estimate how changing diet and physical activity in England affects morbidity and mortality from heart disease, stroke, diabetes, liver disease, and cancers either directly or via raised blood pressure, cholesterol, and body weight.

RESULTS

Model outcomes are change in quality adjusted life years, and change in English National Health Service and social care costs.

CONCLUSION

This paper describes PRIMEtime CE and highlights its main strengths and limitations. The model can be used to compare any number of public policies affecting diet and physical activity, allowing decision makers to understand how they can maximise population health with limited financial resources.

Systematic review of the methods used in economic evaluations of targeted physical activity and sedentary behaviour interventions

The burden of noncommunicable diseases (NCD) on health systems worldwide is substantial. Physical inactivity and sedentary behaviour are major risk factors for NCD. Previous attempts to understand the value for money of preventative interventions targeting physically inactive individuals have proved to be challenging due to key methodological challenges associated with the conduct of economic evaluations in public health. A systematic review was carried out across six databases (Medline, SPORTSDiscus, EconLit, PsychINFO, NHS EED, HTA) along with supplementary searches. The review examines how economic evaluations published between 2009-March 2017 have addressed methodological challenges with the aim of bringing to light examples of good practice for future studies. Fifteen economic evaluations from four high-income countries were retrieved; there is a dearth of studies targeting sedentary behaviour as an independent risk factor from physical activity. Comparability of studies from the healthcare and societal perspectives were limited due to analysts' choice in cost categories, valuation technique and time horizon differing substantially. The scarcity of and inconsistencies across economic evaluations for these two behaviours have exposed a mismatch between calls for more preventative action to tackle NCD and the lack of information available on how resources may be optimally allocated in practice. Consequently, this paper offers a table of recommendations on how future studies can be improved.

A framework for conducting economic evaluations alongside natural experiments

Internationally, policy makers are increasingly focussed on reducing the detrimental consequences and rising costs associated with unhealthy diets, inactivity, smoking, alcohol and other risk factors on the health of their populations. This has led to an increase in the demand for evidence-based, cost-effective Population Health Interventions (PHIs) to reverse this trend. Given that research designs such as randomised controlled trials (RCTs) are often not suited to the evaluation of PHIs, Natural Experiments (NEs) are now frequently being used as a design to evaluate such complex, preventive PHIs. However, current guidance for economic evaluation focusses on RCT designs and therefore does not address the specific challenges of NE designs. Using such guidance can lead to sub-optimal design, data collection and analysis for NEs, leading to bias in the estimated effectiveness and cost-effectiveness of the PHI. As a consequence, there is a growing recognition of the need to identify a robust methodological framework for the design and conducting of economic evaluations alongside such NEs. This paper outlines the challenges inherent to the design and conduct of economic evaluations of PHIs alongside NEs, providing a comprehensive framework and outlining a research agenda in this area.

Understanding and Identifying Key Issues with the Involvement of Clinicians in the Development of Decision-Analytic Model Structures: A Qualitative Study

INTRODUCTION

Decision-analytic models play an essential role in informing healthcare resource allocation decisions; however, their value to decision makers will depend on model structures being clinically valid to determine cost-effectiveness recommendations. Clinician involvement can help modellers to develop clinically valid but straightforward structures; however, there is little guidance available on methods for clinician input to model structure. This study aims to provide an in-depth exploration of clinician involvement in structural development, highlighting key issues and generating recommendations to optimise practices.

METHODS

A qualitative study was undertaken with a range of modellers and clinicians working in different modelling contexts. In-depth interviews and case studies using observations were carried out to understand how clinicians are involved in model structural development and to identify problems and optimal approaches from informants' perspectives.

RESULTS

Twenty-four interviews and two case studies were undertaken with modellers and modelling teams. Key issues included the number and diversity of clinicians contributing to structural development, potentially impacting the generalisability of structures, and problems with clinician understanding of important information to contribute to model pathways. Modellers and clinicians suggested that clinician training in modelling could enhance structural processes.

CONCLUSIONS

Recommendations to optimise current practices include recruiting clinicians from a variety of backgrounds and using discussions between experts to develop valid and generalisable structures. Future research should focus on developing training materials for clinicians and finding ways to help modellers recruit clinicians from different settings.

Integrating Qualitative Techniques in Model Development: A Case Study Using the Framework Approach

BACKGROUND

Despite their potential, there is limited uptake of formal qualitative methods in model development by modellers and health economists. The aim of this case study was to highlight in a real-world context how a qualitative approach has been applied to gain insight into current practice (delineating existing care pathways) for typhoid fever in Ghana, which can then assist in model structure conceptualisation in a model-based cost-effectiveness analysis.

METHODS

The perspectives of a range of healthcare professionals working in different settings and across different practices in the Eastern region of Ghana were captured with a self-administered survey using open-ended questions and analysed using the framework method.

RESULTS

A total of 51 completed questionnaires were retrieved representing a 73% response rate. It was found that two main care pathways for typhoid fever exist in Ghana and there was no consensus on how a new test might be applied to the existing pathways.

CONCLUSION

The two settings in Ghana have different care pathways and any cost-effectiveness analysis should consider the alternative pathways separately. This study demonstrated that framework analysis is a qualitative methodology that is likely to be accessible and feasible across a wide range of health economic settings.

Primary care provider approaches to preventive health delivery: a qualitative study

AimThe objective of this study was to seek decision-making insights on the provider level to gain understanding of the values that shape how providers deliver preventive health in the primary care setting.

BACKGROUND

The primary care clinic is a core site for preventive health delivery. While many studies have identified barriers to preventive health, less is known regarding how primary care providers (PCPs) make preventive health decisions such as what services to provide, under what circumstances, and why they might choose one over another.

METHODS

Qualitative methods were chosen to deeply explore these issues. We conducted semi-structured, one-on-one interviews with 21 PCPs at clinics affiliated with an academic medical center. Interviews with providers were recorded and transcribed. We conducted a qualitative analysis to identify themes and develop a theoretical framework using Grounded Theory methods.FindingsThe following themes were revealed: longitudinal care with an established PCP-patient relationship is perceived as integral to preventive health; conflict and doubt accompany non-preventive visits; PCPs defer preventive health for pragmatic reasons; when preventive health is addressed, providers use multiple contextual factors to decide which interventions are discussed; and PCPs desired team-based preventive health delivery, but wish to maintain their role when shared decision-making is required. We present a conceptual framework called Pragmatic Deferral.

COMPREHENSIVE ASSESSMENT OF COMPLEX TECHNOLOGIES: INTEGRATING VARIOUS ASPECTS IN HEALTH TECHNOLOGY ASSESSMENT

Objectives: Despite recent development of health technology assessment (HTA) methods, there are still methodological gaps for the assessment of complex health technologies. The INTEGRATE-HTA guidance for effectiveness, economic, ethical, socio-cultural, and legal aspects, deals with challenges when assessing complex technologies, such as heterogeneous study designs, multiple stakeholder perspectives, and unpredictable outcomes. The objective of this article is to outline this guidance and describe the added value of integrating these assessment aspects.

Methods: Different methods were used to develop the various parts of the guidance, but all draw on existing, published knowledge and were supported by stakeholder involvement. The guidance was modified after application in a case study and in response to feedback from internal and external reviewers.

Results: The guidance consists of five parts, addressing five core aspects of HTA, all presenting stepwise approaches based on the assessment of complexity, context, and stakeholder involvement. The guidance on effectiveness, health economics and ethics aspects focus on helping users choose appropriate, or further develop, existing methods. The recommendations are based on existing methods’ applicability for dealing with problems arising with complex interventions. The guidance offers new frameworks to identify socio-cultural and legal issues, along with overviews of relevant methods and sources.

Conclusions: The INTEGRATE-HTA guidance outlines a wide range of methods and facilitates appropriate choices among them. The guidance enables understanding of how complexity matters for HTA and brings together assessments from disciplines, such as epidemiology, economics, ethics, law, and social theory. This indicates relevance for a broad range of technologies.

How Qualitative Methods Can be Used to Inform Model Development

Decision-analytic models play a key role in informing healthcare resource allocation decisions. However, there are ongoing concerns with the credibility of models. Modelling methods guidance can encourage good practice within model development, but its value is dependent on its ability to address the areas that modellers find most challenging. Further, it is important that modelling methods and related guidance are continually updated in light of any new approaches that could potentially enhance model credibility. The objective of this article was to highlight the ways in which qualitative methods have been used and recommended to inform decision-analytic model development and enhance modelling practices. With reference to the literature, the article discusses two key ways in which qualitative methods can be, and have been, applied. The first approach involves using qualitative methods to understand and inform general and future processes of model development, and the second, using qualitative techniques to directly inform the development of individual models. The literature suggests that qualitative methods can improve the validity and credibility of modelling processes by providing a means to understand existing modelling approaches that identifies where problems are occurring and further guidance is needed. It can also be applied within model development to facilitate the input of experts to structural development. We recommend that current and future model development would benefit from the greater integration of qualitative methods, specifically by studying 'real' modelling processes, and by developing recommendations around how qualitative methods can be adopted within everyday modelling practice.

Do Economic Evaluations in Primary Care Prevention and the Management of Hypertension Conform to Good Practice Guidelines? A Systematic Review

Background: Results of previous research have identified the need for further investigation into the compliance with good practice guidelines for current decision-analytic modeling (DAM). Objective: To identify the extent to which recent model-based economic evaluations of interventions focused on lowering the blood pressure (BP) of patients with hypertension conform to published guidelines for DAM in health care using a five-dimension framework developed to assess compliance to DAM guidelines. Methods: A systematic review of English language articles was undertaken to identify published model-based economic evaluations that examined interventions aimed at lowering BP. The review covered the period January 2000 to March 2015 and included the following electronic bibliographic databases: EMBASE and Medline via Ovid interface and the Centre for Reviews and Dissemination’s (CRD) NHS-EED. Data were extracted based on different components of good practice across five dimensions utilizing a framework to assess compliance to DAM guidelines. Results: Thirteen articles were included in this review. The review found limited compliance to good practice DAM guidelines, which was most frequently justified by the lack of data. Conclusions: The assessment of structural uncertainty cannot yet be considered common practice in primary prevention and management of hypertension, and researchers seem to face difficulties with identifying sources of structural uncertainty and then handling them correctly. Additional guidelines are needed to aid researchers in identifying and managing sources of potential structural uncertainty. Adherence to guidelines is not always possible and it does pose challenges, in particular when there are limitations due to data availability that restrict, for example, a validation process.