GetReal in mathematical modelling: a review of studies predicting drug effectiveness in the real world

Testing the "RCT augmentation" methodology: A trial simulation study to guide the broadening of trials eligibility criteria and inform on effectiveness

Background

Exclusion criteria that are treatment effect modifiers (TEM) decrease RCTs results generalisability and the potentials of effectiveness estimation. In "augmented RCTs", a small proportion of otherwise-excluded patients are included to allow for effectiveness estimation. In Hodgkin Lymphoma (HL) RCTs, older age and comorbidity are common exclusion criteria, while also TEM. We simulated HL RCTs augmented with age or comorbidity, and explored in each scenario the impact of augmentation on effectiveness estimation accuracy.

Methods

Simulated data with a population of HL individuals initiating drug A or B was generated. There were drug-age and drug-comorbidity interactions in the simulated data, with a greater magnitude of the former compared to the latter. Multiple augmented RCTs were simulated by randomly selecting patients with increasing proportions of older, or comorbid patients. Treatment effect size was expressed using the between-group Restricted Mean Survival Time (RMST) difference at 3 years. For each augmentation proportion, a model estimating the "real-world" treatment effect (effectiveness) was fitted and the estimation error measured (Root Mean Square Error, RMSE).

Results

In simulated RCTs including none (0%), or the real-world proportion (30%) of older patients, the interquartile range of RMST difference was 0.4-0.5 years and 0.2-0.3 years, respectively, and RMSE were 0.198 years (highest possible error) and 0.056 years (lowest), respectively. Augmenting RCTs with 5% older patients decreased estimation error substantially (RMSE = 0.076 years). Augmentation with comorbid patients proved less useful for effectiveness estimation.

Conclusion

In augmented RCTs aiming to inform the effectiveness of drugs, augmentation should concern in priority those exclusion criteria of suspected important TEM magnitude, so as to minimie the proportion of augmentation necessary for good effectiveness estimations.

Thoroughly Calibrated Modular Agent-Based Model of the Human Cardiovascular and Renal Systems for Blood Pressure Regulation in Health and Disease

Here we present a modular agent-based mathematical model of the human cardiovascular and renal systems. It integrates the previous models primarily developed by A. C. Guyton, F. Karaaslan, K. M. Hallow, and Y. V. Solodyannikov. We performed the model calibration to find an equilibrium state within the normal vital sign ranges for a healthy adult. We verified the model's abilities to reproduce equilibrium states with abnormal physiological values related to different combinations of cardiovascular diseases (such as systemic hypertension, chronic heart failure, pulmonary hypertension, etc.). For the model creation and validation, we involved over 200 scientific studies covering known models of the human cardiovascular and renal functions, biosimulation platforms, and clinical measurements of physiological quantities in normal and pathological conditions. We compiled detailed documentation describing all equations, parameters and variables of the model with justification of all formulas and values. The model is implemented in BioUML and available in the web-version of the software.

Quantitative Evidence Synthesis Methods for the Assessment of the Effectiveness of Treatment Sequences for Clinical and Economic Decision Making: A Review and Taxonomy of Simplifying Assumptions

Sequential use of alternative treatments for chronic conditions represents a complex intervention pathway; previous treatment and patient characteristics affect both the choice and effectiveness of subsequent treatments. This paper critically explores the methods for quantitative evidence synthesis of the effectiveness of sequential treatment options within a health technology assessment (HTA) or similar process. It covers methods for developing summary estimates of clinical effectiveness or the clinical inputs for the cost-effectiveness assessment and can encompass any disease condition. A comprehensive review of current approaches is presented, which considers meta-analytic methods for assessing the clinical effectiveness of treatment sequences and decision-analytic modelling approaches used to evaluate the effectiveness of treatment sequences. Estimating the effectiveness of a sequence of treatments is not straightforward or trivial and is severely hampered by the limitations of the evidence base. Randomised controlled trials (RCTs) of sequences were often absent or very limited. In the absence of sufficient RCTs of whole sequences, there is no single best way to evaluate treatment sequences; however, some approaches could be re-used or adapted, sharing ideas across different disease conditions. Each has advantages and disadvantages, and is influenced by the evidence available, extent of treatment sequences (number of treatment lines or permutations), and complexity of the decision problem. Due to the scarcity of data, modelling studies applied simplifying assumptions to data on discrete treatments. A taxonomy for all possible assumptions was developed, providing a unique resource to aid the critique of existing decision-analytic models.

Ten Epidemiological Parameters of COVID-19: Use of Rapid Literature Review to Inform Predictive Models During the Pandemic

Objective: To describe the methods used in a rapid review of the literature and to present the main epidemiological parameters that describe the transmission of SARS-Cov-2 and the illness caused by this virus, coronavirus disease 2019 (COVID-19).

Methods: This is a methodological protocol that enabled a rapid review of COVID-19 epidemiological parameters.

Findings: The protocol consisted of the following steps: definition of scope; eligibility criteria; information sources; search strategies; selection of studies; and data extraction. Four reviewers and three supervisors conducted this review in 40 days. Of the 1,266 studies found, 65 were included, mostly observational and descriptive in content, indicating relative homogeneity as to the quality of the evidence. The variation in the basic reproduction number, between 0.48 and 14.8; and the median of the hospitalization period, between 7.5 and 20.5 days stand out as key findings.

Conclusion: We identified and synthesized 10 epidemiological parameters that may support predictive models and other rapid reviews to inform modeling of this and other future public health emergencies.

A Novel Mathematical Model of Glaucoma Pathogenesis

Background

Conventional experimental approaches to understand glaucoma etiology and pathogenesis and, consequently, predict its course of progression have not seen much success due to the involvement of numerous molecular, cellular, and other moieties. An overwhelming number of these moieties at different levels combined with numerous environmental factors further complicate the intricacy. Interaction patterns between these factors are important to understand yet difficult to probe with conservative experimental approaches.

Methods

We performed a system-level analysis with mathematical modeling by developing and analyzing rate equations with respect to the cellular events in glaucoma pathogenesis. Twenty-two events were enlisted from the literature survey and were analyzed in terms of the sensitivity coefficient of retinal ganglion cells. A separate rate equation was developed for cellular stress also. The results were analyzed with respect to time, and the time course of the events with respect to various cellular moieties was analyzed.

Results

Our results suggest that microglia activation is among the earliest events in glaucoma pathogenesis. This modeling method yields a wealth of useful information which may serve as an important guide to better understand glaucoma pathogenesis and design experimental approaches and also identify useful diagnostic/predictive methods and important therapeutic targets.

Conclusion

We here report the first mathematical model for glaucoma pathogenesis which provides important insight into the sensitivity coefficient and glia-mediated pathology of glaucoma.

How to cite this article

Faiq MA, Sidhu T, et al. A Novel Mathematical Model of Glaucoma Pathogenesis. J Curr Glaucoma Pract 2019; 13(1):3–8.

Scientific Development of HTA—A Proposal by the Health Technology Assessment International Scientific Development and Capacity Building Committee

Objectives

To report from the Scientific Development and Capacity Building Committee of Health Technology Assessment International (HTAi) on activities that are being undertaken within HTAi regarding the promotion of scientific rigor in the field of health technology assessment (HTA).

Methods

Retrieval of definitions of HTA that the SDCB committee considered reflective of the current practice of HTA, followed by a narrative synthesis of the core components of HTA.

Results

Several definitions of HTA have been provided, all sharing the notion that HTA is the formal, systematic, and transparent inquiry into the meaning and value, broadly defined, of health technologies, when used in specific patient populations.

Many frameworks and tools have been developed for assessing the quality of specific tasks that may be conducted in the context of HTA. Collating such frameworks and tools is likely to be helpful in developing standards and in providing guidance as to how the scientific quality of HTA may be secured. Two current trends in HTA were noted: a stronger health systems focus, and the need to involve stakeholders throughout the HTA process. A wider systems’ perspective requires that plausible alternative scenarios are being developed, and wide consultation of various stakeholders is a prerequisite to the development of such scenarios with data from various sources.

Conclusions

Current trends in HTA will lead to different demands on the HTA expert. The task of this emerging policy professional would be not just to provide technical information for problem-solving, but also to combine it with a new function of facilitating public deliberation and learning.

A coupled multiscale model to guide malaria control and elimination

In this paper, we share with the biomathematics community a new coupled multiscale model which has the potential to inform policy and guide malaria control and elimination. The formulation of this multiscale model is based on integrating four submodels which are: (i) a sub-model for the mosquito-to-human transmission of malaria parasite, (ii) a sub-model for the human-to-mosquito transmission of malaria parasite, (iii) a within-mosquito malaria parasite population dynamics sub-model and (iv) a within-human malaria parasite population dynamics sub-model. The integration of the four submodels is achieved by assuming that the transmission parameters of the sub-model for the mosquito-to-human transmission of malaria at the epidemiological scale are functions of the dependent variables of the within-mosquito sporozoite population dynamics while the transmission parameters of the sub-model for the human-to-mosquito transmission of malaria are functions of the dependent variables of the within-human gametocyte population dynamics. This establishes a unidirectionally coupled multiscale model where the within-human and within-mosquito submodels are unidirectionally coupled to the human-to-mosquito and mosquito-to-human submodels. A fast and slow time scale analysis is performed on this system. The result is a simple multiscale model which describes the mechanics of malaria transmission in terms of the major components of the complete malaria parasite life-cycle. This multiscale modelling approach may be found useful in guiding malaria control and elimination.

FROM INDIVIDUAL HEALTH TO COMMUNITY HEALTH: TOWARDS MULTISCALE MODELING OF DIRECTLY TRANSMITTED INFECTIOUS DISEASE SYSTEMS

In this paper, we present a new method for developing a class of nested multiscale models for directly transmitted infectious disease systems that integrates within-host scale and between-host scale using community pathogen load (CPL) as a new public health measure of a community’s level of infectiousness and as an indicator of the effectiveness of health interventions. The approach develops a multiscale modeling science base for directly transmitted infectious disease systems (where the inside-host environment’s biological entities such as cells, tissues, organs, body fluids, whole body are the reservoir of infective pathogen in the community) that is comparable to an existing multiscale modeling science base for environmentally transmitted infectious diseases (where the outside-host geographical environment’s physical entities such as soil, air, formites/contact surfaces, food and water are the reservoir of infective pathogen in the community) where pathogen load in the environment is explicitly incorporated into the model. This is achieved by assuming that infected hosts in the community are homogeneous and unevenly distributed microbial habitats. We illustrate the utility of this multiscale modeling methodology by evaluating the comparative effectiveness of HIV/AIDS preventive and treatment interventions as a case study.

The use of mathematical modeling studies for evidence synthesis and guideline development: A glossary

Mathematical modeling studies are increasingly recognised as an important tool for evidence synthesis and to inform clinical and public health decision‐making, particularly when data from systematic reviews of primary studies do not adequately answer a research question. However, systematic reviewers and guideline developers may struggle with using the results of modeling studies, because, at least in part, of the lack of a common understanding of concepts and terminology between evidence synthesis experts and mathematical modellers. The use of a common terminology for modeling studies across different clinical and epidemiological research fields that span infectious and non‐communicable diseases will help systematic reviewers and guideline developers with the understanding, characterisation, comparison, and use of mathematical modeling studies. This glossary explains key terms used in mathematical modeling studies that are particularly salient to evidence synthesis and knowledge translation in clinical medicine and public health.

A primer on multiscale modelling of infectious disease systems

The development of multiscale models of infectious disease systems is a scientific endeavour whose progress depends on advances on three main frontiers: (a) the conceptual framework frontier, (b) the mathematical technology or technical frontier, and (c) the scientific applications frontier. The objective of this primer is to introduce foundational concepts in multiscale modelling of infectious disease systems focused on these three main frontiers. On the conceptual framework frontier we propose a three-level hierarchical framework as a foundational idea which enables the discussion of the structure of multiscale models of infectious disease systems in a general way. On the scientific applications frontier we suggest ways in which the different structures of multiscale models can serve as infrastructure to provide new knowledge on the control, elimination and even eradication of infectious disease systems, while on the mathematical technology or technical frontier we present some challenges that modelers face in developing appropriate multiscale models of infectious disease systems. We anticipate that the foundational concepts presented in this primer will be central in articulating an integrated and more refined disease control theory based on multiscale modelling - the all-encompassing quantitative representation of an infectious disease system.

The "RCT augmentation": a novel simulation method to add patient heterogeneity into phase III trials

BACKGROUND

Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug's effects when prescribed to wider populations and under routine care ("effectiveness"). Conversely, enrolling heterogeneous populations in RCTs can increase endpoint variability and compromise detection of a drug's effect. We developed the "RCT augmentation" method to quantitatively support RCT design in the identification of exclusion criteria to relax to address both of these considerations. In the present manuscript, we describe the method and a case study in schizophrenia.

METHODS

We applied typical RCT exclusion criteria in a real-world dataset (cohort) of schizophrenia patients to define the "RCT population" subgroup, and assessed the impact of re-including each of the following patient subgroups: (1) illness duration 1-3 years; (2) suicide attempt; (3) alcohol abuse; (4) substance abuse; and (5) private practice management. Predictive models were built using data from different "augmented RCT populations" (i.e., subgroups where patients with one or two of such characteristics were re-included) to estimate the absolute effectiveness of the two most prevalent antipsychotics against real-world results from the entire cohort. Concurrently, the impact on RCT results of relaxing exclusion criteria was evaluated by calculating the comparative efficacy of those two antipsychotics in virtual RCTs drawing on different "augmented RCT populations".

RESULTS

Data from the "RCT population", which was defined with typical exclusion criteria, allowed for a prediction of effectiveness with a bias < 2% and mean squared error (MSE) = 5.8-6.8%. Compared to this typical RCT, RCTs using augmented populations provided improved effectiveness predictions (bias < 2%, MSE = 5.3-6.7%), while returning more variable comparative effects. The impact of augmentation depended on the exclusion criterion relaxed. Furthermore, half of the benefit of relaxing each criterion was gained from re-including the first 10-20% of patients with the corresponding real-world characteristic.

CONCLUSIONS

Simulating the inclusion of real-world subpopulations into an RCT before running it allows for quantification of the impact of each re-inclusion upon effect detection (statistical power) and generalizability of trial results, thereby explicating this trade-off and enabling a controlled increase in population heterogeneity in the RCT design.

Reflections on the Future of Pharmaceutical Public-Private Partnerships: From Input to Impact

Public Private Partnerships (PPPs) are multiple stakeholder partnerships designed to improve research efficacy. We focus on PPPs in the biomedical/pharmaceutical field, which emerged as a logical result of the open innovation model. Originally, a typical PPP was based on an academic and an industrial pillar, with governmental or other third party funding as an incentive. Over time, other players joined in, often health foundations, patient organizations, and regulatory scientists. This review discusses reasons for initiating a PPP, focusing on precompetitive research. It looks at typical expectations and challenges when starting such an endeavor, the characteristics of PPPs, and approaches to assessing the success of the concept. Finally, four case studies are presented, of PPPs differing in size, geographical spread, and research focus.

Improving Realism in Clinical Trial Simulations via Real-World Data

Simulation validity depends on how well sampling distributions used reflect real‐patient characteristics, such as drug adherence, disease progression, and pharmacologic handling in the body. We challenge the current use of growth charts from nondisease‐specific pediatrics in simulations for drug development. Complementary use of data from clinical trials and the real‐world is expected to achieve a more realistic representation of clinical outcomes for decisions in drug development, regulatory approval, and health technology assessment.

GetReal in mathematical modelling: a review of studies predicting drug effectiveness in the real world

The performance of a drug in a clinical trial setting often does not reflect its effect in daily clinical practice. In this third of three reviews, we examine the applications that have been used in the literature to predict real‐world effectiveness from randomized controlled trial efficacy data. We searched MEDLINE, EMBASE from inception to March 2014, the Cochrane Methodology Register, and websites of key journals and organisations and reference lists. We extracted data on the type of model and predictions, data sources, validation and sensitivity analyses, disease area and software. We identified 12 articles in which four approaches were used: multi‐state models, discrete event simulation models, physiology‐based models and survival and generalized linear models. Studies predicted outcomes over longer time periods in different patient populations, including patients with lower levels of adherence or persistence to treatment or examined doses not tested in trials. Eight studies included individual patient data. Seven examined cardiovascular and metabolic diseases and three neurological conditions. Most studies included sensitivity analyses, but external validation was performed in only three studies. We conclude that mathematical modelling to predict real‐world effectiveness of drug interventions is not widely used at present and not well validated. © 2016 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd.

GetReal: from efficacy in clinical trials to relative effectiveness in the real world

The GetReal consortium ("incorporating real-life data into drug development") addresses the efficacy-effectiveness gap that opens between the data from well-controlled randomized trials in selected patient groups submitted to regulators and the real-world evidence on effectiveness and safety of drugs required by decision makers. Workpackage 4 of GetReal develops evidence synthesis and modelling approaches to generate the real-world evidence. In this commentary, we discuss how questions change when moving from the well-controlled randomized trial setting to real-life medical practice, the evidence required to answer these questions, the populations to which estimates will be applicable to and the methods and data sources used to produce these estimates. We then introduce the methodological reviews written by GetReal authors and published in Research Synthesis Methods on network meta-analysis, individual patient data meta-analysis and mathematical modelling to predict drug effectiveness. The critical reviews of key methods are a good starting point for the ambitious programme of work GetReal has embarked on. The different strands of work under way in GetReal have great potential to contribute to making clinical trials research as relevant as it can be to patients, caregivers and policy makers. Copyright © 2016 John Wiley & Sons, Ltd.