1
|
Gruber S, Phillips RV, Lee H, Concato J, van der Laan M. Evaluating and improving real-world evidence with Targeted Learning. BMC Med Res Methodol 2023; 23:178. [PMID: 37533017 PMCID: PMC10394864 DOI: 10.1186/s12874-023-01998-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 07/20/2023] [Indexed: 08/04/2023] Open
Abstract
BACKGROUND The Targeted Learning roadmap provides a systematic guide for generating and evaluating real-world evidence (RWE). From a regulatory perspective, RWE arises from diverse sources such as randomized controlled trials that make use of real-world data, observational studies, and other study designs. This paper illustrates a principled approach to assessing the validity and interpretability of RWE. METHODS We applied the roadmap to a published observational study of the dose-response association between ritodrine hydrochloride and pulmonary edema among women pregnant with twins in Japan. The goal was to identify barriers to causal effect estimation beyond unmeasured confounding reported by the study's authors, and to explore potential options for overcoming the barriers that robustify results. RESULTS Following the roadmap raised issues that led us to formulate alternative causal questions that produced more reliable, interpretable RWE. The process revealed a lack of information in the available data to identify a causal dose-response curve. However, under explicit assumptions the effect of treatment with any amount of ritodrine versus none, albeit a less ambitious parameter, can be estimated from data. CONCLUSIONS Before RWE can be used in support of clinical and regulatory decision-making, its quality and reliability must be systematically evaluated. The TL roadmap prescribes how to carry out a thorough, transparent, and realistic assessment of RWE. We recommend this approach be a routine part of any decision-making process.
Collapse
Affiliation(s)
| | - Rachael V Phillips
- Division of Biostatistics, University of California at Berkeley, Berkeley, CA, USA
| | - Hana Lee
- Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - John Concato
- Office of Medical Policy, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Mark van der Laan
- Division of Biostatistics, University of California at Berkeley, Berkeley, CA, USA
| |
Collapse
|
2
|
Moler-Zapata S, Hutchings A, O'Neill S, Silverwood RJ, Grieve R. Emulating Target Trials With Real-World Data to Inform Health Technology Assessment: Findings and Lessons From an Application to Emergency Surgery. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023; 26:1164-1174. [PMID: 37164043 DOI: 10.1016/j.jval.2023.04.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 04/05/2023] [Accepted: 04/26/2023] [Indexed: 05/12/2023]
Abstract
OBJECTIVES International health technology assessment (HTA) agencies recommend that real-world data (RWD) are used in some circumstances to add to the evidence base about the effectiveness and cost-effectiveness of health interventions. The target trial framework applies the design principles of randomized-controlled trials to RWD and can help alleviate inevitable concerns about bias and design flaws with nonrandomized studies. This article aimed to tackle the lack of guidance and exemplar applications on how this methodology can be applied to RWD to inform HTA decision making. METHODS We use Hospital Episode Statistics data from England on emergency hospital admissions from 2010 to 2019 to evaluate the cost-effectiveness of emergency surgery for 2 acute gastrointestinal conditions. We draw on the case study to describe the main challenges in applying the target trial framework alongside RWD and provide recommendations for how these can be addressed in practice. RESULTS The 4 main challenges when applying the target trial framework to RWD are (1) defining the study population, (2) defining the treatment strategies, (3) establishing time zero (baseline), and (4) adjusting for unmeasured confounding. The recommendations for how to address these challenges, mainly around the incorporation of expert judgment and use of appropriate methods for handling unmeasured confounding, are illustrated within the case study. CONCLUSIONS The recommendations outlined in this study could help future studies seeking to inform HTA decision processes. These recommendations can complement checklists for economic evaluations and design tools for estimating treatment effectiveness in nonrandomized studies.
Collapse
Affiliation(s)
- Silvia Moler-Zapata
- Department of Health Services Research and Policy, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, England, UK.
| | - Andrew Hutchings
- Department of Health Services Research and Policy, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, England, UK
| | - Stephen O'Neill
- Department of Health Services Research and Policy, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, England, UK
| | - Richard J Silverwood
- Centre for Longitudinal Studies, UCL Social Research Institute, University College London, London, England, UK
| | - Richard Grieve
- Department of Health Services Research and Policy, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, England, UK
| |
Collapse
|
3
|
Winterstein AG, Ehrenstein V, Brown JS, Stürmer T, Smith MY. A Road Map for Peer Review of Real-World Evidence Studies on Safety and Effectiveness of Treatments. Diabetes Care 2023; 46:1448-1454. [PMID: 37471605 PMCID: PMC10369122 DOI: 10.2337/dc22-2037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 05/05/2023] [Indexed: 07/22/2023]
Abstract
The growing acceptance of real-world evidence (RWE) in clinical and regulatory decision-making, coupled with increasing availability of health care data and advances in automated analytic approaches, has contributed to a marked expansion of RWE studies of diabetes and other diseases. However, a recent spate of high-profile retractions highlights the need for improvements in the conduct of RWE research as well as in the associated peer review and editorial processes. We review best pharmacoepidemiologic practices and common pitfalls regarding design, measurement, analysis, data validity, appropriateness, and generalizability of RWE studies. To enhance RWE study assessments, we propose that journal editors require 1) study authors to complete RECORD-PE, a reporting guideline for pharmacoepidemiological studies on routinely collected data, 2) availability of predetermined study protocols and analysis plans, 3) inclusion of pharmacoepidemiologists on the peer review team, and 4) provision of detail on data provenance, characterization, and custodianship to facilitate assessment of the data source. We recognize that none of these steps guarantees a high-quality research study. Collectively, however, they permit an informed assessment of whether the study was adequately designed and conducted and whether the data source used was fit for purpose.
Collapse
Affiliation(s)
- Almut G. Winterstein
- Department of Pharmaceutical Outcomes and Policy, Department of Epidemiology, and Center for Drug Evaluation and Safety, University of Florida, Gainesville, FL
- International Network for Epidemiology in Policy, American College of Epidemiology, Washington Avenue Extension, Albany, NY
- International Society for Pharmacoepidemiology, Bethesda, MD
| | - Vera Ehrenstein
- International Network for Epidemiology in Policy, American College of Epidemiology, Washington Avenue Extension, Albany, NY
- International Society for Pharmacoepidemiology, Bethesda, MD
- Department of Clinical Epidemiology, Aarhus University and Aarhus University Hospital, Aarhus, Denmark
| | - Jeffrey S. Brown
- International Network for Epidemiology in Policy, American College of Epidemiology, Washington Avenue Extension, Albany, NY
- International Society for Pharmacoepidemiology, Bethesda, MD
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Cambridge, MA
- TriNetX, LLC, Cambridge, MA
| | - Til Stürmer
- International Network for Epidemiology in Policy, American College of Epidemiology, Washington Avenue Extension, Albany, NY
- International Society for Pharmacoepidemiology, Bethesda, MD
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Meredith Y. Smith
- International Network for Epidemiology in Policy, American College of Epidemiology, Washington Avenue Extension, Albany, NY
- International Society for Pharmacoepidemiology, Bethesda, MD
- Evidera, Inc., PPD, Boston, MA
- School of Pharmacy, University of Southern California, Los Angeles, CA
| |
Collapse
|
4
|
Gregg EW, Patorno E, Karter AJ, Mehta R, Huang ES, White M, Patel CJ, McElvaine AT, Cefalu WT, Selby J, Riddle MC, Khunti K. Use of Real-World Data in Population Science to Improve the Prevention and Care of Diabetes-Related Outcomes. Diabetes Care 2023; 46:1316-1326. [PMID: 37339346 PMCID: PMC10300521 DOI: 10.2337/dc22-1438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 04/11/2023] [Indexed: 06/22/2023]
Abstract
The past decade of population research for diabetes has seen a dramatic proliferation of the use of real-world data (RWD) and real-world evidence (RWE) generation from non-research settings, including both health and non-health sources, to influence decisions related to optimal diabetes care. A common attribute of these new data is that they were not collected for research purposes yet have the potential to enrich the information around the characteristics of individuals, risk factors, interventions, and health effects. This has expanded the role of subdisciplines like comparative effectiveness research and precision medicine, new quasi-experimental study designs, new research platforms like distributed data networks, and new analytic approaches for clinical prediction of prognosis or treatment response. The result of these developments is a greater potential to progress diabetes treatment and prevention through the increasing range of populations, interventions, outcomes, and settings that can be efficiently examined. However, this proliferation also carries an increased threat of bias and misleading findings. The level of evidence that may be derived from RWD is ultimately a function of the data quality and the rigorous application of study design and analysis. This report reviews the current landscape and applications of RWD in clinical effectiveness and population health research for diabetes and summarizes opportunities and best practices in the conduct, reporting, and dissemination of RWD to optimize its value and limit its drawbacks.
Collapse
Affiliation(s)
- Edward W. Gregg
- School of Population Health, RRCSI University of Medicine and Health Sciences, Dublin, Ireland
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, U.K
| | - Elisabetta Patorno
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA
| | - Andrew J. Karter
- Division of Research, Kaiser Permanente, Oakland, CA
- Department of Health Systems and Population Health, University of Washington, Seattle, WA
| | - Roopa Mehta
- Metabolic Research Unit (UIEM), Department of Endocrinology and Metabolism, Instituto Nacional de Ciencias Medicas y Nutricion, Salvador Zubiran (INCMNSZ), Mexico City, Mexico
| | - Elbert S. Huang
- Section of General Internal Medicine, Center for Chronic Disease Research and Policy (CDRP), The University of Chicago, Chicago, IL
| | - Martin White
- Medical Research Council Epidemiology Unit, University of Cambridge, Cambridge, U.K
| | - Chirag J. Patel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA
| | | | - William T. Cefalu
- Division of Diabetes, Endocrinology, and Metabolic Diseases, National Institute of Diabetes, Digestive, and Kidney Diseases, National Institutes of Health, Bethesda, MD
| | - Joseph Selby
- Patient-Centered Outcomes Institute, Washington, DC
| | - Matthew C. Riddle
- Division of Endocrinology, Diabetes, and Clinical Nutrition, Oregon Health & Science University, Portland, OR
| | - Kamlesh Khunti
- Leicester Real World Evidence Unit, Diabetes Research Centre, University of Leicester, Leicester, U.K
| |
Collapse
|
5
|
AlMalki MH, Aldesokey H, Alkhafaji D, Alsheikh A, Braae UC, Lehrskov LL, Magawry W, Yahia M, Haroun A. Glycaemic Control in People with Type 2 Diabetes Treated with Insulin Degludec: A Real-World, Prospective Non-interventional Study-UPDATES Saudi Arabia. Adv Ther 2023; 40:568-584. [PMID: 36417058 PMCID: PMC9685061 DOI: 10.1007/s12325-022-02366-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 10/21/2022] [Indexed: 11/24/2022]
Abstract
INTRODUCTION Insulin degludec (degludec) has proven benefits in type 2 diabetes (T2D), in terms of improved glycaemic control, low risk of hypoglycaemia, and flexibility in dosing time. This prospective non-interventional UPDATES study aimed to investigate whether results obtained from randomised clinical trials and other real-world studies with degludec are generalisable to patients with T2D in routine clinical practice in Saudi Arabia. METHODS Eligible adults (n = 561) with T2D received degludec for 26-34 weeks, at physicians' discretion and in accordance with local routine clinical practice. The primary endpoint was mean change in HbA1c from baseline to end of study (EOS). Secondary endpoints included mean change from baseline to EOS in fasting plasma glucose (FPG), daily insulin dose and rate of hypoglycaemia. RESULTS At baseline, mean age, HbA1c and FPG were 55.7 years, 9.4% and 185.6 mg/dL, respectively. Mean (standard error [SE]) changes from baseline to EOS (crude analysis) were statistically significant for HbA1c (- 1.1 [0.08] %-points, 95% CI - 1.29, - 0.98; P < 0.0001), FPG (- 39.1 [3.42] mg/dL, 95% CI - 45.9, - 32.4; P < 0.0001) and total daily insulin dose (+ 4.7 [1.6] units, 95% CI 1.63, 7.86; P = 0.003, insulin-experienced population). In exploratory analysis of patients switching from insulin glargine U100 or U300 to degludec, similar reductions were seen in HbA1c and FPG. The rate of hypoglycaemia was significantly reduced with degludec versus previous treatment, with no apparent or unexpected safety and tolerability issues. The number of insulin-experienced patients utilising resources associated with severe hypoglycaemia was also reduced. Most patients (95.5%) were willing to continue treatment at EOS, and expressed a preference for degludec over their previous regimen (93.0%). CONCLUSION Patients with T2D treated with degludec in routine clinical practice in Saudi Arabia experienced clinically significant improvements in glycaemic control and a lower rate of hypoglycaemia compared with baseline, with no new safety concerns reported. CLINICAL TRIAL REGISTRATION NCT03785522.
Collapse
Affiliation(s)
- Mussa H. AlMalki
- grid.415277.20000 0004 0593 1832Obesity, Endocrine and Metabolism Center, King Fahad Medical City, PO Box 59046, Riyadh, 11525 Saudi Arabia ,grid.415277.20000 0004 0593 1832King Fahad Medical City, College of Medicine, King Saud Bin Abdul Aziz University for Health Sciences, Riyadh, Saudi Arabia
| | | | - Dania Alkhafaji
- grid.411975.f0000 0004 0607 035XKing Fahd Hospital of the University, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia
| | - Abdulrahman Alsheikh
- grid.412126.20000 0004 0607 9688King Abdulaziz University Hospital, Jeddah, Saudi Arabia
| | | | | | - Waleed Magawry
- grid.412126.20000 0004 0607 9688King Abdulaziz University Hospital, Jeddah, Saudi Arabia
| | | | | |
Collapse
|
6
|
Liu F, Demosthenes P. Real-world data: a brief review of the methods, applications, challenges and opportunities. BMC Med Res Methodol 2022; 22:287. [PMID: 36335315 PMCID: PMC9636688 DOI: 10.1186/s12874-022-01768-6] [Citation(s) in RCA: 60] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 10/22/2022] [Indexed: 11/07/2022] Open
Abstract
Abstract
Background
The increased adoption of the internet, social media, wearable devices, e-health services, and other technology-driven services in medicine and healthcare has led to the rapid generation of various types of digital data, providing a valuable data source beyond the confines of traditional clinical trials, epidemiological studies, and lab-based experiments.
Methods
We provide a brief overview on the type and sources of real-world data and the common models and approaches to utilize and analyze real-world data. We discuss the challenges and opportunities of using real-world data for evidence-based decision making This review does not aim to be comprehensive or cover all aspects of the intriguing topic on RWD (from both the research and practical perspectives) but serves as a primer and provides useful sources for readers who interested in this topic.
Results and Conclusions
Real-world hold great potential for generating real-world evidence for designing and conducting confirmatory trials and answering questions that may not be addressed otherwise. The voluminosity and complexity of real-world data also call for development of more appropriate, sophisticated, and innovative data processing and analysis techniques while maintaining scientific rigor in research findings, and attentions to data ethics to harness the power of real-world data.
Collapse
|
7
|
Penner LS, Gavan SP, Ashcroft DM, Peek N, Elliott RA. Does coprescribing nonsteroidal anti-inflammatory drugs and oral anticoagulants increase the risk of major bleeding, stroke and systemic embolism? Br J Clin Pharmacol 2022; 88:4789-4811. [PMID: 35484847 PMCID: PMC9796910 DOI: 10.1111/bcp.15371] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Revised: 03/21/2022] [Accepted: 04/11/2022] [Indexed: 01/07/2023] Open
Abstract
AIMS To examine the risk of gastrointestinal (GI) bleeding, major bleeding, stroke and systemic embolism associated with prescribing nonsteroidal anti-inflammatory drugs (NSAIDs) to adults receiving oral anticoagulant (OAC) therapy. METHODS We conducted a population-based cohort study in adults receiving OAC therapy using linked primary care (Clinical Practice Research Datalink GOLD) and hospital (Hospital Episodes Statistics) electronic health records. We used cause-specific Cox regression models with time-dependent NSAID treatment in a propensity score matched population to estimate the increased risk of GI bleeding, stroke, major bleeding and systemic embolism associated with NSAID use. RESULTS The matched cohort contained 3177 patients with OAC therapy alone and 3177 with at least 1 concomitant NSAID prescription. Compared with OAC therapy alone, concomitant prescription of NSAIDs with OACs was associated with increased risk of GI bleeding (hazard ratio [HR] 3.01, 95% confidence interval [CI] 1.63 to 5.55), stroke (HR 2.71, 95% CI 1.48 to 4.96) and major bleeding (HR 2.77, 95% CI 1.84 to 4.19). The association with systemic embolism did not reach statistical significance (HR 3.02, 95% CI 0.82 to 11.07). Sensitivity analyses indicated that the results were robust to changes in exclusion criteria and the choice of potential confounding variables. CONCLUSION When OACs are coprescribed with NSAIDs, the risk of adverse bleeding events increases and, simultaneously, the protective effect of OACs to prevent strokes reduces. There is a need for interventions that reduce hazardous prescribing of NSAIDs in people receiving OAC therapy.
Collapse
Affiliation(s)
- Leonie S. Penner
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK,NIHR Greater Manchester Patient Safety Translational Research Centre, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK
| | - Sean P. Gavan
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK
| | - Darren M. Ashcroft
- NIHR Greater Manchester Patient Safety Translational Research Centre, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK,Division of Pharmacy & Optometry, School of Health Sciences, Faculty of BiologyMedicine and Health, University of ManchesterManchesterUK
| | - Niels Peek
- NIHR Greater Manchester Patient Safety Translational Research Centre, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK,Division of Informatics, Imaging and Data Science, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK
| | - Rachel A. Elliott
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and HealthUniversity of ManchesterManchesterUK
| |
Collapse
|
8
|
Saldarriaga EM, Hauber B, Carlson JJ, Barthold D, Veenstra DL, Devine B. Assessing Payers' Preferences for Real-World Evidence in the United States: A Discrete Choice Experiment. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:443-450. [PMID: 35227457 DOI: 10.1016/j.jval.2021.09.016] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 09/20/2021] [Accepted: 09/28/2021] [Indexed: 06/14/2023]
Abstract
OBJECTIVES To rank the US payers' preferences for attributes of real-world evidence (RWE) studies in the context of chronic disease and to quantify trade-offs among them. METHODS We conducted a discrete choice experiment in which 180 employees from payer organizations were tasked to choose between 2 RWE studies assuming they were assessing evidence to inform formulary decisions for chronic disease treatment. Each RWE study was characterized by 7 attributes with 3 levels each: very informative, moderately informative, and not measured. We used a D-optimal main-effects design. Survey data were fitted to a conditional logit model to obtain a relative measure of the ranking of importance for each attribute. RESULTS Clinical outcomes were the most preferred attribute. It was 4.68 times as important as productivity outcomes-the least preferred attribute. It was followed by health-related quality of life (2.78), methodologic rigor (2.09), resource utilization (1.71), and external validity (1.56). CONCLUSIONS This study provides a quantification of the value payers place on key RWE attributes. Across attributes, payers have higher preferences for clinical and health-related quality of life outcomes than the other attributes. Between attributes' levels, payers prefer high levels of information in clinical outcomes and methodologic rigor but are indifferent in other attributes. Our results bridge the gap between the information that payers seek and the attributes that RWE studies prioritize and effectively guide future research design.
Collapse
Affiliation(s)
- Enrique M Saldarriaga
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA
| | - Brett Hauber
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA; Pfizer, Inc., New York, NY, USA
| | - Josh J Carlson
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA
| | - Douglas Barthold
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA
| | - David L Veenstra
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA
| | - Beth Devine
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA.
| |
Collapse
|
9
|
Schneeweiss S, Patorno E. Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments. Endocr Rev 2021; 42:658-690. [PMID: 33710268 PMCID: PMC8476933 DOI: 10.1210/endrev/bnab007] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Indexed: 12/12/2022]
Abstract
Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.
Collapse
Affiliation(s)
- Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MAUSA
| | - Elisabetta Patorno
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MAUSA
| |
Collapse
|
10
|
Blonde L, Bailey T, Sullivan SD, Freemantle N. Insulin glargine 300 units/mL for the treatment of individuals with type 2 diabetes in the real world: A review of the DELIVER programme. Diabetes Obes Metab 2021; 23:1713-1721. [PMID: 33881797 PMCID: PMC8362061 DOI: 10.1111/dom.14405] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 03/30/2021] [Accepted: 04/13/2021] [Indexed: 12/31/2022]
Abstract
Evidence from randomized controlled trials (RCTs) has shown that second-generation basal insulin (BI) analogues, insulin glargine 300 U/mL (Gla-300) and insulin degludec (IDeg), provide similar glycaemic control, with a lower risk of hypoglycaemia compared with the first-generation BI analogue insulin glargine 100 U/mL (Gla-100) in people with type 2 diabetes (T2D). However, the highly selected participants and frequent follow-up of RCTs may not be truly representative of real-life clinical practice. It is important to assess the safety and effectiveness of these second-generation BI analogues in real-life clinical practice settings. The DELIVER programme utilized electronic healthcare records from the United States to compare clinical outcomes in people with T2D who received either Gla-300 or other BI analogues in real-world clinical practice. This review provides a concise overview of the results of the DELIVER studies. Overall, Gla-300 provided similar antihyperglycaemic effectiveness and a lower risk of hypoglycaemia versus the first-generation BI analogues Gla-100 and insulin detemir in people with T2D who had switched BIs. In those who were insulin-naïve, initiation with Gla-300 versus Gla-100 was associated with significantly better antihyperglycaemic effectiveness and similar or lower hypoglycaemic risk. Both glycaemic control and hypoglycaemia risk were also shown to be similar with Gla-300 and IDeg, in people who had switched BIs and in those who were insulin-naïve. In addition, the DELIVER 2 study reported that people with T2D who switched to Gla-300 had reduced healthcare resource utilization, with an overall saving of US$1439 per person per year compared with those who switched to another BI analogue. Overall, the real-world DELIVER programme showed that the glycaemic control with a low risk of hypoglycaemia observed with Gla-300 in RCTs was also seen in standard clinical practice.
Collapse
Affiliation(s)
- Lawrence Blonde
- Frank Riddick Diabetes Institute, Department of EndocrinologyOchsner Medical CenterNew OrleansLouisianaUSA
| | | | - Sean D. Sullivan
- The CHOICE Institute, School of PharmacyUniversity of WashingtonSeattleWashingtonUSA
| | - Nick Freemantle
- Institute of Clinical Trials and MethodologyUniversity College LondonLondonUK
| |
Collapse
|
11
|
Rassen JA, Murk W, Schneeweiss S. Real-world evidence of bariatric surgery and cardiovascular benefits using electronic health records data: A lesson in bias. Diabetes Obes Metab 2021; 23:1453-1462. [PMID: 33566434 DOI: 10.1111/dom.14338] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/15/2020] [Revised: 01/22/2021] [Accepted: 01/30/2021] [Indexed: 12/14/2022]
Abstract
AIM To reproduce and correct studies on bariatric surgery and the reduction in major adverse cardiovascular events (MACE) among patients with obesity and type 2 diabetes (T2D). METHODS We used electronic healthcare records (EHR) from in and outpatient facilities around the United States to identify a cohort of patients with T2D, aged 18 to 80 years and with a body mass index (BMI) of 30 kg/m2 or higher undergoing bariatric surgery. We compared against hip/knee arthroplasty to establish an active comparison group that reduced bias from differential information and confounding. The main outcome was six-point MACE. Pre-exposure characteristics were adjusted in propensity score (PS) models with 1:2 matching plus high-dimensional PS matching. RESULTS After a range of exclusions, the final cohort included 344 bariatric surgery patients (65% female; mean age 58 years) and 551 PS-matched patients undergoing arthroplasty (65% female; 59 years). Median follow-up was 2.5 years in both groups. Bariatric surgery patients showed a sustained 20% weight reduction and an HbA1c reduction by 1% point. We found no benefits of bariatric surgery for six-point MACE (HR = 0.99; 95% CI 0.76-1.30). We observed known increases in risks for vitamin B12 deficiency anaemia (HR = 3.06; 1.10-8.49) and cholelithiasis (HR = 1.72; 0.94-3.13). CONCLUSIONS This real-world evidence study found reductions in HbA1c and BMI following bariatric surgery similar to trials, and no meaningful cardiovascular benefit compatible with the underpowered trials but in contrast to earlier EHR studies. We showed how information bias typical in EHR analyses and confounding may cause substantial bias.
Collapse
Affiliation(s)
| | - William Murk
- Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York, USA
| | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
12
|
Gadde MA, Wang Z, Zozus M, Talburt JB, Greer ML. Rules Based Data Quality Assessment on Claims Database. Stud Health Technol Inform 2020; 272:350-353. [PMID: 32604674 PMCID: PMC7899162 DOI: 10.3233/shti200567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
Data quality problems in coded clinical and administrative data have persisted ever since diagnoses and procedures were first coded and used for healthcare billing. These data are used in clinical decision-making introducing a route for iatrogenesis. As we share data on regional Health Information Exchanges (HIEs) and include them in electronic health records the potential for harm may be increased. To study this problem we applied rules-based data quality checks that have been previously tested on Electronic Health Records (EHR) data on a limited set of aggregated claims data. Medicaid claims data was used exclusively. CMS has clear guidelines for claims submitted for Medicaid patients and penalties are incurred for erroneous claims, which should ensure a high quality data source, however reports of low and varying sensitivity, specificity, positive and negative predictive value of coded diagnoses are common. To identify data quality defects in claims data in a state All Payer Claims Dataset (APCD) we applied and evaluated a recently developed rules-based data quality assessment and monitoring system for Electronic Health Record (EHR) data to test effectiveness in claims data. These rules, that are feasible for “All Payer Claims data” and Medicaid data are identified, applied and the Data Quality issue results are produced.
Collapse
Affiliation(s)
- Mary A Gadde
- University of Arkansas for Medical Sciences, Littlerock, Arkansas, USA
| | - Zhan Wang
- University of Texas Health Science Center, San Antonio, TX, USA
| | - Meredith Zozus
- University of Texas Health Science Center, San Antonio, TX, USA
| | - John B Talburt
- University of Arkansas for Medical Sciences, Littlerock, Arkansas, USA
| | - Melody L Greer
- University of Arkansas for Medical Sciences, Littlerock, Arkansas, USA
| |
Collapse
|