Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative

OpenSAFELY: A platform for analysing electronic health records designed for reproducible research

Electronic health records (EHRs) and other administrative health data are increasingly used in research to generate evidence on the effectiveness, safety, and utilisation of medical products and services, and to inform public health guidance and policy. Reproducibility is a fundamental step for research credibility and promotes trust in evidence generated from EHRs. At present, ensuring research using EHRs is reproducible can be challenging for researchers. Research software platforms can provide technical solutions to enhance the reproducibility of research conducted using EHRs. In response to the COVID-19 pandemic, we developed the secure, transparent, analytic open-source software platform OpenSAFELY designed with reproducible research in mind. OpenSAFELY mitigates common barriers to reproducible research by: standardising key workflows around data preparation; removing barriers to code-sharing in secure analysis environments; enforcing public sharing of programming code and codelists; ensuring the same computational environment is used everywhere; integrating new and existing tools that encourage and enable the use of reproducible working practices; and providing an audit trail for all code that is run against the real data to increase transparency. This paper describes OpenSAFELY's reproducibility-by-design approach in detail.

Real-World Evidence to Reinforce Clinical Trial Evidence in Health Technology Assessment: A Critical Review of Real-World Evidence Requirements from Seven Countries and Recommendations to Improve Acceptance

BACKGROUND

Real-world evidence (RWE) can reinforce clinical trial evidence in health technology assessment (HTA).

OBJECTIVES

Review HTA bodies' (HTAbs) requirements for RWE, real uses, and acceptance across seven countries (Brazil, Canada, France, Germany, Italy, Spain, and the United Kingdom) and outline recommendations that may improve acceptance of RWE in efficacy/effectiveness assessments and appraisals processes.

METHODS

RWE requirements were summarized based on HTAbs' guidelines. Acceptance by HTAbs was evaluated based on industry experience and case studies.

RESULTS

As of June 2022, RWE methodological guidelines were in place in three of the seven countries. HTAbs typically requested analyses based on local data sources, but the preferred study design and data sources differed. HTAbs had individual submission, assessment, and appraisal processes; some allowed early meetings for the protocol and/or results validation, though few involved external experts or medical societies to provide input to assessment and appraisal. The extent of submission, assessment, and appraisal requirements did not necessarily reflect the degree of acceptance.

CONCLUSION

All the countries reviewed face common challenges regarding the use of RWE. Our proposals address the need to facilitate collaboration and communication with industry and regulatory agencies and the need for specific guidelines describing RWE design and criteria of acceptance throughout the assessment and appraisal processes.

From “exceptional ethics” to public health ethics. Patients and proxies facing COVID-19. Ethics and public health during COVID-19

The study aimed to elicit the perception and ethical considerations of patients and proxies with respect both to the individual medical decisions and public health decisions made during the COVID-19 crisis. It used a qualitative, multi-center study based on semi-directive interviews, conducted by an interdisciplinary team. The analysis was conducted using a thematic analysis approach and an ethical framework. Three themes emerged from the analysis: 1) patients, unlike proxies, did not complain about their diminished role in the decision-making process. Both highlighted the importance of “basic care” as opposed to a technical approach to treatment; 2) despite the transparency of the information process, a deep “crisis of trust” has developed between citizens and public authorities; 3) although both patients and proxies accepted the limitations of personal liberties imposed in the name of public health, they argued that these limitations should respect certain boundaries, both temporal and spacial. Above all, they should not affect basic affective human relationships, even if such boundaries are a factor in an increased risk of infection. The study showed that there is a need to reconsider the definition and the main principles of public health ethics, namely transparency and proportionality.

Generating and using real-world data: A worthwhile uphill battle

The precision oncology paradigm challenges the feasibility and data generalizability of traditional clinical trials. Consequently, an unmet need exists for practical approaches to test many subgroups, evaluate real-world drug value, and gather comprehensive, accessible datasets to validate novel biomarkers. Real-world data (RWD) are increasingly recognized to have the potential to fill this gap in research methodology. Established applications of RWD include informing disease epidemiology, pharmacovigilance, and healthcare quality assessment. Currently, concerns regarding RWD quality and comprehensiveness, privacy, and biases hamper their broader application. Nonetheless, RWD may play a pivotal role in supplementing clinical trials, enabling conditional reimbursement and accelerated drug access, and innovating trial conduct. Moreover, purpose-built RWD repositories may support the extension or refinement of drug indications and facilitate the discovery and validation of new biomarkers. This perspective explores the potential of leveraging RWD to advance oncology, highlights its benefits and challenges, and suggests a path forward in this evolving field.

Celebrating the Data from 100,000 Real-World Users of the MiniMed™ 780G System in Europe, Middle East, and Africa Collected Over 3 Years: From Data to Clinical Evidence

Introduction: The present report celebrates the benchmarking of 100,000 MiniMed™ 780G system users in Europe, Middle East, and Africa (EMEA) and summarizes the major insights into the usability and outcomes of this system. Methods: Carelink Personal data (August 2020-August 2023) of users living in EMEA were analyzed. Continuous glucose monitoring-based endpoints were aggregated for (1) the full cohort and (2) a 12-month longitudinal cohort. Subanalyses were done for users on optimal settings (those spending ≥95% of time with glucose target of 100 mg/dL, and ≥95% of time with active insulin time of 2 h), for self-reported age groups (≤15 and ≥56 years) and for various countries/regions. Results: Data from 101,629 users (34 countries) were analyzed. Mean time in range (TIR) was 72.3%, glucose management indicator (GMI) was 7%, time below 70 mg/dL (TBR70) was 2.0% and time below 54 mg/dL (TBR54) was 0.4%. In terms of international targets, 59.6% of users achieved a GMI <7%, 62.5% a TIR >70%, 88.4% a TBR70 < 4%, and 90.0% a TBR54 < 1%. Data improved impressively in optimal setting users (TIR = 78.8%, and users reaching TIR >70% = 86.3%) while safety remained (TBR70 = 2.2% and TBR54 = 0.4%). Data showed consistency across self-reported age groups and geographies. In the longitudinal cohort, TIR reached 75.5% in the first month and remained 73.3% or higher over the 12-month period. Conclusion: Over 100,000 users of the MiniMed™ 780G system have demonstrated consistency in achieving target control of glycemia.

Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis

OBJECTIVES

We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.

DESIGN

Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies.

DATA SOURCES

Systematic search: Medline; Snowballing: starting from three articles (D'Andrea et al, Quigley et al and Faria et al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies.

DATA EXTRACTION AND SYNTHESIS

Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting.

RESULTS

Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items.

CONCLUSION

Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap.

OSF REGISTRATION NUMBER

OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).

A patient-centred and multi-stakeholder co-designed observational prospective study protocol: Example of the adolescent experience of treatment for X-linked hypophosphataemia (XLH)

The importance of patient centricity and keeping the patient at the heart of research design is now well recognised within the healthcare community. The involvement of patient, caregiver and clinician representatives in the study design process may help researchers to achieve this goal and to ensure robust and meaningful data generation. Real-world data collection allows for a more flexible and patient-centred research approach for gaining important insights into the experience of disease and treatments, which is acutely relevant for rare diseases where knowledge about the disease is more likely to be limited. Here, we describe a practical example of a patient-centric, multi-stakeholder approach that led to the co-design of a prospective observational study investigating the lived experience of adolescents with the rare disease, X-linked hypophosphataemia. Specifically, we describe how the knowledge and expertise of a diverse research team, which included expert physicians, research and technology specialists, patients and caregivers, were applied in order to identify the relevant research questions and to ensure the robustness of the study design and its appropriateness to the population of interest within the context of the current clinical landscape. We also demonstrate how a structured patient engagement exercise was key to informing the selection of appropriate outcome measures, data sources, timing of data collection, and to assessing the feasibility and acceptability of the proposed data collection approach.

Building trust in real world evidence (RWE): moving transparency in RWE towards the randomized controlled trial standard

The increasing use of RWE in regulatory and reimbursement decision-making indicates the significant progress that has been made in building trust in RWE through greater transparency. This review of the published literature and key online sources was conducted to provide an update on progress towards greater transparency in RWE, based on four key barriers to trust identified in a 2016 paper and applying learnings from transparency initiatives established for RCTs, such as the US FDA Amendments Act (FDAAA) 2007 Final Rule. Multiple initiatives from the US FDA, EMA and organizations such as the ISPOR-ISPE Joint Task Force have yielded new guidance documents and tools that enable greater transparency in RWE study methodology (STaRT-RWE and HARPER templates), data source selection and quality (SPIFD2 framework, REQueST tool), strategy (the Center for Open Science RWE Study Registry), and will therefore improve transparency in RWE study reporting. Programs such as the REPEAT Initiative and RWE DUPLICATE are investigating reproducibility of RWE studies and improving understanding of the circumstances when valid inference on treatment effects can be obtained from RWE studies. Further work is needed to embed and to implement the new tools and guidance that are available, and thus raise standards for RWE transparency towards the levels expected for RCTs.

Principles for Good Practice in the Conduct of Non-interventional Studies: The View of Industry Researchers

This reflection paper presents a consolidated view of EFPIA on the need for principles for good practice in the generation and use of non-interventional studies (NIS), including overarching principles such as the registration of hypothesis evaluating treatment effect (HETE) studies. We first define NIS and the important adjacencies to clinical trials and relationship with real-world evidence (RWE). We then outline the principles for good practice with respect to appropriate research design, study protocol, fit-for-purpose variables and data quality, analytical methods, bias reduction, transparency in conduct and use, privacy management and ethics review. We conclude with recommendations for action for the research community to promote trust and credibility in the use of NIS.

A Road Map for Peer Review of Real-World Evidence Studies on Safety and Effectiveness of Treatments

The growing acceptance of real-world evidence (RWE) in clinical and regulatory decision-making, coupled with increasing availability of health care data and advances in automated analytic approaches, has contributed to a marked expansion of RWE studies of diabetes and other diseases. However, a recent spate of high-profile retractions highlights the need for improvements in the conduct of RWE research as well as in the associated peer review and editorial processes. We review best pharmacoepidemiologic practices and common pitfalls regarding design, measurement, analysis, data validity, appropriateness, and generalizability of RWE studies. To enhance RWE study assessments, we propose that journal editors require 1) study authors to complete RECORD-PE, a reporting guideline for pharmacoepidemiological studies on routinely collected data, 2) availability of predetermined study protocols and analysis plans, 3) inclusion of pharmacoepidemiologists on the peer review team, and 4) provision of detail on data provenance, characterization, and custodianship to facilitate assessment of the data source. We recognize that none of these steps guarantees a high-quality research study. Collectively, however, they permit an informed assessment of whether the study was adequately designed and conducted and whether the data source used was fit for purpose.

Effectively Leveraging RWD for External Controls: A Systematic Literature Review of Regulatory and HTA Decisions

Real-world data (RWD)-derived external controls can be used to contextualize efficacy findings for investigational therapies evaluated in uncontrolled trials. As the number of submissions to regulatory and health technology assessment (HTA) bodies using external controls rises, and in light of recent regulatory and HTA guidance on the appropriate use of RWD, there is a need to address the operational and methodological challenges impeding the quality of real-world evidence (RWE) generation and the consistency in evaluation of RWE across agencies. This systematic review summarizes publicly available information on the use of external controls to contextualize outcomes from uncontrolled trials for all indications from January 1, 2015, through August 20, 2021, that were submitted to the European Medicines Agency, the US Food and Drug Administration, and/or select major HTA bodies (National Institute for Health and Care Excellence (NICE), Haute Autorité de Santé (HAS), Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), and Gemeinsamer Bundesausschuss (G-BA)). By systematically reviewing submissions to regulatory and HTA bodies in the context of recent guidance, this study provides quantitative and qualitative insights into how external control design and analytic choices may be viewed by different agencies in practice. The primary operational and methodological aspects identified for discussion include, but are not limited to, engagement of regulators and HTA bodies, approaches to handling missing data (a component of data quality), and selection of real-world endpoints. Continued collaboration and guidance to address these and other aspects will inform and assist stakeholders attempting to generate evidence using external controls.

Integration of real-world evidence from different data sources in health technology assessment

Real-world evidence (RWE) is being increasingly used by a wide range of stakeholders involved in the therapeutic product lifecycle but remains underutilized in the health technology assessment (HTA) process. RWE aims to fill the current evidence gaps, reduce the uncertainty around the benefits of medical technologies, and better understand the long-term impact of health technologies in real-world conditions. Despite the minimal use of RWE in some elements of HTA, there has been a larger push to further utilize RWE in the HTA processes. HTA bodies, as other stakeholders, work towards developing more robust means to leverage RWE from various data sources in the HTA processes. However, these agencies need to overcome important challenges before the broader incorporation of RWE into their routine practice. This paper aims to explore the extensive integration of RWE utilizing diverse sources of RWD. We discuss the utilization of RWE in HTA processes, considering aspects such as when, where, and how RWE can be effectively applied. Additionally, we seek the potential challenges and barriers associated with the utilization of different data sources.

Using Real-World Data in the Health Technology Assessment of Pharmaceuticals: Strengths, Difficulties, and a Pragmatic Way Forward

In the past decade, there have been increasing calls for greater use of real-world evidence (RWE) and data (RWD), with the explicit goal of enabling faster provision of effective medicines to patients in need. The push for decision makers to accept RWE is especially noticeable in the pursuit of regulatory approval, but RWE, particularly when used to estimate the relative effectiveness of interventions, is not always readily accepted by agencies responsible for reimbursement and pricing of new pharmaceuticals and, to a varying degree, is not accepted across jurisdictions. This lack of trust hampers the use of RWE despite a very large and growing literature base on the principles of how RWE should be used. In this article, we suggest an important part of the explanation of why this situation has arisen and make suggestions for its alleviation. Given that problems commonly arise that are particular to the question being asked and the data sources being used, general guidance on the principles of how to use RWD cannot cover all eventualities. Therefore, we are suggesting the creation of an archive, or repository, to record uses of RWD in support of decisions by funding bodies or their advisors. This article introduces a proposed, structured classification of decision types using RWE, around which evidence can be assembled in a curated source (RWD/RWE taxonomy) and thus facilitate judgments on when evidence is "good enough." This article is part of a series in a special issue of this journal that looks at the barriers to optimal use of RWE in health technology assessment and how to overcome them. We begin significantly to populate our "taxonomy" with examples in an accompanying article. We also propose recommendations for international standards of evaluating the acceptability of RWD governance practices.

Guidance on using real-world evidence from Western Europe in Central and Eastern European health policy decision making

Aim: Real-world data and real-world evidence (RWE) are becoming more important for healthcare decision making and health technology assessment. We aimed to propose solutions to overcome barriers preventing Central and Eastern European (CEE) countries from using RWE generated in Western Europe. Materials & methods: To achieve this, following a scoping review and a webinar, the most important barriers were selected through a survey. A workshop was held with CEE experts to discuss proposed solutions. Results: Based on survey results, we selected the nine most important barriers. Multiple solutions were proposed, for example, the need for a European consensus, and building trust in using RWE. Conclusion: Through collaboration with regional stakeholders, we proposed a list of solutions to overcome barriers on transferring RWE from Western Europe to CEE countries.

Cost-utility analysis of the UPRIGHT intervention promoting resilience in adolescents

BACKGROUND

As mental health in adulthood is related to mental status during adolescence, school-based interventions have been proposed to improve resilience. The objective of this study was to build a simulation model representing the natural history of mental disorders in childhood, adolescence and youth to estimate the cost-effectiveness of the UPRIGHT school-based intervention in promoting resilience and mental health in adolescence.

METHODS

We built a discrete event simulation model fed with real-world data (cumulative incidence disaggregated into eight clusters) from the Basque Health Service database (609,381 individuals) to calculate utilities (quality-adjusted life years [QALYs]) and costs for the general population in two scenarios (base case and intervention). The model translated changes in the wellbeing of adolescents into different risks of mental illnesses for a time horizon of 30 years.

RESULTS

The number of cases of anxiety was estimated to fall by 5,125 or 9,592 and those of depression by 1,269 and 2,165 if the effect of the intervention lasted 2 or 5 years respectively. From a healthcare system perspective, the intervention was cost-effective for all cases considered with incremental cost-utility ratios always lower than €10,000/QALY and dominant for some subgroups. The intervention was always dominant when including indirect and non-medical costs (societal perspective).

CONCLUSIONS

Although the primary analysis of the trial did not did not detect significant differences, the UPRIGHT intervention promoting positive mental health was dominant in the economic evaluation from the societal perspective. Promoting resilience was more cost-effective in the most deprived group. Despite a lack of information about the spillover effect in some sectors, the economic evaluation framework developed principally for pharmacoeconomics can be applied to interventions to promote resilience in adolescents. As prevention of mental health disorders is even more necessary in the post-coronavirus disease-19 era, such evaluation is essential to assess whether investment in mental health promotion would be good value for money by avoiding costs for healthcare providers and other stakeholders.

What, Where, and How to Collect Real-World Data and Generate Real-World Evidence to Support Drug Reimbursement Decision-Making in Asia: A reflection Into the Past and A Way Forward

BACKGROUND

Globally, there is increasing interest in the use of real-world data (RWD) and real-world evidence (RWE) to inform health technology assessment (HTA) and reimbursement decision-making. Using current practices and case studies shared by eleven health systems in Asia, a non-binding guidance that seeks to align practices for generating and using RWD/RWE for decision-making in Asia was developed by the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) Working Group, addressing a current gap and needs among HTA users and generators.

METHODS

The guidance document was developed over two face-to-face workshops, in addition to an online survey, a face-to-face interview and pragmatic search of literature. The specific focus was on what, where and how to collect RWD/ RWE.

RESULTS

All 11 REALISE member jurisdictions participated in the online survey and the first in-person workshop, 10 participated in the second in-person workshop, and 8 participated in the in-depth face-to-face interviews. The guidance document was iteratively reviewed by all working group members and the International Advisory Panel. There was substantial variation in: (a) sources and types of RWD being used in HTA, and (b) the relative importance and prioritization of RWE being used for policy-making. A list of national-level databases and other sources of RWD available in each country was compiled. A list of useful guidance on data collection, quality assurance and study design were also compiled.

CONCLUSION

The REALISE guidance document serves to align the collection of better quality RWD and generation of reliable RWE to ultimately inform HTA in Asia.

Follicle-stimulating hormone effectiveness in male idiopathic infertility: What happens in daily practice?

OBJECTIVES

To assess the effectiveness of follicle-stimulating hormone (FSH) administration in male idiopathic infertility in a clinical setting.

METHODS

A retrospective real-world study was carried out, including all consecutive FSH-treated infertile men attending the Andrology Unit of Modena (Italy) from June 2015 to May 2022. Medical history, physical and andrological examinations, hormonal and seminal parameters, therapeutic management and pregnancy data were collected. The primary endpoint was the number of pregnancies obtained after FSH administration, whereas semen parameters change was the secondary outcome.

RESULTS

A total of 194 of 362 (53.6%) infertile men, eligible according to the Italian Health System regulations, were treated with FSH (mean age 37.9 ± 6.1 years). Following FSH administration (mean therapy duration 9.1 ± 7.1 months), 43 pregnancies were recorded (27.6%), of which 22 occurred naturally and 21 after assisted reproduction. A significant increase in sperm concentration (9.9 ± 12.2 vs. 18.9 ± 38.9 million/mL, p = 0.045) was detected after treatment, together with a significant increase in normozoospermia (from 1.0% to 5.1%, p = .044) and a reduction in azoospermia rate (from 9.8% to 7%, p = 0.044). Dividing the cohort in FSH-responders and non-responders, in terms of pregnancy achieved, higher sperm concentrations (15.7 ± 26.6 vs. 22.2 ± 25.7 million/mL, p = 0.033) and progressive sperm motility (18.0 ± 18.2 vs. 27.3 ± 11.3, p = 0.044) were found in pregnancy group.

CONCLUSION

Our experience suggests that FSH, empirically administered to men with idiopathic infertility, leads to pregnancy in one out of four patients and increases sperm concentration. Although the expected limits because of a real-world data study, the number of FSH-treated patients required to achieve one pregnancy seems to be lower in clinical setting if compared to previously published data.

Worldwide Trends in Registering Real-World Studies at ClinicalTrials.gov: A Cross-Sectional Analysis

Objective

The purpose of this study was to characterize real-world studies (RWSs) registered at ClinicalTrials.gov to help investigators better conduct relevant research in clinical practice.

Methods

A retrospective analysis of 944 studies was performed on February 28, 2023.

Results

A total of 944 studies were included. The included studies involved a total of 48 countries. China was the leading country in terms of the total number of registered studies (37.9%, 358), followed by the United States (19.7%, 186). Regarding intervention type, 42.4% (400) of the studies involved drugs, and only 9.1% (86) of the studies involved devices. Only 8.5% (80) of the studies mentioned both the detailed study design type and data source in the "Brief Summary". A total of 49.4% (466) of studies had a sample size of 500 participants and above. Overall, 63% (595) of the studies were single-center studies. A total of 213 conditions were covered in the included studies. One-third of the studies (32.7%, 309) involved neoplasms (or tumors). China and the United States were very different regarding the study of different conditions.

Conclusion

Although the pandemic has provided new opportunities for RWSs, the rigor of scientific research still needs to be emphasized. Special attention needs to be given to the correct and comprehensive description of the study design in the Brief Summary of registered studies, thereby promoting communication and understanding. In addition, deficiencies in ClinicalTrials.gov registration data remain prominent.

Clinical trial data-sharing policies among journals, funding agencies, foundations, and other professional organizations: a scoping review

BACKGROUND AND OBJECTIVES

To identify the similarities and differences in data-sharing policies for clinical trial data that are endorsed by biomedical journals, funding agencies, and other professional organizations. Additionally, to determine the beliefs, and opinions regarding data-sharing policies for clinical trials discussed in articles published in biomedical journals.

METHODS

Two searches were conducted, a bibliographic search for published articles that present beliefs, opinions, similarities, and differences regarding policies governing the sharing of clinical trial data. The second search analyzed the gray literature (non-peer-reviewed publications) to identify important data-sharing policies in selected biomedical journals, foundations, funding agencies, and other professional organizations.

RESULTS

A total of 471 articles were included after database search and screening, with 45 from the bibliographic search and 426 from the gray literature search. A total of 424 data-sharing policies were included. Fourteen of the 45 published articles from the bibliographic search (31.1%) discussed only advantages specific to data-sharing policies, 27 (27/45; 60%) discussed both advantages and disadvantages, and 4 (4/45; 8.9%) discussed only disadvantages specific. A total of 216 journals (of 270; 80%) specified a data-sharing policy provided by the journal itself. One hundred industry data-sharing policies were included, and 32 (32%) referenced a data-sharing policy on their website. One hundred and thirty-six (42%) organizations (of 327) specified a data-sharing policy.

CONCLUSION

We found many similarities listed as advantages to data-sharing and fewer disadvantages were discussed within the literature. Additionally, we found a wide variety of commonalities and differences-such as the lack of standardization between policies, and inadequately addressed details regarding the accessibility of research data-that exist in data-sharing policies endorsed by biomedical journals, funding agencies, and other professional organizations. Our study may not include information on all data sharing policies and our data is limited to the entities' descriptions of each policy.

Recommendations to overcome barriers to the use of artificial intelligence-driven evidence in health technology assessment

Background

Artificial intelligence (AI) has attracted much attention because of its enormous potential in healthcare, but uptake has been slow. There are substantial barriers that challenge health technology assessment (HTA) professionals to use AI-generated evidence for decision-making from large real-world databases (e.g., based on claims data). As part of the European Commission-funded HTx H2020 (Next Generation Health Technology Assessment) project, we aimed to put forward recommendations to support healthcare decision-makers in integrating AI into the HTA processes. The barriers, addressed by the paper, are particularly focusing on Central and Eastern European (CEE) countries, where the implementation of HTA and access to health databases lag behind Western European countries.

Methods

We constructed a survey to rank the barriers to using AI for HTA purposes, completed by respondents from CEE jurisdictions with expertise in HTA. Using the results, two members of the HTx consortium from CEE developed recommendations on the most critical barriers. Then these recommendations were discussed in a workshop by a wider group of experts, including HTA and reimbursement decision-makers from both CEE countries and Western European countries, and summarized in a consensus report.

Results

Recommendations have been developed to address the top 15 barriers in areas of (1) human factor-related barriers, focusing on educating HTA doers and users, establishing collaborations and best practice sharing; (2) regulatory and policy-related barriers, proposing increasing awareness and political commitment and improving the management of sensitive information for AI use; (3) data-related barriers, suggesting enhancing standardization and collaboration with data networks, managing missing and unstructured data, using analytical and statistical approaches to address bias, using quality assessment tools and quality standards, improving reporting, and developing better conditions for the use of data; and (4) technological barriers, suggesting sustainable development of AI infrastructure.

Conclusion

In the field of HTA, the great potential of AI to support evidence generation and evaluation has not yet been sufficiently explored and realized. Raising awareness of the intended and unintended consequences of AI-based methods and encouraging political commitment from policymakers is necessary to upgrade the regulatory and infrastructural environment and knowledge base required to integrate AI into HTA-based decision-making processes better.

Developing Patient-Centered Real-World Evidence: Emerging Methods Recommendations From a Consensus Process

OBJECTIVES

The Joint ISPOR-ISPE Special Task Force on Real-World Evidence included patient/stakeholder engagement as a recommended good procedural practice when designing, conducting, and disseminating real-world evidence (RWE). However, there are no guidelines describing how patient experience data (PED) can be applied when designing real-world data (RWD) studies. This article describes development of consensus recommendations to guide researchers in applying PED to develop patient-centered RWE.

METHODS

A multidisciplinary advisory board, identified through recommendations of collaborators, was established to guide development of recommendations. Semistructured interviews were conducted to identify how experienced RWD researchers (n = 15) would apply PED when designing a hypothetical RWD study. Transcripts were analyzed and emerging themes developed into preliminary methods recommendations. An eDelphi survey (n = 26) was conducted to refine/develop consensus on the draft recommendations.

RESULTS

We identified 13 recommendations for incorporating PED throughout the design, conduct, and translation of RWE. The recommendations encompass themes related to the development of a patient-centered research question, designing a study, disseminating RWE, and general considerations. For example, consider how patient input can inform population/subgroups, comparators, and study period. Researchers can leverage existing information describing PED and may be able to apply those insights to studies relying on traditional RWD sources and/or patient registries.

CONCLUSIONS

Applying these emerging recommendations may improve the patient centricity of RWE through improved relevance of RWE to patient communities of interest and foster greater multidisciplinary participation and transparency in RWD research. As researchers gather experience by applying the methods recommendations, further refinement of these consensus recommendations may lead to "best practices."

Which sperm parameter limits could really guide the clinical decision in assisted reproduction?

BACKGROUND

The predictive role of sperm motility and morphology was recently detected in a large sample of more than 20000 assisted reproductive technology (ART) fresh cycles. However, the complete ART procedure consisted of both fresh and frozen-embryos transfers and only a comprehensive evaluation of the entire process could really confirm if these parameters really predict the ART success. The aim of the study was to evaluate which sperm parameter could predict the success of ART.

METHODS

A retrospective, real-world data analysis was performed, enrolling all couples attending ART from 2008 to 2021, including both fresh and frozen cycles and both in vitro fertilization (IVF) and intra-cytoplasmic sperm injection (ICSI) procedures.

RESULTS

Fresh cycles success (considering live birth rate) was predicted by female age (1.04 [1.02-1.06]), injected oocytes (0.96 [0.93-0.99]), embryo number (0.79 [0.75-0.83]) and progressive sperm motility (0.98 [0.97-0.99]). On the contrary, frozen cycle outcomes were predicted only by sperm motility (0.97 [0.95-0.99]). This prediction was confirmed in IVF but not in ICSI cycles.

CONCLUSION

Both female and male parameters predicted the ART success considering the entire path. However, frozen cycle success was predicted only by progressive sperm motility in IVF cycles, suggesting that the potential amelioration of this male parameter is relevant to improve ART success. Those couples expected to obtain the highest embryos after fertilization (low female age and better semen parameters) will have more attempts with frozen cycles and thus would benefit from a potential treatment focused to improve sperm parameters.

Real-world evidence analysis of the follicle-stimulating hormone use in male idiopathic infertility

Male idiopathic infertility remains a therapeutic challenge in the couple infertility management. In this setting, an empirical treatment with follicle-stimulating hormone (FSH) is allowed, although not recommended. Twenty-one clinical trials and four meta-analyses highlighted an overall increased pregnancy rate in case of FSH administration, but the indiscriminate FSH prescription is still unsupported by clinical evidence in idiopathic infertility. This context could represent an example in which real-world data (RWD) could add useful information. From a nationwide clinical practice survey performed in Italy, emerged the clinicians' attitude to prescribe FSH in the case of impaired semen with a significant improvement of semen parameters, identifying FSH treatment as a therapeutic card in the real-life management. Although more robust data are still needed to optimize FSH treatment in male idiopathic infertility, RWD should be included in the body of evidence considered in healthcare decision-making.

Real-world data: a brief review of the methods, applications, challenges and opportunities

Background

The increased adoption of the internet, social media, wearable devices, e-health services, and other technology-driven services in medicine and healthcare has led to the rapid generation of various types of digital data, providing a valuable data source beyond the confines of traditional clinical trials, epidemiological studies, and lab-based experiments.

Methods

We provide a brief overview on the type and sources of real-world data and the common models and approaches to utilize and analyze real-world data. We discuss the challenges and opportunities of using real-world data for evidence-based decision making This review does not aim to be comprehensive or cover all aspects of the intriguing topic on RWD (from both the research and practical perspectives) but serves as a primer and provides useful sources for readers who interested in this topic.

Results and Conclusions

Real-world hold great potential for generating real-world evidence for designing and conducting confirmatory trials and answering questions that may not be addressed otherwise. The voluminosity and complexity of real-world data also call for development of more appropriate, sophisticated, and innovative data processing and analysis techniques while maintaining scientific rigor in research findings, and attentions to data ethics to harness the power of real-world data.

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting

Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.

Value-based evidence across health care sectors: a push for transparent real-world studies, data, and evidence dissemination

There is currently a heightened need for transparency in pharmaceutical sectors. The inclusion of real-world (RW) evidence, in addition to clinical trial evidence, in decision-making processes, was an important step forward toward a more inclusive established value proposition. This advance has introduced new transparency challenges. Increasing transparency is a critical step toward accelerating improvement in type, quality, and access to data, regardless of whether these originate from clinical trials or from RW studies. However, so far, advances in transparency have been relatively restricted to clinical trials, and there remains a lack of similar expectations or standards of transparency concerning the generation and reporting of RW data. This perspective paper aims to highlight the need for transparency concerning RW studies, data, and evidence across health care sectors, to identify areas for improvement, and provide concrete recommendations and practices for the future. Specific issues are discussed from different stakeholder perspectives, culminating in recommended actions, from individual stakeholder perspectives, for improved RW study, data, and evidence transparency. Furthermore, a list of potential guidelines for consideration by stakeholders is proposed. While recommendations from different stakeholder perspectives are made, true transparency in the processes involved in the generation, reporting, and use of RW evidence will require a concerted effort from all stakeholders across health care sectors.

The challenges and opportunities in using real-world data to drive advances in healthcare in East Asia: expert panel recommendations

OBJECTIVE

To provide recommendations for overcoming the challenges associated with the generation and use of real-world evidence (RWE) in regulatory approvals, health technology assessments (HTAs), and reimbursement decision-making in East Asia.

METHODS

A panel of experts convened at the International Society for Pharmacoeconomics and Outcomes Research Asia Pacific 2020 congress to discuss the challenges limiting the use of RWE in healthcare decision-making and to provide insights into the perspectives of regulators, HTA agencies, the pharmaceutical industry, and physicians in China, Japan, and Taiwan. A nonsystematic literature review was conducted to expand on the themes addressed.

RESULTS

The use of RWE in regulatory approvals, HTAs, and reimbursement decision-making remains limited by legal/regulatory, technical, and attitudinal challenges in East Asia.

CONCLUSIONS

We recommend approaches and initiatives that aim to drive improvements in the utilization of RWE in healthcare decision-making in East Asia and other regions. We encourage large-scale collaborations that leverage the full range of skills offered by different stakeholders. Government agencies, hospitals, research organizations, patient groups, and the pharmaceutical industry must collaborate to ensure appropriate access to robust and reliable real-world data and seek alignment on how to address prioritized evidence needs. Increasingly, we believe that this work will be conducted by multidisciplinary teams with expertise in healthcare research and delivery, data science, and information technology. We hope this work will encourage further discussion among all stakeholders seeking to shape the RWE landscape in East Asia and other regions and drive next-generation healthcare.

Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions

Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson’s correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratio_original/hazard ratio_reproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.

Analyses of real-world evidence from digital clinical practice data provide important insights for healthcare decision makers. Here, authors test reproducibility of 150 peer-reviewed studies, reporting strong reproducibility, which could be further improved through more complete reporting in future original studies

Using real-world evidence in healthcare from Western to Central and Eastern Europe: a review of existing barriers

As part of the HTx (Next Generation Health Technology Assessment) project, this study was aimed at identifying the main barriers for application of real-world evidence (RWE) for the purposes of health technology assessment in the Central and Eastern European countries. A mixed methods approach was employed to identify the main barriers: a scoping review of the literature and a series of discussions with stakeholders. Based on the applied approaches, we attempted to summarize the main barriers and challenges related to transferability of RWE in five main groups: technical, regulatory, clinical, scientific and perceptional barriers. Further research should pursue the development of detailed, consensus-based guidelines to improve the harmonization and standardization of RWE.

Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force

Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.

Baseline characteristics and secondary medication adherence among Medicare patients diagnosed with transthyretin amyloid cardiomyopathy and/or receiving tafamidis prescriptions: A retrospective analysis of a Medicare cohort

BACKGROUND: Transthyretin amyloid cardiomyopathy (ATTR-CM) is an underdiagnosed, life-threatening condition that mostly affects older persons. In May 2019, regulatory approval of tafamidis provided the first pharmacologic treatment of ATTR-CM. In the pivotal phase 3 Transthyretin Amyloidosis Cardiomyopathy Clinical Trial (ATTR-ACT), 97.2% of patients were classified as adherent (defined as taking ≥ 80% of scheduled doses). Given its recent approval, there is limited real-world evidence examining patient adherence to tafamidis. OBJECTIVE: To evaluate adherence patterns, demographics, and clinical characteristics of patients in the United States receiving tafamidis prescriptions through Medicare. Secondarily, we aimed to evaluate concomitant medications filled by this patient population. METHODS: We conducted a retrospective cohort study of US Medicare claims data, limited by the Health Insurance Portability and Accountability Act of 1996, in adult patients with an adjudicated pharmacy claim for tafamidis (tafamidis free acid 61-mg capsule once daily or tafamidis meglumine four 20-mg capsules once daily) between May 1, 2019, and June 30, 2021. Gaps in therapy were measured using day gaps between prescription refills and continuous measure of medication gaps. Implementation adherence was assessed through modified medication possession ratio (MPRm), medication refill adherence (MRA), and proportion of days covered (PDC). Patients were grouped based on Medicare coverage. Patients were analyzed by subgroups based on age and at the zip code level, via distressed communities index quartiles and rural-urban tiers. RESULTS: A total of 3,558 patients who received a prescription fill of a tafamidis formulation were identified using Medicare Fee-for-Service (FFS) and Medicare Advantage (MA) claims data from May 1, 2019, to June 30, 2021. The characteristics of this patient population were consistent with published literature, as 98.6% were older than 65 years, 53.4% were between 75 years and 84 years, and 81.5% were male. In the patient population receiving tafamidis refills, adherence was high across all 3 measures, with mean MPRm greater than 90% and mean MRA greater than 80%, across all age groups. Mean PDC adherence rates were 79% or more across all age groups. Concomitant medications were generally indicated for heart failure and thrombosis. Among monotherapy groups with similar demographic makeup, adherence was significantly higher among users of tafamidis free acid vs tafamidis meglumine (P < 0.0001 across all mean adherence measures). CONCLUSIONS: Our results demonstrate that real-world adherence to tafamidis in the Medicare population is high, regardless of age, zip code-level socioeconomic quartile, or geography. Adherence was higher among patients receiving tafamidis free acid, suggesting that the enhanced convenience of a single capsule once daily may positively contribute to adherence among patients with ATTR-CM. DISCLOSURES: Darrin Benjumea is an employee of Genesis Research who has been contracted by Pfizer, Inc., for involvement in this study. Andrew Peterson is an employee of University of the Sciences who has been contracted by Pfizer, Inc., for involvement in this study. Zach Bredl is an employee of Care Journey who has been contracted by Pfizer, Inc., for involvement in this study. Anuja Roy, Nick Marchant, Jose Alvir, Rahul Bhambri, Jason Kemner, and Bhash Parasuraman are employees of Pfizer, Inc., and own stock and/or stock options. This study was supported by Pfizer, Inc.

Methodological considerations on estimating medication adherence from self-report, electronic monitoring, and electronic healthcare databases using the TEOS framework

AIM

Measuring adherence to medication is complex due to the diversity of contexts in which medications are prescribed, dispensed, and used. The Timelines-Events-Objectives-Sources (TEOS) framework outlined a process to operationalize adherence. We aimed to develop practical recommendations for quantification of medication adherence using self-report (SR), electronic monitoring (EM), and electronic healthcare databases (EHD) consistent with the TEOS framework for adherence operationalization.

METHODS

An adherence methodology working group of the International Society for Medication Adherence (ESPACOMP) analysed implications of the process of medication adherence for all data sources and discussed considerations specific to SR, ED, and EHD regarding the information available on the prescribing, dispensing, recommended and actual use timelines, the four events relevant for distinguishing the adherence phases, the study objectives commonly addressed with each type of data, and the potential sources of measurement error and quality criteria applicable.

RESULTS

Four key implications for medication adherence measurement are common to all data sources: adherence is a comparison between two series of events (recommended and actual use); it refers to one or more specific medication(s); it applies to regular repeated events coinciding with known recommended dosing; and it requires separate measurement of the three adherence phases for a complete picture of patients' adherence. We propose recommendations deriving from these statements, and aspects to be considered in study design when measuring adherence with SR, EM and EHD using the TEOS framework.

CONCLUSION

The quality of medication adherence estimates is the result of several design choices that may optimize the data available.

Personalized Reimbursement Model (PRM) program: A real-world data platform of cancer drugs use to improve and personalize drug pricing and reimbursement in France

Objective

This article describes the Personalized Reimbursement Model (PRM) program methodology, limitations, achievement and perspectives in using real-world data of cancer drugs use to improve and personalize drug pricing and reimbursement in France.

Materials and methods

PRM platform aggregates Electronic Pharmacy Records (EPR) data from French medical centers (PRM centers) to build retrospective cohorts of patients treated with injectable cancer drugs in a hospital setting. Data extracted on January 1^st, 2020, from breast cancer (BC) patients who received trastuzumab, trastuzumab emtansin or pertuzumab since January 1st, 2011, and from lung cancer (LC) patients who received bevacizumab or atezolizumab since January 1^st, 2015, enabled recovering their injectable cancer drugs history from diagnosis date until December 30^th, 2019, and served as dataset for assessment.

Results

123 PRM centers provided data from 30,730 patients (25,660 BC and 5,070 LC patients respectively). Overall, 20,942 (82%) of BC and 4,716 (93%) of LC patients were analyzed. Completion rate was above 98% for patients characteristics, diagnostic and treatment related data. PRM centers cover 48% and 33% of BC and LC patients in-hospital therapeutic management in France, respectively. Distribution of BC and LC patients therapeutic management, by medical center category and geographic location, was similar in PRM centers to all French medical centers, ensuring the representativeness of the PRM platform.

Conclusion

PRM Platform enabled building a national database generating on demand Real-World Evidence based on EPR. This enabled the first performance-based risk-sharing arrangements based on PRM data, between the CEPS and Roche, for atezolizumab cancer immunotherapy in metastatic non-small cell lung cancer indication.

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations

OBJECTIVE

To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.

METHODS

We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments.

RESULTS

Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments.

DISCUSSION

These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators.

CONCLUSION

These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.

A review of stakeholder recommendations for defining fit-for-purpose real-world evidence algorithms

Aim: The credibility and value of real-world evidence (RWE) are either supported or undermined by the algorithms (i.e., operational definitions) used. Methods: We conducted a targeted evidence review of key RWE decision makers' published recommendations on RWE algorithms through April 2021. Stakeholders were regulatory bodies, other governmental agencies and payer organizations. Results: Our review identified recommended criteria: relevance, validity, reliability, responsiveness, transparency and replicability, safety, feasibility and quality process. Stakeholders routinely recommended accuracy measures, subgroups evaluation and specific considerations for assessing exposures and covariates and the underlying real-world data (RWD) quality. Conclusion: The importance of stakeholder guidance on fit-for-purpose RWE algorithms is growing. We highlight gaps that future guidance and stakeholder recommendations could address.

The value of anticancer drugs - a regulatory view

The high prices of new anticancer drugs and the marginal added benefit perceived by some stakeholders have fuelled a debate on the value of anticancer drugs in the European Union, even though an agreed definition of what constitutes a drug's value does not exist. In this Perspective, we discuss the value of drugs from different viewpoints and objectives of decision makers: for regulators, assessment of the benefit-risk balance of a drug is a cornerstone for approval; payers rely on cost-effectiveness analyses carried out by health technology assessment agencies for reimbursement decisions; for patients, treatment choices are based on personal preferences and attitudes to risk; and clinicians can use several scales (such as the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS)) that have been developed as an attempt to measure value objectively. Although a unique definition that fully captures the concept of value is unlikely to emerge, herein we discuss the importance of understanding different perspectives, and how regulators can help to inform different decision makers.

Assessing Payers' Preferences for Real-World Evidence in the United States: A Discrete Choice Experiment

OBJECTIVES

To rank the US payers' preferences for attributes of real-world evidence (RWE) studies in the context of chronic disease and to quantify trade-offs among them.

METHODS

We conducted a discrete choice experiment in which 180 employees from payer organizations were tasked to choose between 2 RWE studies assuming they were assessing evidence to inform formulary decisions for chronic disease treatment. Each RWE study was characterized by 7 attributes with 3 levels each: very informative, moderately informative, and not measured. We used a D-optimal main-effects design. Survey data were fitted to a conditional logit model to obtain a relative measure of the ranking of importance for each attribute.

RESULTS

Clinical outcomes were the most preferred attribute. It was 4.68 times as important as productivity outcomes-the least preferred attribute. It was followed by health-related quality of life (2.78), methodologic rigor (2.09), resource utilization (1.71), and external validity (1.56).

CONCLUSIONS

This study provides a quantification of the value payers place on key RWE attributes. Across attributes, payers have higher preferences for clinical and health-related quality of life outcomes than the other attributes. Between attributes' levels, payers prefer high levels of information in clinical outcomes and methodologic rigor but are indifferent in other attributes. Our results bridge the gap between the information that payers seek and the attributes that RWE studies prioritize and effectively guide future research design.

Real-world data in drug development strategies for orphan drugs: tafasitamab in B cell lymphoma, a case study for approval based on a single-arm combination trial

Tafasitamab (TAF) plus lenalidomide (LEN) is a novel treatment option for patients with relapsed/refractory diffuse large B cell lymphoma (rrDLBCL) who are not eligible for autologous stem cell transplantation. The initial US/EU approvals for TAF represent precedents because this is the first time that approval of a novel combination therapy was granted based on a pivotal single-arm trial (SAT). Matching real world-data (RWD) helped to disentangle the contribution of individual agents. In this review, we present the TAF development strategy, the prospective incorporation of RWD within the clinical development plan, the corresponding regulatory hurdles of this strategy, and the prior regulatory actions for other cancer drugs that previously incorporated RWD and propensity score matching in EU and US regulatory submissions. We also outline how RWD could further advance and impact orphan drug development.

Real-World Evidence for Regulatory Decision-Making: Guidance From Around the World

PURPOSE

Interest in leveraging real-world evidence (RWE) to support regulatory decision making for product effectiveness has been increasing globally as evident by the increasing number of regulatory frameworks and guidance documents. However, acceptance of RWE, especially before marketing for regulatory approval, differs across countries. In addition, guidance on the design and conduct of innovative clinical trials, such as randomized controlled registry studies, pragmatic trials, and other hybrid studies, is lacking.

METHODS

We assessed the global regulatory environment with regard to RWE based on regional availability of the following 3 key regulatory elements: (1) RWE regulatory framework, (2) data quality and standards guidance. and (3) study methods guidance.

FINDINGS

This article reviews the available frameworks and existing guidance from across the globe and discusses the observed gaps and opportunities for further development and harmonization.

IMPLICATIONS

Cross-country collaborations are encouraged to further shape and align RWE policies and help establish frameworks in countries without current policies with the goal of creating efficiencies when considering RWE to support regulatory decision-making globally.

Real World Data in Health Technology Assessment of Complex Health Technologies

The available evidence on relative effectiveness and risks of new health technologies is often limited at the time of health technology assessment (HTA). Additionally, a wide variety in real-world data (RWD) policies exist among HTA organizations. This study assessed which challenges, related to the increasingly complex nature of new health technologies, make the acceptance of RWD most likely. A questionnaire was disseminated among 33 EUnetHTA member HTA organizations. The questions focused on accepted data sources, circumstances that allowed for RWD acceptance and barriers to acceptance. The questionnaire was validated and tested for reliability by an expert panel, and pilot-tested before dissemination via LimeSurvey. Twenty-two HTA organizations completed the questionnaire (67%). All reported accepting randomized clinical trials. The most accepted RWD source were patient registries (19/22, 86%), the least accepted were editorials and expert opinions (8/22, 36%). With orphan treatments or companion diagnostics, organizations tended to be most likely to accept RWD sources, 4.3–3.2 on a 5-point Likert scale, respectively. Additional circumstances were reported to accept RWD (e.g., a high disease burden). The two most important barriers to accepting RWD were lacking necessary RWD sources and existing policy structures. European HTA organizations seem positive toward the (wider) use of RWD in HTA of complex therapies. Expanding the use of patient registries could be potentially useful, as a large share of the organizations already accepts this source. However, many barriers still exist to the widespread use of RWD. Our results can be used to prioritize circumstances in which RWD might be accepted.

Use of Real-World Evidence to Drive Drug Development Strategy and Inform Clinical Trial Design

Interest in real-world data (RWD) and real-world evidence (RWE) to expedite and enrich the development of new biopharmaceutical products has proliferated in recent years, spurred by the 21st Century Cures Act in the United States and similar policy efforts in other countries, willingness by regulators to consider RWE in their decisions, demands from third-party payers, and growing concerns about the limitations of traditional clinical trials. Although much of the recent literature on RWE has focused on potential regulatory uses (e.g., product approvals in oncology or rare diseases based on single-arm trials with external control arms), this article reviews how biopharmaceutical companies can leverage RWE to inform internal decisions made throughout the product development process. Specifically, this article will review use of RWD to guide pipeline and portfolio strategy; use of novel sources of RWD to inform product development, use of RWD to inform clinical development, use of advanced analytics to harness "big" RWD, and considerations when using RWD to inform internal decisions. Topics discussed will include the use of molecular, clinicogenomic, medical imaging, radiomic, and patient-derived xenograft data to augment traditional sources of RWE, the use of RWD to inform clinical trial eligibility criteria, enrich trial population based on predicted response, select endpoints, estimate sample size, understand disease progression, and enhance diversity of participants, the growing use of data tokenization and advanced analytical techniques based on artificial intelligence in RWE, as well as the importance of data quality and methodological transparency in RWE.

Baseline Characteristics and Secondary Medication Adherence Patterns Among Patients Receiving Tafamidis Prescriptions: A Retrospective Analysis Using a National Specialty Pharmacy Dispensing Database

INTRODUCTION

Transthyretin amyloid cardiomyopathy (ATTR-CM) is a serious, underrecognized condition, which leads to heart failure and early mortality if left untreated. Until recently, heart transplantation was the only treatment for ATTR-CM. Regulatory approval of tafamidis transformed treatment for patients. In the phase 3 Transthyretin Amyloidosis Cardiomyopathy Clinical Trial (ATTR-ACT), which established the safety and efficacy of tafamidis, medication adherence was high with 97.2% of patients taking ≥80% of scheduled doses. Evidence of real-world adherence to cardiology drugs demonstrates low adherence and suboptimal outcomes; however, real-world adherence to tafamidis has not been investigated. The main objective of this study was to describe adherence patterns of patients filling tafamidis in the Symphony Health database.

METHODS

This retrospective analysis of the Symphony Health Solutions claims database used secondary adherence measures, including modified medication possession ratio (MPRm), days between fills adherence rate, and compliance rate, to assess adherence patterns of 2020 patients filling tafamidis free acid 61-mg capsules or tafamidis meglumine 4x20-mg capsules from June 1, 2019 to August 31, 2020.

RESULTS

Patients receiving a tafamidis formulation had characteristics consistent with the expected patient population; 71.6% were aged 75-84 years, 83.2% were male, and the highest proportion resided in the Northeast region (30.5%) of the United States. Adherence for tafamidis was high, as 75% to 100% of the patients across subgroups met or exceeded the commonly defined adherence threshold of 80%. Median number of refills ordered and received was six refills per patient. Most patients received refills with no gap (n=1633) or a gap <30 days (n=1267/1317 patients). Adherence was high across follow-up time, sex, and age subgroups. Adherence varied by geographic region, with the Northeast being significantly higher than the Midwest (mean MPRm 94.41% vs 88.21%, p=0.0007).

CONCLUSION

These results provide evidence that real-world adherence to tafamidis in patients with ATTR-CM is high.

Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?

Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

Global real-world data on hemodiafiltration: An opportunity to complement clinical trial evidence

Hemodiafiltration (HDF) is a renal replacement therapy that utilizes both diffusive clearance and convective transport to achieve greater clearance of middle-molecular-weight solutes. Among other factors, important prerequisites for the implementation of HDF include access to high-flux dialyzers, achievement of high blood flow rates, and availability of high volumes of sterile substitution/replacement fluids. Online hemodiafiltration (OL-HDF) is an established kidney replacement therapy, frequently used in many countries. Although in the United States, some prerequisites (e.g., access to high-flux dialyzers and achievement of high blood flow rates) for OL-HDF treatment are readily available; however, a machine capable of generating the online solution for OL-HDF is currently not available. As the clinical experience with HDF accumulates globally, it is worth examining the evidence for this kidney replacement therapy as used in routine clinical care. Such real-world evidence is increasingly recognized as valuable by clinicians and may inform regulatory decisions. In this review, we will focus on emerging global real-world data derived from routine clinical practices and examine how these data may complement those derived from clinical trials.

Assessing and Interpreting Real-World Evidence Studies: Introductory Points for New Reviewers

Nonrandomized real-world evidence (RWE) studies are conducted using healthcare data collected as part of clinical practice. As RWE studies are increasingly considered for regulatory, coverage, and other clinical decision making, nonspecialists will find themselves in the position of assessing the validity of RWE studies, a field that may be less familiar to them. This introductory guide provides conceptual guidance for reviewing RWE studies and is particularly directed at professionals for whom this is new or whose prior experience has primarily been in reviewing randomized controlled trial evidence. We focus on RWE studies that make causal inference, evaluating whether one treatment option is better, worse, or neutral compared to another. Although we provide citations to direct the reader to resources with more details on complex issues, this guide cannot substitute for years of training and expertise in the field.

The Structured Process to Identify Fit-for-purpose Data (SPIFD): A data feasibility assessment framework

To complement real‐world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real‐world Evidence (SPACE) framework elucidated a process for designing valid and transparent real‐world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments—a step‐by‐step guide to identify decision grade, fit‐for‐purpose data, which complements the United States Food and Drug Administration (FDA)’s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit‐For‐Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit‐for‐purpose data and study design.

Use of RWE to Inform Regulatory, Public Health Policy, and Intervention Priorities for the Developing World

For low‐ and middle‐income countries (LMICs) to benefit from real‐world evidence (RWE)/real‐world data (RWD) in both product registration (“regulatory”) decision making and in product utilization policy (“policy”) decision making, they need to overcome several challenges. They need to deploy more electronic health records systems (EHRs), adjust for confounder variables, build trust between stakeholders, and create laws and regulations for local generation of data that are assented for secondary use. The role of procurers and their use of RWE/RWD in the LMIC context likewise is in a state of ongoing development. Procurers of health products are strong players currently in the “access” chain as LMICs continue to work on strengthening governmental health technology assessment (HTA) bodies. Procurers’ use of RWE is presently at an early stage and is mostly indirect, leveraging RWE results that are produced by researchers in high‐income countries (HICs), often under considerably different regulatory and policy objectives and constraints compared to LMICs’ epidemiology and priorities. Pending wider deployment of EHRs and other RWE sources, stakeholders must realize that populations from HIC RWE (i) can be devised to closely resemble phenotypic patterns in LMIC populations and (ii) can be analyzed to align with LMICs’ unmet needs.