Advancing a Framework for Regulatory Use of Real-World Evidence: When Real Is Reliable

Clinical Impact of "Real World Data" and Blockchain on Public Health: A Scoping Review

The digitisation of healthcare has allowed a significant rethinking of the previous clinical protocols, improving their interoperability through substantial standardisation. These technological advances have ensured that data are comparable, as they are obtained from 'reliable' and certified processes; however, there are billions of data that are neither structured nor quality-controlled. They are collectively referred to as 'Real World Data' (RWD). Blockchain (BC) is a procedure with specific characteristics and algorithms that ensure that the stored data cannot be tampered with. Nowadays, there is an increasing need to rethink blockchain in a one-health vision, making it more than just a 'repository' of data and information, but rather an active player in the process of data sharing. In this landscape, several scholars have analysed the potential benefits of BC in healthcare, focusing on the sharing and safety of clinical data and its contact tracing applications. There is limited research on this matter; moreover, there is a strategic interest in managing RWD in a reliable and comparable way, despite the lack of knowledge on this topic. Our work aims to analyse systematically the most impacting literature, highlighting the main aspects of BC within the context of the new digital healthcare, and speculating on the unexpressed potential of RWD.

Digital therapeutics from bench to bedside

As a new therapeutic technique based on digital technology, the commercialization and clinical application of digital therapeutics (DTx) are increasing, and the demand for expansion to new clinical fields is remarkably high. However, the use of DTx as a general medical component is still ambiguous, and this ambiguity may be owing to a lack of consensus on a definition, in addition to insufficiencies in research and development, clinical trials, standardization of regulatory frameworks, and technological maturity. In this study, we conduct an in-depth investigation and analysis of definitions, clinical trials, commercial products, and the regulatory status related to DTx using published literature, ClinicalTrials.gov, and web pages of regulatory and private organizations in several countries. Subsequently, we suggest the necessity and considerations for international agreements on the definition and characteristics of DTx, focusing on the commercialization characteristics. In addition, we discuss the status and considerations of clinical research, key technology factors, and the direction of regulatory developments. In conclusion, for the successful settlement of DTx, real-world evidence-based validation should be strengthened by establishing a cooperative system between researchers, manufacturers, and governments, and there should be effective technologies and regulatory systems for overcoming engagement barriers of DTx.

The evolution of pharmacovigilance ecosystems: Does Moore's law invite the use of Occam's razor?

AIMS

Moore's law predicts the doubling of complexity of integrated circuits every 2 years; Kryder's corollary assumes a doubling of data storage every 13 months. With the increasing volume of legislation, pharmacovigilance systems today are inherently complex, and the emphasis has shifted from reactive (responding to emerging risks) to planned, active, risk-proportionate approaches operating throughout the life cycle of medicines.

METHODS

Exploration of the drivers for increasing complexity of pharmacovigilance systems, focusing on regulatory environment, data management and evaluation.

RESULTS

Evaluation of postmarketing data plays an increasingly important role in pharmacovigilance. There is great interest on the part of all stakeholders in optimizing the use of these data. Innovative approaches, including pharmacogenetics and passive measures (sensors), will lead to increased complexity and volumes of data and inevitably to an increase in the volume of case reports. There is a multiplicity of regulations and guidelines on how to manage these data, with an inherent lack of harmonization.

CONCLUSION

We summarize the current characterization of safety data types, sources and the classification of these data. Using this benchmark, we discuss the future requirements of an effective pharmacovigilance ecosystem, keeping the principle of parsimony in mind. In this complex, continuously and rapidly changing environment, there is a need for a return to simplicity and pragmatism. The application of Occam's razor could help to support the rapid provision of new, affordable medicines with a positive benefit to risk profile.

Evaluation of real-world outcomes associated with use of a prescription digital therapeutic to treat substance use disorders

BACKGROUND AND OBJECTIVES

Digital therapeutics can expand the reach and fidelity of behavioral treatment for substance use disorders (SUDs). This analysis evaluated real-world engagement and clinical outcomes in patients diagnosed with SUD who were prescribed reSET®, an FDA-authorized prescription digital therapeutic (PDT).

METHODS

Patients were prescribed a 12-week PDT comprising 61 therapy lessons (31 "core" and 30 "keep learning" lessons) and contingency management rewards (positive reinforcement message or monetary gift cards) based on lesson completion and negative urine drug screens. Engagement (defined as any activity in the PDT), retention (any activity in Weeks 9-12), and substance use data were collected automatically by the PDT and analyzed descriptively. Associations between early lesson completion and end-of-treatment outcomes were assessed.

RESULTS

Six hundred and fifty-eight patients filled their prescription. Evaluated were 602 patients who were exposed to therapeutic content by completing at least one lesson (median age 37 years, 33% female, 41% male, 26% unreported sex). Median lessons completed was 33 (out of 61 possible), and 52% of patients completed all core modules. Retention in treatment during the last 4 weeks of treatment was 74%, and 62% were abstinent (missing data considered positive). [Correction added on 13 December 2022, after first online publication: In the preceding sentence, the treatment percentage values were revised from 74.6% to 74%.] DISCUSSION AND CONCLUSIONS: Patients with SUD exhibited robust engagement with a PDT, high rates of retention through 12 weeks, and substantial rates of abstinence at end of treatment when the therapeutic was used in a real-world setting. PDT's hold promise as a new way to access effective SUD treatment.

SCIENTIFIC SIGNIFICANCE

This study is the first to report real-world PDT engagement and clinical outcomes data from a large, geographically diverse population of patients with SUDs.

Developing Patient-Centered Real-World Evidence: Emerging Methods Recommendations From a Consensus Process

OBJECTIVES

The Joint ISPOR-ISPE Special Task Force on Real-World Evidence included patient/stakeholder engagement as a recommended good procedural practice when designing, conducting, and disseminating real-world evidence (RWE). However, there are no guidelines describing how patient experience data (PED) can be applied when designing real-world data (RWD) studies. This article describes development of consensus recommendations to guide researchers in applying PED to develop patient-centered RWE.

METHODS

A multidisciplinary advisory board, identified through recommendations of collaborators, was established to guide development of recommendations. Semistructured interviews were conducted to identify how experienced RWD researchers (n = 15) would apply PED when designing a hypothetical RWD study. Transcripts were analyzed and emerging themes developed into preliminary methods recommendations. An eDelphi survey (n = 26) was conducted to refine/develop consensus on the draft recommendations.

RESULTS

We identified 13 recommendations for incorporating PED throughout the design, conduct, and translation of RWE. The recommendations encompass themes related to the development of a patient-centered research question, designing a study, disseminating RWE, and general considerations. For example, consider how patient input can inform population/subgroups, comparators, and study period. Researchers can leverage existing information describing PED and may be able to apply those insights to studies relying on traditional RWD sources and/or patient registries.

CONCLUSIONS

Applying these emerging recommendations may improve the patient centricity of RWE through improved relevance of RWE to patient communities of interest and foster greater multidisciplinary participation and transparency in RWD research. As researchers gather experience by applying the methods recommendations, further refinement of these consensus recommendations may lead to "best practices."

External Comparator Groups Derived from Real-world Data Used in Support of Regulatory Decision Making: Use Cases and Challenges

Real-world data (RWD) from electronic health records (EHRs) and administrative claims databases are used increasingly to generate real-world evidence (RWE). RWE is used to support clinical evidence packages for medicines that inform decision-makers. In this review of current issues in the use of RWD-derived external comparator groups to support regulatory filings, we assess a series of topics that generally apply across many disease indications. However, most of the examples and illustrations focus on the oncology clinical research setting. The topics include an overview of current uses of RWD in drug development, a discussion of regulatory filings using RWD-derived external comparators, a brief overview of guidance documents and white papers pertaining to external comparators, a summary of some limitations and methodological issues in the use of external comparator groups and finally, a look at the future of this area and recommendations.

Using real-world evidence in healthcare from Western to Central and Eastern Europe: a review of existing barriers

As part of the HTx (Next Generation Health Technology Assessment) project, this study was aimed at identifying the main barriers for application of real-world evidence (RWE) for the purposes of health technology assessment in the Central and Eastern European countries. A mixed methods approach was employed to identify the main barriers: a scoping review of the literature and a series of discussions with stakeholders. Based on the applied approaches, we attempted to summarize the main barriers and challenges related to transferability of RWE in five main groups: technical, regulatory, clinical, scientific and perceptional barriers. Further research should pursue the development of detailed, consensus-based guidelines to improve the harmonization and standardization of RWE.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Real-World Evidence of Traditional Chinese Medicine (TCM) Treatment on Cancer: A Literature-Based Review

While randomized controlled trials (RCTs) are the gold standard for evidence-based medicine, they do not always reflect the real condition of patients in the real-world setting, which limits their generalizability and external validity. Real-world evidence (RWE), generated during routine clinical practice, is increasingly important in determining external effectiveness of the tightly controlled conditions of RCTs and is well recognized as a valuable complement to RCTs by regulatory bodies currently. Since it could provide new ideas and methods for clinical efficacy and safety evaluation of traditional Chinese medicine (TCM) and high-quality evidence support, real-world study (RWS) has received great attention in the field of medicine, especially in the field of TCM. RWS has shown desirable adaptability in the clinical diagnosis and treatment practice of traditional Chinese medicine. Consequently, it is increasingly essential for physicians and researchers to understand how RWE can be used alongside clinical trial data on TCM. Here, we discuss what real-world study is and outline the benefits and limitations of real-world study. Furthermore, using examples from TCM treatment on cancer, including Chinese herbal medicine, acupuncture, moxibustion, integrated TCM and Western medicine treatment, and other treatments, we elaborate how RWE can be used to help inform treatment decisions when doctoring patients with cancer in the clinic.

Real World-Evidence for Regulatory Use Decision Aid: An Interactive Tool To Inform Clinical Development and Regulatory Strategies

Real-world evidence (RWE) is increasingly used to complement clinical trial data for regulatory decision-making and in certain cases utilized to establish the clinical effectiveness of a therapy. However, the use of RWE is not applicable for all regulatory submissions, and it can be challenging to identify appropriate use cases. An interactive tool was developed ("Decision Aid," https://sn.pub/TpDjZx ) to assist researchers, industry, and other stakeholders in identifying regulatory situations that can benefit from leveraging RWE by organizing precedent cases based on a given regulatory objective (new product approval, labeling expansion for new indication or additional clinical data, post-marketing requirement) and type of RWE study design (external control, observational study, pragmatic trial). Key success factors ensuring fit-for-purpose data and rigorous methods (e.g., clear endpoints, minimizing bias, data completeness) are also described. The tool allows the user to navigate through the precedent cases by selecting certain regulatory objectives and/or study designs. The Decision Aid supports regulatory activities in the RWE space and encourages further use of RWE in regulatory decision-making.

Real-World Reductions in Healthcare Resource Utilization over 6 Months in Patients with Substance Use Disorders Treated with a Prescription Digital Therapeutic

BACKGROUND AND AIMS

Substance use disorders (SUDs) affect approximately 40.3 million people in the USA, yet only approximately 19% receive evidence-based treatment each year. reSET^® is a prescription digital therapeutic (PDT) and the only FDA-authorized treatment for patients with cocaine, cannabis, and stimulant use disorders. This study evaluated real-world healthcare resource utilization (HCRU) and associated costs 6 months after initiation of reSET in patients with SUD.

METHODS

A retrospective analysis of HealthVerity PrivateSource20 data compared the 6-month incidence of all-cause hospital facility encounters and clinician services in patients treated with reSET (re-SET cohort) before (pre-index period) and after (post-index period) reSET initiation (index). Incidence was compared using incidence rate ratios (IRR). HCRU-related costs were also assessed.

RESULTS

The sample included 101 patients (median age 37 years, 50.5% female, 54.5% Medicaid-insured). A statistically significant decrease of 50% was observed in overall hospital encounters from pre-index to post-index (IRR 0.50; 95% CI 0.37-0.67; P < 0.001), which included inpatient stays (56% decrease; IRR 0.44; 95% CI 0.26-0.76; P = 0.003), partial hospitalizations (57% decrease; IRR 0.43; 95% CI 0.21-0.88; P = 0.021), and emergency department visits (45% decrease; IRR 0.55; 95% CI 0.38-0.80; P < 0.004). Additionally, some clinician services declined significantly including pathology and laboratory services: other (54% decrease; IRR 0.46; 95% CI 0.28-0.76; P = 0.003); pathology and laboratory services: drug assays prior to opioid medication prescription (37% decrease; IRR 0.63; 95% CI 0.41-0.96; P = 0.031); and alcohol and drug abuse: medication services (46% decrease; IRR 0.54; 95% CI 0.41-0.70; P < 0.001). Reductions in facility-encounters drove 6-month reSET per-patient cost reductions of $3591 post-index compared to pre-index.

CONCLUSIONS

Use of reSET by patients with SUD is associated with durable reductions in HCRU and lower healthcare costs over 6 months compared to the 6 months before PDT treatment, after adjusting for covariates, providing an economic benefit to the healthcare system.

Comparing Findings From a Friends of Cancer Research Exploratory Analysis of Real-World End Points With the Cancer Analysis System in England

PURPOSE

This study compared real-world end points extracted from the Cancer Analysis System (CAS), a national cancer registry with linkage to national mortality and other health care databases in England, with those from diverse US oncology data sources, including electronic health care records, insurance claims, unstructured medical charts, or a combination, that participated in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0. Consistency between data sets and between real-world overall survival (rwOS) was assessed in patients with immunotherapy-treated advanced non-small-cell lung cancer (aNSCLC).

PATIENTS AND METHODS

Patients with aNSCLC, diagnosed between January 2013 and December 2017, who initiated treatment with approved programmed death ligand-1 (PD-[L]1) inhibitors until March 2018 were included. Real-world end points, including rwOS and real-world time to treatment discontinuation (rwTTD), were assessed using Kaplan-Meier analysis. A synthetic data set, Simulacrum, on the basis of conditional random sampling of the CAS data was used to develop and refine analysis scripts while protecting patient privacy.

RESULTS

Characteristics (age, sex, and histology) of the 2,035 patients with immunotherapy-treated aNSCLC included in the CAS study were broadly comparable with US data sets. In CAS, a higher proportion (46.7%) of patients received a PD-(L)1 inhibitor in the first line than in US data sets (18%-30%). Median rwOS (11.4 months; 95% CI, 10.4 to 12.7) and rwTTD (4.9 months; 95% CI, 4.7 to 5.1) were within the range of US-based data sets (rwOS, 8.6-13.5 months; rwTTD, 3.2-7.0 months).

CONCLUSION

The CAS findings were consistent with those from US-based oncology data sets. Such consistency is important for regulatory decision making. Differences observed between data sets may be explained by variation in health care settings, such as the timing of PD-(L)1 approval and reimbursement, and data capture.

Real-world changes in US health system hospital-based services following treatment with a prescription digital therapeutic for opioid use disorder

Outcomes associated with buprenorphine therapy for the treatment of opioid use disorder (OUD) are suboptimal. reSET-O is an FDA-authorized prescription digital therapeutic (PDT) delivering neurobehavioral therapy via mobile devices to patients with OUD treated with buprenorphine. This analysis evaluated the net impact of reSET-O on medical costs among actively-engaged reSET-O patients using real-world observations. This real-world retrospective analysis of health care claims between October 2018 and October 2019 evaluated health care resource utilization up to 6 months before and 6 months after the initiation of a reSET-O prescription after accounting for the subset of patients not continuing on therapy after week 1 (non-engaged patients). Repeated-measures negative binomial models compared incidences of hospital-based encounters/procedures adjusted for days in each period as well as associated costs. The number needed to treat (NNT) to avoid an inpatient visit was calculated. Of the 351 patients who were prescribed reSET-O, 321 met the criteria of active engagement. Treatment with reSET-O was associated with a substantial reduction in medical costs of -$765,450 (-$2,385/patient, $235/patient greater than a previous analysis in which non-engaged patients were included) in the 6-month period after initiation. The gross reSET-O prescription cost of $584,415 ($1,665/patient) was substantially offset by $49,950 ($142.31/patient) in refunds to payers. The medical cost reduction in engaged patients offset the cost of the therapeutic resulting in an overall cost reduction of -$230,985 in this cohort (net savings of -$720 per patient). The number needed to treat to avoid an inpatient visit was 4.8. Engagement and continued treatment with reSET-O in patients with OUD treated with buprenorphine is associated with substantial real-world reductions in medical costs in the 6-month period following the initiation of the reSET-O prescription.

Uncontrolled Extensions of Clinical Trials and the Use of External Controls-Scoping Opportunities and Methods

Increased interest in real-world evidence (RWE) for clinical and regulatory decision making and the need to evaluate long-term benefits and risks of pharmaceutical products raise the importance of understanding the use of external controls (ECs) for uncontrolled extensions of randomized controlled trials (RCTs). We searched clinicaltrials.gov from 2009 to 2019 for uncontrolled extensions and assessed the use of ECs in the trial protocol registry and PubMed. We present characteristics of identified uncontrolled extensions, their adoption of ECs, and a qualitative appraisal of published uncontrolled extensions with ECs according to good pharmacoepidemiologic practice. The number of uncontrolled extensions increased slightly across the study period, resulting in a total of 1,115 studies. Most originated from phase III RCTs (62.2%) and specified safety outcomes (61.9% among those with specified outcomes). Most uncontrolled extensions incorporated no control group with only 7 out of 1,115 (0.6%) employing ECs. For those studies with ECs, all involved treatments for rare conditions and assessment of effectiveness. Attempts to balance comparison groups varied from none mentioned to propensity score matching. We noted consistent deficiencies in outcome ascertainment methods and approaches to address attrition bias. The contrast of the large and growing number of uncontrolled extensions with the small number of studies that utilized ECs showed clear opportunities for enhancement in design, measurement, and analysis of uncontrolled extensions to allow causal inferences on long-term treatment effects. As extensions continue to expand within RWE regulatory frameworks, development of guidelines for use of EC with uncontrolled extensions is needed.

Characterization and selection of Japanese electronic health record databases used as data sources for non-interventional observational studies

BACKGROUND

Interventional studies are the fundamental method for obtaining answers to clinical questions. However, these studies are sometimes difficult to conduct because of insufficient financial or human resources or the rarity of the disease in question. One means of addressing these issues is to conduct a non-interventional observational study using electronic health record (EHR) databases as the data source, although how best to evaluate the suitability of an EHR database when planning a study remains to be clarified. The aim of the present study is to identify and characterize the data sources that have been used for conducting non-interventional observational studies in Japan and propose a flow diagram to help researchers determine the most appropriate EHR database for their study goals.

METHODS

We compiled a list of published articles reporting observational studies conducted in Japan by searching PubMed for relevant articles published in the last 3 years and by searching database providers' publication lists related to studies using their databases. For each article, we reviewed the abstract and/or full text to obtain information about data source, target disease or therapeutic area, number of patients, and study design (prospective or retrospective). We then characterized the identified EHR databases.

RESULTS

In Japan, non-interventional observational studies have been mostly conducted using data stored locally at individual medical institutions (663/1511) or collected from several collaborating medical institutions (315/1511). Whereas the studies conducted with large-scale integrated databases (330/1511) were mostly retrospective (73.6%), 27.5% of the single-center studies, 47.6% of the multi-center studies, and 73.7% of the post-marketing surveillance studies, identified in the present study, were conducted prospectively. We used our findings to develop an assessment flow diagram to assist researchers in evaluating and choosing the most suitable EHR database for their study goals.

CONCLUSIONS

Our analysis revealed that the non-interventional observational studies were conducted using data stored local at individual medical institutions or collected from collaborating medical institutions in Japan. Disease registries, disease databases, and large-scale databases would enable researchers to conduct studies with large sample sizes to provide robust data from which strong inferences could be drawn.

Understanding Use of Real-World Data and Real-World Evidence to Support Regulatory Decisions on Medical Product Effectiveness

RWE has potential to provide efficient and relevant information on the effectiveness of medical products, complementing the data generated in clinical trials; however, how RWE can support regulatory decision-making is unclear, potentially limiting its use. The objective of this study was to identify and characterize instances where RWE was included in the evidence package to support the effectiveness of a medical product regulated by U.S. Food and Drug Administration. A retrospective landscape analysis was conducted to identify instances where RWE was submitted to support effectiveness through targeted review of white and gray literature and publicly available FDA reviews of medical products. Trained evaluators examined FDA reviews to determine if and how RWE contributed to regulatory decision-making regarding effectiveness. Evaluators identified 34 instances of RWE submitted between 1954 and 2020, where 26% of instances were for oncology, 18% for hematology, and 12% for neurology. Over 50% of the products were indicated for use in rare disease or pediatric populations. 82% of products where RWE was submitted received an orphan designation. RWE was included in the product label in 59% of instances. Stated reasons indicating why submitted RWE did not significantly contribute to regulatory decision-making included lack of pre-specification of study design and analysis as well as data reliability and relevancy concerns. While there is historical use of RWE to support medical product effectiveness for oncology and rare diseases, potential exists to leverage the strengths of RWE to support other therapeutic areas and capture outcomes that are most relevant to patients.

Real-World Evidence: A Review of Real-World Data Sources Used in Orthopaedic Research

SUMMARY

Real-world data (RWD) play an increasingly important role in orthopaedics as demonstrated by the rapidly growing number of publications using registry, administrative, and other databases. Each type of RWD source has its strengths and weaknesses, as does each specific database. Linkages between real-world data sets provide even greater utility and value for research than single data sources. The unique qualities of an RWD data source and all data linkages should be considered before use. Close attention to data quality and use of appropriate analysis methods can help alleviate concerns about validity of orthopaedic studies using RWD. This article describes the main types of RWD used in orthopaedics and provides brief descriptions and a sample listing of publications from selected, key data sources.

Real-World Evidence: A Primer

SUMMARY

The insights that real-world data (RWD) can provide, beyond what can be learned within the traditional clinical trial setting, have gained enormous traction in recent years. RWD, which are increasingly available and accessible, can further our understanding of disease, disease progression, and safety and effectiveness of treatments with the speed and accuracy required by the health care environment and patients today. Over the decades since RWD were first recognized, innovation has evolved to take real-world research beyond finding ways to identify, store, and analyze large volumes of data. The research community has developed strong methods to address challenges of using RWD and as a result has increased the acceptance of RWD in research, practice, and policy. Historic concerns about RWD relate to data quality, privacy, and transparency; however, new tools, methods, and approaches mitigate these challenges and expand the utility of RWD to new applications. Specific guidelines for RWD use have been developed and published by numerous groups, including regulatory authorities. These and other efforts have shown that the more RWD are used and understood and the more the tools for handling it are refined, the more useful it will be.

Real-world evidence for a prescription digital therapeutic to treat opioid use disorder

OBJECTIVES

To evaluate patient engagement and usage of a prescription digital therapeutic (PDT) and associated outcomes of opioid use and treatment retention in a large real-world dataset of patients with opioid use disorder (OUD) treated with buprenorphine medication for opioid use disorder (MOUD). PDTs are software-based disease treatments evaluated for safety and effectiveness in randomized clinical trials (RCTs), and authorized by the U.S. Food and Drug Administration (FDA) to treat disease with approved directions for use (label).

METHODS

A real-world observational evaluation of an all-comer population of patients who redeemed a 12-week prescription for the reSET-O PDT. Engagement and therapeutic use data were collected and analysed on a population level. Substance use was evaluated as a composite of self-reports recorded with reSET-O and urine drug screens (UDS).

RESULTS

Data from 3144 individuals with OUD were evaluated. 45.5% were between ages 30 and 39 years. 80% completed at least 8 of the 67 possible therapeutic modules, 66% completed half of all modules, and 49% completed all modules. Abstinence during the last 4 weeks of treatment was calculated with two imputation methodologies: 66% abstinent using "missing data excluded (patients with no data as positive)", and 91% abstinent with "missing data removed (patients with no data excluded)". 91% of patients met the responder definition of ≥80% of self-report or UDS negative. 74.2% of patients were retained through the last 4 weeks of treatment. Subgroup analysis of patients using reSET-O appropriately (4 or more modules per week for the first 4 weeks) showed 88.1% abstinence using "missing data excluded (patients with no data as positive)", and retention at weeks 9-12 of 85.8%.

CONCLUSIONS

Results demonstrate that reSET-O is readily and broadly used by patients with OUD and that high real-world engagement with the therapeutic is positively associated with abstinence and retention in treatment. ReSET-O is a potentially valuable adjunct to buprenorphine MOUD therapy for patients with OUD.

Real-World Evidence to Contextualize Clinical Trial Results and Inform Regulatory Decisions: Tofacitinib Modified-Release Once-Daily vs Immediate-Release Twice-Daily for Rheumatoid Arthritis

INTRODUCTION

Tofacitinib is an oral Janus kinase inhibitor for the treatment of rheumatoid arthritis (RA). To provide additional clinical evidence in regulatory submissions for a modified-release (MR) once-daily (QD) tofacitinib formulation, we compared real-world adherence and effectiveness between patients initiating the MR QD formulation and patients initiating an immediate-release (IR) twice-daily (BID) formulation.

METHODS

Two noninterventional cohort studies were conducted. First, adherence and two effectiveness proxies were compared between patients with RA who newly initiated tofacitinib MR 11 mg QD or IR 5 mg BID in the IBM® MarketScan® Commercial and Medicare Supplemental US insurance claims databases (March 2016-October 2018). Second, using data collected in the Corrona US RA Registry (February 2016-August 2019), two Clinical Disease Activity Index (CDAI)-based measures of effectiveness were compared between tofacitinib MR 11 mg QD and IR 5 mg BID, and against noninferiority criteria derived from placebo-controlled clinical trials of the tofacitinib IR formulation. Multiple sensitivity analyses of the registry data were conducted to reassure regulators of consistent results across different assumptions.

RESULTS

In each study, approximately two-thirds of patients initiated the MR formulation. In the claims database study, improved adherence and at least comparable effectiveness were observed with tofacitinib MR vs IR over 12 months, particularly in patients without prior advanced therapy. In the registry study, the noninferiority of tofacitinib MR vs IR was demonstrated for both CDAI outcomes at ~6 months; this finding was robust across multiple sensitivity analyses.

CONCLUSION

These results demonstrate the value of real-world evidence from complementary data sources in understanding the impact of medication adherence with a QD formulation in clinical practice. These analyses were suitable for regulatory consideration as an important component of evidence for the comparability of tofacitinib MR 11 mg QD vs IR 5 mg BID in patients with RA.

TRIAL REGISTRATION

Claims database study: ClinicalTrials.gov identifier NCT04018001, retrospectively registered July 12, 2019. Corrona US RA Registry study: ClinicalTrials.gov identifier NCT04267380, retrospectively registered February 12, 2020.

Comparative effectiveness of oral prostacyclin pathway drugs on hospitalization in patients with pulmonary hypertension in the United States: a retrospective database analysis

Two oral medications targeting the prostacyclin pathway are available to treat pulmonary arterial hypertension in the United States: oral treprostinil and selexipag. We compared real-world hospitalization in patients receiving these medications. A retrospective administrative claims study was conducted using the Optum® Clinformatics® Data Mart database. Patients with pulmonary hypertension were identified using diagnostic codes. Cohort inclusion required age ≥ 18 years, first oral treprostinil or selexipag prescription between 1 January 2015 and 30 September 2017 (index date), and continuous enrollment in the prior ≥6 months. Patients who switched index drug were excluded. Follow-up was from index date until the first of end of index drug exposure, end of continuous enrollment, death, or 31 December 2017. Multivariable Cox proportional hazard and Poisson regression were used to compare risk and rate, respectively, of hospitalization associated with oral treprostinil vs. selexipag, adjusting for potential confounders. The study cohort included 99 patients receiving oral treprostinil and 123 receiving selexipag. Mean age was 61 years, and most patients were females (71%). Compared with oral treprostinil, selexipag was associated with a 46% lower risk of all-cause hospitalization (hazard ratio 0.54, 95% confidence interval 0.31, 0.92; P = 0.02), a 47% lower risk of pulmonary hypertension-related hospitalization (hazard ratio 0.53, 95% confidence interval 0.31, 0.93; P = 0.03), a 42% lower all-cause hospitalization rate (rate ratio 0.58, 95% confidence interval 0.39, 0.87; P = 0.01), and a 46% lower pulmonary hypertension-related hospitalization rate (rate ratio 0.54, 95% confidence interval 0.35, 0.82; P = 0.004). This study suggests that selexipag is associated with lower hospitalization risk and rate than oral treprostinil.

Nutritional outcomes in head and neck cancer patients: is intensive nutritional care worth it?

OBJECTIVE

This study aimed to compare nutritional outcomes before and after implementation of weekly dietetic counseling (intensive nutritional care) in head and neck cancers patients.

METHODS

A retrospective study with all head and neck patients, who received radiotherapy between January 2010 and December 2017 were performed. The main outcome was significant weight loss. Compliance to caloric and protein recommendations were also evaluated.

RESULTS

In all, 472 patients were included. Weight loss was not different between before and after implementation (-6.7%; IQ -10.5/-1.9 vs -5.0%; IQ -9.8/-0.7;p=0.06).There were no significant difference in terms of meeting the recommended intake. Higher baseline body mass index and oral nutritional support predicted significant weight loss.

CONCLUSION

Implementation of intensive nutritional care did not have an impact on weight loss and energy and protein intake in head and neck cancer patients. Further research would be of value to determine the appropriate service-delivery model to achieve optimal patient outcomes.

The Certainty Framework for Assessing Real-World Data in Studies of Medical Product Safety and Effectiveness

A fundamental question in using real-world data for clinical and regulatory decision making is: How certain must we be that the algorithm used to capture an exposure, outcome, cohort-defining characteristic, or confounder is what we intend it to be? We provide a practical framework to help researchers and regulators assess and classify the fit-for-purposefulness of real-world data by study variable for a range of data sources. The three levels of certainty (optimal, sufficient, and probable) must be considered in the context of each study variable, the specific question being studied, the study design, and the decision at hand.

Real-world survival outcomes of heavily pretreated patients with refractory HR+, HER2-metastatic breast cancer receiving single-agent chemotherapy-a comparison with MONARCH 1

PURPOSE

In MONARCH 1 (NCT02102490), single-agent abemaciclib demonstrated promising efficacy activity and tolerability in a population of heavily pretreated women with refractory HR+, HER2- metastatic breast cancer (MBC). To help interpret these results and put in clinical context, we compared overall survival (OS) and duration of therapy (DoT) between MONARCH 1 and a real-world single-agent chemotherapy cohort.

METHODS

The real-world chemotherapy cohort was created from a Flatiron Health electronic health records-derived database based on key eligibility criteria from MONARCH 1. The chemotherapies included in the cohort were single-agent capecitabine, gemcitabine, eribulin, or vinorelbine. Results were adjusted for baseline demographics and clinical differences using Mahalanobis distance matching (primary analysis) and entropy balancing (sensitivity analysis). OS and DoT were analyzed using the Kaplan-Meier method and Cox proportional hazards regression.

RESULTS

A real-world single-agent chemotherapy cohort (n = 281) with eligibility criteria similar to the MONARCH 1 population (n = 132) was identified. The MONARCH 1 (n = 108) cohort was matched to the real-world chemotherapy cohort (n = 108). Median OS was 22.3 months in the abemaciclib arm versus 13.6 months in the matched real-world chemotherapy cohort with an estimated hazard ratio (HR) of 0.54. The median DoT was 4.1 months in MONARCH 1 compared to 2.9 months in the real-world chemotherapy cohort with HR of 0.76.

CONCLUSIONS

This study demonstrates an approach to create a real-world chemotherapy cohort suitable to serve as a comparator for trial data. These exploratory results suggest a survival advantage and place the benefit of abemaciclib monotherapy in clinical context.

Bridging the Gap Between RCTs and RWE Through Endpoint Selection

This commentary is authored by several industry real-world evidence (RWE) experts, with support from IQVIA, as part of the 'RWE Leadership Forum': a group of Industry Leaders who have come together as non-competitive partners to understand and respond to RWD/E challenges and opportunities with a single expert voice. Here, the forum discusses the value in bridging the industry disconnect between RTCs and RWE, with a view to promoting the use of RWE in the RCT environment. RCT endpoints are explored along several axes including their clinical relevance and their measure of direct patient benefit, and then compared with their real-world counterparts to identify suitable paths, or gaps, for assimilating RWE endpoints into the RCT environment.

An Exploratory Analysis of Real-World End Points for Assessing Outcomes Among Immunotherapy-Treated Patients With Advanced Non-Small-Cell Lung Cancer

PURPOSE

This pilot study examined the ability to operationalize the collection of real-world data to explore the potential use of real-world end points extracted from data from diverse health care data organizations and to assess how these relate to similar end points in clinical trials for immunotherapy-treated advanced non–small-cell lung cancer.

PATIENTS AND METHODS

Researchers from six organizations followed a common protocol using data from administrative claims and electronic health records to assess real-world end points, including overall survival (rwOS), time to next treatment, time to treatment discontinuation (rwTTD), time to progression, and progression-free survival, among patients with advanced non–small-cell lung cancer treated with programmed death 1/programmed death-ligand 1 inhibitors in real-world settings. Data sets included from 269 to 6,924 patients who were treated between January 2011 and October 2017. Results from contributors were anonymized.

RESULTS

Correlations between real-world intermediate end points (rwTTD and time to next treatment) and rwOS were moderate to high (range, 0.6 to 0.9). rwTTD was the most consistent end points as treatment detail was available in all data sets. rwOS at 1 year post–programmed death-ligand 1 initiation ranged from 40% to 57%. In addition, rwOS as assessed via electronic health records and claims data fell within the range of median OS values observed in relevant clinical trials. Data sources had been used extensively for research with ongoing data curation to assure accuracy and practical completeness before the initiation of this research.

CONCLUSION

These findings demonstrate that real-world end points are generally consistent with each other and with outcomes observed in randomized clinical trials, which substantiates the potential validity of real-world data to support regulatory and payer decision making. Differences observed likely reflect true differences between real-world and protocol-driven practices.

Expanded Access as a source of real-world data: An overview of FDA and EMA approvals

Aims

To identify, characterize and compare all Food and Drug Administration (FDA) and European Medicines Agency (EMA) approvals that included real‐world data on efficacy from expanded access (EA) programmes.

Methods

Cross‐sectional study of FDA (1955–2018) and EMA (1995–2018) regulatory approval documentation. We automated searching for terms related to EA in 22,506 documents using machine learning techniques. We included all approvals where EA terms appeared in the regulatory documentation. Our main outcome was the inclusion of EA data as evidence of clinical efficacy. Characterization was based on approval date, disease area, orphan designation and whether the evidence was supportive or pivotal.

Results

EA terms appeared in 693 out of 22,506 (3.1%) documents, which referenced 187 approvals. For 39 approvals, data from EA programmes were used to inform on clinical efficacy. The yearly number of approvals with EA data increased from 1.25 for 1993–2013 to 4.6 from 2014–2018. In 13 cases, these programmes formed the main evidence for approval. Of these, patients in EA programmes formed over half (median 71%, interquartile range: 34–100) of the total patient population available for efficacy evaluation. Almost all (12/13) approvals were granted orphan designation. In 8/13, there were differences between regulators in approval status and valuation of evidence. Strikingly, 4 treatments were granted approval based solely on efficacy from EA.

Conclusion

Sponsors and regulators increasingly include real‐world data from EA programmes in the efficacy profile of a treatment. The indications of the approved treatments are characterized by orphan designation and high unmet medical need.

Real-world evidence to support regulatory decision-making for medicines: Considerations for external control arms

Randomized clinical trials (RCTs) are the gold standard in producing clinical evidence of efficacy and safety of medical interventions. More recently, a new paradigm is emerging-specifically within the context of preauthorization regulatory decision-making-for some novel uses of real-world evidence (RWE) from a variety of real-world data (RWD) sources to answer certain clinical questions. Traditionally reserved for rare diseases and other special circumstances, external controls (eg, historical controls) are recognized as a possible type of control arm for single-arm trials. However, creating and analyzing an external control arm using RWD can be challenging since design and analytics may not fully control for all systematic differences (biases). Nonetheless, certain biases can be attenuated using appropriate design and analytical approaches. The main objective of this paper is to improve the scientific rigor in the generation of external control arms using RWD. Here we (a) discuss the rationale and regulatory circumstances appropriate for external control arms, (b) define different types of external control arms, and (c) describe study design elements and approaches to mitigate certain biases in external control arms. This manuscript received endorsement from the International Society for Pharmacoepidemiology (ISPE).

Real-World Evidence to Assess Medication Safety or Effectiveness in Children: Systematic Review

Background

The promise of real-world evidence (RWE) is especially relevant to pediatrics, where medicines prescribed for children are often used without evidence derived from randomized clinical trials.

Objectives

The aim of this systematic review was to describe the state of RWE in pediatrics by identifying observational studies published during 2016 that used RWE to assess medication safety or effectiveness in children.

Methods

An electronic search of PubMed was combined with an extended search of references within systematic reviews and expert suggestions. Studies were included if they reported on an infant or child under 18 years with exposure to medications; assessed safety or effectiveness; specified a comparison or control group, and were published in English in 2016. Data extraction was conducted by one team member using a standardized form and reviewed by a second team member. Study quality was assessed using the GRACE checklist for rating the quality of observational studies.

Results

After removing duplicates, 915 citations were screened and 29 studies met the eligibility criteria. Most of the eligible studies relied on primary data collection or chart review at a single institution and did not use the growing number of administrative or electronic health record databases available. One-quarter of the studies did not use well-established statistical methods to control for confounders. No single disease group or medication predominated, and age groups ranged from infants to adolescents.

Conclusions

A small body of observational studies published in 2016 were categorized by the study team as using real-world data to assess medication safety or effectiveness in children. Studies varied in age groups, diseases or conditions, and methods, and may not have fully met the FDA definition of RWE. Our review indicates that the use of RWE is not fully developed in pediatrics, and suggests an opportunity to further develop capabilities and more fully leverage administrative and electronic health record databases to study medication safety and effectiveness in children. Our systematic review appears generalizable to pediatrics broadly, and documents that the high level of activity in RWE in general has had less of an impact on pediatrics.

Electronic supplementary material

The online version of this article (10.1007/s40801-020-00182-y) contains supplementary material, which is available to authorized users.

A scoping review of core outcome sets and their 'mapping' onto real-world data using prostate cancer as a case study

Background

A Core Outcomes Set (COS) is an agreed minimum set of outcomes that should be reported in all clinical studies related to a specific condition. Using prostate cancer as a case study, we identified, summarized, and critically appraised published COS development studies and assessed the degree of overlap between them and selected real-world data (RWD) sources.

Methods

We conducted a scoping review of the Core Outcome Measures in Effectiveness Trials (COMET) Initiative database to identify all COS studies developed for prostate cancer. Several characteristics (i.e., study type, methods for consensus, type of participants, outcomes included in COS and corresponding measurement instruments, timing, and sources) were extracted from the studies; outcomes were classified according to a predefined 38-item taxonomy. The study methodology was assessed based on the recent COS-STAndards for Development (COS-STAD) recommendations. A ‘mapping’ exercise was conducted between the COS identified and RWD routinely collected in selected European countries.

Results

Eleven COS development studies published between 1995 and 2017 were retrieved, of which 8 were classified as ‘COS for clinical trials and clinical research’, 2 as ‘COS for practice’ and 1 as ‘COS patient reported outcomes’. Recommended outcomes were mainly categorized into ‘mortality and survival’ (17%), ‘outcomes related to neoplasm’ (18%), and ‘renal and urinary outcomes’ (13%) with no relevant differences among COS study types. The studies generally fulfilled the criteria for the COS-STAD ‘scope specification’ domain but not the ‘stakeholders involved’ and ‘consensus process’ domains. About 72% overlap existed between COS and linked administrative data sources, with important gaps. Linking with patient registries improved coverage (85%), but was sometimes limited to smaller follow-up patient groups.

Conclusions

This scoping review identified few COS development studies in prostate cancer, some quite dated and with a growing level of methodological quality over time. This study revealed promising overlap between COS and RWD sources, though with important limitations; linking established, national patient registries to administrative data provide the best means to additionally capture patient-reported and some clinical outcomes over time. Thus, increasing the combination of different data sources and the interoperability of systems to follow larger patient groups in RWD is required.

Framework for personalized prediction of treatment response in relapsing remitting multiple sclerosis

BACKGROUND

Personalized healthcare promises to successfully advance the treatment of heterogeneous neurological disorders such as relapsing remitting multiple sclerosis by addressing the caveats of traditional healthcare. This study presents a framework for personalized prediction of treatment response based on real-world data from the NeuroTransData network.

METHODS

A framework for personalized prediction of response to various treatments currently available for relapsing remitting multiple sclerosis patients was proposed. Two indicators of therapy effectiveness were used: number of relapses, and confirmed disability progression. The following steps were performed: (1) Data preprocessing and selection of predictors according to quality and inclusion criteria; (2) Implementation of hierarchical Bayesian generalized linear models for estimating treatment response; (3) Validation of the resulting predictive models based on several performance measures and routines, together with additional analyses that focus on evaluating the usability in clinical practice, such as comparing predicted treatment response with the empirically observed course of multiple sclerosis for different adherence profiles.

RESULTS

The results revealed that the predictive models provide robust and accurate predictions and generalize to new patients and clinical sites. Three different out-of-sample validation schemes (10-fold cross-validation, leave-one-site-out cross-validation, and excluding a test set) were employed to assess generalizability based on three different statistical performance measures (mean squared error, Harrell's concordance statistic, and negative log-likelihood). Sensitivity to different choices of the priors, to the characteristics of the underlying patient population, and to the sample size, was assessed. Finally, it was shown that model predictions are clinically meaningful.

CONCLUSIONS

Applying personalized predictive models in relapsing remitting multiple sclerosis patients is still new territory that is rapidly evolving and has many challenges. The proposed framework addresses the following challenges: robustness and accuracy of the predictions, generalizability to new patients and clinical sites and comparability of the predicted effectiveness of different therapies. The methodological and clinical soundness of the results builds the basis for a future support of patients and doctors when the current treatment is not generating the desired effect and they are considering a therapy switch. (A) The framework is developed using quality-proven real-world data of patients with relapsing remitting multiple sclerosis. Patients have heterogeneous individual characteristics and diverse disease profiles, indicated for example by variations in frequency of relapses and degree of disability. Longitudinal characteristics regarding disease history (e.g. number of previous relapses in the last 12 months) are extracted at the time of an intended therapy switch, i.e. at time point "Today" (left). All clinical parameters are captured in a standardized way (right). (B) The model predicts the course of the disease based on the observed data (panel A), and is able to account for the impact of various available therapies on chosen clinical endpoints. The resulting ranking of therapies has a dependency on patient characteristics, illustrated here by a different highest ranked therapy depending on the number of relapse in the previous 12 months. (C) The model is evaluated for various generalization properties. Compared to performance on the training set (gray) it is able to predict for new patients not part of the training set (red).Top: Prediction for new patients. Middle: Prediction for new clinical sites. Bottom: Prediction for different time windows. (D) In order to assess the clinical impact of the model, disease activity is compared between patients treated with the highest ranked therapy and those treated with any of the other therapies. Patients adhering to the highest ranked therapy are associated with a better disease outcome when compared to those who did not.

Trial designs using real-world data: The changing landscape of the regulatory approval process

Purpose

There is a need to develop hybrid trial methodology combining the best parts of traditional randomized controlled trials (RCTs) and observational study designs to produce real‐world evidence (RWE) that provides adequate scientific evidence for regulatory decision‐making.

Methods

This review explores how hybrid study designs that include features of RCTs and studies with real‐world data (RWD) can combine the advantages of both to generate RWE that is fit for regulatory purposes.

Results

Some hybrid designs include randomization and use pragmatic outcomes; other designs use single‐arm trial data supplemented with external comparators derived from RWD or leverage novel data collection approaches to capture long‐term outcomes in a real‐world setting. Some of these approaches have already been successfully used in regulatory decisions, raising the possibility that studies using RWD could increasingly be used to augment or replace traditional RCTs for the demonstration of drug effectiveness in certain contexts. These changes come against a background of long reliance on RCTs for regulatory decision‐making, which are labor‐intensive, costly, and produce data that can have limited applicability in real‐world clinical practice.

Conclusions

While RWE from observational studies is well accepted for satisfying postapproval safety monitoring requirements, it has not commonly been used to demonstrate drug effectiveness for regulatory purposes. However, this position is changing as regulatory opinions, guidance frameworks, and RWD methodologies are evolving, with growing recognition of the value of using RWE that is acceptable for regulatory decision‐making.

RWE Framework: An Interactive Visual Tool to Support a Real-World Evidence Study Design

OBJECTIVES

Real-world evidence (RWE) is essential for the development of pharmaceutical and medical technologies and informs treatment-related decisions by regulatory agencies, payers, healthcare providers, and patients. Given that planning RWE studies present diverse challenges, we developed the RWE Framework, a concise, visual, interactive tool designed to align multidisciplinary stakeholders toward common goals and encourage a methodical approach to RWE study planning.

METHODS

A search of published literature and internet-based resources was performed to identify guidance on RWE study planning with decision and/or visual aids. A conceptual framework for a study design tool was developed based on best practices for RWE studies, enhanced with an infographic design, and refined by multidisciplinary input from RWE researchers.

RESULTS

The searches confirmed an unmet need for a concise tool to support a broad range of RWE study designs: only two sources with decision/visual aids were identified. The novel RWE Framework comprises sequential decision steps with instructions to guide users through consideration of research objectives, product approval status, study setting, outcomes of interest, data availability in routine practice, need for primary data collection and/or randomization, study type and methodology, and applicable regulatory standards. Pilot testing using case studies of pharmaceutical assets demonstrated the utility of RWE Framework and applicability for use in team environments.

CONCLUSIONS

The RWE Framework is a novel, concise, visual, and interactive tool to inform RWE study planning. It addresses a broad range of real-world study types and research objectives and was found to enhance RWE decision-making by multidisciplinary teams. Further validation is warranted.

When Context Is Hard to Come By: External Comparators and How to Use Them

External comparators, also referred to as historical or synthetic controls, present transformational opportunities for broad context and insights alongside clinical research results. The recent confluence of access to quality real-world data (RWD), advanced epidemiologic methods, and legislative directives to regulators for expanded use of RWD is increasing interest in real-world external comparators, opening the door to achieve broader generalizability and learn more, faster. In this less standardized area of research, tailored scientific methodology must be applied for external comparators to accomplish clinical development objectives. Here, we describe methodological considerations for design and illustrate how RWD comparators have been used for regulatory and reimbursement decisions.

Real-World Evidence Should Be Used in Regulatory Decisions About New Pharmaceutical and Medical Device Products for Diabetes

Randomized clinical trials (RCTs) are no longer the sole source of data to inform guidelines, regulatory, and policy decisions. Real-world data (RWD), collected from registries, electronic health records, insurance claims, pharmacy records, social media, and sensor outputs from devices form real-world evidence (RWE), which can supplement evidence from RCTs. Benefits of using RWE include less time and cost to produce meaningful data; the ability to capture additional information, including social determinants of health that can impact health outcomes; detection of uncommon adverse events; and the potential to apply machine learning and artificial intelligence to the delivery of health care. Overall, combining data from RCTs and RWE would allow regulators to make ongoing and more evidence-based decisions in approving and monitoring products for diabetes.

Future of the drug label: Perspectives from a multistakeholder dialogue

Regulating drugs does not end when market access has been granted. Monitoring drugs over the life cycle has become state of the art, inherent to evolving legislation and societal need. Here, we explore how the drug label could move along in a changing playing‐field and become a sustainable label for the future. A dialogue between academia, government, the pharmaceutical industry and patient/societal organizations was organized by the Regulatory Science Network Netherlands. This is their view.

Lessons on Data Collection and Curation From the NFL Injury Surveillance Program

BACKGROUND

"Research-ready" evidence platforms that link sports data with anonymized electronic health records (EHRs) or other data are important tools for evaluating injury occurrence in response to changes in games, training, rules, and other factors. While there is agreement that high-quality data are essential, there is little evidence to guide data curation.

HYPOTHESIS

We hypothesized that an EHR used in the course of clinical care and curated for research readiness can provide a robust evidence platform. Our purpose was to describe the data curation used for active injury surveillance by the National Football League (NFL).

STUDY DESIGN

Dynamic cohort study.

LEVEL OF EVIDENCE

Level 2.

METHODS

Players provide informed consent for research activities through the collective bargaining process. A league-wide EHR is used to record injuries that come to the attention of the teams' athletic trainers and physicians, NFL medical spotters, or unaffiliated neurotrauma consultants. Information about football activities and injuries are linkable by player, setting, and event to other sports-related data, including game statistics and game-day stadium quality measures, using a unique player identification designed to protect player privacy. Ongoing data curation is used to review data completeness and accuracy and is adjusted over time in response to findings.

RESULTS

The core data curation activities include monthly injury summaries to team staff, queries to resolve incomplete reporting, and periodic external checks. Experiences derived from producing more than 100 reports per year on diverse topics are used to update coding training and related guidance documents in response to missing data or inconsistent coding that is observed. Roughly 20% more injuries were recorded for the same "reportable" injuries after switching from targeted reporting to an EHR.

CONCLUSION

Research-ready databases need systematic curation for quality and completeness, along with related action plans. More injuries were reported through EHR than through targeted reporting.

CLINICAL RELEVANCE

Evidence-driven decision-making thrives on reliable data fine-tuned through systematic use, review, and ongoing adjustments to the curation process.

ISPOR, the FDA, and the Evolving Regulatory Science of Medical Device Products

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) is a key venue for members from private industry, government, and academia to collaborate and share advances in regulatory, clinical, and reimbursement science for drugs, devices, and diagnostics. In parallel, the US Food and Drug Administration (FDA) "is responsible for advancing the public health by helping to speed innovations that make medical products more effective, safer, and more affordable." In 2012, the Medical Device Innovation Consortium (MDIC) was formed as a public-private partnership bringing together government, industry, and nonprofit organizations to advance approaches that promote patient access to safe, innovative medical technologies. With a focus on regulatory science, the MDIC has been assessing how to apply real-world evidence (RWE) regulatory science to medical devices. A key goal of this project is to review the history of RWE regulatory science, define terms, and explain why and how RWE is being considered across the total product life cycle, including regulatory assessment. Unique considerations of real-world data for in vitro diagnostics are also taken into account. We envision that these activities will help ensure a high level of rigor and integrity of RWE necessary for regulatory use cases and demonstrate where RWE can be successfully used for regulatory decision making. The ISPOR, FDA, and MDIC are providing the needed leadership in ensuring that diverse stakeholders share a meaningful voice in determining RWE use and, by so doing, are improving the quality and efficiency of care, enhancing health outcomes, and addressing broader societal concerns of reducing health disparities and costs.

Creating a Synthetic Clinical Trial: Comparative Effectiveness Analyses Using an Electronic Medical Record

PURPOSE

Electronic medical records (EMRs) are a vast resource of potentially mineable data that can be used to complement and extend clinical trials. Extracting and analyzing EMR data are impeded by technical complexities associated with large, multiformat databases. We sought to develop and validate a framework that would overcome the difficulties associated with EMR data and create a simple, portable, and expandable system to better use this resource.

MATERIALS AND METHODS

An Internet-accessible program was developed in Python that applied user-defined criteria to identify and extract patient data from Memorial Sloan Kettering databases. A Worker Application composed of individual modules was developed to identify each patient's functional status, smoking status, and treatment classification. The validity of this approach was tested by identifying, extracting, and analyzing data from a patient cohort that paralleled a practice-changing, prospective, randomized phase III clinical trial performed at a different institution. We called this a synthetic clinical trial.

RESULTS

Our synthetic clinical trial identified and extracted data on a cohort of 281 patients with lung cancer who matched inclusion criteria and received their first treatment between October 2003 and July 2010. The data extraction modules were precise and accurate, with F-measures greater than 0.98. Results were similar in directionality and magnitude to the chosen comparator clinical trial.

CONCLUSION

Our framework offers an accurate and user-friendly interface for identifying and extracting EMR data that can be used to create synthetic clinical trials. Additional studies are needed to validate this approach in other patient cohorts, replicate our findings, and leverage this methodology to improve patient care and accelerate drug development.

A Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence for Regulatory Decisions

Real‐world evidence provides important information about the effects of medicines in routine clinical practice. To engender trust that evidence generated for regulatory purposes is sufficiently valid, transparency in the reasoning that underlies study design decisions is critical. Building on existing guidance and frameworks, we developed the Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real‐world Evidence (SPACE) as a process for identifying design elements and minimal criteria for feasibility and validity concerns, and for documenting decisions. Starting with an articulated research question, we identify key components of the randomized controlled trial needed to maximize validity, and pragmatic choices are considered when required. A causal diagram is used to justify the variables identified for confounding control, and key decisions, assumptions, and evidence are captured in a structured way. In this way, SPACE may improve dialogue and build trust among healthcare providers, patients, regulators, and researchers.

Research methodology and practical issues relating to the conduct of a medical device registry

BACKGROUND

The postmarket research goal is to assess "generalizability" or "external validity" to see if the early results of clinical trials with investigational devices are reproducible in everyday practice in the real world and the longer term. Registries have an important but ambivalent role in achieving this goal.

METHODS

Although registries are common, in practice they follow the regulatory processes that appear designed primarily for pharmaceutical clinical trials and confirmatory studies. We review the literature to assess different definitions and the role of registries in the hierarchy of scientific evidence. We analyze common characteristics affecting registry design, implementation, and governance as well as safety reporting and off-label use while describing the experience of setting up an international, prospective registry for an endovascular device used to treat abdominal aortic aneurysms.

RESULTS

Key areas in which to distinguish registries from trials are as follows: eligibility, setting (patients and institutions), device configurations and iterations, the use of design and quality "spaces," a focus on systematic quality checks (rather than source data monitoring), open-ended follow-up, flexibility in the definition of end points and sample sizes, data sharing, and publishing commitments.

CONCLUSION

Both clinical trials and registries are essential and complementary research methods and the strengths and weaknesses of each need to be recognized. The specific characteristics of registry research deserve to be acknowledged and safeguarded in the regulations governing clinical investigations with medical devices.

Avelumab in gastric cancer

Gastric cancer (GC) is the fifth most common malignancy and the third cause of cancer-related deaths worldwide. Currently, surgery and chemotherapy remain the main therapeutic options and the prognosis of the disease is still poor in the metastatic setting. Avelumab is a human IgG1 antibody directed against PD-L1 approved for Merkel cell carcinoma and urothelial carcinoma that could be useful also for the treatment of GC. This review describes the chemical structure, the pharmacologic properties and the current knowledge of the efficacy of avelumab in the treatment of GC from the data available on the first and later phase clinical trials. The ongoing studies testing this drug either alone or in combination with other drugs are also described.