Real-world evidence to support regulatory decision making: New or expanded medical product indications

Expert survey on real-world data utilization and real-world evidence generation for regulatory decision-making in drug lifecycle in Korea

As the importance of utilizing real-world data (RWD)/real-world evidence (RWE) for supporting regulatory scientific decision-making continues to grow, experiences and inputs from experts become crucial for developing a systematic and practice-oriented plan for the use of fit-for-purpose RWD/RWE. This study aimed to survey relevant experts from government agencies, industries, and academia to identify prerequisites for the drug life cycle in Korea. The questionnaire comprised the following: (A) the definition and categories of RWD/RWE, (B) the suitability and feasibility of using RWD/RWE at each authorization stage by the types of RWD, and (C) the challenges and solutions for the use of RWD/RWE. A total of 46 respondents completed the online survey, with 89.1% of them having prior experience with RWD/RWE usage. A majority of respondents agreed that RWD can be obtained from various sources. Among these sources, the registry was the most suitable source. It is suitable to compensate for the limitations of randomized control trials and ensure quality in data collection. Though there was consensus among the respondents for the use of RWD/RWE in post-marketing surveillance, the use of such data in new drug application (NDA) was disagreeable. Respondents considered it necessary to write a protocol in advance for RWD collection and RWE generation, for all RWD types. In conclusion, this study examined the perceptions of experts for RWD/RWE use at each approval stage of drugs. The results suggest that guidelines for the fit-for-purpose use of RWD/RWE should be developed via careful deliberation among experts in the future.

Real-World Data Quality Framework for Oncology Time to Treatment Discontinuation Use Case: Implementation and Evaluation Study

BACKGROUND

The importance of real-world evidence is widely recognized in observational oncology studies. However, the lack of interoperable data quality standards in the fragmented health information technology landscape represents an important challenge. Therefore, adopting validated systematic methods for evaluating data quality is important for oncology outcomes research leveraging real-world data (RWD).

OBJECTIVE

This study aims to implement real-world time to treatment discontinuation (rwTTD) for a systemic anticancer therapy (SACT) as a new use case for the Use Case Specific Relevance and Quality Assessment, a framework linking data quality and relevance in fit-for-purpose RWD assessment.

METHODS

To define the rwTTD use case, we mapped the operational definition of rwTTD to RWD elements commonly available from oncology electronic health record-derived data sets. We identified 20 tasks to check the completeness and plausibility of data elements concerning SACT use, line of therapy (LOT), death date, and length of follow-up. Using descriptive statistics, we illustrated how to implement the Use Case Specific Relevance and Quality Assessment on 2 oncology databases (Data sets A and B) to estimate the rwTTD of an SACT drug (target SACT) for patients with advanced head and neck cancer diagnosed on or after January 1, 2015.

RESULTS

A total of 1200 (24.96%) of 4808 patients in Data set A and 237 (5.92%) of 4003 patients in Data set B received the target SACT, suggesting better relevance of the former in estimating the rwTTD of the target SACT. The 2 data sets differed with regard to the terminology used for SACT drugs, LOT format, and target SACT LOT distribution over time. Data set B appeared to have less complete SACT records, longer lags in incorporating the latest data, and incomplete mortality data, suggesting a lack of fitness for estimating rwTTD.

CONCLUSIONS

The fit-for-purpose data quality assessment demonstrated substantial variability in the quality of the 2 real-world data sets. The data quality specifications applied for rwTTD estimation can be expanded to support a broad spectrum of oncology use cases.

Case study using RWD in the context of a pivotal trial for regulatory approval in a rare disease

Rare disorders impact millions of children worldwide, and developing new medicines in this setting is associated with multiple challenges. In this paper, we share a successful story of how real-world data (RWD) were leveraged to accelerate evidence generation and patient access to a life-changing therapy in patients with severe manifestations of PIK3CA-related overgrowth spectrum who require systemic therapy. Despite all the existing regulatory guidelines considering real-world evidence (RWE), there is limited regulatory precedent of the use of this framework in support of a new indication. Thus, our case study illustrates design innovations based on the use of a compassionate use program, primarily in children, as a RWD source for approval of a new therapy in a rare disorder. We highlight the systematic considerations and mitigation of potential sources of bias in order to transform the data into actionable evidence. Our experience shows that RWE can be successfully used with appropriate study planning and mitigation in the context of a rare disorder with a high unmet medical need. Some lessons learned from this case study can benefit therapeutic development in rare disorders.

Lung Function and Asthma Clinical Control in N-ERD Patients, Three-Year Follow-Up in the Context of Real-World Evidence

Purpose

To describe the lung function and clinical control of asthma in patients with N-ERD during three years of medical follow-up using GINA guidelines.

Methods

We evaluated 75 N-ERD and 68 asthma patients (AG). Clinical control, lung function, and asthma treatment were evaluated according to GINA-2014. We compared all variables at baseline and one, two, and three years after treatment.

Results

At baseline, the N-ERD group had better basal lung function (LF) than the AG group (p<0.01), and the AG group used higher doses of inhaled corticosteroids than the N-ERD group (52.4% vs 30.5%, p=0.01) and short-term oral corticosteroid (OCS) use (52.4% vs 30.5%, p<0.01). Instead, N-ERD patients needed more use of leukotriene receptor antagonists (LTRA) (29.3% vs 5.9%, p<0.01). This group had better clinical control than the AG group (62.1% vs 34.1%, p<0.01). During the medical follow-up, the LF of the N-ERD group remained at normal values; however, these parameters improved in AG from one year (p<0.01). Likewise, there was a diminished use of high doses of ICS (52.4% vs 33%, p<0.05) and short-term OCS (67.6% vs 20.6%, p<0.01) in asthma patients. However, N-ERD patients still needed more use of LTRAs (p<0.02) during the study. In this context, one-third of N-ERD patients had to use a combination of two drugs to maintain this control. From the second year on, clinical control of asthma was similar in both groups (p>0.05).

Conclusion

According to GINA guidelines, only one-third of patients with N-ERD can gradually achieve adequate lung function and good asthma control with a high ICS dosage. Only a very small portion of patients will require the continued use of a second medication as an LTRA to keep their asthma under control.

A scoping review of the clinical application of machine learning in data-driven population segmentation analysis

OBJECTIVE

Data-driven population segmentation is commonly used in clinical settings to separate the heterogeneous population into multiple relatively homogenous groups with similar healthcare features. In recent years, machine learning (ML) based segmentation algorithms have garnered interest for their potential to speed up and improve algorithm development across many phenotypes and healthcare situations. This study evaluates ML-based segmentation with respect to (1) the populations applied, (2) the segmentation details, and (3) the outcome evaluations.

MATERIALS AND METHODS

MEDLINE, Embase, Web of Science, and Scopus were used following the PRISMA-ScR criteria. Peer-reviewed studies in the English language that used data-driven population segmentation analysis on structured data from January 2000 to October 2022 were included.

RESULTS

We identified 6077 articles and included 79 for the final analysis. Data-driven population segmentation analysis was employed in various clinical settings. K-means clustering is the most prevalent unsupervised ML paradigm. The most common settings were healthcare institutions. The most common targeted population was the general population.

DISCUSSION

Although all the studies did internal validation, only 11 papers (13.9%) did external validation, and 23 papers (29.1%) conducted methods comparison. The existing papers discussed little validating the robustness of ML modeling.

CONCLUSION

Existing ML applications on population segmentation need more evaluations regarding giving tailored, efficient integrated healthcare solutions compared to traditional segmentation analysis. Future ML applications in the field should emphasize methods' comparisons and external validation and investigate approaches to evaluate individual consistency using different methods.

Disease modification in inflammatory skin disorders: opportunities and challenges

Progress in understanding of the mechanisms underlying chronic inflammatory skin disorders, such as atopic dermatitis and psoriasis vulgaris, has led to new treatment options with the primary goal of alleviating symptoms. In addition, this knowledge has the potential to inform on new strategies aimed at inducing deep and therapy-free remission, that is, disease modification, potentially impacting on associated comorbidities. However, to reach this goal, key areas require further exploration, including the definitions of disease modification and disease activity index, further understanding of disease mechanisms and systemic spillover effects, potential windows of opportunity, biomarkers for patient stratification and successful intervention, as well as appropriate study design. This Perspective article assesses the opportunities and challenges in the discovery and development of disease-modifying therapies for chronic inflammatory skin disorders.

Transporting observational study results to a target population of interest using inverse odds of participation weighting

Inverse odds of participation weighting (IOPW) has been proposed to transport clinical trial findings to target populations of interest when the distribution of treatment effect modifiers differs between trial and target populations. We set out to apply IOPW to transport results from an observational study to a target population of interest. We demonstrated the feasibility of this idea with a real-world example using a nationwide electronic health record derived de-identified database from Flatiron Health. First, we conducted an observational study that carefully adjusted for confounding to estimate the treatment effect of fulvestrant plus palbociclib relative to letrozole plus palbociclib as a second-line therapy among estrogen receptor (ER)-positive, human epidermal growth factor receptor (HER2)-negative metastatic breast cancer patients. Second, we transported these findings to the broader cohort of patients who were eligible for a first-line therapy. The interpretation of the findings and validity of such studies, however, rely on the extent that causal inference assumptions are met.

The relevance of real-world data for the evaluation of neuropathic pain treatments

Treatment of neuropathic pain (NP) is challenging. Interest in real-world evidence (RWE) for benefit-risk assessments of NP treatments increases given the paucity of drugs showing efficacy in randomized controlled trials and restricted labels of available medicines. To provide further context, a literature review regarding regulatory use of RWE and a clinical trial registry search for randomized controlled trials over the last 10 years was carried out. Taken together, and especially for available NP treatments, there is increasing support to consider RWE when evaluating their benefit-risk profile. Examples are provided in which RWE could be used effectively for updating the product label and informing treatment recommendations. Collected and analyzed according to state-of-the-art standards, RWE can inform treatment recommendations and product label decisions.

Licensing of Orphan Medicinal Products—Use of Real-World Data and Other External Data on Efficacy Aspects in Marketing Authorization Applications Concluded at the European Medicines Agency Between 2019 and 2021

Background: Reference to so-called real-world data is more often made in marketing authorization applications for medicines intended to diagnose, prevent or treat rare diseases compared to more common diseases. We provide granularity on the type and aim of any external data on efficacy aspects from both real-world data sources and external trial data as discussed in regulatory submissions of orphan designated medicinal products in the EU. By quantifying the contribution of external data according to various regulatory characteristics, we aimed at identifying specific opportunities for external data in the field of orphan conditions.

Methods: Information on external data in regulatory documents covering 72 orphan designations was extracted. Our sample comprised public assessment reports for approved, refused, or withdrawn applications concluded from 2019–2021 at the European Medicines Agency. Products with an active orphan designation at the time of submission were scrutinized regarding the role of external data on efficacy aspects in the context of marketing authorization applications, or on the criterion of “significant benefit” for the confirmation of the orphan designation at the time of licensing. The reports allowed a broad distinction between clinical development, regulatory decision making, and intended post-approval data collection. We defined three categories of external data, administrative data, structured clinical data, and external trial data (from clinical trials not sponsored by the applicant), and noted whether external data concerned the therapeutic context of the disease or the product under review.

Results: While reference to external data with respect to efficacy aspects was included in 63% of the approved medicinal products in the field of rare diseases, 37% of marketing authorization applications were exclusively based on the dedicated clinical development plan for the product under review. Purely administrative data did not play any role in our sample of reports, but clinical data collected in a structured manner (from routine care or clinical research) were often used to inform on the trial design. Two additional recurrent themes for the use of external data were the contextualization of results, especially to confirm the orphan designation at the time of licensing, and reassurance of a large difference in treatment effect size or consistency of effects observed in clinical trials and practice. External data on the product under review were restricted to either active substances already belonging to the standard of care even before authorization or to compassionate use schemes. Furthermore, external data were considered pivotal for marketing authorization only exceptionally and only for active substances already in use within the specific therapeutic indication. Applications for the rarest conditions and those without authorized treatment alternatives were especially prominent with respect to the use of external data from real-world data sources both in the pre- and post-approval setting.

Conclusion: Specific opportunities for external data in the setting of marketing authorizations in the field of rare diseases were identified. Ongoing initiatives of fostering systematic data collection are promising steps for a more efficient medicinal product development in the field of rare diseases.

Effectiveness and Use of RT‐PCR Point of Care Testing in a Large‐Scale COVID ‐19 Surveillance System

Background

Rapid COVID‐19 testing platforms can identify infected individuals at the point of care (POC), allowing immediate isolation of infected individuals and reducing the risk of transmission. While lab‐based nucleic acid amplification testing (NAAT) is often considered the gold standard to detect SARS‐CoV‐2 in the community, results typically take 2–7 days to return, rendering POC testing a critical diagnostic tool for infection control. The National Football League (NFL) and NFL Players Association deployed a new POC testing strategy using a newly available reverse transcriptase polymerase chain reaction (RT‐PCR) rapid test during the 2020 season, and evaluated diagnostic effectiveness compared to other available devices using real‐world population surveillance data.

Methods

RT‐PCR POC test results were compared to NAAT results from same‐day samples by calculation of positive and negative concordance. Sensitivity analyses were performed for three subgroups: (1) individuals symptomatic at time of positive test; (2) individuals tested during the pilot phase of rollout; and (3) individuals tested daily.

Results

Among 4989 same‐day POC/NAAT pairs, 4957 (99.4%) were concordant, with 93.1% positive concordance and 99.6% negative concordance. Based on adjudicated case status, the false negative rate was 0.2% and false positive rate was 2.9%. In 43 instances, the immediate turnaround of results by POC allowed isolation of infected individuals 1 day sooner than lab‐based testing. Positive/negative concordance in sensitivity analyses were relatively stable.

Conclusion

RT‐PCR POC testing provided timely results that were highly concordant with lab‐based NAAT in population surveillance. Expanded use of effective RT‐PCR POC can enable rapid isolation of infected individuals and reduce COVID‐19 infection in the community.

The Role of High Resolution Ultrasonography in Elucidating Features of the Breast Implants in Asymptomatic Patients After Implant-based Augmentation Mammaplasty

BACKGROUND

We conducted this study to describe the feasibility of high-resolution ultrasound (HRUS) in characterizing a breast implant in patients receiving an implant-based augmentation mammaplasty.

METHODS

The current study was conducted in a total of 612 patients (n =6 12) receiving an implant-based augmentation mammaplasty at other hospitals between August 31, 2017 and August 31, 2020. Of these, 136 patients (n = 136; 272 breasts) receiving reoperation were included in the current study. We compared between the patients' subjective awareness of a breast implant and its HRUS findings and an agreement between HRUS findings of a breast implant and its findings at reoperation.

RESULTS

The proportion of the patients receiving a silicone gel-filled breast implant was increased from 65.44% (89/136) to 81.61% (111/136) on HRUS. Moreover, HRUS was effective in identifying a manufacturer of the device.

CONCLUSIONS

In conclusion, our results indicate that HRUS is feasible in characterizing a breast implant in patients receiving an implant-based augmentation mammaplasty. But further prospective, large-scale studies are warranted to corroborate our results.

LEVEL OF EVIDENCE V

This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors. www.springer.com/00266 .

Comparing Findings From a Friends of Cancer Research Exploratory Analysis of Real-World End Points With the Cancer Analysis System in England

PURPOSE

This study compared real-world end points extracted from the Cancer Analysis System (CAS), a national cancer registry with linkage to national mortality and other health care databases in England, with those from diverse US oncology data sources, including electronic health care records, insurance claims, unstructured medical charts, or a combination, that participated in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0. Consistency between data sets and between real-world overall survival (rwOS) was assessed in patients with immunotherapy-treated advanced non-small-cell lung cancer (aNSCLC).

PATIENTS AND METHODS

Patients with aNSCLC, diagnosed between January 2013 and December 2017, who initiated treatment with approved programmed death ligand-1 (PD-[L]1) inhibitors until March 2018 were included. Real-world end points, including rwOS and real-world time to treatment discontinuation (rwTTD), were assessed using Kaplan-Meier analysis. A synthetic data set, Simulacrum, on the basis of conditional random sampling of the CAS data was used to develop and refine analysis scripts while protecting patient privacy.

RESULTS

Characteristics (age, sex, and histology) of the 2,035 patients with immunotherapy-treated aNSCLC included in the CAS study were broadly comparable with US data sets. In CAS, a higher proportion (46.7%) of patients received a PD-(L)1 inhibitor in the first line than in US data sets (18%-30%). Median rwOS (11.4 months; 95% CI, 10.4 to 12.7) and rwTTD (4.9 months; 95% CI, 4.7 to 5.1) were within the range of US-based data sets (rwOS, 8.6-13.5 months; rwTTD, 3.2-7.0 months).

CONCLUSION

The CAS findings were consistent with those from US-based oncology data sets. Such consistency is important for regulatory decision making. Differences observed between data sets may be explained by variation in health care settings, such as the timing of PD-(L)1 approval and reimbursement, and data capture.

RWD-Cockpit: Application for Quality Assessment of Real-World Data (Preprint)

Background

Digital technologies are transforming the health care system. A large part of information is generated as real-world data (RWD). Data from electronic health records and digital biomarkers have the potential to reveal associations between the benefits and adverse events of medicines, establish new patient-stratification principles, expose unknown disease correlations, and inform on preventive measures. The impact for health care payers and providers, the biopharmaceutical industry, and governments is massive in terms of health outcomes, quality of care, and cost. However, a framework to assess the preliminary quality of RWD is missing, thus hindering the conduct of population-based observational studies to support regulatory decision-making and real-world evidence.

Objective

To address the need to qualify RWD, we aimed to build a web application as a tool to translate characterization of some quality parameters of RWD into a metric and propose a standard framework for evaluating the quality of the RWD.

Methods

The RWD-Cockpit systematically scores data sets based on proposed quality metrics and customizable variables chosen by the user. Sleep RWD generated de novo and publicly available data sets were used to validate the usability and applicability of the web application. The RWD quality score is based on the evaluation of 7 variables: manageability specifies access and publication status; complexity defines univariate, multivariate, and longitudinal data; sample size indicates the size of the sample or samples; privacy and liability stipulates privacy rules; accessibility specifies how the data set can be accessed and to what granularity; periodicity specifies how often the data set is updated; and standardization specifies whether the data set adheres to any specific technical or metadata standard. These variables are associated with several descriptors that define specific characteristics of the data set.

Results

To address the need to qualify RWD, we built the RWD-Cockpit web application, which proposes a framework and applies a common standard for a preliminary evaluation of RWD quality across data sets—molecular, phenotypical, and social—and proposes a standard that can be further personalized by the community retaining an internal standard. Applied to 2 different case studies—de novo–generated sleep data and publicly available data sets—the RWD-Cockpit could identify and provide researchers with variables that might increase quality.

Conclusions

The results from the application of the framework of RWD metrics implemented in the RWD-Cockpit application suggests that multiple data sets can be preliminarily evaluated in terms of quality using the proposed metrics. The output scores—quality identifiers—provide a first quality assessment for the use of RWD. Although extensive challenges remain to be addressed to set RWD quality standards, our proposal can serve as an initial blueprint for community efforts in the characterization of RWD quality for regulated settings.