Replacing RCTs with real world data for regulatory decision making: a self-fulfilling prophecy?

Can requests for real-world evidence by the French HTA body be planned? An exhaustive retrospective case-control study of medicinal products appraisals from 2016 to 2021

OBJECTIVES

In France, decisions for pricing and reimbursement for medicinal products are based on appraisals performed by the National authority for health (Haute Autorité de Santé (HAS)). During the appraisal process, additional real-world evidence can be requested as "Post-Registration Studies" (PRS) when there are uncertainties in evidence that could be resolved by additional data collection. To facilitate PRS planning, a retrospective exploratory analysis was conducted to identify the characteristics of medicinal products associated with a PRS request.

METHODS

This analysis encompassed all appraisals finalized between January 1, 2016 and December 31, 2021 and compared products for which the appraisal led to a PRS request with those that did not.

RESULTS

Six hundred positive opinions for reimbursement were identified, with a PRS request present in 17 percent (n = 103) of cases. The independent characteristics associated with a PRS request were a mild or moderate clinical benefit score, a major to moderate or minor clinical added value score, previous availability under an early access program, and certain therapeutic areas (neurology, pulmonology, and endocrinology). These findings suggest two different profiles of PRS requests: (i) products for which there is uncertainty in the size of the clinical benefit and (ii) innovative products for which a substantial benefit is expected but uncertainties persist.

CONCLUSIONS

These results will assist health technology developers to better anticipate data generation to promptly address uncertainties identified by HAS. It may also help HAS and other assessment agencies to work together to improve postlaunch evidence generation according to the characteristics of the medicinal products.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 14

In this latest update we highlight: a publication from the US FDA regarding the definitions of real-world data (RWD) and real-world evidence (RWE); a publication from academic researchers on a demonstration project for target trial emulation; a publication from the National Institute of Health and Care Excellence (NICE) on the 1 year anniversary of their RWE framework; and a publication from NICE and Flatiron Health on the utility of US RWD for initial UK health technology assessment decision making.

The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop

The use of treatment effects derived from nonrandomized studies (NRS) in health technology assessment (HTA) is growing. NRS carry an inherently greater risk of bias than randomized controlled trials (RCTs). Although bias can be mitigated to some extent through appropriate approaches to study design and analysis, concerns around data availability and quality and the absence of randomization mean residual biases typically render the interpretation of NRS challenging. Quantitative bias analysis (QBA) methods are a range of methods that use additional, typically external, data to understand the potential impact that unmeasured confounding and other biases including selection bias and time biases can have on the results (i.e., treatment effects) from an NRS. QBA has the potential to support HTA bodies in using NRS to support decision-making by quantifying the magnitude, direction, and uncertainty of biases. However, there are a number of key aspects of the use of QBA in HTA which have received limited discussion. This paper presents recommendations for the use of QBA in HTA developed using a multi-stakeholder workshop of experts in HTA with a focus on QBA for unmeasured confounding.

What is the role of randomised trials in implementation science?

BACKGROUND

There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.

MAIN BODY

There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.

CONCLUSION

Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

Gaps in the evidence underpinning high-risk medical devices in Europe at market entry, and potential solutions

AIM

To determine the level of evidence for innovative high-risk medical devices at market entry.

METHODS

We reviewed all Belgian healthcare payer (RIZIV-INAMI) assessor reports on novel implants or invasive medical devices (n = 18, Class IIb-III) available between 2018 to mid-2019 on applications submitted for inclusion on their reimbursement list. We also conducted a review of the literature on evidence gaps and an analysis of relevant legal and ethical frameworks within the European context.

FINDINGS

Conformity assessment of medical devices is based on performance, safety, and an acceptable risk-benefit balance. Information submitted for obtaining CE marking is confidential and legally protected, limiting access to clinical evidence. Seven out of the 18 RIZIV-INAMI assessor reports (39%) included a randomized controlled trial (RCT) using the novel device, whilst 2 applications (11%) referred to an RCT that used a different device. The population included was inappropriate or unclear for 3 devices (17%). Only half of the applications presented evidence on quality of life or functioning and 2 (11%) presented overall survival data. Four applications (22%) included no data beyond twelve months. The findings from the literature demonstrated similar problems with the study design and the clinical evidence.

DISCUSSION AND CONCLUSIONS

CE marking does not indicate that a device is effective, only that it complies with the law. The lack of transparency hampers evidence-based decision making. Despite greater emphasis on clinical benefit for the patient, the provisions of the European Medical Device Regulation (MDR) are not yet fully aligned with international ethical standards for clinical research. The MDR fails to address key issues, such as the lack of access to data submitted for CE marking and a failure to require evidence of clinical effectiveness. Indeed, a first report shows no improvement in the clinical evidence for implantable devices generated under the MDR. Thus, patients may continue to be exposed to ineffective or unsafe novel devices. The Health Technology Assessment Regulation plans for Joint Scientific Consultations for specific high-risk devices before companies begin their pivotal clinical investigations. The demanded comparative evidence should facilitate payer decisions. Nevertheless, there is also a need for legislation requiring comparative RCTs assessing patient-relevant outcomes for high-risk devices to ensure implementation, including development and implementation of common specifications for study designs.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 12

In this latest update we highlight the final results from the RCT-DUPLICATE initiative, the publication of guidance from Haute Autorité de Santé (HAS), the joint viewpoint from the Institute for Quality and Efficiency in HealthCare (IQWIG) and the Belgian HealthCare Knowledge Center, and a position from the European Organization for Research and Treatment of Cancer (EORTC). Finally, we discuss how the NICE RWE framework has been implemented to allow consideration of RWE external control arms.

[Genetic tumour diagnostics and personalised medicine from a systems perspective]

High-quality studies are necessary and feasible in personalised medicine in order to evaluate the benefits across the entire treatment chain of biomarker tests and resulting treatments in regard to patient-relevant endpoints. With the introduction of genome sequencing in oncology, a considerable number of new treatment concepts with mostly low-quality evidence can be expected. High quality requirements, interdisciplinary cooperation structures, knowledge-generating care and the connection of patient care at the expense of the statutory health insurance funds, with research at the expense of the manufacturers or public funding, are necessary.