Series: Pragmatic trials and real world evidence: Paper 6. Outcome measures in the real world

Defining the role of pragmatic clinical trials in cancer clinical research: outcomes of a collaborative workshop hosted by the European Organisation for Research and Treatment of Cancer

Explanatory clinical trials, which focus on evaluating therapeutic efficacy under ideal circumstances, are crucial for learning about new therapeutic interventions; however, they also exhibit shortcomings. These include non-representative populations and frequent use of intermediate endpoints, leading to uncertainty about the applicability of study results to patients in the real-world. Moreover, these trials often do not address all clinically meaningful questions, highlighting the need for optimisation within the oncology research framework. Refinements can be partly achieved by incorporating more pragmatic elements into cancer clinical trials. At a virtual European Organisation for Research and Treatment of Cancer workshop, key stakeholders convened to discuss the methodological characteristics and value of pragmatic trials, which focus on evaluating effectiveness in routine clinical practice, and their capacity to address the efficacy-effectiveness gap. This Policy Review outlines and discusses some of the views and perspectives expressed on the role of pragmatic trials in the current framework and their ability to inform decision making, and the recommended priorities for enhancing pragmatism in cancer clinical research.

Comparative Effectiveness of Active and Reactive Mattresses in Pressure Injury Healing for Older People in Their Own Homes: A Pragmatic Equivalence Randomised-Controlled Study

Background: Pressure injuries are an ongoing problem commonly managed with the prescription of pressure mattresses. There is conflicting research about the comparable effectiveness of the two types of pressure mattresses, active and reactive. This, coupled with technological advances and an updated understanding of pressure aetiology, means decision-making when prescribing pressure mattresses is complicated. Objective/Design: A pragmatic approach was used to design an equivalence randomised-controlled trial investigating the comparative effectiveness of active and reactive pressure mattresses in a community setting from a wound healing perspective as well as from a user acceptability perspective. Methods: Participants with an existing pressure injury were provided with an active or reactive mattress for wound healing, with wound stages assessed using photography. Usual clinical care was provided based on the protocols of the health care service, including nursing and occupational therapy input. Participants were monitored for the healing of their existing pressure injuries, using the Revised Photographic Wound Assessment Tool. User acceptability feedback was provided through surveys, including impact on comfort, pain levels and bed mobility. An equivalence design was used for data analysis to determine if the surfaces were comparable. Results: Twelve participants completed the study, which found that people on active mattresses healed 11.71 days (95% CI -55.97-31.78 days) quicker than people on reactive mattresses; however, the small sample size meant that a definitive determination could not be made. Users found bed mobility more challenging, and pain levels decreased, regardless of mattress type. Conclusions: A pragmatic methodology is imperative for research in this field due to the complexity of pressure injury healing. Researchers exploring multi-faceted conditions should consider a pragmatic design to ensure transferability of results to the clinical setting. The results from this study were inconclusive when determining the equivalence of active and reactive mattresses due to the small sample size. When choosing a mattress, prescribers need to consider user preferences and mattress features to ensure user acceptability.

Poor reporting quality and high proportion of missing data in economic evaluations alongside pragmatic trials: a cross-sectional survey

BACKGROUND

Lack of data integrity is a common problem in randomized clinical trials and is more serious in economic evaluations conducted alongside explanatory clinical trials. Despite pragmatic randomized controlled trials (pRCTs) becoming recognized as the best design for economic evaluations, information on the proportion, handling approaches, and reporting quality of missing data in pRCTs-based economic evaluations remains limited. This study aimed to investigate the quantity and reporting quality of missing data in economic evaluations conducted alongside pragmatic clinical trials.

METHODS

In this cross-sectional survey, data were extracted from PubMed and OVID (Embase, CENTRAL, HTA database, and NHS EED) from January 1, 2010, to April 24, 2022. Economic evaluations conducted alongside pRCTs were included. Two independent reviewer groups identified relevant articles, and data were extracted by three groups comprising two reviewers each. Descriptive analyses were performed to assess the characteristics of the included studies, missingness in the included studies, and handling of missing data.

RESULTS

Overall, 715 studies were identified, of which 152 met the inclusion criteria. In total, 113, 119, and 132 articles reported missing data, costs, and effects, respectively. More than 50% (58/113) of the articles reported the proportion or quantity of overall missingness, and 64.71% and 54.55% reported missing costs and effects, respectively. The proportion of missingness of < 5% in the overall group was 3.45%, whereas the proportions of missing costs and effects were both < 10% (5.26% vs. 8.45%, respectively). In terms of the proportion of missing data, the overall missingness rate was 30.22% in 58 studies, whereas the median proportion of missing data was slightly higher than that of missing effects (30.92% vs. 27.78%). Of the included studies, 56 (36.84%) conducted a sensitivity analysis on handling missing data. Of these, 12.50% reported missing mechanisms, and 83.93% examined handling methods.

CONCLUSIONS

Insufficient description and reporting of missing data, along with a high proportion of missing data in pRCT-based economic evaluations, could decrease the reliability and extrapolation of conclusions, leading to misleading decision-making.

Methods for pragmatic randomized clinical trials of pain therapies: IMMPACT statement

ABSTRACT

Pragmatic, randomized, controlled trials hold the potential to directly inform clinical decision making and health policy regarding the treatment of people experiencing pain. Pragmatic trials are designed to replicate or are embedded within routine clinical care and are increasingly valued to bridge the gap between trial research and clinical practice, especially in multidimensional conditions, such as pain and in nonpharmacological intervention research. To maximize the potential of pragmatic trials in pain research, the careful consideration of each methodological decision is required. Trials aligned with routine practice pose several challenges, such as determining and enrolling appropriate study participants, deciding on the appropriate level of flexibility in treatment delivery, integrating information on concomitant treatments and adherence, and choosing comparator conditions and outcome measures. Ensuring data quality in real-world clinical settings is another challenging goal. Furthermore, current trials in the field would benefit from analysis methods that allow for a differentiated understanding of effects across patient subgroups and improved reporting of methods and context, which is required to assess the generalizability of findings. At the same time, a range of novel methodological approaches provide opportunities for enhanced efficiency and relevance of pragmatic trials to stakeholders and clinical decision making. In this study, best-practice considerations for these and other concerns in pragmatic trials of pain treatments are offered and a number of promising solutions discussed. The basis of these recommendations was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) meeting organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks.

Selecting outcomes for pragmatic clinical trials in dementia care: The IMPACT Collaboratory iLibrary

BACKGROUND

Many interventions improve care and outcomes for people with Alzheimer's Disease and related dementias (ADRD), yet are never disseminated. Pragmatic trials facilitate the adoption and dissemination of best practices, but gaps in pragmatic outcome measurement are a critical obstacle. Our objectives are (1) to describe the development and structure of the IMbedded Pragmatic ADRD Clinical Trials Collaboratory (IMPACT) iLibrary of potential outcome measures for ADRD pragmatic trials, and (2) to assess their pragmatic characteristics.

METHODS

We identified potential outcome measures from several sources: a database of administrative and clinical outcome measures from ADRD clinical trials registered in ClinicalTrials.gov, published reviews, and IMPACT pilot pragmatic trial outcome measures. The iLibrary reports (a) number of items, (b) completion time, (c) readability for diverse populations, (d) cost or copyright barriers to use, (e) method of administration, (f) assessor training burden, and (g) feasibility of data capture and interpretation in routine care; a summary of pragmatic characteristics of each outcome measure (high, moderate, low); items or descriptions of items; and links to primary citations regarding development or psychometric properties.

RESULTS

We included 140 outcome measures in the iLibrary: 66 administrative (100% were pragmatic) and 74 clinical (52% were pragmatic). The most commonly addressed outcome domains from administrative assessments included physical function, quality of care or communication concerns, and psychological symptoms or distress behaviors. The most commonly addressed outcome domains from clinical assessments were psychological symptoms or distress behaviors, physical function, cognitive function, and health-related quality of life.

CONCLUSIONS

Pragmatic outcome measures are brief, meaningful to diverse populations, easily scored and interpreted by clinicians, and available in electronic format for analysis. The iLibrary can facilitate the selection of measures for a wide range of outcomes relevant to people with ADRD and their care partners.

Evaluating the pragmatic characteristics of advance care planning outcome measures in dementia clinical trials: A scoping review

BACKGROUND

Advance care planning (ACP) is a process that involves discussing a person's goals, values, and preferences; it is particularly important for persons living with dementia (PLWD) given that dementia is incurable and progressive. To ensure results that will impact real-world practices, ACP outcome measures must be psychometrically strong, meaningful to key partners, and pragmatic to collect. Therefore, we conducted a scoping review of outcome measures utilized in ACP randomized controlled clinical trials (RCTs) enrolling PLWD or their care partners and evaluated their pragmatic characteristics.

METHODS

We searched MEDLINE/PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science for peer-reviewed ACP RCTs enrolling PLWD or their care partners from 2011 to 2021. We abstracted characteristics of primary and secondary outcome measures, including pragmatic characteristics using an adapted Psychometric and Pragmatic Evidence Rating Scale and ACP outcome domains using the standardized ACP Outcome Framework (i.e., process, action, healthcare, or quality of care).

RESULTS

We included 21 ACP RCTs. Trials included 103 outcome measures (39 primary and 64 secondary), of which 11% measured process, 14% measured action, 49% measured healthcare, and 26% measured quality of care. Twenty-four (23%) outcome measures were highly pragmatic, the majority of which (67%) reflected healthcare outcome measures. Sixty-one (59%) outcomes were assessed as highly relevant to PLWD or their care partners. Only 20% (n = 21) of outcome measures were embedded into clinical practice. Most (62%) RCTs were conducted in nursing homes, and 33% were focused PLWD with advanced stage disease.

CONCLUSIONS

In RCTs testing ACP interventions to support PLWD, only 23% of outcome measures were highly pragmatic, and most of these measured healthcare utilizations. Outcome assessments were rarely integrated into the EHR during routine clinical care. New outcome measures that address the lived experience of PLWD and their care partners plus have high pragmatic characteristics are needed for embedded pragmatic clinical trials.

Validation of the GetReal Trial Tool - Facilitating discussion and understanding more pragmatic design choices and their implications

BACKGROUND

The GetReal Trial Tool is a decision support tool to assess the impact of design choices on generalizability of clinical trials to routine clinical practice, while taking into account the risk of bias, precision, acceptability and operational feasibility. This study describes the validation of the GetReal Trial Tool.

METHODS

Twelve experts took part in the GetReal Trial tool validation using the protocols of 6 trials conducted with pragmatic elements. The tool entails 7 domains with a total of 43 questions. A pooled Kappa statistic (95% CI) using random effects model was estimated using Open Meta (analyst) software. The possible operational challenges were collated and discussed with the trialists that conducted the trials.

RESULTS

Agreement in the design choices made for the trial protocols was >50% for all the trials and all teams reached consensus during discussion. The pooled Kappa statistic (95% CI) was 0.236 (0.154-0.318). The GetReal Trial tool highlighted several operational challenges, of which almost half had been experienced previously by the trialists. Out of 25 additional operational challenges mentioned by the trialists, 76% were already highlighted by the tool. The tool was considered helpful to optimize trials right from the design stage.

CONCLUSION

The GetReal Trial Tool helps to scrutinize the choice of study design in the light of Real World Evidence generation. The tool identifies most of the operational challenges experienced by trialists to date. The tool serves the intended purpose of facilitating discussion and understanding more pragmatic design choices and their implications.

Heterogeneity in pragmatic randomised trials: sources and management

BACKGROUND

Pragmatic trials aim to generate evidence to directly inform patient, caregiver and health-system manager policies and decisions. Heterogeneity in patient characteristics contributes to heterogeneity in their response to the intervention. However, there are many other sources of heterogeneity in outcomes. Based on the expertise and judgements of the authors, we identify different sources of clinical and methodological heterogeneity, which translate into heterogeneity in patient responses-some we consider as desirable and some as undesirable. For each of them, we discuss and, using real-world trial examples, illustrate how heterogeneity should be managed over the whole course of the trial.

MAIN TEXT

Heterogeneity in centres and patients should be welcomed rather than limited. Interventions can be flexible or tailored and control interventions are expected to reflect usual care, avoiding use of a placebo. Co-interventions should be allowed; adherence should not be enforced. All these elements introduce heterogeneity in interventions (experimental or control), which has to be welcomed because it mimics reality. Outcomes should be objective and possibly routinely collected; standardised assessment, blinding and adjudication should be avoided as much as possible because this is not how assessment would be done outside a trial setting. The statistical analysis strategy must be guided by the objective to inform decision-making, thus favouring the intention-to-treat principle. Pragmatic trials should consider including process analyses to inform an understanding of the trial results. Needed data to conduct these analyses should be collected unobtrusively. Finally, ethical principles must be respected, even though this may seem to conflict with goals of pragmatism; consent procedures could be incorporated in the flow of care.

The challenges and opportunities in using real-world data to drive advances in healthcare in East Asia: expert panel recommendations

OBJECTIVE

To provide recommendations for overcoming the challenges associated with the generation and use of real-world evidence (RWE) in regulatory approvals, health technology assessments (HTAs), and reimbursement decision-making in East Asia.

METHODS

A panel of experts convened at the International Society for Pharmacoeconomics and Outcomes Research Asia Pacific 2020 congress to discuss the challenges limiting the use of RWE in healthcare decision-making and to provide insights into the perspectives of regulators, HTA agencies, the pharmaceutical industry, and physicians in China, Japan, and Taiwan. A nonsystematic literature review was conducted to expand on the themes addressed.

RESULTS

The use of RWE in regulatory approvals, HTAs, and reimbursement decision-making remains limited by legal/regulatory, technical, and attitudinal challenges in East Asia.

CONCLUSIONS

We recommend approaches and initiatives that aim to drive improvements in the utilization of RWE in healthcare decision-making in East Asia and other regions. We encourage large-scale collaborations that leverage the full range of skills offered by different stakeholders. Government agencies, hospitals, research organizations, patient groups, and the pharmaceutical industry must collaborate to ensure appropriate access to robust and reliable real-world data and seek alignment on how to address prioritized evidence needs. Increasingly, we believe that this work will be conducted by multidisciplinary teams with expertise in healthcare research and delivery, data science, and information technology. We hope this work will encourage further discussion among all stakeholders seeking to shape the RWE landscape in East Asia and other regions and drive next-generation healthcare.

Effectiveness and cost-effectiveness of an intervention to improve Initial Medication Adherence to treatments for cardiovascular diseases and diabetes in primary care: study protocol for a pragmatic cluster randomised controlled trial and economic model (the IMA-cRCT study)

BACKGROUND

Between 2 and 43% of patients who receive a new prescription in PC do not initiate their treatments. Non-initiation is associated with poorer clinical outcomes, more sick leave and higher costs to the healthcare system. Existing evidence suggests that shared decision-making positively impacts medication initiation. The IMA-cRCT assesses the effectiveness of the IMA intervention in improving adherence and clinical parameters compared to usual care in patients with a new treatment for cardiovascular disease and diabetes prescribed in PC, and its cost-effectiveness, through a cRCT and economic modelling.

METHODS

The IMA intervention is a shared decision-making intervention based on the Theoretical Model of Non-initiation. A cRCT will be conducted in 24 PC teams in Catalonia (Spain), randomly assigned to the intervention group (1:1), and community pharmacies in the catchment areas of the intervention PC teams. Healthcare professionals in the intervention group will apply the intervention to all patients who receive a new prescription for cardiovascular disease or diabetes treatment (no other prescription from the same pharmacological group in the previous 6 months). All the study variables will be collected from real-world databases for the 12 months before and after receiving a new prescription. Effectiveness analyses will assess impact on initiation, secondary adherence, cardiovascular risk, clinical parameters and cardiovascular events. Cost-effectiveness analyses will be conducted as part of the cRCT from a healthcare and societal perspective in terms of extra cost per cardiovascular risk reduction and improved adherence; all analyses will be clustered. Economic models will be built to assess the long-term cost-effectiveness of the IMA intervention, in terms of extra cost for gains in QALY and life expectancy, using clinical trial data and data from previous studies.

DISCUSSION

The IMA-cRCT represents an innovative approach to the design and evaluation of behavioural interventions that use the principles of complex interventions, pragmatic trials and implementation research. This study will provide evidence on the IMA intervention and on a new methodology for developing and evaluating complex interventions. The results of the study will be disseminated among stakeholders to facilitate its transferability to clinical practice.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT05026775 . Registered 30th August 2021.

Real-World Evidence of Traditional Chinese Medicine (TCM) Treatment on Cancer: A Literature-Based Review

While randomized controlled trials (RCTs) are the gold standard for evidence-based medicine, they do not always reflect the real condition of patients in the real-world setting, which limits their generalizability and external validity. Real-world evidence (RWE), generated during routine clinical practice, is increasingly important in determining external effectiveness of the tightly controlled conditions of RCTs and is well recognized as a valuable complement to RCTs by regulatory bodies currently. Since it could provide new ideas and methods for clinical efficacy and safety evaluation of traditional Chinese medicine (TCM) and high-quality evidence support, real-world study (RWS) has received great attention in the field of medicine, especially in the field of TCM. RWS has shown desirable adaptability in the clinical diagnosis and treatment practice of traditional Chinese medicine. Consequently, it is increasingly essential for physicians and researchers to understand how RWE can be used alongside clinical trial data on TCM. Here, we discuss what real-world study is and outline the benefits and limitations of real-world study. Furthermore, using examples from TCM treatment on cancer, including Chinese herbal medicine, acupuncture, moxibustion, integrated TCM and Western medicine treatment, and other treatments, we elaborate how RWE can be used to help inform treatment decisions when doctoring patients with cancer in the clinic.

The GetReal Trial Tool: Design, Assess and Discuss Clinical Drug Trials in Light of RWE Generation

Methodologies incorporating Real World Elements into clinical trial design (also called pragmatic trials) offer an attractive opportunity to assess the effect of a treatment strategy in routine care and as such guide decision making in practice. Uptake of these methods is slow for several reasons, including uncertainty about acceptability of trial results, lack of experience with the methodology and operational challenges. We developed the 'Get Real Trial Tool', an easy-to-use interface, which allows users to assess the impact of design choices on generalisability to routine clinical practice, while taking into account risk of bias, precision, acceptability and operational feasibility. The tool is grounded in the scientific literature on pragmatic trials combined with knowledge of experts from academia, pharmaceutical companies, HTA bodies, patient organisations, and regulators. The aim is to help researchers optimise trial design and facilitate translation of evidence from pragmatic trials to clinical practice. In this paper we describe the development, structure and application of the GetReal Trial Tool.

Much ado about something: a response to "COVID-19: underpowered randomised trials, or no randomised trials?"

Non-pharmaceutical interventions (NPI) for infectious diseases such as COVID-19 are particularly challenging given the complexities of what is both practical and ethical to randomize. We are often faced with the difficult decision between having weak trials or not having a trial at all. In a recent article, Dr. Atle Fretheim argues that statistically underpowered studies are still valuable, particularly in conjunction with other similar studies in meta-analysis in the context of the DANMASK-19 trial, asking "Surely, some trial evidence must be better than no trial evidence?" However, informative trials are not always feasible, and feasible trials are not always informative. In some cases, even a well-conducted but weakly designed and/or underpowered trial such as DANMASK-19 may be uninformative or worse, both individually and in a body of literature. Meta-analysis, for example, can only resolve issues of statistical power if there is a reasonable expectation of compatible well-designed trials. Uninformative designs may also invite misinformation. Here, we make the case that-when considering informativeness, ethics, and opportunity costs in addition to statistical power-"nothing" is often the better choice.

The PRECIS-2 tool seems not to be useful to discriminate the degree of pragmatism of medicine masked trials from that of open-label trials

PURPOSE

To assess, with all available trial information, whether the assessment of the PRECIS-2 nine domains could provide a clear distinction between medicine masked pragmatic randomized controlled trials (pRCTs) and open-label pRCTs.

METHODS

A search was conducted of participant-level pRCTs on medicines published on 25 influential medical journals in July 2018-December 2019. All pre-licensing (phases 1-3) and cluster pRCTs were excluded. All trials' available reports were searched through the published article information, Google Scholar, and trial websites. Instead of providing a score to each PRECIS-2 domain, these were classified as E (explanatory), N (neutral), or P (pragmatic).

RESULTS

Of 128 pRCTs, 18 (14%) were participant-level pRCTs on medicines. The full trial protocol was available for 14 trials; 12 had published the protocol and nine had additional reports published. All trials were prospectively registered, and none was funded by industry. Ten and eight were masked and open-label trials, respectively. Masked pRCTS had 34% of pragmatic and 60% of explanatory domains; open-label pRCTS had 45% pragmatic and 45% explanatory domains. Among the 10 masked trials, only one had a majority of five pragmatic domains; among the eight open-label trials, four had a majority of six or five pragmatic domains. "Follow-up" was considered explanatory in the 18 pRCTs; "primary analysis" was pragmatic in 17 pRCTs.

CONCLUSION

The PRECIS-2 tool seems not to be sensitive enough to clearly discriminate between medicine masked pRCTs and open-label pRCTs. When conducting systematic reviews, it is suggested that the PRECIS-2 tool should not be used to support placing masked trials in the pragmatic side of the explanatory/pragmatic continuum.

Patient and healthcare professional experiences of the Salford Lung Studies: qualitative insights for future effectiveness trials

Background

Randomized controlled trials (RCTs) conducted in the routine care setting provide the opportunity to better understand the effectiveness of new medicines but can present recruitment difficulties. An improved understanding of the challenges/opportunities for patient and healthcare professional (HCP) engagement in clinical research is needed to enhance participation and trial experience. In this study, we explored patient and HCP drivers for, and experiences of, participation in the Salford Lung Studies (SLS), and their views on future trial participation and the overall value of such trials.

Methods

This was a qualitative study set in Salford, UK, comprising patient telephone interviews (N = 10) and HCP advisory boards (one with general practitioners [GPs], one with practice managers [PMs]); all individuals had participated in the SLS. Semi-structured telephone interviews were recorded, transcribed and analysed thematically. Advisory board meetings were analysed based on transcriptions of audio recordings and field notes.

Results

For patients, key positive aspects of the SLS were the ease/convenience of study assessments and excellent relationships with study nurses. GPs and PMs considered the SLS to be well-organized and highlighted the value of research nurse support; they also described minor challenges relating to trial systems, initial financial strain on practices and staff turnover. All participants indicated that they were very likely to participate in future trials, citing a design closely aligned with routine care practice as essential. Several strategies to encourage trial participation were suggested, such as clearly communicating benefits to patients and ensuring flexible study assessments.

Conclusions

Patients and HCPs had positive experiences of the SLS. The study design, closely aligned with routine care, was considered important to their high likelihood of participating in future trials. The experiences of patients and HCPs in the SLS provide valuable insights that will help inform future best practice in the design and conduct of future real-world effectiveness RCTs in primary care. The detailed first-hand experiences of HCPs will be of significant value to others considering engaging in clinical research and participating in effectiveness RCTs.

A limited number of medicines pragmatic trials had potential for waived informed consent following the 2016 CIOMS ethical guidelines

OBJECTIVES

European regulations do not allow modification or waiver of informed consent for medicines randomized controlled trials (RCTs) where the three 2016 Council for International Organizations of Medical Sciences (CIOMS) provisions are met (consent would be impractical or unfeasible, yet the trial would have high social value and pose no or minimal risk to participants). We aimed to identify whether any such trials of medicines were being conducted in Europe.

STUDY DESIGN AND SETTING

This is a survey of all phase 4 "ongoing" RCTs on the EU clinical trial register between July 1, 2016 and June 30, 2018, to identify those with potentially high levels of pragmatism. Trials that were excluded were as follows: those conducted on rare diseases; conducted on healthy volunteers (except those assessing vaccines); masked (single-, double-blind) trials; single-center trials; those where one could expect to lead patients to prefer one intervention over the other; and miscellaneous reasons. The degree of pragmatism of the RCTs was self-assessed by trials' investigators by means of the PRECIS-2 tool. Investigators of those trials considered to be highly pragmatic assessed the fulfillment of the three CIOMS provisions. Seven patients assessed the social value of the RCTs. Finally, 33 members of 11 research ethics committees (RECs) assessed the social value of the trials and whether they posed no more than minimal risk to participants. Investigators, patients, and REC members assessed the fulfillment of the CIOMS provisions as "yes," "not sure" or "no."

RESULTS

Of the 638 phase 4 trials, 420 were RCTs, and 21 of these (5%) were candidates to be pragmatic. Investigators of 15 of these 21 RCTs self-assessed their trial's degree of pragmatism: 14 were highly pragmatic. Of these 14, eight fulfilled the three CIOMS provisions. Assessments by patients and RECs were inconsistent for several trials.

CONCLUSIONS

We found few low-risk participant-level pragmatic RCTs that could be suitable for modified or waived participants' informed consent. European regulators should consider amending the current regulation and encouraging the conduct of such trials.

Preimplantation genetic testing for aneuploidy by microarray analysis of polar bodies in advanced maternal age: a randomized clinical trial

STUDY QUESTION

Does preimplantation genetic testing for aneuploidy (PGT-A) by comprehensive chromosome screening (CCS) of the first and second polar body to select embryos for transfer increase the likelihood of a live birth within 1 year in advanced maternal age women aged 36-40 years planning an ICSI cycle, compared to ICSI without chromosome analysis?

SUMMARY ANSWER

PGT-A by CCS in the first and second polar body to select euploid embryos for transfer does not substantially increase the live birth rate in women aged 36-40 years.

WHAT IS KNOWN ALREADY

PGT-A has been used widely to select embryos for transfer in ICSI treatment, with the aim of improving treatment effectiveness. Whether PGT-A improves ICSI outcomes and is beneficial to the patients has remained controversial.

STUDY DESIGN, SIZE, DURATION

This is a multinational, multicentre, pragmatic, randomized clinical trial with intention-to-treat analysis. Of 396 women enroled between June 2012 and December 2016, 205 were allocated to CCS of the first and second polar body (study group) as part of their ICSI treatment cycle and 191 were allocated to ICSI treatment without chromosome screening (control group). Block randomization was performed stratified for centre and age group. Participants and clinicians were blinded at the time of enrolment until the day after intervention.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Infertile couples in which the female partner was 36-40 years old and who were scheduled to undergo ICSI treatment were eligible. In those assigned to PGT-A, array comparative genomic hybridization (aCGH) analysis of the first and second polar bodies of the fertilized oocytes was performed using the 24sure array of Illumina. If in the first treatment cycle all oocytes were aneuploid, a second treatment with PB array CGH was offered. Participants in the control arm were planned for ICSI without PGT-A. Main exclusion criteria were three or more previous unsuccessful IVF or ICSI cycles, three or more clinical miscarriages, poor response or low ovarian reserve. The primary outcome was the cumulative live birth rate after fresh or frozen embryo transfer recorded over 1 year after the start of the intervention.

MAIN RESULTS AND THE ROLE OF CHANCE

Of the 205 participants in the chromosome screening group, 50 (24%) had a live birth with intervention within 1 year, compared to 45 of the 191 in the group without intervention (24%), a difference of 0.83% (95% CI: -7.60 to 9.18%). There were significantly fewer participants in the chromosome screening group with a transfer (relative risk (RR) = 0.81; 95% CI: 0.74-0.89) and fewer with a miscarriage (RR = 0.48; 95% CI: 0.26-0.90).

LIMITATIONS, REASONS FOR CAUTION

The targeted sample size was not reached because of suboptimal recruitment; however, the included sample allowed a 90% power to detect the targeted increase. Cumulative outcome data were limited to 1 year. Only 11 patients out of 32 with exclusively aneuploid results underwent a second treatment cycle in the chromosome screening group.

WIDER IMPLICATIONS OF THE FINDINGS

The observation that the similarity in birth rates was achieved with fewer transfers, less cryopreservation and fewer miscarriages points to a clinical benefit of PGT-A, and this form of embryo selection may, therefore, be considered to minimize the number of interventions while producing comparable outcomes. Whether these benefits outweigh drawbacks such as the cost for the patient, the higher workload for the IVF lab and the potential effect on the children born after prolonged culture and/or cryopreservation remains to be shown.

STUDY FUNDING/COMPETING INTEREST(S)

This study was funded by the European Society of Human Reproduction and Embryology. Illumina provided microarrays and other consumables necessary for aCGH testing of polar bodies. M.B.'s institution (UZBrussel) has received educational grants from IBSA, Ferring, Organon, Schering-Plough, Merck and Merck Belgium. M.B. has received consultancy and speakers' fees from Organon, Serono Symposia and Merck. G.G. has received personal fees and non-financial support from MSD, Ferring, Merck-Serono, Finox, TEVA, IBSA, Glycotope, Abbott and Gedeon-Richter as well as personal fees from VitroLife, NMC Healthcare, ReprodWissen, BioSilu and ZIVA. W.V., C.S., P.M.B., V.G., G.A., M.D., T.E.G., L.G., G.Ka., G.Ko., J.L., M.C.M., M.P., A.S., M.T., K.V., J.G. and K.S. declare no conflict of interest.

TRIAL REGISTRATION NUMBER

NCT01532284.

TRIAL REGISTRATION DATE

7 February 2012.

DATE OF FIRST PATIENT’S ENROLMENT

25 June 2012.

Rare diseases under different levels of economic analysis: current activities, challenges and perspectives

Rare diseases imply clinical and economic burden as well as a significant challenge for health systems. One relevant objective of the activities planned within the European Reference Network on Rare and Complex Connective Tissue and Musculoskeletal Diseases (ERN ReCONNET) is to address the economic dimensions of rare diseases to identify, develop and suggest strategies to improve research and patients' access to orphan drugs (ODs) and highly specialised health technologies. This paper presents a preliminary review of the existing policies on rare diseases in the countries of the Network members. It also introduces and discusses the theme of how to perform health economic evaluations of rare diseases and of existing or new treatments for rare diseases. To obtain a preliminary overview aiming at defining the state of the art of rare diseases policies and initiatives in ERN ReCONNET countries, we collected and analysed the rare diseases national plans of all the eight countries of the ERN ReCONNET participants. The preliminary overview that has been performed showed that in all the ERN ReCONNET countries are in place national plans for rare diseases; however, heterogeneity exists in the reimbursement of ODs, direct provision by the healthcare system, involvement of patients' associations in decision making and implementation of clinical practice guidelines.

Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic?

INTRODUCTION

Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as "pragmatic" or "naturalistic". We focused on 89 of these trials that assessed medicines (drugs or biologics).

DISCUSSION

36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term 'pragmatic' to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as 'pragmatic' and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic).

CONCLUSIONS

To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice.

Commentary on "Regulatory Support Improves Subsequent IRB/REC Approval Rates in Studies Initially Deemed Not Ready for Review: A CTSA Institution's Experience"

In response to researcher concerns a number of initiatives have been developed to support individual researchers seeking ethics review and approval. In this issue, Sonne et al. (2017) outline an example of an intervention to support researchers, which they refer to as a Regulatory Knowledge Support (RKS) service. While the study points to potential benefits, other studies have not had the desired impact on key performance measures. There is a need to develop a community of practice and expand the burgeoning evidence base regarding what interventions work, for whom, and under what circumstances. Advancing the research agenda requires: the development of theoretical models for intervention design and evaluation; developing consensus on key data for collection and measures of effectiveness; conducting evaluations using the strongest possible study designs, and; publishing the findings of evaluations.

Rare diseases under different levels of economic analysis: current activities, challenges and perspectives

Rare diseases imply clinical and economic burden as well as a significant challenge for health systems. One relevant objective of the activities planned within the European Reference Network on Rare and Complex Connective Tissue and Musculoskeletal Diseases (ERN ReCONNET) is to address the economic dimensions of rare diseases to identify, develop and suggest strategies to improve research and patients' access to orphan drugs (ODs) and highly specialised health technologies. This paper presents a preliminary review of the existing policies on rare diseases in the countries of the Network members. It also introduces and discusses the theme of how to perform health economic evaluations of rare diseases and of existing or new treatments for rare diseases. To obtain a preliminary overview aiming at defining the state of the art of rare diseases policies and initiatives in ERN ReCONNET countries, we collected and analysed the rare diseases national plans of all the eight countries of the ERN ReCONNET participants. The preliminary overview that has been performed showed that in all the ERN ReCONNET countries are in place national plans for rare diseases; however, heterogeneity exists in the reimbursement of ODs, direct provision by the healthcare system, involvement of patients' associations in decision making and implementation of clinical practice guidelines.

Series: Pragmatic trials and real world evidence: Paper 8. Data collection and management

Pragmatic trials can improve our understanding of how treatments will perform in routine practice. In a series of eight papers, the GetReal Consortium has evaluated the challenges in designing and conducting pragmatic trials and their specific methodological, operational, regulatory, and ethical implications. The present final paper of the series discusses the operational and methodological challenges of data collection in pragmatic trials. A more pragmatic data collection needs to balance the delivery of highly accurate and complete data with minimizing the level of interference that data entry and verification induce with clinical practice. Furthermore, it should allow for the involvement of a representative sample of practices, physicians, and patients who prescribe/receive treatment in routine care. This paper discusses challenges that are related to the different methods of data collection and presents potential solutions where possible. No one-size-fits-all recommendation can be given for the collection of data in pragmatic trials, although in general the application of existing routinely used data-collection systems and processes seems to best suit the pragmatic approach. However, data access and privacy, the time points of data collection, the level of detail in the data, and the lack of a clear understanding of the data-collection process were identified as main challenges for the usage of routinely collected data in pragmatic trials. A first step should be to determine to what extent existing health care databases provide the necessary study data and can accommodate data collection and management. When more elaborate or detailed data collection or more structured follow-up is required, data collection in a pragmatic trial will have to be tailor-made, often using a hybrid approach using a dedicated electronic case report form (eCRF). In this case, the eCRF should be kept as simple as possible to reduce the burden for practitioners and minimize influence on routine clinical practice.