Premarket Evidence and Postmarketing Requirements for Real-Time Oncology Review Indication Approvals

This cross-sectional study evaluates the use of the US Food and Drug Administration’s Real-Time Oncology Review (RTOR) program in confirming the effectiveness of cancer drugs.

Associations Between Surrogate Markers and Clinical Outcomes for Nononcologic Chronic Disease Treatments

Importance

Surrogate markers are increasingly used as primary end points in clinical trials supporting drug approvals.

Objective

To systematically summarize the evidence from meta-analyses, systematic reviews and meta-analyses, and pooled analyses (hereafter, meta-analyses) of clinical trials examining the strength of association between treatment effects measured using surrogate markers and clinical outcomes in nononcologic chronic diseases.

Data sources

The Food and Drug Administration (FDA) Adult Surrogate Endpoint Table and MEDLINE from inception to March 19, 2023.

Study Selection

Three reviewers selected meta-analyses of clinical trials; meta-analyses of observational studies were excluded.

Data Extraction and Synthesis

Two reviewers extracted correlation coefficients, coefficients of determination, slopes, effect estimates, or results from meta-regression analyses between surrogate markers and clinical outcomes.

Main Outcomes and Measures

Correlation coefficient or coefficient of determination, when reported, was classified as high strength (r ≥ 0.85 or R2 ≥ 0.72); primary findings were otherwise summarized.

Results

Thirty-seven surrogate markers listed in FDA's table and used as primary end points in clinical trials across 32 unique nononcologic chronic diseases were included. For 22 (59%) surrogate markers (21 chronic diseases), no eligible meta-analysis was identified. For 15 (41%) surrogate markers (14 chronic diseases), at least 1 meta-analysis was identified, 54 in total (median per surrogate marker, 2.5; IQR, 1.3-6.0); among these, median number of trials and patients meta-analyzed was 18.5 (IQR, 12.0-43.0) and 90 056 (IQR, 20 109-170 014), respectively. The 54 meta-analyses reported 109 unique surrogate marker-clinical outcome pairs: 59 (54%) reported at least 1 r or R2, 10 (17%) of which reported at least 1 classified as high strength, whereas 50 (46%) reported slopes, effect estimates, or results of meta-regression analyses only, 26 (52%) of which reported at least 1 statistically significant result.

Conclusions and Relevance

Most surrogate markers used as primary end points in clinical trials to support FDA approval of drugs treating nononcologic chronic diseases lacked high-strength evidence of associations with clinical outcomes from published meta-analyses.

Characterization of accelerated approval status, trial endpoints and results, and recommendations in guidelines for oncology drug treatments from the National Comprehensive Cancer Network: cross sectional study

Objectives

To evaluate National Comprehensive Cancer Network (NCCN) guideline recommendations for oncology drug treatments that have been granted accelerated approval, and to determine whether recommendations are updated based on the results of confirmatory trials after approval and based on status updates from the US Food and Drug Administration (FDA).

Design

Cross sectional study.

Setting

US FDA and NCCN guidelines.

Population

Oncology therapeutic indications (ie, specific oncological conditions for which the drug is recommended) that have been granted accelerated approval in 2009-18.

Main outcome measures

NCCN guideline reporting of accelerated approval status and postapproval confirmatory trials, and guideline recommendation alignment with postapproval confirmatory trial results and FDA status updates.

Results

39 oncology drug treatments were granted accelerated approval for 62 oncological indications. Although all indications were recommended in NCCN guidelines, accelerated approval status was reported for 10 (16%) indications. At least one postapproval confirmatory trial was identified for all 62 indications, 33 (53%) of which confirmed benefit; among these indications, NCCN guidelines maintained the previous recommendation or strengthened the category of evidence for 27 (82%). Postapproval confirmatory trials failed to confirm benefit for 12 (19%) indications; among these indications, NCCN guidelines removed the previous recommendation or weakened the category of evidence for five (42%). NCCN guidelines reflected the FDA's decision to convert 30 (83%) of 36 indications from accelerated to traditional approval, of which 20 (67%) had guideline updates before the FDA's conversion decision. NCCN guidelines reflected the FDA's decision to withdraw seven (58%) of 12 indications from the market, of which four (57%) had guidelines updates before the FDA's withdrawal decision.

Conclusions

NCCN guidelines always recommend drug treatments that have been granted accelerated approval for oncological indications, but do not provide information about their accelerated approval status, including surrogate endpoint use and status of postapproval confirmatory trials. NCCN guidelines consistently provide information on postapproval trial results confirming clinical benefit, but not on postapproval trials failing to confirm clinical benefit. NCCN guidelines more frequently update recommendation for indications converted to traditional approval than for those approvals that were withdrawn.

Grilling the data: application of specification curve analysis to red meat and all-cause mortality

OBJECTIVES

To present an application of specification curve analysis-a novel analytic method that involves defining and implementing all plausible and valid analytic approaches for addressing a research question-to nutritional epidemiology.

STUDY DESIGN AND SETTING

We reviewed all observational studies addressing the effect of red meat on all-cause mortality, sourced from a published systematic review, and documented variations in analytic methods (eg, choice of model, covariates, etc.). We enumerated all defensible combinations of analytic choices to produce a comprehensive list of all the ways in which the data may reasonably be analyzed. We applied specification curve analysis to data from National Health and Nutrition Examination Survey 2007 to 2014 to investigate the effect of unprocessed red meat on all-cause mortality. The specification curve analysis used a random sample of all reasonable analytic specifications we sourced from primary studies.

RESULTS

Among 15 publications reporting on 24 cohorts included in the systematic review on red meat and all-cause mortality, we identified 70 unique analytic methods, each including different analytic models, covariates, and operationalizations of red meat (eg, continuous vs quantiles). We applied specification curve analysis to National Health and Nutrition Examination Survey, including 10,661 participants. Our specification curve analysis included 1208 unique analytic specifications, of which 435 (36.0%) yielded a hazard ratio equal to or more than 1 for the effect of red meat on all-cause mortality and 773 (64.0%) less than 1. The specification curve analysis yielded a median hazard ratio of 0.94 (interquartile range: 0.83-1.05). Forty-eight specifications (3.97%) were statistically significant, 40 of which indicated unprocessed red meat to reduce all-cause mortality and eight of which indicated red meat to increase mortality.

CONCLUSION

We show that the application of specification curve analysis to nutritional epidemiology is feasible and presents an innovative solution to analytic flexibility.

Published research on the human health implications of climate change between 2012 and 2021: cross sectional study

Objective

To better understand the state of research on the effects of climate change on human health, including exposures, health conditions, populations, areas of the world studied, funding sources, and publication characteristics, with a focus on topics that are relevant for populations at risk.

Design

Cross sectional study.

Data sources

The National Institute of Environmental Health Sciences climate change and human health literature portal, a curated bibliographical database of global peer reviewed research and grey literature was searched. The database combines searches of multiple search engines including PubMed, Web of Science, and Google Scholar, and includes added-value expert tagging of climate change exposures and health impacts.

Eligibility criteria

Inclusion criteria were peer reviewed, original research articles that investigated the health effects of climate change and were published in English from 2012 to 2021. After identification, a 10% random sample was selected to manually perform a detailed characterisation of research topics and publication information.

Results

10 325 original research articles were published between 2012 and 2021, and the number of articles increased by 23% annually. In a random sample of 1014 articles, several gaps were found in research topics that are particularly relevant to populations at risk, such as those in the global south (134 countries established through the United Nations Office for South-South Cooperation) (n=444; 43.8%), adults aged 65 years or older (n=195; 19.2%), and on topics related to human conflict and migration (n=25; 2.5%) and food and water quality and security (n=148; 14.6%). Additionally, fewer first authors were from the global south (n=349; 34.4%), which may partly explain why research focusing on these countries is disproportionally less.

Conclusions

Although the body of research on the health effects of climate change has grown substantially over the past decade, including those with a focus on the global south, a disproportionate focus continues to be on countries in the global north and less at risk populations. Governments are the largest source of funding for such research, and governments, particularly in the global north, need to re-orient their climate and health research funding to support researchers in the global south and to be more inclusive of issues that are relevant to the global south.

Autoimmune diseases and risk of non-Hodgkin lymphoma: A Mendelian randomisation study

Objective

To examine whether genetically predicted susceptibility to ten autoimmune diseases (Behçet's disease, coeliac disease, dermatitis herpetiformis, lupus, psoriasis, rheumatoid arthritis, sarcoidosis, Sjögren's syndrome, systemic sclerosis, and type 1 diabetes) is associated with risk of non-Hodgkin lymphoma (NHL).

Design

Two sample Mendelian randomization (MR) study.

Setting

Genome wide association studies (GWASs) of ten autoimmune diseases, NHL, and four NHL subtypes (i.e., follicular lymphoma, mature T/natural killer-cell lymphomas, non-follicular lymphoma, and other and unspecified types of NHL).

Analysis

We used data from the largest publicly available GWASs of European ancestry for each autoimmune disease, NHL, and NHL subtypes. For each autoimmune disease, we extracted single nucleotide polymorphisms (SNPs) strongly associated (P < 5×10-8) with that disease and that were independent of one another (R2 < 1×10-3) as genetic instruments. SNPs within the human leukocyte antigen region were not considered due to potential pleiotropy. Our primary MR analysis was the inverse-variance weighted analysis. Additionally, we conducted MR-Egger, weighted mode, and weighted median regression to address potential bias due to pleiotropy, and robust adjusted profile scores to address weak instrument bias. We carried out sensitivity analysis limited to the non-immune pathway for nominally significant findings. To account for multiple testing, we set the thresholds for statistical significance at P < 5×10-3.

Participants

The number of cases and controls identified in the relevant GWASs were 437 and 3,325 for Behçet's disease, 4,918 and 5,684 for coeliac disease, 435 and 341,188 for dermatitis herpetiformis, 4,576 and 8,039 for lupus, 11,988 and 275,335 for psoriasis, 22,350 and 74,823 for rheumatoid arthritis, 3,597 and 337,121 for sarcoidosis, 2,735 and 332,115 for Sjögren's syndrome, 9,095 and 17,584 for systemic sclerosis, 18,942 and 501,638 for type 1 diabetes, 2,400 and 410,350 for NHL; and 296 to 2,340 cases and 271,463 controls for NHL subtypes.

Exposures

Genetic variants predicting ten autoimmune diseases: Behçet's disease, coeliac disease, dermatitis herpetiformis, lupus, psoriasis, rheumatoid arthritis, sarcoidosis, Sjögren's syndrome, systemic sclerosis, and type 1 diabetes.

Main outcome measures

Estimated associations between genetically predicted susceptibility to ten autoimmune diseases and the risk of NHL.

Results

The variance of each autoimmune disease explained by the SNPs ranged from 0.3% to 3.1%. Negative associations between type 1 diabetes and sarcoidosis and the risk of NHL were observed (odds ratio [OR] 0.95, 95% confidence interval [CI]: 0.92 to 0.98, P = 5×10-3, and OR 0.92, 95% CI: 0.85 to 0.99, P = 2.8×10-2, respectively). These findings were supported by the sensitivity analyses accounting for potential pleiotropy and weak instrument bias. No significant associations were found between the other eight autoimmune diseases and NHL risk. Of the NHL subtypes, type 1 diabetes was most strongly associated with follicular lymphoma (OR 0.91, 95% CI: 0.86 to 0.96, P = 1×10-3), while sarcoidosis was most strongly associated with other and unspecified NHL (OR 0.86, 95% CI: 0.75 to 0.97, P = 1.8×10-2).

Conclusions

These findings suggest that genetically predicted susceptibility to type 1 diabetes, and to some extent sarcoidosis, might reduce the risk of NHL. However, future studies with different datasets, approaches, and populations are warranted to further examine the potential associations between these autoimmune diseases and the risk of NHL.

Evidence of subgroup differences in meta-analyses evaluating medications for alcohol use disorder: An umbrella review

Randomized controlled trials (RCTs) evaluating medications for alcohol use disorder (AUD) often examine heterogeneity of treatment effects through subgroup analyses that contrast effect estimates in groups of patients across individual demographic, clinical, and study design-related characteristics. However, these analyses are often not prespecified or adequately powered, highlighting the potential role of subgroup analyses in meta-analysis. Here, we conducted an umbrella review (i.e., a systematic review of meta-analyses) to determine the range and characteristics of reported subgroup analyses in meta-analyses of AUD medications. We searched PubMed to identify meta-analyses of RCTs evaluating medications for the management of AUD, alcohol abuse, or alcohol dependence in adults. We sought studies that measured drinking-related outcomes; quality of life, function, and rates of mortality; adverse events; and dropout. We considered meta-analyses that reported the results from formal subgroup analyses (comparing the summary effects across subgroup levels); summary effect estimates stratified across subgroup levels; and meta-regression, regression, or correlation-based subgroup analyses. We analyzed nine meta-analyses that included 61 formal subgroup analyses (median = 6 per meta-analysis), of which 33 (54%) were based on baseline participant-level and 28 (46%) were based on trial-level characteristics. Of the 58 subgroup analyses with either a p-value from a subgroup test or a statement by the authors that the subgroup analyses were not statistically significant, eight (14%) were statistically significant at the p < 0.05 level. Twelve meta-analyses reported the results of 102 meta-regression analyses, of which 25 (25%) identified statistically significant predictors of the relevant outcome of interest; nine (9%) were based on baseline participant-level and 93 (91%) were based on trial characteristics. Subgroup analyses across meta-analyses of AUD medications often focus on study-level characteristics, which may not be as clinically informative as subgroup analyses based on participant-level characteristics. Opportunities exist for future meta-analyses to standardize their subgroup methodology, focus on more clinically informative participant-level characteristics, and use predictive approaches to account for multiple relevant variables.

Pivotal Trial Demographic Representation and Clinical Development Times for Oncology Therapeutics

This study evaluates whether FDA-approved novel cancer therapeutics supported by pivotal trials with adequate representation of minoritized groups were associated with slower clinical development times than those with inadequate representation.

Association of health benefits and harms of Christmas dessert ingredients in recipes from The Great British Bake Off: umbrella review of umbrella reviews of meta-analyses of observational studies

OBJECTIVE

To determine the health benefits and harms of various ingredients in Christmas desserts from The Great British Bake Off.

DESIGN

Umbrella review of umbrella reviews of meta-analyses of observational studies.

DATA SOURCES

The Great British Bake Off website, Embase, Medline, and Scopus.

INCLUSION CRITERIA

Umbrella reviews of meta-analyses of observational studies evaluating the associations between Christmas dessert ingredients and the risk of death or disease.

MAIN OUTCOME MEASURES

Proportion of protective and harmful summary associations between ingredient groups from The Great British Bake Off Christmas dessert recipes and the risk of death or disease.

RESULTS

48 recipes for Christmas desserts (ie, cakes, biscuits, pastries, and puddings and desserts) were provided on The Great British Bake Off website with 178 unique ingredients that were collapsed into 17 overarching ingredient groups. A literature search identified 7008 titles and abstracts, of which 46 eligible umbrella reviews reported 363 unique summary associations between the ingredient groups and risk of death or disease. Of these summary associations, 149 (41%) were significant, including 110 (74%) that estimated that the ingredient groups reduced the risk of death or disease and 39 (26%) that increased the risk. The most common ingredient groups associated with a reduced risk of death or disease were fruit (44/110, 40%), coffee (17/110, 16%), and nuts (14/110, 13%), whereas alcohol (20/39, 51%) and sugar (5/39, 13%) were the most common ingredient groups associated with increased risk of death or disease.

CONCLUSIONS

Recipes for Christmas desserts from The Great British Bake Off often use ingredient groups that are associated with reductions, rather than increases, in the risk of death or disease. This Christmas, if concerns about the limitations of observational nutrition research are set aside, you can have your cake and eat it too.

Assessing the use of observational methods and real-world data to emulate ongoing randomized controlled trials

BACKGROUND/AIMS

There has been growing interest in better understanding the potential of observational research methods in medical product evaluation and regulatory decision-making. Previously, we used linked claims and electronic health record data to emulate two ongoing randomized controlled trials, characterizing the populations and results of each randomized controlled trial prior to publication of its results. Here, our objective was to compare the populations and results from the emulated trials with those of the now-published randomized controlled trials.

METHODS

This study compared participants' demographic and clinical characteristics and study results between the emulated trials, which used structured data from OptumLabs Data Warehouse, and the published PRONOUNCE and GRADE trials. First, we examined the feasibility of implementing the baseline participant characteristics included in the published PRONOUNCE and GRADE trials' using real-world data and classified each variable as ascertainable, partially ascertainable, or not ascertainable. Second, we compared the emulated trials and published randomized controlled trials for baseline patient characteristics (concordance determined using standardized mean differences <0.20) and results of the primary and secondary endpoints (concordance determined by direction of effect estimates and statistical significance).

RESULTS

The PRONOUNCE trial enrolled 544 participants, and the emulated trial included 2226 propensity score-matched participants. In the PRONOUNCE trial publication, one of the 32 baseline participant characteristics was listed as an exclusion criterion on ClinicalTrials.gov but was ultimately not used. Among the remaining 31 characteristics, 9 (29.0%) were ascertainable, 11 (35.5%) were partially ascertainable, and 10 (32.2%) were not ascertainable using structured data from OptumLabs. For one additional variable, the PRONOUNCE trial did not provide sufficient detail to allow its ascertainment. Of the nine variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 6 (66.7%). The primary endpoint of time from randomization to the first major adverse cardiovascular event and secondary endpoints of nonfatal myocardial infarction and stroke were concordant between the emulated trial and published randomized controlled trial. The GRADE trial enrolled 5047 participants, and the emulated trial included 7540 participants. In the GRADE trial publication, 8 of 34 (23.5%) baseline participant characteristics were ascertainable, 14 (41.2%) were partially ascertainable, and 11 (32.4%) were not ascertainable using structured data from OptumLabs. For one variable, the GRADE trial did not provide sufficient detail to allow for ascertainment. Of the eight variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 4 (50.0%). The primary endpoint of time to hemoglobin A1c ≥7.0% was mostly concordant between the emulated trial and the published randomized controlled trial.

CONCLUSION

Despite challenges, observational methods and real-world data can be leveraged in certain important situations for a more timely evaluation of drug effectiveness and safety in more diverse and representative patient populations.

Systematic review and meta-analysis of the genetics of peripheral arterial disease

Background

Peripheral artery disease (PAD) impacts more than 200 million people worldwide. The understanding of the genetics of the disease and its clinical implications continue to evolve. This systematic review provides a comprehensive summary of all DNA variants that have been studied in association with the diagnosis and progression of PAD, with a meta-analysis of the ones replicated in the literature.

Methods

A systematic review of all studies examining DNA variants associated with the diagnosis and progression of PAD was performed. Candidate gene and genome-wide association studies (GWAS) were included. A meta-analysis of 13 variants derived from earlier smaller candidate gene studies of the diagnosis of PAD was performed. The literature on the progression of PAD was limited, and a meta-analysis was not feasible because of the heterogeneity in the criteria used to characterize it.

Results

A total of 231 DNA variants in 112 papers were studied for the association with the diagnosis of PAD. There were significant variations in the definition of PAD and the selection of controls in the various studies. GWAS have established 19 variants associated with the diagnosis of PAD that were replicated in several large patient cohorts. Only variants in intercellular adhesion molecule-1 (rs5498), IL-6 (rs1800795), and hepatic lipase (rs2070895) showed significant association with the diagnosis of PAD. However, these variants were not noted in the published GWAS.

Conclusions

Genetic research in the diagnosis of PAD has significant heterogeneity, but recent GWAS have demonstrated variants consistently associated with the disease. More research focusing on the progression of PAD is needed to identify patients at risk of adverse events and develop strategies that would improve their outcomes.

Feasibility of Emulating Clinical Trials Supporting US FDA Supplemental Indication Approvals of Drugs and Biologics

This cross-sectional study evaluates the supporting clinical trials for supplemental new drug applications and supplemental biologics license applications from 2017 to 2019.

Comparative effectiveness of second line glucose lowering drug treatments using real world data: emulation of a target trial

Objective

To build on the recently completed GRADE (Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness Study) randomised trial examining the comparative effectiveness of second line glucose lowering drugs in achieving and maintaining glycaemic control in adults with type 2 diabetes.

Design

Emulation of a target trial.

Setting

Medical and pharmacy claims data from the OptumLabs Data Warehouse, a de-identified US national dataset of beneficiaries of commercially insured and Medicare Advantage plans, 29 March 2013 to 30 June 2021.

Participants

Adults (≥18 years) with type 2 diabetes who first started taking glimepiride, sitagliptin, liraglutide, insulin glargine, or canagliflozin between 29 March 2013 and 30 June 2021. Participants were treatment naive or were receiving metformin monotherapy at the time of starting the study drug.

Main outcome measures

The main outcomes were time to primary and secondary metabolic failure of the assigned treatment, calculated as days to haemoglobin A1c levels of ≥7.0% and >7.5%, respectively. Secondary metabolic, cardiovascular, and microvascular outcomes were analysed as specified in the GRADE statistical analysis plan. Propensity scores were estimated with the gradient boosting method, and inverse propensity score weighting was used to emulate randomisation to the treatment groups, which were then compared with Cox proportional hazards regression.

Results

The study cohort included participants starting treatment with glimepiride (n=20 511), liraglutide (n=5569), sitagliptin (n=13 039), insulin glargine (n=7262), and canagliflozin (n=5290). The insulin glargine arm was excluded because of insufficient control of confounding. Median times to primary metabolic failure were 439 (95% confidence interval 400 to 489) days in the canagliflozin arm, 439 (426 to 453) days in the glimepiride arm, 624 (567 to 731) days in the liraglutide arm, and 461 (442 to 482) days in the sitagliptin arm. Median time to secondary metabolic failure was also longest in the liraglutide arm. Adults receiving liraglutide had the lowest one year cumulative incidence rate of primary metabolic failure (0.37, 95% confidence interval 0.35 to 0.40) followed by sitagliptin (0.44, 0.43 to 0.45), glimepiride (0.45, 0.44 to 0.45), and canagliflozin (0.46, 0.44 to 0.48). Similarly, the one year cumulative incidence rate of secondary metabolic failure was 0.27 (0.25 to 0.29) in the canagliflozin arm, 0.28 (0.27 to 0.29) in the glimepiride arm, 0.23 (0.21 to 0.26) in the liraglutide arm, and 0.28 (0.27 to 0.29) in the sitagliptin arm. No differences were observed between the study arms in the rates of microvascular and macrovascular complications.

Conclusions

In this target trial emulation of an expanded GRADE study framework, liraglutide was more effective in achieving and maintaining glycaemic control as a second line glucose lowering drug than canagliflozin, sitagliptin, or glimepiride.

Use of Expedited Regulatory Programs and Clinical Development Times for FDA-Approved Novel Therapeutics

This cross-sectional study evaluates the duration between application to US Food and Drug Administration (FDA) and approval for new drugs and biologics in the US from 2015 to 2022.

Consistency between trials presented at conferences, their subsequent publications and press releases

OBJECTIVE

This study examined the extent to which trials presented at major international medical conferences in 2016 consistently reported their study design, end points and results across conference abstracts, published article abstracts and press releases.

DESIGN

Cross-sectional analysis of clinical trials presented at 12 major medical conferences in the USA in 2016. Conferences were identified from a list of the largest clinical research meetings aggregated by the Healthcare Convention and Exhibitors Association and were included if their abstracts were publicly available. From these conferences, all late-breaker clinical trials were included, as well as a random selection of all other clinical trials, such that the total sample included up to 25 trial abstracts per conference.

MAIN OUTCOME MEASURES

First, it was determined if trials were registered and reported results in an International Committee of Medical Journal Editors-approved clinical trial registry. Second, it was determined if trial results were published in a peer-reviewed journal. Finally, information on trial media coverage and press releases was collected using LexisNexis. For all published trials, the consistency of reporting of the following characteristics was examined, through comparison of the trials' conference and publication abstracts: primary efficacy endpoint definition, safety endpoint identification, sample size, follow-up period, primary end point effect size and characterisation of trial results. For all published abstracts with press releases, the characterisation of trial results across conference abstracts, press releases and publications was compared. Authors determined consistency of reporting when identical information was presented across abstracts and press releases. Primary analyses were descriptive; secondary analyses included χ2 tests and multiple logistic regression.

RESULTS

Among 240 clinical trials presented at 12 major medical conferences, 208 (86.7%) were registered, 95 (39.6%) reported summary results in a registry and 177 (73.8%) were published; 82 (34.2%) were covered by the media and 68 (28.3%) had press releases. Among the 177 published trials, 171 (96.6%) reported the definition of primary efficacy endpoints consistently across conference and publication abstracts, whereas 96/128 (75.0%) consistently identified safety endpoints. There were 107/172 (62.2%) trials with consistent sample sizes across conference and publication abstracts, 101/137 (73.7%) that reported their follow-up periods consistently, 92/175 (52.6%) that described their effect sizes consistently and 157/175 (89.7%) that characterised their results consistently. Among the trials that were published and had press releases, 32/32 (100%) characterised their results consistently across conference abstracts, press releases and publication abstracts. No trial characteristics were associated with reporting primary efficacy end points consistently.

CONCLUSIONS

For clinical trials presented at major medical conferences, primary efficacy endpoint definitions were consistently reported and results were consistently characterised across conference abstracts, registry entries and publication abstracts; consistency rates were lower for sample sizes, follow-up periods, and effect size estimates.

REGISTRATION

This study was registered at the Open Science Framework (https://doi.org/10.17605/OSF.IO/VGXZY).

Comparison of Clinical Study Results Reported in medRxiv Preprints vs Peer-reviewed Journal Articles

IMPORTANCE

Preprints have been widely adopted to enhance the timely dissemination of research across many scientific fields. Concerns remain that early, public access to preliminary medical research has the potential to propagate misleading or faulty research that has been conducted or interpreted in error.

OBJECTIVE

To evaluate the concordance among study characteristics, results, and interpretations described in preprints of clinical studies posted to medRxiv that are subsequently published in peer-reviewed journals (preprint-journal article pairs).

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study assessed all preprints describing clinical studies that were initially posted to medRxiv in September 2020 and subsequently published in a peer-reviewed journal as of September 15, 2022.

MAIN OUTCOMES AND MEASURES

For preprint-journal article pairs describing clinical trials, observational studies, and meta-analyses that measured health-related outcomes, the sample size, primary end points, corresponding results, and overarching conclusions were abstracted and compared. Sample size and results from primary end points were considered concordant if they had exact numerical equivalence.

RESULTS

Among 1399 preprints first posted on medRxiv in September 2020, a total of 1077 (77.0%) had been published as of September 15, 2022, a median of 6 months (IQR, 3-8 months) after preprint posting. Of the 547 preprint-journal article pairs describing clinical trials, observational studies, or meta-analyses, 293 (53.6%) were related to COVID-19. Of the 535 pairs reporting sample sizes in both sources, 462 (86.4%) were concordant; 43 (58.9%) of the 73 pairs with discordant sample sizes had larger samples in the journal publication. There were 534 pairs (97.6%) with concordant and 13 pairs (2.4%) with discordant primary end points. Of the 535 pairs with numerical results for the primary end points, 434 (81.1%) had concordant primary end point results; 66 of the 101 discordant pairs (65.3%) had effect estimates that were in the same direction and were statistically consistent. Overall, 526 pairs (96.2%) had concordant study interpretations, including 82 of the 101 pairs (81.2%) with discordant primary end point results.

CONCLUSIONS AND RELEVANCE

Most clinical studies posted as preprints on medRxiv and subsequently published in peer-reviewed journals had concordant study characteristics, results, and final interpretations. With more than three-fourths of preprints published in journals within 24 months, these results may suggest that many preprints report findings that are consistent with the final peer-reviewed publications.

Characterization and corroboration of safety signals identified from the US Food and Drug Administration Adverse Event Reporting System, 2008-19: cross sectional study

OBJECTIVE

To characterize potential drug safety signals identified from the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS), from 2008 to 2019, to determine how often these signals resulted in regulatory action by the FDA and whether these actions were corroborated by published research findings or public assessments by the Sentinel Initiative.

DESIGN

Cross sectional study.

SETTING

USA.

POPULATION

Safety signals identified from the FAERS and publicly reported by the FDA between 2008 and 2019; and review of the relevant literature published before and after safety signals were reported in 2014-15. Literature searches were performed in November 2019, Sentinel Initiative assessments were searched in December 2021, and data analysis was finalized in December 2021.

MAIN OUTCOME MEASURES

Safety signals and resulting regulatory actions; number and characteristics of published studies, including corroboration of regulatory action as evidenced by significant associations (or no associations) between the drug related to the signal and the adverse event.

RESULTS

From 2008 to 2019, 603 potential safety signals identified from the FAERS were reported by the FDA (median 48 annually, interquartile range 41-61), of which 413 (68.5%) were resolved as of December 2021 (372 of 399 (93.2%) signals ≥3 years old were resolved). Among the resolved safety signals, 91 (22.0%) led to no regulatory action and 322 (78.0%) resulted in regulatory action, including 319 (77.2%) changes to drug labeling and 59 (14.3%) drug safety communications or other public communications from the FDA. For a subset of 82 potential safety signals reported in 2014-15, a literature search identified 1712 relevant publications; 1201 (70.2%) were case reports or case series. Among these 82 safety signals, 76 (92.7%) were resolved, of which relevant published research was identified for 57 (75.0%) signals and relevant Sentinel Initiative assessments for four (5.3%) signals. Regulatory actions by the FDA were corroborated by at least one relevant published research study for 17 of the 57 (29.8%) resolved safety signals; none of the relevant Sentinel Initiative assessments corroborated FDA regulatory action.

CONCLUSIONS

Most potential safety signals identified from the FAERS led to regulatory action by the FDA. Only a third of regulatory actions were corroborated by published research, however, and none by public assessments from the Sentinel Initiative. These findings suggest that either the FDA is taking regulatory actions based on evidence not made publicly available or more comprehensive safety evaluations might be needed when potential safety signals are identified.

Characteristics of Retracted Research Articles About COVID-19 vs Other Topics

This cross-sectional study compares the author and journal characteristics of retracted articles on COVID-19 with retracted articles from other topics.

Emulating the GRADE trial using real world data: retrospective comparative effectiveness study

OBJECTIVE

To emulate the GRADE (Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness Study) trial using real world data before its publication. GRADE directly compared second line glucose lowering drugs for their ability to lower glycated hemoglobin A_1c (HbA_1c).

DESIGN

Observational study.

SETTING

OptumLabs® Data Warehouse (OLDW), a nationwide claims database in the US, 25 January 2010 to 30 June 2019.

PARTICIPANTS

Adults with type 2 diabetes and HbA_1c 6.8-8.5% while using metformin monotherapy, identified according to the GRADE trial specifications, who also used glimepiride, liraglutide, sitagliptin, or insulin glargine.

MAIN OUTCOME MEASURES

The primary outcome was time to HbA_1c ≥7.0%. Secondary outcomes were time to HbA_1c >7.5%, incident microvascular complications, incident macrovascular complications, adverse events, all cause hospital admissions, and all cause mortality. Propensity scores were estimated using the gradient boosting machine method, and inverse propensity score weighting was used to emulate randomization of the treatment groups, which were then compared using Cox proportional hazards regression.

RESULTS

8252 people were identified (19.7% of adults starting the study drugs in OLDW) who met eligibility criteria for the GRADE trial (glimepiride arm=4318, liraglutide arm=690, sitagliptin arm=2993, glargine arm=251). The glargine arm was excluded from analyses owing to small sample size. Median times to HbA_1c ≥7.0% were 442 days (95% confidence interval 394 to 480 days) for glimepiride, 764 (741 to not calculable) days for liraglutide, and 427 (380 to 483) days for sitagliptin. Liraglutide was associated with lower risk of reaching HbA_1c ≥7.0% compared with glimepiride (hazard ratio 0.57, 95% confidence interval 0.43 to 0.75) and sitagliptin (0.55, 0.41 to 0.73). Results were consistent for the secondary outcome of time to HbA_1c >7.5%. No significant differences were observed among treatment groups for the remaining secondary outcomes.

CONCLUSIONS

In this emulation of the GRADE trial, liraglutide was statistically significantly more effective at maintaining glycemic control than glimepiride or sitagliptin when added to metformin monotherapy. Generating timely evidence on medical treatments using real world data as a complement to prospective trials is of value.

Vibration of effect in more than 16 000 pooled analyses of individual participant data from 12 randomised controlled trials comparing canagliflozin and placebo for type 2 diabetes mellitus: multiverse analysis

Objective

To evaluate the impact of conducting all possible pooled analyses across different combinations of randomised controlled trials and endpoints.

Design

Multiverse analysis, consisting of numerous pooled analyses of individual participant data.

Setting

Individual patient data from 12 randomised controlled trials comparing canagliflozin treatment with placebo, shared on the Yale University Open Data Access project (https://yoda.yale.edu/) platform, up to 16 April 2021.

Participants

15 094 people with type 2 diabetes mellitus.

Main outcome measures

Pooled analyses estimated changes in serum glycated haemoglobin (HbA1c), major adverse cardiovascular events, and serious adverse events at weeks 12, 18, 26, and 52. The distribution of effect estimates was calculated for all possible combinations, and the direction and magnitude of the first and 99th centiles of effect estimates were compared.

Results

Across 16 332 distinct pooled analyses comparing canagliflozin with placebo for changes in HbA1c, standardised effect estimates were in favour of canagliflozin treatment at both the first centile (-0.75%) and 99th centile (-0.48%); 15 994 (97.93%) analyses showed significant results (P<0.05) in favour of canagliflozin. For major adverse cardiovascular events, estimated hazard ratios were 0.20 at the first centile and 0.90 at the 99th centile; 2705 of 8144 analyses (33.21%) were significant, all of which were in favour of canagliflozin treatment. For serious adverse events, estimated hazard ratios were 0.59 at the first centile and 1.14 at the 99th centile; 5793 of 16 332 (35.47%) analyses were significant, with 5754 in favour of canagliflozin and 39 in favour of placebo.

Conclusion

Results from pooled analyses can be subject to vibration of effects and should be critically appraised, especially regarding the risk for selection and availability bias in individual participant data retrieved.

Predictors of abstinence, no heavy drinking days, and a 2-level reduction in World Health Organization drinking levels during treatment for alcohol use disorder in the COMBINE study

BACKGROUND

Data from trials of medications for alcohol use disorder (AUD) can be used to identify predictors of drinking outcomes regardless of treatment, which can inform the design of future trials with heterogeneous populations. Here, we identified predictors of abstinence, no heavy drinking days, and a 2-level reduction in World Health Organization (WHO) drinking levels during treatment for AUD in the Combined Pharmacotherapies and Behavioral Interventions (COMBINE) Study.

METHODS

We utilized data from the COMBINE Study, a randomized placebo-controlled trial evaluating the efficacy of naltrexone and acamprosate, both alone and in combination, for AUD (n = 1168). A tree-based machine learning algorithm was used to construct classification trees predicting abstinence, no heavy drinking days, and a 2-level reduction in WHO drinking levels in the last 4 weeks of treatment, based on 89 baseline variables.

RESULTS

The final tree for predicting abstinence had one split based on consecutive days abstinent prior to randomization, with a higher proportion of subjects achieving abstinence among those classified as abstinent for >2 versus ≤2 consecutive weeks prior to randomization (66% vs. 29%). The final tree for predicting no heavy drinking days in the last 4 weeks of treatment had three splits based on consecutive days abstinent, age, and total Alcohol Dependence Scale score at baseline. Seventy-three percent of the subjects classified as abstinent for >2 consecutive weeks prior to randomization had no heavy drinking days in the last 4 weeks of treatment. Among those classified as abstinent ≤2 consecutive weeks prior, three additional splits showed that younger subjects (age ≤44 years; 37%), and older subjects (age >44) with a total Alcohol Dependence Scale score >13 and complete abstinence (56%) or other drinking goals (35%), were less likely to have no heavy drinking days than older subjects with a total Alcohol Dependence Scale score ≤13 (67%). The final tree for predicting a 2-level reduction in WHO levels had no splits.

CONCLUSIONS

Consecutive days abstinent prior to randomization may predict abstinence and no heavy drinking days and total Alcohol Dependence Scale score and age may predict no heavy drinking days. The 2-level reduction in WHO levels outcome may be less likely to discriminate based on multiple patient characteristics.

Environmental risk factors for non-Hodgkin's lymphoma: umbrella review and comparison of meta-analyses of summary and individual participant data

Objectives

To summarise the range, strength, and validity of reported associations between environmental risk factors and non-Hodgkin's lymphoma, and to evaluate the concordance between associations reported in meta-analyses of summary level data and meta-analyses of individual participant data.

Design

Umbrella review and comparison of meta-analyses of summary and individual participant level data.

Data sources

Medline, Embase, Scopus, Web of Science Core Collection, Cochrane Library, and Epistemonikos, from inception to 23 July 2021.

Eligibility criteria for selecting studies

English language meta-analyses of summary level data and of individual participant data evaluating associations between environmental risk factors and incident non-Hodgkin's lymphoma (overall and subtypes).

Data extraction and synthesis

Summary effect estimates from meta-analyses of summary level data comparing ever versus never exposure that were adjusted for the largest number of potential confounders were re-estimated using a random effects model and classified as presenting evidence that was non-significant, weak (P<0.05), suggestive (P<0.001 and >1000 cases), highly suggestive (P<0.000001, >1000 cases, largest study reporting a significant association), or convincing (P<0.000001, >1000 cases, largest study reporting a significant association, I2 <50%, 95% prediction interval excluding the null value, and no evidence of small study effects and excess significance bias) evidence. When the same exposures, exposure contrast levels, and outcomes were evaluated in meta-analyses of summary level data and meta-analyses of individual participant data from the International Lymphoma Epidemiology (InterLymph) Consortium, concordance in terms of direction, level of significance, and overlap of 95% confidence intervals was examined. Methodological quality of the meta-analyses of summary level data was assessed by the AMSTAR 2 tool.

Results

We identified 85 meta-analyses of summary level data reporting 257 associations for 134 unique environmental risk factors and 10 subtypes of non-Hodgkin's lymphoma nearly all (79, 93%) were classified as having critically low quality. Most associations (225, 88%) presented either non-significant or weak evidence. The 11 (4%) associations presenting highly suggestive evidence were primarily for autoimmune or infectious disease related risk factors. Only one association, between history of coeliac disease and risk of non-Hodgkin's lymphoma, presented convincing evidence. Of 40 associations reported in meta-analyses of summary level data that were also evaluated in InterLymph meta-analyses of individual participant data, 22 (55%) pairs were in the same direction, had the same level of statistical significance, and had overlapping 95% confidence intervals; 28 (70%) pairs had summary effect sizes from the meta-analyses of individual participant data that were more conservative.

Conclusion

This umbrella review suggests evidence of many meta-analyses of summary level data reporting weak associations between environmental risk factors and non-Hodgkin's lymphoma. Improvements to primary studies as well as evidence synthesis in evaluations of evironmental risk factors and non-Hodgkin's lymphoma are needed.

Review registration number

PROSPERO CRD42020178010.

Spending by the Centers for Medicare & Medicaid Services Before and After Confirmation of Benefit for Drugs Granted US Food and Drug Administration Accelerated Approval, 2012 to 2017

Question

Findings

Meaning

Importance

Objective

Design and Setting

Main Outcomes and Measures

Results

Conclusions and Relevance

Agreement of treatment effects from observational studies and randomized controlled trials evaluating hydroxychloroquine, lopinavir-ritonavir, or dexamethasone for covid-19: meta-epidemiological study

OBJECTIVE

To systematically identify, match, and compare treatment effects and study demographics from individual or meta-analysed observational studies and randomized controlled trials (RCTs) evaluating the same covid-19 treatments, comparators, and outcomes.

DESIGN

Meta-epidemiological study.

DATA SOURCES

National Institutes of Health Covid-19 Treatment Guidelines, a living review and network meta-analysis published in The BMJ, a living systematic review with meta-analysis and trial sequential analysis in PLOS Medicine (The LIVING Project), and the Epistemonikos "Living OVerview of Evidence" (L·OVE) evidence database.

ELIGIBILITY CRITERIA FOR SELECTION OF STUDIES

RCTs in The BMJ's living review that directly compared any of the three most frequently studied therapeutic interventions for covid-19 across all data sources (that is, hydroxychloroquine, lopinavir-ritonavir, or dexamethasone) for any safety and efficacy outcomes. Observational studies that evaluated the same interventions, comparisons, and outcomes that were reported in The BMJ's living review.

DATA EXTRACTION AND SYNTHESIS

Safety and efficacy outcomes from observational studies were identified and treatment effects for dichotomous (odds ratios) or continuous (mean differences or ratios of means) outcomes were calculated and, when possible, meta-analyzed to match the treatment effects from individual RCTs or meta-analyses of RCTs reported in The BMJ's living review with the same interventions, comparisons, and outcomes (that is, matched pairs). The analysis compared the distribution of study demographics and the agreement between treatment effects from matched pairs. Matched pairs were in agreement if both observational and RCT treatment effects were significantly increasing or decreasing (P<0.05) or if both treatment effects were not significant (P≥0.05).

RESULTS

17 new, independent meta-analyses of observational studies were conducted that compared hydroxychloroquine, lopinavir-ritonavir, or dexamethasone with an active or placebo comparator for any safety or efficacy outcomes in covid-19 treatment. These studies were matched and compared with 17 meta-analyses of RCTs reported in The BMJ's living review. 10 additional matched pairs with only one observational study and/or one RCT were identified. Across all 27 matched pairs, 22 had adequate reporting of demographical and clinical data for all individual studies. All 22 matched pairs had studies with overlapping distributions of sex, age, and disease severity. Overall, 21 (78%) of the 27 matched pairs had treatment effects that were in agreement. Among the 17 matched pairs consisting of meta-analyses of observational studies and meta-analyses of RCTs, 14 (82%) were in agreement; seven (70%) of the 10 matched pairs consisting of at least one observational study or one RCT were in agreement. The 18 matched pairs with treatment effects for dichotomous outcomes had a higher proportion of agreement (n=16, 89%) than did the nine matched pairs with treatment effects for continuous outcomes (n=5, 56%).

CONCLUSIONS

Meta-analyses of observational studies and RCTs evaluating treatments for covid-19 have summary treatment effects that are generally in agreement. Although our evaluation is limited to three covid-19 treatments, these findings suggest that meta-analyzed evidence from observational studies might complement, but should not replace, evidence collected from RCTs.

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014-2017)

Psychologists are navigating an unprecedented period of introspection about the credibility and utility of their discipline. Reform initiatives emphasize the benefits of transparency and reproducibility-related research practices; however, adoption across the psychology literature is unknown. Estimating the prevalence of such practices will help to gauge the collective impact of reform initiatives, track progress over time, and calibrate future efforts. To this end, we manually examined a random sample of 250 psychology articles published between 2014 and 2017. Over half of the articles were publicly available (154/237, 65%, 95% confidence interval [CI] = [59%, 71%]); however, sharing of research materials (26/183; 14%, 95% CI = [10%, 19%]), study protocols (0/188; 0%, 95% CI = [0%, 1%]), raw data (4/188; 2%, 95% CI = [1%, 4%]), and analysis scripts (1/188; 1%, 95% CI = [0%, 1%]) was rare. Preregistration was also uncommon (5/188; 3%, 95% CI = [1%, 5%]). Many articles included a funding disclosure statement (142/228; 62%, 95% CI = [56%, 69%]), but conflict-of-interest statements were less common (88/228; 39%, 95% CI = [32%, 45%]). Replication studies were rare (10/188; 5%, 95% CI = [3%, 8%]), and few studies were included in systematic reviews (21/183; 11%, 95% CI = [8%, 16%]) or meta-analyses (12/183; 7%, 95% CI = [4%, 10%]). Overall, the results suggest that transparency and reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility of psychology research can be compared.

Comparison of Duration of Postapproval vs Pivotal Trials for Therapeutic Agents Granted US Food and Drug Administration Accelerated Approval, 2009-2018

This cross-sectional study compares the duration of postapproval trials with that of the pivotal trials used as the basis for the US Food and Drug Administration’s (FDA’s) approval for all indications receiving accelerated approval from 2009-2018.

Characteristics of Registered Clinical Trials Evaluating the Role of e-Cigarettes in Cessation or Reduction of Cigarette Smoking

This observational study examines the characteristics of completed and ongoing domestic and international clinical trials registered in the World Health Organization Registry Network.

Feasibility of Using Real-world Data to Emulate Postapproval Confirmatory Clinical Trials of Therapeutic Agents Granted US Food and Drug Administration Accelerated Approval

This cross-sectional study examines the feasibility of using real-world data, such as billing, claims, and electronic health records, to emulate US Food and Drug Administration–required confirmatory clinical trials for the 50 new therapeutic agents that received accelerated approval between 2009 and 2018.

Real-world Cardiovascular Outcomes Associated With Degarelix vs Leuprolide for Prostate Cancer Treatment

IMPORTANCE

With a growing interest in the use of real-world evidence for regulatory decision-making, it is important to understand whether real-world data can be used to emulate the results of randomized clinical trials.

OBJECTIVE

To use electronic health record and administrative claims data to emulate the ongoing PRONOUNCE trial (A Trial Comparing Cardiovascular Safety of Degarelix Versus Leuprolide in Patients With Advanced Prostate Cancer and Cardiovascular Disease).

DESIGN, SETTING, AND PARTICIPANTS

This retrospective, propensity-matched cohort study included adult men with a diagnosis of prostate cancer and cardiovascular disease who initiated either degarelix or leuprolide between December 24, 2008, and June 30, 2019. Participants were commercially insured individuals and Medicare Advantage beneficiaries included in a large US administrative claims database.

EXPOSURES

Degarelix or leuprolide.

MAIN OUTCOMES AND MEASURES

The primary end point was time to first occurrence of a major adverse cardiovascular event (MACE), defined as death due to any cause, myocardial infarction, or stroke, analogous to the PRONOUNCE trial. Secondary end points were time to death due to any cause, myocardial infarction, stroke, and angina. Cox proportional hazards regression was used to evaluate primary and secondary end points.

RESULTS

A total of 32 172 men initiated degarelix or leuprolide for prostate cancer; of them, 9490 (29.5%) had cardiovascular disease, and 7800 (24.2%) met the PRONOUNCE trial eligibility criteria and were included in this study. Overall, 165 participants (2.1%) were Asian, 1390 (17.8%) were Black, 663 (8.5%) were Hispanic, and 5258 (67.4%) were White. The mean (SD) age was 74.4 (7.4) years. Among 2226 propensity score-matched patients, no significant difference was observed in the risk of MACE for patients taking degarelix vs those taking leuprolide (10.18 vs 8.60 events per 100 person-years; hazard ratio [HR], 1.18; 95% CI, 0.86-1.61). Degarelix was associated with a higher risk of death from any cause (HR, 1.48; 95% CI, 1.01-2.18) but not of myocardial infarction (HR, 1.16; 95% CI, 0.60-2.25), stroke (HR, 0.92; 95% CI, 0.45-1.85), or angina (HR, 1.36; 95% CI, 0.43-4.27).

CONCLUSIONS AND RELEVANCE

In this emulation of a clinical trial of men with cardiovascular disease undergoing treatment for prostate cancer, degarelix was not associated with a lower risk of cardiovascular events than leuprolide. Comparison of these data with PRONOUNCE trial results, when published, will help enhance our understanding of the appropriate role of using real-world data to emulate clinical trials.

Medical journal requirements for clinical trial data sharing: Ripe for improvement

Florian Naudet and co-authors discuss strengthening requirements for sharing clinical trial data.

Characteristics of available studies and dissemination of research using major clinical data sharing platforms

BACKGROUND/AIMS

Over the past decade, numerous data sharing platforms have been launched, providing access to de-identified individual patient-level data and supporting documentation. We evaluated the characteristics of prominent clinical data sharing platforms, including types of studies listed as available for request, data requests received, and rates of dissemination of research findings from data requests.

METHODS

We reviewed publicly available information listed on the websites of six prominent clinical data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center, ClinicalStudyDataRequest.com, Project Data Sphere, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project. We recorded key platform characteristics, including listed studies and available supporting documentation, information on the number and status of data requests, and rates of dissemination of research findings from data requests (i.e. publications in a peer-reviewed journals, preprints, conference abstracts, or results reported on the platform's website).

RESULTS

The number of clinical studies listed as available for request varied among five data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center (n = 219), ClinicalStudyDataRequest.com (n = 2,897), Project Data Sphere (n = 154), Vivli (n = 5426), and the Yale Open Data Access Project (n = 395); Supporting Open Access to Researchers did not provide a list of Bristol Myers Squibb studies available for request. Individual patient-level data were nearly always reported as being available for request, as opposed to only Clinical Study Reports (Biological Specimen and Data Repository Information Coordinating Center = 211/219 (96.3%); ClinicalStudyDataRequest.com = 2884/2897 (99.6%); Project Data Sphere = 154/154 (100.0%); and the Yale Open Data Access Project = 355/395 (89.9%)); Vivli did not provide downloadable study metadata. Of 1201 data requests listed on ClinicalStudyDataRequest.com, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project platforms, 586 requests (48.8%) were approved (i.e. data access granted). The majority were for secondary analyses and/or developing/validating methods (ClinicalStudyDataRequest.com = 262/313 (83.7%); Supporting Open Access to Researchers-Bristol Myers Squibb = 22/30 (73.3%); Vivli = 63/84 (75.0%); the Yale Open Data Access Project = 111/159 (69.8%)); four were for re-analyses or corroborations of previous research findings (ClinicalStudyDataRequest.com = 3/313 (1.0%) and the Yale Open Data Access Project = 1/159 (0.6%)). Ninety-five (16.1%) approved data requests had results disseminated via peer-reviewed publications (ClinicalStudyDataRequest.com = 61/313 (19.5%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 4/84 (4.8%); the Yale Open Data Access Project = 27/159 (17.0%)). Forty-two (6.8%) additional requests reported results through preprints, conference abstracts, or on the platform's website (ClinicalStudyDataRequest.com = 12/313 (3.8%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 2/84 (2.4%); Yale Open Data Access Project = 25/159 (15.7%)).

CONCLUSION

Across six prominent clinical data sharing platforms, information on studies and request metrics varied in availability and format. Most data requests focused on secondary analyses and approximately one-quarter of all approved requests publicly disseminated their results. To further promote the use of shared clinical data, platforms should increase transparency, consistently clarify the availability of the listed studies and supporting documentation, and ensure that research findings from data requests are disseminated.

Evaluation of Drug Trials in High-, Middle-, and Low-Income Countries and Local Commercial Availability of Newly Approved Drugs

IMPORTANCE

Clinical research supporting US Food and Drug Administration (FDA) drug approvals is largely conducted outside the US.

OBJECTIVE

To characterize where drugs were tested for FDA approval and to determine how commonly and quickly these drugs received marketing approval in the countries where they were tested, both overall and by country income level and geographical region.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional analysis of trials supporting FDA approval of novel drugs in 2012 and 2014, sponsored by large drug companies, did not involve human participants. The settings were the countries hosting trials supporting US drug approval. Data sources included Drugs@FDA, ClinicalTrials.gov, PubMed, Google Scholar, EMBASE, and drug regulatory agency websites. Data analysis was completed March through September 2020.

MAIN OUTCOMES AND MEASURES

The primary outcomes were the proportion of drugs approved for marketing in the countries where they were tested for FDA approval within 1, 2, 3, 4, and 5 years of FDA approval and the proportion of countries contributing participants to trials supporting FDA approvals receiving market access to the drugs they helped test within 1, 2, 3, 4, and 5 years of FDA approval.

RESULTS

In 2012 and 2014, the FDA approved 34 novel drugs sponsored by large companies, on the basis of a total of 898 trials, 563 of which had location information available. Each drug was tested in a median (interquartile range [IQR]) of 25 (18-37) unique countries, including a median (IQR) of 20 (13-25) high-income countries, 6 (4-11) upper-middle-income countries, and 1 (0-2) low-middle-income country. One drug was approved for marketing in all testing countries within 1 year of FDA approval and 15% (5 of 34 drugs) were approved in all testing countries within 5 years of FDA approval. Of the 70 countries contributing research participants for FDA drug approvals, 7% (5 countries) received market access to drugs they helped test within 1 year of FDA approval and 31% (22 countries) did so within 5 years. Access within 1 year occurred in 13% (5 of 39) of high-income countries, 0 of 22 upper-middle-income countries (0%), and 0 of 9 lower-middle-income countries (0%), whereas at 5 years access rates were 46% (18 of 39 countries), 9% (2 of 22 countries), and 22% (2 of 9 countries), respectively. Approvals were faster in high-income countries (median [IQR], 8 [0-11] months) than in upper-middle-income countries (median [IQR], 11 [5-29] months) or lower-middle-income countries (median [IQR], 17 [11-27] months) after FDA approval. Access was lowest in African countries.

CONCLUSIONS AND RELEVANCE

These findings suggest that substantial gaps exist between where FDA-approved drugs are tested and where they ultimately become available to patients, raising concerns about the equitable distribution of research benefits at the population level.

Evaluation of Trials Comparing Single-Enantiomer Drugs to Their Racemic Precursors: A Systematic Review

IMPORTANCE

Chiral switching, a strategy in which drug manufacturers develop a single-enantiomer formulation of a drug to be substituted for a racemic formulation, allows manufacturers to maintain market exclusivity for drugs losing patent protection, even without demonstrating superior efficacy or safety.

OBJECTIVE

To identify and characterize all randomized clinical trials (RCTs) directly comparing a Food and Drug Administration (FDA)-approved single-enantiomer drug against a previously approved racemic drug for 1 or more efficacy or safety end points.

EVIDENCE REVIEW

Drugs were identified using the Drugs@FDA database. Randomized clinical trials were identified using Ovid MEDLINE (1949 to October 22, 2019), Ovid Embase (1974 to October 22, 2019), Web of Science Core Collection (all years), ClinicalTrials.gov, and Cochrane Central Registry of Controlled Trials (CENTRAL, Wiley, Issue 8 of 12, October 22, 2019). Trials were characterized as favoring the single-enantiomer or racemic drugs based on whether the primary efficacy, secondary efficacy, and safety end points achieved each study's defined significance level (eg, P < .05). Trials were characterized as favoring neither drug if no statistically significant differences were reported for any end point or if both drugs were found to be superior for 1 or more separate end points.

FINDINGS

Fifteen FDA-approved single-enantiomer drugs were identified with racemic precursors approved in the US or Europe. For 3 single-enantiomer racemic drug pairs, no RCTs directly comparing the drugs were identified. For the remaining 12 pairs, 185 RCTs comparing efficacy or safety of the drug pairs were identified, 124 (67.0%) of which studied 1 pair (levobupivacaine/bupivacaine). There were 179 RCTs directly comparing drug pairs using efficacy end points, of which 23 (12.8%) favored the single enantiomer based on primary efficacy end point results. There were 124 RCTs directly comparing drug pairs using safety end points, of which 17 (13.7%) favored the single-enantiomer drug. For 9 of the 15 single-enantiomer drugs (60.0%), no RCTs were identified providing evidence of improved efficacy, based on primary end point results, or safety as compared with their racemic precursors.

CONCLUSIONS AND RELEVANCE

The results of this systematic review suggest that most newly marketed FDA-approved single-enantiomer drugs are infrequently directly compared with their racemic precursors, and when compared, they are uncommonly found to provide improved efficacy or safety, despite their greater costs.

US Food and Drug Administration utilization of postmarketing requirements and postmarketing commitments, 2009-2018

BACKGROUND/AIMS

The US Food and Drug Administration outlines clinical studies as postmarketing requirements and commitments to be fulfilled following approval of new drugs and biologics ("therapeutics"). Regulators have increasingly emphasized lifecycle evaluation of approved therapeutics, and postmarketing studies are intended to advance our understanding of therapeutic safety and efficacy. However, little is known about the indications that clinical studies outlined in postmarketing requirements and commitments investigate, including whether they are intended to generate evidence for approved or other clinical indications. Therefore, we characterized US Food and Drug Administration postmarketing requirements and commitments for new therapeutics approved from 2009 to 2018.

METHODS

We conducted a cross-sectional study of all novel therapeutics, including small-molecule drugs and biologics, receiving original US Food and Drug Administration approval from 2009 to 2018, using approval letters accessed through the Drug@FDA database. Outcomes included the number and characteristics of US Food and Drug Administration postmarketing requirements and commitments for new therapeutics at original approval, including the types of studies outlined, the indications to be investigated, and the clinical evidence to be generated.

RESULTS

From 2009 to 2018, the US Food and Drug Administration approved 343 new therapeutics with 1978 postmarketing requirements and commitments. Overall, 750 (37.9%) postmarketing requirements and commitments outlined clinical studies. For 71 of 343 (20.7%) therapeutics, no postmarketing requirements or commitments for clinical studies were outlined, while at least 1 was outlined for 272 (79.3%; median 2 (interquartile range: 1-4)). Among these 272 therapeutics, the number of postmarketing requirements and commitments for clinical studies per therapeutic did not change from 2009 (median: 2 (interquartile range: 1-4)) to 2018 (median: 2 (interquartile range: 1-3)). Among the 750 postmarketing requirements and commitments for clinical studies, 448 (59.7%) outlined new prospective cohort studies, registries, or clinical trials, while the remainder outlined retrospective studies, secondary analyses, or completion of ongoing studies. Although 455 (60.7%) clinical studies investigated only original approved therapeutic indications, 123 (16.4%) enrolled from an expansion of the approved disease population and 61 (8.1%) investigated diseases unrelated to approved indications.

CONCLUSIONS

The US Food and Drug Administration approves most new therapeutics with at least 1 postmarketing requirement or commitment for a clinical study, and outlines investigations of safety or efficacy for both approved and unapproved indications. The median number of 2 clinical studies outlined has remained relatively constant over the last decade. Given increasing emphasis by the US Food and Drug Administration on faster approval and lifecycle evaluation of therapeutics, these findings suggest that more postmarketing requirements and commitments may be necessary to address gaps in the clinical evidence available for therapeutics at approval.

Reporting of Study Participant Demographic Characteristics and Demographic Representation in Premarketing and Postmarketing Studies of Novel Cancer Therapeutics

IMPORTANCE

Adequate representation of demographic subgroups in premarketing and postmarketing clinical studies is necessary for understanding the safety and efficacy associated with novel cancer therapeutics.

OBJECTIVE

To characterize and compare the reporting of demographic data and the representation of individuals by sex, age, and race in premarketing and postmarketing studies used by the Food and Drug Administration (FDA) to evaluate novel cancer therapeutics.

DESIGN, SETTING, AND PARTICIPANTS

In this cross-sectional study, premarketing and postmarketing studies for novel cancer therapeutics approved by the FDA from 2012 through 2016 were identified. Study demographic information was abstracted from publicly available sources, and US cancer population demographic data was abstracted from US Cancer Statistics. Analyses were conducted from February 25 through September 21, 2020.

MAIN OUTCOMES AND MEASURES

The percentages of trials reporting sex, age, and race/ethnicity were calculated, and participation to prevalence ratios (PPRs) were calculated by dividing the percentage of study participants in each demographic group by the percentage of the US cancer population in each group. PPRs were constructed for premarketing and postmarketing studies and by cancer type. Underrepresentation was defined as PPR less than 0.8.

RESULTS

From 2012 through 2016, the FDA approved 45 cancer therapeutics. The study sample included 77 premarketing studies and 56 postmarketing studies. Postmarketing studies, compared with premarketing studies, were less likely to report patient sex (42 studies reporting [75.0%] vs 77 studies reporting [100%]; P < .001) and race (27 studies reporting [48.2%] vs 62 studies reporting [80.5%]; P < .001). Women were adequately represented in premarketing studies (mean [SD] PPR, 0.91; 95% CI, 0.90-0.91) and postmarketing studies (mean PPR, 1.00; 95% CI, 1.00-1.01). Although older adults and Black patients were underrepresented in premarketing studies (older adults: mean PPR, 0.73; 95% CI, 0.72-0.74; Black patients: mean PPR, 0.32; 95% CI, 0.31-0.32), these groups continued to be underrepresented in postmarketing studies (older adults: mean PPR, 0.75; 95% CI, 0.75-0.76; Black patients: mean PPR, 0.21; 95% CI, 0.21-0.21).

CONCLUSIONS AND RELEVANCE

This study found that older adults and Black patients were underrepresented in postmarketing studies of novel cancer therapeutics to a similar degree that they were underrepresented in premarketing studies. These findings suggest that postmarketing studies are not associated with improvements to gaps in demographic representation present at the time of FDA approval.

Vibration of effects in epidemiologic studies of alcohol consumption and breast cancer risk

BACKGROUND

Different analytical approaches can influence the associations estimated in observational studies. We assessed the variability of effect estimates reported within and across observational studies evaluating the impact of alcohol on breast cancer.

METHODS

We abstracted largest harmful, largest protective and smallest (closest to the null value of 1.0) relative risk estimates in studies included in a recent alcohol-breast cancer meta-analysis, and recorded how they differed based on five model specification characteristics, including exposure definition, exposure contrast levels, study populations, adjustment covariates and/or model approaches. For each study, we approximated vibration of effects by dividing the largest by the smallest effect estimate [i.e. ratio of odds ratio (ROR)].

RESULTS

Among 97 eligible studies, 85 (87.6%) reported both harmful and protective relative effect estimates for an alcohol-breast cancer relationship, which ranged from 1.1 to 17.9 and 0.0 to 1.0, respectively. The RORs comparing the largest and smallest estimates in value ranged from 1.0 to 106.2, with a median of 3.0 [interquartile range (IQR) 2.0-5.2]. One-third (35, 36.1%) of the RORs were based on extreme effect estimates with at least three different model specification characteristics; the vast majority (87, 89.7%) had different exposure definitions or contrast levels. Similar vibrations of effect were observed when only extreme estimates with differences based on study populations and/or adjustment covariates were compared.

CONCLUSIONS

Most observational studies evaluating the impact of alcohol on breast cancer report relative effect estimates for the same associations that diverge by >2-fold. Therefore, observational studies should estimate the vibration of effects to provide insight regarding the stability of findings.

Assessment of transparency indicators across the biomedical literature: How open is open?

Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.

Assessment of Concordance and Discordance Among Clinical Studies Posted as Preprints and Subsequently Published in High-Impact Journals

This cross-sectional study examines the concordance between clinical studies posted as preprints and subsequently published in high-impact journals, including key study characteristics, reported results, and study interpretations.

Characteristics and Reporting of Number Needed to Treat, Number Needed to Harm, and Absolute Risk Reduction in Controlled Clinical Trials, 2001-2019

This cross-sectional study assesses the trends and characteristics of absolute measure reporting in highly cited medical journals from 2001 to 2019.

Publishing at any cost: a cross-sectional study of the amount that medical researchers spend on open access publishing each year

OBJECTIVE

To estimate the financial costs paid by individual medical researchers from meeting the article processing charges (APCs) levied by open access journals in 2019.

DESIGN

Cross-sectional analysis.

DATA SOURCES

Scopus was used to generate two random samples of researchers, the first with a senior author article indexed in the 'Medicine' subject area (general researchers) and the second with an article published in the ten highest-impact factor general clinical medicine journals (high-impact researchers) in 2019. For each researcher, Scopus was used to identify all first and senior author original research or review articles published in 2019. Data were obtained from Scopus, institutional profiles, Journal Citation Reports, publisher databases, the Directory of Open Access Journals, and individual journal websites.

MAIN OUTCOME MEASURES

Median APCs paid by general and high-impact researchers for all first and senior author research and review articles published in 2019.

RESULTS

There were 241 general and 246 high-impact researchers identified as eligible for our study. In 2019, the general and high-impact researchers published a total of 914 (median 2, IQR 1-5) and 1471 (4, 2-8) first or senior author research or review articles, respectively. 42% (384/914) of the articles from the general researchers and 29% (428/1471) of the articles from the high-impact medical researchers were published in fully open access journals. The median total APCs paid by general researchers in 2019 was US$191 (US$0-US$2500) and the median total paid by high-impact researchers was US$2900 (US$0-US$5465); the maximum paid by a single researcher in total APCs was US$30115 and US$34676, respectively.

CONCLUSIONS

Medical researchers in 2019 were found to have paid between US$0 and US$34676 in total APCs. As journals with APCs become more common, it is important to continue to evaluate the potential cost to researchers, especially on individuals who may not have the funding or institutional resources to cover these costs.

Transparency of Regulatory Data across the European Medicines Agency, Health Canada, and US Food and Drug Administration

Based on an analysis of relevant laws and policies, regulator data portals, and information requests, we find that clinical data, including clinical study reports, submitted to the European Medicines Agency and Health Canada to support approval of medicines are routinely made publicly available.

Characteristics of Ongoing Clinical Trials for Alcohol Use Disorder Registered on ClinicalTrials.gov

This cross-sectional study evaluates the characteristics of ongoing clinical trials for alcohol use disorder medications registered on ClinicalTrials.gov, including the medications, populations, and end points currently being studied.

Disagreements Within the US Food and Drug Administration Regarding Approval of Novel Therapeutic Agents, 2011-2015

This cross-sectional study examines the frequency of disagreements within the US Food and Drug Administration (FDA) regarding approval of novel therapeutic agents.