Second thoughts on the final rule: An analysis of baseline participant characteristics reports on ClinicalTrials.gov

Population adjusted-indirect comparisons in health technology assessment: A methodological systematic review

In health technology assessment (HTA), population-adjusted indirect comparisons (PAICs) are increasingly considered to adjust for the difference in the target population between studies. We aim to assess the conduct and reporting of PAICs in recent HTA practice, by performing, a methodological systematic review of studies implementing PAICs from PubMed, EMBASE Classic, Embase/Ovid Medline All, and Cochrane databases from January 1, 2010 to Feb 13, 2023. Four independent researchers screened the titles, abstracts, and full-texts of the identified records, then extracted data on methodological and reporting characteristics of 106 eligible articles. Most PAIC analyses (96.9%, n = 157) were conducted by (or received funding from) pharmaceutical companies. Prior to adjustment, 44.5% of analyses (n = 72) (partially) aligned the eligibility criteria of different studies to enhance the similarity of their target populations. In 37.0% of analyses (n = 60), the clinical and methodological heterogeneity across studies were extensively assessed. In 9.3% of analyses (n = 15), the quality (or bias) of individual studies was evaluated. Among 18 analyses using methods that required an outcome model specification, results of the model fitting procedure were adequately reported in three analyses (16.7%). These findings suggest that the conduct and reporting of PAICs are remarkably heterogeneous and suboptimal in current practice. More recommendations and guidelines on PAICs are thus warranted to enhance the quality of these analyses in the future.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

The impact of a yoga-based physical therapy group for individuals with traumatic brain injury: results from a pilot study

OBJECTIVE

To compare the impacts of yoga-based physical therapy versus a seated rest within the context of standard rehabilitation practice on sleep, heart rate variability (HRV), anxiety, and fatigue during acute traumatic brain injury (TBI) rehabilitation.

METHODS

Eleven individuals participated in this crossover study involving the following interventions in a randomized order: group yoga-based physical therapy (YPT), conventional physical therapy (CPT), and group seated rest in a relaxing environment (SR). HRV and self-reported anxiety and fatigue were measured immediately before and after each group, and sleep after each condition and at baseline. Data was analyzed using generalized linear mixed models with repeated measures.

RESULTS

The interaction between time and treatment was statistically significant (p = .0203). For the SR treatment, wake after sleep onset (WASO) rate was reduced from 14.99 to 10.60 (IRR = 0.71; p = .006). Time and treatment were not found to be statistically significantly associated with any of the secondary outcomes.

CONCLUSION

Yoga-based physical therapy is feasible and safe in the inpatient rehabilitation setting following TBI. Sleep quality improved following the addition of a one-hour seated rest in a relaxing environment to a standard rehabilitation daily schedule, suggesting that structured rest time may be beneficial to sleep hygiene during inpatient rehabilitation following TBI. ClinicalTrials.Gov Registration Number: NCT03701594.

Latent Dirichlet Allocation in predicting clinical trial terminations

Background

This study used natural language processing (NLP) and machine learning (ML) techniques to identify reliable patterns from within research narrative documents to distinguish studies that complete successfully, from the ones that terminate. Recent research findings have reported that at least 10 % of all studies that are funded by major research funding agencies terminate without yielding useful results. Since it is well-known that scientific studies that receive funding from major funding agencies are carefully planned, and rigorously vetted through the peer-review process, it was somewhat daunting to us that study-terminations are this prevalent. Moreover, our review of the literature about study terminations suggested that the reasons for study terminations are not well understood. We therefore aimed to address that knowledge gap, by seeking to identify the factors that contribute to study failures.

Method

We used data from the clinicialTrials.gov repository, from which we extracted both structured data (study characteristics), and unstructured data (the narrative description of the studies). We applied natural language processing techniques to the unstructured data to quantify the risk of termination by identifying distinctive topics that are more frequently associated with trials that are terminated and trials that are completed. We used the Latent Dirichlet Allocation (LDA) technique to derive 25 “topics” with corresponding sets of probabilities, which we then used to predict study-termination by utilizing random forest modeling. We fit two distinct models – one using only structured data as predictors and another model with both structured data and the 25 text topics derived from the unstructured data.

Results

In this paper, we demonstrate the interpretive and predictive value of LDA as it relates to predicting clinical trial failure. The results also demonstrate that the combined modeling approach yields robust predictive probabilities in terms of both sensitivity and specificity, relative to a model that utilizes the structured data alone.

Conclusions

Our study demonstrated that the use of topic modeling using LDA significantly raises the utility of unstructured data in better predicating the completion vs. termination of studies. This study sets the direction for future research to evaluate the viability of the designs of health studies.

The 10-year Landscape of United States-Registered Parkinson Disease Clinical Trials: 2007-2016

Background

We know little about how well the goals and results of clinical trials in Parkinson disease (PD) reflect the priorities of patients and the broader PD community.

Objectives

We conducted a review of registered trials on http://clincialtrials.gov from 2007 to 2016 to explore whether PD trials have moved closer to the therapeutic priority goals articulated by the PD community.

Methods

Using the search terms: Parkinson, interventional trials, phase "0-4," we categorized therapeutic PD studies from http://clinicaltrials.gov between January 1, 2007 and December 31, 2016. Seven hundred and sixty-six trials met the criteria for analysis. We explored temporal trends in the utilization of balance problems and falls; mood symptoms, including stress and anxiety; cognitive dysfunction, including dementia; and dyskinesias as primary outcomes. We analyzed trials where recruitment was listed as "completed" (n = 391) to explore publication rates.

Results

Balance problems and falls were listed as primary outcome measures in 125 studies (16.3%), cognitive measures in 48 (6.3%), mood features in 37 (4.8%), and dyskinesias in 30 (3.9%). Trials using balance problems and falls as a primary outcome increased in frequency per year between 2007 and 2016 (Z = -2.128, p = 0.033) unlike the proportion of trials evaluating cognitive dysfunction including dementia (Z = -0.380, p = 0.704), mood symptoms including stress and anxiety (Z = 0.345, p = 0.730), or dyskinesias (Z = 0.340, p = 0.734), which did not show temporal changes. 231 (59.1%) completed trials had results published in manuscript form as of 5/1/2017, leaving 40.9% of trials unpublished.

Conclusions

PD trials focusing on balance problems and falls are becoming more common. About 40% of completed PD trials are unpublished, reflecting suboptimal utilization of participant efforts.