A comprehensive review of randomized clinical trials in three medical journals reveals 396 medical reversals

Reliable crash analysis: Comparing biases and error rates of empirical Bayes before-after analyses to mixed-models

Estimating reliable causal estimates of road safety interventions is challenging, with a number of these challenges addressable through analysis choices. At a minimum, developing reliable crash modification factors (CMFs) needs to address three critical confounding factors, i.e., 1) the regression-to-the-mean (RTM) phenomenon, 2) the effect of traffic volume, and 3) the time trend for the occurrence of crashes. The current preferred crash analysis method is the empirical Bayes (EB) before-after analysis but requires complex bespoke analysis and may not be the best performing method. We compare in a simulation experiment various EB methods to a more straightforward negative binomial generalized linear mixed model (NB-GLMM) with an interaction term between treatment group and time for analysing treatment effects in crash data. Data were simulated using two broad scenarios: 1) an idealized randomized controlled design, and 2) a moderately biased site-selection scenario commonly encountered in road safety crash analyses. Both scenarios varied treatment effects, overdispersion, and sample sizes. The NB-GLMM performed best, maintaining type I error rate and providing least biased estimates across most analyses. Most standard EB methods were too liberal or generally more biased, with the exception of the EB method that incorporated a varying dispersion parameter. Incorporating mixed-effects modelling into the EB procedure improved bias. Overall, we found that using a "standard" NB-GLMM with an interaction term is sufficient for crash analysis, reducing complexity compared to bespoke EB solutions. Chosen methods should also be the least biased and possess the marginal error rates under both ideal and selection-bias conditions. Mixed-effects approaches to analysis of road safety interventions satisfy these criteria outperforming standard or other empirical Bayes approaches tested here.

Andexanet alfa: trials just leave us with more questions

Andexanet Alfa in Acute Intracranial Hemorrhage in Patients Receiving an Oral Factor Xa Inhibitor (ANNEXA-I), the first ever randomized controlled trial of a reversal agent for direct oral anticoagulants, was published in 2024. The trial, which randomized patients with intracranial hemorrhage to andexanet alfa or usual care, was mandated by the United States Food and Drug Administration as part of its conditional approval in 2018. This approval was originally based on the single-arm trial, The Andexanet Alfa, a Novel Antidote to the Anticoagulation Effects of Factor Xa Inhibitors (ANNEXA-4). ANNEXA-I was stopped early for benefit and showed a reduction in the number of patients with significant hematoma expansion. However, the study was not powered for clinical endpoints such as disability or death and showed no difference in these outcomes. It did, however, show an increased risk of thrombosis, predominantly stroke with andexanet alfa. In this perspective, I reflect on some of the key criticisms of the trial and the implications for its interpretation.

Cross-sectional evaluation of medical reversals among National Institute of Health guideline practices implemented during the COVID-19 pandemic: how often did experts err in a time of crisis?

OBJECTIVE

The COVID-19 pandemic required the rapid and often widespread implementation of medical practices without robust data. Many of these practices have since been tested in large, randomised trials and were found to be in error. We sought to identify incorrect recommendations, or reversals, among National Institute of Health COVID-19 guidelines and Food and Drug Administration (FDA) approvals and authorisations.

DESIGN

Retrospective cross-sectional study.

PARTICIPANTS

Recommended medical practices and FDA authorisations or approvals for COVID-19 prevention, treatment and/or management.

MAIN OUTCOME MEASURES

The frequency and characteristics of COVID-19 medical reversals, defined as practices that were implemented and/or recommended during the pandemic, but were later tested in randomised trials that failed to find benefit.

RESULTS

We found 332 COVID-19 recommendations. 85 (25.6%) opposed a medical practice, 23 (6.9%) were to continue a pre-COVID standard of care without deviation and 224 (67.5%) reccommended a new medical practice. We found randomised trials assessing 72 of these practices (32.1%), among which 25 (35%) were found to be in error and deemed medical reversals. Among medical reversals, 21 (84%) were prescription medications and 1 (4%) was convalescent plasma. 17 (68%) were repurposed medications. Two (8%) were procedures or mechanical interventions and one (4%) was a device. 16 (64%) reversals pertained to the hospital setting (4 to intensive care units), 4 (16%) were non-specific (ie, applicable to any setting), 4 (16%) pertained to a non-hospital setting and 1 pertained to healthcare workers.

CONCLUSION

When faced with a novel pandemic, policymakers rapidly made hundreds of specific medical recommendations. More than two out of three were never robustly tested. Among practices tested in a randomised fashion, one in three was made in error. Pandemic recommendation errors were substantial. Early and coordinated efforts to initiate randomised trials, even during dire situations, may mitigate the perpetuation of ineffective practices.

Evidence-based veterinary medicine-potential, practice, and pitfalls

Understanding and incorporating evidence-based veterinary medicine (EBVM) into clinical practice and research continues to pose a challenge for our profession despite over 2 decades of increasing awareness of this concept. Reasons for this include a lack of understanding of its importance to the practice of medicine, veterinary literature that often fails to adhere to evidence-based standards, inadequate attention to teaching EBVM at the university level, and the inherent reluctance of clinicians to alter historical practice styles. For many practitioners, EBVM continues to be an abstract concept they believe requires advanced training in statistics and epidemiology resulting in them relying on less robust sources for clinical guidance. This unfortunately results in suboptimal care for our patients and delayed medical advancements for our profession. As part of the 20th anniversary of the founding of the Evidence-Based Veterinary Medicine Association (EBVMA), we are refocusing our efforts to highlight the need for dedicated teaching of EBVM at the university level, for rigorous adherence to established research reporting guidelines, for expansion of EBVM infrastructure, and for the provision of easily accessible tools that permit clinicians to incorporate EBVM into their daily practice. As the quality of veterinary literature improves, so too will development of more effective clinical practice guidelines that ultimately can be widely adopted if they are flexible enough to support the triadic relationship between veterinarians, our clients and our patients. Ultimately, EBVM is not an end unto itself, but rather a means to improve the quality of care we provide our patients.

Treatment Effects in Randomized and Nonrandomized Studies of Pharmacological Interventions: A Meta-Analysis

Importance

Randomized clinical trials (RCTs) are widely regarded as the methodological benchmark for assessing clinical efficacy and safety of health interventions. There is growing interest in using nonrandomized studies to assess efficacy and safety of new drugs.

Objective

To determine how treatment effects for the same drug compare when evaluated in nonrandomized vs randomized studies.

Data Sources

Meta-analyses published between 2009 and 2018 were identified in MEDLINE via PubMed and the Cochrane Database of Systematic Reviews. Data analysis was conducted from October 2019 to July 2024.

Study Selection

Meta-analyses of pharmacological interventions were eligible for inclusion if both randomized and nonrandomized studies contributed to a single meta-analytic estimate.

Data Extraction and Synthesis

For this meta-analysis using a meta-epidemiological framework, separate summary effect size estimates were calculated for nonrandomized and randomized studies within each meta-analysis using a random-effects model and then these estimates were compared. The reporting of this study followed the Guidelines for Reporting Meta-Epidemiological Methodology Research and relevant portions of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline.

Main Outcome and Measures

The primary outcome was discrepancies in treatment effects obtained from nonrandomized and randomized studies, as measured by the proportion of meta-analyses where the 2 study types disagreed about the direction or magnitude of effect, disagreed beyond chance about the effect size estimate, and the summary ratio of odds ratios (ROR) obtained from nonrandomized vs randomized studies combined across all meta-analyses.

Results

A total of 346 meta-analyses with 2746 studies were included. Statistical conclusions about drug benefits and harms were different for 130 of 346 meta-analyses (37.6%) when focusing solely on either nonrandomized or randomized studies. Disagreements were beyond chance for 54 meta-analyses (15.6%). Across all meta-analyses, there was no strong evidence of consistent differences in treatment effects obtained from nonrandomized vs randomized studies (summary ROR, 0.95; 95% credible interval [CrI], 0.89-1.02). Compared with experimental nonrandomized studies, randomized studies produced on average a 19% smaller treatment effect (ROR, 0.81; 95% CrI, 0.68-0.97). There was increased heterogeneity in effect size estimates obtained from nonrandomized compared with randomized studies.

Conclusions and Relevance

In this meta-analysis of treatment effects of pharmacological interventions obtained from randomized and nonrandomized studies, there was no overall difference in effect size estimates between study types on average, but nonrandomized studies both overestimated and underestimated treatment effects observed in randomized studies and introduced additional uncertainty. These findings suggest that relying on nonrandomized studies as substitutes for RCTs may introduce additional uncertainty about the therapeutic effects of new drugs.

No evidence that ACE2 or TMPRSS2 drive population disparity in COVID risks

Early in the SARS-CoV2 pandemic, in this journal, Hou et al. (BMC Med 18:216, 2020) interpreted public genotype data, run through functional prediction tools, as suggesting that members of particular human populations carry potentially COVID-risk-increasing variants in genes ACE2 and TMPRSS2 far more often than do members of other populations. Beyond resting on predictions rather than clinical outcomes, and focusing on variants too rare to typify population members even jointly, their claim mistook a well known artifact (that large samples reveal more of a population's variants than do small samples) as if showing real and congruent population differences for the two genes, rather than lopsided population sampling in their shared source data. We explain that artifact, and contrast it with empirical findings, now ample, that other loci shape personal COVID risks far more significantly than do ACE2 and TMPRSS2-and that variation in ACE2 and TMPRSS2 per se unlikely exacerbates any net population disparity in the effects of such more risk-informative loci.

Controversy. On pump or off pump: what will I do when I grow up? A narrative systematic review

The transition from the second to the third millennium happened to be a turning point in the history of myocardial revascularization on a beating heart, which moved from technical development to critical evaluation. This article describes how the initial acceptance and spread of off-pump coronary artery bypass grafting (OPCABG) was followed by the general perception that the technique could not fulfill the expectations placed in it and provides some insight on what should we do with the know-how of OPCABG in the present and the future of coronary surgical revascularization.

Utilizing ChatGPT as a scientific reasoning engine to differentiate conflicting evidence and summarize challenges in controversial clinical questions

OBJECTIVE

Synthesizing and evaluating inconsistent medical evidence is essential in evidence-based medicine. This study aimed to employ ChatGPT as a sophisticated scientific reasoning engine to identify conflicting clinical evidence and summarize unresolved questions to inform further research.

MATERIALS AND METHODS

We evaluated ChatGPT's effectiveness in identifying conflicting evidence and investigated its principles of logical reasoning. An automated framework was developed to generate a PubMed dataset focused on controversial clinical topics. ChatGPT analyzed this dataset to identify consensus and controversy, and to formulate unsolved research questions. Expert evaluations were conducted 1) on the consensus and controversy for factual consistency, comprehensiveness, and potential harm and, 2) on the research questions for relevance, innovation, clarity, and specificity.

RESULTS

The gpt-4-1106-preview model achieved a 90% recall rate in detecting inconsistent claim pairs within a ternary assertions setup. Notably, without explicit reasoning prompts, ChatGPT provided sound reasoning for the assertions between claims and hypotheses, based on an analysis grounded in relevance, specificity, and certainty. ChatGPT's conclusions of consensus and controversies in clinical literature were comprehensive and factually consistent. The research questions proposed by ChatGPT received high expert ratings.

DISCUSSION

Our experiment implies that, in evaluating the relationship between evidence and claims, ChatGPT considered more detailed information beyond a straightforward assessment of sentimental orientation. This ability to process intricate information and conduct scientific reasoning regarding sentiment is noteworthy, particularly as this pattern emerged without explicit guidance or directives in prompts, highlighting ChatGPT's inherent logical reasoning capabilities.

CONCLUSION

This study demonstrated ChatGPT's capacity to evaluate and interpret scientific claims. Such proficiency can be generalized to broader clinical research literature. ChatGPT effectively aids in facilitating clinical studies by proposing unresolved challenges based on analysis of existing studies. However, caution is advised as ChatGPT's outputs are inferences drawn from the input literature and could be harmful to clinical practice.

Why some Patients Benefit from Participating in a Sham Surgery Trial

OBJECTIVE

We analyze the ethics of sham surgical trials from a utilitarian perspective and explore whether patients can benefit from participating in these trials.

BACKGROUND

Sham-controlled randomized trials are an essential tool to evaluate the risks and benefits of some surgical procedures. However, sham trials are controversial because they expose patients to the harms of a sham procedure without the possibility of benefit. We argue that ethical analyses of sham trials have focused only on the harms of sham surgery and neglected to account for the harms of the procedure being studied.

METHODS

We develop a theoretical model to estimate the harms and benefits experienced by patients who enter a sham surgery trial, taking into account the harms and benefits of the sham and intervention.

RESULTS

When the procedure in question is found to be ineffective, sham trials typically result in a net benefit to participants because some participants are only exposed to the harms of the sham procedure, which are much lower than the harms of the full procedure. When the procedure is found to be beneficial, the primary harm to patients who underwent the sham is not due to the sham itself but because they suffer a delay in receiving an effective intervention.

CONCLUSIONS

Patients often benefit from participating in sham surgery trials because the harms of the sham procedure are lower than the harms of the full procedure, which may turn out to be ineffective. Our results call for re-thinking the ethics of sham surgery trials.

Advancing healthcare equity through dissemination and implementation science

OBJECTIVE

To provide guiding principles and recommendations for how approaches from the field of dissemination and implementation (D&I) science can advance healthcare equity.

DATA SOURCES AND STUDY SETTING

This article, part of a special issue sponsored by the Agency for Healthcare Research and Quality (AHRQ), is based on an outline drafted to support proceedings of the 2022 AHRQ Health Equity Summit and further revised to reflect input from Summit attendees.

STUDY DESIGN

This is a narrative review of the current and potential applications of D&I approaches for understanding and advancing healthcare equity, followed by discussion and feedback with Summit attendees.

DATA COLLECTION/EXTRACTION METHODS

We identified major themes in narrative and systematic reviews related to D&I science, healthcare equity, and their intersections. Based on our expertise, and supported by synthesis of published studies, we propose recommendations for how D&I science is relevant for advancing healthcare equity. We used iterative discussions internally and at the Summit to refine preliminary findings and recommendations.

PRINCIPAL FINDINGS

We identified four guiding principles and three D&I science domains with strong promise for accelerating progress toward healthcare equity. We present eight recommendations and more than 60 opportunities for action by practitioners, healthcare leaders, policy makers, and researchers.

CONCLUSIONS

Promising areas for D&I science to impact healthcare equity include the following: attention to equity in the development and delivery of evidence-based interventions; the science of adaptation; de-implementation of low-value care; monitoring equity markers; organizational policies for healthcare equity; improving the economic evaluation of implementation; policy and dissemination research; and capacity building.

Trends in use of radiation therapy, chemotherapy, and combination chemoradiotherapy in advanced uterine cancer before, during, and after GOG 258

OBJECTIVE

To explore the use of Gynecologic Oncology Group 258 (GOG 258) study regimens before, during, and after the study.

METHODS

Patients aged 18 years or older with endometrial cancer between 2004-2019 were identified in the National Cancer Database. Inclusion criteria were stage III or IVA of any histology and stage I-IVA clear cell or serous histologies with positive washings that received adjuvant therapy. Adjuvant therapy use was examined in the pre-GOG 258 era (before 2009), during GOG 258 enrollment and maturation (2010-2017), and after results presentation in 2017 (2018-2019). Two-sided Cochran-Armitage tests, Wilcoxen rank sum tests, and χ2 tests were used for continuous and categorical variables. Multi-variable logistic regression assessed factors associated with the receipt of chemoradiotherapy compared with chemotherapy only or radiation therapy only.

RESULTS

From 2004 to 2019, 41 408 high-risk endometrial cancer patients received adjuvant therapy (12% radiation therapy, 38% chemotherapy, 50% chemoradiotherapy). Chemoradiotherapy increased over the GOG 258 study period (40% before study opening, 54% during enrollment, and 59% after results). Serous (OR 0.6, 95% CI 0.6 to 0.7) and clear cell histology (0.7, 0.6 to 0.8), higher grade (0.8, 0.7 to 0.9), and lymph node positivity (0.8, 0.7 to 0.9) were negatively associated with receipt of chemoradiotherapy compared with single-modality treatment. Non-Hispanic Black ethnicity (0.8, 0.8 to 0.9) and residing ≥50 miles from the treatment facility (0.8, 0.7 to 0.9) were also negatively associated with chemoradiotherapy. Private insurance (1.2, 1.0 to 1.4) and treatment at community hospitals (1.2, 1.2 to 1.3) were positively associated with chemoradiotherapy.

CONCLUSION

Despite the lack of benefit in the GOG 258 experimental arm, chemoradiotherapy use increased during study enrollment and after results publication.

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

The Myth of "Reliable Research" in Pediatric Gender Medicine: A critical evaluation of the Dutch Studies-and research that has followed

Two Dutch studies formed the foundation and the best available evidence for the practice of youth medical gender transition. We demonstrate that this work is methodologically flawed and should have never been used in medical settings as justification to scale this "innovative clinical practice." Three methodological biases undermine the research: (1) subject selection assured that only the most successful cases were included in the results; (2) the finding that "resolution of gender dysphoria" was due to the reversal of the questionnaire employed; (3) concomitant psychotherapy made it impossible to separate the effects of this intervention from those of hormones and surgery. We discuss the significant risk of harm that the Dutch research exposed, as well as the lack of applicability of the Dutch protocol to the currently escalating incidence of adolescent-onset, non-binary, psychiatrically challenged youth, who are preponderantly natal females. "Spin" problems-the tendency to present weak or negative results as certain and positive-continue to plague reports that originate from clinics that are actively administering hormonal and surgical interventions to youth. It is time for gender medicine to pay attention to the published objective systematic reviews and to the outcome uncertainties and definable potential harms to these vulnerable youth.

Randomized controlled trials in de-implementation research: a systematic scoping review

BACKGROUND

Healthcare costs are rising, and a substantial proportion of medical care is of little value. De-implementation of low-value practices is important for improving overall health outcomes and reducing costs. We aimed to identify and synthesize randomized controlled trials (RCTs) on de-implementation interventions and to provide guidance to improve future research.

METHODS

MEDLINE and Scopus up to May 24, 2021, for individual and cluster RCTs comparing de-implementation interventions to usual care, another intervention, or placebo. We applied independent duplicate assessment of eligibility, study characteristics, outcomes, intervention categories, implementation theories, and risk of bias.

RESULTS

Of the 227 eligible trials, 145 (64%) were cluster randomized trials (median 24 clusters; median follow-up time 305 days), and 82 (36%) were individually randomized trials (median follow-up time 274 days). Of the trials, 118 (52%) were published after 2010, 149 (66%) were conducted in a primary care setting, 163 (72%) aimed to reduce the use of drug treatment, 194 (85%) measured the total volume of care, and 64 (28%) low-value care use as outcomes. Of the trials, 48 (21%) described a theoretical basis for the intervention, and 40 (18%) had the study tailored by context-specific factors. Of the de-implementation interventions, 193 (85%) were targeted at physicians, 115 (51%) tested educational sessions, and 152 (67%) multicomponent interventions. Missing data led to high risk of bias in 137 (60%) trials, followed by baseline imbalances in 99 (44%), and deficiencies in allocation concealment in 56 (25%).

CONCLUSIONS

De-implementation trials were mainly conducted in primary care and typically aimed to reduce low-value drug treatments. Limitations of current de-implementation research may have led to unreliable effect estimates and decreased clinical applicability of studied de-implementation strategies. We identified potential research gaps, including de-implementation in secondary and tertiary care settings, and interventions targeted at other than physicians. Future trials could be improved by favoring simpler intervention designs, better control of potential confounders, larger number of clusters in cluster trials, considering context-specific factors when planning the intervention (tailoring), and using a theoretical basis in intervention design.

REGISTRATION

OSF Open Science Framework hk4b2.

Evidence reversals in primary care research: a study of randomized controlled trials

BACKGROUND

Evidence-Based Medicine is built on the premise that clinicians can be more confident when their decisions are grounded in high-quality evidence. Furthermore, evidence from studies involving patient-oriented outcomes is preferred when making decisions about tests or treatments. Ideally, the findings of relevant and valid trials should be stable over time, that is, unlikely to be reversed in subsequent research.

OBJECTIVE

To evaluate the stability of evidence from trials relevant to primary healthcare and to identify study characteristics associated with their reversal.

METHODS

We studied synopses of randomized controlled trials (RCTs) published from 2002 to 2005 as "Daily POEMs" (Patient Oriented Evidence that Matters). The initial evidence (E1) from these POEMs (2002-2005) was compared with the updated evidence (E2) on that same topic in a summary resource (DynaMed 2019). Two physician-raters independently categorized each POEM-RCT as (i) reversed when E1 ≠ E2, or as (ii) not reversed, when E1 = E2. For all "Evidence Reversals" (E1 ≠ E2), we assessed the direction of change in the evidence.

RESULTS

We evaluated 408 POEMs on RCTs. Of those, 35 (9%; 95% confidence interval [6-12]) were identified as reversed, 359 (88%) were identified as not reversed, and 14 (3%) were indeterminate. On average, this represents about 2 evidence reversals per annum for POEMs about RCTs.

CONCLUSIONS

Over 12-17 years, 9% of RCTs summarized as POEMs are reversed. Information alerting services that apply strict criteria for relevance and validity of clinical information are likely to identify RCTs whose findings are stable over time.

An Analysis of 5 Years of Randomized Trials in Gastroenterology and Hepatology Reveals 52 Medical Reversals

BACKGROUND AND AIMS

One manifestation of low-value medical practice is the medical reversal, a practice in widespread use that, once subjected to a randomized controlled trial (RCT), is found to be no better-or worse-than a prior established standard of care. We aimed to determine the prevalence of medical reversals in gastroenterology (GI) journals and characterize these reversals.

METHODS

We searched the American Journal of Gastroenterology, Clinical Gastroenterology and Hepatology, Gastroenterology, Gut, Hepatology, and the Journal of Hepatology, reviewing studies published in 2015-2019. We identified RCTs that tested an established clinical practice and produced negative results, considered tentative reversals. Any systematic review or meta-analysis that included the article was categorized as confirming the reversal, refuting the reversal, or providing insufficient data.

RESULTS

During the 5-year period, we identified 5,898 original articles, of which 212 tested an established practice and 52 were categorized as unrefuted medical reversals (25% of articles testing standard of care). Of the reversals, 21 (40%) tested procedures and devices, 15 (29%) tested medications, and 8 (15%) tested vitamins/supplements/diet. Twenty-three (44%) considered the alimentary tract, 12 (23%) considered the liver, pancreas, or biliary tract, and 17 (33%) considered endoscopy. Thirty-eight (73%) were funded exclusively by non-industry sources.

CONCLUSION

This review reveals a total of 52 reversals across all subfields of GI and medical, procedural, screening, and diagnostic interventions, occurring in 25% of randomized trials testing an established practice. More research is needed to determine the optimal way to engage stakeholders and remove reversed practices from medical care.

Extracting and Measuring Uncertain Biomedical Knowledge from Scientific Statements

Purpose

Given the information overload of scientific literature, there is an increasing need for computable biomedical knowledge buried in free text. This study aimed to develop a novel approach to extracting and measuring uncertain biomedical knowledge from scientific statements.

Design/methodology/approach

Taking cardiovascular research publications in China as a sample, we extracted subject–predicate–object triples (SPO triples) as knowledge units and unknown/hedging/conflicting uncertainties as the knowledge context. We introduced information entropy (IE) as potential metric to quantify the uncertainty of epistemic status of scientific knowledge represented at subject-object pairs (SO pairs) levels.

Findings

The results indicated an extraordinary growth of cardiovascular publications in China while only a modest growth of the novel SPO triples. After evaluating the uncertainty of biomedical knowledge with IE, we identified the Top 10 SO pairs with highest IE, which implied the epistemic status pluralism. Visual presentation of the SO pairs overlaid with uncertainty provided a comprehensive overview of clusters of biomedical knowledge and contending topics in cardiovascular research.

Research limitations

The current methods didn’t distinguish the specificity and probabilities of uncertainty cue words. The number of sentences surrounding a given triple may also influence the value of IE.

Practical implications

Our approach identified major uncertain knowledge areas such as diagnostic biomarkers, genetic polymorphism and co-existing risk factors related to cardiovascular diseases in China. These areas are suggested to be prioritized; new hypotheses need to be verified, while disputes, conflicts, and contradictions need to be settled.

Originality/value

We provided a novel approach by combining natural language processing and computational linguistics with informetric methods to extract and measure uncertain knowledge from scientific statements.

Pharmacological Treatment for Pedophilic Disorder and Compulsive Sexual Behavior Disorder: A Review

Guidelines for the pharmacological treatment of paraphilic disorders have historically been based on data from forensic settings and on risk levels for sexual crime. However, emerging treatment options are being evaluated for individuals experiencing distress because of their sexual urges and preferences, targeting both paraphilic disorders such as pedophilic disorder (PeD) and the new diagnosis of compulsive sexual behavior disorder (CSBD) included in the International Classification of Diseases, 11th Revision (ICD-11). As in other mental disorders, this may enable individualized pharmacological treatment plans, taking into account components of sexuality (e.g. high libido, compulsivity, anxiety-driven/sex as coping), medical and psychiatric comorbidity, adverse effects and patient preferences. In order to expand on previous reviews, we conducted a literature search focusing on randomized controlled trials of pharmacological treatment for persons likely to have PeD or CSBD. Our search was not restricted to studies involving forensic or criminal samples. Twelve studies conducted between 1974 and 2021 were identified regardless of setting (outpatient or inpatient), with only one study conducted during the last decade. Of a total of 213 participants included in these studies, 122 (57%) were likely to have PeD, 34 (16%) were likely to have a CSBD, and the remainder had unspecified paraphilias (40, 21%) or sexual offense (17, 8%) as the treatment indication. The diagnostic procedure for PeD and/or CSBD, as well as comorbid psychiatric symptoms, has been described in seven studies. The studies provide some empirical evidence that testosterone-lowering drugs reduce sexual activity for patients with PeD or CSBD, but the body of evidence is meager. There is a need for studies using larger samples, specific criteria for inclusion, longer follow-up periods, and standardized outcome measures with adherence to international reporting guidelines.

Recommendation Reversals in Gastroenterology Clinical Practice Guidelines

BACKGROUND

Recommendations in clinical practice guidelines (CPGs) may be reversed when evidence emerges to show they are futile or unsafe. In this study, we identified and characterized recommendation reversals in gastroenterology CPGs.

METHODS

We searched CPGs published by 20 gastroenterology societies from January 1990 to December 2019. We included guidelines which had at least two iterations of the same topic. We defined reversals as when (a) the more recent iteration of a CPG recommends against a specific practice that was previously recommend in an earlier iteration of a CPG from the same body, and (b) the recommendation in the previous iteration of the CPG is not replaced by a new diagnostic or therapeutic recommendation in the more recent iteration of the CPG. The primary outcome was the number of recommendation reversals. Secondary outcomes included the strength of recommendations and quality of evidence cited for reversals.

RESULTS

Twenty societies published 1022 CPGs from 1990 to 2019. Our sample for analysis included 129 unique CPGs. There were 11 recommendation reversals from 10 guidelines. New evidence was presented for 10 recommendation reversals. Meta-analyses were cited for two reversals, and randomized controlled trials (RCTs) for seven reversals. Recommendations were stronger after the reversal for three cases, weaker in two cases, and of similar strength in three cases. We were unable to compare recommendation strengths for three reversals.

CONCLUSION

Recommendation reversals in gastroenterology CPGs are uncommon but highlight low value or harmful practices.

How Often do Medical Specialties Question the Practices that They Perform? An Empirical, Cross-Sectional Analysis of the Published Literature

An aspect of overuse is who decides which practices are evaluated for overuse and which of the studies on overuse are published in the medical literature. We sought to examine the frequency with which studies in medical journals questioned an established practice. As a secondary objective, we sought to determine if there was variance among medical specialties. We conducted a retrospective, cross-sectional review of the published literature in 14 medical specialty journals. We included studies from one issue in three high-impact journals (November/December 2020) for each specialty. We assessed whether the study reported on a medical practice, whether it reported on an existing practice, whether the author expressed uncertainty regarding the practice, whether the study was a randomized design, and if the authors encouraged further testing in randomized studies. For all medical specialties combined, we found that 37% (n = 98) questioned existing practices, and 15% (n = 40) either tested the practice in a randomized trial or encouraged future randomized testing of the practice. The medical specialties that questioned their practices the most were gastroenterology (61%; n = 10/18), obstetrics/gynecology (52%; n = 11/21), and cardiovascular (50%; n = 5/10). These findings indicate that, although research is being conducted to examine current medical practices, few studies advocate for randomized testing of these practices, and even fewer actually test them in a randomized fashion. Additionally, the variation across medical specialties suggests areas in which to look for potential practices that are low-value, duplicative, and/or wasteful.

All-cause mortality as the primary endpoint for the GRAIL/National Health Service England multi-cancer screening trial

A randomized trial of the GRAIL Galleri^TM multi-cancer screening test is being planned for the National Health Service in England, and will have 140,000 healthy participants aged 50-79: 70,000 exposed to screening and 70,000 unexposed. The test reportedly detects 50 different cancers and is expected to reduce all-cancer mortality by approximately 25%. Given this effect size-and that cancer deaths constitute a large fraction of all deaths-the trial is sufficiently large to test the effect on all-cause mortality. Because most patients believe cancer screening "saves lives", the GRAIL/National Health Service collaboration could set the evaluation standard for multi-cancer screening.

A scoping review of de-implementation frameworks and models

BACKGROUND

Reduction or elimination of inappropriate, ineffective, or potentially harmful healthcare services and public health programs can help to ensure limited resources are used effectively. Frameworks and models (FM) are valuable tools in conceptualizing and guiding the study of de-implementation. This scoping review sought to identify and characterize FM that can be used to study de-implementation as a phenomenon and identify gaps in the literature to inform future model development and application for research.

METHODS

We searched nine databases and eleven journals from a broad array of disciplines (e.g., healthcare, public health, public policy) for de-implementation studies published between 1990 and June 2020. Two raters independently screened titles and abstracts, and then a pair of raters screened all full text records. We extracted information related to setting, discipline, study design, methodology, and FM characteristics from included studies.

RESULTS

The final search yielded 1860 records, from which we screened 126 full text records. We extracted data from 27 articles containing 27 unique FM. Most FM (n = 21) were applicable to two or more levels of the Socio-Ecological Framework, and most commonly assessed constructs were at the organization level (n = 18). Most FM (n = 18) depicted a linear relationship between constructs, few depicted a more complex structure, such as a nested or cyclical relationship. Thirteen studies applied FM in empirical investigations of de-implementation, while 14 articles were commentary or review papers that included FM.

CONCLUSION

De-implementation is a process studied in a broad array of disciplines, yet implementation science has thus far been limited in the integration of learnings from other fields. This review offers an overview of visual representations of FM that implementation researchers and practitioners can use to inform their work. Additional work is needed to test and refine existing FM and to determine the extent to which FM developed in one setting or for a particular topic can be applied to other contexts. Given the extensive availability of FM in implementation science, we suggest researchers build from existing FM rather than recreating novel FM.

REGISTRATION

Not registered.

Adaptive Trials in Cardiology: Some Considerations and Examples

Adaptive trials hold great promise to enhance the evidence base supporting medical interventions. In this review, we will describe the basic principles of an adaptive trial and the different types of adaptive trials, show examples of adaptive trials, and conclude with the advantages and challenges of different types of adaptive trials. While regulatory bodies have expressed a desire to see more adaptive trials, resistance in the community remains. We hope that this review helps to build greater acceptance of the concept of adaptive trial design.

The frequency of medical reversals in a cross-sectional analysis of high-impact oncology journals, 2009-2018

Background

Identifying ineffective practices that have been used in oncology is important in reducing wasted resources and harm. We sought to examine the prevalence of practices that are being used but have been shown in RCTs to be ineffective (medical reversals) in published oncology studies.

Methods

We cross-sectionally analyzed studies published in three high-impact oncology medical journals (2009–2018). We abstracted data relating to the frequency and characterization of medical reversals.

Results

Of the 64 oncology reversals, medications (44%) represented the most common intervention type (39% were targeted). Fourteen (22%) were funded by pharmaceutical/industry only and 56% were funded by an organization other than pharmaceutical/industry. The median number of years that the practice had been in use prior to the reversal study was 9 years (range 1–50 years).

Conclusion

Here we show that oncology reversals most often involve the administration of medications, have been practiced for years, and are often identified through studies funded by non-industry organizations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-021-08632-8.

Ethical Machine Learning in Healthcare

The use of machine learning (ML) in healthcare raises numerous ethical concerns, especially as models can amplify existing health inequities. Here, we outline ethical considerations for equitable ML in the advancement of healthcare. Specifically, we frame ethics of ML in healthcare through the lens of social justice. We describe ongoing efforts and outline challenges in a proposed pipeline of ethical ML in health, ranging from problem selection to postdeployment considerations. We close by summarizing recommendations to address these challenges.

Bowel preparation in colorectal surgery: the day of reckoning is here

Before the use of mechanical bowel preparation is either eliminated or subjected to a clinical trial, its scientific premise should be reconsidered, and consensus of its redesign discussed by a panel of experts including surgeons, infectious disease specialists, microbiome scientists, and clinical trialists.

DeImplementing Ineffective and Low-Value Clinical Practices: Research and Practice Opportunities in Community Oncology Settings

Patients, practitioners, and policy makers are increasingly concerned about the delivery of ineffective or low-value clinical practices in cancer care settings. Research is needed on how to effectively deimplement these types of practices from cancer care. In this commentary, we spotlight the National Cancer Institute Community Oncology Research Program (NCORP), a national network of community oncology practices, and elaborate on how it is an ideal infrastructure for conducting rigorous, real-world research on deimplementation. We describe key multilevel issues that affect deimplementation and also serve as a guidepost for developing strategies to drive deimplementation. We describe optimal study designs for testing deimplementation strategies and elaborate on how and why the NCORP network is uniquely positioned to conduct rigorous and impactful deimplementation trials. The number and diversity of affiliated community oncology care sites, coupled with the overall objective of improving cancer care delivery, make the NCORP an opportune infrastructure for advancing deimplementation research while simultaneously improving the care of millions of cancer patients nationwide.

Towards medical knowmetrics: representing and computing medical knowledge using semantic predications as the knowledge unit and the uncertainty as the knowledge context

In China, Prof. Hongzhou Zhao and Zeyuan Liu are the pioneers of the concept “knowledge unit” and “knowmetrics” for measuring knowledge. However, the definition on “computable knowledge object” remains controversial so far in different fields. For example, it is defined as (1) quantitative scientific concept in natural science and engineering, (2) knowledge point in the field of education research, and (3) semantic predications, i.e., Subject-Predicate-Object (SPO) triples in biomedical fields. The Semantic MEDLINE Database (SemMedDB), a high-quality public repository of SPO triples extracted from medical literature, provides a basic data infrastructure for measuring medical knowledge. In general, the study of extracting SPO triples as computable knowledge unit from unstructured scientific text has been overwhelmingly focusing on scientific knowledge per se. Since the SPO triples would be possibly extracted from hypothetical, speculative statements or even conflicting and contradictory assertions, the knowledge status (i.e., the uncertainty), which serves as an integral and critical part of scientific knowledge has been largely overlooked. This article aims to put forward a framework for Medical Knowmetrics using the SPO triples as the knowledge unit and the uncertainty as the knowledge context. The lung cancer publications dataset is used to validate the proposed framework. The uncertainty of medical knowledge and how its status evolves over time indirectly reflect the strength of competing knowledge claims, and the probability of certainty for a given SPO triple. We try to discuss the new insights using the uncertainty-centric approaches to detect research fronts, and identify knowledge claims with high certainty level, in order to improve the efficacy of knowledge-driven decision support.

[The problem of medical overuse : Finding a definition and solutions]

High-quality medical care including the concepts of "patient-centered medicine" and "precision medicine" imply medical awareness of measures that are "too much" and thus not appropriate for certain patients in a certain context. Physicians occupy a central role as stewards of limited social resources. Numerous influencing factors can cause a cascading into medical overuse. How to identify and avoid overuse? When is "less medicine" the better medicine for an individual patient?

Thickened liquids: do they still have a place in the dysphagia toolkit?

PURPOSE OF REVIEW

The use of commercially or naturally thickened liquids is a well-established treatment for patients with dysphagia to fluids, the aim of which is to improve swallow safety by minimizing risk of aspiration. Although the most recent systematic reviews conclude that this treatment lacks evidential support and leads to patient-reported worsening health and quality of life, thickened liquids continue to be used with patients with dysphagia across clinical settings. This review briefly summarizes the evidence and considers potential reasons for the apparent mismatch between the evidence and clinical practice.

RECENT FINDINGS

Continuing practice with thickened liquids is influenced by a range of factors, including gaps in clinical knowledge, inadequate patient involvement, a culture of common practice and a reliance on invalid surrogate studies or research lacking a credible association between thickened liquids and clinically meaningful endpoints.

SUMMARY

While awaiting further research, clinical decision-making about thickened liquids can be improved by considering the evidence of clinically meaningful endpoints, promoting shared decision-making with patients and underpinning practice with knowledge about the complex relationship between dysphagia, aspiration and pneumonia.

Variability in Human-Animal Interaction Research

The field of Human-Animal Interaction (HAI) is plagued with mixed results. Some findings appear to indicate that interacting with a companion animal is beneficial for some aspect of human health and well-being, while other research outcomes are inconclusive or even indicate the opposite. The purpose of this paper is to take a closer look at this variability in research outcomes and to provide plausible explanations and potential remedies. Some of the reasons for mixed results are likely due to the wide variety of methodologies implemented, intermittent use of standardized measures and manualized protocols, variability in human and animal participants, and limited quantification of human-animal interactions or definitions of pet ownership. Variability in research outcomes is not unique to HAI and is, in fact, not uncommon in many more established fields such as psychology and medicine. However, the potential reasons for the variability may be linked to the unique nature of HAI in that, in its' simplest form, it involves two complex organisms, a human and an animal, interacting in dynamic ways. We argue that this complexity makes research in this field particularly challenging and requires a broad spectrum of theoretical and methodological considerations to improve rigor while ensuring the validity and reliability of conclusions drawn from study results.

What makes a great clinical trial in physiotherapy?

Objective: To identify common characteristics of landmark physiotherapy clinical trials.Methods: The Physiotherapy Evidence Database (PEDro) top five trials were compared to 91 physiotherapy trials published in top medical journals and 99 trials randomly selected from PEDro on the following characteristics: PEDro score, sample size, number of trial sites, use of prospective registration, positive or negative trial, citations, citations in guidelines, Altmetric score, impact factor, publications and citations of first and last author, and PEDro codes (sub-discipline, topic, problem, therapy, and body part). Trials were published from 2014 to 2019. One-way independent ANOVA and Chi-squared test evaluated between-group differences.Results: Compared to a random sample of physiotherapy trials, the PEDro top five trials and trials in top medical journals have higher PEDro scores, larger sample sizes, more study sites, more citations (including in guidelines), higher Altmetric scores, more likely to be prospectively registered, less likely to be positive trials, and have first and last authors with more citations and publications. The problem was the only PEDro code was distributed differently across the trial groups.Conclusion: The PEDro top five trials and physiotherapy trials published in the top medical journals have characteristics that may inform the design, conduct, and reporting of future physiotherapy trials.

Clinical trials may be unethical in certain instances

After review of the literature, I conclude that clinical trials may be unethical.

Developing an open educational resource for open research: Protocol for the PaPOR TRAIL project

Background: Open research involves actions at all stages of the research cycle to make the research process and outputs more transparent and accessible. Though a number of initiatives exist for researchers at PhD, post-doctoral and more senior levels, there remains a critical need for educational resources for research students at earlier career stages and across disciplines. The aim of the Principles and Practices of Open Research: Teaching, Research, Impact, and Learning (PaPOR TRaIL) project is to develop an open educational resource (OER) on the principles and practice of open research for undergraduate and master's students. Methods: In stage 1, interviews and surveys of students and supervisors are being conducted to explore students' and supervisors' knowledge, attitudes, and experiences of open research, in addition to needs and preferences for the content and delivery of the OER. Stage 2 involves development of the OER content and delivery, based on Stage 1 engagement and national and international guidance on best practice in conducting and teaching open research. In Stage 3, students and supervisors will evaluate the developed OER and provide feedback in terms of OER usability, learning experience and learning outcomes. This feedback will guide revisions and finalisation of the OER content, format and learning activities. Discussion: The PaPOR TRaIL project will develop an evidence-based OER that provides a foundation in all aspects of open research theory & practice. Teaching undergraduate and master's students open research will promote development of core research values and equip them with transferable competencies and skills, including how to conduct and use research in a trustworthy and ethical manner within and beyond academia. Enhancing teaching and learning of open research will promote better teaching and research outcomes that will benefit individuals, universities, and science more broadly.

The ethical and legal status of neurosurgical guidelines: the neurosurgeon's golden fleece or Achilles' heel?

Neurosurgical guidelines are fundamental for evidence-based practice and have considerably increased both in number and content over the last decades. Yet, guidelines in neurosurgery are not without limitations, as they are overwhelmingly based on low-level evidence. Such recommendations have in the past been occasionally overturned by well-designed randomized controlled trials (RCTs), demonstrating the volatility of poorly underpinned evidence. Furthermore, even RCTs in surgery come with several limitations; most notably, interventions are often insufficiently standardized and assume a homogeneous patient population, which is not always applicable to neurosurgery. Lastly, guidelines are often outdated by the time they are published and smaller fields such as neurosurgery may lack a sufficient workforce to provide regular updates. These limitations raise the question of whether it is ethical to use low-level evidence for guideline recommendations, and if so, how strictly guidelines should be adhered to from an ethical and legal perspective. This article aims to offer a critical approach to the ethical and legal status of guidelines in neurosurgery. To this aim, the authors discuss: 1) the current state of neurosurgical guidelines and the evidence they are based on; 2) the degree of implementation of these guidelines; 3) the legal status of guidelines in medical disciplinary cases; and 4) the ethical balance between confident and critical use of guidelines. Ultimately, guidelines are neither laws that should always be followed nor purely academic efforts with little practical use. Every patient is unique, and tailored treatment defined by the surgeon will ensure optimal care; guidelines play an important role in creating a solid base that can be adhered to or deviated from, depending on the situation. From a research perspective, it is inevitable to rely on weaker evidence initially in order to generate more robust evidence later, and clinician-researchers have an ethical duty to contribute to generating and improving neurosurgical guidelines.

The Epistemological (Not Reproducibility) Crisis

The current reproducibility crisis is fundamentally a crisis of knowledge, thus in reality it is an epistemological crisis. The current reigning paradigm of null hypothesis testing using a P value of <.05 has made the medical literature prone to be filled with spurious correlations rather than true knowledge. This article brings attention to 3 foundational issues to help navigate the current crisis: The problem of induction, the concept of epistemological access, and the iatrogenics of information. Scientific reasoning is inductive reasoning and the problem of induction highlights the limitations of such knowledge. The concept of epistemological access is introduced to describe the inability of low-level data to extract true findings. This lack of true knowledge brings with it the iatrogenics of information, where having more data are in fact harmful and can lead to patients receiving ineffective treatments.

Quality of evidence of the efficacy of therapeutic interventions on patient-important outcomes in Cochrane's systematic reviews' abstracts: A survey

OBJECTIVE

The objective of this study was to evaluate the proportion of therapeutics that have proven their efficacy on patient-important outcomes with a high quality of evidence among Cochrane systematic reviews (SRs).

METHODS

We surveyed a random sample of 400 SRs' abstracts published between September 2012 and December 2015, which compared therapeutic interventions with at least a placebo or no intervention control. The primary endpoint was the proportion of SRs with a statistically significant efficacy on a patient-important outcome and with a high quality of evidence.

RESULTS

Among the 400 abstracts surveyed, 32 (8%) found efficacy on a patient-important outcome with a high quality of evidence. Half of the 400 SRs (50.2%) evaluated a pharmacological intervention and 12% of these found efficacy of the intervention on a patient-important outcome with a reported high quality of evidence.

CONCLUSION

Based on an analysis of 400 abstracts of SRs from the Cochrane Collaboration, we found that there is a low number of therapeutic interventions which have proven their efficacy on patient-important outcomes with a high quality of evidence.

Evaluation of reproducible and transparent research practices in pulmonology

BACKGROUND

Study reproducibility is valuable for validating or refuting results. Provision of reproducibility indicators, such as materials, protocols, and raw data in a study improve its potential for reproduction. Efforts to reproduce noteworthy studies in the biomedical sciences have resulted in an overwhelming majority of them being found to be unreplicable, causing concern for the integrity of research in other fields, including medical specialties. Here, we analyzed the reproducibility of studies in the field of pulmonology.

METHODS

500 pulmonology articles were randomly selected from an initial PubMed search for data extraction. Two authors scoured these articles for reproducibility indicators including materials, protocols, raw data, analysis scripts, inclusion in systematic reviews, and citations by replication studies as well as other factors of research transparency including open accessibility, funding source and competing interest disclosures, and study preregistration.

FINDINGS

Few publications included statements regarding materials (10%), protocols (1%), data (15%), and analysis script (0%) availability. Less than 10% indicated preregistration. More than half of the publications analyzed failed to provide a funding statement. Conversely, 63% of the publications were open access and 73% included a conflict of interest statement.

INTERPRETATION

Overall, our study indicates pulmonology research is currently lacking in efforts to increase replicability. Future studies should focus on providing sufficient information regarding materials, protocols, raw data, and analysis scripts, among other indicators, for the sake of clinical decisions that depend on replicable or refutable results from the primary literature.