Systematic reviews do not (yet) represent the 'gold standard' of evidence: A position paper

Is the quality of systematic reviews influenced by prospective registration: a methods review of systematic musculoskeletal physical therapy reviews

INTRODUCTION

It is unknown if verified prospective registration of systematic reviews (SRs) and the randomized clinical trials (RCTs) that they use affect an SR's methodological quality on A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2).

METHODS

Data originated from interventional SRs published in International Society of Physiotherapy Journals Editors (ISPJE) member journals, indexed in MEDLINE, between 1 January 2018 and 18 August 2021. Blinded reviewers identified the SRs and extracted the data for the variables of interest for the SRs and the RCTs.

RESULTS

Two of 14 ISPJE member journals required prospective SR registration. Twenty SRs were identified, and 169 unique, retrievable RCTs were included within those SRs. One (5.0%) of the 20 SRs and 15 of the 169 (8.9%) RCTs were prospectively registered and published consistent with this intent. Nineteen (95.0%) of the 20 identified SRs was categorized as 'critically low' on the AMSTAR 2.

DISCUSSION

SRs and the RCTs identified within them were infrequently prospectively registered, prospectively verifiable, or prospectively verified based on the established research record.

CONCLUSIONS

Ensuring that SRs and RCTs have fidelity with the research record from conception to publication may help rule out low-value interventions, decrease variability in physical therapy practice, and solidify evidence-based physical therapy practice.

Flawed, futile, and fabricated-features that limit confidence in clinical research in pain and anaesthesia: a narrative review

The randomised controlled trial is the foundation of clinical research; yet there is concern that many trials have flaws in design, conduct, and reporting that undermine trustworthiness. Common flaws in trials include high risk of bias, small size, outcomes irrelevant to clinical care and patient's experience, and inability to detect efficacy even if present. These flaws carry forward into systematic reviews, which can confer the label of 'high-quality evidence' on inadequate data. Trials can be futile because their flaws mean that they cannot deliver any meaningful result in that different results in a small number of patients would be sufficient to change conclusions. Some trials have been discovered to be fabricated, the number of which is growing. The fields of anaesthesia and pain have more fabricated trials than other clinical fields, possibly because of increased vigilance. This narrative review examines these themes in depth whilst acknowledging an inescapable conclusion: that much of our clinical evidence is in trouble, and special measures are needed to bolster quality and confidence.

Biological and psychological early prognostic factors in complex regional pain syndrome: A systematic review

BACKGROUND AND OBJECTIVE

Several risk factors for the onset of CRPS have been found, but evidence for prognostic factors associated with the progression of this condition remains sparse. However, the detection and management of these factors are necessary to design secondary prevention strategies. The objective of this systematic review was to identify prognostic factors in adult individuals with early CRPS.

DATABASE AND DATA TREATMENT

PubMed, Embase, PsycINFO, Cochrane Library and Scopus, were published between January 1990 and November 2021. Two independent investigators selected cross-sectional and longitudinal studies looking at early (<12 weeks from onset) prognostic factors for pain, CRPS severity score, disability, return to work, or quality of life. The quality in prognostic studies (QUIPS) tool was used to assess the risk of bias. A qualitative meta-synthesis was performed.

RESULTS

Out of 4652 different articles, six studies met the inclusion criteria. We identified 21 early factors associated with a poorer prognosis in type I CRPS. We found moderate evidence to support six of them: higher pain intensity, self-rated disability, anxiety, pain-related fear, being a female and high-energy triggering event. Only two studies had an overall low risk of bias.

CONCLUSIONS

This study showed an important lack of information on early prognostic factors in CRPS. Only one article investigated the link with psychological characteristics. There is a crucial need for larger studies, with a well-defined population using validated measures.

SIGNIFICANCE

This systematic review highlights the lack of knowledge about early prognostic factors in CRPS. A few putative prognostic factors were identified. Most of the moderate evidence is related to a single cohort. Future research is required to find out which patients are vulnerable to chronification.

The problems with systematic reviews: a living systematic review

OBJECTIVES

Systematic reviews and meta-analyses are proliferating as they are an important building block to inform evidence-based guidelines and decision-making. Enforcement of best practice in clinical trials is firmly on the research agenda of good clinical practice, but there is less clarity as to how evidence syntheses that combine these studies can be influenced by bad practice. Our aim was to conduct a living systematic review of articles that highlight flaws in published systematic reviews to formally document and understand these problems.

STUDY DESIGN AND SETTING

We conducted a comprehensive assessment of all literature examining problems, which relate to published systematic reviews.

RESULTS

The first iteration of our living systematic review (https://systematicreviewlution.com/) has found 485 articles documenting 67 discrete problems relating to the conduct and reporting of systematic reviews which can potentially jeopardize their reliability or validity.

CONCLUSION

Many hundreds of articles highlight that there are many flaws in the conduct, methods, and reporting of published systematic reviews, despite the existence and frequent application of guidelines. Considering the pivotal role that systematic reviews have in medical decision-making due to having apparently transparent, objective, and replicable processes, a failure to appreciate and regulate problems with these highly cited research designs is a threat to credible science.

Effect of mirror therapy in the treatment of phantom limb pain in amputees: A systematic review of randomized placebo-controlled trials does not find any evidence of efficacy

BACKGROUND AND OBJECTIVE

Phantom limb pain (PLP) concerns >50% of amputees and has a negative impact on their rehabilitation, mental health and quality of life. Mirror therapy (MT) is a promising strategy, but its effectiveness remains controversial. We performed a systematic review to: (i) evaluate the effectiveness of MT versus placebo in reducing PLP, and (ii) determine MT effect on disability and quality of life.

DATABASES AND DATA TREATMENT

We selected randomized-controlled trials in five databases (Medline, Cochrane Library, CINAHL, PEDro and Embase) that included patients with unilateral lower or upper limb amputation and PLP and that compared the effects on PLP of MT versus a placebo technique. The primary outcome was PLP intensity changes and the secondary outcomes were PLP duration, frequency, patients' disability and quality of life.

RESULTS

Among the five studies included, only one reported a significant difference between the MT group and control group, with a positive MT effect at week 4. Only one study assessed MT effect on disability and found a significant improvement in the MT group at week 10 and month 6.

CONCLUSIONS

Our systematic review did not allow concluding that MT reduces PLP and disability in amputees. This lack of strong evidence is probably due to (i) the low methodological quality of the included studies, and (ii) the lack of statistical power. Future trials should include a higher number of patients, increase the number and frequency of MT sessions, have a long-term follow-up and improve the methodological quality.

SIGNIFICANCE

Recent meta-analyses concluded that MT is effective for reducing phantom limb pain. Conversely, the present systematic review that included only studies with the best level of evidence did not find any evidence about its effectiveness for this condition. We identified many ways to improve future randomized-controlled trials on this topic: increasing the number of participants, reducing the intra-group heterogeneity, using a suitable placebo and intensifying the MT sessions and frequency.

Long-term observational studies with cannabis-based medicines for chronic non-cancer pain: A systematic review and meta-analysis of effectiveness and safety

BACKGROUND AND OBJECTIVE

This systematic review evaluated the effectiveness, tolerability and safety of cannabis-based medicines (CbMs) for chronic non-cancer pain (CNCP) in long-term observational studies.

DATABASES AND DATA TREATMENT

CENTRAL, EMBASE and MEDLINE were searched to December 2021. We included prospective observational studies with a study duration ≥ 26 weeks. Pooled estimates of event rates of categorical data and standardized mean differences (SMD) of continuous variables were calculated using a random effects model.

RESULTS

Six studies were included with 2686 participants, with study duration ranging between 26 and 52 weeks. Pain conditions included were nociceptive, nociplastic, neuropathic and mixed pain. The certainty of evidence for every outcome was very low. The weighted mean difference of mean pain reduction was 1.75 (95% Confidence interval [CI] 0.72 to 2.78) on a 0-10 scale. 20.8 % (95% CI 10.2 % to 34.0 %) of patients reported pain relief of 50% or greater. The effect size for sleep problems was moderate and for depression and anxiety was low. Study completions was reported for 53.3% (95% CI 26.8% to 79.9%) of patients, with dropouts of 6.8 % (95% CI 4.3% to 9.7%) due to adverse events. Serious adverse events occurred in 3.0% (95 CI 0.02 % to 12.8%) and 0.3 % (95% CI 0.1% to 0.6%) of patients died.

CONCLUSIONS

Information included in observational studies should be regarded with caution.Within the context of observational studies, CbMs had positive effects on multiple symptoms for some CNCP patients and were generally well tolerated and safe.