Critically Low Confidence in the Results Produced by Spine Surgery Systematic Reviews: An AMSTAR-2 Evaluation From 4 Spine Journals

Paper 5: a methodological overview of methods and interventions

OBJECTIVES

We aim to (1) evaluate the methods used in systematic reviews of interventions focused on racialized populations to improve racial health equity and (2) examine the types of interventions evaluated for advancing racial health equity in systematic reviews.

STUDY DESIGN AND SETTING

We searched MEDLINE, Cochrane, and Campbell databases for reviews evaluating interventions focused on racialized populations to mitigate racial health inequities, published from January 2020 to January 2023.

RESULTS

We analyzed 157 reviews on racialized populations. Only 22 (14%) reviews addressed racism's role in driving racial health inequities related to the review question. Eleven percent (7) of reviews considered intersectionality when conceptualizing racial inequities. Two-thirds (105, 67%) provided descriptive summaries of included studies rather than synthesizing them. Among those that quantified effect sizes, 54% (21) used biased synthesis methods like vote counting. The most common method assessed was tailoring interventions to meet the needs of racialized populations. Reviews mainly focused on assessing interventions to reduce racial disparities rather than enhancing structural opportunities for racialized populations.

CONCLUSION

Reviews for racial health equity could be improved by enhancing methodologic quality, defining the role of racism in the question, using reliable analytical methods, and assessing process and implementation outcomes. More focus is needed on assessing structural interventions to improve opportunities for racialized populations and prioritize these issues in political and social agendas.

An Overview of the Methodology Used to Develop Clinical Practice Guidelines for the Management of Acute and Intraoperative Spinal Cord Injury

STUDY DESIGN

An overview of the methods used to develop clinical practice guidelines (CPGs).

OBJECTIVES

Acute spinal cord injury (SCI) and intraoperative SCI (ISCI) can have devastating physical and psychological consequences for patients and their families. To date, there are several studies that have discussed the diagnostic and management strategies for both SCI and ISCI. CPGs in SCI help to distill and translate the current evidence into actionable recommendations, standardize care across centers, optimize patient outcomes, and reduce costs and unnecessary interventions. Furthermore, they can be used by patients to assist in making decisions about certain treatments and by policy makers to inform allocation of resources. The objective of this article is to summarize the methods used to develop CPGs for the timing of surgery and hemodynamic management of acute SCI, as well as the identification and treatment of ISCI.

METHODS

The CPGs were developed using standards established by the Institute of Medicine (now the National Academy of Medicine), the Guideline International Network and several other organizations. Systematic reviews were conducted according to accepted methodological standards (eg, Institute of Medicine, Agency for Healthcare Research and Quality and Patient-Centered Outcomes Research Institute) in order to summarize the current body of evidence and inform the guideline development process. Protocols for each guideline were created. A multidisciplinary guideline development group (GDG) was formed that included individuals living with SCI as well as clinicians from the broad range of specialties that encounter patients with SCI: spine or trauma surgeons, critical care physicians, rehabilitation specialists, neurologists, anesthesiologists and other healthcare professionals. Individuals living with SCI were also included in the GDG. The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach was used to rate the certainty of the evidence for each critical outcome. The "evidence to recommendation" framework was then used to translate the evidence obtained from the systematic review to an actionable recommendation. This framework provides structure when assessing the body of evidence and considers several additional factors when rating the strength of the recommendation, including the magnitude of benefits and harms, patient preferences, resource use, health equities, acceptability and feasibility. Finally, the CPGs were appraised both internally and externally.

RESULTS

The results of the CPGs for SCI are provided in separate articles in this focus issue.

CONCLUSIONS

Development of these CPGs for SCI followed the methodology proposed by the Institute of Medicine the Guideline International Network and the GRADE Working Group. It is anticipated that these CPGs will assist clinicians implement the best evidence into practice and facilitate shared-decision making with patients.

Flawed, futile, and fabricated-features that limit confidence in clinical research in pain and anaesthesia: a narrative review

The randomised controlled trial is the foundation of clinical research; yet there is concern that many trials have flaws in design, conduct, and reporting that undermine trustworthiness. Common flaws in trials include high risk of bias, small size, outcomes irrelevant to clinical care and patient's experience, and inability to detect efficacy even if present. These flaws carry forward into systematic reviews, which can confer the label of 'high-quality evidence' on inadequate data. Trials can be futile because their flaws mean that they cannot deliver any meaningful result in that different results in a small number of patients would be sufficient to change conclusions. Some trials have been discovered to be fabricated, the number of which is growing. The fields of anaesthesia and pain have more fabricated trials than other clinical fields, possibly because of increased vigilance. This narrative review examines these themes in depth whilst acknowledging an inescapable conclusion: that much of our clinical evidence is in trouble, and special measures are needed to bolster quality and confidence.

Can Communication Technologies Reduce Loneliness and Social Isolation in Older People? A Scoping Review of Reviews

BACKGROUND

Loneliness and social isolation in older age are considered major public health concerns and research on technology-based solutions is growing rapidly. This scoping review of reviews aims to summarize the communication technologies (CTs) (review question RQ1), theoretical frameworks (RQ2), study designs (RQ3), and positive effects of technology use (RQ4) present in the research field.

METHODS

A comprehensive multi-disciplinary, multi-database literature search was conducted. Identified reviews were analyzed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. A total of N = 28 research reviews that cover 248 primary studies spanning 50 years were included.

RESULTS

The majority of the included reviews addressed general internet and computer use (82% each) (RQ1). Of the 28 reviews, only one (4%) worked with a theoretical framework (RQ2) and 26 (93%) covered primary studies with quantitative-experimental designs (RQ3). The positive effects of technology use were shown in 55% of the outcome measures for loneliness and 44% of the outcome measures for social isolation (RQ4).

CONCLUSION

While research reviews show that CTs can reduce loneliness and social isolation in older people, causal evidence is limited and insights on innovative technologies such as augmented reality systems are scarce.

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019

BACKGROUND

Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines.

METHODS

We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups.

RESULTS

Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items.

CONCLUSION

The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

Systematic reviews and meta-analysis published in indexed Portuguese medical journals: time trends and critical appraisal

Introduction

Over the last years, the number of systematic reviews published is steadily increasing due to the global interest in this type of evidence synthesis. However, little is known about the characteristics of this research published in Portuguese medical journals. This study aims to evaluate the publication trends and overall quality of these systematic reviews.

Material and methods

This was a methodological study. We aimed the most visible Portuguese medical journals indexed in MEDLINE. Systematic reviews were identified through an electronic search (through PUBMED). We included systematic reviews published up to August 2020. Systematic reviews selection and data extraction were done independently by three authors. The overall quality critical appraisal using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) was independently assessed by three authors. Disagreements were solved by consensus.

Results

Sixty-six systematic reviews published in 5 Portuguese medical journals were included. Most (n = 53; 80.3%) were systematic reviews without meta-analysis. Up to 2010 there was a steady increase in the number of systematic reviews published, followed by a period of great variability of publication, ranging from 1 to 10 in a given year. According to the systematic reviews’ typology, most have been predominantly conducted to assess the effectiveness/efficacy of health interventions (n = 27; 40.9%). General and Internal Medicine (n = 20; 30.3%) was the most addressed field. Most systematic reviews (n = 46; 69.7%) were rated as being of “critically low-quality”.

Conclusions

There were consistent flaws in the methodological quality report of the systematic reviews included, particularly in establishing a prior protocol and not assessing the potential impact of the risk of bias on the results.

Through the years, the number of systematic reviews published increased, yet their quality is suboptimal. There is a need to improve the reporting of systematic reviews in Portuguese medical journals, which can be achieved by better adherence to quality checklists/tools.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01591-z.

Methodological assessment and overall confidence in the results of systematic reviews with network meta-analyses in Endodontics

AIM

The aims of the study were to assess the methodological quality of systematic reviews with network meta-analyses (NMAs) in Endodontics using the "A MeaSurement Tool to Assess systematic Reviews" (AMSTAR 2) tool, and to evaluate the overall confidence in the results of the individual reviews included in the analysis.

METHODOLOGY

Systematic reviews with NMAs within the specialty of Endodontics published in English were identified from the PubMed, EbBSCOhost and SCOPUS databases from inception to July 2021. Two reviewers were involved independently in the selection of the reviews, data extraction, methodological quality assessment and overall confidence rating. Disagreements were resolved by discussion between the reviewers to achieve consensus; if disagreements persisted, a third reviewer made the final decision. The methodological quality of the included NMAs was appraised using the AMSTAR 2 checklist, which contains 16 items. The reviewers scored each item - 'Yes' - when the item was fully addressed, 'Partial Yes' - when the item was not fully addressed, or 'No' - when the item was not addressed. The overall confidence in the results of each review was classified as 'High', 'Moderate', 'Low' or 'Critically low' based on the criteria reported by the AMSTAR 2 developers.

RESULTS

Twelve systematic reviews with NMAs were included. All the NMAs adequately reported Item 1 ("Did the research questions and inclusion criteria for the review include the components of PICO?"), Item 8 ("Did the review authors describe the included studies in adequate detail?"), Item 9 ("Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?") and Item 16 ("Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?") , whereas only one NMA reported Item 10 adequately ("Did the review authors report on the sources of funding for the studies included in the review?"). The overall confidence in the results of eight reviews was categorised as "Critically low", one review was "Low", two reviews were "Moderate" and one review was "High".

CONCLUSION

The overall confidence in the results for the majority of systematic reviews with NMAs in Endodontics was judged to be 'Critically low' as their methodological quality was below the necessary standard. AMSTAR 2 and PRISMA for NMA guidelines are available to guide authors to produce high quality systematic reviews with NMAs and for Editors and peer-reviewers when assessing submissions to journals.

Clozapine prescribing barriers in the management of treatment-resistant schizophrenia: A systematic review

BACKGROUND

Treatment-resistant schizophrenia is prevalent and difficult to manage, as patients fail multiple antipsychotic trials before being considered as treatment-resistant. Currently clozapine is the only Food and Drug Administration-approved pharmacotherapy for treatment-resistant schizophrenia but remains under-prescribed. The purpose of this study is to investigate recent literature on clozapine in order to identify barriers to prescribing clozapine and categorize the recommended solutions.

METHODS

We conducted a systematic review following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Using free text and the medical subject headings, we searched MEDLINE/PubMed electronic bibliographic database from 2017 until 2020. Eligible studies included peer-reviewed English language articles with multiple methodologies aiming to identify clozapine barriers in treatment-resistant schizophrenia. We used search terms combining clozapine AND treatment OR treatment-resistant schizophrenia AND barriers AND prescribing OR prescription OR prescriber. We merged search results in a citation manager software, removed duplicates, and screened the remaining articles based on the study eligibility criteria.

RESULTS

We retrieved 123 studies, however, only 10 articles exclusively met the study inclusion criteria for full text review. These studies represented 20 countries; 6 were exclusively conducted in the US. The top barriers delineated by the studies include: providers' lack of knowledge and training (n = 7), concern about side effects (n = 8), and poor adherence (n = 7). All studies described more than 1 barrier. Other barriers included prescriber-perceived barriers (n = 4), administrative barriers (n = 5), and other healthcare systems-related barriers (n = 3). Top recommendations to overcome clozapine prescription barriers included improving prescriber clozapine education/training, utilizing interdisciplinary teams and providing integrated care via clozapine clinics, and simplifying blood test monitoring.

CONCLUSION

Clozapine remains under-prescribed for patients with treatment-resistant schizophrenia due to multiple barriers related to the individual prescriber, system of care, and technology. It is recommended that by improving prescriber knowledge and training, use of integrated care, and use of technology that can enable continuous, real-time blood test monitoring, these barriers may be overcome.

An Overview of Systematic Reviews of Acupuncture for Infertile Women Undergoing in vitro Fertilization and Embryo Transfer

Background: Currently, more and more subfertility couples are opting for combined acupuncture to improve the success rate of in vitro fertilization and embryo transfer (IVF-ET). However, the efficacy and safety of acupuncture in IVF-ET is still highly controversial.

Objectives: The purpose of this overview is to summarize evidence of essential outcomes of systematic reviews (SRs) of acupuncture in IVF-ET and evaluate their methodological quality.

Methods: We conducted a comprehensive literature search for relevant SRs in eight databases from inception to July 31, 2020, without language restriction. We evaluated the methodological quality of the included SRs by using A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2), which was the latest available assessment tool. The Risk of Bias in Systematic Review (ROBIS) tool was used to assess the risk of bias in SRs. We assessed the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) score to determine the strength of evidence. We excluded the overlapping randomized controlled trials (RCTs) and performed a re-meta-analysis of the primary RCTs.

Results: This review included 312 original RCT studies and 65,388 participants. By using AMSTAR-2, we found that the methodological quality of 16 SRs was critically low, because they had more than one critical weakness. Our reviews showed that although the GRADE for quality of evidence profile was suboptimal, acupuncture seemed to be beneficial in increasing the pregnancy rate. Our re-meta-analysis suggested that acupuncture was superior to sham acupuncture in improving the clinical pregnancy rate (CPR) of IVF-ET with substantial heterogeneity (RR = 1.31, 95% CI: 1.13–1.52, p = 0.0004, I² = 66%). No statistical difference was observed regarding the outcomes of live birth rate (LBR), ongoing pregnancy rate (OPR), biochemical pregnancy rate (BPR), and miscarriage rate (MR) between two groups. When compared with no adjunctive treatment groups, acupuncture improved CPR (RR = 1. 25, 95% CI: 1.11–1.42, p = 0.0003) and OPR (RR = 1. 38, 95% CI: 1.04–1.83, p = 0.03). Acupuncture was more superior than no adjunctive treatment in reducing MR (OR = 1.42, 95% CI: 1.03–1.95, p = 0.03) and BPR (RR = 1.19, 95% CI: 1.02–1.37, p = 0.02).

Conclusions: Although the evidence of acupuncture in IVF-ET is insufficient, acupuncture appears to be beneficial to increase the clinical pregnancy rate in women undergoing IVF-ET. However, there are severe heterogeneity and methodological quality defects, which limit the reliability of results. Further, high-quality primary studies are still needed.