Are systematic reviews and meta-analyses still useful research? No

The Australian Traumatic Brain Injury Initiative: Statement of Working Principles and Rapid Review of Methods to Define Data Dictionaries for Neurological Conditions

The Australian Traumatic Brain Injury Initiative (AUS-TBI) aims to develop a health informatics approach to collect data predictive of outcomes for persons with moderate-severe TBI across Australia. Central to this approach is a data dictionary; however, no systematic reviews of methods to define and develop data dictionaries exist to-date. This rapid systematic review aimed to identify and characterize methods for designing data dictionaries to collect outcomes or variables in persons with neurological conditions. Database searches were conducted from inception through October 2021. Records were screened in two stages against set criteria to identify methods to define data dictionaries for neurological conditions (International Classification of Diseases, 11th Revision: 08, 22, and 23). Standardized data were extracted. Processes were checked at each stage by independent review of a random 25% of records. Consensus was reached through discussion where necessary. Thirty-nine initiatives were identified across 29 neurological conditions. No single established or recommended method for defining a data dictionary was identified. Nine initiatives conducted systematic reviews to collate information before implementing a consensus process. Thirty-seven initiatives consulted with end-users. Methods of consultation were "roundtable" discussion (n = 30); with facilitation (n = 16); that was iterative (n = 27); and frequently conducted in-person (n = 27). Researcher stakeholders were involved in all initiatives and clinicians in 25. Importantly, only six initiatives involved persons with lived experience of TBI and four involved carers. Methods for defining data dictionaries were variable and reporting is sparse. Our findings are instructive for AUS-TBI and can be used to further development of methods for defining data dictionaries.

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

Eye movement desensitization and reprocessing for the treatment and early intervention of trauma among first responders: A systematic review

First responders are exposed to repetitive work-related trauma and, thus, are at risk of developing posttraumatic stress disorder (PTSD). Eye-movement desensitization and reprocessing (EMDR) is a psychotherapy intervention designed to treat symptoms of posttraumatic stress. We conducted a systematic review to examine the viability of EMDR among first responders. The primary aim of this review was to identify studies that have trialed EMDR among first responders and evaluate its effectiveness in reducing trauma-related symptoms; a secondary aim was to identify whether EMDR has been used as an early intervention for this cohort and determine its effectiveness as such. Four databases were searched. Studies were included if they evaluated the extent to which EMDR was effective in alleviating symptoms stemming from work-related trauma exposure among first responders. The findings from each study were reported descriptively, and eight studies that evaluated the efficacy of EMDR in this population were included. There was substantial variation in how EMDR was implemented, particularly in the type, duration, frequency, and timing. The findings suggest that EMDR can alleviate symptoms of work-related trauma exposure among first responders; however, findings regarding early intervention were inconclusive, and a methodological quality assessment revealed that all studies were classified as being of either weak or medium quality. Although this review provides preliminary insights into the effectiveness of EMDR for first responders, the conclusions that can be drawn from the literature are limited, and the findings highlight several gaps in the literature.

More consideration is needed for retracted non-Cochrane systematic reviews in medicine: a systematic review

OBJECTIVE

To analyze the retraction status and reasons of non-Cochrane systematic reviews (SRs) in medicine.

STUDY DESIGN AND SETTING

MEDLINE, Embase, Retraction Watch Database and Google Scholar were systematically searched to find all retracted non-Cochrane SRs.

RESULTS

Of 159 non-Cochrane SRs in medicine retracted between 2004 and 2020, more than 70% were led by authors from China and affiliated with hospitals. The largest proportion of retraction notices were issued by the publisher and editor(s) jointly. Fraudulent peer-review was the most common reason for retraction, followed by unreliable data meaning errors in study selection or data analysis. The median time between publication and retraction was 14 months, and SRs retracted due to research misconduct took longer to retract than honest error.

CONCLUSION

The total number of retracted SRs is increasing worldwide, in particular in China. The most common reasons for retraction are fraudulent peer-review and unreliable data, and in most cases the SR is retracted more than a year after publication. Better systems of ethical oversight and culture to improve the process of peer review and adherence to the COPE retraction guidance are needed, and authors should strengthen their skills in SR methodology.

COVID 19 Scoping: A Systemic Review and Meta-Analysis

Introduction:

The world has faced the pandemic of COVID-19 in the march of 2020 and still it continues to effect in 2021. Hence, in the present study we aim to evaluate the gulps in the research so that certain recommendations can be made for the future research. We conducted a scoping review of the COVID meta-analysis.

Materials and Methods:

Online data was collected from the search engines of EBSCO, PubMed, Google Scholar, and Scopus. The searched terms were COVID-19, CORONA, SARS-CoV-2, clinical features, Wuhan, etc. The study articles were collected that from January 2020 to February 2021. Based on the PRISMA guidelines, the meta-analysis was performed.

Results:

In the present study, we finalized 316 articles. On February 2020, the first article was published. We observed a spike in the meta-analysis later on. Most of the meta-analysis were issued in the virology and infection magazines. As expected, the majority studies were from Wuhan. The other countries that published the meta-analysis were the USA, the UK, and Italy. The studies included in each meta-analysis were nearly 25 and the subjects were approximately 16 thousand. However, we noticed a poor quality in majority of these meta-analysis and <10% of all the meta-analysis showed higher confidence.

Conclusion:

A poor quality of the meta-analysis has predominated the data and very few are of high quality. All the journal editors and the reviewing team should verify and thoroughly organize the protocol so that only high quality meta-analysis are encouraged.

Publication of meta-analyses in sleep medicine: a scoping review

STUDY OBJECTIVES

Sleep research has grown substantially in recent decades, producing a large amount of data and an increasing number of meta-analyses. This study sought to establish the volume of meta-analyses in this area and assess how this level of material has developed over time.

METHODS

A bibliographic search of the Web of Science database was conducted (1945-2019). The total number of articles and the total number of meta-analyses were extracted for both sleep medicine and a combination of 6 other medical specialties (cardiology, neurology, psychiatry, pulmonology, otorhinolaryngology, and pediatrics).

RESULTS

A total of 262,384 articles and 1,152 meta-analyses related to sleep medicine were identified. Considering the whole period under analysis, meta-analyses represented 0.44% of the total number of sleep medicine-related articles. Throughout this period, the proportion of meta-analyses published has been increasing in both sleep medicine and the other fields, but it is greater in the other fields. In 2019, meta-analyses in sleep medicine represented 1.10% of the publication output in this area but represented 1.62% of the other areas. However, sleep medicine's growth rate has been consistently higher than in the other fields. The United States, China, and the United Kingdom have been the top meta-analysis producers.

CONCLUSIONS

Meta-analyses in sleep medicine are underused. As a recent medical field, sleep medicine has more potential to grow and is likely to grow faster than other fields. Researchers should be encouraged to perform and publish meta-analyses on sleep medicine, as long as the analyses are reasonable and feasible from methodological, statistical. and practical perspectives.

COVID-19 meta-analyses: a scoping review and quality assessment

OBJECTIVE

To carry out a scoping review of the meta-analyses published regarding about coronavirus disease 2019 (COVID-19), evaluating their main characteristics, publication trends and methodological quality.

METHODS

A bibliometric search was performed in PubMed®, Scopus and Web of Science, focusing on meta-analyses about COVID-2019 disease. Bibliometric and descriptive data for the included articles were extracted and the methodological quality of the included meta-analyses was evaluated using A Measurement Tool to Assess Systematic Reviews.

RESULTS

A total of 348 meta-analyses were considered eligible. The first meta-analysis about COVID-19 disease was published on February 26, 2020, and the number of meta-analyses has grown rapidly since then. Most of them were published in infectious disease and virology journals. The greatest number come from China, followed by the United States, Italy and the United Kingdom. On average, these meta-analyses included 23 studies and 15,200 participants. Overall quality was remarkably low, and only 8.9% of them could be considered as of high confidence level.

CONCLUSION

Although well-designed meta-analyses about COVID-19 disease have already been published, the majority are of low quality. Thus, all stakeholders playing a role in COVID-19 deseases, including policy makers, researchers, publishers and journals, should prioritize well-designed meta-analyses, performed only when the background information seem suitable, and discouraging those of low quality or that use suboptimal methods.

PROTOCOL: When and how to replicate systematic reviews

This is a protocol for a co-registered Cochrane and Campbell Review (Methodology). The objectives are as follows: To identify, describe and assess methods for: when to replicate a systematic review; how to replicate a systematic review.

A world map of evidence-based medicine: Density equalizing mapping of the Cochrane database of systematic reviews

Systematic reviews represent the core and backbone of evidence-based medicine (EBM) strategies in all fields of medicine. In order to depict a first global sketch of the international efforts in the Cochrane database systematic reviews (CDSR), we analyzed the systematic reviews of the Cochrane database. Our global maps of systematic reviewing offer intriguing structural insights into the world of EBM strategies. They demonstrate that for the CDSR, the UK and Commonwealth countries take the lead position. Since patients, care providers and health systems all over the world benefit from systematic reviewing, institutions in other countries should increase their commitment.

Study-based registers reduce waste in systematic reviewing: discussion and case report

BACKGROUND

Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.

METHODS

We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results-reducing considerable redundancy of effort.

RESULTS

SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.

CONCLUSION

Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.