1
|
Sehmbi H, Retter S, Shah UJ, Nguyen D, Martin J, Uppal V. Methodological and reporting quality assessment of network meta-analyses in anesthesiology: a systematic review and meta-epidemiological study. Can J Anaesth 2023; 70:1461-1473. [PMID: 37420161 DOI: 10.1007/s12630-023-02510-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 12/29/2022] [Accepted: 01/04/2023] [Indexed: 07/09/2023] Open
Abstract
PURPOSE The scientific rigour of the conduct and reporting of anesthesiology network meta-analyses (NMAs) is unknown. This systematic review and meta-epidemiological study assessed the methodological and reporting quality of NMAs in anesthesiology. METHODS We searched four databases, including MEDLINE, PubMed, Embase, and the Cochrane Systematic Reviews Database, for anesthesiology NMAs published from inception to October 2020. We assessed the compliance of NMAs against A Measurement Tool to Assess Systematic Reviews (AMSTAR-2), Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement for Network Meta-Analyses (PRISMA-NMA), and PRISMA checklists. We measured the compliance across various items in AMSTAR-2 and PRISMA checklists and provided recommendations to improve quality. RESULTS Using the AMSTAR-2 rating method, 84% (52/62) of NMAs were rated "critically low." Quantitatively, the median [interquartile range] AMSTAR-2 score was 55 [44-69]%, while the PRISMA score was 70 [61-81]%. Methodological and reporting scores showed a strong correlation (R = 0.78). Anesthesiology NMAs had a higher AMSTAR-2 score and PRISMA score if they were published in higher impact factor journals (P = 0.006 and P = 0.01, respectively) or followed PRISMA-NMA reporting guidelines (P = 0.001 and P = 0.002, respectively). Network meta-analyses from China had lower scores (P < 0.001 and P < 0.001, respectively). Neither score improved over time (P = 0.69 and P = 0.67, respectively). CONCLUSION The current study highlights numerous methodological and reporting deficiencies in anesthesiology NMAs. Although the AMSTAR tool has been used to assess the methodological quality of NMAs, dedicated tools for conducting and assessing the methodological quality of NMAs are urgently required. STUDY REGISTRATION PROSPERO (CRD42021227997); first submitted 23 January 2021.
Collapse
Affiliation(s)
- Herman Sehmbi
- Department of Anesthesia & Perioperative Medicine, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
- London Health Sciences Centre, London, ON, Canada
| | - Susanne Retter
- Department of Anesthesia, Pain Management & Perioperative Medicine, Dalhousie University, 10W Victoria Building, 1276 South Park St, Halifax, NS, B3H 2Y9, Canada
- Nova Scotia Health Authority, Halifax, NS, Canada
| | - Ushma J Shah
- Department of Anesthesia & Perioperative Medicine, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Derek Nguyen
- Schulich School of Medicine & Dentistry, London, ON, Canada
| | - Janet Martin
- Department of Anesthesia & Perioperative Medicine, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
- Department of Epidemiology & Biostatistics, London Health Sciences Centre, Western University, London, ON, Canada
| | - Vishal Uppal
- Department of Anesthesia, Pain Management & Perioperative Medicine, Dalhousie University, 10W Victoria Building, 1276 South Park St, Halifax, NS, B3H 2Y9, Canada.
- Nova Scotia Health Authority, Halifax, NS, Canada.
- IWK Health Centre, Halifax, NS, Canada.
| |
Collapse
|
2
|
Veroniki AA, Tsokani S, Zevgiti S, Pagkalidou I, Kontouli KM, Ambarcioglu P, Pandis N, Lunny C, Nikolakopoulou A, Papakonstantinou T, Chaimani A, Straus SE, Hutton B, Tricco AC, Mavridis D, Salanti G. Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis. Syst Rev 2021; 10:246. [PMID: 34507621 PMCID: PMC8434710 DOI: 10.1186/s13643-021-01780-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Accepted: 07/28/2021] [Indexed: 01/07/2023] Open
Abstract
BACKGROUND The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement. METHODS We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015. RESULTS In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6-16%. CONCLUSIONS NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.
Collapse
Affiliation(s)
- Areti Angeliki Veroniki
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece. .,Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada.
| | - Sofia Tsokani
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
| | - Stella Zevgiti
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
| | - Irene Pagkalidou
- Department of Hygiene, Social-Preventive Medicine and Medical Statistics, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Katerina-Maria Kontouli
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
| | - Pinar Ambarcioglu
- Department of Biostatistics, Faculty of Veterinary Medicine, Mustafa Kemal University, Tayfur Sökmen Kampüsü 31060, Antakya, Hatay, Turkey
| | - Nikos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
| | - Carole Lunny
- Cochrane Hypertension Review Group and the Therapeutics Initiative, University of British Columbia, Vancouver, Canada
| | - Adriani Nikolakopoulou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany.,Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | | | - Anna Chaimani
- Université de Paris, Research Center of Epidemiology and Statistics Sorbonne Paris Cité (CRESS UMR1153), INSERM, INRA, Paris, France.,Cochrane France, Paris, France
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada.,Department of Geriatric Medicine, University of Toronto, Toronto, ON, Canada
| | - Brian Hutton
- Ottawa Hospital Research Institute, Ottawa, ON, Canada.,University of Ottawa School of Epidemiology and Public Health, Ottawa, ON, Canada
| | - Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada.,Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Dimitris Mavridis
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece.,Paris Descartes University, Sorbonne Paris CitéFaculté de Médecine, Paris, France
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
3
|
Bae K, Shin IS. Critical evaluation of reporting quality of network meta-analyses assessing the effectiveness of acupuncture. Complement Ther Clin Pract 2021; 45:101459. [PMID: 34388562 DOI: 10.1016/j.ctcp.2021.101459] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Revised: 07/15/2021] [Accepted: 07/25/2021] [Indexed: 11/29/2022]
Abstract
BACKGROUND and purpose: Network meta-analyses (NMAs) comparing the effectiveness of multiple acupuncture have been published but the key concepts underlying NMAs have not been properly reported. This critical evaluation aims to assess the completeness of reporting for NMAs of acupuncture to enhance the validity of findings. MATERIALS AND METHODS Five databases were searched. The characteristics and reporting quality based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension Statement for reporting of NMA (PRISMA-NMA) were evaluated. NMAs were categorized into two groups according to the publication date, reporting guideline, and intervention. The group differences and correlation coefficients were calculated. RESULTS Forty-two NMAs of acupuncture were evaluated. The reporting quality for new items in the key concepts of NMAs was low (median 70.0 % (interquartile range 60.0-80.0)). While the issue of consistency was well reported, the assumption of transitivity and network geometry showed poor reporting. Seventeen studies that followed the PRISMA-NMA guideline showed a higher reporting rate for essential concepts of NMA. The recency of publication did not guarantee clear reporting. CONCLUSION The reporting quality of NMAs of acupuncture was low. The researchers should follow the guidelines on the reporting of NMAs.
Collapse
Affiliation(s)
- Kyeore Bae
- Center for Immunity and Pain, Kwanghye Hospital, Seoul, Republic of Korea.
| | - In-Soo Shin
- Department of Graduate School of Education, Dongguk University, Seoul, Republic of Korea.
| |
Collapse
|