Two alternatives versus the standard Grading of Recommendations Assessment, Development and Evaluation (GRADE) summary of findings (SoF) tables to improve understanding in the presentation of systematic review results: a three-arm, randomised, controlled, non-inferiority trial

Tool to assess recognition and understanding of elements in Summary of Findings Table for health evidence synthesis: a cross-sectional study

of Findings (SoF) tables concisely present the main findings of evidence synthesis of health evidence, but how users navigate it to understand and interpret the presented information is not clear. We quantified the interaction of medical students with an SoF table while answering a knowledge quiz. Read&Learn tool was used to measure the number of target and non-target table cells visited for each question and the time spent on these cells. Students positively identified target elements for quiz questions and answered simpler questions, but struggled with critical thinking and understanding study outcomes. The question on outcomes with the largest improvement post-intervention had the fewest correct answers, the longest interaction with table cells and the most opened cells before answering. Students spent a median of 72% of the time reading target table cells. A heatmap of the interactions showed that they were mostly answer-oriented. Further development of the tool and metrics is needed to use the tool and the metrics to study the cognitive processes during the assessment of health evidence.

Improving grading of recommendations assessment, development, and evaluation evidence tables part 4: a three-arm noninferiority randomized trial demonstrates improved understanding of content in summary of findings tables with a new format

OBJECTIVES

To evaluate alternative formats of summary of findings (SoF) tables for single comparison with multiple outcomes.

STUDY DESIGN AND SETTING

We conducted a three-arm randomized controlled noninferiority trial (RCT) in the following systematic review (SR) users: researchers, clinical practice guideline developers, health care providers, policymakers, and knowledge transfer organizations to measure understanding, accessibility, satisfaction, and preference across the current grading of recommendations assessment, development, and evaluation (GRADE) SoF, an alternative GRADE SoF, or an adapted evidence-based practice center (EPC) program SoF table.

RESULTS

One Hundred Seventy-Nine participants were randomized, and 129 participants completed the RCT (n = 47 current GRADE, n = 41 alternative GRADE, n = 41 adapted EPC). Understanding the certainty of evidence and treatment effect was comparable across groups. The adapted EPC SoF table was inferior for quantifying risk and RD compared to the alternatives (<35% correct vs. >85% correct). Participants reported increased satisfaction when SoF tables presented number needed to treat (NNT), anticipated absolute effect differences, and narrative syntheses for evidence that could not be meta-analyzed. Participants reported accessibility to information as significantly better in both GRADE SoF tables, when compared with the adapted EPC SoF table. Participants preferred the alternative GRADE SoF table format.

CONCLUSION

The alternative GRADE SoF table is a promising format for SR users preferring a comprehensive presentation of SR results for single comparisons.

Comparing the usability of the World Health Organization's conventional tuberculosis guidelines to the eTB recommendations map: A two-arm superiority randomised controlled trial

Best practices for the dissemination of global health guidelines has not undergone rigorous research. We used a new approach to digitizing World Health Organization (WHO) global tuberculosis guideline recommendations (eTB RecMap) and compared its usability to the conventional method of accessing TB recommendations using the WHO website. We conducted a two-arm superiority randomised controlled trial using a survey among global stakeholders who were past or planned future users of TB guidelines, recommendations, or policy advice. We assigned participants randomly (1:1) to complete an activity using the WHO eTB RecMap or the conventional website. The primary outcome was the accessibility of information and secondary outcomes understanding, satisfaction, and preference for one of the two formats. Between February 26 and August 29, 2021, we received 478 responses from stakeholders, of whom 244 (122 per group) were eligible and provided analysable results. Participants rated the eTB RecMap as more accessible, on average, when compared to the conventional website (on a seven-point scale, the mean difference {MD} was 0.9; 95% confidence interval {CI}: 0.6, 1.2; p < 0.001) and were more likely to correctly answer understanding questions. This is the first trial comparing digitized dissemination formats of health guideline recommendations. Stakeholders rated the WHO eTB RecMap as more accessible than the conventional WHO website for the tested recommendations. They also understood presented information better. The findings support better usability of TB information through the eTB RecMap and contribute to the effort to end the TB epidemic. Trial registration: This trial was registered with ClinicalTrials.gov (NCT04745897) on February 9, 2021.

Guidelines rarely used GRADE and applied methods inconsistently: A methodological study of Australian guidelines

OBJECTIVES

The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is accepted methodology to assess the certainty of the evidence included in systematic reviews and clinical practice guidelines. The GRADE approach is endorsed globally, in Australia, the National Health and Medical Research Council advocated for the use of the GRADE approach in 2011. The purpose of this methodological review was to assess how GRADE has been adopted for Australian practice guidelines.

STUDY DESIGN AND SETTING

This methodological review searched of the National Health and Medical Research Council Clinical Practice Guidelines Portal from 2011 to 2018, in an effort to retrieve all practice guidelines available via this medium.

RESULTS

240 guidelines were retrieved authored by 51 different organizations. 15 guidelines followed GRADE methodology. Application of GRADE methods varied between guidelines, some misreported and altered aspects of the GRADE process. Guidelines that closely adhered to the guidance from the GRADE Working Group scored higher in domain 3 (rigor of development) of the Appraisal of Guidelines for Research and Evaluation II tool, indicating a positive linear relationship between GRADE adherence and rigor of development scores.

CONCLUSION

The results of our project suggest that the use of GRADE in Australian guidelines is increasing, however, strategies to increase uptake and reporting within the guideline community need to be explored.