Certainty of evidence and intervention's benefits and harms are key determinants of guidelines' recommendations

Improving methods of clinical practice guidelines: From guidelines to pathways to fast-and-frugal trees and decision analysis to develop individualised patient care

BACKGROUND

Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation-a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"-a lack of framework for explicitly integrating factors such as patient preferences and trade-offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances.

METHODS

We propose augmenting the current guideline development method by converting evidence-based pathways into fast-and-frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations.

RESULTS

We illustrate the process by developing recommendations for the management of heparin-induced thrombocytopenia (HIT). We converted evidence-based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT-predicted probability of HIT exceeded 3%.

CONCLUSIONS

The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk-tailored personalised recommendations and addressing current guideline development critiques.

Emphasis should be placed on identifying and reporting research priorities to increase research value: An empirical analysis

OBJECTIVES

To compared the presentation of research priorities in the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) clinical practice guidelines (CPGs) developed under the guidance of the GRADE working group or its two co-chair, and the Chinese CPGs.

METHODS

This was a methodological empirical analysis. We searched PubMed, Embase, and four Chinese databases (Wanfang, VIP Database for Chinese Technical Periodicals, China National Knowledge Infrastructure and Chinese Biomedical Literature Database) and retrieved nine Chinese guideline databases or Society websites as well as GRADE Pro websites. We included all eligible GRADE CPGs and a random sample of double number of Chinese CPGs, published 2018 to 2022. The reviewers independently screened and extracted the data, and we summarized and analyzed the reporting on the research priorities in the CPGs.

RESULTS

Of the 135 eligible CPGs (45 GRADE CPGs and 90 Chinese CPGs), 668, 138 research priorities were identified respectively. More than 70% of the research priorities in GRADE CPGs and Chinese CPGs had population and intervention (PI) structure. 99 (14.8%) of GRADE CPG research priorities had PIC structures, compared with only 4(2.9%) in Chinese. And 28.4% (190) GRADE CPG research priorities reflected comparisons between PICO elements, approximately double those in Chinese. The types of research priorities among GRADE CPGs and Chinese CPGs were mostly focused on the efficacy of interventions, and the type of comparative effectiveness in the GRADE research priorities was double those in Chinese.

CONCLUSIONS

There was still considerable room for improvement in the developing and reporting of research priorities in Chinese CPGs. Key PICO elements were inadequately presented, with more attention on intervention efficacy and insufficient consideration given to values, preferences, health equity, and feasibility. Identifying and reporting of research priorities deserves greater effort in the future.

Guideline panel social dynamics influence the development of clinical practice recommendations: a mixed-methods systematic review

OBJECTIVES

To synthesize empirical studies that investigate the cognitive and social processes involved in the deliberation process of guideline development meetings and determine the distribution of deliberated topics.

STUDY DESIGN AND SETTING

We conducted a mixed-method systematic review using a convergent segregated approach. We searched for empirical studies that investigate the intragroup dynamics of guideline development meetings indexed in bibliographic databases.

RESULTS

Of the 5,899 citations screened, 12 studies from six countries proved eligible. Chairs, cochairs, and methodologists contributed to at least one-third of the discussion time in guideline development meetings; patient partners contributed the least. In interdisciplinary groups, male gender and occupation as a physician were positively associated with the amount of contribution. Compared to groups that used the Grading of Recommendations Assessment, Development and Evaluation approach, for groups that did not, when faced with insufficient or low-quality evidence, relied more on their clinical experience. The presence of a cognitive "yes" bias was apparent in meetings: panelists tended to acquiesce with positive statements that required less cognitive effort than negative statements.

CONCLUSION

The social dynamics of the discussions were linked to each panelist's activity role, professional background, and gender, all of which influenced the level of contributions they made in guideline development meetings.

Comprehensive consideration of multiple determinants from evidence to recommendations in guidelines for most traditional Chinese medicine was suboptimal: a systematic review

BACKGROUND

The overall comprehensive consideration of the factors influencing the recommendations in the traditional Chinese medicine (TCM) guidelines remains poorly studied. This study systematically evaluate the factors influencing recommendations formation in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) clinical practice guidelines (CPGs) and TCM CPGs.

METHODS

This was a methodological review in which we searched six databases and multiple related websites. The GRADE CPGs were identified as the guidelines developed by the GRADE Working Group or the two Co-Chairs. For the TCM CPGs, we randomly selected guidelines that were published by the TCM or integrative medicine academic societies from China mainland (published by the TCM or integrative medicine academic societies of China mainland). Two reviewers independently screened and extracted data. We included CPGs published in 2018-2022. We extracted information on the influencing factors of evidence to recommendation and conducted the analyses using descriptive statistics and calculated the proportion of relevant items by IBM SPSS Statistics and Microsoft Excel to compare the differences between the GRADE CPGs and the TCM CPGs.

RESULTS

Forty-five GRADE CPGs (including 912 recommendations) and 88 TCM CPGs (including 2452 recommendations) were included. TCM recommendations mainly considered the four key determinants of desirable anticipated effects, undesirable anticipated effects, balance between desirable and undesirable effects, certainty of evidence, with less than 20% of other dimensions. And TCM CPGs presented more strong recommendations (for or against) and inappropriate discordant recommendations than GRADE CPGs. GRADE CPGs were more comprehensive considered about the factors affecting the recommendations, and considered more than 70% of all factors in the evidence to recommendation.

CONCLUSIONS

The TCM CPGs lack a comprehensive consideration of multiple influencing determinants from evidence to recommendations. In the future, the correct application of the GRADE approaches should be emphasized.

[Are the assessments of individual out-of-pocket health services of the IGeL-Monitor in line with clinical guidelines?]

OBJECTIVES

The IGeL-Monitor of the Federal Medical Advisory Service in Germany evaluates the benefits and harms of individual out-of-pocket health services (in German: Individuelle Gesundheitsleistungen / IGeL). The aim of the analysis was to systematically compare IGeL-assessements with the recommendations from evidence-based guidelines.

METHOD

To identify guidelines, we conducted searches in guidelines databases (AWMF, Guidelines International Network and Trip database) and the websites of guideline organisations (February/March 2022). We included guidelines that were not older than 5 years. The methodological quality of the guidelines was assessed using the AGREE II instrument. We compared the recommendations with the IGeL-assessments in terms of content and grade of recommendation.

RESULTS

We identified 41 guidelines covering 24 IGeL-assessements. 19 (79%) assessments (nearly) were in agreement with the guideline recommendations. No comparison was possible for 5 IGeL-assessements, because, for example, the recommendations were more specific. Ten of the 13 IGeL that were rated (tendentially) negatively were also not recommended in the guidelines.

CONCLUSION

Overall, the IGeL-assessments were consistent with the recommendations of current guidelines. Accordingly, guideline groups seem to assess the evidence similarly to the IGeL-Monitor team. Insured persons should be informed honestly about the evidence, particularly for the (tendentially) negatively evaluated IGeL that are not recommended even in guidelines.

Impact of uncertainty intolerance on clinical reasoning: A scoping review of the 21st-century literature

RATIONALE, AIMS AND OBJECTIVES: Clinical reasoning is currently extensively studied to find out how to make proper diagnoses. Literature indicates that intolerance of uncertainty (IU) may have a strong negative impact on clinical reasoning. We summarize the various consequences of IU on clinical reasoning.

METHODS

A scoping review was conducted using relevant keywords to scientific databases (i.e., Google Scholar, Medline, PsycINFO and PBSC) from September to November 2021. Complementary research included relevant articles and articles retrieved through Google Scholar's alert system. We included articles about healthcare professionals as defined by the French Public Health Code (As defined here: https://www.vie-publique.fr/fiches/37855-categories-de-professionnels-de-sante-code-se-la-sante-publique), and articles reporting on the impact of IU or uncertainty management on clinical reasoning.

RESULTS

We retrieved 1853 articles, of which 8 were kept for final analysis considering our inclusion criteria. Two behaviour categories were affected by uncertainty intolerance: investigative and prescriptive behaviours. Regarding the investigation process, mismanagement of uncertainty appeared to lead to reasoning bias, potentially resulting in diagnostic errors. IU was associated with withholding information, more referrals to peers and less use of new medical interventions. Regarding prescription behaviours, IU among health professionals could lead to overprescribing unnecessary or dangerous tests. IU was also associated with more antibiotic prescriptions for conditions where antibiotics are to be used carefully.

CONCLUSION

Few studies have yet addressed the impact of IU on clinical reasoning. IU's influence is primarily observed on investigative and prescribing behaviours. More studies are needed to fully understand the impact of IU on clinical reasoning itself, and not only on practical consequences.

Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines

BACKGROUND

Clinical guidelines should be based on a thorough evaluation of the evidence and generally include a rating of the quality of evidence and assign a strength to recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against making strong recommendations when the certainty of the evidence is low or very low, but has identified five paradigmatic situations (e.g. life-threatening situations) where this may be justified.

AIMS AND OBJECTIVES

We aimed to characterize the strength of recommendations and certainty of the evidence in Irish National Clinical Guidelines using the GRADE approach.

METHODS

All National Clinical Guidelines from the National Clinical Effectiveness Committee (NCEC) website using the GRADE approach (fully or partially) were included. All recommendations and their corresponding certainty of the evidence, strength of recommendations and justifications were extracted. Authors classified instances of strong recommendations with low certainty evidence (referred to as discordant recommendations) into one of the five paradigmatic situations. Descriptive statistics were calculated.

RESULTS

From the 29 NCEC Clinical Guidelines available at the time of analysis, we identified 8 guidelines using GRADE with a total of 240 recommendations; 38 recommendations did not use the GRADE approach and were excluded. Half of the included guidelines focused on emergency situations. In the final dataset of 202 recommendations, 151 (74.7%) were classified as strong and 51 (25.3%) as conditional. Of the 151 strong recommendations, 55 (36.4%) were supported by high or moderate certainty evidence and 96 (63.6%) by low or very low certainty evidence and were considered discordant. Of these 96 discordant recommendations, 55 (73.7%) were consistent with one of the five paradigmatic situations. However, none were specifically described as such within the guidelines.

CONCLUSIONS

The proportion of discordant recommendations identified in this analysis was higher than some previous international studies (range of all strong recommendations being discordant 30-50%), but similar to other guidelines focused on emergency situations. The majority of discordant recommendations could be mapped to one of the five situations, but no National Clinical Guideline explicitly referenced this. Guideline developers require further guidance to enable greater transparency in the reporting of the reasons for discordant recommendations.

Evidence and Decision-Making

Today, every country struggles to provide adequate health care to its citizens. Globally, an average of $8.3 trillion or 10% of gross domestic product (GDP) is annually spent on health services. In 2019, the USA spent nearly 18% ($3.2 trillion) of its GDP on health care, projected to reach $6.2 trillion by 2028.

High quality (certainty) evidence changes less often than low-quality evidence, but the magnitude of effect size does not systematically differ between studies with low versus high-quality evidence

RATIONALE, AIMS, AND OBJECTIVES

It is generally believed that evidence from low quality of evidence generate inaccurate estimates about treatment effects more often than evidence from high (certainty) quality evidence (CoE). As a result, we would expect that (a) estimates of effects of health interventions initially based on high CoE change less frequently than the effects estimated by lower CoE (b) the estimates of magnitude of effect size differ between high and low CoE. Empirical assessment of these foundational principles of evidence-based medicine has been lacking.

METHODS

We reviewed the Cochrane Database of Systematic Reviews from January 2016 through May 2021 for pairs of original and updated reviews for change in CoE assessments based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method. We assessed the difference in effect sizes between the original versus updated reviews as a function of change in CoE, which we report as a ratio of odds ratio (ROR). We compared ROR generated in the studies in which CoE changed from very low/low (VL/L) to moderate/high (M/H) versus M/H to VL/L. Heterogeneity and inconsistency were assessed using the tau and I² statistic. We also assessed the change in precision of effect estimates (by calculating the ratio of standard errors) (seR), and the absolute deviation in estimates of treatment effects (aROR).

RESULTS

Four hundred and nineteen pairs of reviews were included of which 414 (207 × 2) informed the CoE appraisal and 384 (192 × 2) the assessment of effect size. We found that CoE originally appraised as VL/L had 2.1 [95% confidence interval (CI): 1.19-4.12; p = 0.0091] times higher odds to be changed in the future studies than M/H CoE. However, the effect size was not different (p = 1) when CoE changed from VL/L → M/H [ROR = 1.02 (95% CI: 0.74-1.39)] compared with M/H → VL/L (ROR = 1.02 [95% CI: 0.44-2.37]). Similar overlap in aROR between the VL/L → M/H versus M/H → VL/L subgroups was observed [median (IQR): 1.12 (1.07-1.57) vs. 1.21 (1.12-2.43)]. We observed large inconsistency across ROR estimates (I²  = 99%). There was larger imprecision in treatment effects when CoE changed from VL/L → M/H (seR = 1.46) than when it changed from M/H → VL/L (seR = 0.72).

CONCLUSIONS

We found that low-quality evidence changes more often than high CoE. However, the effect size did not systematically differ between the studies with low versus high CoE. The finding that the effect size did not differ between low and high CoE indicate urgent need to refine current EBM critical appraisal methods.

Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis

OBJECTIVE

To investigate whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelines.

DESIGN

Empirical analysis.

DATA SOURCE

Guidelines developed by the American College of Cardiology and the American Heart Association (ACC/AHA) and the American Society of Clinical Oncology (ASCO) up to 27 March 2021.

STUDY SELECTION

Recommendations were clearly categorised as consensus or evidence based, were separated from the remainder of the text, and included both the quality of evidence and the strength of the recommendations.

DATA EXTRACTION

Paired authors independently extracted the recommendation characteristics, including type of recommendation (consensus or evidence based), grading system used for developing recommendations, strength of the recommendation, and quality of evidence. The study team also calculated the number of discordant recommendations (strong recommendations with low quality evidence) and inappropriate discordant recommendations (those that did not meet grading of recommendations assessment, development, and evaluation criteria of appropriateness).

RESULTS

The study included 12 ACC/AHA guidelines that generated 1434 recommendations and 69 ASCO guidelines that generated 1094 recommendations. Of the 504 ACC/AHA recommendations based on low quality evidence, 200 (40%) proved to be consensus based versus 304 (60%) evidence based; of the 404 ASCO recommendations based on low quality evidence, 292 (72%) were consensus based versus 112 (28%) that were evidence based. In both ACC/AHA and ASCO guidelines, the consensus approach yielded more discordant recommendations (ACC/AHA: odds ratio 2.1, 95% confidence interval 1.5 to 3.1; ASCO: 2.9, 1.1 to 7.8) and inappropriate discordant recommendations (ACC/AHA: 2.6, 1.7 to 3.7; ASCO: 5.1, 1.6 to 16.0) than the evidence based approach.

CONCLUSION

Consensus based guidelines produce more recommendations violating the evidence based medicine principles than evidence based guidelines. Ensuring appropriate alignment of quality of evidence with the strength of recommendations is key to the development of "trustworthy" guidelines.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials

Background

The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public.

Methods

In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured.

Findings

Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was ‘low’, rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was ‘high’, and in one of the two studies, those shown ‘low’ quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was ‘high’ showed no statistically significant differences on these measures compared to those given no information about evidence quality.

Conclusions

Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

A few panel members dominated guideline development meeting discussions: Social network analysis

OBJECTIVES

To identify patterns of interactions that may influence guideline panels' decision-making.

STUDY DESIGN AND SETTING

Social network analysis (SNA) to describe the conversation network in a guideline development meeting in United States.

RESULTS

We analyzed one two-day guideline panel meeting that included 20 members who developed a guideline using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. The conversation pattern of the guideline panel indicated a well-connected network (density=0.59, clustering coefficient=0.82). GRADE topics on quality of evidence and benefits versus harms accounted for 46%; non-GRADE factors accounted for 30% of discussion. The chair, co-chair and methodologist initiated 53% and received 60% of all communications in the meeting; 42% of their communications occurred among themselves. SNA metrics (eigenvector, betweenness and closeness) indicated that these individuals also exerted highest influence on discussion, controlled information flow and were at the center of all communications. Members were more likely to continue previous discussion with the same individuals after both morning breaks (r=0.54, P<0.005; r=0.17, P=0.04), and after the last break on day 2 (r=0.44, P=0.015).

CONCLUSION

Non-GRADE factors such as breaks, and the members' roles, affect guideline development more than previously recognized. Collectively, the chair, co-chair and methodologist dominated the discussion.

Perioperative Antiplatelet and Anticoagulant Management with Endodontic Microsurgical Techniques

INTRODUCTION

The purpose of this study was to review evidence-based recommendations for the safe perioperative management of patients undergoing endodontic microsurgery who are currently taking antiplatelet or anticoagulant medications. Using the PICO (Population, Intervention, Comparison, Outcome) format, the following scientific question was asked: In patients taking anticoagulant or antiplatelet agents, what is the available evidence in the management of endodontic microsurgery?

METHODS

MEDLINE, Scopus, Cochrane Library, and ClinicalTrials.gov databases were searched to identify current recommendations regarding the management of antiplatelet and anticoagulant medications in the context of outpatient dental surgical procedures. Additionally, the authors hand searched the bibliographies of all relevant articles, the gray literature, and textbooks. Because of the lack of clinical studies and evidence on this subject, articles and guidelines from other organizations and association position statements were included.

RESULTS

Because any minor surgery can become a major surgery, the treating doctor needs to best assess the risk of bleeding, especially if the surgery is anticipated to take longer than 45 minutes. Every patient should be stratified on a case-by-case basis. Consultation with the patient's physician is highly recommended.

CONCLUSIONS

In order to maximize the effects of these medications (to prevent thrombosis) while minimizing the potential risks (procedural hemorrhage), clinicians should be aware of the best available evidence when considering continuation or discontinuation of antiplatelet and anticoagulant agents perioperatively for endodontic microsurgery. Ideally, a joint effort from an expert panel for microsurgery would be warranted.