Feasibility of national living guideline methods: the Australian Stroke Guidelines

The Utilization of Systematic Reviews and Meta-Analyses in Stroke Guidelines

BACKGROUND

Stroke guideline statements are important references for clinicians due to the rapidly evolving nature of treatments. Guideline statements should be informed by up-to-date systematic reviews (SRs) and meta-analyses (MAs) because they provide the highest level of evidence. To investigate the utilization of SRs/MAs in stroke management guidelines, we conducted a literature review of guidelines and extracted relevant information regarding SRs/MAs.

METHODS

A literature review was conducted in PubMed with supplementation using the Trip medical database with the term "stroke" as the target population, followed by using the filter "guidelines". We extracted the number of included SRs/MAs, the years of publication, the country of origin, and other characteristics of interest. Descriptive statistics were generated using the R software version 4.2.1.

RESULTS

We included 27 guideline statements. The median number of overall SRs or MAs within the guidelines was 4.0 (interquartile range [IQR] = 2-9). For MAs only, the median number included in the guidelines was 3.0 (IQR = 2.0-5.5). Canadian guidelines had the oldest citations, with a median gap of 12.0 (IQR = 5.2-18.0) years for the oldest citation, followed by European (median = 12; IQR = 9.5-13.5) and US (median = 10.0; IQR = 5.2-16) guidelines.

CONCLUSIONS

Stroke guideline writing groups and issuing bodies should devote greater effort to the inclusion of up-to-date SRs/MAs in their guideline statements so that clinicians can reference recent data with the highest level of evidence.

New horizons in clinical practice guidelines for use with older people

Globally, more people are living into advanced old age, with age-associated frailty, disability and multimorbidity. Achieving equity for all ages necessitates adapting healthcare systems. Clinical practice guidelines (CPGs) have an important place in adapting evidence-based medicine and clinical care to reflect these changing needs. CPGs can facilitate better and more systematic care for older people. But they can also present a challenge to patient-centred care and shared decision-making when clinical and/or socioeconomic heterogeneity or personal priorities are not reflected in recommendations or in their application. Indeed, evidence is often lacking to enable this variability to be reflected in guidance. Evidence is more likely to be lacking about some sections of the population. Many older adults are at the intersection of many factors associated with exclusion from traditional clinical evidence sources with higher incidence of multimorbidity and disability compounded by poorer healthcare access and ultimately worse outcomes. We describe these challenges and illustrate how they can adversely affect CPG scope, the evidence available and its summation, the content of CPG recommendations and their patient-centred implementation. In all of this, we take older adults as our focus, but much of what we say will be applicable to other marginalised groups. Then, using the established process of formulating a CPG as a framework, we consider how these challenges can be mitigated, with particular attention to applicability and implementation. We consider why CPG recommendations on the same clinical areas may be inconsistent and describe approaches to ensuring that CPGs remain up to date.

Is it possible to make 'living' guidelines? An evaluation of the Australian Living Stroke Guidelines

BACKGROUND

Keeping best practice guidelines up-to-date with rapidly emerging research evidence is challenging. 'Living guidelines' approaches enable continual incorporation of new research, assisting healthcare professionals to apply the latest evidence to their clinical practice. However, information about how living guidelines are developed, maintained and applied is limited. The Stroke Foundation in Australia was one of the first organisations to apply living guideline development methods for their Living Stroke Guidelines (LSGs), presenting a unique opportunity to evaluate the process and impact of this novel approach.

METHODS

A mixed-methods study was conducted to understand the experience of LSGs developers and end-users. We used thematic analysis of one-on-one semi-structured interview and online survey data to determine the feasibility, acceptability, and facilitators and barriers of the LSGs. Website analytics data were also reviewed to understand usage.

RESULTS

Overall, the living guidelines approach was both feasible and acceptable to developers and users. Facilitators to use included collaboration with multidisciplinary clinicians and stroke survivors or carers. Increased workload for developers, workload unpredictability, and limited information sharing, and interoperability of technological platforms were identified as barriers. Users indicated increased trust in the LSGs (69%), likelihood of following the LSGs (66%), and frequency of access (58%), compared with previous static versions. Web analytics data showed individual access by 16,517 users in 2016 rising to 53,154 users in 2020, a threefold increase. There was also a fourfold increase in unique LSG pageviews from 2016 to 2020.

CONCLUSIONS

This study, the first evaluation of living guidelines, demonstrates that this approach to stroke guideline development is feasible and acceptable, that these approaches may add value to developers and users, and may increase guideline use. Future evaluations should be embedded along with guideline implementation to capture data prospectively.

Exploring the use and impact of the Australian living guidelines for the clinical care of people with COVID-19: where to from here?

OBJECTIVES

The Australian National COVID-19 Clinical Evidence Taskforce has been developing, maintaining, and disseminating living guidelines and decision support tools (clinical flowcharts) for the care of people with suspected or confirmed COVID-19 since 2020. Living guidelines, a form of living evidence, are a relatively new approach; hence, more work is required to determine how to optimize their use to inform practice, policy, and decision-making and to explore implementation, uptake, and impact implications. An update of an earlier impact evaluation was conducted to understand sustained awareness and use of the guidelines; the factors that facilitate the widespread adoption of the guidelines and to explore the perceived strengths and opportunities for improvement of the guidelines.

STUDY DESIGN AND SETTING

A mixed-methods impact evaluation was conducted. Surveys collected both quantitative and qualitative data and were supplemented with qualitative interviews. Participants included Australian healthcare practitioners providing care to individuals with suspected or confirmed COVID-19 and people involved in policy-making. Data were collected on awareness, use, impact, strengths, and opportunities for improvement of the guidelines and flow charts.

RESULTS

A total of 148 participants completed the survey and 21 people were interviewed between January and March 2022. Awareness of the work of the Taskforce was high and more than 75% of participants reported that the guidelines were used within their workplace. Participants described the Taskforce website and guidelines as trustworthy, valuable, and reliable sources of up-to-date evidence-based information. The evaluation highlighted the varied ways the guidelines were being used across a range of settings and the diverse impacts they have from those at a clinical level to impacts at a policy level. Barriers to and enablers of impact and uptake of the guideline were explored.

CONCLUSION

This evaluation highlights the value of living guidelines during a pandemic when the evidence base is rapidly changing and expanding. It presents useful understanding of the ways clinicians and others use living evidence to inform their clinical practice and decision-making and the diverse impacts the guidelines are having around Australia.

Barriers to and facilitators of living guidelines use in low-income and middle-income countries: a scoping review

INTRODUCTION

Living guidelines provide reliable, ongoing evidence surveillance and regularly updated recommendations for healthcare decision-making. As a relatively new concept, most of the initial application of living approaches has been undertaken in high-income countries. However, in this scoping review, we looked at what is currently known about how living guidelines were developed, used and applied in low-income and middle-income countries.

METHODS

Searches for published literature were conducted in Medline, Global Health, Cochrane Library and Embase. Grey literature was identified using Google Scholar and the WHO website. In addition, the reference lists of included studies were checked for missing studies. Studies were included if they described or reflected on the development, application or utility of living guideline approaches for low-income and middle-income countries.

RESULTS

After a full-text review, 21 studies were included in the review, reporting on the development and application of living recommendations in low-income and middle-income countries. Most studies reported living guideline activities conducted by the WHO (15, 71.4%), followed by China (4, 19%), Chile (1, 4.8%) and Lebanon (1, 4.8%). All studies based on WHO reports relate to living COVID-19 management guidelines.

CONCLUSIONS

Most of the studies in this review were WHO-reported studies focusing solely on COVID-19 disease treatment living guidelines. However, there was no clear explanation of how living guidelines were used nor information on the prospects for and obstacles to the implementation of living guidelines in low-income and middle-income countries.

Learning health systems and evidence ecosystems: a perspective on the future of evidence-based medicine and evidence-based guideline development

Despite forming the cornerstone of modern clinical practice for decades, implementation of evidence-based medicine at scale remains a crucial challenge for health systems. As a result, there has been a growing need for conceptual models to better contextualise and pragmatize the use of evidence-based medicine, particularly in tandem with patient-centred care. In this commentary, we highlight the emergence of the learning health system as one such model and analyse its potential role in pragmatizing both evidence-based medicine and patient-centred care. We apply the learning health system lens to contextualise the key activity of evidence-based guideline development and implementation, and highlight how current inefficiencies and bottlenecks in the evidence synthesis phase of evidence-based guideline development threaten downstream adherence. Lastly, we introduce the evidence ecosystem as a complementary model to learning health systems, and propose how innovative developments from the evidence ecosystem may be integrated with learning health systems to better enable health impact at speed and scale.

Living evidence and adaptive policy: perfect partners?

BACKGROUND

While there has been widespread global acceptance of the importance of evidence-informed policy, many opportunities to inform health policy with research are missed, often because of a mismatch between when and where reliable evidence is needed, and when and where it is available. 'Living evidence' is an approach where systematic evidence syntheses (e.g. living reviews, living guidelines, living policy briefs, etc.) are continually updated to incorporate new relevant evidence as it becomes available. Living evidence approaches have the potential to overcome a major barrier to evidence-informed policy, making up-to-date systematic summaries of policy-relevant research available at any time that policy-makers need them. These approaches are likely to be particularly beneficial given increasing calls for policy that is responsive, and rapidly adaptive to changes in the policy context. We describe the opportunities presented by living evidence for evidence-informed policy-making and highlight areas for further exploration.

DISCUSSION

There are several elements of living approaches to evidence synthesis that might support increased and improved use of evidence to inform policy. Reviews are explicitly prioritised to be 'living' by partnerships between policy-makers and researchers based on relevance to decision-making, as well as uncertainty of existing evidence, and likelihood that new evidence will arise. The ongoing nature of the work means evidence synthesis teams can be dynamic and engage with policy-makers in a variety of ways over time; and synthesis topics, questions and methods can be adapted as policy interests or contextual factors shift. Policy-makers can sign-up to be notified when relevant new evidence is found, and can be confident that living syntheses are up-to-date and contain all research whenever they access them. The always up-to-date nature of living evidence syntheses means producers can rapidly demonstrate availability of relevant, reliable evidence when it is needed, addressing a frequently cited barrier to evidence-informed policymaking.

CONCLUSIONS

While there are challenges to be overcome, living evidence provides opportunities to enable policy-makers to access up-to-date evidence whenever they need it and also enable researchers to respond to the issues of the day with up-to-date research; and update policy-makers on changes in the evidence base as they arise. It also provides an opportunity to build flexible partnerships between researchers and policy-makers to ensure that evidence syntheses reflect the changing needs of policy-makers.

Evidence surveillance for a living clinical guideline: Case study of the Australian stroke guidelines

BACKGROUND

Continual evidence surveillance is an integral feature of living guidelines. The Australian Stroke Guidelines include recommendations on 100 clinical topics and have been 'living' since 2018.

OBJECTIVES

To describe the approach for establishing and evaluating an evidence surveillance system for the living Australian Stroke Guidelines.

METHODS

We developed a pragmatic surveillance system based on an analysis of the searches for the 2017 Stroke Guidelines and evaluated its reliability by assessing the potential impact on guideline recommendations. Search retrieval and screening workload are monitored monthly, together with the frequency of changes to the guideline recommendations.

RESULTS

Evidence surveillance was guided by practical considerations of efficiency and sustainability. A single PubMed search covering all guideline topics, limited to systematic reviews and randomised trials, is run monthly. The search retrieves about 400 records a month of which a sixth are triaged to the guideline panels for further consideration. Evaluations with Epistemonikos and the Cochrane Stroke Trials Register demonstrated the robustness of adopting this more restrictive approach. Collaborating with the guideline team in designing, implementing and evaluating the surveillance is essential for optimising the approach.

CONCLUSION

Monthly evidence surveillance for a large living guideline is feasible and sustainable when applying a pragmatic approach.

New horizons in evidence synthesis for older adults

Evidence synthesis, embedded within a systematic review of the literature, is a well-established approach for collating and combining all the relevant information on a particular research question. A robust synthesis can establish the evidence base, which underpins best practice guidance. Such endeavours are frequently used by policymakers and practitioners to inform their decision making. Traditionally, an evidence synthesis of interventions consisted of a meta-analysis of quantitative data comparing two treatment alternatives addressing a specific and focussed clinical question. However, as the methods in the field have evolved, especially in response to the increasingly complex healthcare questions, more advanced evidence synthesis techniques have been developed. These can deal with extended data structures considering more than two treatment alternatives (network meta-analysis) and complex multicomponent interventions. The array of questions capable of being answered has also increased with specific approaches being developed for different evidence types including diagnostic, prognostic and qualitative data. Furthermore, driven by a desire for increasingly up-to-date evidence summaries, living systematic reviews have emerged. All of these methods can potentially have a role in informing older adult healthcare decisions. The aim of this review is to increase awareness and uptake of the increasingly comprehensive array of newer synthesis methods available and highlight their utility for answering clinically relevant questions in the context of older adult research, giving examples of where such techniques have already been effectively applied within the field. Their strengths and limitations are discussed, and we suggest user-friendly software options to implement the methods described.

What could health technology assessment learn from living clinical practice guidelines?

A "living" approach to clinical practice guidelines is when the identification, appraisal and synthesis of evidence is maintained and repeated at an agreed frequency, with a clear process for when and how new evidence is to be incorporated. The value of a living approach to guidelines was emphasised during the COVID-19 pandemic when health professionals and policymakers needed to make decisions regarding patient care in the context of a nascent but rapidly evolving evidence base. In this perspective, we draw on our recent experience developing Australian and international living guidelines and reflect on the feasibility of applying living guideline methods and processes to a lifecycle approach to health technology assessment (HTA). We believe the opportunities and challenges of adopting a living approach in HTA fall into five key themes: identification, appraisal and synthesis of evidence; optimising the frequency of updates; embedding ongoing multi-stakeholder engagement; linking the emergence of new evidence to reimbursement; and system capacity to support a living approach. We acknowledge that the suitability of specific living approaches to HTA will be heavily influenced by the type of health technology, its intended use in the health system, local reimbursement pathways, and other policy settings. But we believe that the methods and processes applied successfully to guideline development to manage evidentiary uncertainty could be applied in the context of HTA and reimbursement decision-making to help manage similar sources of uncertainty.

Practices, beliefs, and attitudes of clinicians in prescribing direct oral anticoagulants for obese adults with atrial fibrillation: a qualitative study

BACKGROUND

Atrial fibrillation (AF) and obesity affect over 60 and 650 million people, respectively.

AIM

This study aimed to explore clinician practices, beliefs, and attitudes towards the use of direct oral anticoagulants (DOACs) in obese adults (BMI ≥ 30 kg/m2) with AF.

METHOD

Semi-structured interviews via video conference were conducted with multidisciplinary clinicians from across Australia, with expertise in DOAC use in adults with AF. Clinicians were invited to participate using purposive and snowball sampling techniques. Data were analysed in NVIVO using thematic analysis.

RESULTS

Fifteen clinicians including cardiologists (n = 5), hospital and academic pharmacists (n = 5), general practitioners (n = 2), a haematologist, a neurologist and a clinical pharmacologist participated. Interviews were on average 31 ± 9 min. Key themes identified were: Health system factors in decision-making Disparities between rural and metropolitan geographic areas, availability of health services, and time limitations for in-patient decision-making, were described; Condition-related factors in decision-making Clinicians questioned the significance of obesity as part of decision-making due to the practical limitations of dose modification, and the rarity of the extremely obese cohort; Decision-making in the context of uncertainty Clinicians reported limited availability, reliability and awareness of primary evidence including limited guidance from clinical guidelines for DOAC use in obesity.

CONCLUSION

This study highlights the complexity of decision-making for clinicians, due to the limited availability, reliability and awareness of evidence, the intrinsic complexity of the obese cohort and limited guidance from clinical guidelines. This highlights the urgent need for contemporary research to improve the quality of evidence to guide informed shared decision-making.

Two Years of Experience and Methodology of Korean COVID-19 Living Clinical Practice Guideline Development

BACKGROUND

In Korea, during the early phase of the coronavirus disease 2019 (COVID-19) pandemic, we responded to the uncertainty of treatments under various conditions, consistently playing catch up with the speed of evidence updates. Therefore, there was high demand for national-level evidence-based clinical practice guidelines for clinicians in a timely manner. We developed evidence-based and updated living recommendations for clinicians through a transparent development process and multidisciplinary expert collaboration.

METHODS

The National Evidence-based Healthcare Collaborating Agency (NECA) and the Korean Academy of Medical Sciences (KAMS) collaborated to develop trustworthy Korean living guidelines. The NECA-supported methodological sections and 8 professional medical societies of the KAMS worked with clinical experts, and 31 clinicians were involved annually. We developed a total of 35 clinical questions, including medications, respiratory/critical care, pediatric care, emergency care, diagnostic tests, and radiological examinations.

RESULTS

An evidence-based search for treatments began in March 2021 and monthly updates were performed. It was expanded to other areas, and the search interval was organized by a steering committee owing to priority changes. Evidence synthesis and recommendation review was performed by researchers, and living recommendations were updated within 3-4 months.

CONCLUSION

We provided timely recommendations on living schemes and disseminated them to the public, policymakers and various stakeholders using webpages and social media. Although the output was successful, there were some limitations. The rigor of development issues, urgent timelines for public dissemination, education for new developers, and spread of several new COVID-19 variants have worked as barriers. Therefore, we must prepare systematic processes and funding for future pandemics.

Weekly updating of guideline recommendations was feasible: the Australian National COVID-19 clinical evidence Taskforce

OBJECTIVES

To investigate how quickly evidence was incorporated into the Australian living guidelines for COVID-19 during the first 12 months of the pandemic.

STUDY DESIGN AND SETTING

For each study concerning drug therapies included in the guideline from April 3, 2020 to April 1, 2021, we extracted the publication date of the study, and the guideline version the study was included in. We analyzed two subgroups of studies as follows: those published in high impact factor journals and those with 100 or more participants.

RESULTS

In the first year, we published 37 major versions of the guidelines, incorporating 129 studies that investigated 48 drug therapies informing 115 recommendations. The median time from first publication of a study to incorporation in the guideline was 27 days (interquartile range [IQR], 16 to 44), ranging from 9 to 234 days. For the 53 studies in the highest impact factor journals, the median was 20 days (IQR 15 to 30), and for the 71 studies with 100 or more participants the median was 22 days (IQR 15 to 36).

CONCLUSION

Developing and sustaining living guidelines where evidence is rapidly incorporated is a resource- and time-intensive undertaking; however, this study demonstrates that it is feasible, even over a long period.

Paper 4: Search methods and approaches for living guidelines

OBJECTIVES

To describe the key features of a continual evidence surveillance process that can be implemented for living guidelines and to outline the considerations and trade-offs in adopting different approaches.

STUDY DESIGN AND SETTING

Members of the Australian Living Evidence Consortium (ALEC), National Institute of Health and Care Excellence (NICE), and the US GRADE Network (USGN) shared their practical experiences of and approaches to establishing surveillance systems for living guidelines. We identified several common components of evidence surveillance and listed the key features and considerations for each component drawn from case studies, highlighting differences with standard guidelines.

RESULTS

We developed guidance that covers the initial information needed to support decisions around suitability for living mode and the practical considerations in setting up continual search surveillance systems (search frequency, sources to search, use of automation, reporting the search, ongoing resources, and evaluation). The case studies draw on our experiences with developing guidelines for COVID-19, as well as for other conditions such as stroke and diabetes, and cover a range of practical approaches, including the use of automation.

CONCLUSION

This paper highlights different approaches to continual evidence surveillance that can be implemented in living guidelines.

Methods for living guidelines: early guidance based on practical experience

OBJECTIVES

To introduce methods for living guidelines based on practical experiences by the Australian Living Evidence Consortium (ALEC), the National Institute of Health and Care Excellence (NICE), and the Infectious Diseases Society of America (IDSA), with methodological support from the US Grading of Recommendations, Assessment, Development and Evaluations (GRADE) Network.

STUDY DESIGN AND SETTING

Members of ALEC, NICE, and the US GRADE Network, convened a working group to share experiences of the methods used to develop living guidelines and outline the key differences between traditional and living guidelines methods.

RESULTS

The guidance includes the following steps: 1) deciding if the guideline is a priority for a living approach, 2) preparing for living guideline development, 3) literature surveillance and frequency of searching, 4) assessment and synthesis of the evidence, 5) publication and dissemination, and 6) transitioning recommendations out of living mode.

CONCLUSION

This paper introduces methods for living guidelines and provides examples of the similarities and differences in approach across multiple organizations conducting living guidelines. It also introduces a series of papers exploring methods for living guidelines based on our practical experiences, including consumer involvement, selecting and prioritizing questions, search decisions, and methods decisions.

Methods for living guidelines: early guidance based on practical experience. Article 5: decisions on methods for evidence synthesis and recommendation development for living guidelines

OBJECTIVES

Producing living guidelines requires making important decisions about methods for evidence identification, appraisal, and integration to allow the living mode to function. Clarifying what these decisions are and the trade-offs between options is necessary. This article provides living guideline developers with a framework to enable them to choose the most suitable model for their living guideline topic, question, or context.

STUDY DESIGN AND SETTING

We developed this guidance through an iterative process informed by interviews, feedback, and a consensus process with an international group of living guideline developers.

RESULTS

Several key decisions need to be made both before commencing and throughout the continual process of living guideline development and maintenance. These include deciding what approach is taken to the systematic review process; decisions about methods to be applied for the evidence appraisal process, including the use of unpublished data; and selection of "triggers" to incorporate new studies into living guideline recommendations. In each case, there are multiple options and trade-offs.

CONCLUSION

We identify trade-offs and important decisions to be considered throughout the living guideline development process. The most appropriate, and most sustainable, mode of development and updating will be dependent on the choices made in each of these areas.

How frequently should "living" guidelines be updated? Insights from the Australian Living Stroke Guidelines

Background

“Living guidelines” are guidelines which are continually kept up to date as new evidence emerges. Living guideline methods are evolving. The aim of this study was to determine how frequently searches for new evidence should be undertaken for the Australian Living Stroke Guidelines.

Methods

Members of the Living Stroke Guidelines Development Group were invited to complete an online survey. Participants nominated one or more recommendation topics from the Living Stroke Guidelines with which they had been involved and answered questions about that topic, assessing whether it met criteria for living evidence synthesis, and how frequently searches for new evidence should be undertaken and why. For each topic we also determined how many studies had been assessed and included, and whether recommendations had been changed.

Results

Fifty-seven assessments were received from 33 respondents, covering half of the 88 guideline topic areas. Nearly all assessments (49, 86%) were that the continual updating process should be maintained. Only three assessments (5%) deemed that searches should be conducted monthly; 3-monthly (14, 25%), 6-monthly (13, 23%) and yearly (17, 30%) searches were far more frequently recommended. Rarely (9, 16%) were topics deemed to meet all three criteria for living review. The vast majority of assessments (45, 79%) deemed the topic a priority for decision-making. Nearly half indicated that there was uncertainty in the available evidence or that new evidence was likely to be available soon.

Since 2017, all but four of the assessed topic areas have had additional studies included in the evidence summary. For eight topics, there have been changes in recommendations, and revisions are underway for an additional six topics.

Clinical importance was the most common reason given for why continual evidence surveillance should be undertaken. Workload for reviewers was a concern, particularly for topics where there is a steady flow of publication of small trials.

Conclusions

Our study found that participants felt that the vast majority of topics assessed in the Living Stroke Guidelines should be continually updated. However, only a fifth of topic areas were assessed as conclusively meeting all three criteria for living review, and the definition of “continual” differed widely. This work has informed decisions about search frequency for the Living Stroke Guidelines and form the basis of further research on methods for frequent updating of guidelines.