Barriers to the uptake of evidence from systematic reviews and meta-analyses: a systematic review of decision makers' perceptions

Does the format of result presentation and type of conclusion in Cochrane plain language summaries matter? A randomised controlled trial

OBJECTIVES

This study aimed to investigate whether the format and type of conclusion in Cochrane plain language summaries (PLSs) influence readers' perception of treatment benefit and decision-making.

DESIGN

An online parallel group, three-arm randomised controlled trial was conducted.

SETTING

The study was conducted online.

PARTICIPANTS

The participants were physiotherapy students.

INTERVENTIONS

The participants read two Cochrane PLSs, one with a positive conclusion (strong evidence of benefit) and another with a negative conclusion (strong evidence of non-benefit). Each participant read the results of both reviews presented in one of three formats: (1) numerical, (2) textual or (3) numerical and textual.

MAIN OUTCOME MEASURES

The primary outcome measure was the participants' perception of treatment benefit.

RESULTS

All three groups of participants perceived the treatment to have positive effects when the Cochrane PLS had a positive conclusion, regardless of the format of presentation (mean perception of treatment benefit score: textual 7.7 (SD 2.3), numerical 7.9 (SD 1.8), numerical and textual 7.7 (SD 1.7), p=0.362). However, when the Cochrane PLS had a negative conclusion, all three groups of participants failed to perceive a negative effect (mean perception of treatment benefit score: textual 5.5 (SD 3.3), numerical 5.6 (SD 2.7), numerical and textual 5.9 (SD 2.8), p=0.019).

CONCLUSIONS

The format of Cochrane PLSs does not appear to significantly impact physiotherapy students' perception of treatment benefit, understanding of evidence, persuasiveness or confidence in their decision. However, participants' perception of treatment benefit does not align with the conclusion when the Cochrane PLS indicates strong evidence of non-benefit from the intervention.

TRIAL REGISTRATION NUMBER

CTRI/2022/10/046476.

Effect of mindfulness-based programmes on elite athlete mental health: a systematic review and meta-analysis

OBJECTIVE

To determine the effectiveness of mindfulness-based programmes (MBPs) on the mental health of elite athletes.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Eight online databases (Embase, PsycINFO, SPORTDiscus, MEDLINE, Scopus, Cochrane CENTRAL, ProQuest Dissertations & Theses and Google Scholar), plus forward and backward searching from included studies and previous systematic reviews.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Studies were included if they were randomised controlled trials (RCTs) that compared an MBP against a control, in current or former elite athletes.

RESULTS

Of 2386 articles identified, 12 RCTs were included in this systematic review and meta-analysis, comprising a total of 614 elite athletes (314 MBPs and 300 controls). Overall, MBPs improved mental health, with large significant pooled effect sizes for reducing symptoms of anxiety (hedges g=-0.87, number of studies (n)=6, p=0.017, I ²=90) and stress (g=-0.91, n=5, p=0.012, I ²=74) and increasing psychological well-being (g=0.96, n=5, p=0.039., I ²=89). Overall, the risk of bias and certainty of evidence was moderate, and all findings were subject to high estimated levels of heterogeneity.

CONCLUSION

MBPs improved several mental health outcomes. Given the moderate degree of evidence, high-quality, adequately powered trials are required in the future. These studies should emphasise intervention fidelity, teacher competence and scalability within elite sport.

PROSPERO REGISTRATION NUMBER

CRD42020176654.

User experiences of structured stakeholder engagement to consider transferability: The TRANSFER approach

BACKGROUND

Systematic reviews are increasingly used to inform decision-making in health, education, social care and environmental protection. However, decision makers still experience barriers to using reviews, including not knowing how findings might translate to their own contexts, and lack of collaboration with systematic review authors. The TRANSFER approach is a novel method that aims to support review authors to systematically and transparently collaborate with stakeholders to consider context and the transferability of review findings from the beginning of the review process. Such collaboration is intended to improve the usefulness and relevance of review findings for decision makers.

OBJECTIVES

We aim to explore the user experience of the TRANSFER approach conversation guide, and in doing so gain a better understanding of the role and perceived value of stakeholder engagement in systematic reviews for informed decision-making.

METHODS

We conducted four user tests of groups using the guide, organized around simulated meetings between review authors and stakeholders. Review authors led the meeting using the TRANSFER approach conversation guide. We audio-recorded and observed the meetings, collected feedback forms and conducted semi-structured interviews with review authors following the meeting. We analysed the data using framework analysis to examine the user experience of the TRANSFER approach conversation guide and of stakeholder engagement more generally.

RESULTS

Seventeen participants in four user groups participated in the user tests. Most participants were generally positive toward the structured approach using the conversation guide, and felt it would be useful in systematic review projects. We observed examples of misunderstanding of the terminology included in the guide, and received multiple suggestions for how to make the conversation guide more user friendly. We observed numerous challenges related to the hypothetical nature of a user test, including lack of familiarity with the review question/topic among participants and lack of preparation for the meeting.

CONCLUSIONS

Review authors and stakeholders are positive toward using a structured approach to guide collaboration within the context of a systematic review. The TRANSFER conversation guide helps participants to discuss the review question and context in a structured way. Such structured collaboration could help to improve the usefulness and relevance of systematic reviews for decision making by improving the review question, inclusion criteria and consideration of transferability of review findings. The conversation guide needs to be modified to improve user experience. Further research is needed to explore stakeholder collaboration and the use of the TRANSFER conversation guide in systematic review processes.

Abdominal aortic aneurysm screening in an academic family practice: Short-term impact of guideline changes

OBJECTIVE

To investigate abdominal aortic aneurysm (AAA) screening rates in the 6 months before and after the introduction of updated Canadian Task Force on Preventive Health Care (CTFPHC) guidelines to determine effects on practice patterns, as well as to determine whether certain patient characteristics impact AAA screening rates.

DESIGN

Retrospective chart review.

SETTING

Academic family health centre in London, Ont.

PARTICIPANTS

Male patients between the ages of 65 and 80.

MAIN OUTCOME MEASURES

Screening rates for AAA before and after the guideline update were compared using the normal approximation of the binomial distribution. Analysis of demographic characteristic effects on screening rates was completed with the Fisher exact test. Number of visits to the clinic with a primary care provider within the study period and imaging type were collected.

RESULTS

Of the 266 patients included in the study, 160 patients were eligible for screening at the start of the study period, 6 months before publication of the CTFPHC AAA guideline. Individuals eligible for screening visited the clinic an average (SD) of 2.44 (1.82) times in the 6 months before and 2.66 (1.99) times in the 6 months after. Overall, 69 individuals had AAA screening completed and 9 had a discussion of AAA screening without any imaging, for a total uptake rate of 88.5% for those who had screening recommended. The overall imaging rate was 48.9%. There was no statistically significant difference in screening rates between the time periods (P=.337) among those eligible for screening. For demographic characteristics for risk stratification, 7 individuals had a documented family history, of whom 5 had imaging of their abdominal aorta performed, plus 1 additional individual who had screening recommended but not completed. This was not statistically significant relative to the total population (P=.0598). Positive smoking status (active or ex-smoker) was more common, with 135 individuals having a relevant smoking history. Approximately half of these current and former smokers (68 individuals [50.4%]) had any sort of abdominal aortic imaging performed or recommended, which was not statistically significantly different compared with non-smokers (62 of 126 imaging performed or recommended, 49.2%; P=.9016).

CONCLUSION

Screening practices did not change appreciably with the introduction of the CTFPHC AAA screening guidelines. Further research is needed to improve AAA screening rates. It is worth exploring electronic medical record-based reminders, nursing staff involvement in screening, screening programs via public health, and point-of-care ultrasound screening in a primary care setting.

Dentists' attitudes and practices toward evidence-based dentistry: a systematic review

INTRODUCTION AND AIMS

Considering that attitudes toward evidence-based dentistry (EBD) may predict implementation behaviors, the objective of this systematic review was to synthesize and evaluate the existing evidence related to dentists' attitudes and practices toward EBD.

METHODS

We included primary studies that collected information from interviews, questionnaires, or conversation sessions with dentists. The following sources were searched: Cochrane Central Register of Controlled Trials, Embase, PubMed, Scopus, and Web of Science, in addition to gray literature. The included studies were appraised according to the assessment tools recommended by the Joanna Briggs Institute for qualitative and quantitative observational studies. Descriptive data were collected in standardized tables and descriptively synthesized.

RESULTS

The selection process resulted in 36 included studies. Dentists share positive opinions about EBD and predominantly report willingness to learn or adopt these practices. Despite high methodological risks and significant heterogeneity, the results collected in this review indicated that scientific journals, clinical practice guidelines, and trusted colleagues are generally perceived as influential and useful by dentists, who highly consulted these information sources.

CONCLUSION

Despite supportive reported attitudes toward EBD, very low certainty exists about actual EBD-related practices.

Bursting out of our bubble: using creative techniques to communicate within the systematic review process and beyond

BACKGROUND

Increasing pressure to publicise research findings and generate impact, alongside an expectation from funding bodies to go beyond publication within academic journals, has generated interest in alternative methods of science communication. Our aim is to describe our experience of using a variety of creative communication tools, reflect on their use in different situations, enhance learning and generate discussion within the systematic review community.

METHODS

Over the last 5 years, we have explored several creative communication tools within the systematic review process and beyond to extend dissemination beyond traditional academic mechanisms. Central to our approach is the co-production of a communication plan with potential evidence users which facilitates (i) the identification of key messages for different audiences, (ii) discussion of appropriate tools to communicate key messages and (iii) exploration of avenues to share them. We aim to involve evidence users in the production of a variety of outputs for each research project cognisant of the many ways in which individuals engage with information.

RESULTS

Our experience has allowed us to develop an understanding of the benefits and challenges of a wide range of creative communication tools. For example, board games can be a fun way of learning, may flatten power hierarchies between researchers and research users and enable sharing of large amounts of complex information in a thought provoking way, but they are time and resource intensive both to produce and to engage with. Conversely, social media shareable content can be quick and easy to produce and to engage with but limited in the depth and complexity of shareable information.

DISCUSSION

It is widely recognised that most stakeholders do not have time to invest in reading large, complex documents; creative communication tools can be a used to improve accessibility of key messages. Furthermore, our experience has highlighted a range of additional benefits of embedding these techniques within our project processes e.g. opening up two-way conversations with end-users of research to discuss the implications of findings.

International guidelines for self-report and proxy completion of paediatric health-related quality of life measures: a protocol for a systematic review

INTRODUCTION

Measures of health-related quality of life (HRQoL), accompanied by the values (or utilities) required to estimate quality-adjusted life-years, are crucial for determining health benefits within economic evaluation and health technology assessment. Several generic and condition-specific measures or instruments of HRQoL, accompanied by values, currently exist for application with child populations. However, there is a lack of a structured summary of guidelines and recommendations for applying these measures in practice. This protocol describes a systematic review of guidelines and recommendations for child and proxy completion of child-specific measures of HRQoL. The aims of the review are to (1) identify and summarise published guidelines and recommendations for existing child-specific measures of HRQoL, (2) determine whether the identified guidelines and recommendations differ by instrument and child characteristics, (3) identify current gaps in these guidelines and recommendations and (4) identify best practices for child self and proxy assessment in paediatric HRQoL measurement for economic evaluation and health technology assessment.

METHODS AND ANALYSIS

The review will identify, collate and synthesise published guidelines and recommendations for existing child-specific utility measures of HRQoL. Electronic databases to be searched include the Cochrane Library, Medline, Scopus, Web of Science, EconLit, PsycINFO, CINAHL, Embase and Informit. The search will be extended to websites of (1) international organisations for health technology assessment, (2) regulation, health economics and HRQoL outcomes research and (3) instrument developers. Three reviewers will independently screen titles and abstracts against the inclusion criteria. A narrative synthesis will describe the key features of the guidelines identified.

ETHICS AND DISSEMINATION

Ethical approval is not required as the proposed systematic review will not use primary data. A paper of the systematic review will be submitted to a peer-reviewed journal for publication.

PROSPERO REGISTRATION NUMBER

CRD42020207160.

Stepped care for musculoskeletal pain is ineffective: a model for utilisation of specialist physiotherapists in primary healthcare management

Specialist physiotherapy services can fill a gap in the management of pain disorders in primary care. Fatima has persistent musculoskeletal pain, not responding to first-line guideline-based management. Despite normal radiology, specialist medical review is requested. Although waiting considerable time for the appointment (partly caused by the backlog of patients like her), Mary develops secondary mental health issues. The usual stepped-care approach is failing her. Specialist physiotherapist review is requested, providing expert-level management suggestions to address complexity in her presentation. The specialist physiotherapist works collaboratively with her treating physiotherapist to achieve positive outcomes for Fatima. Specialist physiotherapy services are an under-recognised value-added step between the GP and medical specialist. Increased utilisation of the skills and competencies of specialist physiotherapists can positively contribute to the health of Australians who suffer the burden of complex/persistent musculoskeletal pain complaints.

Are weak or negative clinical recommendations associated with higher geographical variation in utilisation than strong or positive recommendations? Cross-sectional study of 24 healthcare services

OBJECTIVES

When research evidence is lacking, patient and provider preferences, expected to vary geographically, might have a stronger role in clinical decisions. We investigated whether the strength or the direction of recommendation is associated with the degree of geographic variation in utilisation.

DESIGN

In this cross-sectional study, we selected 24 services following a comprehensive approach. The strength and direction of recommendations were assessed in duplicate. Multilevel models were used to adjust for demographic and clinical characteristics and estimate unwarranted variation.

SETTING

Observational study of claims to mandatory health insurance in Switzerland in 2014.

PARTICIPANTS

Enrolees eligible for the 24 healthcare services.

PRIMARY OUTCOME MEASURES

The variances of regional random effects, also expressed as median odds ratios (MOR). Services grouped by strength and direction of recommendations were compared with Welch's t-test.

RESULTS

The sizes of the eligible populations ranged from 1992 to 409 960 patients. MOR ranged between 1.13 for aspirin in secondary prevention of myocardial infarction to 1.68 for minor surgical procedures performed in inpatient instead of outpatient settings. Services with weak recommendations had a negligibly higher variance and MOR (difference in means (95% CI) 0.03 (-0.06 to 0.11) and 0.05 (-0.11 to 0.21), respectively) compared with strong recommendations. Services with negative recommendations had a slightly higher variance and MOR (difference in means (95% CI) 0.07 (-0.03 to 0.18) and 0.14 (-0.06 to 0.34), respectively) compared with positive recommendations.

CONCLUSIONS

In this exploratory study, the geographical variation in the utilisation of services associated with strong vs weak and negative vs positive recommendations was not substantially different, although the difference was somewhat larger for negative vs positive recommendations. The relationships between the strength or direction of recommendations and the variation may be indirect or modified by other characteristics of services. As initiatives discouraging low-value care are gaining attention worldwide, these findings may inform future research in this area.

Evaluation of the completeness of intervention reporting in Cochrane surgical systematic reviews using the TIDieR-SR checklist: a cross-sectional study

INTRODUCTION

Complete reporting of systematic reviews of interventions is essential to the interpretation of research findings and the reproducibility of research results. The Template for Intervention Description and Replication (TIDieR) checklist-and the version specific to systematic reviews (TIDieR-SR)-was created to provide authors and researchers an evidence-based guide for reporting trial and systematic review interventions. In this study, we apply TIDieR-SR to Cochrane systematic reviews of surgical interventions.

METHODS

We searched the Cochrane Database for relevant systematic reviews. Two investigators applied inclusion/exclusion criteria to all titles/abstracts and full texts. These same investigators extracted all data in duplicate while masked to the other's data. The primary outcome was adherence to TIDieR-SR items.

RESULTS

Two hundred and thirty-eight systematic reviews were included. Overall, included SRs adhered to a median of 6 (IQR 5-7) out of eight TIDieR-SR items. The item with the lowest adherence was item 7 (share intervention materials, 1/238 (0.4%).

DISCUSSION

Our results are encouraging, but the generalisability of our findings is compromised by the inclusion of only Cochrane systematic reviews. Future reporting of intervention materials is likely to improve the application of effective surgical interventions in the clinical practice.

Evidence for implementation of interventions to promote mental health in the workplace: a systematic scoping review protocol

BACKGROUND

Mental health problems are common in the working population and represent a growing concern internationally, with potential impacts on workers, organisations, workplace health and compensation authorities, labour markets and social policies. Workplace interventions that create workplaces supportive of mental health, promote mental health awareness, destigmatise mental illness and support those with mental disorders are likely to improve health and economical outcomes for employees and organisations. Identifying factors associated with successful implementation of these interventions can improve intervention quality and evaluation, and facilitate the uptake and expansion. Therefore, we aim to review research reporting on the implementation of mental health promotion interventions delivered in workplace settings, in order to increase understanding of factors influencing successful delivery.

METHODS AND ANALYSIS

A scoping review will be conducted incorporating a stepwise methodology to identify relevant literature reviews, primary research and grey literature. This review is registered with Research Registry (reviewregistry897). One reviewer will conduct the search to identify English language studies in the following electronic databases from 2008 through to July 1, 2020: Scopus, PROSPERO, Health Technology Assessments, PubMed, Campbell Collaboration, Joanna Briggs Library, PsycINFO, Web of Science Core Collection, CINAHL and Institute of Occupational Safety and Health (IOSH). Reference searching, Google Scholar, Grey Matters, IOSH and expert contacts will be used to identify grey literature. Two reviewers will screen title and abstracts, aiming for 95% agreement, and then independently screen full texts for inclusion. Two reviewers will assess methodological quality of included studies using the Mixed Methods Appraisal Tool and extract and synthesize data in line with the RE-AIM framework, Nielson and Randall's model of organisational-level interventions and Moore's sustainability criteria, if the data allows. We will recruit and consult with international experts in the field to ensure engagement, reach and relevance of the main findings.

DISCUSSION

This will be the first systematic scoping review to identify and synthesise evidence of barriers and facilitators to implementing mental health promotion interventions in workplace settings. Our results will inform future evaluation studies and randomised controlled trials and highlight gaps in the evidence base.

SYSTEMATIC REVIEW REGISTRATION

Research Registry ( reviewregistry897 ).

Translating research evidence into clinical practice: a reminder of important clinical lessons in management of resistant hypertension through a case study in general practice

A case of a 59-year-old man with resistant hypertension, despite 8 months of non-pharmacological and pharmacological management up to maximal doses of triple antihypertensive therapy. Review of the literature found a study that reported improved blood pressure control with bedtime dosing of antihypertensive treatment. Changing to bedtime dosage of antihypertensives resulted in significant improvement in blood pressure control to below target levels. This highlights the importance of the clinicians' awareness and implementation of research findings and hence delivery of best evidence-based care.

Selecting Evidence-Based Content for Inclusion in Self-Management Apps for Pressure Injuries in Individuals With Spinal Cord Injury: Participatory Design Study

BACKGROUND

Technological solutions, particularly mobile health (mHealth), have been shown to be potentially viable approaches for sustaining individuals' self-management of chronic health conditions. Theory-based interventions are more successful, as evidence-based information is an essential prerequisite for appropriate self-management. However, several reviews have shown that many existing mobile apps fail to be either theoretically grounded or based on evidence. Although some authors have attempted to address these two issues by focusing on the design and development processes of apps, concrete efforts to systematically select evidence-based content are scant.

OBJECTIVE

The objective of this study was to present a procedure for the participatory identification of evidence-based content to ground the development of a self-management app.

METHODS

To illustrate the procedure, we focused on the prevention and management of pressure injuries (PIs) in individuals with spinal cord injury (SCI). The procedure involves the following three steps: (1) identification of existing evidence through review and synthesis of existing recommendations on the prevention and self-management of PIs in SCI; (2) a consensus meeting with experts from the field of SCI and individuals with SCI to select the recommendations that are relevant and applicable to community-dwelling individuals in their daily lives; and (3) consolidation of the results of the study.

RESULTS

In this case study, at the end of the three-step procedure, the content for an mHealth intervention was selected in the form of 98 recommendations.

CONCLUSIONS

This study describes a procedure for the participatory identification and selection of disease-specific evidence and professional best practices to inform self-management interventions. This procedure might be especially useful in cases of complex chronic health conditions, as every recommendation in these cases needs to be evaluated and considered in light of all other self-management requirements. Hence, the agreement of experts and affected individuals is essential to ensure the selection of evidence-based content that is considered to be relevant and applicable.

A protocol for a systematic review and meta-analysis to identify measures of breakthrough pain and evaluate their psychometric properties

INTRODUCTION

Breakthrough pain is common in children and adults with cancer and other conditions, including those approaching end-of-life, although it is often poorly managed, possibly partly due to a lack of validated assessment tools. This review aims to (1) identify all available instruments measuring breakthrough pain in infants, children, adolescents or adults and (2) critically appraise, compare and summarise the quality of the psychometric properties of the identified instruments using COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria.

METHODS AND ANALYSIS

Two searches will be carried out between October 2019 and January 2020, one for each aim of the review. The Cochrane Library, International Prospective Register of Systematic Reviews, Embase, Cumulative Index of Nursing and Allied Health Literature, Medical Literature Analysis and Retrieval System Online (MEDLINE), PsycINFO, Web of Science Core Collection, Google Scholar, the ProQuest Dissertations & Theses Database, Evidence Search and OpenGrey databases will be searched from database inception until the date the search is conducted. Reference lists of eligible articles will be screened and authors in the field contacted. For search 1, articles will be screened by two reviewers by abstract, and full-text where necessary, to identify if a breakthrough pain assessment was used. Search 2 will then be conducted to identify studies evaluating measurement properties of these assessments. Two reviewers will screen articles from search 2 by title and abstract. All potentially relevant studies will be screened by full text by both reviewers. For search 2, data will be extracted in parallel with the quality assessment process, as recommended by COSMIN. Two reviewers will assess methodological quality using the COSMIN Risk of Bias checklist and the COSMIN updated criteria for good measurement properties. Findings will be summarised and, if possible, data will be pooled using meta-analysis. The quality of the evidence will be graded and summarised using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) guidelines.

ETHICS AND DISSEMINATION

Results of this review will be submitted for publication in a peer review journal and presented at conferences.

PROSPERO REGISTRATION NUMBER

CRD42019155583.

Revisiones Sistemáticas Exploratorias como metodología para la síntesis del conocimiento científico

Introducción: En la actualidad los sistemas de salud alrededor del mundo apuestan por una toma decisiones clínicas basadas en la evidencia científica. Para ello, es necesario que los profesionales de la salud consulten los resultados de las investigaciones científicas. Sin embargo, dada la gran cantidad de literatura, los investigadores han desarrollado metodologías de revisión para compilar los estudios científicos dentro de un área específica. Aun cuando existen más de 10 tipos de metodologías para la revisión de la literatura, la Revisión Sistemática Exploratoria (RSE) ha recibido poca atención en la literatura sobre métodos de investigación científica de habla hispana. Objetivo: Detallar la metodología de la RSE, sus propósitos y las fases para su desarrollo. Desarrollo: Este trabajo detalla las generalidades de la RSE basándose en la metodología propuesta por Arksey & O’Malley. Así mismo, se describen las áreas o ámbitos donde este tipo de revisión se puede emplear, las fases para desarrollar la revisión y ejemplos de las RSE. Conclusiones: Las RSE tienen la fortaleza de hacer saber a los profesionales de la salud sobre un tema en específico que permita incidir en las políticas públicas. Al igual que las Revisiones Sistemáticas, las RSE utilizan una metodología clara y replicable, aportando datos confiables y científicos para los profesionales de la salud.

Nurses' perceptions of the sustainability of a standardised assessment for preventing complications in a ICU: a qualitative study

Background: Quality improvement projects have been widely adopted to prevent complications in the ICU. Objective: This paper describes nurses' perceptions of implementation strategies and the potential sustainability of a practice change intervention to prevent complications in a Malaysian ICU. Design: A participatory action research using five focus group discussions were undertaken with 19 nurses in a single ICU in regional Malaysia. Focus group transcripts were analysed using thematic analysis. Results: The main themes derived from the interviews were: [1] Empowering staff to embrace evidence-based practices; [2] Staff knowledge, attitudes, and beliefs that impact on behaviour; and [3] management support and leadership are influential in staff behaviours (acceptance & perseverance of change process). Discussion: Resistance to change was recognized as a barrier to adopting evidence based recommendations. There is a need to improve nurses' knowledge, attitude and awareness of the importance of assessment for VAP, CRBSI and PIs in the ICU.

The TRANSFER Approach for assessing the transferability of systematic review findings

BACKGROUND

Systematic reviews are a key input to health and social welfare decisions. Studies included in systematic reviews often vary with respect to contextual factors that may impact on how transferable review findings are to the review context. However, many review authors do not consider the transferability of review findings until the end of the review process, for example when assessing confidence in the evidence using GRADE or GRADE-CERQual. This paper describes the TRANSFER Approach, a novel approach for supporting collaboration between review authors and stakeholders from the beginning of the review process to systematically and transparently consider factors that may influence the transferability of systematic review findings.

METHODS

We developed the TRANSFER Approach in three stages: (1) discussions with stakeholders to identify current practices and needs regarding the use of methods to consider transferability, (2) systematic search for and mapping of 25 existing checklists related to transferability, and (3) using the results of stage two to develop a structured conversation format which was applied in three systematic review processes.

RESULTS

None of the identified existing checklists related to transferability provided detailed guidance for review authors on how to assess transferability in systematic reviews, in collaboration with decision makers. The content analysis uncovered seven categories of factors to consider when discussing transferability. We used these to develop a structured conversation guide for discussing potential transferability factors with stakeholders at the beginning of the review process. In response to feedback and trial and error, the TRANSFER Approach has developed, expanding beyond the initial conversation guide, and is now made up of seven stages which are described in this article.

CONCLUSIONS

The TRANSFER Approach supports review authors in collaborating with decision makers to ensure an informed consideration, from the beginning of the review process, of the transferability of the review findings to the review context. Further testing of TRANSFER is needed.

Stakeholders' experiences with the evidence aid website to support 'real-time' use of research evidence to inform decision-making in crisis zones: a user testing study

Background

Humanitarian action in crisis zones is fraught with many challenges, including lack of timely and accessible research evidence to inform decision-making about humanitarian interventions. Evidence websites have the potential to address this challenge. Evidence Aid is the only evidence website designed for crisis zones that focuses on providing research evidence in the form of systematic reviews. The objective of this study is to explore stakeholders’ views of Evidence Aid, contributing further to our understanding of the use of research evidence in decision-making in crisis zones.

Methods

We designed a qualitative user-testing study to collect interview data from stakeholders about their impressions of Evidence Aid. Eligible stakeholders included those with and without previous experience of Evidence Aid. All participants were either currently working or have worked within the last year in a crisis zone. Participants were asked to perform the same user experience-related tasks and answer questions about this experience and their knowledge needs. Data were analysed using a deductive framework analysis approach drawing on Morville’s seven facets of the user experience — findability, usability, usefulness, desirability, accessibility, credibility and value.

Results

A total of 31 interviews were completed with senior decision-makers (n = 8), advisors (n = 7), field managers (n = 7), analysts/researchers (n = 5) and healthcare providers (n = 4). Participant self-reported knowledge needs varied depending on their role. Overall, participants did not identify any ‘major’ problems (highest order) and identified only two ‘big’ problems (second highest order) with using the Evidence Aid website, namely the lack of a search engine on the home page and that some full-text articles linked to/from the site require a payment. Participants identified seven specific suggestions about how to improve Evidence Aid, many of which can also be applied to other evidence websites.

Conclusions

Stakeholders in crisis zones found Evidence Aid to be useful, accessible and credible. However, they experienced some problems with the lack of a search engine on the home page and the requirement for payment for some full-text articles linked to/from the site.

Palliative Care Evidence Review Service (PaCERS): a knowledge transfer partnership

The importance of linking evidence into practice and policy is recognised as a key pillar of a prudent approach to healthcare; it is of importance to healthcare professionals and decision-makers across the world in every speciality. However, rapid access to evidence to support service redesign, or to change practice at pace, is challenging. This is particularly so in smaller specialties such as Palliative Care, where pressured multidisciplinary clinicians lack time and skill sets to locate and appraise the literature relevant to a particular area. Therefore, we have initiated the Palliative Care Evidence Review Service (PaCERS), a knowledge transfer partnership through which we have developed a clear methodology to conduct evidence reviews to support professionals and other decision-makers working in palliative care.

PaCERS methodology utilises modified systematic review methods as there is no agreed definition or an accepted methodology for conducting rapid reviews. This paper describes the stages involved based on our iterative recent experiences and engagement with stakeholders, who are the potential beneficiaries of the research. Uniquely, we emphasise the process and opportunities of engagement with the clinical workforce and policy-makers throughout the review, from developing and refining the review question at the start through to the importance of demonstrating impact. We are faced with the challenge of the trade-off between the timely transfer of evidence against the risk of impacting on rigour. To address this issue, we try to ensure transparency throughout the review process. Our methodology aligns with key principles of knowledge synthesis in defining a process that is transparent, robust and improving the efficiency and timeliness of the review.

Our reviews are clinically or policy driven and, although we use modified systematic review methods, one of the key differences between published review processes and our review process is in our relationship with the requester. This streamlining approach to synthesising evidence in a timely manner helps to inform decisions faced by clinicians and decision-makers in healthcare settings, supporting, at pace, knowledge transfer and mobilisation.

Diagnostic methods for the canine idiopathic dilated cardiomyopathy: A narrative evidence-based rapid review

Idiopathic dilated cardiomyopathy (DCM) is an important etiology of mortality and morbidity in dogs and its diagnosis relies on systolic dysfunction, chambers dilation, electrical instability and congestion. During the last decades veterinary cardiologists have been joining efforts to obtain diagnostic resources to correctly identify canine DCM in the preclinical stage. Unfortunately, most diagnostic resources have been used with the support of research with weak evidence, without high quality methodologies such as systematic reviews or meta-analysis. Therefore, the support of evidence-based medicine is tailored by empiricism and diagnostic criteria lose'out the ability to properly classify dogs suffering DCM. The presentation of the evidence in medicine is established by multiple sources and the most reliable source has been the presentation of evidence-based medicine from systematic reviews and meta-analysis. Rapid reviews can be interpreted as a pragmatic approach to systematic reviews and although a rapid review follows most of the critical steps of a systematic review to provide timely evidence, some components of a systematic review process are either simplified or omitted. The objective of this narrative evidence-based rapid review is twofold. First: To recognize and to stratify the level of evidence offered by rigorous selected papers about the diagnosis of DCM. Second: To classify the degree of clinical recommendation of the diagnostic resources available.

Pediatric Physical Therapists' Use of the Congenital Muscular Torticollis Clinical Practice Guidelines: A Qualitative Implementation Study

PURPOSE

This study is a follow-up to the quantitative survey to examine the perceptions of pediatric physical therapists (PTs) on the application of the 2013 Congenital Muscular Torticollis Clinical Practice Guideline (CMT CPG).

METHOD

Qualitative semi-structured telephone interviews were completed. Interview questions focused on how the guidelines influenced practice, facilitators and barriers to implementation, and knowledge translation activities.

RESULTS

Thirteen pediatric PTs from a variety of practice settings participated. Positive perceptions about the CMT CPG included the use of flow charts, synthesized literature in one place, and validation of examination and intervention approaches. Negative perceptions included its length and that approaches without published evidence were not addressed. Three major themes were identified: knowledge and evidence for practice, education of clinicians, and the CPG structure and components that influenced practice.

CONCLUSIONS

The CMT CPG provided a number of benefits. Recommendations for future enhancement and development are provided.

Carrying out systematic literature reviews: an introduction

Systematic reviews provide a synthesis of evidence for a specific topic of interest, summarising the results of multiple studies to aid in clinical decisions and resource allocation. They remain among the best forms of evidence, and reduce the bias inherent in other methods. A solid understanding of the systematic review process can be of benefit to nurses that carry out such reviews, and for those who make decisions based on them. An overview of the main steps involved in carrying out a systematic review is presented, including some of the common tools and frameworks utilised in this area. This should provide a good starting point for those that are considering embarking on such work, and to aid readers of such reviews in their understanding of the main review components, in order to appraise the quality of a review that may be used to inform subsequent clinical decision making.

Variability in the validity and reliability of outcome measures identified in a systematic review to assess treatment efficacy of cognitive enhancers for Alzheimer's Dementia

Introduction

Selection of optimal outcome measures is a critical step in a systematic review; inclusion of uncommon or non-validated outcome measures can impact the uptake of systematic review findings. Our goals were to identify the validity and reliability of outcome measures used in primary studies to assess cognition, function, behaviour and global status; and, to use these data to select outcomes for a systematic review (SR) on treatment efficacy of cognitive enhancers for Alzheimer’s Dementia (AD).

Methods

Articles fulfilling the eligibility criteria of the SR were included in a charting exercise to catalogue outcome measures reported. Outcome measures were then assessed for validity and reliability. Two independent reviewers abstracted data on outcome measures and validity and reliability reported for cognition, function, behaviour and global status.

Results

129 studies were included in the charting exercise; 57 outcome measures were identified for cognition, 21 for function, 13 for behaviour and 10 for global status. A total of 35 (61%) cognition measures, 10 (48%) functional measures, 8 (61%) behavioural measures and four (40%) of global status measures were only used once in the literature. Validity and reliability information was found for 51% of cognition measures, 90% of function and global status measures and 100% of behavioural measures.

Conclusions

While a large number of outcome measures were used in primary studies, many of these were used only once. Reporting of validity and reliability varied in AD studies of cognitive enhancers. Core outcome sets should be used when available; when they are not available researchers need to balance frequency of reported outcome measures, their respective validity and reliability, and preferences of knowledge users.

Systematic review registration

CRD#42012001948

Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews

Objectives

Involvement of patients, health professionals, and the wider public (‘stakeholders’) is seen to be beneficial to the quality, relevance and impact of research and may enhance the usefulness and uptake of systematic reviews. However, there is a lack of evidence and resources to guide researchers in how to actively involve stakeholders in systematic reviews. In this paper, we report the development of the ACTIVE framework to describe how stakeholders are involved in systematic reviews.

Methods

We developed a framework using methods previously described in the development of conceptual frameworks relating to other areas of public involvement, including: literature searching, data extraction, analysis, and categorization. A draft ACTIVE framework was developed and then refined after presentation at a conference workshop, before being applied to a subset of 32 systematic reviews. Data extracted from these systematic reviews, identified in a systematic scoping review, were categorized against pre-defined constructs, including: who was involved, how stakeholders were recruited, the mode of involvement, at what stage there was involvement and the level of control or influence.

Results

The final ACTIVE framework described whether patients, carers and/or families, and/or other stakeholders (including health professionals, health decision makers and funders) were involved. We defined: recruitment as either open or closed; the approach to involvement as either one-time, continuous or combined; and the method of involvement as either direct or indirect. The stage of involvement in reviews was defined using the Cochrane Ecosystem stages of a review. The level of control or influence was defined according to the roles and activities of stakeholders in the review process, and described as the ACTIVE continuum of involvement.

Conclusions

The ACTIVE framework provides a structure with which to describe key components of stakeholder involvement within a systematic review, and we have used this to summarize how stakeholders have been involved in a subset of varied systematic reviews. The ACTIVE continuum of involvement provides a new model that uses tasks and roles to detail the level of stakeholder involvement. This work has contributed to the development of learning resources aimed at supporting systematic review authors and editors to involve stakeholders in their systematic reviews. The ACTIVE framework may support the decision-making of systematic review authors in planning how to involve stakeholders in future reviews.

Optimizing the real-world impact of rehabilitation reviews: increasing the relevance and usability of systematic reviews in rehabilitation

BACKGROUND

Despite a growing portfolio of rehabilitation reviews, uptake of review findings into practice remains slow, with review findings perceived to be lacking in relevance and usability for stakeholders. Key aspects of review design, production and dissemination have been identified to contribute to this knowledge translation (KT) gap.

AIM

The aim of this study is to identify strategies relevant to rehabilitation review design, production and dissemination which have the potential to optimize uptake of review findings into practice.

RESULTS

Two strategies are discussed, drawing on case examples of existing rehabilitation reviews, including: 1) involving stakeholders in review design, production and dissemination; and 2) moving towards theory-based, mixed methods review design. The merits of these strategies are discussed with reference to the unique and specific characteristics of the rehabilitation context, where there is complexity inherent in the multiple interacting components across population, intervention, context and implementation processes.

CONCLUSIONS

Moving towards theory-based, mixed methods reviews which involve stakeholders may be a critical first step in supporting uptake of review findings into rehabilitation practice. Doing so also has the potential to support advances in knowledge and practice in rehabilitation through theory development, as well as creating the context for evidence-based practice.

A Critical Appraisal of National and International Clinical Practice Guidelines Reporting Nutritional Recommendations for Age-Related Macular Degeneration: Are Recommendations Evidence-Based?

: Eye care professionals should have access to high quality clinical practice guidelines that ideally are underpinned by evidence from robust systematic reviews of relevant research. The aim of this study was to identify clinical guidelines with recommendations pertaining to dietary modification and/or nutritional supplementation for age-related macular degeneration (AMD), and to evaluate the overall quality of the guidelines using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument. We also mapped recommendations to existing systematic review evidence. A comprehensive search was undertaken using bibliographic databases and other electronic resources for eligible guidelines. Quality appraisal was undertaken to generate scores for each of the six AGREE II domains, and mapping of extracted nutritional recommendations was performed for systematic reviews published up to March 2017. We identified 13 national and international guidelines, developed or updated between 2004 and 2019. These varied substantially in quality. The lowest scoring AGREE II domains were for 'Rigour of Development', 'Applicability' (which measures implementation strategies to improve uptake of recommendations), and 'Editorial Independence'. Only four guidelines used evidence from systematic reviews to support their nutritional recommendations. In conclusion, there is significant scope for improving current Clinical Practice Guidelines for AMD, and guideline developers should use evidence from existing high quality systematic reviews to inform clinical recommendations.

Systematic mapping of checklists for assessing transferability

BACKGROUND

Systematic reviews of research evidence have become an expected basis for decisions about practice guidelines and policy decisions in the health and welfare sectors. Review authors define inclusion criteria to help them determine which studies to search for and include in their reviews. However, these studies may still vary in the extent to which they reflect the context of interest in the review question. While most review authors would agree that systematic reviews should be relevant and useful for decision makers, there appears to be few well known, if any, established methods for supporting review authors to assess the transferability of review findings to the context of interest in the review. With this systematic mapping and content analysis, we aim to identify whether there exists checklists to support review authors in considering transferability early in the systematic review process. The secondary aim was to develop a comprehensive list of factors that influence transferability as discussed in existing checklists.

METHODS

We conducted a systematic mapping of checklists and performed a content analysis of the checklist criteria included in the identified checklists. In June 2016, we conducted a systematic search of eight databases to identify checklists to assess transferability of findings from primary or secondary research, without limitations related to publication type, status, language, or date. We also conducted a gray literature search and searched the EQUATOR repository of checklists for any relevant document. We used search terms such as modified versions of the terms "transferability," "applicability," "generalizability," etc. and "checklist," "guideline," "tool," "criteria," etc. We did not include papers that discussed transferability at a theoretical level or checklists to assess the transferability of guidelines to local contexts.

RESULTS

Our search resulted in 11,752 titles which were screened independently by two review authors. The 101 articles which were considered potentially relevant were subsequently read by two authors, independently in full text and assessed for inclusion. We identified 31 relevant checklists. Six of these examined transferability of economic evaluations, and 25 examined transferability of primary or secondary research findings in health (n = 23) or social welfare (n = 2). The content analysis is based on the 25 health and social welfare checklists. We identified seven themes under which we grouped categories of checklist criteria: population, intervention, implementation context (immediate), comparison intervention, outcomes, environmental context, and researcher conduct.

CONCLUSIONS

We identified a variety of checklists intended to support end users (researchers, review authors, practitioners, etc.) to assess transferability or related concepts. While four of these checklists are intended for use in systematic reviews of effectiveness, we found no checklists for qualitative evidence syntheses or for the field of social welfare practice or policy. Furthermore, none of the identified checklists for review authors included guidance to on how to assess transferability, or present assessments in a systematic review. The results of the content analysis can serve as the basis for developing a comprehensive list of factors to be used in an approach to support review authors in systematically and transparently considering transferability from the beginning of the review process.

Identifying and Addressing Language Needs in Primary Care: a Pilot Implementation Study

BACKGROUND

Medical interpreters improve care for patients with Limited English Proficiency but are underused. Protocols to improve interpreter use in primary care are needed.

METHODS

Medical Assistants (MAs) screened patients for language needs and arranged for telephone interpreters during rooming in two pilot clinics (PCs). We interviewed MAs and providers and analyzed interviews using modified grounded theory, linking themes to the Promoting Action on Research Implementation in Health Services (PARiHS) framework categories of Context, Evidence, and Facilitation. Providers in PCs and four comparison clinics were surveyed.

RESULTS

Context themes included issues with the telephone interpreter vendor; having established teams, roles and workflows; and difficulty incorporating time-sensitive tasks. Evidence themes included engagement in language screening; preferring in-person interpreters; improving the patient experience; and having mixed responses to the protocol. Facilitation themes included MAs needing more support. PC providers were more satisfied with care (OR = 12.7) and communication (OR = 7.6) than comparison clinic providers.

CONCLUSIONS

The protocol may improve patient care and communication, but implementation was inconsistent. Language screening is a complex process and further research is needed to improve screening questions and procedures. Future interventions should capitalize on team members' drives to improve patient care and control costs but also need to consider the impacts of health system changes, and to consider the culture, training needs, roles, and relationships of team members.

Stakeholder involvement in systematic reviews: a scoping review

BACKGROUND

There is increasing recognition that it is good practice to involve stakeholders (meaning patients, the public, health professionals and others) in systematic reviews, but limited evidence about how best to do this. We aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews.

METHODS

We carried out a scoping review, following a published protocol. We searched multiple electronic databases (2010-2016), using a stepwise searching approach, supplemented with hand searching. Two authors independently screened and discussed the first 500 abstracts and, after clarifying selection criteria, screened a further 500. Agreement on screening decisions was 97%, so screening was done by one reviewer only. Pre-planned data extraction was completed, and the comprehensiveness of the description of methods of involvement judged. Additional data extraction was completed for papers judged to have most comprehensive descriptions. Three stakeholder representatives were co-authors for this systematic review.

RESULTS

We included 291 papers in which stakeholders were involved in a systematic review. Thirty percent involved patients and/or carers. Thirty-two percent were from the USA, 26% from the UK and 10% from Canada. Ten percent (32 reviews) were judged to provide a comprehensive description of methods of involving stakeholders. Sixty-nine percent (22/32) personally invited people to be involved; 22% (7/32) advertised opportunities to the general population. Eighty-one percent (26/32) had between 1 and 20 face-to-face meetings, with 83% of these holding ≤ 4 meetings. Meetings lasted 1 h to ½ day. Nineteen percent (6/32) used a Delphi method, most often involving three electronic rounds. Details of ethical approval were reported by 10/32. Expenses were reported to be paid to people involved in 8/32 systematic reviews.

DISCUSSION/CONCLUSION

We identified a relatively large number (291) of papers reporting stakeholder involvement in systematic reviews, but the quality of reporting was generally very poor. Information from a subset of papers judged to provide the best descriptions of stakeholder involvement in systematic reviews provide examples of different ways in which stakeholders have been involved in systematic reviews. These examples arguably currently provide the best available information to inform and guide decisions around the planning of stakeholder involvement within future systematic reviews. This evidence has been used to develop online learning resources.

SYSTEMATIC REVIEW REGISTRATION

The protocol for this systematic review was published on 21 April 2017. Publication reference: Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Watts C, Morley R: Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. Research Involvement and Engagement 2017, 3:9. https://doi.org/10.1186/s40900-017-0060-4 .

Knowledge translation for realist reviews: a participatory approach for a review on scaling up complex interventions

Background

Knowledge syntheses that use a realist methodology are gaining popularity. Yet, there are few reports in the literature that describe how results are summarised, shared and used. This paper aims to inform knowledge translation (KT) for realist reviews by describing the process of developing a KT strategy for a review on pathways for scaling up complex public health interventions.

Methods

The participatory approach used for the realist review was also used to develop the KT strategy. The approach included three main steps, namely (1) an international meeting focused on interpreting preliminary findings from the realist review and seeking input on KT activities; (2) a targeted literature review on KT for realist reviews; and (3) consultations with primary knowledge users of the review.

Results

The international meeting identified a general preference among knowledge users for findings from the review that are action oriented. A need was also identified for understanding how to tailor findings for specific knowledge user groups in relation to their needs. The literature review identified four papers that included brief descriptions of planned or actual KT activities for specific research studies; however, information was minimal on what KT activities or products work for whom, under what conditions and why. The consultations revealed that KT for realist reviews should consider the following: (1) activities closely aligned with the preferences of specific knowledge user groups; (2) key findings that are sensitive to factors within the knowledge user’s context; and (3) actionable statements that can advance KT goals, activities or products. The KT strategy derived from the three activities includes a planning framework and tailored KT activities that address preferences of knowledge users for findings that are action oriented and context relevant.

Conclusions

This paper provides an example of a KT strategy for realist reviews that blends theoretical and practical insights. Evaluation of the strategy’s implementation will provide useful insights on its effectiveness and potential for broader application.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0374-1) contains supplementary material, which is available to authorized users.

Use of post-graduate students' research in evidence informed health policies: a case study of Makerere University College of Health Sciences, Uganda

BACKGROUND

World over, stakeholders are increasingly concerned about making research useful in public policy-making. However, there are hardly any reports linking production of research by students at institutions of higher learning to its application in society. We assessed whether and how post-graduate students' research was used in evidence-informed health policies.

METHODS

This is a multiple case study of master's students' dissertations at Makerere University College of Health Sciences (MakCHS) produced between 1996 and 2010. In a structured review, we applied a theoretical framework of 'research use' and used content analysis to map how research was used in public policy documents. We categorised content of these documents according to the health-related Millennium Development Goals (MDG). We defined a case of 'use' as citation of research products from a master's student's dissertation in a public policy-related document.

RESULTS

We found 22 cases of research use in policy-related documents (0.5%) out of a total 4230 citations from 16 of 1172 total dissertations (1.4%). Additionally, research was mostly cited in primary studies (95.4%), systematic reviews (3%), narrative reviews (0.8%) and cost-effectiveness analyses (0.2%). Research was predominantly used instrumentally, to either frame the problem (burden of disease or health condition) or select an intervention (treatment or diagnostic option) and rarely symbolically to justify strategies already selected. The bulk of the cases of research use addressed child health (MDG 4), focusing on infectious diseases (MDG 6), mainly in international clinical or public health guidelines, working papers, a consensus statement and a global report. We distilled 'synergistic relationships' among organisations or interest groups, 'globalisation of local evidence', 'trade-offs' in the use of research and use of 'negative results' from the documents and text content.

CONCLUSIONS

Research from dissertations of post-graduate students at MakCHS is used in evidence-informed health policies, particularly for infectious diseases in child health. Further, we have delineated pathways of research use in the global arena and highlighted the importance of 'negative results' from dissertations of post-graduate students at MakCHS.

The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review

Background

The objective of this systematic review was to summarize and evaluate evidence about the effectiveness of knowledge translation (KT) interventions to improve the uptake and application of clinical practice guidelines and best practices for a wide range of musculoskeletal (MSK) disorders and health care professionals.

Methods

A search for relevant randomized controlled trials (RCTs) published in English was conducted in MEDLINE (Ovid interface), EMBASE, CINAHL, and CENTRAL (Cochrane library). Two independent reviewers selected studies, assessed risk of bias, and extracted data. All MSK disorders were included except MSK injuries, fractures, trauma, or inflammatory disorders.

Results

A total of 7904 citations yielded 11 eligible RCTs. The targeted MSK disorders included: low back pain (n = 5), neck pain (n = 2), whiplash (1), spinal disorders (n = 1), and osteoarthritis of the hip and knee (n = 2). Studies primarily involved physiotherapists, chiropractors, and a mix of physiotherapists, chiropractors and osteopaths. Results were reported using effect sizes (Cohen’s d). Interactive educational meetings were the most commonly used KT strategy. For professional outcomes, 3 studies using single-component interventions had a small effect (d ranges from 0.14 to 0.28) and 7 studies used multifaceted interventions (3 were effective (d ranges from 0.824 to 2.27). For patient outcomes, 4 studies were ineffective (d ranges from 0.06 to 0.31). The majority of the included RCTs had moderate-to-high risk of bias. About half of the studies used theory-based interventions, but the elements of the interventions and theoretical frameworks were often poorly described. Furthermore, there were no comparable outcome measures to evaluate the impact of the interventions on a similar scale.

Conclusions

The findings suggested that multifaceted educational KT interventions appear to be effective for improving professional outcomes, although effects were inconsistent. The KT strategies were generally not effective on patient outcomes. In general, studies were of low quality, interventions were poorly described, and only half had theoretical underpinning. Researchers are encouraged to use validated professional and patient outcomes.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3253-0) contains supplementary material, which is available to authorized users.

Early impact of guideline publication on angiotensin-receptor neprilysin inhibitor use among patients hospitalized for heart failure

BACKGROUND

On May 20, 2016, US professional organizations in cardiology published joint treatment guidelines recommending the use of angiotensin-receptor neprilysin inhibitor (ARNI) for eligible patients with heart failure with reduced ejection fraction (HFrEF). Using data from the Get With The Guidelines-Heart Failure registry, we evaluated the early impact of this update on temporal trends in ARNI prescription.

METHODS

We analyzed patients with HFrEF who were eligible for ARNI prescription (EF ≤40%, no contraindications) and hospitalized from February 20, 2016, through August 19, 2016-allowing for 13weeks before and after guideline publication. We quantified trends in ARNI use associated with guidelines publication with an interrupted time-series design using logistic regression and accounting for correlations within hospitals using general estimating equation methods.

RESULTS

Of 7,200 eligible patient hospitalizations, 51.9% were discharged in the period directly preceding publication of the guidelines, and 48.1% were discharged after. Odds ratios of ARNI prescription at discharge were significantly higher in the postguideline period compared with the preguideline period in adjusted models (adjusted odds ratio 1.29, 95% CI 1.06-1.57, P=.01). However, there was no significant interaction between observed and expected ARNI use after guideline publication (P_interaction=.14). Results were consistent using a 6-month before and after time frame.

CONCLUSIONS

The model suggested a small increase in ARNI use in HF patients being discharged from the hospital immediately after guideline release. However, the publication of national guidelines recommending ARNI use seemed to have little influence on the adoption of this evidence-based medication in the first 3 to 6months.

A comparative evaluation of PDQ-Evidence

Background

A strategy for minimising the time and obstacles to accessing systematic reviews of health system evidence is to collect them in a freely available database and make them easy to find through a simple ‘Google-style’ search interface. PDQ-Evidence was developed in this way. The objective of this study was to compare PDQ-Evidence to six other databases, namely Cochrane Library, EVIPNet VHL, Google Scholar, Health Systems Evidence, PubMed and Trip.

Methods

We recruited healthcare policy-makers, managers and health researchers in low-, middle- and high-income countries. Participants selected one of six pre-determined questions. They searched for a systematic review that addressed the chosen question and one question of their own in PDQ-Evidence and in two of the other six databases which they would normally have searched. We randomly allocated participants to search PDQ-Evidence first or to search the two other databases first. The primary outcomes were whether a systematic review was found and the time taken to find it. Secondary outcomes were perceived ease of use and perceived time spent searching. We asked open-ended questions about PDQ-Evidence, including likes, dislikes, challenges and suggestions for improvements.

Results

A total of 89 people from 21 countries completed the study; 83 were included in the primary analyses and 6 were excluded because of data errors that could not be corrected. Most participants chose PubMed and Cochrane Library as the other two databases. Participants were more likely to find a systematic review using PDQ-Evidence than using Cochrane Library or PubMed for the pre-defined questions. For their own questions, this difference was not found. Overall, it took slightly less time to find a systematic review using PDQ-Evidence. Participants perceived that it took less time, and most participants perceived PDQ-Evidence to be slightly easier to use than the two other databases. However, there were conflicting views about the design of PDQ-Evidence.

Conclusions

PDQ-Evidence is at least as efficient as other databases for finding health system evidence. However, using PDQ-Evidence is not intuitive for some people.

Trial registration

The trial was prospectively registered in the ISRCTN registry 17 April 2015. Registration number: ISRCTN12742235.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0299-8) contains supplementary material, which is available to authorized users.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 8: developing, implementing and evaluating an evidence dissemination service in a local healthcare setting

BACKGROUND

This is the eighth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE Program was a systematic, integrated, evidence-based program for disinvestment within a large Australian health service. One of the aims was to explore methods to deliver existing high quality synthesised evidence directly to decision-makers to drive decision-making proactively. An Evidence Dissemination Service (EDS) was proposed. While this was conceived as a method to identify disinvestment opportunities, it became clear that it could also be a way to review all practices for consistency with current evidence. This paper reports the development, implementation and evaluation of two models of an in-house EDS.

METHODS

Frameworks for development of complex interventions, implementation of evidence-based change, and evaluation and explication of processes and outcomes were adapted and/or applied. Mixed methods including a literature review, surveys, interviews, workshops, audits, document analysis and action research were used to capture barriers, enablers and local needs; identify effective strategies; develop and refine proposals; ascertain feedback and measure outcomes.

RESULTS

Methods to identify, capture, classify, store, repackage, disseminate and facilitate use of synthesised research evidence were investigated. In Model 1, emails containing links to multiple publications were sent to all self-selected participants who were asked to determine whether they were the relevant decision-maker for any of the topics presented, whether change was required, and to take the relevant action. This voluntary framework did not achieve the aim of ensuring practice was consistent with current evidence. In Model 2, the need for change was established prior to dissemination, then a summary of the evidence was sent to the decision-maker responsible for practice in the relevant area who was required to take appropriate action and report the outcome. This mandatory governance framework was successful. The factors influencing decisions, processes and outcomes were identified.

CONCLUSION

An in-house EDS holds promise as a method of identifying disinvestment opportunities and/or reviewing local practice for consistency with current evidence. The resource-intensive nature of delivery of the EDS is a potential barrier. The findings from this study will inform further exploration.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation

Background

This is the tenth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. After more than a decade of research, there is little published evidence of active and successful disinvestment. The paucity of frameworks, methods and tools is reported to be a factor in the lack of success. However there are clear and consistent messages in the literature that can be used to inform development of a framework for operationalising disinvestment. This paper, along with the conceptual review of disinvestment in Paper 9 of this series, aims to integrate the findings of the SHARE Program with the existing disinvestment literature to address the lack of information regarding systematic organisation-wide approaches to disinvestment at the local health service level.

Discussion

A framework for disinvestment in a local healthcare setting is proposed. Definitions for essential terms and key concepts underpinning the framework have been made explicit to address the lack of consistent terminology. Given the negative connotations of the word ‘disinvestment’ and the problems inherent in considering disinvestment in isolation, the basis for the proposed framework is ‘resource allocation’ to address the spectrum of decision-making from investment to disinvestment. The focus is positive: optimising healthcare, improving health outcomes, using resources effectively.

The framework is based on three components: a program for decision-making, projects to implement decisions and evaluate outcomes, and research to understand and improve the program and project activities. The program consists of principles for decision-making and settings that provide opportunities to introduce systematic prompts and triggers to initiate disinvestment. The projects follow the steps in the disinvestment process. Potential methods and tools are presented, however the framework does not stipulate project design or conduct; allowing application of any theories, methods or tools at each step. Barriers are discussed and examples illustrating constituent elements are provided.

Conclusions

The framework can be employed at network, institutional, departmental, ward or committee level. It is proposed as an organisation-wide application, embedded within existing systems and processes, which can be responsive to needs and priorities at the level of implementation. It can be used in policy, management or clinical contexts.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2506-7) contains supplementary material, which is available to authorized users.

Facilitators and barriers of change toward an elder-friendly surgical environment: perspectives of clinician stakeholder groups

BACKGROUND

Current acute care surgical practices do not focus on the unique needs of older adults. Adverse outcomes in older patients result from a complex interrelationship between baseline vulnerability and insults experienced during hospitalization. The purpose of this study is to assess the organizational readiness and the barriers and facilitators for the implementation of elder-friendly interventions in the acute care of unplanned abdominal surgery patients.

METHODS

This cross-sectional mixed methods study included a convenience sample of clinician stakeholder groups. Eight focus groups were conducted with 33 surgical team members including: 10 health care aides, 6 licensed practical nurses, 6 registered nurses, 4 nurse managers and 7 surgeons, to identify barriers and facilitators to the implementation of an elder-friendly surgical unit. Audio recordings of the focus groups were transcribed verbatim and analysed using interpretive description techniques. Transcripts were coded along with explanatory memos to generate a detailed description of participant experiences. Themes were identified followed by refining the codes. Participants also completed the Organizational Readiness for Implementing Change questionnaire. Differences in organizational readiness scores across clinician stakeholder groups were assessed using Kruskal-Wallice tests. Mann-Whitney tests (Bonferroni's corrections for multiple comparisons) were conducted to assess pair-wise relationships.

RESULTS

The focus group data were conceptualized to represent facilitators and barriers to change at two levels of care delivery. Readiness to change at the organizational level was evident in five categories that reflected the barriers and facilitators to implementing an elder-friendly surgical unit. These included education, environment, staffing, policies and other research projects. At the individual level barriers and facilitators were apparent in staff members' acceptance of new roles and duties with other staff, family and patients. Examples of these included communication, teamwork and leadership. The mean change commitment and change efficacy scores on the Organizational Readiness for Implementing Change Questionnaire were 3.7 (0.8) and 3.5 (0.9) respectively. No statistically significant differences were detected between the stakeholder groups.

CONCLUSIONS

Staff are interested in contributing to improved care for elderly surgical patients; however, opportunities were identified to enhance facilitators and reduce barriers in advance of implementing the elder-friendly surgical unit intervention.

Supporting Evidence-Informed Teaching in Biomedical and Health Professions Education Through Knowledge Translation: An Interdisciplinary Literature Review

Phenomenon: The purpose of "systematic" reviews/reviewers of medical and health professions educational research is to identify best practices. This qualitative article explores the question of whether systematic reviews can support "evidence informed" teaching and contrasts traditional systematic reviewing with a knowledge translation (KT) approach to this objective.

APPROACH

Degrees of freedom analysis (DOFA) is used to examine the alignment of systematic review methods with educational research and the pedagogical strategies and approaches that might be considered with a decision-making framework developed to support valid assessment. This method is also used to explore how KT can be used to inform teaching and learning.

FINDINGS

The nature of educational research is not compatible with most (11/14) methods for systematic review. The inconsistency of systematic reviewing with the nature of educational research impedes both the identification and implementation of "best-evidence" pedagogy and teaching. This is primarily because research questions that do support the purposes of review do not support educational decision making. By contrast to systematic reviews of the literature, both a DOFA and KT are fully compatible with informing teaching using evidence. A DOFA supports the translation of theory to a specific teaching or learning case, so could be considered a type of KT. The DOFA results in a test of alignment of decision options with relevant educational theory, and KT leads to interventions in teaching or learning that can be evaluated. Examples of how to structure evaluable interventions are derived from a KT approach that are simply not available from a systematic review. Insights: Systematic reviewing of current empirical educational research is not suitable for deriving or supporting best practices in education. However, both "evidence-informed" and scholarly approaches to teaching can be supported as KT projects, which are inherently evaluable and can generate actionable evidence about whether the decision or intervention worked for students, instructors, and the institution. A DOFA can also support evidence- and theory-informed teaching to develop an understanding of what works, why, and for whom. Thus KT, but not systematic reviewing, can support decision making around pedagogy (and pedagogical innovation) that can also inform new teaching and learning initiatives; it can also point to new avenues of empirical research in education that are informed by, and can inform, theory.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 7: supporting staff in evidence-based decision-making, implementation and evaluation in a local healthcare setting

BACKGROUND

This is the seventh in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE Program was a systematic, integrated, evidence-based program for resource allocation within a large Australian health service. It aimed to facilitate proactive use of evidence from research and local data; evidence-based decision-making for resource allocation including disinvestment; and development, implementation and evaluation of disinvestment projects. From the literature and responses of local stakeholders it was clear that provision of expertise and education, training and support of health service staff would be required to achieve these aims. Four support services were proposed. This paper is a detailed case report of the development, implementation and evaluation of a Data Service, Capacity Building Service and Project Support Service. An Evidence Service is reported separately.

METHODS

Literature reviews, surveys, interviews, consultation and workshops were used to capture and process the relevant information. Existing theoretical frameworks were adapted for evaluation and explication of processes and outcomes.

RESULTS

Surveys and interviews identified current practice in use of evidence in decision-making, implementation and evaluation; staff needs for evidence-based practice; nature, type and availability of local health service data; and preferred formats for education and training. The Capacity Building and Project Support Services were successful in achieving short term objectives; but long term outcomes were not evaluated due to reduced funding. The Data Service was not implemented at all. Factors influencing the processes and outcomes are discussed.

CONCLUSION

Health service staff need access to education, training, expertise and support to enable evidence-based decision-making and to implement and evaluate the changes arising from those decisions. Three support services were proposed based on research evidence and local findings. Local factors, some unanticipated and some unavoidable, were the main barriers to successful implementation. All three proposed support services hold promise as facilitators of EBP in the local healthcare setting. The findings from this study will inform further exploration.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 6: investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting

BACKGROUND

This is the sixth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE program was established to investigate a systematic, integrated, evidence-based approach to disinvestment within a large Australian health service. This paper describes the methods employed in undertaking pilot disinvestment projects. It draws a number of lessons regarding the strengths and weaknesses of these methods; particularly regarding the crucial first step of identifying targets for disinvestment.

METHODS

Literature reviews, survey, interviews, consultation and workshops were used to capture and process the relevant information. A theoretical framework was adapted for evaluation and explication of disinvestment projects, including a taxonomy for the determinants of effectiveness, process of change and outcome measures. Implementation, evaluation and costing plans were developed.

RESULTS

Four literature reviews were completed, surveys were received from 15 external experts, 65 interviews were conducted, 18 senior decision-makers attended a data gathering workshop, 22 experts and local informants were consulted, and four decision-making workshops were undertaken. Mechanisms to identify disinvestment targets and criteria for prioritisation and decision-making were investigated. A catalogue containing 184 evidence-based opportunities for disinvestment and an algorithm to identify disinvestment projects were developed. An Expression of Interest process identified two potential disinvestment projects. Seventeen additional projects were proposed through a non-systematic nomination process. Four of the 19 proposals were selected as pilot projects but only one reached the implementation stage. Factors with potential influence on the outcomes of disinvestment projects are discussed and barriers and enablers in the pilot projects are summarised.

CONCLUSION

This study provides an in-depth insight into the experience of disinvestment in one local healthcare service. To our knowledge, this is the first paper to report the process of disinvestment from identification, through prioritisation and decision-making, to implementation and evaluation, and finally explication of the processes and outcomes.

Policymaker experiences with rapid response briefs to address health-system and technology questions in Uganda

Background

Health service and systems researchers have developed knowledge translation strategies to facilitate the use of reliable evidence for policy, including rapid response briefs as timely and responsive tools supporting decision making. However, little is known about users’ experience with these newer formats for presenting evidence. We sought to explore Ugandan policymakers’ experience with rapid response briefs in order to develop a format acceptable for policymakers.

Methods

We used existing research regarding evidence formats for policymakers to inform the initial version of rapid response brief format. We conducted user testing with healthcare policymakers at various levels of decision making in Uganda, employing a concurrent think-aloud method, collecting data on elements including usability, usefulness, understandability, desirability, credibility and value of the document. We modified the rapid response briefs format based on the results of the user testing and sought feedback on the new format.

Results

The participants generally found the format of the rapid response briefs usable, credible, desirable and of value. Participants expressed frustrations regarding several aspects of the document, including the absence of recommendations, lack of clarity about the type of document and its potential uses (especially for first time users), and a crowded front page. Participants offered conflicting feedback on preferred length of the briefs and use and placement of partner logos. Users had divided preferences for the older and newer formats.

Conclusion

Although the rapid response briefs were generally found to be of value, there are major and minor frustrations impeding an optimal user experience. Areas requiring further research include how to address policymakers’ expectations of recommendations in these briefs and their optimal length.

Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects

PLAIN ENGLISH SUMMARY

Researchers are expected to actively involve stakeholders (including patients, the public, health professionals, and others) in their research. Although researchers increasingly recognise that this is good practice, there is limited practical guidance about how to involve stakeholders. Systematic reviews are a research method in which international literature is brought together, using carefully designed and rigorous methods to answer a specified question about healthcare. We want to investigate how researchers have involved stakeholders in systematic reviews, and how involvement has potentially affected the quality and impact of reviews. We plan to bring this information together by searching and reviewing the literature for reports of stakeholder involvement in systematic reviews. This paper describes in detail the methods that we plan to use to do this. After carrying out comprehensive searches for literature, we will: 1. Provide an overview of identified reports, describing key information such as types of stakeholders involved, and how. 2. Pick out reports of involvement which include detailed descriptions of how researchers involved people in a systematic review and summarise the methods they used. We will consider who was involved, how people were recruited, and how the involvement was organised and managed. 3. Bring together any reports which have explored the effect, or impact, of involving stakeholders in a systematic review. We will assess the quality of these reports, and summarise their findings. Once completed, our review will be used to produce training resources aimed at helping researchers to improve ways of involving stakeholders in systematic reviews.

ABSTRACT

Background There is an expectation for stakeholders (including patients, the public, health professionals, and others) to be involved in research. Researchers are increasingly recognising that it is good practice to involve stakeholders in systematic reviews. There is currently a lack of evidence about (A) how to do this and (B) the effects, or impact, of such involvement. We aim to create a map of the evidence relating to stakeholder involvement in systematic reviews, and use this evidence to address the two points above. Methods We will complete a mixed-method synthesis of the evidence, first completing a scoping review to create a broad map of evidence relating to stakeholder involvement in systematic reviews, and secondly completing two contingent syntheses. We will use a stepwise approach to searching; the initial step will include comprehensive searches of electronic databases, including CENTRAL, AMED, Embase, Medline, Cinahl and other databases, supplemented with pre-defined hand-searching and contacting authors. Two reviewers will undertake each review task (i.e., screening, data extraction) using standard systematic review processes. For the scoping review, we will include any paper, regardless of publication status or study design, which investigates, reports or discusses involvement in a systematic review. Included papers will be summarised within structured tables. Criteria for judging the focus and comprehensiveness of the description of methods of involvement will be applied, informing which papers are included within the two contingent syntheses. Synthesis A will detail the methods that have been used to involve stakeholders in systematic reviews. Papers from the scoping review that are judged to provide an adequate description of methods or approaches will be included. Details of the methods of involvement will be extracted from included papers using pre-defined headings, presented in tables and described narratively. Synthesis B will include studies that explore the effect of stakeholder involvement on the quality, relevance or impact of a systematic review, as identified from the scoping review. Study quality will be appraised, data extracted and synthesised within tables. Discussion This review should help researchers select, improve and evaluate methods of involving stakeholders in systematic reviews. Review findings will contribute to Cochrane training resources.

Determinants of evidence use in public health policy making: Results from a study across six EU countries

The knowledge-practice gap in public health is widely known. The importance of using different types of evidence for the development of effective health promotion has also been emphasized. Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking the most effective approaches, thus limiting the impact of health promotion strategies. This article focuses on facilitators and barriers in the use of evidence in developing health enhancing physical activity policies. Data was collected in 2012 by interviewing 86 key stakeholders from six EU countries (FI, DK, UK, NL, IT, RO) using a common topic guide. Content analysis and concept mapping was used to construct a map of facilitators and barriers. Barriers and facilitators experienced by most stakeholders and policy context in each country are analysed. A lack of locally useful and concrete evidence, evidence on costs, and a lack of joint understanding were specific hindrances. Also users' characteristics and the role media play were identified as factors of influence. Attention for individual and social factors within the policy context might provide the key to enhance more sustainable evidence use. Developing and evaluating tailored approaches impacting on networking, personal relationships, collaboration and evidence coproduction is recommended.

Technical Packages in Injury and Violence Prevention to Move Evidence Into Practice: Systematic Reviews and Beyond

Injury and violence prevention strategies have greater potential for impact when they are based on scientific evidence. Systematic reviews of the scientific evidence can contribute key information about which policies and programs might have the greatest impact when implemented. However, systematic reviews have limitations, such as lack of implementation guidance and contextual information, that can limit the application of knowledge. "Technical packages," developed by knowledge brokers such as the federal government, nonprofit agencies, and academic institutions, have the potential to be an efficient mechanism for making information from systematic reviews actionable. Technical packages provide information about specific evidence-based prevention strategies, along with the estimated costs and impacts, and include accompanying implementation and evaluation guidance to facilitate adoption, implementation, and performance measurement. We describe how systematic reviews can inform the development of technical packages for practitioners, provide examples of technical packages in injury and violence prevention, and explain how enhancing review methods and reporting could facilitate the use and applicability of scientific evidence.