Practice guidelines and measurement: state-of-the-science

Are we ready for conformance checking in healthcare? Measuring adherence to clinical guidelines: A scoping systematic literature review

Clinical guidelines are recommendations of how to diagnose, treat, and manage a patient's medical condition. Health organizations must measure adherence to clinical guidelines to enhance the quality of service, but due to the complexity of the medical environment, there is no simple way of measuring adherence to clinical guidelines. This scoping review will systematically assess the criteria used to measure adherence to clinical guidelines in the past 20 years and explore the suitability of using process mining techniques. We will use a workflow protocol based on declarative and temporal constraints to translate the narrative text rules in the publications into a high-level process model. This approach will enable us to explore the main patterns and gaps identified when measuring adherence to clinical guidelines and how they affect the adoption of process mining techniques. The main contributions of this paper are a) a comprehensive analysis of the criteria used for measuring adherence, considering a diverse set of medical conditions b) a framework that will classify the level of complexity of the rules used to measure adherence based on declarative and temporal constraints c) list of key trends and gaps identified in the literature and how they relate to the use of process mining techniques in healthcare.

Adherence to guidelines in the management of urolithiasis: are there differences among distinct patient care settings?

INTRODUCTION

Urolithiasis is a common diagnosis in urology. New technologies offer a variety of diagnostic and therapy and consequently display a financial burden on healthcare systems. Hence, clinical practice guidelines (CPG) are essential to implement evidence-based medicine and assure a standard of care considering limited resources. To date, there is no evidence of the use and adherence to CPG on urolithiasis.

MATERIAL AND METHODS

Therefore, we performed a cross-sectional study to analyze the use of CPG on urolithiasis. Data collection was carried out by a questionnaire given to 400 German urologists. The survey included use and adherence to guidelines, evaluation of the clinical situation, therapy spectrum, and workplace. In total, 150 (37%) questionnaires were received and included in our survey. Statistics were performed by SPSS using Chi-quadrat test/Fisher's exact test.

RESULTS

In our study, urologists were office based, hospital affiliated, non-academic, or academic centers in 53%, 32%, 16% and 5%, respectively. In 74% and 70%, urologists adhere to CPG in diagnostic and therapy. Interestingly, workplace and therapy spectrum determines the use of different CPG (p = 0.01; p = 0.022). Academic urologists were more likely to use international CPG of EAU (40%), while outpatient urologists significantly orientated on national CPG (46%). 86% of urologists with high volume of urolithiasis practice interventions in contrast to 53% in low volume (p = 0.001). More than 80% of urologists use short versions and app version of CPG.

CONCLUSION

We firstly describe compliance and the use of CPG on urolithiasis. EAU and DGU present the most commonly used CPG. Short version and app version of CPG find frequent clinical utilization.

Feasibility of using algorithm-based clinical decision support for symptom assessment and management in lung cancer

CONTEXT

Distressing symptoms interfere with the quality of life in patients with lung cancer. Algorithm-based clinical decision support (CDS) to improve evidence-based management of isolated symptoms seems promising, but no reports yet address multiple symptoms.

OBJECTIVES

This study examined the feasibility of CDS for a Symptom Assessment and Management Intervention targeting common symptoms in patients with lung cancer (SAMI-L) in ambulatory oncology. The study objectives were to evaluate completion and delivery rates of the SAMI-L report and clinician adherence to the algorithm-based recommendations.

METHODS

Patients completed a web-based symptom assessment and SAMI-L created tailored recommendations for symptom management. Completion of assessments and delivery of reports were recorded. Medical record review assessed clinician adherence to recommendations. Feasibility was defined as 75% or higher report completion and delivery rates and 80% or higher clinician adherence to recommendations. Descriptive statistics and generalized estimating equations were used for data analyses.

RESULTS

Symptom assessment completion was 84% (95% CI=81-87%). Delivery of completed reports was 90% (95% CI=86-93%). Depression (36%), pain (30%), and fatigue (18%) occurred most frequently, followed by anxiety (11%) and dyspnea (6%). On average, overall recommendation adherence was 57% (95% CI=52-62%) and was not dependent on the number of recommendations (P=0.45). Adherence was higher for anxiety (66%; 95% CI=55-77%), depression (64%; 95% CI=56-71%), pain (62%; 95% CI=52-72%), and dyspnea (51%; 95% CI=38-64%) than for fatigue (38%; 95% CI=28-47%).

CONCLUSION

The CDS systems, such as SAMI-L, have the potential to fill a gap in promoting evidence-based care.

Methodology issues in implementation science

BACKGROUND

Putting evidence into practice at the point of care delivery requires an understanding of implementation strategies that work, in what context and how.

OBJECTIVE

To identify methodological issues in implementation science using 4 studies as cases and make recommendations for further methods development.

RESEARCH DESIGN

Four cases are presented and methodological issues identified. For each issue raised, evidence on the state of the science is described.

RESULTS

Issues in implementation science identified include diverse conceptual frameworks, potential weaknesses in pragmatic study designs, and the paucity of standard concepts and measurement.

CONCLUSIONS

Recommendations to advance methods in implementation include developing a core set of implementation concepts and metrics, generating standards for implementation methods including pragmatic trials, mixed methods designs, complex interventions and measurement, and endorsing reporting standards for implementation studies.

Differences in quality of antenatal care provided by midwives to low-risk pregnant dutch women in different ethnic groups

INTRODUCTION

The objective of this study was to evaluate whether differences existed in the adherence to the Dutch national guidelines regarding basic antenatal care by Dutch midwives for low-risk women of different ethnic groups.

METHODS

This was an observational study using data from electronic antenatal charts of 7 midwife practices (23 midwives), participating in the Generation R Study. The Generation R Study is a multiethnic, population-based, prospective, cohort study that is investigating the growth, development, and health of urban children from fetal life until young adulthood. The study is conducted in Rotterdam, The Netherlands. The antenatal charts of 2093 low-risk pregnant women with an expected birthing date in 2002 through 2004 were used to determine the mean quality of antenatal care scores for 7 ethnic groups. These scores reflected the degree of adherence to the guidelines regarding 10 tests and examinations.

RESULTS

Few differences between ethnic groups were found in adherence to the guidelines that addressed the obstetric-technical quality of antenatal care. This finding applied more to nulliparous than to multiparous women. Adherence to guidelines was not always better in the antenatal care provided to native Dutch multiparous women when compared to other ethnic groups. Midwives adhered well to the guidelines regarding most tests. For all women, irrespective of ethnic background, hemoglobin was not measured as often as recommended, and this was especially the case for Moroccan, Surinamese-Creole, and Dutch-Antillean multiparous women.

DISCUSSION

The poorer adherence regarding screening for hemoglobin needs further investigation, as women with African or Mediterranean heritage are more at risk for hemoglobinopathies. However, in general, midwives adhered well to the clinical guidelines regarding most tests irrespective of the ethnic background of the pregnant women. When differences were present, these were not systematically less favorable for non-Dutch pregnant women.

The cancer pain practice index: a measure of evidence-based practice adherence for cancer pain management in older adults in hospice care

Various clinical practice guidelines addressing pain assessment and management have been available for several years that pertain, at least to some extent, to older patients with cancer. Nonetheless, systematic evaluations or methodologically sound studies of adherence to pain management practice guidelines within Medicare-certified hospice programs are lacking. As part of a larger translating-research-into-practice pain improvement study involving older patients with cancer in hospice programs, we recognized the need to create a valid and reliable tool that can facilitate critical evaluation of hospice medical records for nurse and physician adherence to pain management guidelines to create a consolidated score for comparative and quality improvement purposes. We report the process used to create this tool, named the Cancer Pain Practice Index, and a guide to its use.

Summative index: Acute pain management in older adults

One of the challenges in measuring adoption of complex evidence-based practices (EBPs) such as acute pain management is determining what constitutes adherence to an EBP guideline. Traditionally, individual process indicators extrapolated from an EBP guideline are selected as dependent measures of guideline adoption. When using multiple indicators, the challenge is determining the number of indicators that must be met to define adherence to the EBP guideline. The primary goal of the study reported herein was to develop and test a summative index (SI) of guideline adherence for acute pain management of hospitalized older adults. Steps in formulating the initial index are described as well as refinement of this metric. Techniques used included factor analysis, discriminate validity, and split-half reliability. The resulting SI is composed of 18 indicators each scored as 0 (not present) or 1 (present), with a total SI score of 0 to 18.

Outcomes of adoption: measuring evidence uptake by individuals and organizations

BACKGROUND

The translation and diffusion of findings into health care validate the potential of evidence-based innovation to improve clinical practice and affirm the benefits of society's investment in advancing science.

AIMS

This article briefly reviews key concepts in the knowledge use process, considers theoretical implications for measuring outcomes and uptake of innovation, discusses issues to consider in planning for measurement of adoption, and provides an example of confronting those challenges from a project now in progress. THEORETICAL PERSPECTIVES AND TRANSLATION OUTCOMES: Four principal conceptual frameworks related to translational science, Lewin, Rogers, Havelock, and Promoting Action on Research Implementation in Health Services (PARIHS), explicate a process that catalyzes new knowledge adoption and use by individuals and systems to solve problems. Each conceptual perspective suggests that translation is not complete until the extent and impact of use is examined and understood. Most perspectives support evaluation of impact of evidence-based practice using process measures that integrate clinician knowledge, actual performance of the practice, and patient/clinician outcomes. Based on the Agency for Healthcare Research and Quality (AHRQ) framework, additional measures might include changes in patterns of care and changes in policies, procedures, or protocols. CASE EXAMPLE: A description of process and outcome measures used in an in-progress quality improvement demonstration project, California Nursing Outcomes Coalition (CalNOC) Partners in Quality Translating Research into Practice to Reduce Patient Falls Project, is presented. IMPLICATION(S) FOR PRACTICE, RESEARCH, AND POLICY: Since the adoption of evidence-based innovation is a process that is often described along a continuum, investigators seeking to measure the impact of an evidence-based innovation must gather evidence that uptake of the innovation has occurred. The theoretical perspective and practical measurement issues of a given project will drive selection of process and outcome measures.

CONCLUSIONS

Efforts to change practice in order to enhance evidence-based patient care must integrate monitoring and evaluation of specific target outcomes of adoption as the basis for validating the impact of the change.

State of the science of translational research: from demonstration projects to intervention testing

An overview of the effort of nursing in translation research begins with studies in research utilization. Delineation of issues in the uptake of evidence using guidelines and systematic reviews is discussed next. The federal initiatives in translation research are described with a focus on the Translating Research Into Practice (TRIP) studies. The article closes with future directions.

Implications for implementation science

Implementing evidence into healthcare practice is essential to maximize the benefits of research and billions of dollars spent generating new knowledge. Implementation science is the investigation of methods, interventions (strategies), and variables to influence adoption of evidence-based healthcare practices by individuals and organizations to improve clinical and operational decision making, and includes testing the effectiveness of interventions to promote and sustain use of evidence-based healthcare practices. Estabrooks and her research team are to be applauded not only for the excellent contributions to the understanding of research utilization but also for setting forth a series of articles to stimulate thinking and comments across national boundaries and lines of inquiry. Described here are the implications for implementation science with regard to conceptual frameworks, measurement issues, and research designs, stimulated by the papers of the Estabrooks investigative team set forth in this special issue.

Prognostic implications of clinical practice guidelines among hemodialysis patients

Although the National Kidney Foundation (NKF) has published clinical practice guidelines for the management of risk factors for cardiovascular disease, these guidelines have not been tested rigorously for their effectiveness. We conducted an observational study among patients with end-stage kidney disease to examine the prognostic impact of threshold levels recommended by the NKF for blood pressure, hemoglobin, calcium-phosphate product, parathyroid hormone, low-density lipoprotein, and glycosylated hemoglobin. The study population (N = 197) was assembled from a previously completed randomized trial examining arteriovenous graft thrombosis. Cox proportional hazard analysis was used to calculate hazard ratios for the association of levels outside guideline recommended targets and death, adjusting for age, comorbidity, race, and albumin. The proportion of patients outside guideline targets ranged from 33% to 81%, and the impact of levels outside guideline targets on mortality varied substantially. Elevated calcium-phosphate product and glycosylated hemoglobin had harmful effects, with adjusted hazard ratios of 1.58 (95% CI 1.00-2.50; p = 0.050) and 2.21 (95% CI 0.99-4.97; p = 0.054), respectively. Nontarget levels for blood pressure, hemoglobin, and parathyroid hormone had little effect, with adjusted hazard ratios of 1.15 (95% CI 0.74-1.78; p = 0.542), 1.04 (95% CI 0.65-1.68; p = 0.866), and 0.90 (95% CI 0.50-1.61; p = 0.722), respectively. Elevated low-density lipoprotein had a paradoxically beneficial effect, with an adjusted hazard ratio of 0.48 (95% CI 0.23-1.00; p = 0.049). These results suggest that the prognostic impact of current threshold levels recommended by select NKF guidelines on mortality is variable. Accordingly, the development and implementation of clinical practice guidelines should be accompanied by corresponding efforts to confirm their impact on patient outcomes. Such efforts are essential for the improvement of guidelines and to inform health policy optimally.

Hypertension guidelines and their effects on the health system

INTRODUCTION

Hypertension guidelines, which have existed for many years and primarily used in the USA, Canada and Great Britain, are now becoming an issue in Germany. Strong efforts are presently underway for a German version comparable to the guidelines developed for the mentioned countries. The development of guidelines is a part of the implementation system of guidelines in Germany. It covers the mode of operation of the AWMF (work community of the scientific medical subject companies) with the clearinghouse for guidelines (CLA) and the cooperation with the centre for medical quality (ÄZQ). In the HTA report the real use of the hypertension guidelines shall be investigated for Germany from the development trends and further possibilities of use according to a medical applicability. Economic issues and an optimisation of use are also discussed.

QUESTION

THE FOLLOWING QUESTIONS SHALL BE ANSWERED IN PARTICULAR: How much are the guidelines used concerning hypertension? Can effects (or their influence) be established on the medical procedures? Are there statements available about costs and cost effectiveness? Are there recommendations for further use?

METHODOLOGY

To answer these questions, a comprehensive literature search was done. No empirical investigation was carried out. From this enquiry 206 articles were checked in detail but not all of them were available in full text. Only those publications which directly dealt with high blood pressure guidelines or articles with a direct reference to the topic have been considered in the HTA report. Publications concerning screening or methods of prevention, medical studies of the hypertension syndrome without a direct reference to guidelines and publications concerned with putting guidelines into action were excluded.

RESULTS

After an analysis of the selected literature addressing the topic of hypertension guidelines, it was evident that the use of these guidelines cannot be gathered from existing literature at the present time. One can assume from international studies with analogical reasoning that these are confessed and have a high level of acceptance in the medical community. Unfortunately the actual usage is not represented satisfactorily in the scientific literature. The effects of the guidelines on the medical procedures seem to be very strongly individual and the analyses to the compliance show at least an observable effect within the last few years. No publications could be found for the cost effectiveness of the guidelines. The actual compliance with guidelines seems to be in relation with the duration of the professional practice. It seems the shorter the professional practice takes place, the stronger the guidelines are adhered.

DISCUSSION

At present, there are only a few notes for the German health service regarding the actual effect of the hypertonus guidelines. However, the reason is not that the effect would not be possibly strong but at the methodical challenge to evaluate the sustaining effects of the application of the hypertonus guidelines. For this reason the literature is very rare regarding this topic. For Germany it can be derived by analogical reasoning from foreign studies that guidelines will facilitate a more and more essential contribution to the design of the health system. Considering that primarily younger physicians accepted guidelines mode, the further construction, update and implementation of guidelines are essential, particularly with regard to the quality assurance. Straight guidelines can express a standard of the quality of a health system as a benchmark. The existence of guidelines or the lack thereof is considered also as a quality indicator of a health system at the organisation for economic cooperation and development (OECD).

CONCLUSION

Guidelines should be evaluated - especially the hypertonus guideline. Also further development and implementation should be emphasised. Methodically oriented work to the approach is pretty recent. It is undeniable that guidelines represent a very essential and important contribution for the successful dealing with significant morbidity problems in a health system. The fact that primarily younger doctors more frequently adopt, employ and adhere to guidelines leads to the assumption that expected sustainability for practical use will increase. Furthermore intensified use of guidelines can be considered in the "mainstream" of the development of the public health system also in an international perspective. Not one single publication contradicts that a further acquirement, update and distribution of guidelines for the use of practices is necessary. The importance of the guideline is also not questioned in any article.

Implementation of an Evidence-based Feeding Protocol and Aspiration Risk Reduction Algorithm

Aspiration pneumonia is a serious complication of mechanical ventilation and enteral tube feedings. It results in increased patient mortality, increased length of hospital stay, and increased healthcare costs. This article describes an evidence-based practice approach to the creation of an enteral feeding protocol and an aspiration risk reduction algorithm. These tools were piloted in a Medical Intensive Care Unit at a Midwest tertiary care center. Chart audits show an increase in the percentage of patients who reach their goal rate for enteral feedings from 78% to 85%. Reported aspiration pneumonias decreased from an average count of 4.8 patients per month to 4.3 per month and ventilator-associated pneumonia rates decreased from 6.8 to 3.2 per 1000 patient days.

Methods in Translation Science

BACKGROUND

The gap between research findings and practice has been, and continues to be, a concern for the international community. A number of descriptive studies have elucidated barriers and facilitators of evidence-based practice in nursing. It is argued that it is now time to use findings from these studies to design and test interventions that explicitly target barriers to the use of evidence in practice rather than doing further research to describe generic barriers and facilitators to evidence-based practice. This article discusses research methods to advance our knowledge regarding the efficacy of translating research into practice (TRIP) interventions that promote and hasten adoption of evidence in practice.

APPROACH

Published research is reviewed and used as exemplars of translation studies. The benefits and challenges in studying natural experiments are addressed as are the benefits and challenges in conducting TRIP studies using experimental designs.

DISCUSSION

The article discusses factors such as sample attainment, unit of analysis, intervention characteristics, outcome measurement, and sustainability. Although there are a myriad of initiatives aimed at increasing use of evidence in practice, there is little systematic evidence of the effectiveness of these initiatives. In order to advance knowledge, translation research needs more experimental studies that test TRIP interventions through, for example, partnership models.

Evidence-based guidelines for managing mucositis

OBJECTIVES

To discuss implementation of evidence-based clinical practice guidelines for mucositis.

DATA SOURCE

Published articles, book chapters, web sources, clinical experience, unpublished manuscripts.

CONCLUSION

Nurses can implement evidence-based guidelines but must include an evaluation component to determine effect on clinical outcomes.

IMPLICATIONS FOR NURSING PRACTICE

Nurses have an integral role implementing and evaluating evidence-based practice guidelines for managing mucositis. When evidence is lacking and guidelines have gaps, nurses can play important roles in research to overcome these gaps.