The contribution of benchmarking to quality improvement in healthcare. A systematic literature review

Progressing the Death Literacy Index: the development of a revised version (DLI-R) and a short format (DLI-9)

Background

Since the development of the Death Literacy Index (DLI) in 2019 in Australia, subsequent internationally validated versions have prompted rewording and refinement of the original survey questions. Use of the DLI in the community has also resulted in requests for a short format.

Objectives

To examine and report on the psychometric properties of a revised version of the DLI-R and develop a short format DLI-9.

Design

A cross-sectional national survey was conducted for the validation of the revised DLI.

Methods

The DLI items were revised by the research team using the international literature. DLI data were collected from a representative online non-probability panel of 1202 Australian adults, based on age, gender, and geographical location. Confirmatory factor analysis (CFA) was conducted to ensure the revised version (DLI-R) was consistent with the original. To develop a short format version of the DLI (DLI-9), items were first removed based on face validity, followed by an exploratory factor analysis (EFA) and CFA. The internal reliability of the DLI-R and the DLI-9 was assessed using Cronbach's alpha. The intraclass correlation coefficient was calculated to examine the inter-rater reliability between the DLI-R and DLI-9.

Results

Twenty-four questions in the DLI were reworded for clarity. A CFA on the 29 items of this modified version of the DLI indicated a good model fit (Tucker-Lewis Index (TLI): 0.93; Comparative Fit Index (CFI): 0.93; root mean square of approximation (RMSEA): 0.06; standardized root mean residual (SRMR): 0.06), with six latent variables and an underlying latent variable "death literacy." For the DLI-9, an EFA identified a nine-item, two-factor structure model (DLI-9). A subsequent CFA in a separate sample demonstrated a good model fit for the DLI-9 (TLI: 0.92; CFI: 0.94; RMSEA: 0.089; SRMR: 0.07). Excellent inter-rater reliability (0.98) was observed between DLI-9 and DLI-R. Cronbach's alpha coefficients for DLI-R scales and subscales and the DLI-9 all exceeded 0.8, indicating high internal consistency.

Conclusion

The DLI-R and the DLI-9 were found to have acceptable psychometric properties. The development of a shorter version of the DLI provides a valid measure of overall death literacy.

Construction and validation of an instrument for event-related sterility of processed healthcare products

OBJECTIVE

To construct and validate an instrument to assess events related to maintaining the sterility of processed healthcare products.

METHODS

This methodological study developed the instrument through analysis by a panel of experts, focusing on the integrity of commonly used packaging: spunbond-meltblown-spunbond and medical-grade paper. The instrument was analyzed using the Content Validity Index and Content Validity Ratio (≥ 0.80) and modified Kappa (≥ 0.74). The instrument underwent pre-testing.

RESULTS

Six experienced professionals participated in the expert panel. After two rounds, the final version of the instrument contained five dimensions. In the pre-test, 30 nursing professionals participated, of whom 86.67% considered the instrument good, and 90% found it understandable.

CONCLUSION

The construction and validation followed literature recommendations. The instrument is available, aiding in the safe use of processed healthcare products.

The Benchmarking Exercise Programme for Older People (BEPOP): Design, Results and Recommendations from The First Wave of Data Collection

Objectives

The Benchmarking Exercise Programme for Older People (BEPOP) service improvement project seeks to determine and promote the exercise training characteristics associated with positive outcomes for resistance exercise for older people living with, or at risk of, sarcopenia or physical frailty.

Methods

Mixed-methods service improvement project. Ten UK National Health Service physiotherapist-led therapy services delivering exercise interventions for older people submitted anonymized data for up to 20 consecutive patients. A multidisciplinary expert panel generated a report and recommendations with site-specific benchmarking data and feedback. In parallel, participating physiotherapy team members were interviewed to elicit feedback on BEPOP rationale, processes and perceived value.

Results

Data from 188 patients were included, mean age 80 years (range 60-101). 115 (61%) received objective assessment of strength-based physical performance. Bodyweight exercises (173 [92%]) and resistance bands (49 [26%]) were the commonest exercise modalities. Exercises progressed predominantly through increased repetitions (163 [87%]) rather than increased load. 50 (30%) had no reassessment of outcomes; only 68 (41%) were signposted to follow-on exercise services. Staff interviews identified themes around knowledge, diagnosis, data collection and practice reflection.

Conclusion

BEPOP was feasible to deliver and generated actionable insights for service improvement via improved diagnosis, measurement and progression of resistance exercise.

Temporal Trends in Acute Adrenal Insufficiency Events in Children With Congenital Adrenal Hyperplasia During 2019-2022

Background

It is unclear whether targeted monitoring of acute adrenal insufficiency (AI) related adverse events (AE) such as sick day episodes (SDEs) and hospitalization rate in congenital adrenal hyperplasia (CAH) is associated with a change in the occurrence of these events.

Aim

Study temporal trends of AI related AE in the I-CAH Registry.

Methods

In 2022, data on the occurrence of AI-related AE in children aged <18 years with 21-hydroxylase deficiency CAH were compared to data collected in 2019.

Results

In 2022, a total of 513 children from 38 centers in 21 countries with a median of 8 children (range 1-58) per center had 2470 visits evaluated over a 3-year period (2019-2022). The median SDE per patient year in 2022 was 0 (0-2.5) compared to 0.3 (0-6) in 2019 (P = .01). Despite adjustment for age, CAH phenotype and duration of study period, a difference in SDE rate was still apparent between the 2 cohorts. Of the 38 centers in the 2022 cohort, 21 had also participated in 2019 and a reduction in SDE rate was noted in 13 (62%), an increase was noted in 3 (14%), and in 5 (24%) the rate remained the same. Of the 474 SDEs reported in the 2022 cohort, 103 (22%) led to hospitalization compared to 299 of 1099 SDEs (27%) in the 2019 cohort (P = .02).

Conclusion

The I-CAH Registry can be used for targeted monitoring of important clinical benchmarks in CAH. However, changes in reported benchmarks need careful interpretation and longer-term monitoring.

Towards collaborative care for severe and enduring Anorexia Nervosa - a mixed-method approach

BACKGROUND

Severe and Enduring Eating Disorders (SEED), in particular SEED-Anorexia Nervosa (SE-AN), may represent the most difficult disorder to treat in psychiatry. Furthermore, the lack of empirical research in this patient group, and, consequently the lack of guidelines, call for an urgent increase in research and discussion within this field. Meanwhile experts concur that effective care should be structured in a collaborative manner.

OBJECTIVE

To identify the challenges in providing care to patients with SE-AN in the Dutch healthcare context, and propose a collaborative care treatment model to address these issues.

METHODS

A pragmatic mixed-method approach was used, structured as follows: (1) Identifying perceived barriers and treatment needs from the viewpoint of both patients and eating disorder healthcare professionals through an evaluation questionnaire; (2) Investigating current treatment practices for SEED/SE-AN via benchmarking; (3) Gaining insight into the optimal structure and content of care by interviewing network partners and experts-by-experience. Based on these findings, and drawing from literature on severe and enduring disorders, a treatment model for SE-AN was proposed and implemented.

RESULTS

The key challenges identified included a lack of knowledge about eating disorders among network partners, treatment ambivalence among patients and poor collaboration between professionals. The proposed model enhances self-management and collaborative relationships with healthcare providers, offers user-friendly and practical guidance, and aims at stabilization, reducing relapses, deterioration, and readmissions, thereby being cost-effective. Importantly, the model operates across levels of care (primary, secondary, tertiary).

CONCLUSION

This study, describing a collaborative care program for SE-AN, developed and implemented in a highly specialized treatment center for eating disorders, sets the stage for further explanatory/efficacy research to build on the findings in this study, with the following aims: addressing the critical gap in care for SEED/SE-AN, improving better healthcare organization, reducing relapse rates, and lowering costs for this often overlooked patient group.

Creating a postgraduate syllabus for a team care diploma examination: a Delphi study

Objectives

There is no agreed standard assessment of the minimum knowledge and skills that are required to provide healthcare to participants in individual or team sports. This study aims to develop a syllabus for the Faculty of Sport and Exercise Medicine (FSEM) Team Care Diploma examination. This will provide a recognised assessment of the minimum required skills and knowledge for healthcare professionals providing care in an individual and team sport environment.

Methods

A modified Delphi approach was used. A syllabus was developed by a purposeful selection of members of the FSEM, all of whom have significant team care experience. This was then reviewed by the Delphi expert panel who were team care practitioners with at least 5 years of experience. A two-round Delphi approach was used to develop a consensus.

Results

The expert panel consisted of 50 individuals, with 46 (92%) completing both rounds. Of the 447 learning objectives (LOs) proposed; 430 (96%) were accepted outright, 17 (4%) were rejected and four new LOs were introduced based on expert panel feedback. The final syllabus contained 434 LOs across 6 modules (clinical governance, safe and effective practice, interdisciplinary teamwork, specific athlete groups, specific health conditions and duties of the medical team).

Conclusion

This standardised syllabus will be used as the basis for the new FSEM Team Care Diploma examination which will aim to provide world-leading standardised assessment of the minimum skills and knowledge required for healthcare professionals across the multidisciplinary team providing care in individual and team sport.

Exploring subjective quality-of-life indicators in long-term care facilities: a mixed-methods research protocol

INTRODUCTION

Improving quality of life has become a priority in the long-term care (LTC) sector internationally. With development and implementation guidance, standardised quality-of-life monitoring tools based on valid, self-report surveys could be used more effectively to benefit LTC residents, families and organisations. This research will explore the potential for subjective quality-of-life indicators in the interRAI Self-Reported Quality of Life Survey for Long-Term Care Facilities (QoL-LTCF).

METHODS AND ANALYSIS

Guided by the Medical Research Council Framework, this research will entail a (1) modified Delphi study, (2) feasibility study and (3) realist synthesis. In study 1, we will evaluate the importance of statements and scales in the QoL-LTCF by administering Delphi surveys and focus groups to purposively recruited resident and family advisors, researchers, and LTC clinicians, staff, and leadership from international quality improvement organisations. In study 2, we will critically examine the feasibility and implications of risk-adjusting subjective quality-of-life indicators. Specifically, we will collect expert stakeholder perspectives with interviews and apply a risk-adjustment methodology to QoL-LTCF data. In study 3, we will iteratively review and synthesise literature, and consult with expert stakeholders to explore the implementation of quality-of-life indicators.

ETHICS AND DISSEMINATION

This study has received approval through a University of Waterloo Research Ethics Board and the Social and Societal Ethics Committee of KU Leuven. We will disseminate our findings in conferences, journal article publications and presentations for a variety of stakeholders.

6,126 hepatectomies in 2022: current trend of outcome in Italy

PURPOSE

Whether hospital volume affects outcome of patients undergoing hepatobiliary surgery, and whether the centralization of such procedures is justified remains to be investigated. The aim of this study was to analyze the outcome of liver surgery in Italy in relationship of hospital volume.

METHODS

This is a nationwide retrospective observational study conducted on data collected by the National Italian Registry "Piano Nazionale Esiti" (PNE) 2023 that included all liver procedures performed in 2022. Outcome measure were case volume and 30-day mortality. Hospitals were classified as very high-volume (H-Vol), intermediate-volume (I-Vol), low-volume (L-Vol) and very low-volume (VL-VoL). A review on centralization process and outcome measures was added.

RESULTS

6,126 liver resections for liver tumors were performed in 327 hospitals in 2022. The 30-day mortality was 2.2%. There were 14 H-Vol, 19 I-Vol, 31 L-Vol and 263 VL-Vol hospitals with 30-day mortality of 1.7%, 2.2%, 2.6% and 3.6% respectively (P < 0.001); 220 centers (83%) performed less than 10 resections, and 78 (29%) centers only 1 resection in 2022. By considering the geographical macro-areas, the median count of liver resection performed in northern Italy exceeded those in central and southern Italy (57% vs. 23% vs. 20%, respectively).

CONCLUSIONS

High-volume has been confirmed to be associated to better outcome after hepatobiliary surgical procedures. Further studies are required to detail the factors associated with mortality. The centralization process should be redesigned and oversight.

Characterising trends in the initiation, timing, and completion of recommended summary plan for emergency care and treatment (ReSPECT) plans: Retrospective analysis of routine data from a large UK hospital trust

AIM

To assess patient socio-demographic and disease characteristics associated with the initiation, timing, and completion of emergency care and treatment planning in a large UK-based hospital trust.

METHODS

Secondary retrospective analysis of data across 32 months extracted from digitally stored Recommended Summary Plan for Emergency Care and Treatment (ReSPECT) plans within the electronic health record system of an acute hospital trust in England, UK.

RESULTS

Data analysed from ReSPECT plans (n = 23,729), indicate an increase in the proportion of admissions having a plan created from 4.2% in January 2019 to 6.9% in August 2021 (mean = 8.1%). Forms were completed a median of 41 days before death (a median of 58 days for patients with capacity, and 21 days for patients without capacity). Do not attempt cardiopulmonary resuscitation was more likely to be recorded for patients lacking capacity, with increasing age (notably for patients aged over 74 years), being female and the presence of multiple disease groups. 'Do not attempt cardiopulmonary resuscitation' was less likely to be recorded for patients having ethnicity recorded as Asian or Asian British and Black or Black British compared to White. Having a preferred place of death recorded as 'hospital' led to a five-fold increase in the likelihood of dying in hospital.

CONCLUSION

Variation in the initiation, timing, and completion of ReSPECT plans was identified by applying an evaluation framework. Digital storage of ReSPECT plan data presents opportunities for assessing trends and completion of the ReSPECT planning process and benchmarking across sites. Further research is required to monitor and understand any inequity in the implementation of the ReSPECT process in routine care.

Risk-adjusted trend in national inpatient fall rates observed from 2011 to 2019 in acute care hospitals in Switzerland: a repeated multicentre cross-sectional study

OBJECTIVES

This study aimed to investigate whether a significant trend regarding inpatient falls in Swiss acute care hospitals between 2011 and 2019 could be confirmed on a national level, and whether the trend persists after risk adjustment for patient-related fall risk factors.

DESIGN

A secondary data analysis was conducted based on annual multicentre cross-sectional studies carried out between 2011 and 2019.

SETTING

All Swiss acute care hospitals were obliged to participate in the surveys. Except for emergency departments, outpatient wards and recovery rooms, all wards were included.

PARTICIPANTS

All inpatients aged 18 or older who had given their informed consent and whose data were complete and available were included.

OUTCOME MEASURE

Whether a patient had fallen in the hospital was retrospectively determined on the survey day by asking patients the following question: Have you fallen in this institution in the last 30 days?

RESULTS

Based on data from 110 892 patients from 222 Swiss hospitals, a national inpatient fall rate of 3.7% was determined over the 9 survey years. A significant linear decreasing trend (p=0.004) was observed using the Cochran-Armitage trend test. After adjusting for patient-related fall risk factors in a two-level random intercept logistic regression model, a significant non-linear decreasing trend was found at the national level.

CONCLUSIONS

A significant decrease in fall rates in Swiss hospitals, indicating an improvement in the quality of care provided, could be confirmed both descriptively and after risk adjustment. However, the non-linear trend, that is, an initial decrease in inpatient falls that flattens out over time, also indicates a possible future increase in fall rates. Monitoring of falls in hospitals should be maintained at the national level. Risk adjustment accounts for the observed increase in patient-related fall risk factors in hospitals, thus promoting a fairer comparison of the quality of care provided over time.

Exploring Risk Factors for Predicting 30-Day Postoperative Morbidity in Musculoskeletal Tumor Surgery

Background: This study investigates the risk factors associated with postoperative complications in musculoskeletal tumor surgeries and evaluates the impact of benchmarking in enhancing surgical outcomes. Methods: Conducted at a tertiary referral center, this retrospective analysis included 196 patients who underwent surgeries for various musculoskeletal tumors, ranging from soft tissue to bone sarcomas. Patient and tumor characteristics, along with surgical interventions and outcomes, were comprehensively assessed using the Charlson Comorbidity Index and the Clavien-Dindo classification. Results: Key findings indicate that surgical reconstruction, ASA 3 status, bone tumor presence, and the need for multiple erythrocyte transfusions significantly increase postoperative morbidity. Notably, no significant correlation was found between the Charlson Comorbidity Index scores and the occurrence or severity of complications, challenging the utility of this index in predicting short-term surgical outcomes. Conclusions: This study highlights the importance of tailored surgical approaches and emphasizes rigorous preoperative assessments to mitigate risks and enhance patient care. Despite its insights, limitations include its retrospective nature and single-center scope, suggesting a need for broader, multicenter studies to generalize findings. Overall, our results underscore the necessity of integrating clinical assessments with benchmarking data to optimize outcomes in the complex field of musculoskeletal tumor surgery.

Sustainable Approach to Justice, Equity, Diversity, and Inclusion Through Better Quality Measurement

The US health care industry has broadly adopted performance and quality measures that are extracted from electronic health records and connected to payment incentives that hope to improve declining life expectancy and health status and reduce costs. While the development of a quality measurement infrastructure based on electronic health record data was an important first step in addressing US health outcomes, these metrics, reflecting the average performance across diverse populations, do not adequately adjust for population demographic differences, social determinants of health, or ecosystem vulnerability. Like society as a whole, health care must confront the powerful impact that social determinants of health, race, ethnicity, and other demographic variations have on key health care performance indicators and quality metrics. Tools that are currently available to capture and report the health status of Americans lack the granularity, complexity, and standardization needed to improve health and address disparities at the local level. In this article, we discuss the current and future state of electronic clinical quality measures through a lens of equity.

Reporting interhospital neonatal intensive care transport: international five-step Delphi-based template

OBJECTIVE

To develop a general and internationally applicable template of data variables for reporting interhospital neonatal intensive care transports.

DESIGN

A five-step Delphi method.

SETTING

A group of experts was guided through a formal consensus process using email.

SUBJECTS

12 experts in neonatal intensive care transports from Canada, Denmark, Norway, the UK and the USA. Four women and eight men. The experts were neonatologists, anaesthesiologists, intensive care nurse, anaesthetic nurse, medical leaders, researchers and a parent representative.

MAIN OUTCOME MEASURES

37 data variables were included in the final template.

RESULTS

Consensus was achieved on a template of 37 data variables with definitions. 30 variables to be registered for each transport and 7 for annual registration of the system of the transport service. 11 data variables under the category structure, 20 under process and 6 under outcome.

CONCLUSIONS

We developed a template with a set of data variables to be registered for neonatal intensive care transports. To register the same data will enable larger datasets and comparing services.

Centre-level variation in the survival of patients receiving haemodialysis in India: findings from a nationwide private haemodialysis network

Background

There are no large studies examining survival in patients receiving haemodialysis in India or considering centre-level effects on survival. We measured survival variation between dialysis centres across India and evaluated the extent to which differences are explained by measured centre characteristics.

Methods

This is a multilevel analysis of patient survival in centres of the NephroPlus dialysis network consisting of 193 centres across India. Patients receiving haemodialysis at a centre for ≥90 days between April 2014 and June 2019 were included, with analyses restricted to centres with ≥10 such patients. The primary outcome was all-cause mortality, measured from 90 days after joining a centre. Proportional hazards models with shared frailty were used to model centre- and patient-level effects on survival.

Findings

Amongst 23,601 patients (median age 53 years; 29% female), the unadjusted centre-specific 180-day Kaplan-Meier survival estimates ranged between 55% (95% confidence interval [CI] 38-80%) and 100%, with a median of 88% (interquartile interval 83%-92%). After accounting for multilevel factors, estimated 180-day survival ranged between 83% (73-89%) and 97% (95-98%), with 90% 180-day survival in the average centre. The mortality rate in patients attending rural centres was 32% (Hazard Ratio 1.32; 95% CI 1.06-1.65) higher than those at urban centres in adjusted analyses. Multiple patient characteristics were associated with mortality.

Interpretation

This is the first national benchmark for survival amongst dialysis patients in India. Centre- and patient-level characteristics are associated with survival but there remains unexplained variation between centres. As India continues to widen dialysis access, ongoing quality improvement programs will be an important part of ensuring that patients experience the best possible outcomes at the point of care.

Funding

This project received no external funding.

'Naming and faming' maternity care providers: A mixed-methods study

BACKGROUND

Positive benchmarking can serve as a catalyst for maternity care improvement.

AIM

To retrospectively benchmark Flemish maternity care providers' qualities, based on women's positive care experiences, and to explore which attributes of the different care providers contribute to these experiences.

METHODS

A sequential, two-phased mixed-methods study benchmarking the qualities of the community midwife, the hospital midwife, and the obstetrician. An online questionnaire was used to collect the data among pregnant and postpartum women, who rated their care experiences with the various care providers using the Net Promoter Score. Non-parametric and post hoc tests established the differences between types of clinicians and between antenatal, intrapartum, and postpartum Net Promoter Score mean scores. Content analysis was used to construct a final pool of keywords representing attributes of care professionals, accumulated from the promoters' free text responses. Ranks were assigned to each keyword based on its frequency.

FINDINGS

A total of 2385 Net Promoter Scale scores and 1856 free-text responses of 1587 responders were included. The community midwife received the overall highest NPS scores (p < .001). The promoters (n = 1015) assigned community midwives the highest NPS scores (9.67), followed by obstetricians (9.57) and hospital-based midwives (9.51). The distinct benchmarking attributes of community midwives were availability (p < .001), supportiveness (p = .04) and personalised care (p < .001). Being honest (p < .001), empathic (p < .001) and inexhaustible (p = .04) benchmarked hospital midwives. Calmness (p < .001), a no-nonsense approach (p < .001), being humane (p = .01) and comforting (p = .02) benchmarked obstetricians.

DISCUSSION/CONCLUSION

The findings indicate that all care providers are highly valued, but community midwives are ranked the highest. The distinct differences between the care professionals can serve as exemplary performance for professional development and shape the profiles of maternity care professionals.

Electronic health record time-tracking provides real-time data to measure and benchmark dietitian productivity

BACKGROUND

Traditional methods for benchmarking dietitian productivity are time-consuming and fail to accurately measure the total time spent providing nutrition care. An electronic health record (EHR)-based tool that allows for daily tracking of both face-to-face and patient care coordination time for dietitians was created. We assessed whether it provided consistent, continuous measurement of time and productivity.

METHODS

This tool was created in an independent paediatric academic healthcare system in the USA. Time spent by dietitians in face-to-face settings and care coordination were tracked. Changes in time spent between the years 2013-2016 versus 2018-2019 were also analysed.

RESULTS

The outpatient dietitian spent a mean total of 66.4 min per patient (37.8 ± 6.0 min in face-to-face care and 28.6 ± 5.2 min in care coordination). The total times and fractions spent on face-to-face and care coordination time varied by specialty. Comparison of the two periods of time revealed 75% more productivity on average of dietitians in different outpatient settings after including care coordination tracking. In addition, dietitians were more likely to document time spent in 5-min increments after the institution of this methodology as opposed to 15-min increments.

CONCLUSIONS

An EHR-based tool that facilitates the documentation of both face-to-face time and patient care coordination time is feasible and enables consistent, continuous measurement of time and productivity. The real-time data from this tool can be used to support adequate dietitian staffing and be used to create a multicentre database to measure the actual time dietitians need to provide care and generate consistent staffing benchmarks.

Pediatric endoscopy: how can we improve patient outcomes and ensure best practices?

INTRODUCTION

Strategies to promote high-quality endoscopy in children require consensus around pediatric-specific quality standards and indicators. Using a rigorous guideline development process, the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) was developed to support continuous quality improvement efforts within and across pediatric endoscopy services.

AREAS COVERED

This review presents a framework, informed by the PEnQuIN guidelines, for assessing endoscopist competence, granting procedural privileges, audit and feedback, and for skill remediation, when required. As is critical for promoting quality, PEnQuIN indicators can be benchmarked at the individual endoscopist, endoscopy facility, and endoscopy community levels. Furthermore, efforts to incorporate technologies, including electronic medical records and artificial intelligence, into endoscopic quality improvement processes can aid in creation of large-scale networks to facilitate comparison and standardization of quality indicator reporting across sites.

EXPERT OPINION

PEnQuIN quality standards and indicators provide a framework for continuous quality improvement in pediatric endoscopy, benefiting individual endoscopists, endoscopy facilities, and the broader endoscopy community. Routine and reliable measurement of data, facilitated by technology, is required to identify and drive improvements in care. Engaging all stakeholders in endoscopy quality improvement processes is crucial to enhancing patient outcomes and establishing best practices for safe, efficient, and effective pediatric endoscopic care.

The Measure of an Outcome: Comparing Norming and Stacking to Benchmark the Effectiveness of Brain Injury Rehabilitation Services

Practitioners have a clinical, ethical, academic, and economic responsibility to dispassionately consider how effective their services are. Approaches to measure how "good" or "bad" healthcare is include clinical audit, satisfaction surveys, and routine outcome measurement. However, the process of comparing the clinical outcomes of a specific service against the 'best' services in the same specialism, also known as benchmarking, remains challenging, and it is unclear how it affects quality improvement. This paper piloted and compared two different approaches to benchmarking to assess clinical outcomes in neurorehabilitation. Norming involved comparing routine measures of clinical outcome with external validators. Stacking involved pooling and comparing internal data across several years. The analyses of routine clinical outcome data from 167 patients revealed significant differences in the patient characteristics of those admitted to the same service provider over time, but no differences in outcomes achieved when comparing with historical data or with external reference data. These findings illustrate the potential advantages and limitations of using stacking and norming to benchmark clinical outcomes, and how the results from each approach might be used to evaluate service effectiveness and inform quality improvement within the field of brain injury rehabilitation.

Real-life benchmarking bladder cancer care: A population-based study

INTRODUCTION

Radical cystectomy (RC) is a complex oncological surgical procedure and population studies of routine surgical care have suggested suboptimal results compared to high-volume centers of excellence. A previous Canadian bladder cancer quality-of-care consensus led to adoption of multiple key quality-of-care indicators, with associated benchmarks created using available evidence and expert opinion to inform and measure future performance. Herein, we report real-life benchmark performance for the management of muscle-invasive bladder cancer (MIBC) relative to expert opinion guidance.

METHODS

This is a population-based, retrospective, cohort study that used the Ontario Cancer Registry (OCR) to identify all incident patients who underwent RC from 2009-2013. Electronic records of treatment from 1573 patients were linked to OCR; pathology records were obtained for all cases and reviewed by a team of trained data abstractors. The primary objective was to describe benchmarks for identified indicators, first as median values obtained across hospitals or providers, as well as a "pared-mean" approach to identify a benchmark population of "top performance," as defined as the best outcome accomplished for at least 10% of the population.

RESULTS

Overall, performance in Ontario across all indicators fell short of expert opinion-determined benchmarks. Annual surgical volume by each surgeon performing a RC (benchmark >6, percent of institutions meeting benchmark=20%), percent of patients with MIBC referred preoperatively to medical oncology (MO; benchmark>90%, percent of institutions meeting benchmark=2%) and radiation oncology (RO; benchmark>50%, percent of institutions meeting benchmark=0%), time to cystectomy within six weeks of transurethral resection of bladder tumor (TURBT) in patients without neoadjuvant chemotherapy (benchmark <6 weeks, percent of institutions meeting benchmark=0%), percent of patients with adequate lymph node dissection (defined as >14 nodes, benchmark>85%, percent of institutions meeting benchmark=0%), percent of patients with positive margins post-RC (benchmark <10%, percent of institutions meeting benchmark=46%), and 90-day mortality (benchmark<5%, percent of institutions meeting benchmark=37%) fell considerably short. Simply evaluating benchmarks across the province as median performance significantly underestimated benchmarks that were possible by top-performing hospitals.

CONCLUSIONS

Performance through most bladder cancer quality-of-care indicators fall short of benchmarks proposed by expert opinion. Different methodologies, such as a paredmean approach of top performers, may provide more realistic benchmarking.

Physical activity service provision in hospice care: A national mixed-methods study

BACKGROUND

Physical activity (PA) interventions help people with advanced incurable diseases to manage symptoms and improve their quality of life. However, little is known about the extent to which PA is currently delivered in hospice care in England.

OBJECTIVES

To determine the extent of and intervention features of PA service provision in hospice care in England alongside barriers and facilitators to their delivery.

METHODS

An embedded mixed-methods design using (1) a nationwide online survey of 70 adult hospices in England and (2) focus groups and individual interviews with health professionals from 18 hospices. Analysis of the data involved applying descriptive statistics to the numeric items and thematic analysis to the open-ended questions. Quantitative and qualitative data were collected and analyzed separately.

RESULTS

The majority of responding hospices (n = 47/70, 67%) promoted PA in routine care. Sessions were most often delivered by a physiotherapist (n = 40/47, 85%) using a personalized approach (n = 41/47, 87%) and included resistance/thera bands, Tai Chi/Chi Qong, circuit exercises, and yoga. The following qualitative findings were revealed: (1) variation among hospices in their capacity to deliver PA, (2) a desire to embed a hospice culture of PA, and (3) a need for an organizational commitment to PA service provision.

SIGNIFICANCE OF RESULTS

While many hospices in England deliver PA, there is considerable variation in its delivery across sites. Funding and policy action may be needed to support hospices to initiate or scale up services and address inequity in access to high-quality interventions.

Benchmarking: a novel measuring tool for outcome comparisons in surgery

INTRODUCTION

Benchmarking, a novel measuring tool for outcome comparisons, is a recent concept in surgery. The objectives of this review are to examine the concept, definition, and evolution of benchmarking and its application in surgery.

METHODS

The literature about benchmarking was reviewed through an ever-narrowing search strategy, commencing from the concept, definition, and evolution of benchmarking to the application of benchmarking and experiences of benchmarking in surgery. PubMed, Web of Science, Embase, and Science Direct databases were searched until 20 September 2022, in the English language according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines.

RESULTS

In the first phase of the literature search, the development of benchmarking was identified. The definitions of benchmarking evolved from a surveying term to a novel quality-improvement tool to assess the best achievable results in surgery. In the second phase, a total of 23 studies were identified about benchmarking in surgery, including esophagectomy, hepatic surgery, pancreatic surgery, rectum resection, and bariatric surgery. All studies were multicenter analyses from national, international, or global expert centers. Most studies (87.0%) adopted the definition that benchmark was the 75th percentile of the median values of centers. Performance metrics to define benchmarks were clinically relevant intraoperative and postoperative outcome indicators.

CONCLUSION

Benchmarking in surgery is a novel quality-improvement tool to define and measure the best achievable results, establishing a meaningful reference to evaluate surgical performance.