Should interventions to reduce variation in care quality target doctors or hospitals?

The Role of Medical Student Quality Improvement Projects in Health Care: A Scoping Review

INTRODUCTION

Across Australia, there is considerable variation in the quality of health care and the risk of associated complications. Hence, many patients are receiving poor-quality care. This may be exacerbated in rural and remote areas where the availability of health care is relatively limited. Addressing this requires a multipronged approach that supports the workforce. Involving medical practitioners in medical students' quality improvement (QI) scholarly projects may be one strategy to assist with bridging this gap. The aim of this review was to synthesise the evidence relating to medical students' compulsory curriculum-based quality improvement projects and associated practice outcomes.

METHOD

Scoping review of peer-reviewed literature (January 2000-June 2024).

RESULTS

Of the 239 articles, six empirical studies from Australia, New Zealand, the UK and the USA were included. Half of these were based in community settings and the rest in hospitals. Only one was in a rural setting. The time allocated to projects was between 5 and 12 weeks. Five of the six studies reported that student project recommendations had been implemented. One study reported that the process enhanced doctors' adherence to best practice guidelines.

DISCUSSION

Much of the research about the outcomes of medical student curriculum-based projects focuses on research outputs. These relatively short student QI projects are one strategy to improve evidence-based practice while upskilling clinicians. Further work is needed to examine their impact, particularly in rural areas.

CONCLUSION

Integrating medical students into 'real-world' QI health service projects can enhance the quality of health care whilst building the skills of the medical workforce.

Adherence to national trauma triage criteria in Norway: a cross-sectional study

BACKGROUND

Norwegian hospitals employed individual trauma triage criteria until 2015 when nationwide criteria were implemented. There is a lack of empirical evidence regarding adherence to Norwegian national criteria for activation of the trauma team (NTrC) and the decision-making processes regarding trauma team activation (TTA) within Norwegian trauma hospitals. The objectives of this study were to investigate institutional adherence to the NTrC and to investigate similarities and differences in the decision-making process leading to TTA in Norwegian trauma hospitals.

METHODS

A digital semi-structured questionnaire regarding adherence to criteria, TTA decision-making and criteria documentation was distributed to all Norwegian trauma hospitals (n = 38) in the spring of 2022. Contact details of trauma coordinators and registrars were provided by the Norwegian Trauma Registry secretariat. Follow-up telephone interviews were conducted at the investigator's discretion in cases of non-respondents or need to clarify answers.

RESULTS

Thirty-eight trauma hospitals were invited to answer the survey, where 35 hospitals responded (92%), making 35 the denominator of the results. Thirty-four (97.1%) hospitals stated that they followed NTrC. Thirty-three (94.3%) of the responding hospitals provided documentation of their criteria in use, of which twenty-eight (80%) of responding hospitals adhered to the NTrC. Three (8.6%) hospitals employed a tiered TTA approach with different sized teams. In addition four hospitals (11.4%) used specialized teams to meet the needs of defined patient groups (e.g. geriatric patients, traumatic brain injury). Twenty-one (60%) of the responding hospitals had written guidelines on who could perform TTA and in 18 hospitals (51.4%) TTA could be performed by pre-hospital personnel. Twenty-three (65.7%) of the hospitals documented which criteria that were used for TTA.

CONCLUSION

There is good adherence to the national criteria for activation of the trauma team among Norwegian trauma hospitals after implementation of national guidelines. Individual hospitals argue the use of certain local criteria and trauma team activation decision-making processes to increase their precision in specific patient populations and demographics. Further steps should be done to reduce the variation in TTA decision-making processes among hospitals and improve documentation quality.

Surgery for phaeochromocytomas and paragangliomas: Current practice in the United Kingdom

INTRODUCTION

There is wide variability in the perioperative management of phaeochromocytoma and paraganglioma (PPGL) in different centres. This study aimed to summarise the management of PPGLs as reported in the United Kingdom Registry for Endocrine and Thyroid Surgery (UKRETS) database and to determine current perioperative management of PPGLs by surveying UK clinicians.

METHODS

Data recorded on UKRETS from 2005 to 2021 were subjected to descriptive analyses. British Association of Endocrine and Thyroid Surgeons members were invited to participate in an open survey relating to the perioperative management of patients with PPGLs.

RESULTS

A total of 2,007 operations for PPGL from 49 participating centres were included. The median annual workload in each centre was four cases. Operations were performed predominantly laparoscopically (69%). The median length of stay (4 days) was the same in groups of surgeons stratified by volume. The survey had 29 respondents from 22 centres across the UK, and a formal protocol for perioperative management exists in 48% of the centres. Phenoxybenzamine (72%) was preferred for alpha-blockade. The practice of admitting patients for optimisation from 1 to 7 days before the day of surgery was common (62%). Central venous pressure and blood glucose monitoring were mentioned as routine intraoperative adjuncts by 72% of the responders.

CONCLUSIONS

There is significant variation in the workload and perioperative management of PPGLs in the UK. This is potentially detrimental to patient outcomes and a consensus document might be beneficial to harmonise practice across the UK.

Between-hospital variation in indicators of quality of care: a systematic review

BACKGROUND

Efforts to mitigate unwarranted variation in the quality of care require insight into the 'level' (eg, patient, physician, ward, hospital) at which observed variation exists. This systematic literature review aims to synthesise the results of studies that quantify the extent to which hospitals contribute to variation in quality indicator scores.

METHODS

Embase, Medline, Web of Science, Cochrane and Google Scholar were systematically searched from 2010 to November 2023. We included studies that reported a measure of between-hospital variation in quality indicator scores relative to total variation, typically expressed as a variance partition coefficient (VPC). The results were analysed by disease category and quality indicator type.

RESULTS

In total, 8373 studies were reviewed, of which 44 met the inclusion criteria. Casemix adjusted variation was studied for multiple disease categories using 144 indicators, divided over 5 types: intermediate clinical outcomes (n=81), final clinical outcomes (n=35), processes (n=10), patient-reported experiences (n=15) and patient-reported outcomes (n=3). In addition to an analysis of between-hospital variation, eight studies also reported physician-level variation (n=54 estimates). In general, variation that could be attributed to hospitals was limited (median VPC=3%, IQR=1%-9%). Between-hospital variation was highest for process indicators (17.4%, 10.8%-33.5%) and lowest for final clinical outcomes (1.4%, 0.6%-4.2%) and patient-reported outcomes (1.0%, 0.9%-1.5%). No clear pattern could be identified in the degree of between-hospital variation by disease category. Furthermore, the studies exhibited limited attention to the reliability of observed differences in indicator scores.

CONCLUSION

Hospital-level variation in quality indicator scores is generally small relative to residual variation. However, meaningful variation between hospitals does exist for multiple indicators, especially for care processes which can be directly influenced by hospital policy. Quality improvement strategies are likely to generate more impact if preceded by level-specific and indicator-specific analyses of variation, and when absolute variation is also considered.

PROSPERO REGISTRATION NUMBER

CRD42022315850.

Providers, peers and patients. How do physicians' practice environments affect patient outcomes?

We study how physicians' practice environments affect their treatment decisions and quality of care. Using clinical registry data from Sweden, we compare stent choices of cardiologists moving across hospitals over time. To disentangle changes in practice styles attributable to hospital- and peer group-specific factors, we exploit quasi-random variation on cardiologists working together on the same days. We find that migrating cardiologists' stent choices rapidly adapt to their new practice environment after relocation and are equally driven by the hospital and peer environments. In contrast, while decision errors increase, treatment costs and adverse clinical events remain largely unchanged despite the altered practice styles.

Between-Hospital and Between-Physician Variation in Outcomes and Costs in High- and Low-Complex Surgery: A Nationwide Multilevel Analysis

OBJECTIVES

Clinicians and policy makers are increasingly exploring strategies to reduce unwarranted variation in outcomes and costs. Adequately accounting for case mix and better insight into the levels at which variation exists is crucial for such strategies. This nationwide study investigates variation in surgical outcomes and costs at the level of hospitals and individual physicians and evaluates whether these can be reliably compared on performance.

METHODS

Variation was analyzed using 92 330 patient records collected from 62 Dutch hospitals who underwent surgery for colorectal cancer (n = 6640), urinary bladder cancer (n = 14 030), myocardial infarction (n = 31 870), or knee osteoarthritis (n = 39 790) in the period 2018 to 2019. Multilevel regression modeling with and without case-mix adjustment was used to partition variation in between-hospital and between-physician components for in-hospital mortality, intensive care unit admission, length of stay, 30-day readmission, 30-day reintervention, and in-hospital costs. Reliability was calculated for each treatment-outcome combination at both levels.

RESULTS

Across outcomes, hospital-level variation relative to total variation ranged between ≤ 1% and 15%, and given the high caseloads, this typically yielded high reliability (> 0.9). In contrast, physician-level variation components were typically ≤ 1%, with limited opportunities to make reliable comparisons. The impact of case-mix adjustment was limited, but nonnegligible.

CONCLUSIONS

It is not typically possible to make reliable comparisons among physicians due to limited partitioned variation and low caseloads. Nevertheless, for hospitals, the opposite often holds. Although variation-reduction efforts directed at hospitals are thus more likely to be successful, this should be approached cautiously, partly because level-specific variation and the impact of case mix vary considerably across treatments and outcomes.

Multiple hospital organisational factors are associated with adverse patient outcomes post-hip fracture in England and Wales: the REDUCE record-linkage cohort study

OBJECTIVES

Despite established standards and guidelines, substantial variation remains in the delivery of hip fracture care across the United Kingdom. We aimed to determine which hospital-level organisational factors predict adverse patient outcomes in the months following hip fracture.

METHODS

We examined a national record-linkage cohort of 178,757 patients aged ≥60 years who sustained a hip fracture in England and Wales in 2016-19. Patient-level hospital admissions datasets, National Hip Fracture Database and mortality data were linked to metrics from 18 hospital-level organisational-level audits and reports. Multilevel models identified organisational factors, independent of patient case-mix, associated with three patient outcomes: length of hospital stay (LOS), 30-day all-cause mortality and emergency 30-day readmission.

RESULTS

Across hospitals mean LOS ranged from 12 to 41.9 days, mean 30-day mortality from 3.7 to 10.4% and mean readmission rates from 3.7 to 30.3%, overall means were 21.4 days, 7.3% and 15.3%, respectively. In all, 22 organisational factors were independently associated with LOS; e.g. a hospital's ability to mobilise >90% of patients promptly after surgery predicted a 2-day shorter LOS (95% confidence interval [CI]: 1.2-2.6). Ten organisational factors were independently associated with 30-day mortality; e.g. discussion of patient experience feedback at clinical governance meetings and provision of prompt surgery to >80% of patients were each associated with 10% lower mortality (95%CI: 5-15%). Nine organisational factors were independently associated with readmissions; e.g. readmissions were 17% lower if hospitals reported how soon community therapy would start after discharge (95%CI: 9-24%).

CONCLUSIONS

Receipt of hip fracture care should be reliable and equitable across the country. We have identified multiple, potentially modifiable, organisational factors associated with important patient outcomes following hip fracture.

Is There a Doctors' Effect on Patients' Physical Health, Beyond the Intervention and All Known Factors? A Systematic Review

Purpose

Despite billions of doctor visits worldwide each year, little is known on whether doctors themselves affect patients' physical health after accounting for intervention and confounders such as patients' and doctors' data, hospital effects, nor how strong that doctors' effect is. Knowledge of surgeons' and psychotherapists' effects exists, but not for 102 other medical specialties notwithstanding the importance of such knowledge.

Methods

Eligibility Criteria: Randomized controlled trials (RCTs), case-control, and cohort studies including medical doctors except surgeons for any intervention, reporting the proportion of variance in patients' outcomes owing to the doctors (random effects), or the fixed effects of grading doctors by outcomes, after multivariate adjustment. Exclusions: studies of <15 doctors or solely reporting doctors' effects for known variables.

Sources

Medline, Embase, PsycINFO, inception to June 2020. Manual search for papers referring/referred to by resulting studies.

Risk of Bias

Using Newcastle-Ottawa scale.

Results

Despite all medical interventions bar surgery being eligible, only thirty cohort papers were found, covering 36,239 doctors, with 10 specialties, 21 interventions, 60 outcomes (17 unique). Studies reported doctors' effects by grading doctors from best to worst, or by diversely calculating the doctor-attributed percentage of patients' outcome variation, ie the intra-class correlation coefficient (ICC). Sixteen studies presented fixed effects, 18 random effects, and 3 another approach. No RCTs found. Thirteen studies reported exceptionally good and/or poor performers with confidence intervals wholly outside the average performance. ICC range 0 to 33%, mean 3.9%. Highly diverse reporting, meta-analysis therefore not applicable.

Conclusion

Doctors, on their own, can affect patients' physical health for many interventions and outcomes. Effects range from negligible to substantial, even after accounting for all known variables. Many published cohorts may reveal valuable information by reanalyzing their data for doctors' effects. Positive and negative doctor outliers appear regularly. Therefore, it can matter which doctor is chosen.

The Doctors' Effect on Patients' Physical Health Outcomes Beyond the Intervention: A Methodological Review

Background

Previous research suggests that when a treatment is delivered, patients' outcomes may vary systematically by medical practitioner.

Objective

To conduct a methodological review of studies reporting on the effect of doctors on patients' physical health outcomes and to provide recommendations on how this effect could be measured and reported in a consistent and appropriate way.

Methods

The data source was 79 included studies and randomized controlled trials from a systematic review of doctors' effects on patients' physical health. We qualitatively assessed the studies and summarized how the doctors' effect was measured and reported.

Results

The doctors' effects on patients' physical health outcomes were reported as fixed effects, identifying high and low outliers, or random effects, which estimate the variation in patient health outcomes due to the doctor after accounting for all available variables via the intra-class correlation coefficient. Multivariable multilevel regression is commonly used to adjust for patient risk, doctor experience and other demographics, and also to account for the clustering effect of hospitals in estimating both fixed and random effects.

Conclusion

This methodological review identified inconsistencies in how the doctor's effect on patients' physical health outcomes is measured and reported. For grading doctors from worst to best performances and estimating random effects, specific recommendations are given along with the specific data points to report.

Identifying prognostic factors for clinical outcomes and costs in four high-volume surgical treatments using routinely collected hospital data

Identifying prognostic factors (PFs) is often costly and labor-intensive. Routinely collected hospital data provide opportunities to identify clinically relevant PFs and construct accurate prognostic models without additional data-collection costs. This multicenter (66 hospitals) study reports on associations various patient-level variables have with outcomes and costs. Outcomes were in-hospital mortality, intensive care unit (ICU) admission, length of stay, 30-day readmission, 30-day reintervention and in-hospital costs. Candidate PFs were age, sex, Elixhauser Comorbidity Score, prior hospitalizations, prior days spent in hospital, and socio-economic status. Included patients dealt with either colorectal carcinoma (CRC, n = 10,254), urinary bladder carcinoma (UBC, n = 17,385), acute percutaneous coronary intervention (aPCI, n = 25,818), or total knee arthroplasty (TKA, n = 39,214). Prior hospitalization significantly increased readmission risk in all treatments (OR between 2.15 and 25.50), whereas prior days spent in hospital decreased this risk (OR between 0.55 and 0.95). In CRC patients, women had lower risk of in-hospital mortality (OR 0.64), ICU admittance (OR 0.68) and 30-day reintervention (OR 0.70). Prior hospitalization was the strongest PF for higher costs across all treatments (31–64% costs increase/hospitalization). Prognostic model performance (c-statistic) ranged 0.67–0.92, with Brier scores below 0.08. R-squared ranged from 0.06–0.19 for LoS and 0.19–0.38 for costs. Identified PFs should be considered as building blocks for treatment-specific prognostic models and information for monitoring patients after surgery. Researchers and clinicians might benefit from gaining a better insight into the drivers behind (costs) prognosis.

Creating a culture of quality: our experience with providing feedback to frontline hospitalists

BACKGROUND

One way to provide performance feedback to hospitalists is through the use of dashboards, which deliver data based on agreed-upon standards. Despite the growing trend on feedback performance on quality metrics, there remain limited data on the means, frequency and content of feedback that should be provided to frontline hospitalists.

OBJECTIVE

The objective of our research is to report our experience with a comprehensive feedback system for frontline hospitalists, as well as report the change in our quality metrics after implementation.

DESIGN, SETTING AND PARTICIPANTS

This quality improvement project was conducted at a tertiary academic medical centre among our hospitalist group consisting of 46 full-time faculty members.

INTERVENTION OR EXPOSURE

A monthly performance feedback report was distributed to provide ongoing feedback to our hospitalist faculty, including an individual dashboard and a peer comparison report, complemented by coaching to incorporate process improvement tactics into providers' daily workflow.

MAIN OUTCOMES AND MEASURES

The main outcome of our study is the change in quality metrics after implementation of the monthly performance feedback report RESULTS: The dashboard and rank order list were sent to all faculty members every month. An improvement was seen in the following quality metrics: length of stay index, 30-day readmission rate, catheter-associated urinary tract infections, central line-associated bloodstream infections, provider component of Healthcare Consumer Assessment of Healthcare Providers and Systems scores, attendance at care coordination rounds and percentage of discharge orders placed by 10:00.

CONCLUSIONS

Implementation of a monthly performance feedback report for hospitalists, complemented by peer comparison and guidance on tactics to achieve these metrics, created a culture of quality and improvement in the quality of care delivered.

Models of generalist and specialist care in smaller hospitals in England: a mixed-methods study

Background

The increasing number of older, complex patients who require emergency admission to hospital has prompted calls for better models of medical generalist care, especially for smaller hospitals, whose size constrains resources and staffing.

Objective

To investigate the strengths and weaknesses of the current models of medical generalism used in smaller hospitals from patient, professional and service perspectives.

Methods

The design was a mixed-methods study. Phase 1 was a scoping and mapping exercise to create a typology of models of care, which was then explored further through 11 case studies. Phase 2 created a classification using the Hospital Episode Statistics of acute medical ‘generalist’ and ‘specialist’ work and described differences in workload and explored the links between case mix, typology and length of stay and between case mix and skill mix. Phase 3 analysed the relationships between models of care and patient-level costs. Phase 4 examined the strengths and weaknesses of the models of care through focus groups, a discrete choice experiment and an exploration of the impact of typology on other outcomes.

Results

In total, 50 models of care were explored through 48 interviews. A typology was constructed around generalist versus specialist patterns of consultant working. Twenty-five models were deployed by 48 hospitals, and no more than four hospitals used any one model of care. From the patient perspective, analysis of Hospital Episode Statistics data of 1.9 million care episodes found that the differences in case mix between hospitals were relatively small, with 65–70% of episodes accounted for by 20 case types. The skill mix of hospital staff varied widely; there were no relationships with case mix. Patients exhibited a preference for specialist care in the discrete choice experiment but indicated in focus groups that overall hospital quality was more important. From a service perspective, qualitative work found that models of care were contingent on complex constellations of factors, including staffing, the local hospital environment and policy imperatives. Neither the model of care nor the case mix accounted for variability in the length of stay (no associations were significant at p < 0.05). No significant differences were found in the costs of the models. Professionally, the preferences of doctors for specialist versus generalist work depended on their experiences of providing care and were associated with a healthy organisational culture and a co-operative approach to managing emergency work. Concepts of medical generalism were found to be complex and difficult to define, with theoretical models differing markedly from models in action.

Limitations

Smaller hospitals in multisite trusts were excluded, potentially leading to sample bias. The rapidly changing nature of the models limited the analysis of typology against outcomes.

Conclusions

The case mix of smaller hospitals was dominated by patients with presentations amenable to generalist approaches to care; however, there was no evidence to support any particular pattern of consultant working. Matching hospital staff to better meet local need and the creation of more collaborative working environments appear more likely to improve care in smaller hospitals than changing models.

Future work

The exploration of the relationships between workforce, measures of hospital culture, models of care, costs and outcomes in both smaller and larger hospitals is urgently required to underpin service reforms.

Study registration

This study is registered as Integrated Research Application System project ID 191393.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 4. See the NIHR Journals Library website for further project information.

How operations matters in healthcare standardization

Purpose

Process management approaches all pursue standardization, of which evidence-based medicine (EBM) is the most common form in healthcare. While EBM addresses improvement in clinical performance, it is unclear whether EBM also enhances operational performance. Conversely, operational process standardization (OPS) does not necessarily yield better clinical performance. The authors have therefore looked at the relationship between clinical practise standardization (CPS) and OPS and the way in which they jointly affect operational performance. The paper aims to discuss this issue.

Design/methodology/approach

The authors conducted a comparative case study analysis of a cataract surgery treatment at five Belgium hospital sites. Data collection involved 218 h of observations of 274 cataract surgeries. Both qualitative and quantitative methods were used.

Findings

Findings suggest that CPS does not automatically lead to improved resource or throughput efficiency. This can be explained by the low level of OPS across the five units, notwithstanding CPS. The results indicate that a wide range of variables on different levels (patient, physician and organization) affect OPS.

Research limitations/implications

Considering one type of care treatment in which clinical outcome variations are small complicates translating the findings to unstructured and complex care treatments.

Originality/value

With the introduction of OPS as a complementary view of CPS, the study clearly shows the potential of OPS to support CPS in practice. Operations matters in healthcare standardization, but only when it is managed in a deliberate way on a hospital and policy level.

A Cost-effectiveness Analysis of Albumin in Septic Shock: A Patient-level Data Analysis

PURPOSE

Albumin-based fluid therapy in septic shock is a matter of debate and criticism. The aim of this study was to assess the cost-effectiveness of albumin therapy in patients with septic shock.

METHODS

A retrospective cohort study was conducted in Imam Khomeini, Sina, and Shariati hospitals on patients with septic shock admitted to intensive care units from March 31, 2016 to September 22, 2017. Data sources were the health information system database and patient medical records. The patients with potential septic shock were identified based on norepinephrine use. Septic shock was confirmed after medical record review based on systemic inflammatory response syndrome criteria, antibiotic use, and fluid therapy. Patients who received albumin in the fluid therapy were compared with patients treated without albumin. The 28-day mortality, life-year gain, and cost-effectiveness were evaluated.

FINDINGS

The addition of albumin had no significant increase in life-year gain (mean difference = 0.67; 95% CI, -2.25 to 3.58). However, the addition of albumin increased the total cost of treatment by US $3846.07 (95% CI, US $2093.46-US $5598.98). The incremental cost-effectiveness ratio calculated based on the mean life-years gained was US$5740.40 per a life-year gained. The net monetary benefit was negative (-355.4; 95% CI, -15,387.61 to 14,676.81), and the probability that the addition of albumin will be cost-effective at a gross domestic product per capita was 40.0%.

IMPLICATIONS

Albumin-based fluid therapy does not improve the 28-day mortality of patients with septic shock. The addition of albumin in the fluid therapy of patients with septic shock was not cost-effective. Both the observational and retrospective nature of the study was expected to introduce bias. We recommend a cost-effectiveness analysis combined with clinical trials to settle the debate once and for all.

Contribution of primary care organisation and specialist care provider to variation in GP referrals for suspected cancer: ecological analysis of national data

OBJECTIVES

To examine how much of the variation between general practices in referral rates and cancer detection rates is attributable to local health services rather than the practices or their populations.

DESIGN

Ecological analysis of national data on fast-track referrals for suspected cancer from general practices. Data were analysed at the levels of general practice, primary care organisation (Clinical Commissioning Group) and secondary care provider (Acute Hospital Trust) level. Analysis of variation in detection rate was by multilevel linear and Poisson regression.

SETTING

6379 group practices with data relating to more than 50 cancer cases diagnosed over the 5 years from 2013 to 2017.

OUTCOMES

Proportion of observed variation attributable to primary and secondary care organisations in standardised fast-track referral rate and in cancer detection rate before and after adjustment for practice characteristics.

RESULTS

Primary care organisation accounted for 21% of the variation between general practices in the standardised fast-track referral rate and 42% of the unadjusted variation in cancer detection rate. After adjusting for standardised fast-track referral rate, primary care organisation accounted for 31% of the variation in cancer detection rate (compared with 18% accounted for by practice characteristics). In areas where a hospital trust was the main provider for multiple primary care organisations, hospital trusts accounted for the majority of the variation attributable to local health services (between 63% and 69%).

CONCLUSION

This is the first large-scale finding that a substantial proportion of the variation between general practitioner practices in referrals is attributable to their local healthcare systems. Efforts to reduce variation need to focus not just on individual practices but on local diagnostic service provision and culture at the interface of primary and secondary care.