Physician-level variation in clinical outcomes and resource use in inpatient general internal medicine: an observational study

Identifying high-impact-opportunity hospitals for improving healthcare quality based on a national population analysis of inter-hospital variation in mortality, readmissions and prolonged length of stay

OBJECTIVES

To study between-hospital variation in mortality, readmissions and prolonged length of stay across Belgian hospitals.

DESIGN

A retrospective nationwide observational study.

SETTING

Secondary and tertiary acute-care hospitals in Belgium.

PARTICIPANTS

We studied 4 560 993 hospital stays in 99 (98%) Belgian acute-care hospitals between 2016 and 2018.

PRIMARY OUTCOME MEASURES

Using generalised linear mixed models, we calculated hospital-specific and Major Diagnostic Category (MDC)-specific risk-adjusted in-hospital mortality, readmissions within 30 days and length of stay above the MDC-specific 90th percentile and assessed between-hospital variation through estimated variance components.

RESULTS

There was strong evidence of between-hospital variation in mortality, readmissions and prolonged length of stay across the vast majority of patient service lines. Overall, should hospitals with upper-quartile risk-standardised rates succeed in improving to the median level, a yearly 4076 hospital deaths, 3671 readmissions and 15 787 long patient stays could potentially be avoided in those hospitals. Our analysis revealed a select set of 'high-impact-opportunity hospitals' characterised by poor performance across outcomes and across a large number of MDCs.

CONCLUSIONS

Analysis of between-hospital variation highlights important differences in patient outcomes that are not explained by known patient or hospital characteristics. Identifying 'high-impact-opportunity hospitals' can help government inspection bodies and hospital managers to establish targeted audits and inspections to generate effective quality improvement initiatives.

Real-world use of glucocorticoids and clinical outcomes in adults hospitalized with community-acquired pneumonia on medical wards

BACKGROUND

Little is known about the real-world use of systemic glucocorticoids to treat patients hospitalized with community-acquired pneumonia (CAP) outside of the intensive care unit (ICU).

METHODS

This retrospective cohort study included 11,588 hospitalizations for CAP without chronic pulmonary disease at seven hospitals in Ontario, Canada. We report physician-level variation in the use of glucocorticoids and trends over time. We investigated the association between glucocorticoid prescriptions and clinical outcomes, using propensity score overlap weighting to account for confounding by indication.

RESULTS

Glucocorticoids were prescribed in 1283 (11.1%) patients, increasing over time from 10.0% in 2010 to 11.9% in 2020 (p = .008). Physician glucocorticoid prescribing ranged from 2.9% to 34.6% (median 10.0%, inter quartile range [IQR]: 6.7%-14.6%). Patients receiving glucocorticoids tended to be younger (median age 73 vs. 79), have higher Charlson comorbidity scores (score of 2 or more: 42.4% vs. 31.0%), more cancer (26.6% vs. 13.2%), more renal disease (11.5% vs. 6.6%), and less dementia (7.8% vs. 14.8%). Patients treated with glucocorticoids had higher rates of in-hospital mortality (weighted Risk Difference = 1.72, 95% confidence interval [95% CI]: 0.16-3.3, p = .033). Glucocorticoid use was not associated with ICU admission, hospital length-of-stay, or 30-day readmission.

CONCLUSION

Glucocorticoids were prescribed in 11.1% of patients hospitalized with CAP outside of ICU and one in four physicians prescribed glucocorticoids in more than 14% of patients. Glucocorticoid use was associated with greater in-hospital mortality, although these findings are limited by large selection effects. Clinicians should exercise caution in prescribing glucocorticoids for nonsevere CAP, and definitive trials are needed in this population.

A simple electronic medical record-based predictors of illness severity in sepsis (sepsis) score

OBJECTIVE

Current scores for predicting sepsis outcomes are limited by generalizability, complexity, and electronic medical record (EMR) integration. Here, we validate a simple EMR-based score for sepsis outcomes in a large multi-centre cohort.

DESIGN

A simple electronic medical record-based predictor of illness severity in sepsis (SEPSIS) score was developed (4 additive lab-based predictors) using a population-based retrospective cohort study.

SETTING

Internal medicine services across four academic teaching hospitals in Toronto, Canada from April 2010-March 2015 (primary cohort) and 2015-2019 (secondary cohort).

PATIENTS

We identified patients admitted with sepsis based upon receipt of antibiotics and positive cultures.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was in-hospital mortality and secondary outcomes were ICU admission at 72 hours, and hospital length of stay (LOS). We calculated the area under the receiver operating curve (AUROC) for the SEPSIS score, qSOFA, and NEWS2. We then evaluated the SEPSIS score in a secondary cohort (2015-2019) of hospitalized patients receiving antibiotics. Our primary cohort included 1,890 patients with a median age of 72 years (IQR: 56-83). 9% died during hospitalization, 18.6% were admitted to ICU, and mean LOS was 12.7 days (SD: 21.5). In the primary and secondary (2015-2019, 4811 patients) cohorts, the AUROCs of the SEPSIS score for predicting in-hospital mortality were 0.63 and 0.64 respectively, which were similar to NEWS2 (0.62 and 0.67) and qSOFA (0.62 and 0.68). AUROCs for predicting ICU admission at 72 hours, and length of stay > 14 days, were similar between scores, in the primary and secondary cohorts. All scores had comparable calibration for predicting mortality.

CONCLUSIONS

An EMR-based SEPSIS score shows a similar ability to predict important clinical outcomes compared with other validated scores (qSOFA and NEWS2). Because of the SEPSIS score's simplicity, it may prove a useful tool for clinical and research applications.

Data-driven approach to identifying potential laboratory overuse in general internal medicine (GIM) inpatients

BACKGROUND

Reducing laboratory test overuse is important for high quality, patient-centred care. Identifying priorities to reduce low value testing remains a challenge.

OBJECTIVE

To develop a simple, data-driven approach to identify potential sources of laboratory overuse by combining the total cost, proportion of abnormal results and physician-level variation in use of laboratory tests.

DESIGN, SETTING AND PARTICIPANTS

A multicentre, retrospective study at three academic hospitals in Toronto, Canada. All general internal medicine (GIM) hospitalisations between 1 April 2010 and 31 October 2017.

RESULTS

There were 106 813 GIM hospitalisations during the study period, with median hospital length-of-stay of 4.6 days (IQR: 2.33-9.19). There were 21 tests which had a cumulative cost >US$15 400 at all three sites. The costliest test was plasma electrolytes (US$4 907 775), the test with the lowest proportion of abnormal results was red cell folate (0.2%) and the test with the greatest physician-level variation in use was antiphospholipid antibodies (coefficient of variation 3.08). The five tests with the highest cumulative rank based on greatest cost, lowest proportion of abnormal results and highest physician-level variation were: (1) lactate, (2) antiphospholipid antibodies, (3) magnesium, (4) troponin and (5) partial thromboplastin time. In addition, this method identified unique tests that may be a potential source of laboratory overuse at each hospital.

CONCLUSIONS

A simple multidimensional, data-driven approach combining cost, proportion of abnormal results and physician-level variation can inform interventions to reduce laboratory test overuse. Reducing low value laboratory testing is important to promote high value, patient-centred care.

Between-Hospital and Between-Physician Variation in Outcomes and Costs in High- and Low-Complex Surgery: A Nationwide Multilevel Analysis

OBJECTIVES

Clinicians and policy makers are increasingly exploring strategies to reduce unwarranted variation in outcomes and costs. Adequately accounting for case mix and better insight into the levels at which variation exists is crucial for such strategies. This nationwide study investigates variation in surgical outcomes and costs at the level of hospitals and individual physicians and evaluates whether these can be reliably compared on performance.

METHODS

Variation was analyzed using 92 330 patient records collected from 62 Dutch hospitals who underwent surgery for colorectal cancer (n = 6640), urinary bladder cancer (n = 14 030), myocardial infarction (n = 31 870), or knee osteoarthritis (n = 39 790) in the period 2018 to 2019. Multilevel regression modeling with and without case-mix adjustment was used to partition variation in between-hospital and between-physician components for in-hospital mortality, intensive care unit admission, length of stay, 30-day readmission, 30-day reintervention, and in-hospital costs. Reliability was calculated for each treatment-outcome combination at both levels.

RESULTS

Across outcomes, hospital-level variation relative to total variation ranged between ≤ 1% and 15%, and given the high caseloads, this typically yielded high reliability (> 0.9). In contrast, physician-level variation components were typically ≤ 1%, with limited opportunities to make reliable comparisons. The impact of case-mix adjustment was limited, but nonnegligible.

CONCLUSIONS

It is not typically possible to make reliable comparisons among physicians due to limited partitioned variation and low caseloads. Nevertheless, for hospitals, the opposite often holds. Although variation-reduction efforts directed at hospitals are thus more likely to be successful, this should be approached cautiously, partly because level-specific variation and the impact of case mix vary considerably across treatments and outcomes.

Methodology paper for the General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED): a retrospective cohort study of internal medicine resident case-mix, clinical care and patient outcomes

INTRODUCTION

Unwarranted variation in patient care among physicians is associated with negative patient outcomes and increased healthcare costs. Care variation likely also exists for resident physicians. Despite the global movement towards outcomes-based and competency-based medical education, current assessment strategies in residency do not routinely incorporate clinical outcomes. The widespread use of electronic health records (EHRs) may enable the implementation of in-training assessments that incorporate clinical care and patient outcomes.

METHODS AND ANALYSIS

The General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED) is a retrospective cohort study of senior residents (postgraduate year 2/3) enrolled in the University of Toronto Internal Medicine (IM) programme between 1 April 2010 and 31 December 2020. This study focuses on senior IM residents and patients they admit overnight to four academic hospitals. Senior IM residents are responsible for overseeing all overnight admissions; thus, care processes and outcomes for these clinical encounters can be at least partially attributed to the care they provide. Call schedules from each hospital, which list the date, location and senior resident on-call, will be used to link senior residents to EHR data of patients admitted during their on-call shifts. Patient data will be derived from the GEMINI database, which contains administrative (eg, demographic and disposition) and clinical data (eg, laboratory and radiological investigation results) for patients admitted to IM at the four academic hospitals. Overall, this study will examine three domains of resident practice: (1) case-mix variation across residents, hospitals and academic year, (2) resident-sensitive quality measures (EHR-derived metrics that are partially attributable to resident care) and (3) variations in patient outcomes across residents and factors that contribute to such variation.

ETHICS AND DISSEMINATION

GEMINI MedED was approved by the University of Toronto Ethics Board (RIS#39339). Results from this study will be presented in academic conferences and peer-reviewed journals.

Is There a Doctors' Effect on Patients' Physical Health, Beyond the Intervention and All Known Factors? A Systematic Review

Purpose

Despite billions of doctor visits worldwide each year, little is known on whether doctors themselves affect patients' physical health after accounting for intervention and confounders such as patients' and doctors' data, hospital effects, nor how strong that doctors' effect is. Knowledge of surgeons' and psychotherapists' effects exists, but not for 102 other medical specialties notwithstanding the importance of such knowledge.

Methods

Eligibility Criteria: Randomized controlled trials (RCTs), case-control, and cohort studies including medical doctors except surgeons for any intervention, reporting the proportion of variance in patients' outcomes owing to the doctors (random effects), or the fixed effects of grading doctors by outcomes, after multivariate adjustment. Exclusions: studies of <15 doctors or solely reporting doctors' effects for known variables.

Sources

Medline, Embase, PsycINFO, inception to June 2020. Manual search for papers referring/referred to by resulting studies.

Risk of Bias

Using Newcastle-Ottawa scale.

Results

Despite all medical interventions bar surgery being eligible, only thirty cohort papers were found, covering 36,239 doctors, with 10 specialties, 21 interventions, 60 outcomes (17 unique). Studies reported doctors' effects by grading doctors from best to worst, or by diversely calculating the doctor-attributed percentage of patients' outcome variation, ie the intra-class correlation coefficient (ICC). Sixteen studies presented fixed effects, 18 random effects, and 3 another approach. No RCTs found. Thirteen studies reported exceptionally good and/or poor performers with confidence intervals wholly outside the average performance. ICC range 0 to 33%, mean 3.9%. Highly diverse reporting, meta-analysis therefore not applicable.

Conclusion

Doctors, on their own, can affect patients' physical health for many interventions and outcomes. Effects range from negligible to substantial, even after accounting for all known variables. Many published cohorts may reveal valuable information by reanalyzing their data for doctors' effects. Positive and negative doctor outliers appear regularly. Therefore, it can matter which doctor is chosen.

The Doctors' Effect on Patients' Physical Health Outcomes Beyond the Intervention: A Methodological Review

Background

Previous research suggests that when a treatment is delivered, patients' outcomes may vary systematically by medical practitioner.

Objective

To conduct a methodological review of studies reporting on the effect of doctors on patients' physical health outcomes and to provide recommendations on how this effect could be measured and reported in a consistent and appropriate way.

Methods

The data source was 79 included studies and randomized controlled trials from a systematic review of doctors' effects on patients' physical health. We qualitatively assessed the studies and summarized how the doctors' effect was measured and reported.

Results

The doctors' effects on patients' physical health outcomes were reported as fixed effects, identifying high and low outliers, or random effects, which estimate the variation in patient health outcomes due to the doctor after accounting for all available variables via the intra-class correlation coefficient. Multivariable multilevel regression is commonly used to adjust for patient risk, doctor experience and other demographics, and also to account for the clustering effect of hospitals in estimating both fixed and random effects.

Conclusion

This methodological review identified inconsistencies in how the doctor's effect on patients' physical health outcomes is measured and reported. For grading doctors from worst to best performances and estimating random effects, specific recommendations are given along with the specific data points to report.