Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Temporal Clustering of Critical Illness Events on Medical Wards

Importance

Recognizing and preventing patient deterioration is important for hospital safety.

Objective

To investigate whether critical illness events (in-hospital death or intensive care unit [ICU] transfer) are associated with greater risk of subsequent critical illness events for other patients on the same medical ward.

Design, Setting, and Participants

Retrospective cohort study in 5 hospitals in Toronto, Canada, including 118 529 hospitalizations. Patients were admitted to general internal medicine wards between April 1, 2010, and October 31, 2017. Data were analyzed between January 1, 2020, and April 10, 2023.

Exposures

Critical illness events (in-hospital death or ICU transfer).

Main Outcomes and Measures

The primary outcome was the composite of in-hospital death or ICU transfer. The association between critical illness events on the same ward across 6-hour intervals was studied using discrete-time survival analysis, adjusting for patient and situational factors. The association between critical illness events on different comparable wards in the same hospital was measured as a negative control.

Results

The cohort included 118 529 hospitalizations (median age, 72 years [IQR, 56-83 years]; 50.7% male). Death or ICU transfer occurred in 8785 hospitalizations (7.4%). Patients were more likely to experience the primary outcome after exposure to 1 prior event (adjusted odds ratio [AOR], 1.39; 95% CI, 1.30-1.48) and more than 1 prior event (AOR, 1.49; 95% CI, 1.33-1.68) in the prior 6-hour interval compared with no exposure. The exposure was associated with increased odds of subsequent ICU transfer (1 event: AOR, 1.67; 95% CI, 1.54-1.81; >1 event: AOR, 2.05; 95% CI, 1.79-2.36) but not death alone (1 event: AOR, 1.08; 95% CI, 0.97-1.19; >1 event: AOR, 0.88; 95% CI, 0.71-1.09). There was no significant association between critical illness events on different wards within the same hospital.

Conclusions and Relevance

Findings of this cohort study suggest that patients are more likely to be transferred to the ICU in the hours after another patient's critical illness event on the same ward. This phenomenon could have several explanations, including increased recognition of critical illness and preemptive ICU transfers, resource diversion to the first event, or fluctuations in ward or ICU capacity. Patient safety may be improved by better understanding the clustering of ICU transfers on medical wards.

Quality improvement strategies for diabetes care: Effects on outcomes for adults living with diabetes

BACKGROUND

There is a large body of evidence evaluating quality improvement (QI) programmes to improve care for adults living with diabetes. These programmes are often comprised of multiple QI strategies, which may be implemented in various combinations. Decision-makers planning to implement or evaluate a new QI programme, or both, need reliable evidence on the relative effectiveness of different QI strategies (individually and in combination) for different patient populations.

OBJECTIVES

To update existing systematic reviews of diabetes QI programmes and apply novel meta-analytical techniques to estimate the effectiveness of QI strategies (individually and in combination) on diabetes quality of care.

SEARCH METHODS

We searched databases (CENTRAL, MEDLINE, Embase and CINAHL) and trials registers (ClinicalTrials.gov and WHO ICTRP) to 4 June 2019. We conducted a top-up search to 23 September 2021; we screened these search results and 42 studies meeting our eligibility criteria are available in the awaiting classification section.

SELECTION CRITERIA

We included randomised trials that assessed a QI programme to improve care in outpatient settings for people living with diabetes. QI programmes needed to evaluate at least one system- or provider-targeted QI strategy alone or in combination with a patient-targeted strategy. - System-targeted: case management (CM); team changes (TC); electronic patient registry (EPR); facilitated relay of clinical information (FR); continuous quality improvement (CQI). - Provider-targeted: audit and feedback (AF); clinician education (CE); clinician reminders (CR); financial incentives (FI). - Patient-targeted: patient education (PE); promotion of self-management (PSM); patient reminders (PR). Patient-targeted QI strategies needed to occur with a minimum of one provider or system-targeted strategy.

DATA COLLECTION AND ANALYSIS

We dual-screened search results and abstracted data on study design, study population and QI strategies. We assessed the impact of the programmes on 13 measures of diabetes care, including: glycaemic control (e.g. mean glycated haemoglobin (HbA1c)); cardiovascular risk factor management (e.g. mean systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), proportion of people living with diabetes that quit smoking or receiving cardiovascular medications); and screening/prevention of microvascular complications (e.g. proportion of patients receiving retinopathy or foot screening); and harms (e.g. proportion of patients experiencing adverse hypoglycaemia or hyperglycaemia). We modelled the association of each QI strategy with outcomes using a series of hierarchical multivariable meta-regression models in a Bayesian framework. The previous version of this review identified that different strategies were more or less effective depending on baseline levels of outcomes. To explore this further, we extended the main additive model for continuous outcomes (HbA1c, SBP and LDL-C) to include an interaction term between each strategy and average baseline risk for each study (baseline thresholds were based on a data-driven approach; we used the median of all baseline values reported in the trials). Based on model diagnostics, the baseline interaction models for HbA1c, SBP and LDL-C performed better than the main model and are therefore presented as the primary analyses for these outcomes. Based on the model results, we qualitatively ordered each QI strategy within three tiers (Top, Middle, Bottom) based on its magnitude of effect relative to the other QI strategies, where 'Top' indicates that the QI strategy was likely one of the most effective strategies for that specific outcome. Secondary analyses explored the sensitivity of results to choices in model specification and priors.  Additional information about the methods and results of the review are available as Appendices in an online repository. This review will be maintained as a living systematic review; we will update our syntheses as more data become available.

MAIN RESULTS

We identified 553 trials (428 patient-randomised and 125 cluster-randomised trials), including a total of 412,161 participants. Of the included studies, 66% involved people living with type 2 diabetes only. Participants were 50% female and the median age of participants was 58.4 years. The mean duration of follow-up was 12.5 months. HbA1c was the commonest reported outcome; screening outcomes and outcomes related to cardiovascular medications, smoking and harms were reported infrequently. The most frequently evaluated QI strategies across all study arms were PE, PSM and CM, while the least frequently evaluated QI strategies included AF, FI and CQI. Our confidence in the evidence is limited due to a lack of information on how studies were conducted.  Four QI strategies (CM, TC, PE, PSM) were consistently identified as 'Top' across the majority of outcomes. All QI strategies were ranked as 'Top' for at least one key outcome. The majority of effects of individual QI strategies were modest, but when used in combination could result in meaningful population-level improvements across the majority of outcomes. The median number of QI strategies in multicomponent QI programmes was three.  Combinations of the three most effective QI strategies were estimated to lead to the below effects:  - PR + PSM + CE: decrease in HbA1c by 0.41% (credibility interval (CrI) -0.61 to -0.22) when baseline HbA1c < 8.3%; - CM + PE + EPR: decrease in HbA1c by 0.62% (CrI -0.84 to -0.39) when baseline HbA1c > 8.3%;  - PE + TC + PSM: reduction in SBP by 2.14 mmHg (CrI -3.80 to -0.52) when baseline SBP < 136 mmHg; - CM + TC + PSM: reduction in SBP by 4.39 mmHg (CrI -6.20 to -2.56) when baseline SBP > 136 mmHg;  - TC + PE + CM: LDL-C lowering of 5.73 mg/dL (CrI -7.93 to -3.61) when baseline LDL < 107 mg/dL; - TC + CM + CR: LDL-C lowering by 5.52 mg/dL (CrI -9.24 to -1.89) when baseline LDL > 107 mg/dL. Assuming a baseline screening rate of 50%, the three most effective QI strategies were estimated to lead to an absolute improvement of 33% in retinopathy screening (PE + PR + TC) and 38% absolute increase in foot screening (PE + TC + Other).

AUTHORS' CONCLUSIONS

There is a significant body of evidence about QI programmes to improve the management of diabetes. Multicomponent QI programmes for diabetes care (comprised of effective QI strategies) may achieve meaningful population-level improvements across the majority of outcomes. For health system decision-makers, the evidence summarised in this review can be used to identify strategies to include in QI programmes. For researchers, this synthesis identifies higher-priority QI strategies to examine in further research regarding how to optimise their evaluation and effects. We will maintain this as a living systematic review.

Potentially avoidable admissions to general internal medicine at an academic teaching hospital: an observational study

BACKGROUND

Identifying potentially avoidable admissions to Canadian hospitals is an important health system goal. With general internal medicine (GIM) accounting for 40% of hospital admissions, we sought to develop a method to identify potentially avoidable admissions and characterize patient, provider and health system factors.

METHODS

We conducted an observational study of GIM admissions at our institution from August 2019 to February 2020. We defined potentially avoidable admissions as admissions that could be managed in an appropriate and safe manner in the emergency department or ambulatory setting and asked staff physicians to screen admissions daily and flag candidates as potentially avoidable admissions. For each candidate, we prepared a case review and debriefed with members of the admitting team. We then reviewed each candidate with our research team, assigned an avoidability score (1 [low] to 4 [high]) and identified contributing factors for those with scores of 3 or more.

RESULTS

We screened 601 total admissions and staff physicians flagged 117 (19.5%) of these as candidate potential avoidable admissions. Consensus review identified 67 candidates as potentially avoidable admissions (11.1%, 95% confidence interval 8.8%-13.9%); these patients were younger (mean age 65 yr v. 72 yr), had fewer comorbidities (Canadian Institute for Health Information Case Mix Group+ 0.42 v. 1.14), had lower resource-intensity weighting scores (0.72 v. 1.50) and shorter hospital lengths of stay (29 h v. 105 h) (p < 0.01). Common factors included diagnostic and therapeutic uncertainty, perceived need for short-term monitoring, government directive of a 4-hour limit for admission decision-making and subspecialist request to admit.

INTERPRETATION

Our prospective method of screening, flagging and case review showed that 1 in 9 GIM admissions were potentially avoidable. Other institutions could consider adapting this methodology to ascertain their rate of potentially avoidable admissions and to understand contributing factors to inform improvement endeavours.

Striving for high reliability in healthcare: a qualitative study of the implementation of a hospital safety programme

BACKGROUND

Healthcare leaders look to high-reliability organisations (HROs) for strategies to improve safety, despite questions about how to translate these strategies into practice. Weick and Sutcliffe describe five principles exhibited by HROs. Interventions aiming to foster these principles are common in healthcare; however, there have been few examinations of the perceptions of those who have planned or experienced these efforts.

OBJECTIVE

This single-site qualitative study explores how healthcare professionals understand and enact the HRO principles in response to an HRO-inspired hospital-wide safety programme.

METHODS

We interviewed 71 participants representing hospital executives, programme leadership, and staff and physicians from three clinical services. We observed and collected data from unit and hospital-wide quality and safety meetings and activities. We used thematic analysis to code and analyse the data.

RESULTS

Participants reported enactment of the HRO principles 'preoccupation with failure', 'reluctance to simplify interpretations' and 'sensitivity to operations', and described the programme as adding legitimacy, training, and support. However, the programme was more often targeted at, and taken up by, nurses compared with other groups. Participants were less able to identify interventions that supported the HRO principles 'commitment to resilience' and 'deference to expertise' and reported limited examples of changes in practices related to these principles. Moreover, we identified inconsistent, and even conflicting, understanding of concepts related to the HRO principles, often related to social and professional norms and practices. Finally, an individualised rather than systemic approach hindered collective actions underlying high reliability.

CONCLUSION

Our findings demonstrate that the safety programme supported some HRO principles more than others, and was targeted at, and perceived differently across professional groups leading to inconsistent understanding and enactments of the principles across the organisation. Combining HRO-inspired interventions with more targeted attention to each of the HRO principles could produce greater, more consistent high-reliability practices.

A Novel Collaborative Care Program to Augment Nursing Home Care During and After the COVID-19 Pandemic

The 2019 novel coronavirus (COVID-19) pandemic created an immediate need to enhance current efforts to reduce transfers of nursing home (NH) residents to acute care. Long-Term Care Plus (LTC+), a collaborative care program developed and implemented during the COVID-19 pandemic, aimed to enhance care in the NH setting while also decreasing unnecessary acute care transfers. Using a hub-and-spoke model, LTC+ was implemented in 6 hospitals serving as central hubs to 54 geographically associated NHs with 9574 beds in Toronto, Canada. LTC+ provided NHs with the following: (1) virtual general internal medicine (GIM) consultations; (2) nursing navigator support; (3) rapid access to laboratory and diagnostic imaging services; and (4) educational resources. From April 2020 to June 2021, LTC+ provided 381 GIM consultations that addressed abnormal bloodwork (15%), cardiac problems (13%), and unexplained fever (11%) as the most common reasons for consultation. Sixty-five nurse navigator calls addressed requests for non-GIM specialist consultations (34%), wound care assessments (14%), and system navigation (12%). One hundred seventy-seven (46%, 95% CI 41%-52%) consults addressed care concerns sufficiently to avoid the need for acute care transfer. All 36 primary care physicians who consulted the LTC+ program reported strong satisfaction with the advice provided. Early results demonstrate the feasibility and acceptability of an integrated care model that enhances care delivery for NH residents where they reside and has the potential to positively impact the long-term care sector by ensuring equitable and timely access to care for people living in NHs. It represents an important step toward health system integration that values the expertise within the long-term care sector.

Association of Cardiology Billing Amounts With Health Care Utilization and Clinical Outcomes in Patients With Atrial Fibrillation

Background The relationship between health care utilization and outcomes in patients with atrial fibrillation is unknown. The objective of this study was to investigate whether cardiologists' billing amounts in a fee-for-service environment are associated with better patient-level clinical outcomes. Methods and Results A retrospective cohort study was conducted using administrative claims data of cardiologists in Ontario, Canada between April 1, 2011 and March 31, 2016. The cardiologists were stratified into quintiles based on their median billing patterns per patient over the observation period. The primary outcomes were patient-level receipt of repeat visits, cardiac diagnostic tests, and medications ≤1 year of index date. The secondary clinical outcomes were death, emergency department visits, and all-cause hospitalization 1-year post-index visit. The patient cohort comprised 182 572 patients with atrial fibrillation (median age 74 years, 58% male) from 467 cardiologists. Patients with atrial fibrillation seen by higher-billing cardiologists were 26% more likely to have an echocardiogram (adjusted odds ratio [aOR], 1.26 [95% CI, 1.10-1.43] for quintile 5 versus 2), 28% a stress test (aOR, 1.28 [1.12-1.46] for quintile 5 versus 2), 25% continuous electrocardiographic monitoring (aOR, 1.25 [1.08-1.46] for quintile 4 versus 2), and 79% more likely to get a stress echocardiogram (aOR, 1.79 [1.32-2.42] for quintile 5 versus 2). They also had a higher rate of all-cause hospitalization (aOR, 1.13 [1.07-1.20]). Mortality rates were similar across cardiologists billing quintiles (eg, aOR, 0.98 [0.87-1.11] for quintile 4 versus 2). Conclusions Higher-billing cardiologists ordered more diagnostic tests per patient with atrial fibrillation but these are not associated with improvements in outcomes.

Implementation of a central-line bundle: a qualitative study of three clinical units

Background

Evidence for the central line-associated bloodstream infection (CLABSI) bundle effectiveness remains mixed, possibly reflecting implementation challenges and persistent ambiguities in how CLABSIs are counted and bundle adherence measured. In the context of a tertiary pediatric hospital that had reduced CLABSI by 30% as part of an international safety program, we aimed to examine unit-based socio-cultural factors influencing bundle practices and measurement, and how they come to be recognized and attended to by safety leaders over time in an organization-wide bundle implementation effort.

Methods

We used an interpretivist qualitative research approach, based on 74 interviews, approximately 50 h of observations, and documents. Data collection focused on hospital executives and safety leadership, and three clinical units: a medical specialty unit, an intensive care unit, and a surgical unit. We used thematic analysis and constant comparison methods for data analysis.

Results

Participants had variable beliefs about the central-line bundle as a quality improvement priority based on their professional roles and experiences and unit setting, which influenced their responses. Nursing leaders were particularly concerned about CLABSI being one of an overwhelming number of QI targets for which they were responsible. Bundle implementation strategies were initially reliant on unit-based nurse education. Over time there was recognition of the need for centralized education and reinforcement tactics. However, these interventions achieved limited impact given the influence of competing unit workflow demands and professional roles, interactions, and routines, which were variably targeted in the safety program. The auditing process, initially a responsibility of units, was performed in different ways based on individuals’ approaches to the process. Given concerns about auditing reliability, a centralized approach was implemented, which continued to have its own variability.

Conclusions

Our findings report on a contextualized, dynamic implementation approach that required movement between centralized and unit-based approaches and from a focus on standardization to some recognition of a role for customization. However, some factors related to bundle compliance and measurement remain unaddressed, including harder to change socio-cultural factors likely important to sustainability of the CLABSI reductions and fostering further improvements across a broader safety agenda.

Experiential Learning in Project-Based Quality Improvement Education: Questioning Assumptions and Identifying Future Directions

PURPOSE

Project-based experiential learning is a defining element of quality improvement (QI) education despite ongoing challenges and uncertainties. The authors examined stakeholders' perceptions and experiences of QI project-based learning to increase understanding of factors that influence learning and project experiences.

METHOD

The authors used a case study approach to examine QI project-based learning in 3 advanced longitudinal QI programs, 2 at the University of Toronto and 1 at an academic tertiary-care hospital. From March 2016 to June 2017, they undertook 135 hours of education program observation and 58 interviews with learners, program directors, project coaches, and institutional leaders and reviewed relevant documents. They analyzed data using a conventional and directed data analysis approach.

RESULTS

The findings provide insight into 5 key factors that influenced participants' project-based learning experiences and outcomes: (1) variable emphasis on learning versus project objectives and resulting benefits, tensions, and consequences; (2) challenges integrating the QI project into the curriculum timeline; (3) project coaching factors (e.g., ability, capacity, role clarity); (4) participants' differing access to resources and ability to direct a QI project given their professional roles; and (5) workplace environment influence on project success.

CONCLUSIONS

The findings contribute to an empirical basis toward more effective experiential learning in QI by identifying factors to target and optimize. Expanding conceptualizations of project-based learning for QI education beyond learner-initiated, time-bound projects, which are at the core of many QI educational initiatives, may be necessary to improve learning and project outcomes.

Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials

OBJECTIVE

To report the improvements achieved with clinical decision support systems and examine the heterogeneity from pooling effects across diverse clinical settings and intervention targets.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Medline up to August 2019.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES AND METHODS

Randomised or quasi-randomised controlled trials reporting absolute improvements in the percentage of patients receiving care recommended by clinical decision support systems. Multilevel meta-analysis accounted for within study clustering. Meta-regression was used to assess the degree to which the features of clinical decision support systems and study characteristics reduced heterogeneity in effect sizes. Where reported, clinical endpoints were also captured.

RESULTS

In 108 studies (94 randomised, 14 quasi-randomised), reporting 122 trials that provided analysable data from 1 203 053 patients and 10 790 providers, clinical decision support systems increased the proportion of patients receiving desired care by 5.8% (95% confidence interval 4.0% to 7.6%). This pooled effect exhibited substantial heterogeneity (I²=76%), with the top quartile of reported improvements ranging from 10% to 62%. In 30 trials reporting clinical endpoints, clinical decision support systems increased the proportion of patients achieving guideline based targets (eg, blood pressure or lipid control) by a median of 0.3% (interquartile range -0.7% to 1.9%). Two study characteristics (low baseline adherence and paediatric settings) were associated with significantly larger effects. Inclusion of these covariates in the multivariable meta-regression, however, did not reduce heterogeneity.

CONCLUSIONS

Most interventions with clinical decision support systems appear to achieve small to moderate improvements in targeted processes of care, a finding confirmed by the small changes in clinical endpoints found in studies that reported them. A minority of studies achieved substantial increases in the delivery of recommended care, but predictors of these more meaningful improvements remain undefined.

Modelling resource requirements and physician staffing to provide virtual urgent medical care for residents of long-term care homes: a cross-sectional study

BACKGROUND

The coronavirus disease 2019 (COVID-19) outbreak increases the importance of strategies to enhance urgent medical care delivery in long-term care (LTC) facilities that could potentially reduce transfers to emergency departments. The study objective was to model resource requirements to deliver virtual urgent medical care in LTC facilities.

METHODS

We used data from all general medicine inpatient admissions at 7 hospitals in the Greater Toronto Area, Ontario, Canada, over a 7.5-year period (Apr. 1, 2010, to Oct. 31, 2017) to estimate historical patterns of hospital resource use by LTC residents. We estimated an upper bound of potentially avoidable transfers by combining data on short admissions (≤ 72 h) with historical data on the proportion of transfers from LTC facilities for which patients were discharged from the emergency department without admission. Regression models were used to extrapolate future resource requirements, and queuing models were used to estimate physician staffing requirements to perform virtual assessments.

RESULTS

There were 235 375 admissions to general medicine wards, and residents of LTC facilities (age 16 yr or older) accounted for 9.3% (n = 21 948) of these admissions. Among the admissions of residents of LTC facilities, short admissions constituted 24.1% (n = 5297), and for 99.8% (n = 5284) of these admissions, the patient received laboratory testing, for 86.9% (n = 4604) the patient received plain radiography, for 41.5% (n = 2197) the patient received computed tomography and for 81.2% (n = 4300) the patient received intravenous medications. If all patients who have short admissions and are transferred from the emergency department were diverted to outpatient care, the average weekly demand for outpatient imaging per hospital would be 2.6 ultrasounds, 11.9 computed tomographic scans and 23.9 radiographs per week. The average daily volume of urgent medical virtual assessments would range from 2.0 to 5.8 per hospital. A single centralized virtual assessment centre staffed by 2 or 3 physicians would provide services similar in efficiency (measured by waiting time for physician assessment) to 7 separate centres staffed by 1 physician each.

INTERPRETATION

The provision of acute medical care to LTC residents at their facility would probably require rapid access to outpatient diagnostic imaging, within-facility access to laboratory services and intravenous medication and virtual consultations with physicians. The results of this study can inform efforts to deliver urgent medical care in LTC facilities in light of a potential surge in COVID-19 cases.

Building the Bridge to Quality: An Urgent Call to Integrate Quality Improvement and Patient Safety Education With Clinical Care

Current models of quality improvement and patient safety (QIPS) education are not fully integrated with clinical care delivery, representing a major impediment toward achieving widespread QIPS competency among health professions learners and practitioners. The Royal College of Physicians and Surgeons of Canada organized a 2-day consensus conference in Niagara Falls, Ontario, Canada, called Building the Bridge to Quality, in September 2016. Its goal was to convene an international group of educational and health system leaders, educators, frontline clinicians, learners, and patients to engage in a consensus-building process and generate a list of actionable strategies that individuals and organizations can use to better integrate QIPS education with clinical care.Four strategic directions emerged: prioritize the integration of QIPS education and clinical care, build structures and implement processes to integrate QIPS education and clinical care, build capacity for QIPS education at multiple levels, and align educational and patient outcomes to improve quality and patient safety. Individuals and organizations can refer to the specific tactics associated with the 4 strategic directions to create a road map of targeted actions most relevant to their organizational starting point.To achieve widespread change, collaborative efforts and alignment of intrinsic and extrinsic motivators are needed on an international scale to shift the culture of educational and clinical environments and build bridges that connect training programs and clinical environments, align educational and health system priorities, and improve both learning and care, with the ultimate goal of achieving improved outcomes and experiences for patients, their families, and communities.

Evidence-based medicine: A cornerstone for clinical care but not for quality improvement

Quality improvement (QI) as a clinical improvement science has been criticized for failing to deliver broad patient outcome improvement and for being a top-down regulatory and compliance construct. These critics have argued that the focus of QI should be on increasing adherence to clinical practice guidelines (CPGs) and, as a result, should be consolidated into research structures with the science of evidence-based medicine (EBM) at the helm. We argue that EBM often overestimates the role of knowledge as the root cause of quality problems and focuses almost exclusively on the effectiveness of care while often neglecting the domains of safety, efficiency, patient-centredness, and equity. Successfully addressing quality problems requires a much broader, systems-based view of health-care delivery. Although essential to clinical decision-making and practice, EBM cannot act as the cornerstone of health system improvement.

Follow Up of Incidental High-Risk Pulmonary Nodules on Computed Tomography Pulmonary Angiography at Care Transitions

BACKGROUND

Computed tomography pulmonary angiography (CTPA) detects incidental findings that require follow-up. In just over 50% of cases, those incidental findings are pulmonary nodules. Fleischner guidelines recommend that patients with nodules that have a high risk of malignancy should undergo CT follow-up within 3-12 months.

OBJECTIVE

We examined the proportion of patients with pulmonary nodules requiring follow up who received repeat imaging within six weeks of the time frame recommended by the radiologist.

DESIGN

This retrospective cohort study included all patients who underwent CTPA in the emergency department and inpatient settings at three teaching hospitals in Toronto, Canada between September 1, 2014, and August 31, 2015. Natural language processing software was applied to a linked radiology information system to identify all CTPAs that contained pulmonary nodules. Using manual review and prespecified exclusion criteria, we generated a cohort with possible new lung malignancy eligible for follow-up imaging; then we reviewed available health records to determine whether follow-up had occurred.

RESULTS

Of the 1,910 CTPAs performed over the study period, 674 (35.3%) contained pulmonary nodules. Of the 259 patients with new nodules eligible for follow-up imaging, 65 received an explicit suggestion for follow-up by radiology (25.1%). Of these 65 patients, 35 (53.8%) did not receive repeat imaging within the recommended time frame. Explicit mention that follow-up was required in the discharge summary (P = .03), attending an outpatient follow-up visit (P < .001), and younger age (P = .03) were associated with receiving timely follow-up imaging.

CONCLUSIONS

Over 50% of patients with new high-risk pulmonary nodules detected incidentally on CTPA did not receive timely follow-up imaging.

Putting out fires: a qualitative study exploring the use of patient complaints to drive improvement at three academic hospitals

BACKGROUND AND OBJECTIVES

Recent years have seen increasing calls for more proactive use of patient complaints to develop effective system-wide changes, analogous to the intended functions of incident reporting and root cause analysis (RCA) to improve patient safety. Given recent questions regarding the impact of RCAs on patient safety, we sought to explore the degree to which current patient complaints processes generate solutions to recurring quality problems.

DESIGN/SETTING

Qualitative analysis of semistructured interviews with 21 patient relations personnel (PRP), nursing and physician leaders at three teaching hospitals (Toronto, Canada).

RESULTS

Challenges to using the patient complaints process to drive hospital-wide improvement included: (1) Complaints often reflect recalcitrant system-wide issues (eg, wait times) or well-known problems which require intensive efforts to address (eg, poor communication). (2) The use of weak change strategies (eg, one-off educational sessions). (3) The handling of complaints by unit managers so they never reach the patient relations office. PRP identified giving patients a voice as their primary goal. Yet their daily work, which they described as 'putting out fires', focused primarily on placating patients in order to resolve complaints as quickly as possible, which may in effect suppress the patient voice.

CONCLUSIONS

Using patient complaints to drive improvement faces many of the challenges affecting incident reporting and RCA. The emphasis on 'putting out fires' may further detract from efforts to improve care for future patients. Systemically incorporating patients' voices in clinical operations, as with co-design and other forms of authentic patient engagement, may hold greater promise for meaningful improvements in the patient experience than do RCA-like analyses of patient complaints.

Physician Characteristics Associated With Ordering 4 Low-Value Screening Tests in Primary Care

IMPORTANCE

Efforts to reduce low-value tests and treatments in primary care are often ineffective. These efforts typically target physicians broadly, most of whom order low-value care infrequently.

OBJECTIVES

To measure physician-level use rates of 4 low-value screening tests in primary care to investigate the presence and characteristics of primary care physicians who frequently order low-value care.

DESIGN, SETTING, AND PARTICIPANTS

A retrospective cohort study was conducted using administrative health care claims collected between April 1, 2012, and March 31, 2016, in Ontario, Canada. This study measured use of 4 low-value screening tests-repeated dual-energy x-ray absorptiometry (DXA) scans, electrocardiograms (ECGs), Papanicolaou (Pap) tests, and chest radiographs (CXRs)-among low-risk outpatients rostered to a common cohort of primary care physicians.

EXPOSURES

Physician sex, years since medical school graduation, and primary care model.

MAIN OUTCOMES AND MEASURES

This study measured the number of tests to which a given physician ranked in the top quintile by ordering rate. The resulting cross-test score (range, 0-4) reflects a physician's propensity to order low-value care across screening tests. Physicians were then dichotomized into infrequent or isolated frequent users (score, 0 or 1, respectively) or generalized frequent users for 2 or more tests (score, ≥2).

RESULTS

The final sample consisted of 2394 primary care physicians (mean [SD] age, 51.3 [10.0] years; 50.2% female), who were predominantly Canadian medical school graduates (1701 [71.1%]), far removed from medical school graduation (median, 25.3 years; interquartile range, 17.3-32.3 years), and reimbursed via fee-for-service in a family health group (1130 [47.2%]). They ordered 302 509 low-value screening tests (74 167 DXA scans, 179 855 ECGs, 19 906 Pap tests, and 28 581 CXRs) after 3 428 557 ordering opportunities. Within the cohort, generalized frequent users represented 18.4% (441 of 2394) of physicians but ordered 39.2% (118 665 of 302 509) of all low-value screening tests. Physicians who were male (odds ratio, 1.29; 95% CI, 1.01-1.64), further removed from medical school graduation (odds ratio, 1.03; 95% CI, 1.02-1.04), or in an enhanced fee-for-service payment model (family health group) vs a capitated payment model (family health team) (odds ratio, 2.04; 95% CI, 1.42-2.94) had increased odds of being generalized frequent users.

CONCLUSIONS AND RELEVANCE

This study identified a group of primary care physicians who frequently ordered low-value screening tests. Tailoring future interventions to these generalized frequent users might be an effective approach to reducing low-value care.

Do Combined Pharmacist and Prescriber Efforts on Medication Reconciliation Reduce Postdischarge Patient Emergency Department Visits and Hospital Readmissions?

BACKGROUND

Although medication reconciliation (Med Rec) has demonstrated a reduction in potential adverse drug events, its effect on hospital readmissions remains inconclusive.

OBJECTIVE

To evaluate the impact of an interprofessional Med Rec bundle from admission to discharge on patient emergency department visits and hospital readmissions (hospital visits).

METHODS

The design was a retrospective, cohort study. Patients discharged from general internal medicine over a 57-month interval were identified through administrative databases. Patients who received an enhanced, Gold level, Med Rec bundle (including both admission Med Rec and interprofessional pharmacist-prescriber collaboration on discharge Med Rec) were assigned to the intervention group. Patients who received partial Med Rec services, Silver and Bronze level, comprised the control group. The primary outcome was hospital visits within 30 days of discharge.

RESULTS

Over a 57-month period, 9931 unique patient visits (n = 8678 patients) met the study criteria. The main analysis did not detect a difference in 30-day hospital visits between the intervention (Gold level bundle) and control (21.25% vs 19.26%; adjusted odds ratio, 1.06; 95% confidence interval [CI], 0.95-1.19). Propensity score adjustment also did not detect an effect (16.7% vs18.9%; relative risk of readmission, 0.88; 95% CI, 0.59-1.32).

CONCLUSION

A long-term, observational evaluation of interprofessional Med Rec did not detect a difference in 30- day postdischarge patient hospital visits between patients who received enhanced versus partial Med Rec patient care bundles. In future prospective studies, researchers could focus on evaluating high-risk populations and specific elements of Med Rec services on avoidable, medication-related hospital admissions and postdischarge adverse drug events.

Faculty-Resident "Co-learning": A Longitudinal Exploration of an Innovative Model for Faculty Development in Quality Improvement

PURPOSE

To examine the effectiveness of co-learning, wherein faculty and trainees learn together, as a novel approach for building quality improvement (QI) faculty capacity.

METHOD

From July 2012 through September 2015, the authors conducted 30 semistructured interviews with 23 faculty participants from the Co-Learning QI Curriculum of the Department of Medicine, Faculty of Medicine, University of Toronto, and collected descriptive data on faculty participation and resident evaluations of teaching effectiveness. Interviewees were from 13 subspecialty residency programs at their institution.

RESULTS

Of the 56 faculty participants, the Co-Learning QI Curriculum trained 29 faculty mentors, 14 of whom taught formally. Faculty leads with an academic QI role, many of whom had prior QI training, reinforced their QI knowledge while also developing QI mentorship and teaching skills. Co-learning elements that contributed to QI teaching skills development included seeing first how the QI content is taught, learning through project mentorship, building experience longitudinally over time, a graded transition toward independent teaching, and a supportive program lead. Faculty with limited QI experience reported improved QI knowledge, skills, and project facilitation but were ambivalent about assuming a teacher role. Unplanned outcomes for both groups included QI teaching outside of the curriculum, applying QI principles to other work, networking, and strengthening one's QI professional role.

CONCLUSIONS

The Co-Learning QI Curriculum was effective in improving faculty QI knowledge and skills and increased faculty capacity to teach and mentor QI. Findings suggest that a combination of curriculum and contextual factors were critical to realizing the curriculum's full potential.

“Rheum to Improve”: Quality Improvement in Outpatient Rheumatology

The commitment to improve care processes and patient outcomes is a professional mandate for clinicians and is also seen as an operational priority for institutions. Quality improvement now figures in the accreditation of training programs, specialty examinations, and hospital scorecards. Rheumatologists have traditionally focused primarily on quality problems such as guideline adherence; however, improvement goals should also include other aspects of care that are helpful to patients and are professionally rewarding for practitioners. This review makes use of improvement projects in outlining tangible tools rheumatologists can use to resolve quality concerns in their practices.

Do physicians clean their hands? Insights from a covert observational study

Physicians are notorious for poor hand hygiene (HH) compliance. We wondered if lower performance by physicians compared with other health professionals might reflect differences in the Hawthorne effect. We introduced covert HH observers to see if performance differences between physicians and nurses decreased and to gain further insights into physician HH behaviors. Following training and validation with a hospital HH auditor, 2 students covertly measured HH during clinical rotations. Students rotated off clinical services every week to increase exposure to different providers and minimize risk of exposing the covert observation. We compared covertly measured HH compliance with data from overt observation by hospital auditors during the same time period. Covert observation produced much lower HH compliance than recorded by hospital auditors during the same time period: 50.0% (799/1597) versus 83.7% (2769/3309) (P < 0.0002). The difference in physician compliance between hospital auditors and covert observers was 19.0% (73.2% vs 54.2%); for nurses this difference was much higher at 40.7% (85.8% vs 45.1%) (P < 0.0001). Physician trainees showed markedly better compliance when attending staff cleaned their hands compared with encounters when attending did not (79.5% vs 18.9%; P < 0.0002). Our study suggests that traditional HH audits not only overstate HH performance overall, but can lead to inaccurate inferences about performance by professional groupings due to relative differences in the Hawthorne effect. We suggest that future improvement efforts will rely on more accurate HH monitoring systems and strong attending physician leadership to set an example for trainees. Journal of Hospital Medicine 2015;11:862-864. © 2015 Society of Hospital Medicine.

Expanding the scope of Critical Care Rapid Response Teams: a feasible approach to identify adverse events. A prospective observational cohort

INTRODUCTION

Adverse events (AEs) affect 3-12% of hospitalised patients. These are estimates from a labour-intensive chart review process,which is not feasible outside research. Clinical deterioration on the wards triggers a rapid response teams (RRTs) consult and can be used to identify an AE prospectively.

OBJECTIVES

To demonstrate the feasibility of using RRT to detect AEs and compare this methodology to the rates reported using an electronic safety reporting system.

METHODS

Prospective observational cohort of RRT consults. Three independent physicians reviewed all cases for the occurrence of an AE and its preventability. We summarise AEs as rates per 1000 patient-days, and compared the rates between RRT and the safety reporting system using a Poisson model.

RESULTS

There were 8713 hospital admissions, with 531 RRT consults and 247 (2.8%) cases included. Forty-four (17.8%) and 35 cases (14.2%) were judged as AEs and preventable AEs, respectively. RRT identified 0.52 AE/1000 patient-days, compared with 0.21 AE/1000 patient-days detected through the electronic safety reporting system (rate ratio 2.4, 95% CI 1.4 to 4.2, p=0.0014). Patients in surgical wards had more AEs (0.83/1000 vs 0.36/1000, p<0.01) and preventable AEs (0.70 vs 0.21, p<0.01) than patients in medical wards. Agreement for AE (κ 0.46, 95% CI 0.39 to 0.53) and preventable AE (κ 0.47, 95% CI 0.40 to 0.53) was moderate among reviewers.

CONCLUSIONS

Reviewing RRT consults identified a high proportion of AEs and preventable AEs. This methodology detected twice as many AEs as the hospital's safety reporting system. RRT clinicians provide a complementary and more sensitive mechanism than traditional safety reporting systems to identify possible AEs in hospitals.