Quality improvement in radiology: white paper report of the Sun Valley Group meeting

Causes and diagnostic utility of musculoskeletal MRI recall examinations

AIM

To determine the causes and diagnostic utility of musculoskeletal (MSK) magnetic resonance imaging (MRI) recall examinations.

MATERIALS AND METHODS

An institutional review board-approved retrospective review was conducted of all MSK MRI examinations performed at a single academic institution over 10 years where radiologists requested the patient return for additional imaging. The reason for the recall was documented. Recalls were reviewed in consensus by two MSK radiologists to determine whether additional sequences resulted in a change in the final report. Recall causes were divided into four categories: (1) radiologist-related: incorrect field of view (FOV) or incorrect protocol; (2) technologist-related: incorrect FOV or incorrect/incomplete protocol performed, or technically poor-quality images; (3) patient-related motion artefact; (4) unexpected lesion discovered. Fisher's exact test was used to assess for statistical significance.

RESULTS

The recall rate was 0.25% (156/62,930). Of the total 129 recalls returning for imaging, 42 (33%) were radiologist-related, 45 (35%) were technologist-related, six (5%) were patient-related, and 36 (28%) had an unexpected lesion requiring additional sequences. For clinical utility, 42% resulted in a change from the initial report. Recalls due to radiologist error, incorrect FOV, or unexpected lesion caused a significant change in the final report; however, recalls due to technologist error, patient motion artefact, or incorrect protocol did not.

CONCLUSION

MRI MSK recalls are uncommon, and the most common reasons are incorrect FOV, incorrect protocol, and unexpected lesion. Radiologist-related errors in protocols and FOV led to a significant change in the final report and should be targeted as areas for improvement to reduce recall examinations.

Use of a PACS Embedded System for Communicating Radiologist to Technologist Learning Opportunities and Patient Callbacks

OBJECTIVE

This study aimed to determine effect of modality, care setting, and radiology subspecialty on frequency of diagnostic image quality issues identified by radiologists during image interpretation.

METHODS

This Institutional Review Board-exempt retrospective study was performed 10/1/18-6/30/20 at an academic radiology practice performing 700,000+ examinations annually. A closed-loop communication tool integrated in PACS workflow enabled radiologists to alert technologists to image quality issues. Radiologists categorized communications as requiring patient callback, or as technologist learning opportunities if image quality was adequate to generate a diagnostic report. Fisher's exact test assessed impact of imaging modality, radiology subspecialty, and care setting on radiologist-identified image quality issues.

RESULTS

976,915 imaging examinations were performed during the study period. Radiologists generated 1,935 technologist learning opportunities (0.20%) and 208 callbacks (0.02%). Learning opportunity rates were highest for MRI (0.60%) when compared to CT (0.26%) and radiography (0.08%) (p<0.0001). The same was true for patient callbacks (MRI 0.13%, CT 0.02%, radiography 0.0006%; p<0.0001). Outpatient examinations generated more learning opportunities (1479/637,092; 0.23%) vs. inpatient (305/200,206; 0.15%) and Emergency Department (151/139,617; 0.11%) (p<0.0001). Abdominal subspecialists were most likely to generate learning opportunities when compared to other subspecialists and cardiovascular imagers were most likely to call a patient back.

CONCLUSIONS

Image quality issues identified by radiologists during the interpretation process were rare and 10 times more commonly categorized as learning opportunities not interfering with a clinically adequate report than as requiring patient callback. Further work is necessary to determine if creating learning opportunities leads to fewer patients requiring repeat examinations.

Assessing information sources to elucidate diagnostic process errors in radiologic imaging - a human factors framework

Objective

To assess information sources that may elucidate errors related to radiologic diagnostic imaging, quantify the incidence of potential safety events from each source, and quantify the number of steps involved from diagnostic imaging chain and socio-technical factors.

Materials and Methods

This retrospective, Institutional Review Board-approved study was conducted at the ambulatory healthcare facilities associated with a large academic hospital. Five information sources were evaluated: an electronic safety reporting system (ESRS), alert notification for critical result (ANCR) system, picture archive and communication system (PACS)-based quality assurance (QA) tool, imaging peer-review system, and an imaging computerized physician order entry (CPOE) and scheduling system. Data from these sources (January-December 2015 for ESRS, ANCR, QA tool, and the peer-review system; January-October 2016 for the imaging ordering system) were collected to quantify the incidence of potential safety events. Reviewers classified events by the step(s) in the diagnostic process they could elucidate, and their socio-technical factors contributors per the Systems Engineering Initiative for Patient Safety (SEIPS) framework.

Results

Potential safety events ranged from 0.5% to 62.1% of events collected from each source. Each of the information sources contributed to elucidating diagnostic process errors in various steps of the diagnostic imaging chain and contributing socio-technical factors, primarily Person, Tasks, and Tools and Technology.

Discussion

Various information sources can differentially inform understanding diagnostic process errors related to radiologic diagnostic imaging.

Conclusion

Information sources elucidate errors in various steps within the diagnostic imaging workflow and can provide insight into socio-technical factors that impact patient safety in the diagnostic process.

Classifying Safety Events Related to Diagnostic Imaging From a Safety Reporting System Using a Human Factors Framework

Purpose:

To measure diagnostic imaging safety events reported to an electronic safety reporting system (ESRS) and assess steps where they occurred within the diagnostic imaging workflow and contributing socio-technical factors.

Methods:

We evaluated all ESRS safety reports related to diagnostic imaging during calendar 2015 at an academic medical center with 50,000 admissions, 950,000 ambulatory visits, and performing 680,000 diagnostic imaging studies annually. Each report was assigned a 0-4 harm score by the reporter; we classified scores of 2 (minor harm) to 4 (death) as “potential harm”. Two reviewers manually classified reports into steps involved in the diagnostic imaging chain and socio-technical factors per the Systems Engineering Initiative for Patient Safety (SEIPS) framework. Kappa measured inter-reviewer agreement on 10% of reports. The percentage of reports that could cause “potential harm” was compared for each step and socio-technical factor using chi-square analysis.

Results:

Of 11,570 safety reports submitted in 2015, 854 (7%) were related to diagnostic imaging. Although the most common step was Imaging Procedure (54% of reports), potential harm occurred more in Report Communication (Odds Ratio=2.36, p=0.05). Person factors most commonly contributed to safety reports (71%). Potential harm occurred more in safety reports that were related to Task compared to Person factors (OR=5.03, p<0.0001). Kappa was 0.79.

Conclusion:

Safety events were related to diagnostic imaging in 7% of reports and potential harm occurred primarily during Imaging Procedure and Report Communication. Safety events were attributed to multifactorial socio-technical factors. Further work is necessary to decrease safety events related to diagnostic imaging.

Online Error Reporting for Managing Quality Control Within Radiology

Information technology systems within health care, such as picture archiving and communication system (PACS) in radiology, can have a positive impact on production but can also risk compromising quality. The widespread use of PACS has removed the previous feedback loop between radiologists and technologists. Instead of direct communication of quality discrepancies found for an examination, the radiologist submitted a paper-based quality-control report. A web-based issue-reporting tool can help restore some of the feedback loop and also provide possibilities for more detailed analysis of submitted errors. The purpose of this study was to evaluate the hypothesis that data from use of an online error reporting software for quality control can focus our efforts within our department. For the 372,258 radiologic examinations conducted during the 6-month period study, 930 errors (390 exam protocol, 390 exam validation, and 150 exam technique) were submitted, corresponding to an error rate of 0.25 %. Within the category exam protocol, technologist documentation had the highest number of submitted errors in ultrasonography (77 errors [44 %]), while imaging protocol errors were the highest subtype error for computed tomography modality (35 errors [18 %]). Positioning and incorrect accession had the highest errors in the exam technique and exam validation error category, respectively, for nearly all of the modalities. An error rate less than 1 % could signify a system with a very high quality; however, a more likely explanation is that not all errors were detected or reported. Furthermore, staff reception of the error reporting system could also affect the reporting rate.

Radiology Research in Quality and Safety: Current Trends and Future Needs

Promoting quality and safety research is now essential for radiology as reimbursement is increasingly tied to measures of quality, patient safety, efficiency, and appropriateness of imaging. This article provides an overview of key features necessary to promote successful quality improvement efforts in radiology. Emphasis is given to current trends and future opportunities for directing research. Establishing and maintaining a culture of safety is paramount to organizations wishing to improve patient care. The correct culture must be in place to support quality initiatives and create accountability for patient care. Focused educational curricula are necessary to teach quality and safety-related skills and behaviors to trainees, staff members, and physicians. The increasingly complex healthcare landscape requires that organizations build effective data infrastructures to support quality and safety research. Incident reporting systems designed specifically for medical imaging will benefit quality improvement initiatives by identifying and learning from system errors, enhancing knowledge about safety, and creating safer systems through the implementation of standardized practices and standards. Finally, validated performance measures must be developed to accurately reflect the value of the care we provide for our patients and referring providers. Common metrics used in radiology are reviewed with focus on current and future opportunities for investigation.

Key Principles in Quality and Safety in Radiology

OBJECTIVE

The purpose of this article is to introduce the reader to basic concepts of quality and safety in radiology.

CONCLUSION

Concepts are introduced that are keys to identifying, understanding, and utilizing certain quality tools with the aim of making process improvements. Challenges, opportunities, and change drivers can be mapped from the radiology quality perspective. Best practices, informatics, and benchmarks can profoundly affect the outcome of the quality improvement initiative we all aim to achieve.

Teaching Critical Thinking in Graduate Medical Education: Lessons Learned in Diagnostic Radiology

The 2014 Institute of Medicine report, Graduate Medical Education that Meets the Nation's Health Needs, challenged the current graduate medical training process and encouraged new opportunities to redefine the fundamental skills and abilities of the physician workforce. This workforce should be skilled in critically evaluating the current systems to improve care delivery and health. To meet these goals, current challenges, motivations, and educational models at the medical school and graduate medical education levels related to formal training in nonclinical aspects of medicine, especially critical thinking, are reviewed. Our diagnostic radiology training program is presented as a "case study" to frame the review.

Improving Procedure Start Times and Decreasing Delays in Interventional Radiology: A Department's Quality Improvement Initiative

RATIONALE AND OBJECTIVES

To identify and reduce reasons for delays in procedure start times, particularly the first cases of the day, within the interventional radiology (IR) divisions of the Department of Radiology using principles of continuous quality improvement.

MATERIALS AND METHODS

An interdisciplinary team representative of the IR and preprocedure/postprocedure care area (PPCA) health care personnel, managers, and data analysts was formed. A standardized form was used to document both inpatient and outpatient progress through the PPCA and IR workflow in six rooms and to document reasons for delays. Data generated were used to identify key problems areas, implement improvement interventions, and monitor their effects. Project duration was 6 months.

RESULTS

The average number of on-time starts for the first case of the day increased from 23% to 56% (P value < .01). The average number of on-time, scheduled outpatients increased from 30% to 45% (P value < .01). Patient wait time to arrive at treatment room once they were ready for their procedure was reduced on average by 10 minutes (P value < .01). Patient care delay duration per 100 patients was reduced from 30.3 to 21.6 hours (29% reduction). Number of patient care delays per 100 patients was reduced from 46.6 to 40.1 (17% reduction). Top reasons for delay included waiting for consent (26% of delays duration) and laboratory tests (12%).

CONCLUSIONS

Many complex factors contribute to procedure start time delays within an IR practice. A data-driven and patient-centered, interdisciplinary team approach was effective in reducing delays in IR.

Advancing Patient-centered Outcomes in Emergency Diagnostic Imaging: A Research Agenda

Diagnostic imaging is integral to the evaluation of many emergency department (ED) patients. However, relatively little effort has been devoted to patient-centered outcomes research (PCOR) in emergency diagnostic imaging. This article provides background on this topic and the conclusions of the 2015 Academic Emergency Medicine consensus conference PCOR work group regarding "Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization." The goal was to determine a prioritized research agenda to establish which outcomes related to emergency diagnostic imaging are most important to patients, caregivers, and other key stakeholders and which methods will most optimally engage patients in the decision to undergo imaging. Case vignettes are used to emphasize these concepts as they relate to a patient's decision to seek care at an ED and the care received there. The authors discuss applicable research methods and approaches such as shared decision-making that could facilitate better integration of patient-centered outcomes and patient-reported outcomes into decisions regarding emergency diagnostic imaging. Finally, based on a modified Delphi process involving members of the PCOR work group, prioritized research questions are proposed to advance the science of patient-centered outcomes in ED diagnostic imaging.

Performance measures in radiology

Performance measures in radiology play an increasingly significant role in health care quality assessment and now form the basis for a variety of pay-for-performance programs, including those administered by CMS. This article introduces the measure development process, beginning with topic selection, followed by measure development and testing, National Quality Forum endorsement, and implementation. Once implemented, measures may undergo further testing and be re-endorsed, modified, or retired. Radiologists should familiarize themselves with the measures relevant to their practice, develop ways to collect and report data efficiently, and implement the necessary practice changes to meet measure criteria and improve the quality of their practice.

CT of acute appendicitis: can diagnostic accuracy serve as a practical performance metric for readers specialized in abdominal imaging?

PURPOSE

To investigate diagnostic accuracy for acute appendicitis at computed tomography (CT) as a performance metric for radiologists specialized in abdominal imaging.

MATERIALS AND METHODS

We retrospectively identified six attending abdominal imagers who each independently interpreted over 100 CT studies for suspected acute appendicitis.

RESULTS

The mean number of studies per reader was 311 (range, 129-386). Mean reader diagnostic accuracy was 95.0% (range, 91.4-97.1%). Only one had a diagnostic accuracy (91.4%) that was significantly lower than all others.

CONCLUSION

Diagnostic accuracy for acute appendicitis at CT may be an impractical performance metric for radiologists specialized in abdominal imaging.

Using health services research to meet ACGME resident research requirements

Health services research is a field of research that crosses many disciplines. It represents a novel way to address the Accreditation Council for Graduate Medical Education requirements for scholarly activity of residents and faculty in academic radiology departments. In addition to meeting training requirements, it offers future radiologists the opportunity to develop skills that will be essential as we transition from a volume based health care delivery system to an outcome based delivery system. This report introduces examples of health services research projects and the types of data resources that are available to facilitate such projects.

Global quality imaging: emerging issues

Quality imaging may be described as "a timely access to and delivery of integrated and appropriate procedures, in a safe and responsive practice, and a prompt delivery of an accurately interpreted report by capable personnel in an efficient, effective, and sustainable manner." For this article, radiation safety is considered as one of the key quality elements. The stakeholders are the drivers of quality imaging. These include those that directly provide or use imaging procedures and others indirectly supporting the system. Imaging is indispensable in health care, and its use has greatly expanded worldwide. Globalization, consumer sophistication, communication and technological advances, corporatization, rationalization, service outsourcing, teleradiology, workflow modularization, and commoditization are reshaping practice. This article defines the emerging issues; an earlier article in the May 2011 issue described possible improvement actions. The issues that could threaten the quality use of imaging for all countries include workforce shortage; increased utilization, population radiation exposure, and cost; practice changes; and efficiency drive and budget constraints. In response to these issues, a range of quality improvement measures, strategies, and actions are used to maximize the benefits and minimize the risks. The 3 measures are procedure justification, optimization of image quality and radiation protection, and error prevention. The development and successful implementation of such improvement actions require leadership, collaboration, and the active participation of all stakeholders to achieve the best outcomes that we all advocate.

Tracking delays in report availability caused by incorrect exam status with Web-based issue tracking: a quality initiative

Many radiology information systems (RIS) cannot accept a final report from a dictation reporting system before the exam has been completed in the RIS by a technologist. A radiologist can still render a report in a reporting system once images are available, but the RIS and ancillary systems may not get the results because of the study's uncompleted status. This delay in completing the study caused an alarming number of delayed reports and was undetected by conventional RIS reporting techniques. We developed a Web-based reporting tool to monitor uncompleted exams and automatically page section supervisors when a report was being delayed by its incomplete status in the RIS. Institutional Review Board exemption was obtained. At four imaging centers, a Python script was developed to poll the dictation system every 10 min for exams in five different modalities that were signed by the radiologist but could not be sent to the RIS. This script logged the exams into an existing Web-based tracking tool using PHP and a MySQL database. The script also text-paged the modality supervisor. The script logged the time at which the report was finally sent, and statistics were aggregated onto a separate Web-based reporting tool. Over a 1-year period, the average number of uncompleted exams per month and time to problem resolution decreased at every imaging center and in almost every imaging modality. Automated feedback provides a vital link in improving technologist performance and patient care without assigning a human resource to manage report queues.

Establishing a CT colonography service

This article describes the steps involved in establishing a screening computed tomographic colonography (CTC) practice and integrating that practice within a gastroenterology practice. The standard operating procedures followed at the National Naval Medical Center's Colon Health Initiative are presented and are followed by a discussion of practical aspects of establishing a CTC practice, such as equipment specifications, CTC performance, and interpretation training requirements for radiologists and nonradiologists. Regulatory considerations involved in establishing a screening CTC program are examined along with the salient features of a CTC business plan. Finally, reimbursement issues, quality control, and the potential impact of screening CTC on colonoscopy practice are discussed.

IRQN Award Article: Labeling Defects in Radiology

The Institute of Medicine's 1999 report on medical errors in the American health care system, To Err Is Human, and the follow-up report in 2001, Crossing the Quality Chasm, have focused the public eye on improving quality in medicine. Governmental agencies such as the Centers for Medicare and Medicaid Services have mandated the development of metrics to measure the quality of care. The Sun Valley Group is an informal assembly of individuals interested in improving quality in radiology. One of the initiatives that arose from the initial meeting of that group was to pilot programs in quality metric measurement. One of these programs was to focus on a safety initiative on correct image labeling. This is the report of that pilot project.

Leadership and management in quality radiology

The practice of medical imaging and interventional radiology are undergoing rapid change in recent years due to technological advances, workload escalation, workforce shortage, globalisation, corporatisation, commercialisation and commoditisation of healthcare. These professional and economical changes are challenging the established norm but may bring new opportunities. There is an increasing awareness of and interest in the quality of care and patient safety in medical imaging and interventional radiology. Among the professional organisations, a range of quality systems are available to address individual, facility and system needs. To manage the limited resources successfully, radiologists and professional organisations must be leaders and champion for the cause of quality care and patient safety. Close collaboration with other stakeholders towards the development and management of proactive, long-term, system-based strategies and infrastructures will underpin a sustainable future in quality radiology. The International Radiology Quality Network can play a useful facilitating role in this worthwhile but challenging endeavour.

Defining Quality in Radiology

The introduction of pay for performance in medicine represents an opportunity for radiologists to define quality in radiology. Radiology quality can be defined on the basis of the production model that currently drives reimbursement, codifying the role of radiologists as being limited to the production of timely and accurate radiology reports produced in conditions of maximum patient safety and communicated in a timely manner. Alternately, quality in radiology can also encompass the professional role of radiologists as diagnostic imaging specialists responsible for the appropriate use, selection, interpretation, and application of imaging. Although potentially challenging to implement, the professional model for radiology quality is a comprehensive assessment of the ways in which radiologists add value to patient care. This essay is a discussion of the definition of radiology quality and the implications of that definition.