Moral Distress, Moral Injury, and Burnout in Radiology Practice

Moral distress, which causes burnout, is a growing issue in health care since its initial description in 1984. The relationship among moral distress, moral injury (sustained moral distress), and burnout is critical to understanding the implications for physicians' mental and physical health and the impact on patient care. Moral distress can lead to an increase in medical errors and result in low quality of care for patients. There are five common causes of moral distress in radiology that can affect patient care. These include high workload, lack of leadership support, clinical demands interfering with teaching mission, lack of team communication, and disregard for professional expertise by pressuring radiologists to perform unnecessary or inappropriate imaging. This article analyzes current work environment challenges contributing to these issues, including causes of high workload, staffing crisis in radiology, and lack of time for nonclinical missions (eg, teaching, research, continuing medical education, practice building, reading literature, mentoring, and society volunteering). Moral distress and intention to leave were compared between radiology and other specialties. Concrete solutions to address causes of moral distress are outlined. These solutions include developing guidelines for safe workloads, servant leadership models, and tips for maintaining the teaching mission in a busy academic work environment and improving communication between clinicians. Possible solutions to national problems such as high workload, reduction of inappropriate imaging, seeking reimbursement for noninterpretative tasks, and short staffing are also described.

Interventions to improve timely cancer diagnosis: an integrative review

Cancer will affect more than one in three U.S. residents in their lifetime, and although the diagnosis will be made efficiently in most of these cases, roughly one in five patients will experience a delayed or missed diagnosis. In this integrative review, we focus on missed opportunities in the diagnosis of breast, lung, and colorectal cancer in the ambulatory care environment. From a review of 493 publications, we summarize the current evidence regarding the contributing factors to missed or delayed cancer diagnosis in ambulatory care, as well as evidence to support possible strategies for intervention. Cancer diagnoses are made after follow-up of a positive screening test or an incidental finding, or most commonly, by following up and clarifying non-specific initial presentations to primary care. Breakdowns and delays are unacceptably common in each of these pathways, representing failures to follow-up on abnormal test results, incidental findings, non-specific symptoms, or consults. Interventions aimed at 'closing the loop' represent an opportunity to improve the timeliness of cancer diagnosis and reduce the harm from diagnostic errors. Improving patient engagement, using 'safety netting,' and taking advantage of the functionality offered through health information technology are all viable options to address these problems.

Use of ChatGPT Large Language Models to Extract Details of Recommendations for Additional Imaging From Free-Text Impressions of Radiology Reports

BACKGROUND. Automated extraction of actionable details of recommendations for additional imaging (RAIs) from radiology reports could facilitate tracking and timely completion of clinically necessary RAIs and thereby potentially reduce diagnostic delays. OBJECTIVE. The purpose of the study was to assess the performance of large language models (LLMs) in extracting actionable details of RAIs from radiology reports. METHODS. This retrospective single-center study evaluated reports of diagnostic radiology examinations performed across modalities and care settings within five subspecialties (abdominal imaging, musculoskeletal imaging, neuroradiology, nuclear medicine, thoracic imaging) in August 2023. Of reports identified by a previously validated natural language processing algorithm to contain an RAI, 250 were randomly selected; 231 of these reports were confirmed to contain an RAI on manual review and formed the study sample. Twenty-five reports were used to engineer a prompt instructing an LLM, when inputted in a report impression containing an RAI, to extract details about the modality, body part, time frame, and rationale of the RAI; the remaining 206 reports were used for testing the prompt in combination with GPT-3.5 and GPT-4. A 4th-year medical student and radiologist from the relevant subspecialty independently classified the LLM outputs as correct versus incorrect for extracting the four actionable details of RAIs in comparison with the report impressions; a third reviewer assisted in resolving discrepancies. Extraction accuracy was summarized and compared between LLMs using consensus assessments. RESULTS. For GPT-3.5 and GPT-4, the two reviewers agreed about classification of LLM output as correct versus incorrect with respect to report impressions for 95.6% and 94.2% for RAI modality, 89.3% and 88.3% for RAI body part, 96.1% and 95.1% for RAI time frame, and 89.8% and 88.8% for RAI rationale, respectively. GPT-4 was more accurate than GPT-3.5 in extracting RAI modality (94.2% [194/206] vs 85.4% [176/206], p < .001), RAI body part (86.9% [179/206] vs 77.2% [159/206], p = .004), and RAI time frame (99.0% [204/206] vs 95.6% [197/206], p = .02). Both LLMs had accuracy of 91.7% (189/206) for extracting RAI rationale. CONCLUSION. LLMs were used to extract actionable details of RAIs from free-text impression sections of radiology reports; GPT-4 outperformed GPT-3.5. CLINICAL IMPACT. The technique could represent an innovative method to facilitate timely completion of clinically necessary radiologist recommendations.

Patient Photograph Association With Radiologist Recommendations for Additional Imaging

OBJECTIVE

Assess whether display of a patient photograph in the electronic health record (EHR) alongside head and neck CT or MRI radiology examinations is associated with recommendations for additional imaging (RAI) and whether self-reported race modifies that association.

METHODS

This multi-institution health care system retrospective observational study from June 1, 2021 to May 31, 2022 included all patients with a head/neck CT or MRI report. We investigated association of photograph with RAIs using mixed-effects models adjusting for age, sex, complexity score, race, and area deprivation index while conditioning on patient and radiologist. Race was subsequently included as an interaction term. Multiple imputation was used as sensitivity analysis to address missing race data.

RESULTS

In all, 60,543 reports were included from 48,143 patients (55.6% female; median age 58 years, interquartile range 40-70). The EHR included a photograph at the time 18.2% (11,048 of 60,543) of reports were signed. RAIs were included in 7.5% (4,522 of 60,543) of reports. Reports signed when a photograph was displayed had lower estimated odds of containing RAIs (odds ratio: 0.85, 95% confidence interval: 0.77-0.93, P < .001), consistent in sensitivity analysis, with no clear interaction between race and photograph (odds ratio: 0.99, 95% confidence interval: 0.69-1.46, P = .97).

DISCUSSION

Patients with a photograph in the EHR had a lower probability of receiving RAIs and this difference did not seem to be the result of implicit racial bias but may be due to personalization of the encounter. This effect may influence radiology reporting for millions of patients per year. Further research is needed to determine whether the association has a positive or negative impact on care quality and outcomes.

Incidental findings in research brain MRI: Definition, prevalence and ethical implications

Radiological incidental findings (IFs) are previously undetected abnormalities which are unrelated to the original indication for imaging and are unexpectedly discovered. In brain magnetic resonance imaging (MRI), the prevalence of IFs is increasing. By reviewing the literature on IFs in brain MRI performed for research purposes and discussing ethical considerations of IFs, this paper provides an overview of brain IF research results and factors contributing to inconsistencies and considers how the consent process can be improved from an ethical perspective. We found that despite extensive literature regarding IFs in research MRI of the brain, there are major inconsistencies in the reported prevalence, ranging from 1.3% to 99%. Many factors appear to contribute to this broad range: lack of standardised definition, participant demographics variance, heterogenous MRI scanner strength and sequences, reporter variation and results classification. We also found significant discrepancies in the review, consent and clinical communication processes pertaining to the ethical nature of these studies. These findings have implications for future studies, particularly those involving artificial intelligence. Further research, particularly in relation to MRI brain IFs would be useful to explore the generalisability of study results.

Communication of Incidental Imaging Findings on Inpatient Discharge Summaries After Implementation of Electronic Health Record Notification System

OBJECTIVES

Inadequate follow-up of incidental imaging findings (IIFs) can result in poor patient outcomes, patient dissatisfaction, and provider malpractice. At our institution, radiologists flag IIFs during report dictation to trigger electronic health record (EHR) notifications to providers and patients. Nurse coordinators directly contact patients or their primary care physicians (PCPs) regarding IIFs if follow-up is not completed within the recommended time frame. Despite these interventions, many patients and their PCPs remain unaware of IIFs. In an effort to improve awareness of IIFs, we aim to investigate communication of IIFs on inpatient discharge summaries after implementation of our EHR notification system.

METHODS

Inpatient records with IIFs from 2018 to 2021 were retrospectively reviewed to determine type of IIFs, follow-up recommendations, and mention of IIFs on discharge summaries. Nurse coordinators spoke to patients and providers to determine their awareness of IIFs.

RESULTS

Incidental imaging findings were reported in 51% of discharge summaries (711/1383). When nurse coordinators called patients and PCPs regarding IIFs at the time follow-up was due, the patients and PCPs were aware of 79% of IIFs (1096/1383).

CONCLUSIONS

With implementation of EHR notifications to providers regarding IIFs, IIFs were included in 51% of discharge summaries. Lack of inclusion of IIFs on discharge summaries could be related to transitions of care within hospitalization, provider alert fatigue, and many diagnostic testing results to distill. These findings demonstrate the need to improve communication of IIFs, possibly via automating mention of IIFs on discharge summaries, and the need for care coordinators to follow up on IIFs.

The Impact of Closed-Loop Imaging on Actionable CT-Detected Breast Findings

PURPOSE

Closed-loop imaging programs (CLIPs) are designed to ensure that patients receive appropriate follow-up, but a review of incidental CT-detected breast findings in the setting of CLIPs has not been performed.

METHODS

A retrospective review was conducted of CT reports at a single academic institution from July 1, 2020, to January 31, 2022, to identify reports with recommendations for breast imaging follow-up. Medical records were reviewed to evaluate patient adherence to follow-up, CLIP intervention, subsequent BI-RADS assessment, and diagnosis. Adherence was defined as diagnostic breast imaging performed within 6 months of the CT recommendation.

RESULTS

Follow-up recommendations for breast imaging were included in CT report impressions for 311 patients. Almost half of patients (47.3% [147 of 311]) underwent follow-up breast imaging within 6 months, yielding breast cancer diagnoses in 12.9% (19 of 147) and a biopsy-proven positive predictive value of 65.5% (19 of 29). Most patients who returned for follow-up within 6 months did so without CLIP intervention. The majority of CT report impressions in the follow-up group (85.0% [125 of 147]) contained specific recommendations for "diagnostic breast imaging." For patients who did not receive follow-up, the CLIP team tracked all cases and intervened in 19.1% (28 of 147). The most common intervention was a phone call and/or fax to the primary care provider. Outpatient CT examination setting and specific recommendation for diagnostic breast imaging were significantly associated with higher follow-up adherence (P < .0001).

CONCLUSIONS

Actionable CT-detected breast findings require follow-up diagnostic breast imaging because of a relevant cancer detection rate of 12.9%. Although many patients return for breast imaging without intervention, almost half of patients did not receive follow-up and may account for a significant number of missed cancer diagnoses. Specific CT recommendation verbiage is associated with higher follow-up adherence, which can be addressed across settings even without CLIPs.

Recommendations for Additional Imaging on Head and Neck Imaging Examinations: Interradiologist Variation and Associated Factors

BACKGROUND. A paucity of relevant guidelines may lead to pronounced variation among radiologists in issuing recommendations for additional imaging (RAI) for head and neck imaging. OBJECTIVE. The purpose of this article was to explore associations of RAI for head and neck imaging examinations with examination, patient, and radiologist factors and to assess the role of individual radiologist-specific behavior in issuing such RAI. METHODS. This retrospective study included 39,200 patients (median age, 58 years; 21,855 women, 17,315 men, 30 with missing sex information) who underwent 39,200 head and neck CT or MRI examinations, interpreted by 61 radiologists, from June 1, 2021, through May 31, 2022. A natural language processing (NLP) tool with manual review of NLP results was used to identify RAI in report impressions. Interradiologist variation in RAI rates was assessed. A generalized mixed-effects model was used to assess associations between RAI and examination, patient, and radiologist factors. RESULTS. A total of 2943 (7.5%) reports contained RAI. Individual radiologist RAI rates ranged from 0.8% to 22.0% (median, 7.1%; IQR, 5.2-10.2%), representing a 27.5-fold difference between minimum and a maximum values and 1.8-fold difference between 25th and 75th percentiles. In multivariable analysis, RAI likelihood was higher for CTA than for CT examinations (OR, 1.32), for examinations that included a trainee in report generation (OR, 1.23), and for patients with self-identified race of Black or African American versus White (OR, 1.25); was lower for male than female patients (OR, 0.90); and was associated with increasing patient age (OR, 1.09 per decade) and inversely associated with radiologist years since training (OR, 0.90 per 5 years). The model accounted for 10.9% of the likelihood of RAI. Of explainable likelihood of RAI, 25.7% was attributable to examination, patient, and radiologist factors; 74.3% was attributable to radiologist-specific behavior. CONCLUSION. Interradiologist variation in RAI rates for head and neck imaging was substantial. RAI appear to be more substantially associated with individual radiologist-specific behavior than with measurable systemic factors. CLINICAL IMPACT. Quality improvement initiatives, incorporating best practices for incidental findings management, may help reduce radiologist preference-sensitive decision-making in issuing RAI for head and neck imaging and associated care variation.

Impact of Early Direct Patient Notification on Follow-Up Completion for Nonurgent Actionable Incidental Radiologic Findings

PURPOSE

The aim of this study was to evaluate whether early direct patient notification in addition to an existing multistage recommendation-tracking system (Backstop) increases follow-up completion rates for actionable incidental findings (AIFs). Patient attitudes toward early notification were also assessed.

METHODS

This prospective, randomized controlled trial recruited patients with AIFs requiring follow-up being enrolled into the Backstop system. Patients were randomized into four groups: those receiving additional early direct notification in a mailed letter (group 1, similar to Pennsylvania Act 112), by phone (group 2), or in an electronic portal message (group 3) and a control group (group 4) without additional notifications added to the existing Backstop system. Differences in follow-up completion rates among these groups were determined using χ2 tests. Patients were surveyed on binary yes/no and Likert-type scale questions, and descriptive statistics are reported.

RESULTS

Data from 2,548 randomized patients were analyzed for the study, including 593 patients notified by letter, 637 notified by phone, 701 notified by portal, and 617 control patients. Group 3 demonstrated the lowest rate of follow-up completion within 1 month of the follow-up due date at 36.4%, compared with 58.7% for group 1, 60.4% for group 2, and 53.2% for group 4 (P < .0001 for all). Group 2 was the only group to have a significantly higher completion rate than group 4 (P = .014). Patients responded positively regarding early notification and preferred electronic portal communication.

CONCLUSIONS

Early direct notification had a mixed impact on follow-up completion rates on the basis of communication modality but was positively received by patients and may have health care benefits when implemented within a recommendation-tracking system.

Medical Malpractice and Diagnostic Radiology: Challenges and Opportunities

Medicolegal challenges in radiology are broad and impact both radiologists and patients. Radiologists may be affected directly by malpractice litigation or indirectly due to defensive imaging ordering practices. Patients also could be harmed physically, emotionally, or financially by unnecessary tests or procedures. As technology advances, the incorporation of artificial intelligence into medicine will bring with it new medicolegal challenges and opportunities. This article reviews the current and emerging direct and indirect effects of medical malpractice on radiologists and summarizes evidence-based solutions.

Revenue and Cost Analysis of a System Utilizing Natural Language Processing and a Nurse Coordinator for Radiology Follow-up Recommendations

Radiology reports often contain recommendations for follow-up imaging, Provider adherence to these radiology recommendations can be incomplete, which may result in patient harm, lost revenue, or litigation. This study sought to perform a revenue assessment of a hybrid natural language processing (NLP) and human follow-up system. Reports generated from January 2020 to April 2021 that were indexed as overdue from follow-up recommendations by mPower Follow-Up Recommendation Algorithm (Nuance Communications Inc., Burlington, MA), were assessed for follow up and revenue. Follow-up exams completed because of the hybrid system were tabulated and given revenue amounts based on Medicare national reimbursement rates. These rates were then summated. A total of n =3011 patients were flagged via the mPower algorithm as having not received a timely follow-up indicated for procedure. Of these, n = 427 required the quality nurse to contact their healthcare provider to place orders. The follow-up imaging of these patients accounted for $62,937.66 of revenue. This revenue was calculated as higher than personnel cost (based on national average quality and safety nurse salary and time allotted on follow-ups). Our results indicate that a hybrid human-artificial intelligence follow-up system can be profitable, while potentially adding to patient safety. Our revenue figure likely significantly underestimates the true revenue obtained at our institution. This was due to the use of Medicare national reimbursement rates to calculate revenue, for the purposes of generalizability.

Applying Implementation Science Principles to Systematize High-Quality Care for Potentially Significant Imaging Findings

OBJECTIVE

Use principles of implementation science to improve the diagnosis and management of potentially significant imaging findings.

METHODS

Multidisciplinary stakeholders codified the diagnosis and management of potentially significant imaging findings in eight organs and created a finding tracking management system that was embedded in radiologist workflows and IT systems. Radiologists were trained to use this system. An automated finding tracking management system was created to support consistent high-quality care through care pathway visualizations, increased awareness of specific findings in the electronic medical record, templated notifications, and creation of an electronic safety net. Primary outcome was the rate of quality reviews related to eight targeted imaging findings. Secondary outcome was radiologist use of the finding tracking management tool.

RESULTS

In the 4 years after implementation, the tool was used to track findings in 7,843 patients who received 10,015 ultrasound, CT, MRI, x-ray, and nuclear medicine examinations that were interpreted by all 34 radiologists. Use of the tool lead to a decrease in related quality reviews (from 8.0% to 0.0%, P < .007). Use of the system increased from 1.7% of examinations in the early implementation phase to 3.1% (+82%, P < .00001) in the postimplementation phase. Each radiologist used the tool on an average of 294.6 unique examinations (SD 404.8). Overall, radiologists currently use the tool approximately 4,000 times per year.

DISCUSSION

Radiologists frequently used a finding tracking management system to ensure effective communication and raise awareness of the importance of recommended future follow-up studies. Use of this system was associated with a decrease in the rate of quality review requests in this domain.

Implementation of Incidental Liver Lesion Clinically Integrated Workflow Increases Compliance With ACR Follow-Up Guidelines, Closing Care Gaps

OBJECTIVE

Follow established management guidelines from the ACR and improve adherence to follow-up recommendations for incidental liver lesions (ILLs) for all patients undergoing CT abdomen and pelvis with contrast (CTAPw) examinations, with advocacy from a multidisciplinary care team.

METHODS

A mandatory structured radiology reporting module was developed for use in CTAPw reports for ILL recommendations. Data from the electronic medical record describing patients with radiology-reported ILLs and their clinical risk diagnosis categories were tabulated in a queryable electronic database. A nurse co-ordinator initiated workflow to communicate the need for ILL follow-up MRI to ordering physicians and primary care providers. MRIs were ordered by the ILL team. An interactive process was undertaken with continuous review to improve identification of eligible patients and adherence to recommendations.

RESULTS

During the initial launch phase from December 2020 to March 2021, 1,577 ILLs were detected on 20,667 CTAPw examinations, and for those with the characterize now recommendation, 36 of 114 (31.6%) received follow-up in 30 days. Between January 2021 and June 2022, 117,520 CTAPws were performed and 4,371 ILLs were detected. Using the ILL workflow, in the MRI now cohort, follow-up occurred within 30 days in 202 of 542 (36.2%) patients, and a total of 368 of 542 (67.9%) patients have completed their follow-up to date.

DISCUSSION

Using a focused effort to close a gap in ILL care, adherence to follow-up recommendations improved over the long term, although there remains a gap in adherence to short-term interventions. A multidisciplinary approach, radiology reporting, and software solutions were leveraged to improve a complex process.

Patient-level factors influencing adherence to follow-up imaging recommendations

PURPOSE

To determine which, if any, patient-level factors were associated with differences in completion of follow-up imaging recommendations at a tertiary academic medical center.

METHODS

In this IRB-approved, retrospective cohort study, approximately one month of imaging recommendations were reviewed from 2017 at a single academic institution that contained key words recommending follow-up imaging. Age, gender, race/ethnicity, insurance, smoking history, primary language, BMI, and home address were recorded via chart extraction. Home addresses were geocoded to Census Block Groups and assigned to a quintile of neighborhood socioeconomic status. A multivariate logistic regression model was used to evaluate each predictor variable with significance set to p = 0.05.

RESULTS

A total of 13,421 imaging reports that included additional follow-up recommendations were identified. Of the 1013 included reports that recommended follow-up, 350 recommended additional imaging and were analyzed. Three hundred eight (88.00%) had corresponding follow-up imaging present and the insurance payor was known for 266 (86.36%) patients: 146 (47.40%) had commercial insurance, 35 (11.36%) had Medicaid, and 85 (27.60%) had Medicare. Patients with Medicaid had over four times lower odds of completing follow-up imaging compared to patients with commercial insurance (OR 0.24, 95% CI 0.06-0.88, p = 0.032). Age, gender, race/ethnicity, smoking history, primary language, BMI, and neighborhood socioeconomic status were not independently associated with differences in follow-up imaging completion.

CONCLUSION

Patients with Medicaid had decreased odds of completing follow-up imaging recommendations compared to patients with commercial insurance.

Analysis of Radiology Report Recommendation Characteristics and Rate of Recommended Action Performance

IMPORTANCE

Following up on recommendations from radiologic findings is important for patient care, but frequently there are failures to carry out these recommendations. The lack of reliable systems to characterize and track completion of actionable radiology report recommendations poses an important patient safety challenge.

OBJECTIVES

To characterize actionable radiology recommendations and, using this taxonomy, track and understand rates of loop closure for radiology recommendations in a primary care setting.

DESIGN, SETTING, AND PARTICIPANTS

Radiology reports in a primary care clinic at a large academic center were redesigned to include actionable recommendations in a separate dedicated field. Manual review of all reports generated from imaging tests ordered between January 1 and December 31, 2018, by primary care physicians that contained actionable recommendations was performed. For this quality improvement study, a taxonomy system that conceptualized recommendations was developed based on 3 domains: (1) what is recommended (eg, repeat a test or perform a different test, specialty referral), (2) specified time frame in which to perform the recommended action, and (3) contingency language qualifying the recommendation. Using this framework, a 2-stage process was used to review patients' records to classify recommendations and determine loop closure rates and factors associated with failure to complete recommended actions. Data analysis was conducted from April to July 2021.

MAIN OUTCOMES AND MEASURES

Radiology recommendations, time frames, and contingencies. Rates of carrying out vs not closing the loop on these recommendations in the recommended time frame were assessed.

RESULTS

A total of 598 radiology reports were identified with structured recommendations: 462 for additional or future radiologic studies and 196 for nonradiologic actions (119 specialty referrals, 47 invasive procedures, and 43 other actions). The overall rate of completed actions (loop closure) within the recommended time frame was 87.4%, with 31 open loop cases rated by quality expert reviewers to pose substantial clinical risks. Factors associated with successful loop closure included (1) absence of accompanying contingency language, (2) shorter recommended time frames, and (3) evidence of direct radiologist communication with the ordering primary care physicians. A clinically significant lack of loop closure was found in approximately 5% of cases.

CONCLUSIONS AND RELEVANCE

The findings of this study suggest that creating structured radiology reports featuring a dedicated recommendations field permits the development of taxonomy to classify such recommendations and determine whether they were carried out. The lack of loop closure suggests the need for more reliable systems.

Novel Quality Measure Set: Closing the Completion Loop on Radiology Follow-up Recommendations for Noncritical Actionable Incidental Findings

BACKGROUND

Care gaps occur when radiology follow-up recommendations are poorly communicated or not completed, resulting in missed or delayed diagnosis potentially leading to worse patient outcomes. This ACR-led initiative assembled a technical expert panel (TEP) to advise development of quality measures intended to improve communication and drive increased completion rates for radiology follow-up recommendations.

MATERIALS AND METHODS

A multistakeholder TEP was assembled to advise the development of quality measures. The project scope, limited to noncritical actionable incidental findings (AIFs), encourages practices to develop and implement systems ensuring appropriate communication and follow-up to completion.

RESULTS

A suite of nine measures were developed: four outcome measures include closing the loop on completion of radiology follow-up recommendations for nonemergent AIFs (with pulmonary nodule and abdominal aortic aneurysm use cases) and overall cancer diagnoses. Five process measures address communication and tracking of AIFs: inclusion of available evidence or guidelines informing the recommendation, communication of AIFs to the practice managing ongoing care, identifying when AIFs have been communicated to the patient, and employing tracking and reminder systems for AIFs.

CONCLUSION

This ACR-led initiative developed a measure set intended to improve patient outcomes by ensuring that AIFs are appropriately communicated and followed up. The intent of these measures is to focus improvement on specific areas in which gaps in communication and AIF follow-up may occur, prompting systems to devote resources that will identify and implement solutions to improve patient care.

Management of Incidental Thyroid Nodules on Chest CT: Using Natural Language Processing to Assess White Paper Adherence and Track Patient Outcomes

OBJECTIVE

The purpose of this study was to develop a natural language processing (NLP) pipeline to identify incidental thyroid nodules (ITNs) meeting criteria for sonographic follow-up and to assess both adherence rates to white paper recommendations and downstream outcomes related to these incidental findings.

METHODS

21583 non-contrast chest CT reports from 2017 and 2018 were retrospectively evaluated to identify reports which included either an explicit recommendation for thyroid ultrasound, a description of a nodule ≥ 1.5 cm, or description of a nodule with suspicious features. Reports from 2018 were used to train an NLP algorithm called fastText for automated identification of such reports. Algorithm performance was then evaluated on the 2017 reports. Next, any patient from 2017 with a report meeting criteria for ultrasound follow-up was further evaluated with manual chart review to determine follow-up adherence rates and nodule-related outcomes.

RESULTS

NLP identified reports with ITNs meeting criteria for sonographic follow-up with an accuracy of 96.5% (95% CI 96.2-96.7) and sensitivity of 92.1% (95% CI 89.8-94.3). In 10006 chest CTs from 2017, ITN follow-up ultrasound was indicated according to white paper criteria in 81 patients (0.8%), explicitly recommended in 46.9% (38/81) of patients, and obtained in less than half of patients in which it was appropriately recommended (17/35, 48.6%).

DISCUSSION

NLP accurately identified chest CT reports meeting criteria for ITN ultrasound follow-up. Radiologist adherence to white paper guidelines and subsequent referrer adherence to radiologist recommendations showed room for improvement.

Validation of an electronic trigger to measure missed diagnosis of stroke in emergency departments

OBJECTIVE

Diagnostic errors are major contributors to preventable patient harm. We validated the use of an electronic health record (EHR)-based trigger (e-trigger) to measure missed opportunities in stroke diagnosis in emergency departments (EDs).

METHODS

Using two frameworks, the Safer Dx Trigger Tools Framework and the Symptom-disease Pair Analysis of Diagnostic Error Framework, we applied a symptom-disease pair-based e-trigger to identify patients hospitalized for stroke who, in the preceding 30 days, were discharged from the ED with benign headache or dizziness diagnoses. The algorithm was applied to Veteran Affairs National Corporate Data Warehouse on patients seen between 1/1/2016 and 12/31/2017. Trained reviewers evaluated medical records for presence/absence of missed opportunities in stroke diagnosis and stroke-related red-flags, risk factors, neurological examination, and clinical interventions. Reviewers also estimated quality of clinical documentation at the index ED visit.

RESULTS

We applied the e-trigger to 7,752,326 unique patients and identified 46,931 stroke-related admissions, of which 398 records were flagged as trigger-positive and reviewed. Of these, 124 had missed opportunities (positive predictive value for "missed" = 31.2%), 93 (23.4%) had no missed opportunity (non-missed), 162 (40.7%) were miscoded, and 19 (4.7%) were inconclusive. Reviewer agreement was high (87.3%, Cohen's kappa = 0.81). Compared to the non-missed group, the missed group had more stroke risk factors (mean 3.2 vs 2.6), red flags (mean 0.5 vs 0.2), and a higher rate of inadequate documentation (66.9% vs 28.0%).

CONCLUSION

In a large national EHR repository, a symptom-disease pair-based e-trigger identified missed diagnoses of stroke with a modest positive predictive value, underscoring the need for chart review validation procedures to identify diagnostic errors in large data sets.

Closing the Compliance Loop on Follow-Up Imaging Recommendations: Comparing Radiologists' and Administrators' Attitudes

PURPOSE

To compare non-physician healthcare professional and radiologists' survey responses regarding attitudes and current practices, policies, and procedures related to the follow-up of nonemergent actionable incidental findings (AIF).

MATERIALS AND METHODS

The American College of Radiology (ACR) developed a survey with input from a technical expert panel (TEP). Survey items were developed by TEP members, refined by an ACR market research expert, and were examined for face and construct validity. The survey was distributed among ACR membership and various medical professional organizations. Responses from non-physician responders and radiologists were analyzed and compared using descriptive statistics.

RESULTS

The analysis included 375 responses, 247 from radiologists and 128 from non-physicians. All respondent groups stated that radiology follow-up recommendations are evidence-based. Both respondent groups indicated that there is up to moderate risk associated with AIF follow-up. Both respondent groups similarly favored that the accountability for communicating AIF lies first with the ordering provider, followed by primary care providers, then the patient, and lastly an automated process that is managed by a staff member and/or the radiologist. All respondent groups indicated that tracking processes were more commonly funded by the healthcare system than through the radiology budget.

CONCLUSION

There is alignment between non-physicians and radiologists regarding the implementation of tracking systems that assure completion of radiology follow-up recommendations. Building tracking systems represents an opportunity for multi-disciplinary collaboration to address care transition communication and process gaps.

Notification System for Overdue Radiology Recommendations Improves Rates of Follow-Up and Diagnosis

OBJECTIVE. The purpose of this study was to quantify improved rates of follow-up and additional important diagnoses made after notification for overdue workups recommended by radiologists. MATERIALS AND METHODS. Standard reports from imaging studies performed at our institution from October through November 2016 were searched for the words "recommend" or "advised," yielding 9784 studies. Of these, 5245 were excluded, yielding 4539 studies; reports for 1599 of these 4539 consecutive studies were reviewed to identify firm or soft recommendations or findings requiring immediate management. If recommended follow-ups were incomplete within 1 month of the advised time, providers were notified. Compliance was calculated before and after notification and was compared using a one-sample test of proportion. RESULTS. Of 1599 patients, 92 were excluded because they had findings requiring immediate management, and 684 were excluded because of soft recommendations, yielding 823 patients. Of these patients, 125 were not yet overdue for follow-up and were excluded, and 18 were excluded because of death or transfer to another institution. Of the remaining 680 patients, follow-up was completed for 503 (74.0%). A total of 177 (26.0%) of the 680 patients were overdue for follow-up, and providers were notified. Of these 177 patients, 36 (20.3%) completed their follow-ups after notification, 34 (19.2%) had follow-up designated by the provider as nonindicated, and 107 (60.5%) were lost to follow-up, yielding four clinically important diagnoses: one biopsy-proven malignancy, one growing mass, and two thyroid nodules requiring biopsy. The rate of incomplete follow-ups after communication decreased from 26.0% (177/680) to 20.7% (141/680) (95% CI, 17.7-23.9%; p = .002), with a 20.4% reduction in relative risk of noncompliance, and 39.5% (70/177) of overdue cases were resolved when nonindicated studies were included. CONCLUSION. Notification of overdue imaging recommendations reduces incomplete follow-ups and yields clinically important diagnoses.

Impact of PTRIA (Patient Test Result Information Act) on patient follow up management

OBJECTIVE

We aim to study if direct patient notification in accordance with the Patient Test Results Information Act (Act 112) in Pennsylvania leads to decreased loss to follow up and prompt management of actionable imaging findings.

METHODS

For this IRB-approved study, radiology reports were randomly identified using the Nuance mPower™ search engine. The actionable finding group (prior to Act-112) contained 300 patients for which a voice notification was sent by radiologists to alert ordering physicians about significant imaging findings. The PTRIA group (after Act-112) contained 300 patients who were mailed a standardized letter one day after the final report was issued. The electronic medical records were reviewed to evaluate how patients were managed.

RESULTS

There was no difference in loss to follow up rates and time to follow up completion between the two groups. In both groups, 34% of patients were lost to follow up in transition of care from generalists to specialists; 24% cases were lost to follow up when imaging findings were not in the area of the initial ordering provider expertise.

CONCLUSION

The goal of Act 112 is to increase patients' role in the timely management of their significant medical conditions and prevent medical errors, specifically loss to follow up. Our study suggests that presumed patients' awareness does not contribute to improved follow up rates or decreased time to a follow up visit. 13% of patients are lost to follow up in both groups. A tracking system is required to prevent delayed management of the significant findings.

Engaging patients and families in pediatric radiology

While patient and family-centered care (PFCC) is currently a hot topic in medicine, it has long been a specific focus of pediatrics. The concept of PFCC includes a change in culture where physicians and patients move away from paternalism and instead view patients and families as partners in care. Although there are many ways in which adult-focused radiologists can learn from pediatric radiologists as leaders in PFCC, there remain many opportunities for improvement for all radiologists.

How Communication "Failed" or "Saved the Day": Counterfactual Accounts of Medical Errors

Communication breakdowns among clinicians, patients, and family members can lead to medical errors, yet effective communication may prevent such mistakes. This investigation examined patients' and family members' experiences where they believed communication failures contributed to medical errors or where effective communication prevented a medical error ("close calls"). The study conducted a thematic analysis of open-ended responses to an online survey of patients' and family members' past experiences with medical errors or close calls. Of the 93 respondents, 56 (60%) provided stories of medical errors, and the remaining described close calls. Two predominant themes emerged in medical error stories that were attributed to health care providers-information inadequacy (eg, delayed, inaccurate) and not listening to or being dismissive of a patient's or family member's concerns. In stories of close calls, a patient's or family member's proactive communication (eg, being assertive, persistent) most often "saved the day." The findings highlight the importance of encouraging active patient/family involvement in a patient's medical care to prevent errors and of improving systems to provide meaningful information in a timely manner.

Closing the Loop-A Pilot in Health System Improvement

A significant number of patients are reported to not receive timely completion of their recommended follow-up intervention following the interpretation of their imaging studies, contributing to patient deaths resulting from inaccurate or delayed diagnosis. Though automated critical test notification systems and computerized communication mechanisms currently exist, many institutions are discovering that there continue to be gaps in the completion of follow-up recommendations. Herein, we describe how we developed and implemented a closed-loop program dedicated to identifying such gaps and ensuring patients were aware of and received appropriate follow-up.

The Importance of Imaging Informatics and Informaticists in the Implementation of AI

Imaging informatics is critical to the success of AI implementation in radiology. An imaging informaticist is a unique individual who sits at the intersection of clinical radiology, data science, and information technology. With the ability to understand each of the different domains and translate between the experts in these domains, imaging informaticists are now essential players in the development, evaluation, and deployment of AI in the clinical environment.

Trends of Follow-Up Recommendations Made on Musculoskeletal MRI Reports

OBJECTIVE. The purpose of this article was to analyze trends in follow-up recommendations made on musculoskeletal MRI reports. MATERIALS AND METHODS. An IRB-approved retrospective study identified 790 musculoskeletal MRI reports from our database between January 1, 2016, and January 1, 2018, containing follow-up recommendations made by the interpreting radiologist. Metadata were automatically extracted and classification of the recommendations was performed by manual review. Clinical outcome data were collected from the electronic health record. After exclusion criteria were applied, 654 reports were included in the study. Descriptive statistics, Fisher exact tests, and chi-square tests were used for analysis. RESULTS. Clinicians acknowledged 83% and followed 73% of the recommendations. Follow-up compliance varied with the type of recommendation made: 98% for clinical intervention versus 67% for additional imaging (p < 0.001). Subspecialties acknowledged and followed recommendations at different rates: 92% and 85% for internists versus 76% and 64% for orthopedists (p < 0.001 and p < 0.001), respectively. Patient age, practice setting, radiologist experience, recommendation conditionality, and specified follow-up time intervals made no difference in compliance rate (all p > 0.05). There was no difference in compliance rate among various pathologic findings of concern (p = 0.995). Compliance rate increased significantly after direct communication between the radiologist and clinician compared with when there was no direct communication (93% vs 71%, p < 0.001). Concern for neoplasm comprised the greatest number of unacknowledged recommendations (73%). CONCLUSION. Musculoskeletal MRI recommendations are followed independent of the finding of concern and compliance is lowest for requests of additional imaging. Direct communication improves compliance and may be particularly helpful for orthopedic referrers.

Patient-centered Radiology: Overview and Opportunities in Cardiothoracic Imaging

Radiology is unique compared with most other medical specialties in that care can sometimes be delivered without speaking to or touching the patient. Although radiologists have increasingly become involved in patient safety, quality improvement, informatics, and advocacy, they must still work harder than other medical specialties to be considered "patient-facing." While cardiothoracic radiologists have likely experienced fewer opportunities to directly interface with patients, shared decision-making with patients around lung cancer screening and radiation dose optimization are both excellent examples of patient-centered and family-centered care in cardiothoracic imaging. Many cardiothoracic examinations necessitate medication administration or customized breath-holds not required of other examinations and create an opportunity for discussion between cardiothoracic radiologists and patients. Opportunities to increase the patient-centered focus in radiology exist at every interface between the radiology practice and the patient. Implementing the principles of patient-centered and family-centered care in a radiology department or practice requires the participation and engagement of all stakeholders, including patients.

The impact of distinct triple-negative breast cancer subtypes on misdiagnosis and diagnostic delay

BACKGROUND

Triple-negative breast cancer (TNBC) includes mostly aggressive types of breast cancer with poor prognosis. Due to its growth pattern, misinterpretation in clinical imaging is more frequent than in non-TNBC. As the group of TNBC contains heterogeneous types of tumors, marker expression-based subtypes have recently been established. We analyzed clinical features and false-negative imaging findings that could potentially lead to diagnostic delay within the subtypes.

METHODS

An exploratory analysis compared the imaging features across the a priori defined subtypes and related these findings to molecular subtype, disease stage, potential diagnostic delay, and patient outcome.

RESULTS

TNBC cases were categorized into basal-like (BL; 38.6%), mesenchymal-like (ML; 19.9%), luminal androgen receptor (LAR; 28.3%), and immunomodulatory (IM; 13.3%) subtype. In almost every third patient, malignant classification was missed in at least one imaging method. Misclassification in mammogram was more frequent in ML, while benign ultrasound features were reported more often in the BL subtype. Diagnostic delay due to misclassification in imaging led to tumor growth and/or upgrading of the tumor stage in 8.9% of BL tumors, which had the lowest overall survivals. Despite misclassification rate was higher in the ML subtype it showed better outcomes. Misdiagnosis of axillary lymph node metastasis was higher in LAR; however, this subtype showed a higher percentage of affected axillary lymph nodes.

CONCLUSION

TNBC subtypes have different clinical features, benign appearances, and diagnostic delay, which can lead to tumor stage upgrade. Future clinical studies on TNBC outcomes might consider the confounder of clinical delay in the subtypes.

Adoption of a Closed-Loop Communication Tool to Establish and Execute a Collaborative Follow-Up Plan for Incidental Pulmonary Nodules

OBJECTIVE. The purpose of this study is to assess radiologists' adoption of a closed-loop communication and tracking system, Result Alert and Development of Automated Resolution (RADAR), for incidental pulmonary nodules and to measure its effect on the completeness of radiologists' follow-up recommendations. MATERIALS AND METHODS. This retrospective study was performed at a tertiary academic center that performs more than 600,000 radiology examinations annually. Before RADAR, the institution's standard of care was for radiologists to generate alerts for newly discovered incidental pulmonary nodules using a previously described PACS-embedded software tool. RADAR is a new closed-loop communication tool embedded in the PACS and enterprise provider workflow that enables establishing a collaborative follow-up plan between a radiologist and referring provider and helps automate collaborative follow-up plan tracking and execution. We assessed RADAR adoption for incidental pulmonary nodules, the primary outcome, in our thoracic radiology division (study period March 9, 2018, through August 2, 2018). The secondary outcome was the completeness of follow-up recommendation for incidental pulmonary nodules, defined as explicit imaging modality and time frame for follow-up. RESULTS. After implementation, 106 of 183 (58%) incidental pulmonary nodules alerts were generated using RADAR. RADAR adoption increased by 75% during the study period (40% in the first 3 weeks vs 70% in the last 3 weeks; p < 0.001 test for trend). All RADAR alerts had explicit documentation of imaging modality and follow-up time frame, compared with 71% for non-RADAR alerts for incidental pulmonary nodules (p < 0.001). CONCLUSION. A closed-loop communication system that enables establishing and executing a collaborative follow-up plan for incidental pulmonary nodules can be adopted and improves the quality of radiologists' follow-up recommendations.

Application of electronic trigger tools to identify targets for improving diagnostic safety

Progress in reducing diagnostic errors remains slow partly due to poorly defined methods to identify errors, high-risk situations, and adverse events. Electronic trigger (e-trigger) tools, which mine vast amounts of patient data to identify signals indicative of a likely error or adverse event, offer a promising method to efficiently identify errors. The increasing amounts of longitudinal electronic data and maturing data warehousing techniques and infrastructure offer an unprecedented opportunity to implement new types of e-trigger tools that use algorithms to identify risks and events related to the diagnostic process. We present a knowledge discovery framework, the Safer Dx Trigger Tools Framework, that enables health systems to develop and implement e-trigger tools to identify and measure diagnostic errors using comprehensive electronic health record (EHR) data. Safer Dx e-trigger tools detect potential diagnostic events, allowing health systems to monitor event rates, study contributory factors and identify targets for improving diagnostic safety. In addition to promoting organisational learning, some e-triggers can monitor data prospectively and help identify patients at high-risk for a future adverse event, enabling clinicians, patients or safety personnel to take preventive actions proactively. Successful application of electronic algorithms requires health systems to invest in clinical informaticists, information technology professionals, patient safety professionals and clinicians, all of who work closely together to overcome development and implementation challenges. We outline key future research, including advances in natural language processing and machine learning, needed to improve effectiveness of e-triggers. Integrating diagnostic safety e-triggers in institutional patient safety strategies can accelerate progress in reducing preventable harm from diagnostic errors.