Quality control in neuroradiology: impact of trainees on discrepancy rates

Radiologic blind spots in hip and pelvic radiographs

PURPOSE

The purpose of our study was to identify the locations at which hip and pelvic fractures are commonly missed on radiographs.

METHODS

A retrospective study was performed at four non-Level 1 trauma center emergency departments. IRB approval was obtained. All emergency department hip or pelvic radiographs with subsequent CT performed within 48 h were identified from 2017 to 2022. Reports for each radiograph and CT were scored for the presence or the absence of fracture in the following locations: ilium, sacrum, superior pubic ramus, inferior pubic ramus, pubis, acetabulum, subcapital femoral, femoral neck, greater trochanter, lesser trochanter, intertrochanteric, and subtrochanteric. The CT report was used as the gold standard. The false-negative radiography cases were reviewed on a PACS workstation to determine whether the case had an "unexpected miss" of a fracture, a subtle fracture, radiographically occult fracture, or exam was limited by artifact. The percentage of missed fractures at each location was calculated.

RESULTS

Nine hundred seventy-five radiography cases with subsequent CT were identified. One hundred forty-six cases did not meet entry criteria; therefore, 829 cases were analyzed further. Seventy-four percent of patients were female with age of 74 ± 16 (mean ± standard deviation) years (range 1-103). Three hundred fifty-two cases had at least one fracture, and many cases had multiple fractures. There were 68 false-negative cases. The most commonly missed fractures by percentage were pubis, ilium, and greater trochanter. The most common unexpectedly missed fractures were greater trochanter and femoral subcapital.

CONCLUSION

A careful systematic evaluation of hip and pelvic radiographs, with particular attention to the pubis, ilium, greater trochanter, and subcapital region, may improve radiographic fracture detection and decrease delays in diagnosis.

Analysis of perceptual errors in skull-base pathology

PURPOSE

Medical errors result in significant mortality and morbidity. The purpose of this study is to analyze skull-base errors at a single tertiary institution, identify common anatomic sites of errors, and offer strategies to reduce errors in this region.

METHODS

A Neuroradiology Quality Assurance Database of radiologic errors was searched for attending physician computer tomography and magnetic resonance imaging errors in skull-base pathology from 2014 to 2020. Data were limited to CT and MRI reports. Errors were separated into four subcategories (tumor, trauma, vascular, and congenital) and further divided by relevant anatomic site.

RESULTS

A total of 90 skull-based errors were identified. Most errors were perceptual (87%), with common study types including MRI Brain (39%) and CT Head (24%). Most common errors were tumors (55%), followed by trauma (24%), vascular (10%), and congenital (7%). Six anatomic sites were identified and encompassed over half of errors (58%): sella, occipital bone, cerebellopontine angle/internal auditory canal (CPA/IAC), foramen magnum and clivus, cavernous sinus, and dural venous sinus.

SUMMARY

Most of the skull-base errors were perceptual. Placing a strong emphasis on both the pathology and closely examining its critical anatomic site (sella, occipital bone, CPA/IAC, foramen magnum and clivus, cavernous sinus, and dural venous sinus) could potentially reduce up to 60% of errors in these regions.

Factors Associated With Neuroradiologic Diagnostic Errors at a Large Tertiary-Care Academic Medical Center: A Case-Control Study

BACKGROUND. Numerous studies have explored factors associated with diagnostic errors in neuroradiology; however, large-scale multivariable analyses are lacking. OBJECTIVE. The purpose of this study was to evaluate associations of interpretation time, shift volume, care setting, day of week, and trainee participation with diagnostic errors by neuroradiologists at a large academic medical center. METHODS. This retrospective case-control study using a large tertiary-care academic medical center's neuroradiology quality assurance database evaluated CT and MRI examinations for which neuroradiologists had assigned RADPEER scores. The database was searched from January 2014 through March 2020 for examinations without (RADPEER score of 1) or with (RADPEER scores of 2a, 2b, 3a, 3b, or 4) diagnostic error. For each examination with error, two examinations without error were randomly selected (unless only one examination could be identified) and matched by interpreting radiologist and examination type to form case and control groups. Marginal mixed-effects logistic regression models were used to assess associations of diagnostic error with interpretation time (number of minutes since the immediately preceding report's completion), shift volume (number of examinations interpreted during the shift), emergency/inpatient setting, weekend interpretation, and trainee participation in interpretation. RESULTS. The case group included 564 examinations in 564 patients (mean age, 50.0 ± 25.0 [SD] years; 309 men, 255 women); the control group included 1019 examinations in 1019 patients (mean age, 52.5 ± 23.2 years; 540 men, 479 women). In the case versus control group, mean interpretation time was 16.3 ± 17.2 [SD] minutes versus 14.8 ± 16.7 minutes; mean shift volume was 50.0 ± 22.1 [SD] examinations versus 45.4 ± 22.9 examinations. In univariable models, diagnostic error was associated with shift volume (OR = 1.22, p < .001) and weekend interpretation (OR = 1.60, p < .001) but not interpretation time, emergency/inpatient setting, or trainee participation (p > .05). However, in multivariable models, diagnostic error was independently associated with interpretation time (OR = 1.18, p = .003), shift volume (OR = 1.27, p < .001), and weekend interpretation (OR = 1.69, p = .02). In subanalysis, diagnostic error showed independent associations on weekdays with interpretation time (OR = 1.18, p = .003) and shift volume (OR = 1.27, p < .001); such associations were not observed on weekends (interpretation time: p = .62; shift volume: p = .58). CONCLUSION. Diagnostic errors in neuroradiology were associated with longer interpretation times, higher shift volumes, and weekend interpretation. CLINICAL IMPACT. These findings should be considered when designing work-flow-related interventions seeking to reduce neuroradiology interpretation errors.

Impact of Shift Volume on Neuroradiology Diagnostic Errors at a Large Tertiary Academic Center

BACKGROUND AND PURPOSE

Medical errors can result in significant morbidity and mortality. The goal of our study is to evaluate correlation between shift volume and errors made by attending neuroradiologists at an academic medical center, using a large data set.

MATERIALS AND METHODS

CT and MRI reports from our Neuroradiology Quality Assurance database (years 2014 - 2020) were searched for attending physician errors. Data were collected on shift volume, category of missed findings, error type, interpretation setting, exam type, clinical significance.

RESULTS

654 reports contained diagnostic error. There was a significant difference between mean volume of interpreted studies on shifts when an error was made compared with shifts in which no error was documented (46.58 (SD=22.37) vs 34.09 (SD=18.60), p<0.00001); and between shifts when perceptual error was made compared with shifts when interpretive errors were made (49.50 (SD=21.9) vs 43.26 (SD=21.75), p=0.0094). 59.6% of errors occurred in the emergency/inpatient setting, 84% were perceptual and 91.1% clinically significant. Categorical distribution of errors was: vascular 25.8%, brain 23.4%, skull base 13.8%, spine 12.4%, head/neck 11.3%, fractures 10.2%, other 3.1%. Errors were detected most often on brain MRI (25.4%), head CT (18.7%), head/neck CTA (13.8%), spine MRI (13.7%).

CONCLUSION

Errors were associated with higher volume shifts, were primarily perceptual and clinically significant. We need National guidelines establishing a range of what is a safe number of interpreted cross-sectional studies per day.

Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center

BACKGROUND AND PURPOSE

Diagnostic errors affect 2%-8% of neuroradiology studies, resulting in significant potential morbidity and mortality. This retrospective analysis of a large database at a single tertiary academic institution focuses on diagnostic misses in cerebrovascular pathology and suggests error-reduction strategies.

MATERIALS AND METHODS

CT and MR imaging reports from a consecutive database spanning 2015-2020 were searched for errors of attending physicians in cerebrovascular pathology. Data were collected on missed findings, study types, and interpretation settings. Errors were categorized as ischemic, arterial, venous, hemorrhagic, and "other."

RESULTS

A total of 245,762 CT and MR imaging neuroradiology examinations were interpreted during the study period. Vascular diagnostic errors were present in 165 reports, with a mean of 49.6 (SD, 23.3) studies on the shifts when an error was made, compared with 34.9 (SD, 19.2) on shifts without detected errors (P < .0001). Seventy percent of examinations occurred in the hospital setting; 93.3% of errors were perceptual; 6.7% were interpretive; and 93.9% (n = 155) were clinically significant (RADPEER 2B or 3B). The distribution of errors was arterial and ischemic each with 33.3%, hemorrhagic with 21.8%, and venous with 7.5%. Most errors involved brain MR imaging (30.3%) followed by head CTA (27.9%) and noncontrast head CT (26.1%). The most common misses were acute/subacute infarcts (25.1%), followed by aneurysms (13.7%) and subdural hematomas (9.7%).

CONCLUSIONS

Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.

Neuroradiology diagnostic errors at a tertiary academic centre: effect of participation in tumour boards and physician experience

AIM

To quantify and correlate the diagnostic error rates in radiological interpretation with the experience of the attending neuroradiologist at a tertiary academic medical centre.

MATERIALS AND METHODS

The institution's Neuroradiology Quality Assurance Database of diagnostic errors was searched for misses from 2014-2020. Attendance at Head and Neck (H&N), Brain, and Paediatric Neuroradiology (PN) tumour boards (TB) as the presenting radiologist was recorded. Number of post-fellowship years of clinical practice (CPY) and frequency of TB attendance were considered separate metrics of a radiologist's experience. Radiological errors were categorised as Total, H&N, Skull Base (SKB), Brain, or PN diagnostic errors. Diagnostic error rates per attending neuroradiologist within each category were correlated with the frequency of TB participation and CPY using Spearman's rank correlation coefficients.

RESULTS

A total 607 examinations contained a diagnostic error. Spearman's rank correlation coefficients between Total TB participation and Total, H&N, SKB, Brain error rates were: -0.89 (p=0.0002); -0.81 (p=0.002); -0.66 (p=0.03); -0.82 (p=0.002); respectively. Spearman's rank correlation coefficients between CPY and Total, H&N, SKB, Brain and PN error rates were: 0.05 (p=0.88); 0.08 (p=0.82); 0.28 (p=0.41); -0.10 (p=0.77); -0.16 (p=0.63), respectively. Spearman's rank correlation coefficients between H&N TB and H&N, SKB error rates; and between Brain TB attendance and Brain error rates were statistically significant (p<0.05).

CONCLUSION

The present study shows a strong correlation between high TB participation rates and low diagnostic error rates. The number of years in practice did not appear to influence error rate.

Analysis of misses in imaging of head and neck pathology by attending neuroradiologists at a single tertiary academic medical centre

AIM

To analyse errors in head and neck (H&N) pathology made by attending neuroradiologists at a single tertiary-care centre.

MATERIALS AND METHODS

A neuroradiology quality assurance (QA) database of radiological errors was searched for attending physician errors in H&N pathology from 2014-2020. Data were limited to computed tomography (CT) and magnetic resonance imaging (MRI) reports. Data were collected on missed pathologies and study types. Misses were grouped into three categories: central neck (thyroid gland, aerodigestive tract), lateral neck (salivary glands, lymph nodes, soft tissues), and face/orbits (orbits, sinuses, masticator space).

RESULTS

During the study period, a total of 283,248 CT and MRI neuroradiology examinations were interpreted (all indications). Seventy-four H&N misses were identified comprising 85.1% perceptual and 14.9% interpretive errors. The distribution of errors was face/orbits (37.8%), central neck (36.5%), and lateral neck (25.7%). Clinically significant errors were found most commonly in the aerodigestive tract (21%), orbits (17.7%), masticator space, and parotid glands (14.5% each). The majority (67.6%) of the misses were detected on examinations that were not performed for a primary H&N indication; MRI brain was the most common examination (27%). Clearly malignant or potentially malignant masses accounted for 48.6% of all misses.

CONCLUSION

The majority of H&N misses were perceptual and were detected on examinations not performed for a H&N indication. Clearly malignant or potentially malignant masses represented half of all misses.

Comparing Preliminary and Final Neuroradiology Reports: What Factors Determine the Differences?

BACKGROUND AND PURPOSE

Trainees' interpretations of neuroradiologic studies are finalized by faculty neuroradiologists. We aimed to identify the factors that determine the degree to which the preliminary reports are modified.

MATERIALS AND METHODS

The character length of the preliminary and final reports and the percentage character change between the 2 reports were determined for neuroradiology reports composed during November 2012 to October 2013. Examination time, critical finding flag, missed critical finding flag, trainee level, faculty experience, imaging technique, and native-versus-non-native speaker status of the reader were collected. Multivariable linear regression models were used to evaluate the association between mean percentage character change and the various factors.

RESULTS

Of 34,661 reports, 2322 (6.7%) were read by radiology residents year 1; 4429 (12.8%), by radiology residents year 2; 3663 (10.6%), by radiology residents year 3; 2249 (6.5%), by radiology residents year 4; and 21,998 (63.5%), by fellows. The overall mean percentage character change was 14.8% (range, 0%-701.8%; median, 6.6%). Mean percentage character change increased for a missed critical finding (+41.6%, P < .0001), critical finding flag (+1.8%, P < .001), MR imaging studies (+3.6%, P < .001), and non-native trainees (+4.2%, P = .018). Compared with radiology residents year 1, radiology residents year 2 (-5.4%, P = .002), radiology residents year 3 (-5.9%, P = .002), radiology residents year 4 (-8.2%, P < .001), and fellows (-8.7%; P < .001) had a decreased mean percentage character change. Senior faculty had a lower mean percentage character change (-6.88%, P < .001). Examination time and non-native faculty did not affect mean percentage character change.

CONCLUSIONS

A missed critical finding, critical finding flag, MR imaging technique, trainee level, faculty experience level, and non-native-trainee status are associated with a higher degree of modification of a preliminary report. Understanding the factors that influence the extent of report revisions could improve the quality of report generation and trainee education.

Influences of Radiology Trainees on Screening Mammography Interpretation

PURPOSE

Participation of radiology trainees in screening mammographic interpretation is a critical component of radiology residency and fellowship training. The aim of this study was to investigate and quantify the effects of trainee involvement on screening mammographic interpretation and diagnostic outcomes.

METHODS

Screening mammograms interpreted at an academic medical center by six dedicated breast imagers over a three-year period were identified, with cases interpreted by an attending radiologist alone or in conjunction with a trainee. Trainees included radiology residents, breast imaging fellows, and fellows from other radiology subspecialties during breast imaging rotations. Trainee participation, patient variables, results of diagnostic evaluations, and pathology were recorded.

RESULTS

A total of 47,914 mammograms from 34,867 patients were included, with an overall recall rate for attending radiologists reading alone of 14.7% compared with 18.0% when involving a trainee (P < .0001). Overall cancer detection rate for attending radiologists reading alone was 5.7 per 1,000 compared with 5.2 per 1,000 when reading with a trainee (P = .517). When reading with a trainee, dense breasts represented a greater portion of recalls (P = .0001), and more frequently, greater than one abnormality was described in the breast (P = .013). Detection of ductal carcinoma in situ versus invasive carcinoma or invasive cancer type was not significantly different. The mean size of cancers in patients recalled by attending radiologists alone was smaller, and nodal involvement was less frequent, though not statistically significantly.

CONCLUSIONS

These results demonstrate a significant overall increase in recall rate when interpreting screening mammograms with radiology trainees, with no change in cancer detection rate. Radiology faculty members should be aware of this potentiality and mitigate tendencies toward greater false positives.

Errors in neuroradiology

Approximately 4 % of radiologic interpretation in daily practice contains errors and discrepancies that should occur in 2-20 % of reports. Fortunately, most of them are minor degree errors, or if serious, are found and corrected with sufficient promptness; obviously, diagnostic errors become critical when misinterpretation or misidentification should significantly delay medical or surgical treatments. Errors can be summarized into four main categories: observer errors, errors in interpretation, failure to suggest the next appropriate procedure, failure to communicate in a timely and a clinically appropriate manner. Misdiagnosis/misinterpretation percentage should rise up in emergency setting and in the first moments of the learning curve, as in residency. Para-physiological and pathological pitfalls in neuroradiology include calcification and brain stones, pseudofractures, and enlargement of subarachnoid or epidural spaces, ventricular system abnormalities, vascular system abnormalities, intracranial lesions or pseudolesions, and finally neuroradiological emergencies. In order to minimize the possibility of error, it is important to be aware of various presentations of pathology, obtain clinical information, know current practice guidelines, review after interpreting a diagnostic study, suggest follow-up studies when appropriate, communicate significant abnormal findings appropriately and in a timely fashion directly with the treatment team.