Objective comparison of errors and report length between structured and freeform abdominopelvic computed tomography reports

Bigram frequency analysis for detection of radiology report errors

PURPOSE

This pilot study evaluates the utility of analyzing bigram frequencies for detecting radiology report errors.

METHODS

A corpus of 48,050 CT reports was used to enumerate the frequency of each bigram (F_AB), and the expected frequency of each bigram in the corpus based on the constituent unigram frequencies (P_AB). A test set consisted of a separate random sample of 200 radiology reports dictated by attendings for CT scans of the abdomen in 2019, as well as a random sample of 200 radiology reports for CT scans of the abdomen dictated in 2019 by 52 different residents or fellows prior to editing by the signing attendings. Bigrams in the test reports that occurred either rarely or not at all in the corpus were flagged for manual review by an abdominal radiologist.

FINDINGS

Of 682 n-grams flagged in attending reports, 11.6% were true errors, while of 1378 n-grams flagged in trainee reports, 7.9% were true errors. The largest group of flagged n-grams in both test sets involved bigrams that did not appear in the corpus, but whose constituent words did appear in the corpus. Subsets of 50 attending and 50 resident reports were manually reviewed, revealing that the flagging procedure had a sensitivity for errors of 58% (22/38) in the attending reports and 97% (31/32) in the resident reports.

CONCLUSION

Bigram frequency analysis may be of practical value in reviewing radiology reports for errors. Further methodological refinement to improve the positive predictive value of error detection is required.

Structured reporting in radiology: a systematic review to explore its potential

OBJECTIVES

Structured reporting (SR) in radiology reporting is suggested to be a promising tool in clinical practice. In order to implement such an emerging innovation, it is necessary to verify that radiology reporting can benefit from SR. Therefore, the purpose of this systematic review is to explore the level of evidence of structured reporting in radiology. Additionally, this review provides an overview on the current status of SR in radiology.

METHODS

A narrative systematic review was conducted, searching PubMed, Embase, and the Cochrane Library using the syntax 'radiol*' AND 'structur*' AND 'report*'. Structured reporting was divided in SR level 1, structured layout (use of templates and checklists), and SR level 2, structured content (a drop-down menu, point-and-click or clickable decision trees). Two reviewers screened the search results and included all quantitative experimental studies that discussed SR in radiology. A thematic analysis was performed to appraise the evidence level.

RESULTS

The search resulted in 63 relevant full text articles out of a total of 8561 articles. Thematic analysis resulted in 44 SR level 1 and 19 level 2 reports. Only one paper was scored as highest level of evidence, which concerned a double cohort study with randomized trial design.

CONCLUSION

The level of evidence for implementing SR in radiology is still low and outcomes should be interpreted with caution.

KEY POINTS

• Structured reporting is increasingly being used in radiology, especially in abdominal and neuroradiological CT and MRI reports. • SR can be subdivided into structured layout (SR level 1) and structured content (SR level 2), in which the first is defined as being a template in which the reporter has to report; the latter is an IT-based manner in which the content of the radiology report can be inserted and displayed into the report. • Despite the extensive amount of research on the subject of structured reporting, the level of evidence is low.

[Structured reporting in daily practice-status quo and future]

CLINICAL/METHODICAL ISSUE

Introduction and application of structured reporting in radiology private practice.

STANDARD RADIOLOGICAL METHODS

Ultrasound (US), magnetic resonance imaging (MRI), computed tomography (CT), digital x‑ray (DR), mammography (MG).

METHODOLOGICAL INNOVATIONS

Structuring and predefinition of entire reports and report parts.

PERFORMANCE

Structured reporting in radiology presumably allows for higher report quality, reproducibility and easier access to data mining. Also the communication with the referring physicians can be improved. Although these advantages are well acknowledged, structured reporting does not prevail in private radiology practices. This article sheds light on the current situation of structured reporting in radiology private practice and provides information for the faltering implementation as well as perspective for individual potential solutions to overcome this situation.

ACHIEVEMENTS

Structured reporting is not yet standard. Growing acceptance of structured reporting can be expected.

PRACTICAL RECOMMENDATIONS

The application of structured reporting can improve report quality. The use of structured reporting should be considered for common examinations.