Impact of structured reporting on developing head and neck ultrasound skills

[Increased report completeness and satisfaction with structured neurotological reporting in the interdisciplinary assessment of vertigo]

BACKGROUND

Results of neurotological function diagnostics in the context of interdisciplinary vertigo assessment are usually formulated as free-text reports (FTR). These are often subject to high variability, which may lead to loss of information. The aim of the present study was to evaluate the completeness of structured reports (SR) and referrer satisfaction in the neurotological assessment of vertigo.

MATERIALS AND METHODS

Neurotological function diagnostics performed as referrals (n = 88) were evaluated retrospectively. On the basis of the available raw data, SRs corresponding to FTRs from clinical routine were created by means of a specific SR template for neurotological function diagnostics. FTRs and SRs were evaluated for completeness and referring physician satisfaction (n = 8) using a visual analog scale (VAS) questionnaire.

RESULTS

Compared to FTRs, SRs showed significantly increased overall completeness (73.7% vs. 51.7%, p < 0.001), especially in terms of patient history (92.5% vs. 66.7%, p < 0.001), description of previous findings (87.5% vs. 38%, p < 0.001), and neurotological (33.5% vs. 26.7%, p < 0.001) and audiometric function diagnostics (58% vs. 32.3%, p < 0.001). In addition, SR showed significantly increased referring physician satisfaction (VAS 8.8 vs. 4.9, p < 0.001).

CONCLUSION

Neurotological SRs enable a significantly increased report completeness with higher referrer satisfaction in the context of interdisciplinary assessment of vertigo. Furthermore, SRs are particularly suitable for scientific data analysis, especially in the context of big data analyses.

Natural language processing to convert unstructured COVID-19 chest-CT reports into structured reports

Background

Structured reporting has been demonstrated to increase report completeness and to reduce error rate, also enabling data mining of radiological reports. Still, structured reporting is perceived by radiologists as a fragmented reporting style, limiting their freedom of expression.

Purpose

A deep learning-based natural language processing method was developed to automatically convert unstructured COVID-19 chest CT reports into structured reports.

Methods

Two hundred-two COVID-19 chest CT were retrospectively reviewed by two experienced radiologists, who wrote for each exam a free-form text radiological report and coherently filled the template provided by the Italian Society of Medical and Interventional Radiology, used as ground-truth. A semi-supervised convolutional neural network was implemented to extract 62 categorical variables from the report. Two iterations were carried-out, the first without fine-tuning, the second one performing a fine-tuning. The performance was measured using the mean accuracy and the F1 mean score. An error analysis was performed to identify errors entirely attributable to incorrect processing of the model.

Results

The algorithm achieved a mean accuracy of 93.7% and an F1 score 93.8% in the first iteration. Most of the errors were exclusively attributable to wrong inference (46%). In the second iteration the model achieved for both parameters 95,8% and percentage of errors attributable to wrong inference decreased to 26%.

Conclusions

The convolutional neural network achieved an optimal performance in the automated conversion of free-form text into structured radiological reports, overcoming all the limitation attributed to structured reporting and finally paving the way for data mining of radiological report.

Improving data adequacy of ultrasonography reports for non-alcoholic fatty liver disease (NAFLD) through a national structured template

BACKGROUND

Non-alcoholic fatty liver disease (NAFLD) is a prevalent disorder that increases due to lifestyle, the rising rate of obesity, and population ages worldwide. Diagnostic ways, including sonography, do not have an explicit reporting structure.

PURPOSE

To create a structure template for NAFLD reporting, investigate its completeness, and assess the specialist opinions of using it in clinical practice.

MATERIAL AND METHODS

A structured reporting template (SRT) was designed and implemented in four stages. At first, important features were extracted from a comprehensive literature review and were evaluated by 10 radiologists and gastroenterologists using the Likert scale. Finally, the usefulness of the SRT in comparison with the conventional reporting template (CRT) was judged by 10 gastroenterologists completing the questionnaire.

RESULTS

Demographic information and sonography of the liver, gallbladder, and spleen organs were the most critical features. The completeness scores of SRT reports were higher than CRT scores for almost all the factors studied. The difference in the scores was significant for most of the parameters. Moreover, the total completeness score increased from 42% in CRT to 92% in SRT. A comparison of the report adequacy of two reports was seen in all items. The SRT obtained more rates from specialists.

CONCLUSION

Introduction of the SRT for NAFLD significantly enhanced the completeness of reporting to reduce variability in the interpretation of the related reports by clinicians. Nevertheless, more studies are needed to generalize the results in real scales for patients with NAFLD.

Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability

Purpose Ultrasound examinations are often criticized for having higher examiner dependency compared to other imaging techniques. Compared to free-text reporting, structured reporting (SR) of head and neck sonography (HNS) achieves superior time efficiency as well as report quality. However, there are no findings concerning the influence of SR on the interrater reliability (IRR) of HNS. Materials and Methods Typical pathologies (n=4) in HNS were documented by video/images by two certified head and neck ultrasound instructors. Consequently, structured reports of these videos/images were created by n=9 senior physicians at departments of otolaryngology or maxillofacial surgery with DEGUM instructors on staff. Reports (n=36) were evaluated regarding overall completeness and IRR. Additionally, user satisfaction was assessed by a visual analog scale (VAS). Results SR yielded very high report completeness (91.8%) in all four cases with a substantial IRR (Fleiss' κ 0.73). Interrater agreement was high at 87.2% with very good user satisfaction (VAS 8.6). Conclusion SR has the potential to ensure high-quality examination reports with substantial comparability and very high user satisfaction. Furthermore, big data collection and analysis are facilitated by SR. Therefore, process quality, workflow, and scientific output are potentially enhanced by SR.

Head and Neck Ultrasound - EFSUMB Training Recommendations for the Practice of Medical Ultrasound in Europe

Different surgical and medical specialists increasingly use head and neck ultrasound and ultrasound-guided interventions as part of their clinical practice. We need to ensure high quality and standardized practice across specialties, and this position paper of the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) describes the training requirements for head and neck ultrasound. Traditionally, a minimum number of ultrasound examinations indicates competence, but this is unreliable, and a general shift towards competence-based training is ongoing. For each EFSUMB level, we will outline the theoretical knowledge and skills needed for clinical practice. The recommendations follow the three EFSUMB competency levels for medical ultrasound practice. Level 1 describes the skills required to perform essential head and neck ultrasound examinations independently, level 2 includes ultrasound-guided interventions, while level 3 involves the practice of high-level neck ultrasound and use of advanced technologies. Our goal is to ensure high quality and standardized head and neck ultrasound practice performed by different clinical specialists with these recommendations.

Professional Standards in Medical Ultrasound - EFSUMB Position Paper (Short Version) - General Aspects

This first position paper of the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) on professional standards presents a common position across the different medical professions within EFSUMB regarding optimal standards for the performing and reporting of ultrasound examinations by any professional ultrasound operator. It describes general aspects of professionality that ensure procedure quality, effectiveness, efficiency, and sustainability in virtually all application fields of medical ultrasound. Recommendations are given related to safety and indication of ultrasound examinations, requirements for examination rooms, structured examination, systematic reporting of results, and management, communication and archiving of ultrasound data. The print version of this article is a short version. The long version is published online.

Professional Standards in Medical Ultrasound - EFSUMB Position Paper (Long Version) - General Aspects

This first position paper of the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) on professional standards presents a common position across the different medical professions within EFSUMB regarding optimal standards for the performing and reporting of ultrasound examinations by any professional ultrasound operator. It describes general aspects of professionality that ensure procedure quality, effectiveness, efficiency, and sustainability in virtually all application fields of medical ultrasound. Recommendations are given related to safety and indication of ultrasound examinations, requirements for examination rooms, structured examination, systematic reporting of results, and management, communication and archiving of ultrasound data. The print version of this article is a short version. The long version is published online.

“A Head Start or a Pain in the Neck?”—Establishment and Evaluation of a Video-Based “Hands-On” Head and Neck Ultrasound Course

The COVID-19 pandemic has strongly highlighted the need for more digitalization in healthcare. Teaching ultrasound skills in online courses is a key challenge in this context. The aim of this study was to establish an online video-based head and neck ultrasound course with an evaluation of the quality, effectiveness, and feasibility of this teaching method compared to in-person teaching. Twenty-two medical students were taught head and neck ultrasound in two groups: one group in an in-person course and the other one in a video-based course. Learning success was analyzed using self-evaluation forms and external assessment by an experienced ultrasonographer. Comparing pre- and post-training self-evaluation, all participants showed statistically significant learning progress. In the external assessment, the overall scores in both groups did not differ significantly. The courses themselves were positively evaluated by all participants. Herein, we present the first feasibility study of a web-based head and neck-ultrasound course for medical students. The methodology provides the potential for future changes in telemedical education and sustainable improvements in telemedical teaching and global intra-clinical and interdisciplinary patient care.

ENT Residents Benefit from a Structured Operation Planning Approach in the Training of Functional Endoscopic Sinus Surgery

Background and Objectives: Preoperative planning utilizing computed tomographies (CT) is of utmost importance in functional endoscopic sinus surgery (FESS). Frequently, no uniform documentation and planning structures are available to residents in training. Consequently, overall completeness and quality of operation planning may vary greatly. The objective of the present study was to evaluate the impact of a structured operation planning (SOP) approach on the report quality and user convenience during a 4-day sinus surgery course. Materials and Methods: Fifteen participant were requested to plan a FESS procedure based on a CT scan of the paranasal sinuses that exhibited common pathological features, in a conventional manner, using a free text. Afterwards, the participants reevaluated the same scans by means of a specifically designed structured reporting template. Two experienced ENT surgeons assessed the collected conventional operation planning (COP) and SOP methods independently with regard to time requirements, overall quality, and legibility. User convenience data were collected by utilizing visual analogue scales. Results: A significantly greater time expenditure was associated with SOPs (183 s vs. 297 s, p = 0.0003). Yet, legibility (100% vs. 72%, p < 0.0001) and overall completeness (61.3% vs. 22.7%, p < 0.0001) of SOPs was significantly superior to COPs. Additionally, description of highly relevant variants in anatomy and pathologies were outlined in greater detail. User convenience data delineated a significant preference for SOPs (VAS 7.9 vs. 6.9, p = 0.0185). Conclusions: CT-based planning of FESS procedures by residents in training using a structured approach is more time-consuming while producing a superior report quality in terms of detailedness and readability. Consequently, SOP can be considered as a valuable tool in the process of preoperative evaluations, especially within residency.

[Current developments on digitalization : Analysis of quality and economics in healthcare]

Hintergrund

Im deutschen Gesundheitssystem und damit auch im Fachgebiet der Urologie gewinnen ökonomische Rahmenbedingungen zunehmend an Bedeutung und parallel dazu werden digitale Anwendungen vermehrt eingesetzt.

Fragestellung

Die Fragestellung betrifft die gesundheitsökonomische Auseinandersetzung mit den Rahmenbedingungen der Digitalisierung im deutschen Gesundheitssystem sowie ausgewählter Anwendungsbereiche in der Urologie.

Material und Methoden

Das Gutachten des Sachverständigenrates zur Begutachtung der Entwicklung im Gesundheitswesen (SVR) wird analysiert und eine systematische Literaturanalyse zum Einsatz der strukturierten Befundung und Analyse ausgewählter Literatur zu telemedizinischen Anwendungen in der Urologie unter gesundheitsökonomischen Gesichtspunkten durchgeführt.

Ergebnisse

Als zentrale Hemmnisse bei der Digitalisierung des deutschen Gesundheitswesens identifiziert der SVR dessen Regulierung und Komplexität sowie den Umgang mit Datenschutz und -sicherheit. Der Einsatz strukturierter Befundung kann Qualität, Effektivität und Effizienz der Befundung in der Urologie steigern. Im Hinblick auf die Kosten können signifikante Einsparungen mit zunehmender Digitalisierung in der Medizin realisiert werden.

Schlussfolgerungen

Aus medizinischer und gesundheitsökonomischer Perspektive besteht bei der Ausgestaltung von Rahmenbedingungen für digitale Anwendungen im deutschen Gesundheitssystem hinsichtlich der Informationssicherheit und des Datenschutzes weiterer Gestaltungsbedarf. Bei zielgerichtetem Einsatz von digitalen Anwendungen wie der strukturierten Befundung und der Telemedizin können optimale Voraussetzungen für den zunehmenden Einsatz von künstlicher Intelligenz im Fachgebiet der Urologie geschaffen werden.

Structured Reporting of Computed Tomography Examinations in Post-Lung Transplantation Patients

OBJECTIVE

The aim of this study was to evaluate the benefits and potential of structured reports (SR) for chest computed tomography after lung transplantation.

METHODS

Free-text reports (FTR) and SR were generated for 49 computed tomography scans. Clinical routine reports were used as FTR. Two pulmonologists rated formal aspects, completeness, clinical utility, and overall quality. Wilcoxon and McNemar tests were used for statistical analysis.

RESULTS

Structured reports received significantly higher ratings for all formals aspects (P < 0.001, respectively). Completeness was higher in SR with regard to evaluation of bronchiectases, bronchial anastomoses, bronchiolitic and fibrotic changes (P < 0.001, respectively), and air trapping (P = 0.012), but not signs of pneumonia (P = 0.5). Clinical utility and overall quality were rated significantly higher for SR than FTR (P < 0.001, respectively). However, report type did not influence initiation of further diagnostic or therapeutic measures (P = 0.307 and 1.0).

CONCLUSIONS

Structured reports are superior to FTR with regard to formal aspects, completeness, clinical utility, and overall satisfaction of referring pulmonologists.

Structured MRI reporting increases completeness of radiological reports and requesting physicians' satisfaction in the diagnostic workup for pelvic endometriosis

PURPOSE

MRI plays an important role in the diagnosis and surgical planning of pelvic endometriosis (PE), and imaging reports should contain all relevant information (completeness). As structured reports are being increasingly utilized, we aimed to evaluate whether structured MRI reporting increases the quality of reports regarding completeness and, consequently, their perceived value by gynecologists, in comparison to free-text reports. We also aimed to compare the diagnostic performance of both formats.

METHODS

We retrospectively included 28 consecutive women with histologically proven PE who underwent MRI within one month before surgery. Two abdominal radiologists (Rd1/Rd2, 3y/12y experience), blinded to clinical and surgical data, individually elaborated free-text reports and, four months later, structured reports. Completeness (defined as description of six key anatomical sites deemed essential for surgical planning in a consensus of four-blinded external experts) and diagnostic performance (sensitivity and specificity) by site (histology as reference) were compared between reports using the McNemar test. The satisfaction of gynecologists was compared using the marginal homogeneity test.

RESULTS

Structured reporting increased completeness for both Rd1 (rectosigmoid, retrocervical/uterosacral ligament, vagina, and ureter) and Rd2 (vagina, ureter, and bladder) (p < 0.05), without compromising sensitivity or specificity at any of the evaluated sites. Gynecologists' satisfaction was superior with structured reports in most comparisons.

CONCLUSION

Structured MRI reports perform better in fully documenting essential features of PE and are similar in terms of diagnostic performance, therefore having higher potential for surgical planning. Gynecologists found them easier to assess and were more satisfied with the information provided by structured reports.

Structured Reporting in the Characterization of Renal Cysts by Contrast-Enhanced Ultrasound (CEUS) Using the Bosniak Classification System-Improvement of Report Quality and Interdisciplinary Communication

BACKGROUND

This study aims to evaluate the potential benefits of structured reporting (SR) compared to conventional free-text reporting (FTR) in contrast-enhanced ultrasound (CEUS) of cystic renal lesions, based on the Bosniak classification.

METHODS

Fifty patients with cystic renal lesions who underwent CEUS were included in this single-center study. FTR created in clinical routine were compared to SR retrospectively generated by using a structured reporting template. Two experienced urologists evaluated the reports regarding integrity, effort for information extraction, linguistic quality, and overall quality.

RESULTS

The required information could easily be extracted by the reviewers in 100% of SR vs. 82% of FTR (p < 0.001). The reviewers trusted the information given by SR significantly more with a mean of 5.99 vs. 5.52 for FTR (p < 0.001). SR significantly improved the linguistic quality (6.0 for SR vs. 5.68 for FTR (p < 0.001)) and the overall report quality (5.98 for SR vs. 5.58 for FTR (p < 0.001)).

CONCLUSIONS

SR significantly increases the quality of radiologic reports in CEUS examinations of cystic renal lesions compared to conventional FTR and represents a promising approach to facilitate interdisciplinary communication in the future.

Structured Reporting Using CEUS LI-RADS for the Diagnosis of Hepatocellular Carcinoma (HCC)-Impact and Advantages on Report Integrity, Quality and Interdisciplinary Communication

Simple Summary

Contrast-enhanced ultrasound (CEUS) is an increasingly accepted imaging modality for visualizing hepatocellular carcinoma (HCC) and is recommended as a secondary imaging option by most leading hepatology societies. In recent years, the use of structured reporting (SR) has been recommended by several societies to standardize report content and improve report quality of various diagnostic modalities when compared to conventional free-text reports (FTR). Our single-center study aimed to evaluate the use of SR using a CEUS LI-RADS software template in CEUS examinations of 50 HCC patients. SR significantly increased report integrity, satisfaction of the referring physicians, linguistic quality and overall report quality compared to FTR. Therefore, the use of SR in CEUS examinations of HCC patients may represent a valuable tool to facilitate clinical decision-making and improve interdisciplinary communication in the future.

Abstract

Background: Our retrospective single-center study aims to evaluate the impact of structured reporting (SR) using a CEUS LI-RADS template on report quality compared to conventional free-text reporting (FTR) in contrast-enhanced ultrasound (CEUS) for the diagnosis of hepatocellular carcinoma (HCC). Methods: We included 50 patients who underwent CEUS for HCC staging. FTR created after these examinations were compared to SR retrospectively generated by using template-based online software with clickable decision trees. The reports were evaluated regarding report completeness, information extraction, linguistic quality and overall report quality by two readers specialized in internal medicine and visceral surgery. Results: SR significantly increased report completeness with at least one key feature missing in 31% of FTR vs. 2% of SR (p < 0.001). Information extraction was considered easy in 98% of SR vs. 86% of FTR (p = 0.004). The trust of referring physicians in the report was significantly increased by SR with a mean of 5.68 for SR vs. 4.96 for FTR (p < 0.001). SR received significantly higher ratings regarding linguistic quality (5.79 for SR vs. 4.83 for FTR (p < 0.001)) and overall report quality (5.75 for SR vs. 5.01 for FTR (p < 0.001)). Conclusions: Using SR instead of conventional FTR increases the overall quality of reports in CEUS examinations of HCC patients and may represent a valuable tool to facilitate clinical decision-making and improve interdisciplinary communication in the future.

[Quality in the appraisal of head and neck sonography results in university hospitals-a random sample]

Hintergrund

Die Ultraschalldiagnostik gilt für den Radiologen, Hals-Nasen-Ohren-Arzt (HNO) oder Mund-Kiefer-Gesichts-Chirurgen als Standard in der Abklärung zahlreicher Pathologien. Es besteht ein Konsens, dass die digitale Dokumentation heute dringend notwendig ist, um die Qualität der sonographischen Dokumentationen zu verbessern und zu standardisieren. Es häufen sich Publikationen zur Implementierung standardisierter Befunddokumentation einschließlich der Kopf- und Halssonographie.

Ziel der Arbeit

Die vorliegende Arbeit zielt darauf ab, die Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden nach Kriterien der Kassenärztlichen Vereinigung (KV) Bayern an einer Auswahl deutscher HNO-Universitätskliniken stichprobenartig zu ermitteln.

Material und Methoden

Insgesamt wurden retrospektiv 70 zufällig ausgewählte, anonymisierte schriftliche Befunde einschließlich Bildmaterial von insgesamt 7 HNO-Universitätskliniken stichprobenartig nach KV-Kriterien durch einen erfahrenen Prüfer der KV Bayern ausgewertet und deskriptiv analysiert.

Ergebnisse

Von 70 Befunden konnten 69 ausgewertet werden. Die Dokumentationsvollständigkeit lag im Mittel bei 80,6 %. Neun Befunde waren vollständig korrekt dokumentiert (13 %). Die Dokumentationsvollständigkeit der einzelnen Kliniken lag zwischen 68,1 % und 93 %. Mit 88,5 % vs. 75 % erbrachte eine strukturierte Befundung eine höhere Befundvollständigkeit. In 75 % der Fälle verfügten die Kliniken mit strukturiertem Befund auch über digitale Dokumentationslösungen.

Schlussfolgerung

Die Vollständigkeit und Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden an einer Auswahl von HNO-Universitätskliniken ist insgesamt optimierbar. Die Implementierung strukturierter Befundmasken und die Umstellung der analogen Dokumentation auf digitale Lösungen sowie Vernetzung mit dem Klinikinformationssystem (KIS) und Bildarchivierungs- und Kommunikationssystem (PACS) sollte weiter vorangetrieben werden. Darüber hinaus sind leitende Ärzte dazu angehalten, die Befundqualität unerfahrener Kollegen regelmäßig zu prüfen und im Rahmen der Facharztausbildung auf die Erfüllung entsprechender Standards wie der KV-Ultraschallvereinbarung hinzuarbeiten.

The role of structured reporting and structured operation planning in functional endoscopic sinus surgery

Computed tomography (CT) scans represent the gold standard in the planning of functional endoscopic sinus surgeries (FESS). Yet, radiologists and otolaryngologists have different perspectives on these scans. In general, residents often struggle with aspects involved in both reporting and operation planning. The aim of this study was to compare the completeness of structured reports (SR) of preoperative CT images and structured operation planning (SOP) to conventional reports (CR) and conventional operation planning (COP) to potentially improve future treatment decisions on an individual level. In total, 30 preoperative CT scans obtained for surgical planning of patients scheduled for FESS were evaluated using SR and CR by radiology residents. Subsequently, otolaryngology residents performed a COP using free texts and a SOP using a specific template. All radiology reports and operation plannings were evaluated by two experienced FESS surgeons regarding their completeness for surgical planning. User satisfaction of otolaryngology residents was assessed by using visual analogue scales. Overall radiology report completeness was significantly higher using SRs regarding surgically important structures compared to CRs (84.4 vs. 22.0%, p<0.001). SOPs produced significantly higher completeness ratings (97% vs. 39.4%, p<0.001) regarding pathologies and anatomical variances. Moreover, time efficiency was not significantly impaired by implementation of SR (148 s vs. 160 s, p = 0.61) and user satisfaction was significantly higher for SOP (VAS 8.1 vs. 4.1, p<0.001). Implementation of SR and SOP results in a significantly increased completeness of radiology reports and operation planning for FESS. Consequently, the combination of both facilitates surgical planning and may decrease potential risks during FESS.

The use of structured reporting of head and neck ultrasound ensures time-efficiency and report quality during residency

PURPOSE

Free text reports (FTR) of head and neck ultrasound studies are currently deployed in most departments. Because of a lack of composition and language, these reports vary greatly in terms of quality and reliability. This may impair the learning process during residency. The purpose of the study was to analyze the longitudinal effects of using structured reports (SR) of head and neck ultrasound studies during residency.

METHODS

Attending residents (n = 24) of a tripartite course on head and neck ultrasound, accredited by the German Society for Ultrasound in Medicine (DEGUM), were randomly allocated to pictures of common diseases. Both SRs and FTRs were compiled. All reports were analyzed concerning completeness, acquired time and legibility. Overall user contentment was evaluated by a questionnaire.

RESULTS

SRs achieved significantly higher ratings regarding completeness (95.6% vs. 26.4%, p < 0.001), description of pathologies (72.2% vs. 58.9%, p < 0.001) and legibility (100% vs. 52.4%, p < 0.001) with a very high inter-rater reliability (Fleiss' kappa 0.9). Reports were finalized significantly faster (99.1 s vs. 115.0 s, p < 0.001) and user contentment was significantly better when using SRs (8.3 vs. 6.3, p < 0.001). In particular, only SRs showed a longitudinally increasing time efficiency (- 20.1 s, p = 0.036) while maintaining consistent completeness ratings.

CONCLUSIONS

The use of SRs of head and neck ultrasound studies results in an increased longitudinal time-efficiency while upholding the report quality at the same time. This may indicate an additive learning effect of structured reporting. Superior outcomes in terms of comprehensiveness, legibility and time-efficiency can be observed immediately after implementation.