Is artificial intelligence capable of generating hospital discharge summaries from inpatient records?

Automated generation of discharge summaries: leveraging large language models with clinical data

This study explores the use of open-source large language models (LLMs) to automate generation of German discharge summaries from structured clinical data. The structured data used to produce AI-generated summaries were manually extracted from electronic health records (EHRs) by a trained medical professional. By leveraging structured documentation collected for research and quality management, the goal is to assist physicians with editable draft summaries. After de-identifying 25 patient datasets, we optimized the output of the LLaMA3 model through prompt engineering and evaluated it using error analysis, as well as quantitative and qualitative metrics. The LLM-generated summaries were rated by physicians on comprehensiveness, conciseness, correctness, and fluency. Key results include an error rate of 2.84 mistakes per summary, and low-to-moderate alignment between generated and physician-written summaries (ROUGE-1: 0.25, BERTScore: 0.64). Medical professionals rated the summaries 3.72 ± 0.89 for comprehensiveness and 3.88 ± 0.97 for factual correctness on a 5-point Likert-scale; however, only 60% rated the comprehensiveness as good (4 or 5 out of 5). Despite overall informativeness, essential details-such as patient history, lifestyle factors, and intraoperative findings-were frequently omitted, reflecting gaps in summary completeness. While the LLaMA3 model captured much of the clinical information, complex cases and temporal reasoning presented challenges, leading to factual inaccuracies, such as incorrect age calculations. Limitations include a small dataset size, missing structured data elements, and the model's limited proficiency with German medical terminology, highlighting the need for large, more complete datasets and potential model fine-tuning. In conclusion, this work provides a set of real-world methods, findings, experiences, insights, and descriptive results for a focused use case that may be useful to guide future work in the LLM generation of discharge summaries, perhaps especially for those working with German and possibly other non-English content.

Using artificial intelligence (AI) for form and content checks of medical reports: Proofreading by ChatGPT4.0 in a neurology department

INTRODUCTION

Medical reports contain critical information and require concise language, yet often display errors despite advances in digital tools. This study compared the effectiveness of ChatGPT 4.0 in reporting orthographic, grammatical, and content errors in German neurology reports to a human expert.

MATERIALS AND METHODS

Ten neurology reports were embedded with ten linguistic errors each, including typographical and grammatical mistakes, and one significant content error. The reports were reviewed by ChatGPT 4.0 using three prompts: (1) check the text for spelling and grammatical errors and report them in a list format without altering the original text, (2) identify spelling and grammatical errors and generate a revised version of the text, ensuring content integrity, (3) evaluate the text for factual inaccuracies, including incorrect information and treatment errors, and report them without modifying the original text. Human control was provided by an experienced medical secretary. Outcome parameters were processing time, percentage of identified errors, and overall error detection rate.

RESULTS

Artificial intelligence (AI) accuracy in error detection was 35% (median) for Prompt 1 and 75% for Prompt 2. The mean word count of erroneous medical reports was 980 (SD = 180). AI-driven report generation was significantly faster than human review (AI Prompt 1: 102.4 s; AI Prompt 2: 209.4 s; Human: 374.0 s; p < 0.0001). Prompt 1, a tabular error report, was faster but less accurate than Prompt 2, a revised version of the report (p = 0.0013). Content analysis by Prompt 3 identified 70% of errors in 34.6 seconds.

CONCLUSIONS

AI-driven text processing for medical reports is feasible and effective. ChatGPT 4.0 demonstrated strong performance in detecting and reporting errors. The effectiveness of AI depends on prompt design, significantly impacting quality and duration. Integration into medical workflows could enhance accuracy and efficiency. AI holds promise in improving medical report writing. However, proper prompt design seems to be crucial. Appropriately integrated AI can significantly enhance supervision and quality control in health care documentation.

Using ChatGPT for writing hospital inpatient discharge summaries - perspectives from an inpatient infectious diseases service

BACKGROUND

Hospital discharge summaries are important tools for communication between healthcare professionals. They convey events that occurred during hospitalisation, as well as the subsequent follow-up plans. Artificial intelligence models can be used to summarise information succinctly from large amounts of raw data input. We explored ChatGPT's ability to generate effective discharge summaries to assist junior doctors in writing these documents.

METHODS

We constructed three hypothetical scenarios of inpatient encounters, with three different outcomes: i) discharge home with follow-up with a general practitioner, ii) discharge to a stepdown facility for further physical rehabilitation, iii) transfer to a tertiary centre for more advanced care. ChatGPT was used to generate discharge summaries for these three scenarios. The quality of the responses provided were evaluated.

RESULTS

ChatGPT was able to provide an effective framework for discharge summaries. It processed large volumes of text, summarising pertinent issues and communicating follow-up plans clearly. It is a potentially useful tool for documentation for clinicians. However, pitfalls remain, where close reading is still required to ensure the veracity of the output provided.

CONCLUSIONS

ChatGPT was able to synthesize patient information from a long prosaic format to provide a structured discharge summary. Future prospective study could evaluate if this framework provided by ChatGPT is helpful to aid junior doctors in learning about and writing discharge summaries more efficiently.

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model

OBJECTIVE

To use an artificial intelligence (AI)-powered large language model (LLM) to improve readability of patient handouts.

STUDY DESIGN

Review of online material modified by AI.

SETTING

Academic center.

METHODS

Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI's ChatGPT-4 after prompting: "Rewrite the following at a 6th-grade reading level." The understandability and actionability of both native and LLM-revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank-sum tests.

RESULTS

The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to "difficult," with reading categories ranging between high school and university grade levels. Conversely, the LLM-revised handouts had an average seventh-grade reading level. LLM-revised handouts had better readability in nearly all metrics tested: Flesch-Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman-Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM-revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials.

CONCLUSION

Patient-facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials.

LEVEL OF EVIDENCE

Level VI.

Evaluating the Impact of Artificial Intelligence (AI) on Clinical Documentation Efficiency and Accuracy Across Clinical Settings: A Scoping Review

Artificial intelligence (AI) technologies (natural language processing (NLP), speech recognition (SR), and machine learning (ML)) can transform clinical documentation in healthcare. This scoping review evaluates the impact of AI on the accuracy and efficiency of clinical documentation across various clinical settings (hospital wards, emergency departments, and outpatient clinics). We found 176 articles by applying a specific search string on Ovid. To ensure a more comprehensive search process, we also performed manual searches on PubMed and BMJ, examining any relevant references we encountered. In this way, we were able to add 46 more articles, resulting in 222 articles in total. After removing duplicates, 208 articles were screened. This led to the inclusion of 36 studies. We were mostly interested in articles discussing the impact of AI technologies, such as NLP, ML, and SR, and their accuracy and efficiency in clinical documentation. To ensure that our research reflected recent work, we focused our efforts on studies published in 2019 and beyond. This criterion was pilot-tested beforehand and necessary adjustments were made. After comparing screened articles independently, we ensured inter-rater reliability (Cohen's kappa=1.0), and data extraction was completed on these 36 articles. We conducted this study according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. This scoping review shows improvements in clinical documentation using AI technologies, with an emphasis on accuracy and efficiency. There was a reduction in clinician workload, with the streamlining of the documentation processes. Subsequently, doctors also had more time for patient care. However, these articles also raised various challenges surrounding the use of AI in clinical settings. These challenges included the management of errors, legal liability, and integration of AI with electronic health records (EHRs). There were also some ethical concerns regarding the use of AI with patient data. AI shows massive potential for improving the day-to-day work life of doctors across various clinical settings. However, more research is needed to address the many challenges associated with its use. Studies demonstrate improved accuracy and efficiency in clinical documentation with the use of AI. With better regulatory frameworks, implementation, and research, AI can significantly reduce the burden placed on doctors by documentation.

Transforming healthcare documentation: harnessing the potential of AI to generate discharge summaries

BACKGROUND

Hospital discharge summaries play an essential role in informing GPs of recent admissions to ensure excellent continuity of care and prevent adverse events; however, they are notoriously poorly written, time-consuming, and can result in delayed discharge.

AIM

To evaluate the potential of artificial intelligence (AI) to produce high-quality discharge summaries equivalent to the level of a doctor who has completed the UK Foundation Programme.

DESIGN & SETTING

Feasibility study using 25 mock patient vignettes.

METHOD

Twenty-five mock patient vignettes were written by the authors. Five junior doctors wrote discharge summaries from the case vignettes (five each). The same case vignettes were input into ChatGPT. In total, 50 discharge summaries were generated; 25 by Al and 25 by junior doctors. Quality and suitability were determined through both independent GP evaluators and adherence to a minimum dataset.

RESULTS

Of the 25 AI-written discharge summaries 100% were deemed by GPs to be of an acceptable quality compared with 92% of the junior doctor summaries. They both showed a mean compliance of 97% with the minimum dataset. In addition, the ability of GPs to determine if the summary was written by ChatGPT was poor, with only a 60% accuracy of detection. Similarly, when run through an AI-detection tool all were recognised as being very unlikely to be written by AI.

CONCLUSION

AI has proven to produce discharge summaries of equivalent quality to a junior doctor who has completed the UK Foundation Programme; however, larger studies with real-world patient data with NHS-approved AI tools will need to be conducted.

The Effect of Teach-Back on Readmission Rates in Rehabilitation Patients

PURPOSE

Thirty-day readmissions often occur in rehabilitation patients and can happen for many reasons. One of those reasons is when patients do not fully understand how to effectively manage their health after discharge. The purpose of this evidence-based quality improvement project was to determine if implementing the teach-back intervention from the Agency for Healthcare Research and Quality's (AHRQ) Health Literacy Universal Precautions Toolkit would impact 30-day readmission rates among adult rehabilitation patients.

METHODS

Data were collected from the electronic health record of rehabilitation patients. The comparative group included all rehabilitation admissions for 8 weeks prior to the intervention. The implementation group was composed of the rehabilitation admissions for 8 weeks post-implementation. All patients were then followed for 30 days postdischarge to capture readmissions.

RESULTS

The total sample size was 79 ( n = 43 in the comparative group, n = 36 in the implementation group). There was a 45% decrease in the mean percentage of the 30-day readmission rate in the implementation group as compared with the comparative group.

CONCLUSION

Based on the results, using the teach-back intervention from AHRQ's Health Literacy Universal Precautions Toolkit may impact 30-day readmission rates.

ChatGPT in Plastic and Reconstructive Surgery

Background  Chat Generative Pre-Trained Transformer (ChatGPT) is a versatile large language model-based generative artificial intelligence. It is proficient in a variety of tasks from drafting emails to coding to composing music to passing medical licensing exams. While the potential role of ChatGPT in plastic surgery is promising, evidence-based research is needed to guide its implementation in practice. Methods  This review aims to summarize the literature surrounding ChatGPT's use in plastic surgery. Results  A literature search revealed several applications for ChatGPT in the field of plastic surgery, including the ability to create academic literature and to aid the production of research. However, the ethical implications of using such chatbots in scientific writing requires careful consideration. ChatGPT can also generate high-quality patient discharge summaries and operation notes within seconds, freeing up busy junior doctors to complete other tasks. However, currently clinical information must still be manually inputted, and clinicians must consider data privacy implications. Its use in aiding patient communication and education and training is also widely documented in the literature. However, questions have been raised over the accuracy of answers generated given that current versions of ChatGPT cannot access the most up-to-date sources. Conclusions  While one must be aware of its shortcomings, ChatGPT is a useful tool for plastic surgeons to improve productivity for a range of tasks from manuscript preparation to healthcare communication generation to drafting teaching sessions to studying and learning. As access improves and technology becomes more refined, surely more uses for ChatGPT in plastic surgery will become apparent.