Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science

Exploring the impact of artificial intelligence-enabled decision aids in improving patient inclusivity, empowerment, and education in urology: a systematic review by EAU endourology

PURPOSE OF REVIEW

The implementation of artificial intelligence (AI) in urology has the potential to enhance patient outcomes through the provision of intelligent tools, such as AI-enabled decision aids (AIDAs), which can support personalized care. The objective of this systematic review is to determine the role of AIDAs in educating and empowering patients, particularly those from underrepresented populations.

RECENT FINDINGS

We conducted a comprehensive systematic review following PRISMA guidelines to explore the potential for AIDAs to address healthcare inequalities and promote patient education and empowerment. From 1078 abstracts screened, 21 articles were suitable for inclusion, all of which utilized chatbots. Three main themes of studies were identified. Fourteen studies focused on enhancing patient education, four studies investigated whether chatbots can improve the accessibility of urological literature and three studies explored chatbots role in providing lifestyle guidance. While chatbots demonstrated great potential as educational and lifestyle support tools, current research found mixed accuracy and a tendency for them to produce unreliable information. In terms of accessibility, chatbots were able to effectively enhance readability and translate literature, potentially bridging language, and literacy barriers.

SUMMARY

Through chatbots, AIDAs show strong potential to enhance urological education and empower underrepresented communities. However, chatbots must show greater consistency in accuracy before they can be confidently relied upon in clinical contexts. Further research evaluating chatbots' efficacy in clinical settings, especially with underrepresented groups, would enable greater understanding of their role in improving patient inclusivity, empowerment, and education.

Analyzing the Strengths, Weaknesses, Opportunities, and Threats (SWOT) of Chatbots in Emergency Nursing: A Narrative Review of Literature

BACKGROUND

Artificial intelligence (AI) has the potential to enhance health care by optimizing patient management and supporting workforce demands. AI-driven chatbots have potential in emergency and triage nursing, although their adoption is currently limited. Objective: This review explores the role of chatbots in emergency nursing using a Strengths, Weaknesses, Opportunities, and Threats (SWOT) framework. Methods: A narrative review of 16 studies published between 2018 and 2024 was conducted using PubMed, ScienceDirect, and Google Scholar, synthesizing data into the SWOT framework. Results: Use of chatbots was associated with improved decision-making, operational efficiency, and staff development through educational tools. However, chatbots' incapacity for critical thinking raises concerns about the usefulness of their output in complex scenarios. Opportunities include predictive analytics and better care coordination, while threats include ethical challenges, provider resistance, and overreliance on automation. Conclusion: Chatbots can transform emergency nursing by enhancing efficiency and care quality when used as complementary tools with human oversight.

The impact of AI-driven sentiment analysis on patient outcomes in psychiatric care: A narrative review

This article addresses the pressing question of how advanced analytical tools, specifically artificial intelligence (AI)-driven sentiment analysis, can be effectively integrated into psychiatric care to enhance patient outcomes. Utilizing specific search phrases like "AI-driven sentiment analysis," "psychiatric care," and "patient outcomes," a comprehensive survey of English-language publications from the years 2014-2024 was performed. This examination encompassed multiple databases such as PubMed, PsycINFO, Google Scholar, and IEEE Xplore. Through a comprehensive analysis of qualitative case studies and quantitative metrics, the study uncovered that the implementation of sentiment analysis significantly improves clinicians' ability to monitor and respond to patient emotions, leading to more tailored treatment plans and increased patient engagement. Key findings indicated that sentiment analysis improves early mood disorder detection, personalizes treatments, enhances patient-provider communication, and boosts treatment adherence, leading to better mental health outcomes. The significance of these findings lies in their potential to revolutionize psychiatric care by providing healthcare professionals with real-time insights into patient feelings and responses, thereby facilitating more proactive and empathetic care strategies. Furthermore, this study highlights the broader implications for healthcare systems, suggesting that the incorporation of sentiment analysis can lead to a paradigm shift in how mental health services are delivered, ultimately enhancing the efficacy and quality of care. By addressing barriers to new technology adoption and demonstrating its practical benefits, this research contributes vital knowledge to the ongoing discourse on optimizing healthcare delivery through innovative solutions in psychiatric settings.

Evaluating the novel role of ChatGPT-4 in addressing corneal ulcer queries: An AI-powered insight

PurposeChatGPT-4, a natural language processing-based AI model, is increasingly being applied in healthcare, facilitating education, research, and clinical decision-making support. This study explores ChatGPT-4's capability to deliver accurate and detailed information on corneal ulcers, assessing its application in medical education and clinical decision-making.MethodsThe study engaged ChatGPT-4 with 12 structured questions across different categories related to corneal ulcers. For each inquiry, five unique ChatGPT-4 sessions were initiated, ensuring that the output was not affected by previous queries. A panel of five ophthalmology experts including optometry teaching and research staff assessed the responses using a Likert scale (1-5) (1: very poor; 2: poor; 3: acceptable; 4: good; 5: very good) for quality and accuracy. Median scores were calculated, and inter-rater reliability was assessed to gauge consistency among evaluators.ResultsThe evaluation of ChatGPT-4's responses to corneal ulcer-related questions revealed varied performance across categories. Median scores were consistently high (4.0) for risk factors, etiology, symptoms, treatment, complications, and prognosis, with narrow IQRs (3.0-4.0), reflecting strong agreement. However, classification and investigations scored slightly lower (median 3.0). Signs of corneal ulcers had a median of 2.0, showing significant variability. Of 300 responses, 45% were rated 'good,' 41.7% 'acceptable,' 10% 'poor,' and only 3.3% 'very good,' highlighting areas for improvement. Notably, Evaluator 2 gave 35 'good' ratings, while Evaluators 1 and 3 assigned 10 'poor' ratings each. Inter-evaluator variability, along with gaps in diagnostic precision, underscores the need for refining AI responses. Continuous feedback and targeted adjustments could boost ChatGPT-4's utility in delivering high-quality ophthalmic education.ConclusionChatGPT-4 shows promising utility in providing educational content on corneal ulcers. Despite the variance in evaluator ratings, the numerical analysis suggests that with further refinement, ChatGPT-4 could be a valuable tool in ophthalmological education and clinical support.

Development and Systematic Evaluation of a Progressive Web Application for Women With Cardiac Pain: Usability Study

BACKGROUND

Cardiac pain has been widely considered to be the primary indicator of coronary artery disease. The presentation of cardiac pain and associated symptoms vary in women, making it challenging to interpret as cardiac, possibly cardiac, or noncardiac. Women prefer to consult with family and friends instead of seeking immediate medical care.

OBJECTIVE

This study aimed to assess the user performance (ie, ease of use, efficiency, and errors) and user satisfaction (System Usability Scale; SUS) of a progressive web application for women with cardiac pain.

METHODS

Following ethics approval, a purposive sample of women aged >18 years with cardiac pain or associated symptoms lasting >3 months and able to speak and read English was recruited to participate in 2 iterative usability testing cycles. The first cycle assessed the performance of and satisfaction with at heart using a web application, and the second cycle assessed the performance of and satisfaction with at heart across various Android and iOS devices. In total, 2 investigators recorded user comments and documented problems. At the end of the testing session, the participants completed the SUS and 4 semistructured interview questions.

RESULTS

In total, 10 eligible women participated in usability testing from March 31, 2020, to April 17, 2020 (cycle 1), and from November 17, 2020, to November 30, 2020 (cycle 2). Women across usability testing cycles had a mean age of 55.6 (SD 7.3) years, and most (9/10, 90%) were well educated. In total, 50% (5/10) were employed full or part time, and 60% (6/10) earned >CAD $70,000 (US $48,881.80) annually. Participants across 2 testing cycles reported the overall usability of the at heart progressive web application as highly acceptable (mean SUS score 81.75, SD 10.41). In total, 90% (9/10) of participants rated the user-friendliness of at heart as good or excellent. All participants (10/10, 100%) thought at heart was easy to use and efficient. Only 2 testing errors were noted as high priority; these were low contrast or small font and clarification that the chatbot was not a real person. User satisfaction was assessed using themes that emerged from the debrief and 4 semistructured interview questions; at heart was engaging, comprehensive, understandable, credible, relevant, affirming, personalized, and innovative.

CONCLUSIONS

This study provides initial support for the at heart progressive web application for women living with cardiac pain and symptoms. Ongoing evaluations in phases 3 and 4 should aim to examine the feasibility and acceptability of and the extent of engagement with the at heart core feature set: Heart Check, Wellness Check, and the library. In addition to assessing effectiveness in the phase-4 effectiveness-implementation hybrid trial (type I), describing and better understanding the context for implementation (eg, race and ethnicity and geography) will be necessary.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2019-033092.

Assessing the performance of ChatGPT-4o on the Turkish Orthopedics and Traumatology Board Examination

OBJECTIVES

This study aims to assess the overall performance of ChatGPT version 4-omni (GPT-4o) on the Turkish Orthopedics and Traumatology Board Examination (TOTBE) using actual examinees as a reference point to evaluate and compare the performance of GPT-4o with that of human participants.

MATERIALS AND METHODS

In this study, GPT-4o was tested with multiple-choice questions that formed the first step of 14 TOTBEs conducted between 2010 and 2023. The assessment of image-based questions was conducted separately for all exams. The questions were classified based on the subspecialties for the five exams (2010-2014). The performance of GPT-4o was assessed and compared to those of actual examinees of the TOTBE.

RESULTS

The mean total score of GPT-4o was 70.2±5.64 (range, 61 to 84), whereas that of actual examinees was 58±3.28 (range, 53.6 to 64.6). Considering accuracy rates, GPT-4o demonstrated 62% accuracy on image-based questions and 70% accuracy on text-based questions. It also demonstrated superior performance in the field of basic sciences, whereas actual examinees performed better in the specialty of reconstruction. Both GPT-4o and actual examinees exhibited the lowest scores in the subspecialty of lower extremity and foot.

CONCLUSION

Our study results showed that GPT-4o performed well on the TOTBE, particularly in basic sciences. While it demonstrated accuracy comparable to actual examinees in some areas, these findings highlight its potential as a helpful tool in medical education.

AI-enabled language models (LMs) to large language models (LLMs) and multimodal large language models (MLLMs) in drug discovery and development

BACKGROUND

Due to the recent revolution of artificial intelligence (AI), AI-enabled large language models (LLMs) have flourished and started to be applied in various sectors of science and medicine. Drug discovery and development are time-consuming, complex processes that require high investment. The conventional method of drug discovery is costly and has a high failure rate. AI-enabled LLMs are used in various steps of drug discovery to solve the challenges of time and cost.

AIM OF REVIEW

The article aims to provide a comprehensive understanding of AI-enabled LLMs and their use in various steps of drug discovery to ease the challenges.

KEY SCIENTIFIC CONCEPTS OF REVIEW

The review provides an overview of the LLMs and their current state-of-the-art application in structure-based drug molecule design and de novo drug design. The different applications of AI-enabled LLMshave been illustrated, such as drug target identification, validation, interaction, and ADME/ADMET. Several domain-specific models of LLMs are developed in this direction and applied in drug discovery and development to speed up the process. We discussed all these domain-specific models of LLMs and their applications in this field. Finally, we illustrated the challenges and future perspectives on the applications of AI-enabled LLMs in drug discovery and development.

Assessment of ChatGPT-generated medical Arabic responses for patients with metabolic dysfunction-associated steatotic liver disease

BACKGROUND AND AIM

Artificial intelligence (AI)-powered chatbots, such as Chat Generative Pretrained Transformer (ChatGPT), have shown promising results in healthcare settings. These tools can help patients obtain real-time responses to queries, ensuring immediate access to relevant information. The study aimed to explore the potential use of ChatGPT-generated medical Arabic responses for patients with metabolic dysfunction-associated steatotic liver disease (MASLD).

METHODS

An English patient questionnaire on MASLD was translated to Arabic. The Arabic questions were then entered into ChatGPT 3.5 on November 12, 2023. The responses were evaluated for accuracy, completeness, and comprehensibility by 10 Saudi MASLD experts who were native Arabic speakers. Likert scales were used to evaluate: 1) Accuracy, 2) Completeness, and 3) Comprehensibility. The questions were grouped into 3 domains: (1) Specialist referral, (2) Lifestyle, and (3) Physical activity.

RESULTS

Accuracy mean score was 4.9 ± 0.94 on a 6-point Likert scale corresponding to "Nearly all correct." Kendall's coefficient of concordance (KCC) ranged from 0.025 to 0.649, with a mean of 0.28, indicating moderate agreement between all 10 experts. Mean completeness score was 2.4 ± 0.53 on a 3-point Likert scale corresponding to "Comprehensive" (KCC: 0.03-0.553; mean: 0.22). Comprehensibility mean score was 2.74 ± 0.52 on a 3-point Likert scale, which indicates the responses were "Easy to understand" (KCC: 0.00-0.447; mean: 0.25).

CONCLUSION

MASLD experts found that ChatGPT responses were accurate, complete, and comprehensible. The results support the increasing trend of leveraging the power of AI chatbots to revolutionize the dissemination of information for patients with MASLD. However, many AI-powered chatbots require further enhancement of scientific content to avoid the risks of circulating medical misinformation.

Artificial intelligence chatbots versus traditional medical resources for patient education on "Labor Epidurals": an evaluation of accuracy, emotional tone, and readability

BACKGROUND

Labor epidural analgesia is a widely used method for pain relief in childbirth, yet information accessibility for expectant mothers remains a challenge. Artificial intelligence (AI) chatbots like Chat Generative Pre-Trained Transformer (ChatGPT) and Google Gemini offer potential solutions for improving patient education. This study evaluates the accuracy, readability, and emotional tone of AI chatbot responses compared to the American Society of Anesthesiologists (ASA) online materials on labor epidurals.

METHODS

Eight common questions about labor epidurals were posed to ChatGPT and Gemini. Seven obstetric anaesthesiologists evaluated the generated responses for accuracy and completeness on a 1-10 Likert scale, comparing them with ASA-sourced content. Statistical analysis (one-way ANOVA, Tukey HSD), sentiment analysis and readability metrics (Flesch Reading ease) were used to assess differences.

RESULTS

ASA materials scored highest for accuracy (8.80 ± 0.40) and readability, followed by Gemini and ChatGPT. Completeness scores showed ASA and Gemini performing significantly better than ChatGPT (P <0.001). ASA materials were the most accessible, while Gemini content was more complex. Sentiment analysis indicated a neutral tone for ASA and Gemini, with ChatGPT displaying a less consistent tone.

CONCLUSION

AI chatbots exhibit promise in patient education for labor epidurals but require improvements in readability and tone consistency to enhance engagement. Further refinement of AI chatbots may support more accessible, patient-centred healthcare information.

A deep neural network model for classifying pharmacy practice publications into research domains

BACKGROUND

Pharmacy practice faculty research profiles extend beyond the clinical and social domains, which are core elements of pharmacy practice. But as highlighted by journal editors in the Granada Statements, there is no consensus on these terms. Four domains (clinical, education, social & administrative, and basic & translational) of pharmacy practice faculty research are proposed.

OBJECTIVES

To develop a classifier for categorizing pharmacy practice faculty publications into four proposed domains, and to compare the model with zero-shot performances of state-of-the-art, general purpose large language models (gpLLMs).

METHODS

One thousand abstracts from 2018 to 2021 documents published by pharmacy practice faculty were reviewed, labelled and used to screen and finetune several Bidirectional Encoders Representations from Transformers (BERT) models. The selected model was compared with zero-shot performances of 7 state-of-the-art gpLLMs including ChatGPT-4o, Gemini-1.5-Pro, Claude-3.5, LLAMA-3.1 and Mistral Large, using 80 randomly selected abstracts from 2023 publications labelled with ≥80% consensus by all authors. Classification metrics included F1, recall, precision and accuracy, and reproducibility was measured with Cohen's kappa. A use case was demonstrated by testing the null hypothesis that the research domain distribution of faculty publications was independent of the pandemic.

RESULT

The model - Pharmacy Practice Research Domain Classifier (PPRDC) produced a 5-fold stratified cross-validation metrics of 89.4 ± 1.7, 90.2 ± 2.2, 89.0 ± 1.7, and 95.5 ± 0.6, for F1, recall, precision and accuracy, respectively. PPRDC produced perfectly reproducible classifications (Cohen's kappa = 1.0) and outperformed zero-shot performances of all gpLLMs. F1 scores were 96.2 ± 1.6, 92.7 ± 1.2, 85.8 ± 3.2, and 83.1 ± 9.8 for education, clinical, social, and translational domains, respectively.

CONCLUSIONS

PPRDC (https://sadeosun-pprdc.streamlit.app) performed better than gpLLMs in this abstract classification task. Among several other impacts, PPRDC opens a new frontier in bibliometric studies; it will also advance the goals of the Grenada Statements by aiding authors and journal editors in journal selection and article prioritization decisions, respectively.

Artificial intelligence for image recognition in diagnosing oral and oropharyngeal cancer and leukoplakia

Visual diagnosis is one of the key features of squamous cell carcinoma of the oral cavity (OSCC) and oropharynx (OPSCC), both subsets of head and neck squamous cell carcinoma (HNSCC) with a heterogeneous clinical appearance. Advancements in artificial intelligence led to Image recognition being introduced recently into large language models (LLMs) such as ChatGPT 4.0. This exploratory study, for the first time, evaluated the application of image recognition by ChatGPT to diagnose squamous cell carcinoma and leukoplakia based on clinical images, with images without any lesion as a control group. A total of 45 clinical images were analyzed, comprising 15 cases each of SCC, leukoplakia, and non-lesion images. ChatGPT 4.0 was tasked with providing the most likely diagnosis based on these images in scenario one. In scenario two the image and the clinical history were provided, whereas in scenario three only the clinical history was given. The results and the accuracy of the LLM were rated by two independent reviewers and the overall performance was evaluated using the modified Artificial Intelligence Performance Index (AIPI. In this study, ChatGPT 4.0 demonstrated the ability to correctly identify leukoplakia cases using image recognition alone, while the ability to diagnose SCC was insufficient, but improved by including the clinical history in the prompt. Providing only the clinical history resulted in a misclassification of most leukoplakia and some SCC cases. Oral cavity lesions were more likely to be diagnosed correctly. In this exploratory study of 45 images of oral lesions, ChatGPT 4.0 demonstrated a convincing performance for detecting SCC only when the clinical history was added, whereas Leukoplakia was detected solely by image recognition. ChatGPT is therefore currently insufficient for reliable OPSCC and OSCC diagnosis, but further technological advancements may pave the way for the use in the clinical setting.

Exploring Chatbot contributions to enhancing vaccine literacy and uptake: A scoping review of the literature

BACKGROUND

The increasing integration of chatbots across various sectors marks a significant shift in digital communication, and their role in healthcare makes no exception. This scoping review aims to systematically examine the role of chatbots in the perspective of organizational vaccine literacy, particularly in enhancing vaccine literacy and facilitating the dissemination of vaccine-related information, evaluating the potential of chatbots to transform vaccination communication strategies and improve health education outcomes.

METHODS

This scoping review adhered to the Joanna Briggs Institute methodology and the PRISMA-ScR checklist. A systematic search of MEDLINE, Embase, Scopus, and PsycInfo was conducted from January 2020 to October 30, 2024, using keywords related to "chatbots" and "vaccination." Study selection involved a two-stage screening process, focusing on studies reporting the use of chatbots to improve vaccine literacy and uptake. Data were thematically analyzed and presented in a narrative format.

RESULTS

Twenty-two studies were included in the review: these studies demonstrate the effectiveness of chatbots in enhancing vaccine literacy and acceptance, mainly focusing on COVID-19 but also addressing HPV and childhood vaccinations. They highlight chatbots' role in improving the vaccine-literate environment through countering misinformation and improving communication with healthcare professionals, showcasing their potential to significantly influence public health outcomes and their adaptability to diverse populations and geographic regions.

CONCLUSIONS

These digital assistants could provide personalized and up-to-date information, improving not only knowledge but also attitudes and intentions towards vaccinations.

Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review

Artificial intelligence (AI) is becoming increasingly influential in ophthalmology, particularly through advancements in machine learning, deep learning, robotics, neural networks, and natural language processing (NLP). Among these, NLP-based chatbots are the most readily accessible and are driven by AI-based large language models (LLMs). These chatbots have facilitated new research avenues and have gained traction in both clinical and surgical applications in ophthalmology. They are also increasingly being utilized in studies on ophthalmology-related exams, particularly those containing multiple-choice questions (MCQs). This narrative review evaluates both the opportunities and the challenges of integrating chatbots into ophthalmology research, with separate assessments of studies involving open- and close-ended questions. While chatbots have demonstrated sufficient accuracy in handling MCQ-based studies, supporting their use in education, additional exam security measures are necessary. The research on open-ended question responses suggests that AI-based LLM chatbots could be applied across nearly all areas of ophthalmology. They have shown promise for addressing patient inquiries, offering medical advice, patient education, supporting triage, facilitating diagnosis and differential diagnosis, and aiding in surgical planning. However, the ethical implications, confidentiality concerns, physician liability, and issues surrounding patient privacy remain pressing challenges. Although AI has demonstrated significant promise in clinical patient care, it is currently most effective as a supportive tool rather than as a replacement for human physicians.

Exploring the ability of ChatGPT to create quality patient education resources about kidney transplant

BACKGROUND

Chat Generative Pre-trained Transformer (ChatGPT) is a language model that may have the potential to revolutionize health care. The study purpose was to test whether ChatGPT could be used to create educational brochures about kidney transplant tailored for three target audiences: caregivers, teens and children.

METHODS

Using a list of 25 educational topics, standardized prompts were employed to ensure content consistency in ChatGPT generation. An expert panel assessed the accuracy of the content by rating agreement on a Likert scale (1 = <25 % agreement; and 5 = 100 % agreement). The understandability, actionability and readability of the brochures were assessed using the Patient Education Materials Assessment Tool for printable materials (PEMAT-P) and standard readability scales. A caregiver and patient reviewed and provided written feedback.

RESULTS

We found mean understandability scores of 69 %, 66 %, and 73 % for caregiver, teen, and child brochures respectively, with 90.7 % of the ChatGPT generated brochures scoring 40 % on the actionability scale. Generated caregiver and teen materials achieved readability levels of grades 9-14, while child-specific brochures achieved readability levels of grades 6-11. Brochures were formatted appropriately but lacked depth.

CONCLUSION

ChatGPT demonstrates potential for rapidly generating patient education materials; however, challenges remain in ensuring content specificity. We share the lessons learned to assist other healthcare providers with using this technology.

Hepatic encephalopathy post-TIPS: Current status and prospects in predictive assessment

Transjugular intrahepatic portosystemic shunt (TIPS) is an essential procedure for the treatment of portal hypertension but can result in hepatic encephalopathy (HE), a serious complication that worsens patient outcomes. Investigating predictors of HE after TIPS is essential to improve prognosis. This review analyzes risk factors and compares predictive models, weighing traditional scores such as Child-Pugh, Model for End-Stage Liver Disease (MELD), and albumin-bilirubin (ALBI) against emerging artificial intelligence (AI) techniques. While traditional scores provide initial insights into HE risk, they have limitations in dealing with clinical complexity. Advances in machine learning (ML), particularly when integrated with imaging and clinical data, offer refined assessments. These innovations suggest the potential for AI to significantly improve the prediction of post-TIPS HE. The study provides clinicians with a comprehensive overview of current prediction methods, while advocating for the integration of AI to increase the accuracy of post-TIPS HE assessments. By harnessing the power of AI, clinicians can better manage the risks associated with TIPS and tailor interventions to individual patient needs. Future research should therefore prioritize the development of advanced AI frameworks that can assimilate diverse data streams to support clinical decision-making. The goal is not only to more accurately predict HE, but also to improve overall patient care and quality of life.

Assessing the use of the novel tool Claude 3 in comparison to ChatGPT 4.0 as an artificial intelligence tool in the diagnosis and therapy of primary head and neck cancer cases

OBJECTIVES

Head and neck squamous cell carcinoma (HNSCC) is a complex malignancy that requires a multidisciplinary tumor board approach for individual treatment planning. In recent years, artificial intelligence tools have emerged to assist healthcare professionals in making informed treatment decisions. This study investigates the application of the newly published LLM Claude 3 Opus compared to the currently most advanced LLM ChatGPT 4.0 for the diagnosis and therapy planning of primary HNSCC. The results were compared to that of a conventional multidisciplinary tumor board; (2) Materials and Methods: We conducted a study in March 2024 on 50 consecutive primary head and neck cancer cases. The diagnostics and MDT recommendations were compared to the Claude 3 Opus and ChatGPT 4.0 recommendations for each patient and rated by two independent reviewers for the following parameters: clinical recommendation, explanation, and summarization in addition to the Artificial Intelligence Performance Instrument (AIPI); (3) Results: In this study, Claude 3 achieved better scores for the diagnostic workup of patients than ChatGPT 4.0 and provided treatment recommendations involving surgery, chemotherapy, and radiation therapy. In terms of clinical recommendations, explanation and summarization Claude 3 scored similar to ChatGPT 4.0, listing treatment recommendations which were congruent with the MDT, but failed to cite the source of the information; (4) Conclusion: This study is the first analysis of Claude 3 for primary head and neck cancer cases and demonstrates a superior performance in the diagnosis of HNSCC than ChatGPT 4.0 and similar results for therapy recommendations. This marks the advent of a newly launched advanced AI model that may be superior to ChatGPT 4.0 for the assessment of primary head and neck cancer cases and may assist in the clinical diagnostic and MDT setting.

Performance of Multimodal Artificial Intelligence Chatbots Evaluated on Clinical Oncology Cases

Importance

Multimodal artificial intelligence (AI) chatbots can process complex medical image and text-based information that may improve their accuracy as a clinical diagnostic and management tool compared with unimodal, text-only AI chatbots. However, the difference in medical accuracy of multimodal and text-only chatbots in addressing questions about clinical oncology cases remains to be tested.

Objective

To evaluate the utility of prompt engineering (zero-shot chain-of-thought) and compare the competency of multimodal and unimodal AI chatbots to generate medically accurate responses to questions about clinical oncology cases.

Design, Setting, and Participants

This cross-sectional study benchmarked the medical accuracy of multiple-choice and free-text responses generated by AI chatbots in response to 79 questions about clinical oncology cases with images.

Exposures

A unique set of 79 clinical oncology cases from JAMA Network Learning accessed on April 2, 2024, was posed to 10 AI chatbots.

Main Outcomes and Measures

The primary outcome was medical accuracy evaluated by the number of correct responses by each AI chatbot. Multiple-choice responses were marked as correct based on the ground-truth, correct answer. Free-text responses were rated by a team of oncology specialists in duplicate and marked as correct based on consensus or resolved by a review of a third oncology specialist.

Results

This study evaluated 10 chatbots, including 3 multimodal and 7 unimodal chatbots. On the multiple-choice evaluation, the top-performing chatbot was chatbot 10 (57 of 79 [72.15%]), followed by the multimodal chatbot 2 (56 of 79 [70.89%]) and chatbot 5 (54 of 79 [68.35%]). On the free-text evaluation, the top-performing chatbots were chatbot 5, chatbot 7, and the multimodal chatbot 2 (30 of 79 [37.97%]), followed by chatbot 10 (29 of 79 [36.71%]) and chatbot 8 and the multimodal chatbot 3 (25 of 79 [31.65%]). The accuracy of multimodal chatbots decreased when tested on cases with multiple images compared with questions with single images. Nine out of 10 chatbots, including all 3 multimodal chatbots, demonstrated decreased accuracy of their free-text responses compared with multiple-choice responses to questions about cancer cases.

Conclusions and Relevance

In this cross-sectional study of chatbot accuracy tested on clinical oncology cases, multimodal chatbots were not consistently more accurate than unimodal chatbots. These results suggest that further research is required to optimize multimodal chatbots to make more use of information from images to improve oncology-specific medical accuracy and reliability.

Can Natural Language Processing (NLP) Provide Consultancy to Patients About Edentulism Teeth Treatment?

Aim This study aimed to evaluate the accuracy and quality of the answers given by artificial intelligence (AI) applications to the questions directed at tooth deficiency treatments. Materials and methods Fifteen questions asked by patients/ordinary people about missing tooth treatment were selected from the Quora platform. Questions were asked to the ChatGPT-4 (OpenAI Inc., San Francisco, California, United States) and Copilot (Microsoft Corporation, Redmond, Washington, United States) models. Responses were assessed by two expert physicians using a five-point Likert scale (LS) for accuracy and the Global Quality Scale (GQS) for quality. To assess the internal consistency and inter-rater agreement of ChatGPT-4 and Copilot, Cronbach's alpha, Spearman-Brown's coefficient, and Guttman's split-half coefficient were calculated to measure the reliability and internal consistency of both instruments (α=0.05). Results Copilot showed a mean LS value of 3.83±0.36 and ChatGPT-4 showed a lower mean value of 3.93±0.32. ChatGPT-4's GQS mean value (3.9±0.28) is also higher than Copilot (3.83±0.06) (p<0.001). Conclusion It can be said that AI chatbots gave highly accurate and consistent answers to questions about the treatment of toothlessness. With the ever-developing technology, AI chatbots can be used as consultants for dental treatments in the future.

Assessing the response quality and readability of chatbots in cardiovascular health, oncology, and psoriasis: A comparative study

BACKGROUND

Chatbots using the Large Language Model (LLM) generate human responses to questions from all categories. Due to staff shortages in healthcare systems, patients waiting for an appointment increasingly use chatbots to get information about their condition. Given the number of chatbots currently available, assessing the responses they generate is essential.

METHODS

Five chatbots with free access were selected (Gemini, Microsoft Copilot, PiAI, ChatGPT, ChatSpot) and blinded using letters (A, B, C, D, E). Each chatbot was asked questions about cardiology, oncology, and psoriasis. Responses were compared to guidelines from the European Society of Cardiology, American Academy of Dermatology and American Society of Clinical Oncology. All answers were assessed using readability scales (Flesch Reading Scale, Gunning Fog Scale Level, Flesch-Kincaid Grade Level and Dale-Chall Score). Using a 3-point Likert scale, two independent medical professionals assessed the compliance of the responses with the guidelines.

RESULTS

A total of 45 questions were asked of all chatbots. Chatbot C gave the shortest answers, 7.0 (6.0 - 8.0), and Chatbot A the longest 17.5 (13.0 - 24.5). The Flesch Reading Ease Scale ranged from 16.3 (12.2 - 21.9) (Chatbot D) to 39.8 (29.0 - 50.4) (Chatbot A). Flesch-Kincaid Grade Level ranged from 12.5 (10.6 - 14.6) (Chatbot A) to 15.9 (15.1 - 17.1) (Chatbot D). Gunning Fog Scale Level ranged from 15.77 (Chatbot A) to 19.73 (Chatbot D). Dale-Chall Score ranged from 10.3 (9.3 - 11.3) (Chatbot A) to 11.9 (11.5 - 12.4) (Chatbot D).

CONCLUSION

This study indicates that chatbots vary in length, quality, and readability. They answer each question in their own way, based on the data they have pulled from the web. Reliability of the responses generated by chatbots is high. This suggests that people who want information from a chatbot need to be careful and verify the answers they receive, particularly when they ask about medical and health aspects.

The changing scenario of drug discovery using AI to deep learning: Recent advancement, success stories, collaborations, and challenges

Due to the transformation of artificial intelligence (AI) tools and technologies, AI-driven drug discovery has come to the forefront. It reduces the time and expenditure. Due to these advantages, pharmaceutical industries are concentrating on AI-driven drug discovery. Several drug molecules have been discovered using AI-based techniques and tools, and several newly AI-discovered drug molecules have already entered clinical trials. In this review, we first present the data and their resources in the pharmaceutical sector for AI-driven drug discovery and illustrated some significant algorithms or techniques used for AI and ML which are used in this field. We gave an overview of the deep neural network (NN) models and compared them with artificial NNs. Then, we illustrate the recent advancement of the landscape of drug discovery using AI to deep learning, such as the identification of drug targets, prediction of their structure, estimation of drug-target interaction, estimation of drug-target binding affinity, design of de novo drug, prediction of drug toxicity, estimation of absorption, distribution, metabolism, excretion, toxicity; and estimation of drug-drug interaction. Moreover, we highlighted the success stories of AI-driven drug discovery and discussed several collaboration and the challenges in this area. The discussions in the article will enrich the pharmaceutical industry.

Large language model to multimodal large language model: A journey to shape the biological macromolecules to biological sciences and medicine

After ChatGPT was released, large language models (LLMs) became more popular. Academicians use ChatGPT or LLM models for different purposes, and the use of ChatGPT or LLM is increasing from medical science to diversified areas. Recently, the multimodal LLM (MLLM) has also become popular. Therefore, we comprehensively illustrate the LLM and MLLM models for a complete understanding. We also aim for simple and extended reviews of LLMs and MLLMs for a broad category of readers, such as researchers, students in diversified fields, and other academicians. The review article illustrates the LLM and MLLM models, their working principles, and their applications in diversified fields. First, we demonstrate the technical concept of LLMs, working principle, Black Box, and the evolution of LLMs. To explain the working principle, we discuss the tokenization process, token representation, and token relationships. We also extensively demonstrate the application of LLMs in biological macromolecules, medical science, biological science, and other areas. We illustrate the multimodal applications of LLMs or MLLMs. Finally, we illustrate the limitations, challenges, and future prospects of LLMs. The review acts as a booster dose for clinicians, a primer for molecular biologists, and a catalyst for scientists, and also benefits diversified academicians.

Assessing the role of advanced artificial intelligence as a tool in multidisciplinary tumor board decision-making for recurrent/metastatic head and neck cancer cases - the first study on ChatGPT 4o and a comparison to ChatGPT 4.0

Background

Recurrent and metastatic head and neck squamous cell carcinoma (HNSCC) is characterized by a complex therapeutic management that needs to be discussed in multidisciplinary tumor boards (MDT). While artificial intelligence (AI) improved significantly to assist healthcare professionals in making informed treatment decisions for primary cases, an application in the even more complex recurrent/metastatic setting has not been evaluated yet. This study also represents the first evaluation of the recently published LLM ChatGPT 4o, compared to ChatGPT 4.0 for providing therapy recommendations.

Methods

The therapy recommendations for 100 HNSCC cases generated by each LLM, 50 cases of recurrence and 50 cases of distant metastasis were evaluated by two independent reviewers. The primary outcome measured was the quality of the therapy recommendations measured by the following parameters: clinical recommendation, explanation, and summarization.

Results

In this study, ChatGPT 4o and 4.0 provided mostly general answers for surgery, palliative care, or systemic therapy. ChatGPT 4o proved to be 48.5% faster than ChatGPT 4.0. For clinical recommendation, explanation, and summarization both LLMs obtained high scores in terms of performance of therapy recommendations, with no significant differences between both LLMs, but demonstrated to be mostly an assisting tool, requiring validation by an experienced clinician due to a lack of transparency and sometimes recommending treatment modalities that are not part of the current treatment guidelines.

Conclusion

This research demonstrates that ChatGPT 4o and 4.0 share a similar performance, while ChatGPT 4o is significantly faster. Since the current versions cannot tailor therapy recommendations, and sometimes recommend incorrect treatment options and lack information on the source material, advanced AI models at the moment can merely assist in the MDT setting for recurrent/metastatic HNSCC.

Integrating Artificial Intelligence (AI) Chatbots for Depression Management: A New Frontier in Primary Care

Depression is a prevalent mental health disorder that significantly impacts primary care settings. This editorial explores the potential of artificial intelligence (AI)-powered chatbots in managing depression within primary care environments. AI chatbots offer innovative solutions to challenges faced by healthcare providers, including limited appointment times, delayed access to specialists, and stigma associated with mental health issues. These digital tools provide continuous support, personalized interactions, and early symptom detection, potentially improving accessibility and outcomes in depression management. The integration of AI chatbots in primary care presents opportunities for round-the-clock patient support, personalized interventions, and the reduction of mental health stigma. However, challenges persist, including concerns about assessment accuracy, data privacy, and integration with existing healthcare systems. Successful implementation requires systematic approaches, stakeholder engagement, and comprehensive training for healthcare providers. Ethical considerations, such as ensuring informed consent, managing algorithmic biases, and maintaining the human element in care, are crucial for responsible deployment. As AI technology evolves, future directions may include enhanced natural language processing, multimodal integration, and AI-augmented clinical decision support. This editorial emphasizes the need for a balanced approach that leverages the potential of AI while acknowledging its limitations and the irreplaceable value of human clinical judgment in depression management within primary care settings.

Can artificial intelligence models serve as patient information consultants in orthodontics?

BACKGROUND

To evaluate the accuracy, reliability, quality, and readability of responses generated by ChatGPT-3.5, ChatGPT-4, Gemini, and Copilot in relation to orthodontic clear aligners.

METHODS

Frequently asked questions by patients/laypersons about clear aligners on websites were identified using the Google search tool and these questions were posed to ChatGPT-3.5, ChatGPT-4, Gemini, and Copilot AI models. Responses were assessed using a five-point Likert scale for accuracy, the modified DISCERN scale for reliability, the Global Quality Scale (GQS) for quality, and the Flesch Reading Ease Score (FRES) for readability.

RESULTS

ChatGPT-4 responses had the highest mean Likert score (4.5 ± 0.61), followed by Copilot (4.35 ± 0.81), ChatGPT-3.5 (4.15 ± 0.75) and Gemini (4.1 ± 0.72). The difference between the Likert scores of the chatbot models was not statistically significant (p > 0.05). Copilot had a significantly higher modified DISCERN and GQS score compared to both Gemini, ChatGPT-4 and ChatGPT-3.5 (p < 0.05). Gemini's modified DISCERN and GQS score was statistically higher than ChatGPT-3.5 (p < 0.05). Gemini also had a significantly higher FRES compared to both ChatGPT-4, Copilot and ChatGPT-3.5 (p < 0.05). The mean FRES was 38.39 ± 11.56 for ChatGPT-3.5, 43.88 ± 10.13 for ChatGPT-4 and 41.72 ± 10.74 for Copilot, indicating that the responses were difficult to read according to the reading level. The mean FRES for Gemini is 54.12 ± 10.27, indicating that Gemini's responses are more readable than other chatbots.

CONCLUSIONS

All chatbot models provided generally accurate, moderate reliable and moderate to good quality answers to questions about the clear aligners. Furthermore, the readability of the responses was difficult. ChatGPT, Gemini and Copilot have significant potential as patient information tools in orthodontics, however, to be fully effective they need to be supplemented with more evidence-based information and improved readability.

The potential of large language model chatbots for application to epilepsy: Let's talk about physical exercise

In this paper, we discuss how artificial intelligence chatbots based on large-scale language models (LLMs) can be used to disseminate information about the benefits of physical exercise for individuals with epilepsy. LLMs have demonstrated the ability to generate increasingly detailed text and allow structured dialogs. These can be useful tools, providing guidance and advice to people with epilepsy on different forms of treatment as well as physical exercise. We also examine the limitations of LLMs, which include the need for human supervision and the risk of providing imprecise and unreliable information regarding specific or controversial aspects of the topic. Despite these challenges, LLM chatbots have demonstrated the potential to support the management of epilepsy and break down barriers to information access, particularly information on physical exercise.

Exploring ChatGPT's potential in the clinical stream of neurorehabilitation

In several medical fields, generative AI tools such as ChatGPT have achieved optimal performance in identifying correct diagnoses only by evaluating narrative clinical descriptions of cases. The most active fields of application include oncology and COVID-19-related symptoms, with preliminary relevant results also in psychiatric and neurological domains. This scoping review aims to introduce the arrival of ChatGPT applications in neurorehabilitation practice, where such AI-driven solutions have the potential to revolutionize patient care and assistance. First, a comprehensive overview of ChatGPT, including its design, and potential applications in medicine is provided. Second, the remarkable natural language processing skills and limitations of these models are examined with a focus on their use in neurorehabilitation. In this context, we present two case scenarios to evaluate ChatGPT ability to resolve higher-order clinical reasoning. Overall, we provide support to the first evidence that generative AI can meaningfully integrate as a facilitator into neurorehabilitation practice, aiding physicians in defining increasingly efficacious diagnostic and personalized prognostic plans.

ChatGPT vs. web search for patient questions: what does ChatGPT do better?

PURPOSE

Chat generative pretrained transformer (ChatGPT) has the potential to significantly impact how patients acquire medical information online. Here, we characterize the readability and appropriateness of ChatGPT responses to a range of patient questions compared to results from traditional web searches.

METHODS

Patient questions related to the published Clinical Practice Guidelines by the American Academy of Otolaryngology-Head and Neck Surgery were sourced from existing online posts. Questions were categorized using a modified Rothwell classification system into (1) fact, (2) policy, and (3) diagnosis and recommendations. These were queried using ChatGPT and traditional web search. All results were evaluated on readability (Flesch Reading Ease and Flesch-Kinkaid Grade Level) and understandability (Patient Education Materials Assessment Tool). Accuracy was assessed by two blinded clinical evaluators using a three-point ordinal scale.

RESULTS

54 questions were organized into fact (37.0%), policy (37.0%), and diagnosis (25.8%). The average readability for ChatGPT responses was lower than traditional web search (FRE: 42.3 ± 13.1 vs. 55.6 ± 10.5, p < 0.001), while the PEMAT understandability was equivalent (93.8% vs. 93.5%, p = 0.17). ChatGPT scored higher than web search for questions the 'Diagnosis' category (p < 0.01); there was no difference in questions categorized as 'Fact' (p = 0.15) or 'Policy' (p = 0.22). Additional prompting improved ChatGPT response readability (FRE 55.6 ± 13.6, p < 0.01).

CONCLUSIONS

ChatGPT outperforms web search in answering patient questions related to symptom-based diagnoses and is equivalent in providing medical facts and established policy. Appropriate prompting can further improve readability while maintaining accuracy. Further patient education is needed to relay the benefits and limitations of this technology as a source of medial information.

Can Artificial Intelligence "Hold" a Dermoscope?-The Evaluation of an Artificial Intelligence Chatbot to Translate the Dermoscopic Language

This survey represents the first endeavor to assess the clarity of the dermoscopic language by a chatbot, unveiling insights into the interplay between dermatologists and AI systems within the complexity of the dermoscopic language. Given the complex, descriptive, and metaphorical aspects of the dermoscopic language, subjective interpretations often emerge. The survey evaluated the completeness and diagnostic efficacy of chatbot-generated reports, focusing on their role in facilitating accurate diagnoses and educational opportunities for novice dermatologists. A total of 30 participants were presented with hypothetical dermoscopic descriptions of skin lesions, including dermoscopic descriptions of skin cancers such as BCC, SCC, and melanoma, skin cancer mimickers such as actinic and seborrheic keratosis, dermatofibroma, and atypical nevus, and inflammatory dermatosis such as psoriasis and alopecia areata. Each description was accompanied by specific clinical information, and the participants were tasked with assessing the differential diagnosis list generated by the AI chatbot in its initial response. In each scenario, the chatbot generated an extensive list of potential differential diagnoses, exhibiting lower performance in cases of SCC and inflammatory dermatoses, albeit without statistical significance, suggesting that the participants were equally satisfied with the responses provided. Scores decreased notably when practical descriptions of dermoscopic signs were provided. Answers to BCC scenario scores in the diagnosis category (2.9 ± 0.4) were higher than those with SCC (2.6 ± 0.66, p = 0.005) and inflammatory dermatoses (2.6 ± 0.67, p = 0). Similarly, in the teaching tool usefulness category, BCC-based chatbot differential diagnosis received higher scores (2.9 ± 0.4) compared to SCC (2.6 ± 0.67, p = 0.001) and inflammatory dermatoses (2.4 ± 0.81, p = 0). The abovementioned results underscore dermatologists' familiarity with BCC dermoscopic images while highlighting the challenges associated with interpreting rigorous dermoscopic images. Moreover, by incorporating patient characteristics such as age, phototype, or immune state, the differential diagnosis list in each case was customized to include lesion types appropriate for each category, illustrating the AI's flexibility in evaluating diagnoses and highlighting its value as a resource for dermatologists.

Assessing the role of advanced artificial intelligence as a tool in multidisciplinary tumor board decision-making for primary head and neck cancer cases

Background

Head and neck squamous cell carcinoma (HNSCC) is a complex malignancy that requires a multidisciplinary approach in clinical practice, especially in tumor board discussions. In recent years, artificial intelligence has emerged as a tool to assist healthcare professionals in making informed decisions. This study investigates the application of ChatGPT 3.5 and ChatGPT 4.0, natural language processing models, in tumor board decision-making.

Methods

We conducted a pilot study in October 2023 on 20 consecutive head and neck cancer patients discussed in our multidisciplinary tumor board (MDT). Patients with a primary diagnosis of head and neck cancer were included. The MDT and ChatGPT 3.5 and ChatGPT 4.0 recommendations for each patient were compared by two independent reviewers and the number of therapy options, the clinical recommendation, the explanation and the summarization were graded.

Results

In this study, ChatGPT 3.5 provided mostly general answers for surgery, chemotherapy, and radiation therapy. For clinical recommendation, explanation and summarization ChatGPT 3.5 and 4.0 scored well, but demonstrated to be mostly an assisting tool, suggesting significantly more therapy options than our MDT, while some of the recommended treatment modalities like primary immunotherapy are not part of the current treatment guidelines.

Conclusions

This research demonstrates that advanced AI models at the moment can merely assist in the MDT setting, since the current versions list common therapy options, but sometimes recommend incorrect treatment options and in the case of ChatGPT 3.5 lack information on the source material.

Regulatory and Ethical Considerations on Artificial Intelligence for Occupational Medicine

Generative artificial intelligence and Large Language Models are reshaping labor dynamics and occupational health practices. As AI continues to evolve, there's a critical need to customize ethical considerations for its specific impacts on occupational health. Recognizing potential ethical challenges and dilemmas, stakeholders and physicians are urged to proactively adjust the practice of occupational medicine in response to shifting ethical paradigms. By advocating for a comprehensive review of the International Commission on Occupational Health ICOH code of Ethics, we can ensure responsible medical AI deployment, safeguarding the well-being of workers amidst the transformative effects of automation in healthcare.

ChatGPT as an aid for pathological diagnosis of cancer

Diagnostic workup of cancer patients is highly reliant on the science of pathology using cytopathology, histopathology, and other ancillary techniques like immunohistochemistry and molecular cytogenetics. Data processing and learning by means of artificial intelligence (AI) has become a spearhead for the advancement of medicine, with pathology and laboratory medicine being no exceptions. ChatGPT, an artificial intelligence (AI)-based chatbot, that was recently launched by OpenAI, is currently a talk of the town, and its role in cancer diagnosis is also being explored meticulously. Pathology workflow by integration of digital slides, implementation of advanced algorithms, and computer-aided diagnostic techniques extend the frontiers of the pathologist's view beyond a microscopic slide and enables effective integration, assimilation, and utilization of knowledge that is beyond human limits and boundaries. Despite of it's numerous advantages in the pathological diagnosis of cancer, it comes with several challenges like integration of digital slides with input language parameters, problems of bias, and legal issues which have to be addressed and worked up soon so that we as a pathologists diagnosing malignancies are on the same band wagon and don't miss the train.