1
|
Kachman MM, Brennan I, Oskvarek JJ, Waseem T, Pines JM. How artificial intelligence could transform emergency care. Am J Emerg Med 2024; 81:40-46. [PMID: 38663302 DOI: 10.1016/j.ajem.2024.04.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2024] [Revised: 04/13/2024] [Accepted: 04/15/2024] [Indexed: 06/07/2024] Open
Abstract
Artificial intelligence (AI) in healthcare is the ability of a computer to perform tasks typically associated with clinical care (e.g. medical decision-making and documentation). AI will soon be integrated into an increasing number of healthcare applications, including elements of emergency department (ED) care. Here, we describe the basics of AI, various categories of its functions (including machine learning and natural language processing) and review emerging and potential future use-cases for emergency care. For example, AI-assisted symptom checkers could help direct patients to the appropriate setting, models could assist in assigning triage levels, and ambient AI systems could document clinical encounters. AI could also help provide focused summaries of charts, summarize encounters for hand-offs, and create discharge instructions with an appropriate language and reading level. Additional use cases include medical decision making for decision rules, real-time models that predict clinical deterioration or sepsis, and efficient extraction of unstructured data for coding, billing, research, and quality initiatives. We discuss the potential transformative benefits of AI, as well as the concerns regarding its use (e.g. privacy, data accuracy, and the potential for changing the doctor-patient relationship).
Collapse
Affiliation(s)
- Marika M Kachman
- US Acute Care Solutions, Canton, OH, United States of America; Department of Emergency Medicine, Virginia Hospital Center, Arlington, VA, United States of America
| | - Irina Brennan
- US Acute Care Solutions, Canton, OH, United States of America; Department of Emergency Medicine, Inova Alexandria Hospital, Alexandria, VA, United States of America
| | - Jonathan J Oskvarek
- US Acute Care Solutions, Canton, OH, United States of America; Department of Emergency Medicine, Summa Health, Akron, OH, United States of America
| | - Tayab Waseem
- Department of Emergency Medicine, George Washington University, Washington, DC, United States of America
| | - Jesse M Pines
- US Acute Care Solutions, Canton, OH, United States of America; Department of Emergency Medicine, George Washington University, Washington, DC, United States of America.
| |
Collapse
|
2
|
Meczner A, Cohen N, Qureshi A, Reza M, Sutaria S, Blount E, Bagyura Z, Malak T. Controlling Inputter Variability in Vignette Studies Assessing Web-Based Symptom Checkers: Evaluation of Current Practice and Recommendations for Isolated Accuracy Metrics. JMIR Form Res 2024; 8:e49907. [PMID: 38820578 PMCID: PMC11179013 DOI: 10.2196/49907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/10/2023] [Accepted: 04/24/2024] [Indexed: 06/02/2024] Open
Abstract
BACKGROUND The rapid growth of web-based symptom checkers (SCs) is not matched by advances in quality assurance. Currently, there are no widely accepted criteria assessing SCs' performance. Vignette studies are widely used to evaluate SCs, measuring the accuracy of outcome. Accuracy behaves as a composite metric as it is affected by a number of individual SC- and tester-dependent factors. In contrast to clinical studies, vignette studies have a small number of testers. Hence, measuring accuracy alone in vignette studies may not provide a reliable assessment of performance due to tester variability. OBJECTIVE This study aims to investigate the impact of tester variability on the accuracy of outcome of SCs, using clinical vignettes. It further aims to investigate the feasibility of measuring isolated aspects of performance. METHODS Healthily's SC was assessed using 114 vignettes by 3 groups of 3 testers who processed vignettes with different instructions: free interpretation of vignettes (free testers), specified chief complaints (partially free testers), and specified chief complaints with strict instruction for answering additional symptoms (restricted testers). κ statistics were calculated to assess agreement of top outcome condition and recommended triage. Crude and adjusted accuracy was measured against a gold standard. Adjusted accuracy was calculated using only results of consultations identical to the vignette, following a review and selection process. A feasibility study for assessing symptom comprehension of SCs was performed using different variations of 51 chief complaints across 3 SCs. RESULTS Intertester agreement of most likely condition and triage was, respectively, 0.49 and 0.51 for the free tester group, 0.66 and 0.66 for the partially free group, and 0.72 and 0.71 for the restricted group. For the restricted group, accuracy ranged from 43.9% to 57% for individual testers, averaging 50.6% (SD 5.35%). Adjusted accuracy was 56.1%. Assessing symptom comprehension was feasible for all 3 SCs. Comprehension scores ranged from 52.9% and 68%. CONCLUSIONS We demonstrated that by improving standardization of the vignette testing process, there is a significant improvement in the agreement of outcome between testers. However, significant variability remained due to uncontrollable tester-dependent factors, reflected by varying outcome accuracy. Tester-dependent factors, combined with a small number of testers, limit the reliability and generalizability of outcome accuracy when used as a composite measure in vignette studies. Measuring and reporting different aspects of SC performance in isolation provides a more reliable assessment of SC performance. We developed an adjusted accuracy measure using a review and selection process to assess data algorithm quality. In addition, we demonstrated that symptom comprehension with different input methods can be feasibly compared. Future studies reporting accuracy need to apply vignette testing standardization and isolated metrics.
Collapse
Affiliation(s)
- András Meczner
- Healthily, London, United Kingdom
- Institute for Clinical Data Management, Semmelweis University, Budapest, Hungary
| | | | | | | | | | | | - Zsolt Bagyura
- Institute for Clinical Data Management, Semmelweis University, Budapest, Hungary
| | | |
Collapse
|
3
|
Hammoud M, Douglas S, Darmach M, Alawneh S, Sanyal S, Kanbour Y. Evaluating the Diagnostic Performance of Symptom Checkers: Clinical Vignette Study. JMIR AI 2024; 3:e46875. [PMID: 38875676 PMCID: PMC11091811 DOI: 10.2196/46875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 06/15/2023] [Accepted: 03/02/2024] [Indexed: 06/16/2024]
Abstract
BACKGROUND Medical self-diagnostic tools (or symptom checkers) are becoming an integral part of digital health and our daily lives, whereby patients are increasingly using them to identify the underlying causes of their symptoms. As such, it is essential to rigorously investigate and comprehensively report the diagnostic performance of symptom checkers using standard clinical and scientific approaches. OBJECTIVE This study aims to evaluate and report the accuracies of a few known and new symptom checkers using a standard and transparent methodology, which allows the scientific community to cross-validate and reproduce the reported results, a step much needed in health informatics. METHODS We propose a 4-stage experimentation methodology that capitalizes on the standard clinical vignette approach to evaluate 6 symptom checkers. To this end, we developed and peer-reviewed 400 vignettes, each approved by at least 5 out of 7 independent and experienced primary care physicians. To establish a frame of reference and interpret the results of symptom checkers accordingly, we further compared the best-performing symptom checker against 3 primary care physicians with an average experience of 16.6 (SD 9.42) years. To measure accuracy, we used 7 standard metrics, including M1 as a measure of a symptom checker's or a physician's ability to return a vignette's main diagnosis at the top of their differential list, F1-score as a trade-off measure between recall and precision, and Normalized Discounted Cumulative Gain (NDCG) as a measure of a differential list's ranking quality, among others. RESULTS The diagnostic accuracies of the 6 tested symptom checkers vary significantly. For instance, the differences in the M1, F1-score, and NDCG results between the best-performing and worst-performing symptom checkers or ranges were 65.3%, 39.2%, and 74.2%, respectively. The same was observed among the participating human physicians, whereby the M1, F1-score, and NDCG ranges were 22.8%, 15.3%, and 21.3%, respectively. When compared against each other, physicians outperformed the best-performing symptom checker by an average of 1.2% using F1-score, whereas the best-performing symptom checker outperformed physicians by averages of 10.2% and 25.1% using M1 and NDCG, respectively. CONCLUSIONS The performance variation between symptom checkers is substantial, suggesting that symptom checkers cannot be treated as a single entity. On a different note, the best-performing symptom checker was an artificial intelligence (AI)-based one, shedding light on the promise of AI in improving the diagnostic capabilities of symptom checkers, especially as AI keeps advancing exponentially.
Collapse
|
4
|
Wimbarti S, Kairupan BHR, Tallei TE. Critical review of self-diagnosis of mental health conditions using artificial intelligence. Int J Ment Health Nurs 2024; 33:344-358. [PMID: 38345132 DOI: 10.1111/inm.13303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 01/26/2024] [Accepted: 01/30/2024] [Indexed: 03/10/2024]
Abstract
The advent of artificial intelligence (AI) has revolutionised various aspects of our lives, including mental health nursing. AI-driven tools and applications have provided a convenient and accessible means for individuals to assess their mental well-being within the confines of their homes. Nonetheless, the widespread trend of self-diagnosing mental health conditions through AI poses considerable risks. This review article examines the perils associated with relying on AI for self-diagnosis in mental health, highlighting the constraints and possible adverse outcomes that can arise from such practices. It delves into the ethical, psychological, and social implications, underscoring the vital role of mental health professionals, including psychologists, psychiatrists, and nursing specialists, in providing professional assistance and guidance. This article aims to highlight the importance of seeking professional assistance and guidance in addressing mental health concerns, especially in the era of AI-driven self-diagnosis.
Collapse
Affiliation(s)
- Supra Wimbarti
- Faculty of Psychology, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - B H Ralph Kairupan
- Department of Psychiatry, Faculty of Medicine, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
| | - Trina Ekawati Tallei
- Department of Biology, Faculty of Mathematics and Natural Sciences, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
- Department of Biology, Faculty of Medicine, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
| |
Collapse
|
5
|
Chorney W, Wang H. Towards federated transfer learning in electrocardiogram signal analysis. Comput Biol Med 2024; 170:107984. [PMID: 38244469 DOI: 10.1016/j.compbiomed.2024.107984] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 12/17/2023] [Accepted: 01/13/2024] [Indexed: 01/22/2024]
Abstract
Modern methods in artificial intelligence perform very well on many healthcare datasets, at times outperforming trained doctors. However, many assumptions made in model training are not justifiable in clinical settings. In this work, we propose a method to train classifiers for electrocardiograms, able to deal with data of disparate input dimensions, distributed across different institutions, and able to protect patient privacy. In addition, we propose a simple method for creating federated datasets from any centralized dataset. We use autoencoders in conjunction with federated learning to model a highly heterogeneous modeling problem using the Massachusetts Institute of Technology Beth Israel Hospital Arrhythmia dataset, the Computing in Cardiology 2017 challenge dataset, and the PTB-XL dataset. For an encoding dimension of 1000, our federated classifier achieves an accuracy, precision, recall, and F1 score of 73.0%, 66.6%, 73.0%, and 69.7%, respectively. Our results suggest that dropping commonly made assumptions significantly complicate training and that as a result, estimates of performance of many machine learning models may overestimate performance when adopted for clinical settings.
Collapse
Affiliation(s)
- Wesley Chorney
- Computational Engineering, Mississippi State University, Mississippi State, 39762, USA.
| | - Haifeng Wang
- Industrial and Systems Engineering, Mississippi State University, Mississippi State, 39762, USA.
| |
Collapse
|
6
|
Li J, Dada A, Puladi B, Kleesiek J, Egger J. ChatGPT in healthcare: A taxonomy and systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 245:108013. [PMID: 38262126 DOI: 10.1016/j.cmpb.2024.108013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 12/29/2023] [Accepted: 01/08/2024] [Indexed: 01/25/2024]
Abstract
The recent release of ChatGPT, a chat bot research project/product of natural language processing (NLP) by OpenAI, stirs up a sensation among both the general public and medical professionals, amassing a phenomenally large user base in a short time. This is a typical example of the 'productization' of cutting-edge technologies, which allows the general public without a technical background to gain firsthand experience in artificial intelligence (AI), similar to the AI hype created by AlphaGo (DeepMind Technologies, UK) and self-driving cars (Google, Tesla, etc.). However, it is crucial, especially for healthcare researchers, to remain prudent amidst the hype. This work provides a systematic review of existing publications on the use of ChatGPT in healthcare, elucidating the 'status quo' of ChatGPT in medical applications, for general readers, healthcare professionals as well as NLP scientists. The large biomedical literature database PubMed is used to retrieve published works on this topic using the keyword 'ChatGPT'. An inclusion criterion and a taxonomy are further proposed to filter the search results and categorize the selected publications, respectively. It is found through the review that the current release of ChatGPT has achieved only moderate or 'passing' performance in a variety of tests, and is unreliable for actual clinical deployment, since it is not intended for clinical applications by design. We conclude that specialized NLP models trained on (bio)medical datasets still represent the right direction to pursue for critical clinical applications.
Collapse
Affiliation(s)
- Jianning Li
- Institute for Artificial Intelligence in Medicine, University Hospital Essen (AöR), Girardetstraße 2, 45131 Essen, Germany
| | - Amin Dada
- Institute for Artificial Intelligence in Medicine, University Hospital Essen (AöR), Girardetstraße 2, 45131 Essen, Germany
| | - Behrus Puladi
- Institute of Medical Informatics, University Hospital RWTH Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Department of Oral and Maxillofacial Surgery, University Hospital RWTH Aachen, Pauwelsstraße 30, 52074 Aachen, Germany
| | - Jens Kleesiek
- Institute for Artificial Intelligence in Medicine, University Hospital Essen (AöR), Girardetstraße 2, 45131 Essen, Germany; TU Dortmund University, Department of Physics, Otto-Hahn-Straße 4, 44227 Dortmund, Germany
| | - Jan Egger
- Institute for Artificial Intelligence in Medicine, University Hospital Essen (AöR), Girardetstraße 2, 45131 Essen, Germany; Center for Virtual and Extended Reality in Medicine (ZvRM), University Hospital Essen, University Medicine Essen, Hufelandstraße 55, 45147 Essen, Germany.
| |
Collapse
|
7
|
Bernardi S, Vallati M, Gatta R. Artificial Intelligence-Based Management of Adult Chronic Myeloid Leukemia: Where Are We and Where Are We Going? Cancers (Basel) 2024; 16:848. [PMID: 38473210 DOI: 10.3390/cancers16050848] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Revised: 02/08/2024] [Accepted: 02/15/2024] [Indexed: 03/14/2024] Open
Abstract
Artificial intelligence (AI) is emerging as a discipline capable of providing significant added value in Medicine, in particular in radiomic, imaging analysis, big dataset analysis, and also for generating virtual cohort of patients. However, in coping with chronic myeloid leukemia (CML), considered an easily managed malignancy after the introduction of TKIs which strongly improved the life expectancy of patients, AI is still in its infancy. Noteworthy, the findings of initial trials are intriguing and encouraging, both in terms of performance and adaptability to different contexts in which AI can be applied. Indeed, the improvement of diagnosis and prognosis by leveraging biochemical, biomolecular, imaging, and clinical data can be crucial for the implementation of the personalized medicine paradigm or the streamlining of procedures and services. In this review, we present the state of the art of AI applications in the field of CML, describing the techniques and objectives, and with a general focus that goes beyond Machine Learning (ML), but instead embraces the wider AI field. The present scooping review spans on publications reported in Pubmed from 2003 to 2023, and resulting by searching "chronic myeloid leukemia" and "artificial intelligence". The time frame reflects the real literature production and was not restricted. We also take the opportunity for discussing the main pitfalls and key points to which AI must respond, especially considering the critical role of the 'human' factor, which remains key in this domain.
Collapse
Affiliation(s)
- Simona Bernardi
- Department of Clinical and Experimental Sciences, University of Brescia, 25123 Brescia, Italy
- CREA-Centro di Ricerca Emato-Oncologica AIL, ASST Spedali Civili of Brescia, 25123 Brescia, Italy
| | - Mauro Vallati
- School of Computing and Engineering, University of Huddersfield, Huddersfield HD1 3DH, UK
| | - Roberto Gatta
- Department of Clinical and Experimental Sciences, University of Brescia, 25123 Brescia, Italy
| |
Collapse
|
8
|
Cao B, Huang S, Tang W. AI triage or manual triage? Exploring medical staffs' preference for AI triage in China. PATIENT EDUCATION AND COUNSELING 2024; 119:108076. [PMID: 38029576 DOI: 10.1016/j.pec.2023.108076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 10/23/2023] [Accepted: 11/13/2023] [Indexed: 12/01/2023]
Abstract
OBJECTIVES The introduction of AI technology in healthcare presents both opportunities and challenges. The aim of this study was to investigate medical staffs' preference for AI triage and the influencing factors. METHODS A survey was conducted online among medical staffs in China from March 4th to April 28th, 2021. Participants were recruited through multiple channels, including medical professional platforms and social media. A total of 677 valid responses were obtained from medical staff members located in 28 provinces across China. RESULTS The results showed that AI triage had an overall acceptance rate of 77.1%, and 45.2% of the medical staffs surveyed preferred "AI triage exclusively." Direct experience was positively associated with medical staffs' preference for AI triage (β = 0.223, p < .001). Additionally, greater exposure to a variety of media was positively associated with the perceived value of AI technology, which, in turn, increased preference for AI triage (β = 0.040, SE = 0.013, p < .001, 95% CI = [0.017, 0.067]). CONCLUSION Medical staffs generally hold a favorable attitude towards AI triage, particularly in areas with a high medical burden and during pandemics. In a multimedia environment, media exposure variety impacts medical staffs' preferences through their perceived value of AI technology. This study has implications for the implementation of AI triage on a larger scale.
Collapse
Affiliation(s)
- Bolin Cao
- School of Media and Communication, Shenzhen University, Shenzhen, China.
| | - Shiyi Huang
- School of Media and Communication, Shenzhen University, Shenzhen, China
| | - Weiming Tang
- Division of Infectious Disease, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|
9
|
Dubinski D, Won SY, Trnovec S, Behmanesh B, Baumgarten P, Dinc N, Konczalla J, Chan A, Bernstock JD, Freiman TM, Gessler F. Leveraging artificial intelligence in neurosurgery-unveiling ChatGPT for neurosurgical discharge summaries and operative reports. Acta Neurochir (Wien) 2024; 166:38. [PMID: 38277081 PMCID: PMC10817836 DOI: 10.1007/s00701-024-05908-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2023] [Accepted: 11/06/2023] [Indexed: 01/27/2024]
Abstract
PURPOSE Chat generative pre-trained transformer (GPT) is a novel large pre-trained natural language processing software that can enable scientific writing amongst a litany of other features. Given this, there is a growing interest in exploring the use of ChatGPT models as a modality to facilitate/assist in the provision of clinical care. METHODS We investigated the time taken for the composition of neurosurgical discharge summaries and operative reports at a major University hospital. In so doing, we compared currently employed speech recognition software (i.e., SpeaKING) vs novel ChatGPT for three distinct neurosurgical diseases: chronic subdural hematoma, spinal decompression, and craniotomy. Furthermore, factual correctness was analyzed for the abovementioned diseases. RESULTS The composition of neurosurgical discharge summaries and operative reports with the assistance of ChatGPT leads to a statistically significant time reduction across all three diseases/report types: p < 0.001 for chronic subdural hematoma, p < 0.001 for decompression of spinal stenosis, and p < 0.001 for craniotomy and tumor resection. However, despite a high degree of factual correctness, the preparation of a surgical report for craniotomy proved to be significantly lower (p = 0.002). CONCLUSION ChatGPT assisted in the writing of discharge summaries and operative reports as evidenced by an impressive reduction in time spent as compared to standard speech recognition software. While promising, the optimal use cases and ethics of AI-generated medical writing remain to be fully elucidated and must be further explored in future studies.
Collapse
Affiliation(s)
- Daniel Dubinski
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany.
| | - Sae-Yeon Won
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany
| | - Svorad Trnovec
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany
| | - Bedjan Behmanesh
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany
| | - Peter Baumgarten
- Department of Neurosurgery, University Hospital, Schiller University Jena, Jena, Germany
| | - Nazife Dinc
- Department of Neurosurgery, University Hospital, Schiller University Jena, Jena, Germany
| | - Juergen Konczalla
- Department of Neurosurgery, Goethe-University Hospital, Frankfurt am Main, Germany
| | - Alvin Chan
- David H. Koch Institute for Integrated Cancer Research, MIT, Cambridge, MA, USA
| | - Joshua D Bernstock
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Thomas M Freiman
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany
| | - Florian Gessler
- Department of Neurosurgery, University Medicine Rostock, Rostock, Germany
| |
Collapse
|
10
|
Oniani D, Hilsman J, Peng Y, Poropatich RK, Pamplin JC, Legault GL, Wang Y. Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare. NPJ Digit Med 2023; 6:225. [PMID: 38042910 PMCID: PMC10693640 DOI: 10.1038/s41746-023-00965-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Accepted: 11/15/2023] [Indexed: 12/04/2023] Open
Abstract
In 2020, the U.S. Department of Defense officially disclosed a set of ethical principles to guide the use of Artificial Intelligence (AI) technologies on future battlefields. Despite stark differences, there are core similarities between the military and medical service. Warriors on battlefields often face life-altering circumstances that require quick decision-making. Medical providers experience similar challenges in a rapidly changing healthcare environment, such as in the emergency department or during surgery treating a life-threatening condition. Generative AI, an emerging technology designed to efficiently generate valuable information, holds great promise. As computing power becomes more accessible and the abundance of health data, such as electronic health records, electrocardiograms, and medical images, increases, it is inevitable that healthcare will be revolutionized by this technology. Recently, generative AI has garnered a lot of attention in the medical research community, leading to debates about its application in the healthcare sector, mainly due to concerns about transparency and related issues. Meanwhile, questions around the potential exacerbation of health disparities due to modeling biases have raised notable ethical concerns regarding the use of this technology in healthcare. However, the ethical principles for generative AI in healthcare have been understudied. As a result, there are no clear solutions to address ethical concerns, and decision-makers often neglect to consider the significance of ethical principles before implementing generative AI in clinical practice. In an attempt to address these issues, we explore ethical principles from the military perspective and propose the "GREAT PLEA" ethical principles, namely Governability, Reliability, Equity, Accountability, Traceability, Privacy, Lawfulness, Empathy, and Eutonomy, for generative AI in healthcare. Furthermore, we introduce a framework for adopting and expanding these ethical principles in a practical way that has been useful in the military and can be applied to healthcare for generative AI, based on contrasting their ethical concerns and risks. Ultimately, we aim to proactively address the ethical dilemmas and challenges posed by the integration of generative AI into healthcare practice.
Collapse
Affiliation(s)
- David Oniani
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA
| | - Jordan Hilsman
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA
| | - Yifan Peng
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
| | - Ronald K Poropatich
- Division of Pulmonary, Allergy, Critical Care & Sleep Medicine, University of Pittsburgh, Pittsburgh, PA, USA
- Center for Military Medicine Research, University of Pittsburgh, Pittsburgh, PA, USA
| | - Jeremy C Pamplin
- Telemedicine & Advanced Technology Research Center, US Army, Fort Detrick, Frederick, MD, USA
| | - Gary L Legault
- Department of Surgery, Uniformed Services University, Bethesda, MD, USA
- Virtual Medical Center, Brooke Army Medical Center, San Antonio, TX, USA
| | - Yanshan Wang
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA.
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA.
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA.
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA.
- University of Pittsburgh Medical Center, Pittsburgh, PA, USA.
| |
Collapse
|
11
|
Weerarathna IN, Kamble AR, Luharia A. Artificial Intelligence Applications for Biomedical Cancer Research: A Review. Cureus 2023; 15:e48307. [PMID: 38058345 PMCID: PMC10697339 DOI: 10.7759/cureus.48307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 11/05/2023] [Indexed: 12/08/2023] Open
Abstract
Artificial intelligence (AI) has rapidly evolved and demonstrated its potential in transforming biomedical cancer research, offering innovative solutions for cancer diagnosis, treatment, and overall patient care. Over the past two decades, AI has played a pivotal role in revolutionizing various facets of cancer clinical research. In this comprehensive review, we delve into the diverse applications of AI across the cancer care continuum, encompassing radiodiagnosis, radiotherapy, chemotherapy, immunotherapy, targeted therapy, surgery, and nanotechnology. AI has revolutionized cancer diagnosis, enabling early detection and precise characterization through advanced image analysis techniques. In radiodiagnosis, AI-driven algorithms enhance the accuracy of medical imaging, making it an invaluable tool for clinicians in the detection and assessment of cancer. AI has also revolutionized radiotherapy, facilitating precise tumor boundary delineation, optimizing treatment planning, and enabling real-time adjustments to improve therapeutic outcomes while minimizing collateral damage to healthy tissues. In chemotherapy, AI models have emerged as powerful tools for predicting patient responses to different treatment regimens, allowing for more personalized and effective strategies. In immunotherapy, AI analyzes genetic and imaging data to select ideal candidates for treatment and predict responses. Targeted therapy has seen great advancements with AI, aiding in the identification of specific molecular targets for tailored treatments. AI plays a vital role in surgery by offering real-time navigation and support, enhancing surgical precision. Moreover, the synergy between AI and nanotechnology promises the development of personalized nanomedicines, offering more efficient and targeted cancer treatments. While challenges related to data quality, interpretability, and ethical considerations persist, the future of AI in cancer research holds tremendous promise for improving patient outcomes through advanced and individualized care.
Collapse
Affiliation(s)
- Induni N Weerarathna
- Biomedical Sciences, School of Allied Health Sciences, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| | - Aahash R Kamble
- Artificial Intelligence and Data Science, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| | - Anurag Luharia
- Radiotherapy, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| |
Collapse
|
12
|
Martinengo L, Lin X, Jabir AI, Kowatsch T, Atun R, Car J, Tudor Car L. Conversational Agents in Health Care: Expert Interviews to Inform the Definition, Classification, and Conceptual Framework. J Med Internet Res 2023; 25:e50767. [PMID: 37910153 PMCID: PMC10652195 DOI: 10.2196/50767] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 09/21/2023] [Accepted: 09/29/2023] [Indexed: 11/03/2023] Open
Abstract
BACKGROUND Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in health care settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes and unreliable research findings. We developed and published a conceptual framework, designing, developing, evaluating, and implementing a smartphone-delivered, rule-based conversational agent (DISCOVER), consisting of 3 iterative stages of CA design, development, and evaluation and implementation, complemented by 2 cross-cutting themes (user-centered design and data privacy and security). OBJECTIVE This study aims to perform in-depth, semistructured interviews with multidisciplinary experts in health care CAs to share their views on the definition and classification of health care CAs and evaluate and validate the DISCOVER conceptual framework. METHODS We conducted one-on-one semistructured interviews via Zoom (Zoom Video Communications) with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed using thematic analysis. RESULTS Following participants' input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using ≥1 communication modality, such as text, voice, images, or video. CAs were classified by 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of health care intervention. Experts considered that the conceptual framework could be adapted for artificial intelligence-based CAs. However, despite recent advances in artificial intelligence, including large language models, the technology is not able to ensure safety and reliability in health care settings. Finally, aligned with participants' feedback, we present an updated iteration of the conceptual framework for health care conversational agents (CHAT) with key considerations for CA design, development, and evaluation and implementation, complemented by 3 cross-cutting themes: ethics, user involvement, and data privacy and security. CONCLUSIONS We present an expanded, validated CHAT and aim at guiding researchers from a variety of backgrounds and with different levels of expertise in the design, development, and evaluation and implementation of rule-based CAs in health care settings.
Collapse
Affiliation(s)
- Laura Martinengo
- Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore
| | - Xiaowen Lin
- Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore
| | - Ahmad Ishqi Jabir
- Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore
- Future Health Technologies Programme, Campus for Research Excellence and Technological Enterprise, Singapore-ETH Centre, Singapore, Singapore
| | - Tobias Kowatsch
- Future Health Technologies Programme, Campus for Research Excellence and Technological Enterprise, Singapore-ETH Centre, Singapore, Singapore
- Institute for Implementation Science in Health Care, University of Zurich, Zurich, Switzerland
- School of Medicine, University of St.Gallen, St.Gallen, Switzerland
- Centre for Digital Health Interventions, Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland
| | - Rifat Atun
- Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Harvard University, Cambridge, MA, United States
| | - Josip Car
- Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United Kingdom
| | - Lorainne Tudor Car
- Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United Kingdom
| |
Collapse
|
13
|
Choi E, Leonard KW, Jassal JS, Levin AM, Ramachandra V, Jones LR. Artificial Intelligence in Facial Plastic Surgery: A Review of Current Applications, Future Applications, and Ethical Considerations. Facial Plast Surg 2023; 39:454-459. [PMID: 37353051 DOI: 10.1055/s-0043-1770160] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2023] Open
Abstract
From virtual chat assistants to self-driving cars, artificial intelligence (AI) is often heralded as the technology that has and will continue to transform this generation. Among widely adopted applications in other industries, its potential use in medicine is being increasingly explored, where the vast amounts of data present in electronic health records and need for continuous improvements in patient care and workflow efficiency present many opportunities for AI implementation. Indeed, AI has already demonstrated capabilities for assisting in tasks such as documentation, image classification, and surgical outcome prediction. More specifically, this technology can be harnessed in facial plastic surgery, where the unique characteristics of the field lends itself well to specific applications. AI is not without its limitations, however, and the further adoption of AI in medicine and facial plastic surgery must necessarily be accompanied by discussion on the ethical implications and proper usage of AI in healthcare. In this article, we review current and potential uses of AI in facial plastic surgery, as well as its ethical ramifications.
Collapse
Affiliation(s)
- Elizabeth Choi
- Wayne State University School of Medicine, Detroit, Michigan
| | - Kyle W Leonard
- Department of Otolaryngology, Henry Ford Hospital, Detroit, Michigan
| | - Japnam S Jassal
- Department of Otolaryngology, Henry Ford Hospital, Detroit, Michigan
| | - Albert M Levin
- Department of Public Health Science, Henry Ford Health, Detroit, Michigan
- Center for Bioinformatics, Henry Ford Health, Detroit, Michigan
| | - Vikas Ramachandra
- Department of Public Health Science, Henry Ford Health, Detroit, Michigan
- Center for Bioinformatics, Henry Ford Health, Detroit, Michigan
| | - Lamont R Jones
- Department of Otolaryngology, Henry Ford Hospital, Detroit, Michigan
| |
Collapse
|
14
|
Iqbal J, Cortés Jaimes DC, Makineni P, Subramani S, Hemaida S, Thugu TR, Butt AN, Sikto JT, Kaur P, Lak MA, Augustine M, Shahzad R, Arain M. Reimagining Healthcare: Unleashing the Power of Artificial Intelligence in Medicine. Cureus 2023; 15:e44658. [PMID: 37799217 PMCID: PMC10549955 DOI: 10.7759/cureus.44658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/04/2023] [Indexed: 10/07/2023] Open
Abstract
Artificial intelligence (AI) has opened new medical avenues and revolutionized diagnostic and therapeutic practices, allowing healthcare providers to overcome significant challenges associated with cost, disease management, accessibility, and treatment optimization. Prominent AI technologies such as machine learning (ML) and deep learning (DL) have immensely influenced diagnostics, patient monitoring, novel pharmaceutical discoveries, drug development, and telemedicine. Significant innovations and improvements in disease identification and early intervention have been made using AI-generated algorithms for clinical decision support systems and disease prediction models. AI has remarkably impacted clinical drug trials by amplifying research into drug efficacy, adverse events, and candidate molecular design. AI's precision and analysis regarding patients' genetic, environmental, and lifestyle factors have led to individualized treatment strategies. During the COVID-19 pandemic, AI-assisted telemedicine set a precedent for remote healthcare delivery and patient follow-up. Moreover, AI-generated applications and wearable devices have allowed ambulatory monitoring of vital signs. However, apart from being immensely transformative, AI's contribution to healthcare is subject to ethical and regulatory concerns. AI-backed data protection and algorithm transparency should be strictly adherent to ethical principles. Vigorous governance frameworks should be in place before incorporating AI in mental health interventions through AI-operated chatbots, medical education enhancements, and virtual reality-based training. The role of AI in medical decision-making has certain limitations, necessitating the importance of hands-on experience. Therefore, reaching an optimal balance between AI's capabilities and ethical considerations to ensure impartial and neutral performance in healthcare applications is crucial. This narrative review focuses on AI's impact on healthcare and the importance of ethical and balanced incorporation to make use of its full potential.
Collapse
Affiliation(s)
| | - Diana Carolina Cortés Jaimes
- Epidemiology, Universidad Autónoma de Bucaramanga, Bucaramanga, COL
- Medicine, Pontificia Universidad Javeriana, Bogotá, COL
| | - Pallavi Makineni
- Medicine, All India Institute of Medical Sciences, Bhubaneswar, Bhubaneswar, IND
| | - Sachin Subramani
- Medicine and Surgery, Employees' State Insurance Corporation (ESIC) Medical College, Gulbarga, IND
| | - Sarah Hemaida
- Internal Medicine, Istanbul Okan University, Istanbul, TUR
| | - Thanmai Reddy Thugu
- Internal Medicine, Sri Padmavathi Medical College for Women, Sri Venkateswara Institute of Medical Sciences (SVIMS), Tirupati, IND
| | - Amna Naveed Butt
- Medicine/Internal Medicine, Allama Iqbal Medical College, Lahore, PAK
| | | | - Pareena Kaur
- Medicine, Punjab Institute of Medical Sciences, Jalandhar, IND
| | | | | | - Roheen Shahzad
- Medicine, Combined Military Hospital (CMH) Lahore Medical College and Institute of Dentistry, Lahore, PAK
| | - Mustafa Arain
- Internal Medicine, Civil Hospital Karachi, Karachi, PAK
| |
Collapse
|
15
|
Alexeenko V, Jeevaratnam K. Artificial intelligence: Is it wizardry, witchcraft, or a helping hand for an equine veterinarian? Equine Vet J 2023; 55:719-722. [PMID: 37551620 DOI: 10.1111/evj.13969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 06/14/2023] [Indexed: 08/09/2023]
Affiliation(s)
- Vadim Alexeenko
- School of Veterinary Medicine, University of Surrey, Surrey, UK
| | | |
Collapse
|
16
|
Polevikov S. Advancing AI in healthcare: A comprehensive review of best practices. Clin Chim Acta 2023; 548:117519. [PMID: 37595864 DOI: 10.1016/j.cca.2023.117519] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 08/14/2023] [Accepted: 08/15/2023] [Indexed: 08/20/2023]
Abstract
Artificial Intelligence (AI) and Machine Learning (ML) are powerful tools shaping the healthcare sector. This review considers twelve key aspects of AI in clinical practice: 1) Ethical AI; 2) Explainable AI; 3) Health Equity and Bias in AI; 4) Sponsorship Bias; 5) Data Privacy; 6) Genomics and Privacy; 7) Insufficient Sample Size and Self-Serving Bias; 8) Bridging the Gap Between Training Datasets and Real-World Scenarios; 9) Open Source and Collaborative Development; 10) Dataset Bias and Synthetic Data; 11) Measurement Bias; 12) Reproducibility in AI Research. These categories represent both the challenges and opportunities of AI implementation in healthcare. While AI holds significant potential for improving patient care, it also presents risks and challenges, such as ensuring privacy, combating bias, and maintaining transparency and ethics. The review underscores the necessity of developing comprehensive best practices for healthcare organizations and fostering a diverse dialogue involving data scientists, clinicians, patient advocates, ethicists, economists, and policymakers. We are at the precipice of significant transformation in healthcare powered by AI. By continuing to reassess and refine our approach, we can ensure that AI is implemented responsibly and ethically, maximizing its benefit to patient care and public health.
Collapse
|
17
|
Kopka M, Scatturin L, Napierala H, Fürstenau D, Feufel MA, Balzer F, Schmieding ML. Characteristics of Users and Nonusers of Symptom Checkers in Germany: Cross-Sectional Survey Study. J Med Internet Res 2023; 25:e46231. [PMID: 37338970 DOI: 10.2196/46231] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 04/12/2023] [Accepted: 05/03/2023] [Indexed: 06/21/2023] Open
Abstract
BACKGROUND Previous studies have revealed that users of symptom checkers (SCs, apps that support self-diagnosis and self-triage) are predominantly female, are younger than average, and have higher levels of formal education. Little data are available for Germany, and no study has so far compared usage patterns with people's awareness of SCs and the perception of usefulness. OBJECTIVE We explored the sociodemographic and individual characteristics that are associated with the awareness, usage, and perceived usefulness of SCs in the German population. METHODS We conducted a cross-sectional online survey among 1084 German residents in July 2022 regarding personal characteristics and people's awareness and usage of SCs. Using random sampling from a commercial panel, we collected participant responses stratified by gender, state of residence, income, and age to reflect the German population. We analyzed the collected data exploratively. RESULTS Of all respondents, 16.3% (177/1084) were aware of SCs and 6.5% (71/1084) had used them before. Those aware of SCs were younger (mean 38.8, SD 14.6 years, vs mean 48.3, SD 15.7 years), were more often female (107/177, 60.5%, vs 453/907, 49.9%), and had higher formal education levels (eg, 72/177, 40.7%, vs 238/907, 26.2%, with a university/college degree) than those unaware. The same observation applied to users compared to nonusers. It disappeared, however, when comparing users to nonusers who were aware of SCs. Among users, 40.8% (29/71) considered these tools useful. Those considering them useful reported higher self-efficacy (mean 4.21, SD 0.66, vs mean 3.63, SD 0.81, on a scale of 1-5) and a higher net household income (mean EUR 2591.63, SD EUR 1103.96 [mean US $2798.96, SD US $1192.28], vs mean EUR 1626.60, SD EUR 649.05 [mean US $1756.73, SD US $700.97]) than those who considered them not useful. More women considered SCs unhelpful (13/44, 29.5%) compared to men (4/26, 15.4%). CONCLUSIONS Concurring with studies from other countries, our findings show associations between sociodemographic characteristics and SC usage in a German sample: users were on average younger, of higher socioeconomic status, and more commonly female compared to nonusers. However, usage cannot be explained by sociodemographic differences alone. It rather seems that sociodemographics explain who is or is not aware of the technology, but those who are aware of SCs are equally likely to use them, independently of sociodemographic differences. Although in some groups (eg, people with anxiety disorder), more participants reported to know and use SCs, they tended to perceive them as less useful. In other groups (eg, male participants), fewer respondents were aware of SCs, but those who used them perceived them to be more useful. Thus, SCs should be designed to fit specific user needs, and strategies should be developed to help reach individuals who could benefit but are not aware of SCs yet.
Collapse
Affiliation(s)
- Marvin Kopka
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Division of Ergonomics, Department of Psychology and Ergonomics, Technische Universität Berlin, Berlin, Germany
| | - Lennart Scatturin
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Hendrik Napierala
- Institute of General Practice and Family Medicine, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Daniel Fürstenau
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Department of Business IT, IT University of Copenhagen, København, Denmark
| | - Markus A Feufel
- Division of Ergonomics, Department of Psychology and Ergonomics, Technische Universität Berlin, Berlin, Germany
| | - Felix Balzer
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Malte L Schmieding
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
18
|
Patel SB, Lam K. ChatGPT: the future of discharge summaries? Lancet Digit Health 2023; 5:e107-e108. [PMID: 36754724 DOI: 10.1016/s2589-7500(23)00021-3] [Citation(s) in RCA: 206] [Impact Index Per Article: 206.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Accepted: 01/24/2023] [Indexed: 02/08/2023]
Affiliation(s)
- Sajan B Patel
- Department of Surgery and Cancer, St Mary's Hospital, London W2 1NY, UK
| | - Kyle Lam
- Department of Surgery and Cancer, St Mary's Hospital, London W2 1NY, UK.
| |
Collapse
|
19
|
Teleconsultation in respiratory medicine - A position paper of the Portuguese Pulmonology Society. Pulmonology 2023; 29:65-76. [PMID: 35705437 PMCID: PMC9188666 DOI: 10.1016/j.pulmoe.2022.04.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 04/06/2022] [Accepted: 04/06/2022] [Indexed: 01/06/2023] Open
Abstract
The COVID-19 pandemic crisis, among so many social, economic and health problems, also brought new opportunities. The potential of telemedicine to improve health outcomes had already been recognised in the last decades, but the pandemic crisis has accelerated the digital revolution. In 2020, a rapid increase in the use of remote consultations occurred due to the need to reduce attendance and overcrowding in outpatient clinics. However, the benefit of their use extends beyond the pandemic crisis, as an important tool to improve both the efficiency and capacity of future healthcare systems. This article reviews the literature regarding telemedicine and teleconsultation standards and recommendations, collects opinions of Portuguese experts in respiratory medicine and provides guidance in teleconsultation practices for Pulmonologists.
Collapse
|
20
|
Au Yeung J, Kraljevic Z, Luintel A, Balston A, Idowu E, Dobson RJ, Teo JT. AI chatbots not yet ready for clinical use. Front Digit Health 2023; 5:1161098. [PMID: 37122812 PMCID: PMC10130576 DOI: 10.3389/fdgth.2023.1161098] [Citation(s) in RCA: 32] [Impact Index Per Article: 32.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 03/27/2023] [Indexed: 05/02/2023] Open
Abstract
As large language models (LLMs) expand and become more advanced, so do the natural language processing capabilities of conversational AI, or "chatbots". OpenAI's recent release, ChatGPT, uses a transformer-based model to enable human-like text generation and question-answering on general domain knowledge, while a healthcare-specific Large Language Model (LLM) such as GatorTron has focused on the real-world healthcare domain knowledge. As LLMs advance to achieve near human-level performances on medical question and answering benchmarks, it is probable that Conversational AI will soon be developed for use in healthcare. In this article we discuss the potential and compare the performance of two different approaches to generative pretrained transformers-ChatGPT, the most widely used general conversational LLM, and Foresight, a GPT (generative pretrained transformer) based model focused on modelling patients and disorders. The comparison is conducted on the task of forecasting relevant diagnoses based on clinical vignettes. We also discuss important considerations and limitations of transformer-based chatbots for clinical use.
Collapse
Affiliation(s)
- Joshua Au Yeung
- Department of Neuroscience, Kings College Hospital, London, United Kingdom
- Guys & St Thomas Hospital, London, United Kingdom
- Correspondence: Joshua Au Yeung
| | - Zeljko Kraljevic
- Department of Biostatistics, Kings College London, London, United Kingdom
| | - Akish Luintel
- Department of Neuroscience, Kings College Hospital, London, United Kingdom
| | | | - Esther Idowu
- Guys & St Thomas Hospital, London, United Kingdom
| | - Richard J. Dobson
- Department of Biostatistics, Kings College London, London, United Kingdom
- NIHR Biomedical Research Centre, South London and Maudsley NHS Foundation Trust and King's College London, London, United Kingdom
| | - James T. Teo
- Department of Neuroscience, Kings College Hospital, London, United Kingdom
- Guys & St Thomas Hospital, London, United Kingdom
| |
Collapse
|
21
|
Müller R, Klemmt M, Ehni HJ, Henking T, Kuhnmünch A, Preiser C, Koch R, Ranisch R. Ethical, legal, and social aspects of symptom checker applications: a scoping review. MEDICINE, HEALTH CARE, AND PHILOSOPHY 2022; 25:737-755. [PMID: 36181620 PMCID: PMC9613552 DOI: 10.1007/s11019-022-10114-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 09/03/2022] [Indexed: 06/16/2023]
Abstract
Symptom Checker Applications (SCA) are mobile applications often designed for the end-user to assist with symptom assessment and self-triage. SCA are meant to provide the user with easily accessible information about their own health conditions. However, SCA raise questions regarding ethical, legal, and social aspects (ELSA), for example, regarding fair access to this new technology. The aim of this scoping review is to identify the ELSA of SCA in the scientific literature. A scoping review was conducted to identify the ELSA of SCA. Ten databases (e.g., Web of Science and PubMed) were used. Studies on SCA that address ELSA, written in English or German, were included in the review. The ELSA of SCA were extracted and synthesized using qualitative content analysis. A total of 25,061 references were identified, of which 39 were included in the analysis. The identified aspects were allotted to three main categories: (1) Technology; (2) Individual Level; and (3) Healthcare system. The results show that there are controversial debates in the literature on the ethical and social challenges of SCA usage. Furthermore, the debates are characterised by a lack of a specific legal perspective and empirical data. The review provides an overview on the spectrum of ELSA regarding SCA. It offers guidance to stakeholders in the healthcare system, for example, patients, healthcare professionals, and insurance providers and could be used in future empirical research to investigate the perspectives of those affected, such as users.
Collapse
Affiliation(s)
- Regina Müller
- Institute of Ethics and History of Medicine, University of Tübingen, Gartenstraße 47, 72074 Tübingen, Germany
| | - Malte Klemmt
- Institute of Applied Social Sciences, University of Applied Sciences Würzburg-Schweinfurt, Münzstraße 12, 97070 Würzburg, Germany
| | - Hans-Jörg Ehni
- Institute of Ethics and History of Medicine, University of Tübingen, Gartenstraße 47, 72074 Tübingen, Germany
| | - Tanja Henking
- Institute of Applied Social Sciences, University of Applied Sciences Würzburg-Schweinfurt, Münzstraße 12, 97070 Würzburg, Germany
| | - Angelina Kuhnmünch
- Institute of Ethics and History of Medicine, University of Tübingen, Gartenstraße 47, 72074 Tübingen, Germany
| | - Christine Preiser
- Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Wilhelmstraße 27, 72074 Tübingen, Germany
| | - Roland Koch
- Institute for General Practice and Interprofessional Care, University Medicine Tübingen, Osianderstraße 5, 72076 Tübingen, Germany
| | - Robert Ranisch
- Faculty of Health Sciences Brandenburg, University of Potsdam, Karl-Liebknecht-Str. 24-25, House 16, 14476 Potsdam, Golm, Germany
| |
Collapse
|
22
|
Liu VDM, Kaila M, Koskela T. User initiated symptom assessment with an electronic symptom checker. Study protocol for mixed-methods validation. (Preprint). JMIR Res Protoc 2022. [PMID: 37467041 PMCID: PMC10398552 DOI: 10.2196/41423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/30/2023] Open
Abstract
BACKGROUND The national Omaolo digital social welfare and health care service of Finland provides a symptom checker, Omaolo, which is a medical device (based on Duodecim Clinical Decision Support EBMEDS software) with a CE marking (risk class IIa), manufactured by the government-owned DigiFinland Oy. Users of this service can perform their triage by using the questions in the symptom checker. By completing the symptom checker, the user receives a recommendation for action and a service assessment with appropriate guidance regarding their health problems on the basis of a selected specific symptom in the symptom checker. This allows users to be provided with appropriate health care services, regardless of time and place. OBJECTIVE This study describes the protocol for the mixed methods validation process of the symptom checker available in Omaolo digital services. METHODS This is a mixed methods study using quantitative and qualitative methods, which will be part of the clinical validation process that takes place in primary health care centers in Finland. Each organization provides a space where the study and the nurse triage can be done in order to include an unscreened target population of users. The primary health care units provide walk-in model services, where no prior phone call or contact is required. For the validation of the Omaolo symptom checker, case vignettes will be incorporated to supplement the triage accuracy of rare and acute cases that cannot be tested extensively in real-life settings. Vignettes are produced from a variety of clinical sources, and they test the symptom checker in different triage levels by using 1 standardized patient case example. RESULTS This study plan underwent an ethics review by the regional permission, which was requested from each organization participating in the research, and an ethics committee statement was requested and granted from Pirkanmaa hospital district's ethics committee, which is in accordance with the University of Tampere's regulations. Of 964 clinical user-filled symptom checker assessments, 877 cases were fully completed with a triage result, and therefore, they met the requirements for clinical validation studies. The goal for sufficient data has been reached for most of the chief symptoms. Data collection was completed in September 2019, and the first feasibility and patient experience results were published by the end of 2020. Case vignettes have been identified and are to be completed before further testing the symptom checker. The analysis and reporting are estimated to be finalized in 2024. CONCLUSIONS The primary goals of this multimethod electronic symptom checker study are to assess safety and to provide crucial information regarding the accuracy and usability of the Omaolo electronic symptom checker. To our knowledge, this will be the first study to include real-life clinical cases along with case vignettes. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/41423.
Collapse
|
23
|
Jones AM, Jones DR. A Novel Bayesian General Medical Diagnostic Assistant Achieves Superior Accuracy With Sparse History. Front Artif Intell 2022; 5:727486. [PMID: 35937138 PMCID: PMC9355422 DOI: 10.3389/frai.2022.727486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 06/21/2022] [Indexed: 11/25/2022] Open
Abstract
Online AI symptom checkers and diagnostic assistants (DAs) have tremendous potential to reduce misdiagnosis and cost, while increasing the quality, convenience, and availability of healthcare, but only if they can perform with high accuracy. We introduce a novel Bayesian DA designed to improve diagnostic accuracy by addressing key weaknesses of Bayesian Network implementations for clinical diagnosis. We compare the performance of our prototype DA (MidasMed) to that of physicians and six other publicly accessible DAs (Ada, Babylon, Buoy, Isabel, Symptomate, and WebMD) using a set of 30 publicly available case vignettes, and using only sparse history (no exam findings or tests). Our results demonstrate superior performance of the MidasMed DA, with the correct diagnosis being the top ranked disorder in 93% of cases, and in the top 3 in 96% of cases.
Collapse
|
24
|
van Bussel MJP, Odekerken-Schröder GJ, Ou C, Swart RR, Jacobs MJG. Analyzing the determinants to accept a virtual assistant and use cases among cancer patients: a mixed methods study. BMC Health Serv Res 2022; 22:890. [PMID: 35804356 PMCID: PMC9270807 DOI: 10.1186/s12913-022-08189-7] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 06/13/2022] [Indexed: 11/17/2022] Open
Abstract
Background Technological progress in artificial intelligence has led to the increasing popularity of virtual assistants, i.e., embodied or disembodied conversational agents that allow chatting with a technical system in a natural language. However, only little comprehensive research is conducted about patients' perceptions and possible applications of virtual assistant in healthcare with cancer patients. This research aims to investigate the key acceptance factors and value-adding use cases of a virtual assistant for patients diagnosed with cancer. Methods Qualitative interviews with eight former patients and four doctors of a Dutch radiotherapy institute were conducted to determine what acceptance factors they find most important for a virtual assistant and gain insights into value-adding applications. The unified theory of acceptance and use of technology (UTAUT) was used to structure perceptions and was inductively modified as a result of the interviews. The subsequent research model was triangulated via an online survey with 127 respondents diagnosed with cancer. A structural equation model was used to determine the relevance of acceptance factors. Through a multigroup analysis, differences between sample subgroups were compared. Results The interviews found support for all factors of the UTAUT: performance expectancy, effort expectancy, social influence and facilitating conditions. Additionally, self-efficacy, trust, and resistance to change, were added as an extension of the UTAUT. Former patients found a virtual assistant helpful in receiving information about logistic questions, treatment procedures, side effects, or scheduling appointments. The quantitative study found that the constructs performance expectancy (ß = 0.399), effort expectancy (ß = 0.258), social influence (ß = 0.114), and trust (ß = 0.210) significantly influenced behavioral intention to use a virtual assistant, explaining 80% of its variance. Self-efficacy (ß = 0.792) acts as antecedent of effort expectancy. Facilitating conditions and resistance to change were not found to have a significant relationship with user intention. Conclusions Performance and effort expectancy are the leading determinants of virtual assistant acceptance. The latter is dependent on a patient’s self-efficacy. Therefore, including patients during the development and introduction of a VA in cancer treatment is important. The high relevance of trust indicates the need for a reliable, secure service that should be promoted as such. Social influence suggests using doctors in endorsing the VA. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-022-08189-7.
Collapse
Affiliation(s)
- Martien J P van Bussel
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands.
| | - Gaby J Odekerken-Schröder
- Department of Marketing and Supply Chain Management, Maastricht University, Maastricht, The Netherlands
| | - Carol Ou
- Tilburg School of Economics and Management, Department of Management, Tilburg University, Tilburg, The Netherlands
| | - Rachelle R Swart
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Maria J G Jacobs
- Tilburg School of Economics and Management, Department of Management, Tilburg University, Tilburg, The Netherlands
| |
Collapse
|
25
|
Cotte F, Mueller T, Gilbert S, Blümke B, Multmeier J, Hirsch MC, Wicks P, Wolanski J, Tutschkow D, Schade Brittinger C, Timmermann L, Jerrentrup A. Safety of Triage Self-assessment Using a Symptom Assessment App for Walk-in Patients in the Emergency Care Setting: Observational Prospective Cross-sectional Study. JMIR Mhealth Uhealth 2022; 10:e32340. [PMID: 35343909 PMCID: PMC9002590 DOI: 10.2196/32340] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Revised: 12/17/2021] [Accepted: 02/18/2022] [Indexed: 01/29/2023] Open
Abstract
Background Increasing use of emergency departments (EDs) by patients with low urgency, combined with limited availability of medical staff, results in extended waiting times and delayed care. Technological approaches could possibly increase efficiency by providing urgency advice and symptom assessments. Objective The purpose of this study is to evaluate the safety of urgency advice provided by a symptom assessment app, Ada, in an ED. Methods The study was conducted at the interdisciplinary ED of Marburg University Hospital, with data collection performed between August 2019 and March 2020. This study had a single-center cross-sectional prospective observational design and included 378 patients. The app’s urgency recommendation was compared with an established triage concept (Manchester Triage System [MTS]), including patients from the lower 3 MTS categories only. For all patients who were undertriaged, an expert physician panel assessed the case to detect potential avoidable hazardous situations (AHSs). Results Of 378 participants, 344 (91%) were triaged the same or more conservatively and 34 (8.9%) were undertriaged by the app. Of the 378 patients, 14 (3.7%) had received safe advice determined by the expert panel and 20 (5.3%) were considered to be potential AHS. Therefore, the assessment could be considered safe in 94.7% (358/378) of the patients when compared with the MTS assessment. From the 3 lowest MTS categories, 43.4% (164/378) of patients were not considered as emergency cases by the app, but could have been safely treated by a general practitioner or would not have required a physician consultation at all. Conclusions The app provided urgency advice after patient self-triage that has a high rate of safety, a rate of undertriage, and a rate of triage with potential to be an AHS, equivalent to telephone triage by health care professionals while still being more conservative than direct ED triage. A large proportion of patients in the ED were not considered as emergency cases, which could possibly relieve ED burden if used at home. Further research should be conducted in the at-home setting to evaluate this hypothesis. Trial Registration German Clinical Trial Registration DRKS00024909; https://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00024909
Collapse
Affiliation(s)
- Fabienne Cotte
- Charité Universitäsmedizin Berlin, Berlin, Germany.,Department of Emergency Medicine, University Clinic Marburg, Philipps-University, Marburg, Germany.,Ada Health GmbH, Berlin, Germany
| | - Tobias Mueller
- Center for Unknown and Rare Diseases, UKGM GmbH, University Clinic Marburg, Philipps-University, Marburg, Germany
| | - Stephen Gilbert
- Ada Health GmbH, Berlin, Germany.,Else Kröner Fresenius Center for Digital Health, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | | | | | - Martin Christian Hirsch
- Ada Health GmbH, Berlin, Germany.,Institute of Artificial Intelligence, Philipps-University Marburg, Marburg, Germany
| | | | | | - Darja Tutschkow
- Coordinating Center for Clinical Trials, Philipps University Marburg, Marburg, Germany, Marburg, Germany
| | - Carmen Schade Brittinger
- Coordinating Center for Clinical Trials, Philipps University Marburg, Marburg, Germany, Marburg, Germany
| | - Lars Timmermann
- Department of Neurology, University Hospital of Marburg, Marburg, Germany
| | - Andreas Jerrentrup
- Department of Emergency Medicine, University Clinic Marburg, Philipps-University, Marburg, Germany
| |
Collapse
|
26
|
The Role of Artificial Intelligence in Early Cancer Diagnosis. Cancers (Basel) 2022; 14:cancers14061524. [PMID: 35326674 PMCID: PMC8946688 DOI: 10.3390/cancers14061524] [Citation(s) in RCA: 46] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 03/08/2022] [Accepted: 03/10/2022] [Indexed: 02/01/2023] Open
Abstract
Improving the proportion of patients diagnosed with early-stage cancer is a key priority of the World Health Organisation. In many tumour groups, screening programmes have led to improvements in survival, but patient selection and risk stratification are key challenges. In addition, there are concerns about limited diagnostic workforces, particularly in light of the COVID-19 pandemic, placing a strain on pathology and radiology services. In this review, we discuss how artificial intelligence algorithms could assist clinicians in (1) screening asymptomatic patients at risk of cancer, (2) investigating and triaging symptomatic patients, and (3) more effectively diagnosing cancer recurrence. We provide an overview of the main artificial intelligence approaches, including historical models such as logistic regression, as well as deep learning and neural networks, and highlight their early diagnosis applications. Many data types are suitable for computational analysis, including electronic healthcare records, diagnostic images, pathology slides and peripheral blood, and we provide examples of how these data can be utilised to diagnose cancer. We also discuss the potential clinical implications for artificial intelligence algorithms, including an overview of models currently used in clinical practice. Finally, we discuss the potential limitations and pitfalls, including ethical concerns, resource demands, data security and reporting standards.
Collapse
|
27
|
White E, Koulaouzidis A, Patience L, Wenzek H. How a managed service for colon capsule endoscopy works in an overstretched healthcare system. Scand J Gastroenterol 2022; 57:359-363. [PMID: 34854333 DOI: 10.1080/00365521.2021.2006299] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Lower gastrointestinal diagnostics have been facing significant capacity constraints, which the COVID-19 pandemic has exacerbated due to significant reductions in endoscopy procedures. Colon Capsule Endoscopy (CCE) provides a safe, viable solution to offset ongoing demand and could be a valuable tool for the recovery of endoscopy services post-COVID. NHS Scotland has already begun a country-wide rollout of CCE as a managed service, and NHS England have committed to a pilot scheme of 11,000 capsules via hospital-based delivery. Here, we outline a proven method of CCE delivery that ensures the CCE and results are delivered in an efficient, clinically robust manner with high patient acceptability levels through a managed service. Delivering CCE without a managed service is likely to be slower, more costly, and less effective, limiting the many benefits of CCE as an addition to the standard diagnostic pathway for bowel cancer.
Collapse
Affiliation(s)
| | - Anastasios Koulaouzidis
- Department of Social Medicine & Public Health, Pomeranian Medical University, Szczecin, Poland
| | | | - Hagen Wenzek
- CorporateHealth International ApS, Odense, Denmark
| |
Collapse
|
28
|
Hennemann S, Kuhn S, Witthöft M, Jungmann SM. Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients. JMIR Ment Health 2022; 9:e32832. [PMID: 35099395 PMCID: PMC8844983 DOI: 10.2196/32832] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Accepted: 11/09/2021] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Digital technologies have become a common starting point for health-related information-seeking. Web- or app-based symptom checkers aim to provide rapid and accurate condition suggestions and triage advice but have not yet been investigated for mental disorders in routine health care settings. OBJECTIVE This study aims to test the diagnostic performance of a widely available symptom checker in the context of formal diagnosis of mental disorders when compared with therapists' diagnoses based on structured clinical interviews. METHODS Adult patients from an outpatient psychotherapy clinic used the app-based symptom checker Ada-check your health (ADA; Ada Health GmbH) at intake. Accuracy was assessed as the agreement of the first and 1 of the first 5 condition suggestions of ADA with at least one of the interview-based therapist diagnoses. In addition, sensitivity, specificity, and interrater reliabilities (Gwet first-order agreement coefficient [AC1]) were calculated for the 3 most prevalent disorder categories. Self-reported usability (assessed using the System Usability Scale) and acceptance of ADA (assessed using an adapted feedback questionnaire) were evaluated. RESULTS A total of 49 patients (30/49, 61% women; mean age 33.41, SD 12.79 years) were included in this study. Across all patients, the interview-based diagnoses matched ADA's first condition suggestion in 51% (25/49; 95% CI 37.5-64.4) of cases and 1 of the first 5 condition suggestions in 69% (34/49; 95% CI 55.4-80.6) of cases. Within the main disorder categories, the accuracy of ADA's first condition suggestion was 0.82 for somatoform and associated disorders, 0.65 for affective disorders, and 0.53 for anxiety disorders. Interrater reliabilities ranged from low (AC1=0.15 for anxiety disorders) to good (AC1=0.76 for somatoform and associated disorders). The usability of ADA was rated as high in the System Usability Scale (mean 81.51, SD 11.82, score range 0-100). Approximately 71% (35/49) of participants would have preferred a face-to-face over an app-based diagnostic. CONCLUSIONS Overall, our findings suggest that a widely available symptom checker used in the formal diagnosis of mental disorders could provide clinicians with a list of condition suggestions with moderate-to-good accuracy. However, diagnostic performance was heterogeneous between disorder categories and included low interrater reliability. Although symptom checkers have some potential to complement the diagnostic process as a screening tool, the diagnostic performance should be tested in larger samples and in comparison with further diagnostic instruments.
Collapse
Affiliation(s)
- Severin Hennemann
- Department of Clinical Psychology, Psychotherapy and Experimental Psychopathology, University of Mainz, Mainz, Germany
| | - Sebastian Kuhn
- Department of Digital Medicine, Medical Faculty OWL, Bielefeld University, Bielefeld, Germany
| | - Michael Witthöft
- Department of Clinical Psychology, Psychotherapy and Experimental Psychopathology, University of Mainz, Mainz, Germany
| | - Stefanie M Jungmann
- Department of Clinical Psychology, Psychotherapy and Experimental Psychopathology, University of Mainz, Mainz, Germany
| |
Collapse
|
29
|
Deep contextual multi-task feature fusion for enhanced concept, negation and speculation detection from clinical notes. INFORMATICS IN MEDICINE UNLOCKED 2022. [DOI: 10.1016/j.imu.2022.101109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
|
30
|
Boucher EM, Harake NR, Ward HE, Stoeckl SE, Vargas J, Minkel J, Parks AC, Zilca R. Artificially intelligent chatbots in digital mental health interventions: a review. Expert Rev Med Devices 2021; 18:37-49. [PMID: 34872429 DOI: 10.1080/17434440.2021.2013200] [Citation(s) in RCA: 46] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
INTRODUCTION Increasing demand for mental health services and the expanding capabilities of artificial intelligence (AI) in recent years has driven the development of digital mental health interventions (DMHIs). To date, AI-based chatbots have been integrated into DMHIs to support diagnostics and screening, symptom management and behavior change, and content delivery. AREAS COVERED We summarize the current landscape of DMHIs, with a focus on AI-based chatbots. Happify Health's AI chatbot, Anna, serves as a case study for discussion of potential challenges and how these might be addressed, and demonstrates the promise of chatbots as effective, usable, and adoptable within DMHIs. Finally, we discuss ways in which future research can advance the field, addressing topics including perceptions of AI, the impact of individual differences, and implications for privacy and ethics. EXPERT OPINION Our discussion concludes with a speculative viewpoint on the future of AI in DMHIs, including the use of chatbots, the evolution of AI, dynamic mental health systems, hyper-personalization, and human-like intervention delivery.
Collapse
|
31
|
Miles O, West R, Nadarzynski T. Health chatbots acceptability moderated by perceived stigma and severity: A cross-sectional survey. Digit Health 2021; 7:20552076211063012. [PMID: 34917391 PMCID: PMC8670785 DOI: 10.1177/20552076211063012] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Revised: 10/21/2021] [Accepted: 11/10/2021] [Indexed: 11/17/2022] Open
Abstract
Background Chatbots and virtual voice assistants are increasingly common in primary care
without sufficient evidence for their feasibility and effectiveness. We
aimed to assess how perceived stigma and severity of various health issues
are associated with the acceptability for three sources of health
information and consultation: an automated chatbot, a General Practitioner
(GP), or a combination of both. Methods Between May and June 2019, we conducted an online study, advertised via
Facebook, for UK citizens. It was a factorial simulation experiment with
three within-subject factors (perceived health issue stigma, severity, and
consultation source) and six between-subject covariates. Acceptability
rating for each consultation source was the dependant variable. A single
mixed-model ANOVA was performed. Results Amongst 237 participants (65% aged over 45 years old, 73% women), GP
consultations were seen as most acceptable, followed by GP-chatbot service.
Chatbots were seen least acceptable as a consultation source for severe
health issues, while the acceptability was significantly higher for
stigmatised health issues. No associations between participants’
characteristics and acceptability were found. Conclusions Although healthcare professionals are perceived as the most desired sources
of health information, chatbots may be useful for sensitive health issues in
which disclosure of personal information is challenging. However, chatbots
are less acceptable for health issues of higher severity and should not be
recommended for use within that context. Policymakers and digital service
designers need to recognise the limitations of health chatbots. Future
research should establish a set of health topics most suitable for
chatbot-led interventions and primary healthcare services.
Collapse
Affiliation(s)
- Oliver Miles
- Health Behaviour Research Centre, University College London Epidemiology and Public Health, London, UK
| | - Robert West
- Health Behaviour Research Centre, University College London Epidemiology and Public Health, London, UK
| | - Tom Nadarzynski
- School of Social Sciences, The University of Westminster, Westminster, UK
| |
Collapse
|
32
|
Hah H, Goldin DS. How Clinicians Perceive Artificial Intelligence-Assisted Technologies in Diagnostic Decision Making: Mixed Methods Approach. J Med Internet Res 2021; 23:e33540. [PMID: 34924356 PMCID: PMC8726017 DOI: 10.2196/33540] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2021] [Revised: 10/26/2021] [Accepted: 11/16/2021] [Indexed: 12/19/2022] Open
Abstract
Background With the rapid development of artificial intelligence (AI) and related technologies, AI algorithms are being embedded into various health information technologies that assist clinicians in clinical decision making. Objective This study aimed to explore how clinicians perceive AI assistance in diagnostic decision making and suggest the paths forward for AI-human teaming for clinical decision making in health care. Methods This study used a mixed methods approach, utilizing hierarchical linear modeling and sentiment analysis through natural language understanding techniques. Results A total of 114 clinicians participated in online simulation surveys in 2020 and 2021. These clinicians studied family medicine and used AI algorithms to aid in patient diagnosis. Their overall sentiment toward AI-assisted diagnosis was positive and comparable with diagnoses made without the assistance of AI. However, AI-guided decision making was not congruent with the way clinicians typically made decisions in diagnosing illnesses. In a quantitative survey, clinicians reported perceiving current AI assistance as not likely to enhance diagnostic capability and negatively influenced their overall performance (β=–0.421, P=.02). Instead, clinicians’ diagnostic capabilities tended to be associated with well-known parameters, such as education, age, and daily habit of technology use on social media platforms. Conclusions This study elucidated clinicians’ current perceptions and sentiments toward AI-enabled diagnosis. Although the sentiment was positive, the current form of AI assistance may not be linked with efficient decision making, as AI algorithms are not well aligned with subjective human reasoning in clinical diagnosis. Developers and policy makers in health could gather behavioral data from clinicians in various disciplines to help align AI algorithms with the unique subjective patterns of reasoning that humans employ in clinical diagnosis.
Collapse
Affiliation(s)
- Hyeyoung Hah
- Information Systems and Business Analytics, College of Business, Florida International University, Miami, FL, United States
| | - Deana Shevit Goldin
- Nicole Wertheim College of Nursing & Health Sciences, Florida International University, Miami, FL, United States
| |
Collapse
|
33
|
Woodcock C, Mittelstadt B, Busbridge D, Blank G. The Impact of Explanations on Layperson Trust in Artificial Intelligence-Driven Symptom Checker Apps: Experimental Study. J Med Internet Res 2021; 23:e29386. [PMID: 34730544 PMCID: PMC8600426 DOI: 10.2196/29386] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 07/11/2021] [Accepted: 07/27/2021] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND Artificial intelligence (AI)-driven symptom checkers are available to millions of users globally and are advocated as a tool to deliver health care more efficiently. To achieve the promoted benefits of a symptom checker, laypeople must trust and subsequently follow its instructions. In AI, explanations are seen as a tool to communicate the rationale behind black-box decisions to encourage trust and adoption. However, the effectiveness of the types of explanations used in AI-driven symptom checkers has not yet been studied. Explanations can follow many forms, including why-explanations and how-explanations. Social theories suggest that why-explanations are better at communicating knowledge and cultivating trust among laypeople. OBJECTIVE The aim of this study is to ascertain whether explanations provided by a symptom checker affect explanatory trust among laypeople and whether this trust is impacted by their existing knowledge of disease. METHODS A cross-sectional survey of 750 healthy participants was conducted. The participants were shown a video of a chatbot simulation that resulted in the diagnosis of either a migraine or temporal arteritis, chosen for their differing levels of epidemiological prevalence. These diagnoses were accompanied by one of four types of explanations. Each explanation type was selected either because of its current use in symptom checkers or because it was informed by theories of contrastive explanation. Exploratory factor analysis of participants' responses followed by comparison-of-means tests were used to evaluate group differences in trust. RESULTS Depending on the treatment group, two or three variables were generated, reflecting the prior knowledge and subsequent mental model that the participants held. When varying explanation type by disease, migraine was found to be nonsignificant (P=.65) and temporal arteritis, marginally significant (P=.09). Varying disease by explanation type resulted in statistical significance for input influence (P=.001), social proof (P=.049), and no explanation (P=.006), with counterfactual explanation (P=.053). The results suggest that trust in explanations is significantly affected by the disease being explained. When laypeople have existing knowledge of a disease, explanations have little impact on trust. Where the need for information is greater, different explanation types engender significantly different levels of trust. These results indicate that to be successful, symptom checkers need to tailor explanations to each user's specific question and discount the diseases that they may also be aware of. CONCLUSIONS System builders developing explanations for symptom-checking apps should consider the recipient's knowledge of a disease and tailor explanations to each user's specific need. Effort should be placed on generating explanations that are personalized to each user of a symptom checker to fully discount the diseases that they may be aware of and to close their information gap.
Collapse
Affiliation(s)
- Claire Woodcock
- Oxford Internet Institute, University of Oxford, Oxford, United Kingdom
| | - Brent Mittelstadt
- Oxford Internet Institute, University of Oxford, Oxford, United Kingdom
| | | | - Grant Blank
- Oxford Internet Institute, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
34
|
Delshad S, Dontaraju VS, Chengat V. Artificial Intelligence-Based Application Provides Accurate Medical Triage Advice When Compared to Consensus Decisions of Healthcare Providers. Cureus 2021; 13:e16956. [PMID: 34405077 PMCID: PMC8352839 DOI: 10.7759/cureus.16956] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/06/2021] [Indexed: 12/23/2022] Open
Abstract
Accurate medical triage is essential for improving patient outcomes and efficient healthcare delivery. Patients increasingly rely on artificial intelligence (AI)-based applications to access healthcare information, including medical triage advice. We assessed the accuracy of triage decisions provided by an AI-based application. We presented 50 clinical vignettes to the AI-based application, seven emergency medicine providers, and five internal medicine physicians. We compared the triage decisions of the AI-based application to those of the individual providers as well as their consensus decisions. When compared to the human clinicians’ consensus triage decisions, the AI-based application performed equal or better than individual human clinicians.
Collapse
Affiliation(s)
- Sean Delshad
- Internal Medicine, David Geffen School of Medicine at University of California Los Angeles, Los Angeles, USA
| | | | | |
Collapse
|
35
|
Sætra HS, Fosch-Villaronga E. Healthcare Digitalisation and the Changing Nature of Work and Society. Healthcare (Basel) 2021; 9:1007. [PMID: 34442144 PMCID: PMC8394196 DOI: 10.3390/healthcare9081007] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2021] [Revised: 07/15/2021] [Accepted: 08/05/2021] [Indexed: 11/21/2022] Open
Abstract
Digital technologies have profound effects on all areas of modern life, including the workplace. Certain forms of digitalisation entail simply exchanging digital files for paper, while more complex instances involve machines performing a wide variety of tasks on behalf of humans. While some are wary of the displacement of humans that occurs when, for example, robots perform tasks previously performed by humans, others argue that robots only perform the tasks that robots should have carried out in the very first place and never by humans. Understanding the impacts of digitalisation in the workplace requires an understanding of the effects of digital technology on the tasks we perform, and these effects are often not foreseeable. In this article, the changing nature of work in the health care sector is used as a case to analyse such change and its implications on three levels: the societal (macro), organisational (meso), and individual level (micro). Analysing these transformations by using a layered approach is helpful for understanding the actual magnitude of the changes that are occurring and creates the foundation for an informed regulatory and societal response. We argue that, while artificial intelligence, big data, and robotics are revolutionary technologies, most of the changes we see involve technological substitution and not infrastructural change. Even though this undermines the assumption that these new technologies constitute a fourth industrial revolution, their effects on the micro and meso level still require both political awareness and proportional regulatory responses.
Collapse
Affiliation(s)
- Henrik Skaug Sætra
- Faculty of Computer Sciences, Engineering and Economics, Østfold University College, N-1757 Halden, Norway
| | - Eduard Fosch-Villaronga
- eLaw Center for Law and Digital Technologies, School of Law, Leiden University, 2311 EZ Leiden, The Netherlands;
| |
Collapse
|
36
|
Conley CC, Otto AK, McDonnell GA, Tercyak KP. Multiple approaches to enhancing cancer communication in the next decade: translating research into practice and policy. Transl Behav Med 2021; 11:2018-2032. [PMID: 34347872 DOI: 10.1093/tbm/ibab089] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022] Open
Abstract
Communicating risk and other health information in a clear, understandable, and actionable manner is critical for the prevention and control of cancer, as well as the care of affected individuals and their family members. However, the swift pace of development in communication technologies has dramatically changed the health communication landscape. This digital era presents new opportunities and challenges for cancer communication research and its impact on practice and policy. In this article, we examine the science of health communication focused on cancer and highlight important areas of research for the coming decade. Specifically, we discuss three domains in which cancer communication may occur: (a) among patients and their healthcare providers; (b) within and among families and social networks; and (c) across communities, populations, and the public more broadly. We underscore findings from the prior decade of cancer communication research, provide illustrative examples of future directions for cancer communication science, and conclude with considerations for diverse populations. Health informatics studies will be necessary to fully understand the growing and complex communication settings related to cancer: such works have the potential to change the face of information exchanges about cancer and elevate our collective discourse about this area as newer clinical and public health priorities emerge. Researchers from a wide array of specialties are interested in examining and improving cancer communication. These interdisciplinary perspectives can rapidly advance and help translate findings of cancer communication in the field of behavioral medicine.
Collapse
Affiliation(s)
- Claire C Conley
- Cancer Prevention and Control Program, Lombardi Comprehensive Cancer Center, Georgetown University Medical Center, Washington, DC, USA
| | - Amy K Otto
- Department of Public Health Sciences, University of Miami, Miller School of Medicine, Miami, FL, USA
| | - Glynnis A McDonnell
- Cancer Prevention and Control Program, Lombardi Comprehensive Cancer Center, Georgetown University Medical Center, Washington, DC, USA
| | - Kenneth P Tercyak
- Cancer Prevention and Control Program, Lombardi Comprehensive Cancer Center, Georgetown University Medical Center, Washington, DC, USA
| |
Collapse
|
37
|
Dingler T, Kwasnicka D, Wei J, Gong E, Oldenburg B. The Use and Promise of Conversational Agents in Digital Health. Yearb Med Inform 2021; 30:191-199. [PMID: 34479391 PMCID: PMC8416202 DOI: 10.1055/s-0041-1726510] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
OBJECTIVES To describe the use and promise of conversational agents in digital health-including health promotion andprevention-and how they can be combined with other new technologies to provide healthcare at home. METHOD A narrative review of recent advances in technologies underpinning conversational agents and their use and potential for healthcare and improving health outcomes. RESULTS By responding to written and spoken language, conversational agents present a versatile, natural user interface and have the potential to make their services and applications more widely accessible. Historically, conversational interfaces for health applications have focused mainly on mental health, but with an increase in affordable devices and the modernization of health services, conversational agents are becoming more widely deployed across the health system. We present our work on context-aware voice assistants capable of proactively engaging users and delivering health information and services. The proactive voice agents we deploy, allow us to conduct experience sampling in people's homes and to collect information about the contexts in which users are interacting with them. CONCLUSION In this article, we describe the state-of-the-art of these and other enabling technologies for speech and conversation and discuss ongoing research efforts to develop conversational agents that "live" with patients and customize their service offerings around their needs. These agents can function as 'digital companions' who will send reminders about medications and appointments, proactively check in to gather self-assessments, and follow up with patients on their treatment plans. Together with an unobtrusive and continuous collection of other health data, conversational agents can provide novel and deeply personalized access to digital health care, and they will continue to become an increasingly important part of the ecosystem for future healthcare delivery.
Collapse
Affiliation(s)
- Tilman Dingler
- NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, School of Computing and Information Systems, University of Melbourne, Parkville, Australia
| | - Dominika Kwasnicka
- NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Australia
| | - Jing Wei
- NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, School of Computing and Information Systems, University of Melbourne, Parkville, Australia
| | - Enying Gong
- NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Australia
| | - Brian Oldenburg
- NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Australia
| |
Collapse
|
38
|
Chen J, Lu C, Huang H, Zhu D, Yang Q, Liu J, Huang Y, Deng A, Han X. Cognitive Computing-Based CDSS in Medical Practice. HEALTH DATA SCIENCE 2021; 2021:9819851. [PMID: 38487503 PMCID: PMC10880153 DOI: 10.34133/2021/9819851] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Accepted: 06/28/2021] [Indexed: 03/17/2024]
Abstract
Importance. The last decade has witnessed the advances of cognitive computing technologies that learn at scale and reason with purpose in medicine studies. From the diagnosis of diseases till the generation of treatment plans, cognitive computing encompasses both data-driven and knowledge-driven machine intelligence to assist health care roles in clinical decision-making. This review provides a comprehensive perspective from both research and industrial efforts on cognitive computing-based CDSS over the last decade.Highlights. (1) A holistic review of both research papers and industrial practice about cognitive computing-based CDSS is conducted to identify the necessity and the characteristics as well as the general framework of constructing the system. (2) Several of the typical applications of cognitive computing-based CDSS as well as the existing systems in real medical practice are introduced in detail under the general framework. (3) The limitations of the current cognitive computing-based CDSS is discussed that sheds light on the future work in this direction.Conclusion. Different from medical content providers, cognitive computing-based CDSS provides probabilistic clinical decision support by automatically learning and inferencing from medical big data. The characteristics of managing multimodal data and computerizing medical knowledge distinguish cognitive computing-based CDSS from other categories. Given the current status of primary health care like high diagnostic error rate and shortage of medical resources, it is time to introduce cognitive computing-based CDSS to the medical community which is supposed to be more open-minded and embrace the convenience and low cost but high efficiency brought by cognitive computing-based CDSS.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Aijun Deng
- The Affiliated Hospital of Weifang Medical University, Shandong, China
| | - Xiaoxu Han
- National Clinical Research Center for Laboratory MedicineChina
- The First Affiliated Hospital, China Medical University, Liaoning, China
| |
Collapse
|
39
|
Ceney A, Tolond S, Glowinski A, Marks B, Swift S, Palser T. Accuracy of online symptom checkers and the potential impact on service utilisation. PLoS One 2021; 16:e0254088. [PMID: 34265845 PMCID: PMC8282353 DOI: 10.1371/journal.pone.0254088] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Accepted: 06/13/2021] [Indexed: 02/03/2023] Open
Abstract
Objectives The aims of our study are firstly to investigate the diagnostic and triage performance of symptom checkers, secondly to assess their potential impact on healthcare utilisation and thirdly to investigate for variation in performance between systems. Setting Publicly available symptom checkers for patient use. Participants Publicly available symptom-checkers were identified. A standardised set of 50 clinical vignettes were developed and systematically run through each system by a non-clinical researcher. Primary and secondary outcome measures System accuracy was assessed by measuring the percentage of times the correct diagnosis was a) listed first, b) within the top five diagnoses listed and c) listed at all. The safety of the disposition advice was assessed by comparing it with national guidelines for each vignette. Results Twelve tools were identified and included. Mean diagnostic accuracy of the systems was poor, with the correct diagnosis being present in the top five diagnoses on 51.0% (Range 22.2 to 84.0%). Safety of disposition advice decreased with condition urgency (being 71.8% for emergency cases vs 87.3% for non-urgent cases). 51.0% of systems suggested additional resource utilisation above that recommended by national guidelines (range 18.0% to 61.2%). Both diagnostic accuracy and appropriate resource recommendation varied substantially between systems. Conclusions There is wide variation in performance between available symptom checkers and overall performance is significantly below what would be accepted in any other medical field, though some do achieve a good level of accuracy and safety of disposition. External validation and regulation are urgently required to ensure these public facing tools are safe.
Collapse
Affiliation(s)
- Adam Ceney
- Methods Analytics Ltd, Sheffield, United Kingdom
- * E-mail:
| | | | | | - Ben Marks
- Methods Analytics Ltd, Sheffield, United Kingdom
| | - Simon Swift
- Methods Analytics Ltd, Sheffield, United Kingdom
- University of Exeter Business School (INDEX), Exeter, United Kingdom
| | - Tom Palser
- Methods Analytics Ltd, Sheffield, United Kingdom
- Department of Surgery, University Hospitals of Leicester NHS Trust, Leicester, United Kingdom
- SAPPHIRE, Department of Health Sciences, University of Leicester, Leicester, United Kingdom
| |
Collapse
|
40
|
Alam L, Mueller S. Examining the effect of explanation on satisfaction and trust in AI diagnostic systems. BMC Med Inform Decis Mak 2021; 21:178. [PMID: 34082719 PMCID: PMC8176739 DOI: 10.1186/s12911-021-01542-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Accepted: 05/21/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Artificial Intelligence has the potential to revolutionize healthcare, and it is increasingly being deployed to support and assist medical diagnosis. One potential application of AI is as the first point of contact for patients, replacing initial diagnoses prior to sending a patient to a specialist, allowing health care professionals to focus on more challenging and critical aspects of treatment. But for AI systems to succeed in this role, it will not be enough for them to merely provide accurate diagnoses and predictions. In addition, it will need to provide explanations (both to physicians and patients) about why the diagnoses are made. Without this, accurate and correct diagnoses and treatments might otherwise be ignored or rejected. METHOD It is important to evaluate the effectiveness of these explanations and understand the relative effectiveness of different kinds of explanations. In this paper, we examine this problem across two simulation experiments. For the first experiment, we tested a re-diagnosis scenario to understand the effect of local and global explanations. In a second simulation experiment, we implemented different forms of explanation in a similar diagnosis scenario. RESULTS Results show that explanation helps improve satisfaction measures during the critical re-diagnosis period but had little effect before re-diagnosis (when initial treatment was taking place) or after (when an alternate diagnosis resolved the case successfully). Furthermore, initial "global" explanations about the process had no impact on immediate satisfaction but improved later judgments of understanding about the AI. Results of the second experiment show that visual and example-based explanations integrated with rationales had a significantly better impact on patient satisfaction and trust than no explanations, or with text-based rationales alone. As in Experiment 1, these explanations had their effect primarily on immediate measures of satisfaction during the re-diagnosis crisis, with little advantage prior to re-diagnosis or once the diagnosis was successfully resolved. CONCLUSION These two studies help us to draw several conclusions about how patient-facing explanatory diagnostic systems may succeed or fail. Based on these studies and the review of the literature, we will provide some design recommendations for the explanations offered for AI systems in the healthcare domain.
Collapse
Affiliation(s)
- Lamia Alam
- Michigan Technological University, Houghton, MI, 49931, USA.
| | - Shane Mueller
- Michigan Technological University, Houghton, MI, 49931, USA
| |
Collapse
|
41
|
You Y, Gui X. Self-Diagnosis through AI-enabled Chatbot-based Symptom Checkers: User Experiences and Design Considerations. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2021; 2020:1354-1363. [PMID: 33936512 PMCID: PMC8075525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Recently, there has been a growing interest in developing AI-enabled chatbot-based symptom checker (CSC) apps in the healthcare market. CSC apps provide potential diagnoses for users and assist them with self-triaging based on Artificial Intelligence (AI) techniques using human-like conversations. Despite the popularity of such CSC apps, little research has been done to investigate their functionalities and user experiences. To do so, we conducted a feature review, a user review analysis, and an interview study. We found that the existing CSC apps lack the functions to support the whole diagnostic process of an offline medical visit. We also found that users perceive the current CSC apps to lack support for a comprehensive medical history, flexible symptom input, comprehensible questions, and diverse diseases and user groups. Based on these results, we derived implications for the future features and conversational design of CSC apps.
Collapse
Affiliation(s)
- Yue You
- Pennsylvania State University, University Park, PA, USA
| | - Xinning Gui
- Pennsylvania State University, University Park, PA, USA
| |
Collapse
|
42
|
Baker A, Perov Y, Middleton K, Baxter J, Mullarkey D, Sangar D, Butt M, DoRosario A, Johri S. A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis. Front Artif Intell 2020; 3:543405. [PMID: 33733203 PMCID: PMC7861270 DOI: 10.3389/frai.2020.543405] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2020] [Accepted: 11/04/2020] [Indexed: 11/18/2022] Open
Abstract
AI virtual assistants have significant potential to alleviate the pressure on overly burdened healthcare systems by enabling patients to self-assess their symptoms and to seek further care when appropriate. For these systems to make a meaningful contribution to healthcare globally, they must be trusted by patients and healthcare professionals alike, and service the needs of patients in diverse regions and segments of the population. We developed an AI virtual assistant which provides patients with triage and diagnostic information. Crucially, the system is based on a generative model, which allows for relatively straightforward re-parameterization to reflect local disease and risk factor burden in diverse regions and population segments. This is an appealing property, particularly when considering the potential of AI systems to improve the provision of healthcare on a global scale in many regions and for both developing and developed countries. We performed a prospective validation study of the accuracy and safety of the AI system and human doctors. Importantly, we assessed the accuracy and safety of both the AI and human doctors independently against identical clinical cases and, unlike previous studies, also accounted for the information gathering process of both agents. Overall, we found that the AI system is able to provide patients with triage and diagnostic information with a level of clinical accuracy and safety comparable to that of human doctors. Through this approach and study, we hope to start building trust in AI-powered systems by directly comparing their performance to human doctors, who do not always agree with each other on the cause of patients’ symptoms or the most appropriate triage recommendation.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Arnold DoRosario
- Northeast Medical Group, Yale New Haven Health, New Haven, CT, United States
| | | |
Collapse
|
43
|
Lin SY, Mahoney MR, Sinsky CA. Ten Ways Artificial Intelligence Will Transform Primary Care. J Gen Intern Med 2019; 34:1626-1630. [PMID: 31090027 PMCID: PMC6667610 DOI: 10.1007/s11606-019-05035-1] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Revised: 03/12/2019] [Accepted: 04/05/2019] [Indexed: 01/16/2023]
Abstract
Artificial intelligence (AI) is poised as a transformational force in healthcare. This paper presents a current environmental scan, through the eyes of primary care physicians, of the top ten ways AI will impact primary care and its key stakeholders. We discuss ten distinct problem spaces and the most promising AI innovations in each, estimating potential market sizes and the Quadruple Aims that are most likely to be affected. Primary care is where the power, opportunity, and future of AI are most likely to be realized in the broadest and most ambitious scale. We propose how these AI-powered innovations must augment, not subvert, the patient-physician relationship for physicians and patients to accept them. AI implemented poorly risks pushing humanity to the margins; done wisely, AI can free up physicians' cognitive and emotional space for patients, and shift the focus away from transactional tasks to personalized care. The challenge will be for humans to have the wisdom and willingness to discern AI's optimal role in twenty-first century healthcare, and to determine when it strengthens and when it undermines human healing. Ongoing research will determine the impact of AI technologies in achieving better care, better health, lower costs, and improved well-being of the workforce.
Collapse
Affiliation(s)
- Steven Y Lin
- Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA.
| | - Megan R Mahoney
- Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | | |
Collapse
|
44
|
Challenges for digital services in the NHS: drowning in a sea of innovation. Br J Gen Pract 2019; 69:326-327. [PMID: 31249071 DOI: 10.3399/bjgp19x704177] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/31/2022] Open
|