1
|
Buitrago-Esquinas EM, Puig-Cabrera M, Santos JAC, Custódio-Santos M, Yñiguez-Ovando R. Developing a hetero-intelligence methodological framework for sustainable policy-making based on the assessment of large language models. MethodsX 2024; 12:102707. [PMID: 38650999 PMCID: PMC11033193 DOI: 10.1016/j.mex.2024.102707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 04/09/2024] [Indexed: 04/25/2024] Open
Abstract
This work delves into the increasing relevance of Large Language Models (LLMs) in the realm of sustainable policy-making, proposing an innovative hetero-intelligence framework that blends human and artificial intelligence (AI) for tackling modern sustainability challenges. The research methodology includes a hetero-intelligence performance test, which juxtaposes human intelligence with AI in the formulation and implementation of sustainable policies. After testing this hetero-intelligence methodology, seven steps are rigorously described so that it can be replicated in any sustainability planning related context. The results underscore the capabilities and limitations of LLMs, underscoring the critical role of human intelligence in enhancing the efficacy of hetero-intelligence systems. This work fulfils the need of a rigorous methodological framework based on empirical steps that can provide unbiased outcomes to be integrated into sustainable planning and decision-making processes.•Assesses LLMs' limitations and capabilities regarding sustainable planning issues•A replicable methodology is proposed based on the combination of both human and artificial intelligence•It proposes and systematises the integration of a hetero-intelligent approach into the formulation of sustainability policies to be more efficient and effective.
Collapse
Affiliation(s)
- Eva M. Buitrago-Esquinas
- Faculty of Economics and Business Sciences, Universidad de Sevilla, Spain
- Research Centre for Tourism, Sustainability and Well-being (CinTurs), Universidade do Algarve, Faro, Portugal
| | - Miguel Puig-Cabrera
- Faculty of Economics and Business Sciences, Universidad de Sevilla, Spain
- Research Centre for Tourism, Sustainability and Well-being (CinTurs), Universidade do Algarve, Faro, Portugal
| | - José António C. Santos
- Faculty of Economics and Business Sciences, Universidad de Sevilla, Spain
- Research Centre for Tourism, Sustainability and Well-being (CinTurs), Universidade do Algarve, Faro, Portugal
| | - Margarida Custódio-Santos
- Faculty of Economics and Business Sciences, Universidad de Sevilla, Spain
- Research Centre for Tourism, Sustainability and Well-being (CinTurs), Universidade do Algarve, Faro, Portugal
| | - Rocío Yñiguez-Ovando
- Faculty of Economics and Business Sciences, Universidad de Sevilla, Spain
- Research Centre for Tourism, Sustainability and Well-being (CinTurs), Universidade do Algarve, Faro, Portugal
| |
Collapse
|
2
|
Garg S. ChatGPT is still struggling to revolutionize mental health policy. Asian J Psychiatr 2024; 93:103906. [PMID: 38217966 DOI: 10.1016/j.ajp.2023.103906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Accepted: 12/31/2023] [Indexed: 01/15/2024]
Affiliation(s)
- Sunny Garg
- Department of Psychiatry, Bhagat Phool Singh Government Medical College for Women Khanpur Kalan, Sonipat, Haryana, India.
| |
Collapse
|
3
|
Madrid-García A, Rosales-Rosado Z, Freites-Nuñez D, Pérez-Sancristóbal I, Pato-Cour E, Plasencia-Rodríguez C, Cabeza-Osorio L, Abasolo-Alcázar L, León-Mateos L, Fernández-Gutiérrez B, Rodríguez-Rodríguez L. Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training. Sci Rep 2023; 13:22129. [PMID: 38092821 PMCID: PMC10719375 DOI: 10.1038/s41598-023-49483-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 12/08/2023] [Indexed: 12/17/2023] Open
Abstract
The emergence of large language models (LLM) with remarkable performance such as ChatGPT and GPT-4, has led to an unprecedented uptake in the population. One of their most promising and studied applications concerns education due to their ability to understand and generate human-like text, creating a multitude of opportunities for enhancing educational practices and outcomes. The objective of this study is twofold: to assess the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain (MIR), and to evaluate the medical reasoning followed by these LLM to answer those questions. A dataset, RheumaMIR, of 145 rheumatology-related questions, extracted from the exams held between 2010 and 2023, was created for that purpose, used as a prompt for the LLM, and was publicly distributed. Six rheumatologists with clinical and teaching experience evaluated the clinical reasoning of the chatbots using a 5-point Likert scale and their degree of agreement was analyzed. The association between variables that could influence the models' accuracy (i.e., year of the exam question, disease addressed, type of question and genre) was studied. ChatGPT demonstrated a high level of performance in both accuracy, 66.43%, and clinical reasoning, median (Q1-Q3), 4.5 (2.33-4.67). However, GPT-4 showed better performance with an accuracy score of 93.71% and a median clinical reasoning value of 4.67 (4.5-4.83). These findings suggest that LLM may serve as valuable tools in rheumatology education, aiding in exam preparation and supplementing traditional teaching methods.
Collapse
Affiliation(s)
- Alfredo Madrid-García
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain.
| | - Zulema Rosales-Rosado
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | - Dalifer Freites-Nuñez
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | - Inés Pérez-Sancristóbal
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | - Esperanza Pato-Cour
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | | | - Luis Cabeza-Osorio
- Medicina Interna, Hospital Universitario del Henares, Avenida de Marie Curie, 0, 28822, Madrid, Spain
- Facultad de Medicina, Universidad Francisco de Vitoria, Carretera Pozuelo, Km 1800, 28223, Madrid, Spain
| | - Lydia Abasolo-Alcázar
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | - Leticia León-Mateos
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| | - Benjamín Fernández-Gutiérrez
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
- Facultad de Medicina, Universidad Complutense de Madrid, Madrid, Spain
| | - Luis Rodríguez-Rodríguez
- Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain
| |
Collapse
|
4
|
Morita P, Abhari S, Kaur J. Do ChatGPT and Other Artificial Intelligence Bots Have Applications in Health Policy-Making? Opportunities and Threats. Int J Health Policy Manag 2023; 12:8131. [PMID: 38618768 PMCID: PMC10843407 DOI: 10.34172/ijhpm.2023.8131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 10/16/2023] [Indexed: 04/16/2024] Open
Affiliation(s)
- Plinio Morita
- School of Public Health Sciences, University of Waterloo, Waterloo, ON, Canada
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
- Research Institute for Aging, University of Waterloo, Waterloo, ON, Canada
- Centre for Digital Therapeutics, Techna Institute, University Health Network, Toronto, ON, Canada
- Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Shahabeddin Abhari
- School of Public Health Sciences, University of Waterloo, Waterloo, ON, Canada
| | - Jasleen Kaur
- School of Public Health Sciences, University of Waterloo, Waterloo, ON, Canada
| |
Collapse
|
5
|
Ray PP, Majumder P. The Double-Edged Sword of AI in Biomedical Engineering: ChatGPT's Controversial Impact on Research and Collaboration Paradigms. Ann Biomed Eng 2023; 51:1904-1905. [PMID: 37202644 DOI: 10.1007/s10439-023-03237-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 05/10/2023] [Indexed: 05/20/2023]
Abstract
As artificial intelligence (AI) permeates various fields, ChatGPT emerges as a contentious force in biomedical engineering, stirring both excitement and concern. This letter explores the disruptive potential and controversial implications of ChatGPT on research, collaboration, and the future of the field. By presenting thought-provoking questions and confronting divisive issues, we aim to spark a lively debate on the responsible integration of AI technology in biomedical engineering while safeguarding the essence of human expertise.
Collapse
Affiliation(s)
| | - Poulami Majumder
- Maulana Abul Kalam Azad University of Technology, Kolkata, India
| |
Collapse
|
6
|
Ray PP, Majumder P. The Potential of ChatGPT to Transform Healthcare and Address Ethical Challenges in Artificial Intelligence-Driven Medicine. J Clin Neurol 2023; 19:509-511. [PMID: 37635433 PMCID: PMC10471548 DOI: 10.3988/jcn.2023.0158] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 05/22/2023] [Accepted: 06/05/2023] [Indexed: 08/29/2023] Open
Affiliation(s)
| | - Poulami Majumder
- Maulana Abul Kalam Azad University of Technology, Kolkata, India
| |
Collapse
|
7
|
Cheng K, Guo Q, He Y, Lu Y, Xie R, Li C, Wu H. Artificial Intelligence in Sports Medicine: Could GPT-4 Make Human Doctors Obsolete? Ann Biomed Eng 2023:10.1007/s10439-023-03213-1. [PMID: 37097528 DOI: 10.1007/s10439-023-03213-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2023] [Accepted: 04/17/2023] [Indexed: 04/26/2023]
Abstract
Sports medicine, an essential branch of orthopedics, focuses on preserving, restoring, improving, and rebuilding the function of the human motor system. As a thriving interdisciplinary field, sports medicine attracts not only the interest of the orthopedic community, but also artificial intelligence (AI). In this study, our team summarized the potential applications of GPT-4 in sports medicine including diagnostic imaging, exercise prescription, medical supervision, surgery treatment, sports nutrition, and science research. In our opinion, it is impossible that GPT-4 could make sports physicians obsolete. Instead, it could become an indispensable scientific assistant for sport doctors in future.
Collapse
Affiliation(s)
- Kunming Cheng
- Department of Intensive Care Unit, The Second Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan, China
| | - Qiang Guo
- Department of Orthopedics, Baodi Clinical College of Tianjin Medical University, Tianjin, China
| | - Yongbin He
- School of Sport Medicine and Rehabilitation, Beijing Sport University, Beijing, China
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Yanqiu Lu
- Department of Intensive Care Unit, The Second Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan, China
| | - Ruijie Xie
- Department of Microsurgery, The Affiliated Nanhua Hospital, Hengyang Medical School, University of South China, Hengyang, China.
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, Heidelberg, Germany.
| | - Cheng Li
- Department of Orthopaedic Surgery, Beijing Jishuitan Hospital, Fourth Clinical College of Peking University, Beijing, China.
- Center for Musculoskeletal Surgery (CMSC), Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt University of Berlin, and Berlin Institute of Health, Berlin, Germany.
| | - Haiyang Wu
- Department of Graduate School, Tianjin Medical University, Tianjin, China.
- Duke Molecular Physiology Institute, Duke University School of Medicine, Durham, NC, USA.
| |
Collapse
|