Abstracts written by ChatGPT fool scientists

ChatGPT: Is this version good for healthcare and research?

BACKGROUND AND AIMS

There have been advancements in artificial intelligence (AI) and deep learning in the past decade. Recently, OpenAI Inc. has launched a new chatbot, called ChatGPT that interacts in a conversational way and its dialogue format makes is user friendly and fast. In this paper we aimed to explore the current position and the accuracy of currently available version of ChatGPT in relation to healthcare and medical research.

METHODS

We searched the PubMed, Scopus, and Google databases from 15th to 25th February 2023, using the keywords: 'ChatGPT' AND 'medical research, healthcare, and scientific writing'. We found 29 results in PubMed and 9 results in Scopus database., in English language. In addition, we (RV, AM) interacted with ChatGPT multiple times to review accuracy of responses of various medical questions.

RESULTS

Using literature search and interactions with ChatGPT with medical questions, we infer that this version generates answers rapidly but narrates data from existing internet literature in a general manner. However, as emphasised by the company in the landing page of ChatGPT, we found errors in responses to medical questions, Further, narrated data were limited up to September 2021. Positive features include admission of its limitations in medical field, and as it has been designed, learning from previous answers.

CONCLUSION

Current version of ChatGPT may be useful in a limited manner as a narrative AI chatbot for medical personnel, however, researchers are advised to fact check all statements provided, keeping in mind its limitations.

Evaluating the use of large language model in identifying top research questions in gastroenterology

The field of gastroenterology (GI) is constantly evolving. It is essential to pinpoint the most pressing and important research questions. To evaluate the potential of chatGPT for identifying research priorities in GI and provide a starting point for further investigation. We queried chatGPT on four key topics in GI: inflammatory bowel disease, microbiome, Artificial Intelligence in GI, and advanced endoscopy in GI. A panel of experienced gastroenterologists separately reviewed and rated the generated research questions on a scale of 1-5, with 5 being the most important and relevant to current research in GI. chatGPT generated relevant and clear research questions. Yet, the questions were not considered original by the panel of gastroenterologists. On average, the questions were rated 3.6 ± 1.4, with inter-rater reliability ranging from 0.80 to 0.98 (p < 0.001). The mean grades for relevance, clarity, specificity, and originality were 4.9 ± 0.1, 4.6 ± 0.4, 3.1 ± 0.2, 1.5 ± 0.4, respectively. Our study suggests that Large Language Models (LLMs) may be a useful tool for identifying research priorities in the field of GI, but more work is needed to improve the novelty of the generated research questions.

Extraventricular Neurocytoma of the Posterior Fossa: A Case Report Written by ChatGPT

ChatGPT (Chat Generative Pre-trained Transformer) is the latest artificial intelligence (AI) tool that has taken the internet by storm. A large language model-based AI tool, ChatGPT is used to generate natural prose-like answers learning from a large database of text. A lot is being written and spoken about the future of academic writing using this tool.We attempt to generate a short case report of a brain tumour labelled extraventriuclar neurocytoma (EVN) which is a rare tumour of the central nervous system. When posed with a set of questions on this uncommon topic, ChatGPT did well with relevant but not very specific answers. Validation of the information is an issue like with most machine learning solutions. This case report is an attempt to summarise EVN and its management. Q: Summarise extraventricular neurocytoma. ChatGPT: EVN is a rare and benign tumor that typically occurs in the brain. It often causes no symptoms, but as it grows, it can cause headaches, seizures, and changes in vision or balance. Surgery is usually curative, but there is a lack of studies to know the best treatment options. The prognosis is generally good with high rates of long-term survival and low rates of recurrence. Text italicised in the article is generated by ChatGPT.

Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios

This paper aims to highlight the potential applications and limits of a large language model (LLM) in healthcare. ChatGPT is a recently developed LLM that was trained on a massive dataset of text for dialogue with users. Although AI-based language models like ChatGPT have demonstrated impressive capabilities, it is uncertain how well they will perform in real-world scenarios, particularly in fields such as medicine where high-level and complex thinking is necessary. Furthermore, while the use of ChatGPT in writing scientific articles and other scientific outputs may have potential benefits, important ethical concerns must also be addressed. Consequently, we investigated the feasibility of ChatGPT in clinical and research scenarios: (1) support of the clinical practice, (2) scientific production, (3) misuse in medicine and research, and (4) reasoning about public health topics. Results indicated that it is important to recognize and promote education on the appropriate use and potential pitfalls of AI-based LLMs in medicine.

The Transparency of Science with ChatGPT and the Emerging Artificial Intelligence Language Models: Where Should Medical Journals Stand?

N/a.

Can artificial intelligence help for scientific writing?

This paper discusses the use of Artificial Intelligence Chatbot in scientific writing. ChatGPT is a type of chatbot, developed by OpenAI, that uses the Generative Pre-trained Transformer (GPT) language model to understand and respond to natural language inputs. AI chatbot and ChatGPT in particular appear to be useful tools in scientific writing, assisting researchers and scientists in organizing material, generating an initial draft and/or in proofreading. There is no publication in the field of critical care medicine prepared using this approach; however, this will be a possibility in the next future. ChatGPT work should not be used as a replacement for human judgment and the output should always be reviewed by experts before being used in any critical decision-making or application. Moreover, several ethical issues arise about using these tools, such as the risk of plagiarism and inaccuracies, as well as a potential imbalance in its accessibility between high- and low-income countries, if the software becomes paying. For this reason, a consensus on how to regulate the use of chatbots in scientific writing will soon be required.

Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis

We reflect on our experiences of using Generative Pre-trained Transformer ChatGPT, a chatbot launched by OpenAI in November 2022, to draft a research article. We aim to demonstrate how ChatGPT could help researchers to accelerate drafting their papers. We created a simulated data set of 100 000 health care workers with varying ages, Body Mass Index (BMI), and risk profiles. Simulation data allow analysts to test statistical analysis techniques, such as machine-learning based approaches, without compromising patient privacy. Infections were simulated with a randomized probability of hospitalisation. A subset of these fictitious people was vaccinated with a fictional vaccine that reduced this probability of hospitalisation after infection. We then used ChatGPT to help us decide how to handle the simulated data in order to determine vaccine effectiveness and draft a related research paper. AI-based language models in data analysis and scientific writing are an area of growing interest, and this exemplar analysis aims to contribute to the understanding of how ChatGPT can be used to facilitate these tasks.

ChatGPT Output Regarding Compulsory Vaccination and COVID-19 Vaccine Conspiracy: A Descriptive Study at the Outset of a Paradigm Shift in Online Search for Information

BACKGROUND

Being on the verge of a revolutionary approach to gathering information, ChatGPT (an artificial intelligence (AI)-based language model developed by OpenAI, and capable of producing human-like text) could be the prime motive of a paradigm shift on how humans will acquire information. Despite the concerns related to the use of such a promising tool in relation to the future of the quality of education, this technology will soon be incorporated into web search engines mandating the need to evaluate the output of such a tool. Previous studies showed that dependence on some sources of online information (e.g., social media platforms) was associated with higher rates of vaccination hesitancy. Therefore, the aim of the current study was to describe the output of ChatGPT regarding coronavirus disease 2019 (COVID-19) vaccine conspiracy beliefs. and compulsory vaccination.

METHODS

The current descriptive study was conducted on January 14, 2023 using the ChatGPT from OpenAI (OpenAI, L.L.C., San Francisco, CA, USA). The output was evaluated by two authors and the degree of agreement regarding the correctness, clarity, conciseness, and bias was evaluated using Cohen's kappa.

RESULTS

The ChatGPT responses were dismissive of conspiratorial ideas about severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) origins labeling it as non-credible and lacking scientific evidence. Additionally, ChatGPT responses were totally against COVID-19 vaccine conspiracy statements. Regarding compulsory vaccination, ChatGPT responses were neutral citing the following as advantages of this strategy: protecting public health, maintaining herd immunity, reducing the spread of disease, cost-effectiveness, and legal obligation, and on the other hand, it cited the following as disadvantages of compulsory vaccination: ethical and legal concerns, mistrust and resistance, logistical challenges, and limited resources and knowledge.

CONCLUSIONS

The current study showed that ChatGPT could be a source of information to challenge COVID-19 vaccine conspiracies. For compulsory vaccination, ChatGPT resonated with the divided opinion in the scientific community toward such a strategy; nevertheless, it detailed the pros and cons of this approach. As it currently stands, the judicious use of ChatGPT could be utilized as a user-friendly source of COVID-19 vaccine information that could challenge conspiracy ideas with clear, concise, and non-biased content. However, ChatGPT content cannot be used as an alternative to the original reliable sources of vaccine information (e.g., the World Health Organization [WHO] and the Centers for Disease Control and Prevention [CDC]).

A publishing infrastructure for AI-assisted academic authoring

In this work we investigate how models with advanced natural language processing capabilities can be used to reduce the time-consuming process of writing and revising scholarly manuscripts. To this end, we integrate large language models into the Manubot publishing ecosystem to suggest revisions for scholarly text. We tested our AI-based revision workflow in three case studies of existing manuscripts, including the present one. Our results suggest that these models can capture the concepts in the scholarly text and produce high-quality revisions that improve clarity. Given the amount of time that researchers put into crafting prose, we anticipate that this advance will revolutionize the type of knowledge work performed by academics.

The Future of AI in Ovarian Cancer Research: The Large Language Models Perspective

Conversational large language model (LLM)-based chatbots utilize neural networks to process natural language. By generating highly sophisticated outputs from contextual input text, they revolutionize the access to further learning, leading to the development of new skills and personalized interactions. Although they are not developed to provide healthcare, their potential to address biomedical issues is rather unexplored. Healthcare digitalization and documentation of electronic health records is now developing into a standard practice. Developing tools to facilitate clinical review of unstructured data such as LLMs can derive clinical meaningful insights for ovarian cancer, a heterogeneous but devastating disease. Compared to standard approaches, they can host capacity to condense results and optimize analysis time. To help accelerate research in biomedical language processing and improve the validity of scientific writing, task-specific and domain-specific language models may be required. In turn, we propose a bespoke, proprietary ovarian cancer-specific natural language using solely in-domain text, whereas transfer learning drifts away from the pretrained language models to fine-tune task-specific models for all possible downstream applications. This venture will be fueled by the abundance of unstructured text information in the electronic health records resulting in ovarian cancer research ultimately reaching its linguistic home.