Readership awareness series - Paper 4: Chatbots and ChatGPT - Ethical considerations in scientific publications

Tracing the Footprints of AI in Radiology Literature: A Detailed Analysis of Journal Abstracts

PURPOSE

 To assess and compare the probabilities of AI-generated content within scientific abstracts from selected Q1 journals in the fields of radiology, nuclear medicine, and imaging, published between May and August 2022 and May and August 2023.

MATERIALS AND METHODS

 An extensive list of Q1 journals was acquired from Scopus in the fields of radiology, nuclear medicine, and imaging. All articles in these journals were acquired from the Medline databases, focusing on articles published between May and August in 2022 and 2023. The study specifically compared abstracts for limitations of the AI detection tool in terms of word constraints. Extracted abstracts from the two different periods were categorized into two groups, and each abstract was analyzed using the AI detection tool, a system capable of distinguishing between human and AI-generated content with a validated accuracy of 97.06 %. This tool assessed the probability of each abstract being AI-generated, enabling an in-depth comparison between the two groups in terms of the prevalence of AI-generated content probability.

RESULTS

 Group 1 and Group 2 exhibit significant variations in the characteristics of AI-generated content probability. Group 1, consisting of 4,727 abstracts, has a median AI-generated content probability of 3.8 % (IQR1.9-9.9 %) and peaks at 49.9 %, with the computation times contained within a range of 2 to 10 seconds (IQR 3-8 s). In contrast, Group 2, which is composed of 3,917 abstracts, displays a significantly higher median AI-generated content probability at 5.7 % (IQR2.8-12.9 %) surging to a maximum of 69.9 %, with computation times spanning from 2 to 14 seconds (IQR 4-11 s). This comparison yields a statistically significant difference in median AI-generated content probability between the two groups (p = 0.005). No significant correlation was observed between word count and AI probability, as well as between article type, primarily original articles and reviews, and AI probability, indicating that AI probability is independent of these factors.

CONCLUSION

 The comprehensive analysis reveals significant differences and variations in AI-generated content probabilities between 2022 and 2023, indicating a growing presence of AI-generated content. However, it also illustrates that abstract length or article type does not impact the likelihood of content being AI-generated.

KEY POINTS

  · The study examines AI-generated content probability in scientific abstracts from Q1 journals between 2022 to 2023.. · The AI detector tool indicates an increased median AI content probability from 3.8 % to 5.7 %.. · No correlation was found between abstract length or article type and AI probability..

Detection of fake papers in the era of artificial intelligence

OBJECTIVES

Paper mills, companies that write scientific papers and gain acceptance for them, then sell authorships of these papers, present a key challenge in medicine and other healthcare fields. This challenge is becoming more acute with artificial intelligence (AI), where AI writes the manuscripts and then the paper mills sell the authorships of these papers. The aim of the current research is to provide a method for detecting fake papers.

METHODS

The method reported in this article uses a machine learning approach to create decision trees to identify fake papers. The data were collected from Web of Science and multiple journals in various fields.

RESULTS

The article presents a method to identify fake papers based on the results of decision trees. Use of this method in a case study indicated its effectiveness in identifying a fake paper.

CONCLUSIONS

This method to identify fake papers is applicable for authors, editors, and publishers across fields to investigate a single paper or to conduct an analysis of a group of manuscripts. Clinicians and others can use this method to evaluate articles they find in a search to ensure they are not fake articles and instead report actual research that was peer reviewed prior to publication in a journal.

ChatGPT in Medical Education and Research: A Boon or a Bane?

Science fiction literature and films no longer focus on artificial intelligence. In contrast to all other aspects of life, medical education and clinical patient care have been progressing slowly. Recently, a lot of text from the internet was used to construct and train chatbots, especially ChatGPT. The language model ChatGPT, created by OpenAI, has emerged as a useful resource for medical research and education. It has proven to be a useful tool for researchers, students, and medical professionals because of its capacity to produce human-like answers to challenging medical queries. However, using ChatGPT also has significant drawbacks. The possibility of erroneous or biased information being spread, which could have negative effects on patient care, is one of the key worries. Moreover, the overreliance on technology in medical education could also lead to a decline in critical thinking and clinical decision-making skills. Overall, ChatGPT has the potential to be a boon to medical education and research, but its use must be accompanied by caution and critical evaluation.