A cross-sectional study of the endorsement proportion of reporting guidelines in 1039 Chinese medical journals

The policies on the use of large language models in radiological journals are lacking: a meta-research study

OBJECTIVE

To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.

METHODS

In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.

RESULTS

The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).

CONCLUSION

The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.

CRITICAL RELEVANCE STATEMENT

It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.

KEY POINTS

The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.

Endorsements of five reporting guidelines for biomedical research by journals of prominent publishers

Biomedical research reporting guidelines provide a framework by which journal editors and the researchers who conduct studies can ensure that the reported research is both complete and transparent. With more than 16 different guidelines for the 11 major study types of medical and health research, authors need to be familiar with journal reporting standards. To assess the current endorsements of reporting guidelines for biomedical and health research, this study examined the instructions for authors (IFAs) of 559 biomedical journals by 11 prominent publishers that publish original research or systematic reviews/meta-analyses. Data from the above original sources were cleaned and restructured, and analyzed in a database and text miner. Each journal's instructions or information for authors were examined to code if any of five prominent reporting guidelines were mentioned and what form the guideline adherence demonstration took. Seventeen journals published the reporting guidelines. Four of the five reporting guidelines listed journals as endorsers. For journals with open peer review reports, a sample of journals and peer reviews was analyzed for mention of adherence to reporting guidelines. The endorsement of research guidelines by publishers and their associated journals is inconsistent for some publishers, with only a small number of journals endorsing relevant guidelines. Based on the analysis of open peer reviews, there is evidence that some reviewers check the adherence to the endorsed reporting guidelines. Currently, there is no universal endorsement of reporting guidelines by publishers nor ways of demonstrating adherence to guidelines. Journals may not directly inform authors of their guideline endorsements, making it more difficult for authors to adhere to endorsed guidelines. Suggestions derived from the findings are provided for authors, journals, and reporting guidelines to ensure increased adequate use of endorsed reporting guidelines.

STROBE, CONSORT, PRISMA, MOOSE, STARD, SPIRIT, and other guidelines - Overview and application

The purpose of research is to seek answers and new knowledge. When conducted properly and systematically, research adds to humanity's corpus of knowledge and hence to our general advancement. However, this is only possible if reported research is accurate and transparent. Guidelines for all the major types of studies (STROBE, CONSORT, PRISMA, MOOSE, STARD, and SPIRIT) have been developed and refined over the years, and their inception, development, and application are briefly discussed in this paper. Indeed, there are currently over 250 of these guidelines for various types of medical research, and these are published by the EQUATOR network. This paper will also briefly review progress in acceptance and adoption of these guidelines.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.