What web-based information is available for people with Parkinson's disease interested in aquatic physiotherapy? A social listening study

Assessing the efficacy of artificial intelligence to provide peri-operative information for patients with a stoma

BACKGROUND

Stomas present significant lifestyle and psychological challenges for patients, requiring comprehensive education and support. Current educational methods have limitations in offering relevant information to the patient, highlighting a potential role for artificial intelligence (AI). This study examined the utility of AI in enhancing stoma therapy management following colorectal surgery.

MATERIAL AND METHODS

We compared the efficacy of four prominent large language models (LLM)-OpenAI's ChatGPT-3.5 and ChatGPT-4.0, Google's Gemini, and Bing's CoPilot-against a series of metrics to evaluate their suitability as supplementary clinical tools. Through qualitative and quantitative analyses, including readability scores (Flesch-Kincaid, Flesch-Reading Ease, and Coleman-Liau index) and reliability assessments (Likert scale, DISCERN score and QAMAI tool), the study aimed to assess the appropriateness of LLM-generated advice for patients managing stomas.

RESULTS

There are varying degrees of readability and reliability across the evaluated models, with CoPilot and ChatGPT-4 demonstrating superior performance in several key metrics such as readability and comprehensiveness. However, the study underscores the infant stage of LLM technology in clinical applications. All responses required high school to college level education to comprehend comfortably. While the LLMs addressed users' questions directly, the absence of incorporating patient-specific factors such as past medical history generated broad and generic responses rather than offering tailored advice.

CONCLUSION

The complexity of individual patient conditions can challenge AI systems. The use of LLMs in clinical settings holds promise for improving patient education and stoma management support, but requires careful consideration of the models' capabilities and the context of their use.

Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms

BACKGROUND

The widespread diffusion of Artificial Intelligence (AI) platforms is revolutionizing how health-related information is disseminated, thereby highlighting the need for tools to evaluate the quality of such information. This study aimed to propose and validate the Quality Assessment of Medical Artificial Intelligence (QAMAI), a tool specifically designed to assess the quality of health information provided by AI platforms.

METHODS

The QAMAI tool has been developed by a panel of experts following guidelines for the development of new questionnaires. A total of 30 responses from ChatGPT4, addressing patient queries, theoretical questions, and clinical head and neck surgery scenarios were assessed by 27 reviewers from 25 academic centers worldwide. Construct validity, internal consistency, inter-rater and test-retest reliability were assessed to validate the tool.

RESULTS

The validation was conducted on the basis of 792 assessments for the 30 responses given by ChatGPT4. The results of the exploratory factor analysis revealed a unidimensional structure of the QAMAI with a single factor comprising all the items that explained 51.1% of the variance with factor loadings ranging from 0.449 to 0.856. Overall internal consistency was high (Cronbach's alpha = 0.837). The Interclass Correlation Coefficient was 0.983 (95% CI 0.973-0.991; F (29,542) = 68.3; p < 0.001), indicating excellent reliability. Test-retest reliability analysis revealed a moderate-to-strong correlation with a Pearson's coefficient of 0.876 (95% CI 0.859-0.891; p < 0.001).

CONCLUSIONS

The QAMAI tool demonstrated significant reliability and validity in assessing the quality of health information provided by AI platforms. Such a tool might become particularly important/useful for physicians as patients increasingly seek medical information on AI platforms.

Content, educational value and quality analysis of videos about neurorehabilitation in people with multiple sclerosis on YouTube®

INTRODUCTION

The use of YouTube® has spread among patients with chronic diseases such as multiple sclerosis (MS). These patients consult the available videos to learn more about their disease in terms of diagnosis and making decisions about treatments, including rehabilitation. The aim of this study was to evaluate the content, educational value, and quality analysis of MS videos about neurorehabilitation on YouTube® using quantitative instruments.

MATERIALS AND METHODS

A search was conducted on YouTube®. The first 30 videos that met the inclusion criteria were reviewed. The videos were classified according to the upload source and the content. All videos included in the review were assessed by the DISCERN questionnaire, the JAMA benchmark, the global quality scale (GQS) and the video information and quality index (VIQI).

RESULTS

The mean scores were: 28.3 (±9.33) in DISCERN, 2 (±0.81) in JAMA, 2.57 (±1.22) in GQS, and 11.73 (±4.06) in VIQI. JAMA score statistically significantly differed according to upload source (p = 0.002), video content (p = 0.023) and the speaker (p = 0.002). The DISCERN, JAMA, GQS, and VIQI scores showed significant correlations with each other.

CONCLUSIONS

The analyzed videos about neurorehabilitation in people with MS on YouTube® were quite old since the upload, with a moderate duration and number of views, but with a poor quality of the content, educational value, and quality analysis of the videos. Our research showed that there were statistically significant differences in terms of quality, transparency, and reliability of the information, depending on the upload source, video content and the speaker.

Examining the Type, Quality, and Content of Web-Based Information for People With Chronic Pain Interested in Spinal Cord Stimulation: Social Listening Study

BACKGROUND

The increased availability of web-based medical information has encouraged patients with chronic pain to seek health care information from multiple sources, such as consultation with health care providers combined with web-based information. The type and quality of information that is available on the web is very heterogeneous, in terms of content, reliability, and trustworthiness. To date, no studies have evaluated what information is available about neuromodulation on the web for patients with chronic pain.

OBJECTIVE

This study aims to explore the type, quality, and content of web-based information regarding spinal cord stimulation (SCS) for chronic pain that is freely available and targeted at health care consumers.

METHODS

The social listening tool Awario was used to search Facebook (Meta Platforms, Inc), Twitter (Twitter, Inc), YouTube (Google LLC), Instagram (Meta Platforms, Inc), blogs, and the web for suitable hits with "pain" and "neuromodulation" as keywords. Quality appraisal of the extracted information was performed using the DISCERN instrument. A thematic analysis through inductive coding was conducted.

RESULTS

The initial search identified 2174 entries, of which 630 (28.98%) entries were eventually withheld, which could be categorized as web pages, including news and blogs (114/630, 18.1%); Reddit (Reddit, Inc) posts (32/630, 5.1%); Vimeo (Vimeo, Inc) hits (38/630, 6%); or YouTube (Google LLC) hits (446/630, 70.8%). Most posts originated in the United States (519/630, 82.4%). Regarding the content of information, 66.2% (383/579) of the entries discussed (fully discussed or partially discussed) how SCS works. In total, 55.6% (322/579) of the entries did not elaborate on the fact that there may be >1 potential treatment choice and 47.7% (276/579) did not discuss the influence of SCS on the overall quality of life. The inductive coding revealed 4 main themes. The first theme of pain and the burden of pain (1274/8886, 14.34% coding references) explained about pain, pain management, individual impact of pain, and patient experiences. The second theme included neuromodulation as a treatment approach (3258/8886, 36.66% coding references), incorporating the background on neuromodulation, patient-centered care, SCS therapy, and risks. Third, several device-related aspects (1722/8886, 19.38% coding references) were presented. As a final theme, patient benefits and testimonials of treatment with SCS (2632/8886, 29.62% coding references) were revealed with subthemes regarding patient benefits, eligibility, and testimonials and expectations.

CONCLUSIONS

Health care consumers have access to web-based information about SCS, where details about the surgical procedures, the type of material, working mechanisms, risks, patient expectations, testimonials, and the potential benefits of this therapy are discussed. The reliability, trustworthiness, and correctness of web-based sources should be carefully considered before automatically relying on the content.

Evidence for the Efficacy of Commercially Available Wearable Biofeedback Gait Devices: Consumer-Centered Review

BACKGROUND

The number of wearable technological devices or sensors that are commercially available for gait training is increasing. These devices can fill a gap by extending therapy outside the clinical setting. This was shown to be important during the COVID-19 pandemic when people could not access one-on-one treatment. These devices vary widely in terms of mechanisms of therapeutic effect, as well as targeted gait parameters, availability, and strength of the evidence supporting the claims.

OBJECTIVE

This study aimed to create an inventory of devices targeting improvement in gait pattern and walking behavior and identify the strength of the evidence underlying the claims of effectiveness for devices that are commercially available to the public.

METHODS

As there is no systematic or reproducible way to identify gait training technologies available to the public, we used a pragmatic, iterative approach using both the gray and published literature. Four approaches were used: simple words, including some suggested by laypersons; devices endorsed by condition-specific organizations or charities; impairment-specific search terms; and systematic reviews. A findable list of technological devices targeting walking was extracted separately by 3 authors. For each device identified, the evidence for efficacy was extracted from material displayed on the websites, and full-text articles were obtained from the scientific databases PubMed, Ovid MEDLINE, Scopus, or Google Scholar. Additional information on the target population, mechanism of feedback, evidence for efficacy or effectiveness, and commercial availability was obtained from the published material or websites. A level of evidence was assigned to each study involving the device using the Oxford Centre for Evidence-Based Medicine classification. We also proposed reporting guidelines for the clinical appraisal of devices targeting movement and mobility.

RESULTS

The search strategy for this consumer-centered review yielded 17 biofeedback devices that claim to target gait quality improvement through various sensory feedback mechanisms. Of these 17 devices, 11 (65%) are commercially available, and 6 (35%) are at various stages of research and development. Of the 11 commercially available devices, 4 (36%) had findable evidence for efficacy potential supporting the claims. Most of these devices were targeted to people living with Parkinson disease. The reporting of key information about the devices was inconsistent; in addition, there was no summary of research findings in layperson's language.

CONCLUSIONS

The amount of information that is currently available to the general public to help them make an informed choice is insufficient, and, at times, the information presented is misleading. The evidence supporting the effectiveness does not cover all aspects of technology uptake. Commercially available technologies help to provide continuity of therapy outside the clinical setting, but there is a need to demonstrate effectiveness to support claims made by the technologies.