Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites

Appraisal of Spanish-language online patient education resources for kidney stones

INTRODUCTION

The internet is a primary source of health information, our study addresses the need to assess the credibility, readability, and reliability of Spanish-language medical websites concerning nephrolithiasis. With the Spanish-speaking population in the United States rapidly approaching 20% and growing, our research endeavors to fill a critical gap by evaluating the quality of online resources in Spanish for this common medical condition.

METHODS

In our study, we evaluated the quality of Spanish-language online resources on nephrolithiasis for the growing Spanish-speaking population in the United States. Using Google searches with terms like "cálculos renales" and "nefrolitiasis," we analyzed 80 unique websites using Fernandez-Huerta readability software, HON-code certification, JAMA benchmark criteria, source classification, and brief DISCERN. The evaluation was performed by an expert urologist and a medical student, with discrepancies resolved by a third experienced reviewer.

RESULTS

Interrater reliability was strong for JAMA (ICC: 0.98) and DISCERN (ICC: 0.85). Websites mainly originated from Physician/Community Hospitals (43.75%) and University/Academic institutions (23.75%). Only 32.5% had HON-code certification, and 17.5% met all JAMA criteria. Readability typically matched an 8th to 9th grade level, but 58.75% scored below the good quality threshold on DISCERN, with University/Academic sources performing significantly better than Physician/Community sources (p = 0.0078). No significant differences were noted in credibility or readability between these groups.

CONCLUSIONS

Our findings indicate that Spanish-language resources on nephrolithiasis often lack in quality, readability, and credibility, highlighting the need for enhanced online materials to support patient and provider management of kidney stones. University sources provide notably higher-quality information than other resource types.

A comparative analysis of CDC and AI-generated health information using computer-aided text analysis

BACKGROUND

AI-generated content is easy to access. Members of the public use it as an alternative or to supplement official sources, such as the Centers for Disease Control and Prevention (CDC). However, the quality and reliability of AI-generated health information is questionable. This study aims to understand how AI-generated health information differs from that provided by the CDC, particularly in terms of sentiment, readability, and overall quality. Language expectancy theory serves as a framework and offers insights into how people's expectations of message content from different sources can influence perceived credibility and persuasiveness of such information.

METHODS

Computer-aided text analysis was used to analyze 20 text entries from the CDC and 20 entries generated by ChatGPT 3.5. Content analysis utilizing human coders was used to assess the quality of information.

RESULTS

ChatGPT used more negative sentiments, particularly words associated with anger, sadness, and disgust. The CDC's health messages were significantly easier to read than those generated by ChatGPT. Furthermore, ChatGPT's responses required a higher reading grade level. In terms of quality, the CDC's information was a little higher quality than that of ChatGPT, with significant differences in DISCERN scores.

CONCLUSION

Public health professionals need to educate the general public about the complexity and quality of AI-generated health information. Health literacy programs should address topics about quality and readability of AI-generated content. Other recommendations for using AI-generated health information are provided.

Cross-language insights: Comparing Spanish and English-language cervical cancer content on TikTok

OBJECTIVES

To investigate themes, quality, and reliability of cervical cancer (CC) related Spanish-language content on TikTok as compared to English-language content.

METHODS

TikTok was systematically searched for the top 200 Spanish-language and 100 English-language posts for CC in August 2024. Data were collected including creator demographics, timing of posts in relation to diagnosis, tone, intent, and themes. The modified DISCERN, a validated tool including structured questions assessing reliability and of health-related resources was utilized to assess videos with educational intent. Differences were assessed.

RESULTS

There were more physician Spanish-language creators (27.8 %) compared to physician English-language creators (11.00 %, p < 0.001). Most Spanish-language education videos were created by physicians (82.25 %). Spanish-language posts were more likely to feature themes of treatment toxicities (7.5 % vs 1.7 %, p < 0.001), faith (3.7 % vs 0.7 %, p = 0.003), pain (2.0 % vs 0 %, p = 0.001), family support (1.6 % vs 0 %, p = 0.004), hair loss (1.8 % vs 0.3 %, p = 0.039), and long-term treatment sequelae (1.4 % vs 0 %, p = 0.007); and less likely to address themes of diagnosis (11.8 % vs 18.0 %, p < 0.001), fear or anxiety about cancer (3.4 % vs 8.2 %, p = 0.002), impact on sexual function (0.6 % vs 6.1 %, p < 0.001) and parenthood (0 % vs 3.7 %, p < 0.001). The median DISCERN score was 1.42 (Interquartile Range [IQR] 1.33-1.67) and 1.33 (IQR 1.00-1.92), for Spanish and English-language posts, respectively (p = 0.385).

CONCLUSIONS

There were significantly more Spanish-language physician creators suggesting a strong interest in physician-driven content. Thematic differences between Spanish-language and English-language posts highlight important differences in experiences. CC-related content is of poor educational quality, regardless of language. Improving TikTok educational content could increase CC awareness and prevention.

Assessing the efficacy of artificial intelligence to provide peri-operative information for patients with a stoma

BACKGROUND

Stomas present significant lifestyle and psychological challenges for patients, requiring comprehensive education and support. Current educational methods have limitations in offering relevant information to the patient, highlighting a potential role for artificial intelligence (AI). This study examined the utility of AI in enhancing stoma therapy management following colorectal surgery.

MATERIAL AND METHODS

We compared the efficacy of four prominent large language models (LLM)-OpenAI's ChatGPT-3.5 and ChatGPT-4.0, Google's Gemini, and Bing's CoPilot-against a series of metrics to evaluate their suitability as supplementary clinical tools. Through qualitative and quantitative analyses, including readability scores (Flesch-Kincaid, Flesch-Reading Ease, and Coleman-Liau index) and reliability assessments (Likert scale, DISCERN score and QAMAI tool), the study aimed to assess the appropriateness of LLM-generated advice for patients managing stomas.

RESULTS

There are varying degrees of readability and reliability across the evaluated models, with CoPilot and ChatGPT-4 demonstrating superior performance in several key metrics such as readability and comprehensiveness. However, the study underscores the infant stage of LLM technology in clinical applications. All responses required high school to college level education to comprehend comfortably. While the LLMs addressed users' questions directly, the absence of incorporating patient-specific factors such as past medical history generated broad and generic responses rather than offering tailored advice.

CONCLUSION

The complexity of individual patient conditions can challenge AI systems. The use of LLMs in clinical settings holds promise for improving patient education and stoma management support, but requires careful consideration of the models' capabilities and the context of their use.

Modern Digital Query Analytics of Patient Education Materials on Acanthosis Nigricans: Systematic Search and Content Analysis

Background

Online digital materials are integral to patient education and health care outcomes in dermatology. Acanthosis nigricans (AN) is a common condition, often associated with underlying diseases such as insulin resistance. Patients frequently search the internet for information related to this cutaneous finding. To our knowledge, the quality of online educational materials for AN has not been systematically examined.

Objective

The primary objective of this study was to profile the readability and quality of the content of publicly available digital educational materials on AN and identify questions frequently asked by patients.

Methods

This study analyzed publicly available internet sources to identify the most frequent questions searched by patients regarding AN using the Google Rankbrain algorithm. Furthermore, available articles on AN were evaluated for quality and reading level using metrics such as the Brief DISCERN score, and readability was determined using three specific scales including the Flesch-Kincaid score, Gunning Fog index, and the Coleman-Liau index, based on literature.

Results

Patients most frequently accessed facts on AN from government sources, which comprised 30% (n=15) of the analyzed sources. The available articles did not meet quality standards and were at a reading level not appropriate for the general public. The majority of articles (n=29/50, 58%) had substandard Brief DISCERN scores, failing to meet the criteria for good quality.

Conclusions

Clinicians should be aware of the paucity of valuable online educational material on AN and educate their patients accordingly.

Spanish language version of the "Medical Quality Video Evaluation Tool" (MQ-VET): Cross-cultural AI-supported adaptation and validation study

BACKGROUND

The Medical Quality Video Evaluation Tool (MQ-VET) is a standardized instrument for assessing health-related video quality, yet it is only available in English. This study addresses the growing demand for a Spanish version to better support the increasing Spanish-speaking population seeking reliable digital health content.

OBJECTIVE

To adapt and validate the MQ-VET into Spanish, ensuring robust psychometric reliability and validity through rigorous cross-cultural adaptation methods, augmented by the integration of artificial intelligence (AI) tools.

MATERIALS AND METHODS

Following international guidelines, the MQ-VET was translated, back-translated, and reviewed by experts. AI-based tools were employed to refine linguistic and cultural accuracy. Psychometric properties were evaluated by 60 participants (30 healthcare and 30 nonhealthcare professionals), focusing on reliability, agreement, and concurrent validity with the DISCERN instrument.

RESULTS

The Spanish MQ-VET showed excellent reliability (Cronbach's alpha>0.90, ICC=0.81) and strong concurrent validity (Pearson r = 0.9435, Spearman r = 0.9482, p < 0.0001), alongside with a robust linear regression result (R²=0.8902). Bland-Altman analysis confirmed a robust agreement, and AI-driven tools performed the factorial analysis that revealed a clear three-factor structure explaining 81.1% of the variance.

CONCLUSIONS

The Spanish MQ-VET is a reliable and valid instrument for assessing the quality of health-related videos, applicable to both healthcare professionals and individuals outside the healthcare field. Leveraging AI-driven methodologies, it serves as a robust resource for enhancing digital health literacy and promoting critical appraisal of video content among Spanish-speaking populations.

How trustworthy is online dental information for individuals with Down syndrome from Ibero-American countries?

BACKGROUND

Internet has become an indispensable source of health-related information. However, several studies have shown there to be a lack of quality control for webpages related to disability. Specifically, available content concerning Down syndrome (DS) and dentistry is limited and of dubious quality.

OBJECTIVE

The aim of the present study was to assess the quality of online content in Spanish and Portuguese on dental care for individuals with DS.

METHODS

A simultaneous search in Google and Bing using the terms "Down syndrome" and "odontology/dentist/dental treatment" in Spanish and Portuguese was conducted in seven Ibero-American countries (Argentina, Brazil, Chile, Colombia, Spain, Mexico, and Portugal). The first 100 consecutive pages of results from the three combinations of terms in each of the search engines were accessed and selected by applying conventional exclusion criteria. The selected pages were classified according to their authorship, specificity and dissemination potential. The quality of the online content was assessed using the DISCERN questionnaire and the Questionnaire to Evaluate Health Web Sites According to European Criteria (QEEC). The presence of the Health On Net (HON) and Accredited Medical Website (AMW) seals was also assessed.

RESULTS

The mean DISCERN score was 2.51 ± 0.85 and 2.57 ± 0.86 for the Spanish and Portuguese webpages, respectively. The mean readability score was 3.43 ± 1.26 and 3.25 ± 1.08 for the Spanish and Portuguese webpages, respectively. None of the selected webpages presented the HONcode or AMW trust seals.

CONCLUSIONS

The content available online in Spanish and Portuguese regarding Down syndrome and dentistry is scarce and of highly questionable quality.

Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms

BACKGROUND

The widespread diffusion of Artificial Intelligence (AI) platforms is revolutionizing how health-related information is disseminated, thereby highlighting the need for tools to evaluate the quality of such information. This study aimed to propose and validate the Quality Assessment of Medical Artificial Intelligence (QAMAI), a tool specifically designed to assess the quality of health information provided by AI platforms.

METHODS

The QAMAI tool has been developed by a panel of experts following guidelines for the development of new questionnaires. A total of 30 responses from ChatGPT4, addressing patient queries, theoretical questions, and clinical head and neck surgery scenarios were assessed by 27 reviewers from 25 academic centers worldwide. Construct validity, internal consistency, inter-rater and test-retest reliability were assessed to validate the tool.

RESULTS

The validation was conducted on the basis of 792 assessments for the 30 responses given by ChatGPT4. The results of the exploratory factor analysis revealed a unidimensional structure of the QAMAI with a single factor comprising all the items that explained 51.1% of the variance with factor loadings ranging from 0.449 to 0.856. Overall internal consistency was high (Cronbach's alpha = 0.837). The Interclass Correlation Coefficient was 0.983 (95% CI 0.973-0.991; F (29,542) = 68.3; p < 0.001), indicating excellent reliability. Test-retest reliability analysis revealed a moderate-to-strong correlation with a Pearson's coefficient of 0.876 (95% CI 0.859-0.891; p < 0.001).

CONCLUSIONS

The QAMAI tool demonstrated significant reliability and validity in assessing the quality of health information provided by AI platforms. Such a tool might become particularly important/useful for physicians as patients increasingly seek medical information on AI platforms.

Evaluating the Readability, Credibility, and Quality of Spanish-Language Online Educational Resources for Knee Arthroplasty: Implications for Patient Education and Health Disparities

INTRODUCTION

Spanish-speaking individuals may experience language-based disparities related to elective orthopaedic procedures. Because patients often seek online health information, we assessed the readability, credibility, and quality of Spanish-language educational websites for knee arthroplasty.

METHODS

We queried "Google," "Yahoo," and "Bing" using the term "reemplazo de rodilla" (knee replacement in Spanish) and extracted the top 50 websites per search engine. Websites were categorized by information source (physician/community hospital, university/academic, other) and presence of HONcode certification. Information was assessed for readability (Fernández-Huerta formula), credibility (Journal of the American Medical Association benchmark criteria), and quality (Brief DISCERN tool); scores were compared between the categories.

RESULTS

A total of 77 unique websites were included (40.3% physician/community hospital, 35.1% university/academic). The median readability score was 59.4 (10th to 12th-grade reading level); no websites achieved the recommended level of ≤6th grade. The median Journal of the American Medical Association benchmark score was 2 (interquartile range 1 to 3), with only 7.8% of websites meeting all criteria. The median Brief DISCERN score was 16 (interquartile range 12 to 20), with 50.7% meeting the threshold for good quality. University/academic websites had better readability (P = 0.02) and credibility (P = 0.002) but similar quality (P > 0.05) compared with physician/community hospital websites. In addition, HONcode-certified websites had better quality scores (P = 0.045) but similar readability and credibility (P > 0.05) compared with noncertified websites.

DISCUSSION

We identified limitations in readability, credibility, and quality of Spanish-language online educational resources for knee arthroplasty. Healthcare providers should be aware of these patient education barriers when counseling patients, and efforts should be made to support the online information needs of Spanish-speaking orthopaedic patients and mitigate language-based disparities.

Digital Education on Hospital Nutrition Diets: What Do Patients Want to Know?

INTRODUCTION

Therapeutic nutrition plays an imperative role during a patient's hospital course. There is a tremendous body of literature that emphasizes the systematic delivery of information regarding hospital nutrition diets. A major component of delivering healthcare information is the principle of providing quality healthcare information, but this has not yet been investigated on hospital nutrition diets. This study aimed to evaluate the comprehension and readability of patient education materials regarding therapeutic hospital diets.

METHODOLOGY

The methodology employed the use of publicly available questions regarding hospital nutrition diets and categorized them per Rothwell's Classification of Questions. Additionally, the questions were extracted online and have an associated digital article linked to the question. These articles underwent analysis for readability scores.

RESULTS

This study's findings reveal that most hospital diets do not meet the recommended grade-reading levels.

CONCLUSIONS

This underscores the need for healthcare providers to enhance patient education regarding hospital diets. The prevalence of "Fact" questions showcases the importance of clearly explaining diets and dietary restrictions to patients.

Readability and quality assessment of online patient education materials for spinal and epidural anesthesia

PURPOSE

Guidelines recommend that health-related information for patients should be written at or below the sixth-grade level. We sought to evaluate the readability level and quality of online patient education materials regarding epidural and spinal anesthesia.

METHODS

We evaluated webpages with content written specifically regarding either spinal or epidural anesthesia, identified using 11 relevant search terms, with seven commonly used readability formulas: Flesh-Kincaid Grade Level (FKGL), Gunning Fox Index (GFI), Coleman-Liau Index (CLI), Automated Readability Index (ARI), Simple Measure of Gobbledygook (SMOG), Flesch Reading Ease (FRE), and New Dale-Chall (NDC). Two evaluators assessed the quality of the reading materials using the Brief DISCERN tool.

RESULTS

We analyzed 261 webpages. The mean (standard deviation) readability scores were: FKGL = 8.8 (1.9), GFI = 11.2 (2.2), CLI = 10.3 (1.9), ARI = 8.1 (2.2), SMOG = 11.6 (1.6), FRE = 55.7 (10.8), and NDC = 5.4 (1.0). The mean grade level was higher than the recommended sixth-grade level when calculated with six of the seven readability formulas. The average Brief DISCERN score was 16.0.

CONCLUSION

Readability levels of online patient education materials pertaining to epidural and spinal anesthesia are higher than recommended. When we evaluated the quality of the information using a validated tool, the materials were found to be just below the threshold of what is considered good quality. Authors of educational materials should provide not only readable but also good-quality information to enhance patient understanding.

Examining the Public's Most Frequently Asked Questions About Carpal Tunnel Syndrome and Appraising Online Information About Treatment

BACKGROUND

Carpal tunnel syndrome (CTS) is the most common entrapment neuropathy. Patients often search online for health information regarding common musculoskeletal complaints. Thus, the purpose of this study was to use language processing information from Google to assess the content of CTS frequently asked questions (FAQs) searched online and the transparency and quality of online CTS information.

METHODS

On March 11, 2021, we searched Google for 3 terms "carpal tunnel syndrome treatment," "carpal tunnel syndrome surgical treatment," and "carpal tunnel syndrome non-surgical treatment" until a minimum of 100 FAQs and their answer links were extracted from each search. We used Rothwell classification to categorize the FAQs. The Journal of the American Medical Association's benchmark criteria were used to assess information transparency. Information quality was assessed using the Brief DISCERN tool.

RESULTS

Our Google search returned 124 unique FAQs. Fifty-six (45.2%) were value based and most were related to the evaluation of treatment options (45/56, 80.4%). The most common source type was medical practices (26.6%). Nearly half of the answer sources (45.9%) were found to be lacking in transparency. One-way analysis of variance revealed a significant difference in mean Brief DISCERN scores among the 5 source types, F(4, 119) = 5.93, P = .0002, with medical practices averaging the worst score (13.73/30).

CONCLUSIONS

Patients are most commonly searching Google to gain information regarding CTS treatment options. Online sources such as medical practices should use widely accepted rubrics for ensuring transparency and quality prior to publishing CTS information.

Assessment of the reliability and quality of breast cancer related videos on TikTok and Bilibili: cross-sectional study in China

Background

As the most common malignant tumor in the world, breast cancer also brings a huge disease burden to China. Ordinary people are increasingly inclined to use the Internet, especially video social platforms, as a source of health information. Educating the public to obtain correct information is important to reduce the incidence of breast cancer and improve the prognosis. However, the quality and reliability of breast cancer-related video content have not been fully studied.

Objective

This study aims to evaluate the quality of the information of breast cancer-related videos on TikTok and Bilibili video sharing platforms and factors related to video quality.

Methods

We collected the top 100 videos about breast cancer on TikTok and Bilibili, respectively. Categorize videos according to video source and video content. Video quality and reliability were assessed using Global Quality Score (GQS) and modified DISCERN (mDISCERN) tools. We also analyzed the correlation between video quality and video likes, comments, saves, and shares.

Results

Although the quality and reliability of Bilibili's breast cancer videos were higher than TikTok (p = 0.002 and p = 0.001, respectively), the video quality of both video sharing platforms was not satisfactory, with a median GQS scores of 2.00 and 3.00 and mDISCERN scores of 1.00 and 2.00, respectively. In general, the quality and reliability of videos released by medical practitioners were higher than those of non-medical practitioners, and the quality and reliability of videos covering disease-related knowledge were higher than those of news reports (all p < 0.001). Among medical practitioners, the quality of videos uploaded by doctors in breast disease was significantly lower than that of doctors in other areas (p < 0.05). There was a significant positive correlation between video quality and duration (r = 0.240, p < 0.001), a weak negative correlation between video quality and likes (r = 0.191, p < 0.01), video quality and comments (r = 0.256, p < 0.001), video reliability and likes (r = 0.198, p < 0.001), video reliability and comments (r = 0.243, p < 0.01).

Conclusion

Our study shows that the quality and reliability of breast cancer-related videos on TikTok and Bilibili are poor, and the overall quality is unsatisfactory. But videos uploaded by medical practitioners covering disease knowledge, prevention and treatment are of higher quality. Medical practitioners are encouraged to publish more high-quality videos, while video social platforms should formulate relevant policies to censor and supervise health education videos, so as to enable the public to obtain reliable health information.

The quality, suitability, and readability of web-based resources on endometriosis-associated dyspareunia: A systematic review

People commonly and increasingly rely on the internet to search for health information, including those related to endometriosis-associated dyspareunia. Yet the content of such websites may be of variable accuracy and quality. This review aims to evaluate the quality, readability, and suitability of web-based resources on endometriosis-associated dyspareunia for patients. We searched 3 databases - Google, Bing, and Yahoo - to identify websites related to endometriosis-associated dyspareunia. Two independent reviewers screened the search results against inclusion and exclusion criteria. Another set of two reviewers evaluated the selected websites using validated measurement instruments. Out of 450 websites, 21 met the inclusion criteria and were evaluated. More than half of the websites had information on content updates, reported on authorship, or disclosed sponsorship information. The mean quality and suitability scores were 47.5 (SD = 13.3) and 65.2 (SD = 13.6) respectively, thus suggesting generally adequate quality and suitability levels. However, the mean readability scores exceeded the recommended level for health-related websites. The poor readability of the websites might limit accessibility for a significant proportion of patients with low educational levels. The findings of this review have implications for designing high-quality, readable and up-to-date web interventions for people who rely on web platforms as an alternative or complementary source of health information on dyspareunia.

Caution! AI Bot Has Entered the Patient Chat: ChatGPT Has Limitations in Providing Accurate Urologic Healthcare Advice

OBJECTIVE

To conduct the first study examining the accuracy of ChatGPT, an artificial intelligence (AI) chatbot, derived patient counseling responses based on clinical care guidelines in urology using a validated questionnaire.

METHODS

We asked ChatGPT a set of 13 urological guideline-based questions three times. Answers were evaluated for appropriateness and using Brief DISCERN (BD), a validated healthcare information assessment questionnaire. Data analysis included descriptive statistics and Student's t test (SAS Studio).

RESULTS

60% (115/195) of ChatGPT responses were deemed appropriate. Variability existed between responses to the same prompt, with 25% of the 13 question sets having discordant appropriateness designations. The average BD score was 16.8 ± 3.59. Only 7 (54%) of 13 topics and 21 (54%) of 39 responses met the BD cut-off score of ≥16 to denote good-quality content. Appropriateness was associated with higher overall and Relevance domain scores (both P < .01). The lowest BD domain scores were for Source categories, since ChatGPT does not provide references by default. With prompting, 92.3% had ≥1 incorrect, misinterpreted, or nonfunctional citations.

CONCLUSION

While ChatGPT provides appropriate responses to urological questions more than half of the time, it misinterprets clinical care guidelines, dismisses important contextual information, conceals its sources, and provides inappropriate references. Chatbot models hold great promise, but users should be cautious when interpreting healthcare-related advice from existing AI models. Additional training and modifications are needed before these AI models will be ready for reliable use by patients and providers.

Frequently asked questions about mobility devices among older adults

PURPOSE

To assess frequently asked questions (FAQs) about mobility devices among older adults.

MATERIALS AND METHODS

We searched multiple terms on Google to find FAQs. Rothwell's classification, JAMA benchmark criteria, and Brief DISCERN were used to categorize and assess each entry.

RESULTS

Our search yielded 224 unique combinations of questions and linked answer sources. Viewing questions alone resulted in 214 unique FAQs, with the majority seeking factual information (130/214, 60.7%). Viewing website sources alone resulted in 175 unique answer sources, most of which were retail commercial sites (68/175, 38.9%) followed by non-retail commercial sites (65/175, 37.1%). Statistical analysis showed a significant difference between the JAMA benchmark scores by source type (p < 0.00010) and Brief DISCERN scores by source type (p = 0.0001).

DISCUSSION

Our findings suggest government, academic, and possibly non-retail commercial sources may provide better quality information about the use of mobility devices. We recommend medical providers be prepared to promote and provide quality resources on the risks, benefits, and proper techniques for using mobility devices.

Digital Patient Education on Xanthelasma Palpebrarum: A Content Analysis

Patient education has been transformed using digital media and online repositories which disseminate information with greater efficiency. In dermatology, this transformation has allowed for patients to gain education on common cutaneous conditions and improve health literacy. Xanthelasma palpebrarum is one of the most common cutaneous conditions, yet there is a poor understanding of how digital materials affect health literacy on this condition. Our study aimed to address this paucity of literature utilizing Brief DISCERN, Rothwell's Classification of Questions, and six readability calculations. The findings of this study indicate a poor-quality profile (Brief DISCERN < 16) regarding digital materials and readability scores which do not meet grade-level recommendations in the United States. This indicates a need to improve the current body of educational materials used by clinicians for diagnosing and managing xanthelasma palpebrarum.

Internet search analysis on the treatment of rheumatoid arthritis: What do people ask and read online?

OBJECTIVES

This study aimed to characterize the content of frequently asked questions about the treatment of rheumatoid arthritis (RA) on the internet in Japan and to evaluate the quality of websites related to the questions.

METHODS

We searched terms on the treatment of RA on Google and extracted frequently asked questions generated by the Google "people also ask" function. The website that answered each question was also obtained. We categorized the questions based on the content. The quality of the websites was evaluated using the brief DISCERN, Journal of American Medical Association benchmark criteria, and Clear Communication Index.

RESULTS

Our search yielded 83 questions and the corresponding websites. The most frequently asked questions were regarding the timeline of treatment (n = 17, 23%) and those on the timeline of the clinical course (n = 13, 16%). The median score of brief DISCERN was 11 points, with only 7 (8%) websites having sufficient quality. Websites having sufficient quality based on the Journal of American Medical Association benchmark criteria and Clear Communication Index were absent.

CONCLUSIONS

The questions were most frequently related to the timeline of treatment and clinical course. Physicians should provide such information to patients with RA in the counseling and education materials.

"More than a song and dance": Exploration of patient perspectives and educational quality of gynecologic cancer content on TikTok

OBJECTIVE

To investigate themes, quality, and reliability of gynecologic cancer-related content on the social media application TikTok.

METHODS

TikTok was systematically searched for the 100 most popular posts for ovarian cancer (OC), endometrial cancer (EC), cervical cancer (CC), vulvar cancer (VC), and gestational trophoblastic disease (GTD) in August 2022. Data was collected for demographics, tone, and themes. Educational videos were rated for quality and reliability utilizing the modified DISCERN scale. Relationships between content demographics, disease sites, and themes were assessed.

RESULTS

As of August 2022, the top five hashtags for each gynecologic cancer on TikTok had 466.7 million views. 430 of the top 500 posts were eligible for inclusion (OC: n = 86, CC: n = 93, EC: n = 98, GTD: n = 63, VC: n = 90). The majority of creators (n = 323, 75.1%) were White, 33 (7.7%) were Black, 20 (4.6%) were Asian/Pacific Islander (API), 10 (2.3%) were South Asian, 20 (4.7%) were Hispanic/Latino/a, 24 (5.5%) were unable to determine. Eleven central themes were identified, with significant differences when analyzed by disease site and race. The median DISCERN score for all posts was 1.0, indicating poor educational quality and reliability. When compared by race, South Asian/API posters received the highest scores (3, IQR 2.5) versus Black (2: IQR 3), Hispanic/Latino/a (2: IQR 0), and White posters (1, IQR 2) (p = 0.0013).

CONCLUSION(S)

Gynecologic cancer-related content on TikTok is of poor educational quality, and racial disparities in gynecologic cancer extend to social media. Opportunities exist to create more diverse content to support racial and cultural experiences in gynecologic cancer treatment.

Insights Into Patients Questions Over Bunion Treatments: A Google Study

Background

Approximately 1 in 4 adults will develop hallux valgus (HV). Up to 80% of adult Internet users reference online sources for health-related information. Overall, with the high prevalence of HV combined with the numerous treatment options, we believe patients are likely turning to Internet search engines for questions relevant to HV. Using Google's people also ask (PAA) or frequently asked questions (FAQs) feature, we sought to classify these questions, categorize the sources, as well as assess their levels of quality and transparency.

Methods

On October 9, 2022, we searched Google using these 4 phrases: "hallux valgus treatment," "hallux valgus surgery," "bunion treatment," and "bunion surgery." The FAQs were classified in accordance with the Rothwell Classification schema and each source was categorized. Lastly, transparency and quality of the sources' information were evaluated with the Journal of the American Medical Association's (JAMA) Benchmark tool and Brief DISCERN, respectively.

Results

Once duplicates and FAQs unrelated to HV were removed, our search returned 299 unique FAQs. The most common question in our sample was related to the evaluation of treatment options (79/299, 26.4%). The most common source type was medical practices (158/299, 52.8%). Nearly two-thirds of the answer sources (184/299; 61.5%) were lacking in transparency. One-way analysis of variance revealed a significant difference in mean Brief DISCERN scores among the 5 source types, F(4) = 54.49 (P < .001), with medical practices averaging the worst score (12.1/30).

Conclusion

Patients seeking online information concerning treatment options for HV search for questions pertaining to the evaluation of treatment options. The source type encountered most by patients is medical practices; these were found to have both poor transparency and poor quality. Publishing basic information such as the date of publication, authors or reviewers, and references would greatly improve the transparency and quality of online information regarding HV treatment.

Level of Evidence

Level V, mechanism-based reasoning.

A quality analysis of thyroid cancer videos available on TikTok

Background

Thyroid cancer (TC) is becoming an increasing public health problem worldwide. TikTok is a global video-sharing social media app, which can be a source of information on TC. However, the information quality of these videos remains unknown.

Methods

A search of TikTok was performed with the term ""and "" ("thyroid cancer" and "thyroid neoplasm" in Chinese). Videos included were independently assessed using six predefined questions for content scores and DISCERN scale for information quality, while the video popularity index (VPI) was also calculated. A correlation analysis was performed among duration, presence of animation, VPI, DISCERN scores, and content scores.

Results

A total of 56 videos were finally included, of which 49 were uploaded by physicians, 4 by health organizations, and 3 by hospitals. While 43 were real content videos, 13 were animated. The mean of DISCERN score and content score was 3.44 ± 0.72 and 5.19 ± 0.95, respectively. Good consistency was found between the two reviewers in terms of both DISCERN scores and content scores. The video duration and presence of animation were positively correlated with VPI, while DISCERN scores and content scores were not correlated with VPI.

Conclusion

The overall quality of TC-related videos was satisfactory, although the quality varies greatly depending on the type of source. Patients should be cautious when using TikTok as a source of TC-related information.

A comparative study of strategies for identifying credible sources of mental health information online: Can clinical services deliver a youth-specific internet prescription?

BACKGROUND

Adolescents and young adults undertake internet searches before and after attending mental health consultations, but they express concerns about how they can identify reliable online mental health information (OMHI). This action research project examines how clinicians might direct young people towards credible OMHI by analysing which search strategies are most helpful for developing an 'internet prescription' targeted at a youth audience.

METHODS

Recently qualified doctors undertook searches for OMHI about the diagnosis and treatment of seven mental health conditions. Rankings of websites in search results were compared with reliability and quality ratings derived from established evaluation instruments [the Brief DISCERN and the 12-item Health on the Net (HoN) tool].

RESULTS

Of 140 websites identified through seven Google searches, only 15% of those evaluated were categorized as high quality. About 17% of websites were selling a treatment or service and about 25% were regarded as untrustworthy. Also, higher rankings in the search results were not indicative of better quality OMHI (Wilcoxon signed rank test: Z = -5.28; p < .001). Although the HoN and DISCERN tools provided useful insights into the credibility of OMHI, investigators did not think either instrument could be recommended for independent use by youth.

CONCLUSIONS

Our findings suggest that clinical services should take responsibility for identifying a range of credible sources of OMHI. The next step in translating the research strategy into clinical practice involves collaboration with young people to select those platforms most orientated to their needs and to develop rating instruments that enable youth to evaluate OMHI.

Research to Clinical Practice-Youth seeking mental health information online and its impact on the first steps in the patient journey

BACKGROUND

Online searches about anxiety and depression are recorded every 3-5 s. As such, information and communication technologies (ICT) have enormous potential to enable or impair help-seeking and patient-professional interactions. Youth studies indicate that ICT searches are undertaken before initial mental health consultations, but no publications have considered how this online activity affects the first steps of the patient journey in youth mental health settings.

METHODS

State-of-the-art review using an iterative, evidence mapping approach to identify key literature and expert consensus to synthesize and prioritise clinical and research issues.

RESULTS

Adolescents and young adults are more likely to seek health advice via online search engines or social media platforms than from a health professional. Young people not only search user-generated content and social media to obtain advice and support from online communities but increasingly contribute personal information online.

CONCLUSIONS

A major clinical challenge is to raise professional awareness of the likely impact of this activity on mental health consultations. Potential strategies range from modifying the structure of clinical consultations to ensure young people are able to disclose ICT activities related to mental health, through to the development and implementation of 'internet prescriptions' and a youth-focused 'toolkit'.

Assessing the Accuracy, Quality, and Readability of Information Related to the Surgical Management of Benign Prostatic Hyperplasia

Objectives: To assess the accuracy, quality, and readability of online educational health information in English related to the most common benign prostatic hyperplasia (BPH) guideline-approved surgical treatments. Methods: The terms "benign prostatic hyperplasia," "BPH," and all eight guideline-approved treatment modalities studied, were searched to retrieve the first five relevant websites and first two paid advertised websites related to the surgical treatment options for BPH. These modalities included transurethral resection of the prostate (TURP), GreenLight photovaporization, endoscopic enucleation of the prostate, Rezum, Urolift, Aquablation, open simple prostatectomy, and robotic simple prostatectomy (RSP). All relevant websites were assessed for their accuracy, quality, and readability using standardized scoring systems. Results: The mean accuracy score for each of the treatment modalities were all indicative of good accuracy, with 76%-99% of the information presented as being accurate. The median quality score was statistically different across the eight treatment modalities (p = 0.015). The median readability grade level was statistically different across the eight treatment modalities (p = 0.009). Websites that described TURP (median readability grade level, 9.00 [interquartile range (IQR) 8.00-10.80]) were significantly easier to read than those related to RSP (median readability grade level, 14.35 [IQR, 11.08-16.50]) (p = 0.011). No other statistically significant differences were found within the other treatment modality websites. Conclusions: The majority of websites retrieved were found to be of high accuracy, good quality, and poor readability. Additionally, it was found that none of the retrieved websites included descriptions for all the other included treatment modalities. Given these findings, the authors recommend the development of centralized resources with all guideline-approved treatment modalities and accurate, readable, and high-quality information related to the surgical treatment of BPH.

Development and validation of a tool for evaluating YouTube-based medical videos

Background/aims

Today, one of the ways to access medical information is the internet. Our objective was to develop a measurement tool to assess the quality of online medical videos.

Methods

Online videos covering a variety of subjects (COVID-19, low back pain, weight loss, hypertension, cancer, chest pain, vaccination, asthma, allergy, and cataracts) were evaluated using our Medical Quality Video Evaluation Tool (MQ-VET) by 25 medical and 25 non-medical professionals. Exploratory factor analysis, Cronbach’s alpha, and correlation coefficients were used to assess the validity and reliability of the MQ-VET.

Results

The final MQ-VET consisted of 15 items and four sections. The Cronbach’s alpha reliability coefficient for the full MQ-VET was 0.72, and the internal consistency for all factors was good (between 0.73 and 0.81). The correlation between the DISCERN questionnaire scores and MQ-VET scores was significant.

Conclusion

Collectively, our findings indicated that the MQ-VET is a valid and reliable tool that will help to standardize future evaluations of online medical videos.

Documentary quality versus veracity of information of the websites on syphilis and gonorrhea

Objective: Analyze the possible relation between the documentary quality (DQ) versus the veracity of information (VI) on the syphilis and gonorrhea web pages. Methods: Descriptive cross-sectional study of websites containing information about syphilis and gonorrhea, by accessing this population through a Google. The quality was studied by using 8 variables belonging to DQ and 7 variables of the VI. Results: A total of 440 active websites mainly belonging to mass media and private entities was assessed. The fulfillment of DQ gave the following results: Mean 3.46 ± 0.07, median 4 and range from 0 to 7. The VI result was: median 4.07 ± 0.09, median 4 and range from 0 to 7. According to the search athwart syphilis or gonorrhea, the contrast of the median of the two indicators was: 3.55 vs 3.37 p = 0.181 and 4.14 vs 4.00 p = 0.442. No correlation was verified amid the data of DQ and VI (R = 0.04); p = 0.368. Similarly, no significance was observed when segregating data by disease, in the case of syphilis R = -0.03; p = 0.625 and on gonorrhea R = 0.12; p = 0.064. Conclusions: The DQ and VI bestowed low outcomes, which implies poor quality of syphilis and gonorrhea websites. According to infection (syphilis or gonorrhea), there were no meaningful differences amid the median values of the two indicators. Being acquainted with the authorship and affiliation of a website and the fact that it is tied to a prestigious web may be a factor to be deemed when predicting the VI of a website. The correlation amid the two indicators did not demonstrate an association, thus, knowing the DQ does not imply having the security of an adequate VI.

The Quality and Content of Internet-Based Information on Orthopaedic Sports Medicine Requires Improvement: A Systematic Review

Purpose

To evaluate the quality and content of internet-based information available for some of the most common orthopaedic sports medicine terms.

Methods

A search of the PubMed, Embase, and Cochrane databases following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines was performed. All English-language literature published from 2010 to 2020 discussing information quality pertaining to orthopaedic sports medicine terms was included. Outcomes included the search engines used, number and type of websites evaluated, platform, and quality scoring metrics. Descriptive statistics are presented.

Results

This review includes 21 studies. Of these, 3 evaluated both the upper and lower extremity. Twelve focused on either the upper or lower extremity, most commonly rotator cuff tears (3 of 12) and/or anterior cruciate ligament pathologies (7 of 12). The most common engines were Google (18 of 21), Bing (16 of 21), Yahoo (16 of 21), YouTube (3 of 21), Ask (3 of 21), and AOL (2 of 21). The average number of media files assessed per study was 87 ± 55. Website quality was assessed with DISCERN (7 of 21), Flesch-Kincaid (9 of 21), Health on the Net (7 of 21), and/or Journal of the American Medical Association Benchmark (7 of 21) scores. YouTube was evaluated with Journal of the American Medical Association Benchmark scores (1.74 ± 1.00). Image quality was reported in 2 studies and varied with search terminology.

Conclusions

The results of this systematic review suggest that physicians should improve the quality of online information and encourage patients to access credible sources when conducting their own research.

Clinical Relevance

Doctors can and should play an active role in closing the gap between the level of health literacy of their patients and that of most common online resources.

Online educational materials for appendectomy patients have good quality but poor readability

BACKGROUND

Guidelines recommend patient health-related information be written at or below the sixth-grade level. This study evaluates the readability level and quality of online appendectomy patient education materials.

METHODS

Webpages were evaluated using seven readability formulae: Flesh-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Automated Readability Index (ARI), Simple Measure of Gobbledygook (SMOG), Flesch Reading Ease (FRE), and New Dale-Chall (NDC). Two evaluators assessed quality using the Brief DISCERN tool.

RESULTS

Thirty seven webpages were analyzed. The mean readability scores were: FKGL = 9.11, GFI = 11.82, CLI = 10.84, ARI = 7.99, SMOG = 11.88, FRE = 51.17, and NDC = 5.48. 6 of the 7 readability formulae indicate that the materials were written at too high a level. The average Brief DISCERN score was 17.81, indicating good quality.

CONCLUSIONS

Readability levels for online appendectomy patient education materials are higher than recommended but are of good quality. Authors of such materials should not only provide good quality information but also ensure readability.

Compulsive Health-Related Internet Use and Cyberchondria

BACKGROUND

Cyberchondria denotes excessive and repeated online health-related searches associated with an increase in health anxiety. Such searches persist in those with cyberchondria, despite the negative consequences, resembling a pattern of compulsive Internet use.

OBJECTIVES

The aim of the present study was to assess compulsive health-related Internet use in relation to cyberchondria while controlling for related variables.

METHOD

Adult participants (N = 749) were recruited from an online platform. They completed questionnaires assessing the severity of cyberchondria (via the Cyberchondria Severity Scale [CSS]), compulsive Internet use adapted for online health-related seeking (via the adapted Compulsive Internet Use Scale [CIUS]), and levels of intolerance of uncertainty and anxiety, as well as depressive, somatic, and obsessive-compulsive symptoms. A logistic regression analysis was carried out to identify predictors of scores above a cutoff value on the CIUS, indicating compulsive health-related Internet use.

RESULTS

The regression output showed that only the CSS total score and sex made a unique, statistically significant contribution to the model, leading to the correct classification of 78.6% of the cases. Of the CSS subscales, compulsion and distress were the most strongly associated with compulsive health-related Internet use.

CONCLUSIONS

The finding that the adapted CIUS scores are associated with cyberchondria indicates that cyberchondria has a compulsive component, at least in terms of health-related Internet use. It also suggests that compulsive health-related Internet use persists despite the distress associated with this activity. Males may engage in cyberchondria more compulsively than females. These findings have implications for research and clinical practice.

Examining the Public's Most Frequently Asked Questions Regarding COVID-19 Vaccines Using Search Engine Analytics in the United States: Observational Study

BACKGROUND

The emergency authorization of COVID-19 vaccines has offered the first means of long-term protection against COVID-19-related illness since the pandemic began. It is important for health care professionals to understand commonly held COVID-19 vaccine concerns and to be equipped with quality information that can be used to assist in medical decision-making.

OBJECTIVE

Using Google's RankBrain machine learning algorithm, we sought to characterize the content of the most frequently asked questions (FAQs) about COVID-19 vaccines evidenced by internet searches. Secondarily, we sought to examine the information transparency and quality of sources used by Google to answer FAQs on COVID-19 vaccines.

METHODS

We searched COVID-19 vaccine terms on Google and used the "People also ask" box to obtain FAQs generated by Google's machine learning algorithms. FAQs are assigned an "answer" source by Google. We extracted FAQs and answer sources related to COVID-19 vaccines. We used the Rothwell Classification of Questions to categorize questions on the basis of content. We classified answer sources as either academic, commercial, government, media outlet, or medical practice. We used the Journal of the American Medical Association's (JAMA's) benchmark criteria to assess information transparency and Brief DISCERN to assess information quality for answer sources. FAQ and answer source type frequencies were calculated. Chi-square tests were used to determine associations between information transparency by source type. One-way analysis of variance was used to assess differences in mean Brief DISCERN scores by source type.

RESULTS

Our search yielded 28 unique FAQs about COVID-19 vaccines. Most COVID-19 vaccine-related FAQs were seeking factual information (22/28, 78.6%), specifically about safety and efficacy (9/22, 40.9%). The most common source type was media outlets (12/28, 42.9%), followed by government sources (11/28, 39.3%). Nineteen sources met 3 or more JAMA benchmark criteria with government sources as the majority (10/19, 52.6%). JAMA benchmark criteria performance did not significantly differ among source types (χ 2 4=7.40; P=.12). One-way analysis of variance revealed a significant difference in mean Brief DISCERN scores by source type (F 4,23=10.27; P<.001).

CONCLUSIONS

The most frequently asked COVID-19 vaccine-related questions pertained to vaccine safety and efficacy. We found that government sources provided the most transparent and highest-quality web-based COVID-19 vaccine-related information. Recognizing common questions and concerns about COVID-19 vaccines may assist in improving vaccination efforts.

Plausibility of Using a Checklist With YouTube to Facilitate the Discovery of Acute Low Back Pain Self-Management Content: Exploratory Study

Background

Access to guideline-consistent effective care for acute low back pain (ALBP) is generally regarded as limited. Researchers have recognized the potential of YouTube as a clinical and patient education resource that may improve access to appropriate care. However, the heterogeneity of evaluation approaches and variable quality of health information have generally limited the potential of YouTube as a self-management intervention.

Objective

This study aims to increase the understanding of ALBP content available on YouTube in 2020 and to establish the plausibility of using a simple checklist to facilitate the discovery of YouTube content consistent with current guidelines. We examined the following 4 research questions: how was the data set defined, what are the metadata characteristics of the videos in the data set, what is the information quality of ALBP YouTube videos, and what are the characteristics of the YouTube data set based on an ALBP self-management checklist?

Methods

This was an exploratory, qualitative infodemiology study. We identified videos in our data set through YouTube search based on popular ALBP-relevant search terms identified through Google Trends for YouTube. We accessed YouTube metadata using the YouTube data tools developed by the University of Amsterdam. We used a modified Brief DISCERN checklist to examine the information quality. We developed a checklist based on the 2018 Lancet Low Back Pain guidelines to examine self-management content.

Results

We analyzed a data set of 202 YouTube videos authored by chiropractors, physicians, physiotherapists, and instructors of yoga and other disciplines. We identified clear differences in the ALBP videos in our data set based on the authors’ disciplines. We found that the videos authored by each discipline strongly featured a specific intervention domain, that is, education, treatment, or exercise. We also found that videos authored by physicians were consistently coded with the highest ALBP self-management content scores than all other disciplines.

Conclusions

The results returned by YouTube in response to a search for back pain–related content were highly variable. We suggest that a simple checklist may facilitate the discovery of guideline-concordant ALBP self-management content on YouTube. Further research may identify the clinical contexts in which the use of an ALBP checklist with YouTube is feasible.

Quality of information on YouTube about artificial intelligence in dental radiology

OBJECTIVES

This study was designed to investigate Artificial Intelligence in Dental Radiology (AIDR) videos on YouTube in terms of popularity, content, reliability, and educational quality.

METHODS

Two researchers systematically searched about AIDR on YouTube on January 27, 2020, by using the terms "artificial intelligence in dental radiology," "machine learning in dental radiology," and "deep learning in dental radiology." The search was performed in English, and 60 videos for each keyword were assessed. Video source, content type, time since upload, duration, and number of views, likes, and dislikes were recorded. Video popularity was reported using Video Power Index (VPI). The accuracy and reliability of the source of information were measured using the adapted DISCERN score. The quality of the videos was measured using JAMAS and modified Global Quality Score (mGQS) and content via Total Concent Evaluation (TCE).

RESULTS

There was high interobserver agreement for DISCERN (intraclass correlation coefficient [ICC]: 0.975; 95% confidence interval [CI]: 0.957-0.985; P: 0.000; P < 0.05) and mGQS (ICC: 0.904; 95% CI: 0.841-0.943; P: 0.000; P < 0.05). Academic source videos had higher DISCERN, GQS, and TCE, revealing both reliability and quality. Also, positive relationship of VPI with mGQS (30.1%) (P: 0.035) and DISCERN (38.1%) (P: 0.007) is detected. The scores revealed 51.9% relationship between mGQS and DISCERN (P: 0.001); and educational quality predictor scores revealed 62.5% relationship between TCE and GQS (P: 0.000).

CONCLUSION

Despite the limited number of relevant videos, YouTube involves reliable and quality videos that can be used by dentists about learning AIDR.

AutoDiscern: rating the quality of online health information with hierarchical encoder attention-based neural networks

Background

Patients increasingly turn to search engines and online content before, or in place of, talking with a health professional. Low quality health information, which is common on the internet, presents risks to the patient in the form of misinformation and a possibly poorer relationship with their physician. To address this, the DISCERN criteria (developed at University of Oxford) are used to evaluate the quality of online health information. However, patients are unlikely to take the time to apply these criteria to the health websites they visit.

Methods

We built an automated implementation of the DISCERN instrument (Brief version) using machine learning models. We compared the performance of a traditional model (Random Forest) with that of a hierarchical encoder attention-based neural network (HEA) model using two language embeddings, BERT and BioBERT.

Results

The HEA BERT and BioBERT models achieved average F1-macro scores across all criteria of 0.75 and 0.74, respectively, outperforming the Random Forest model (average F1-macro = 0.69). Overall, the neural network based models achieved 81% and 86% average accuracy at 100% and 80% coverage, respectively, compared to 94% manual rating accuracy. The attention mechanism implemented in the HEA architectures not only provided ’model explainability’ by identifying reasonable supporting sentences for the documents fulfilling the Brief DISCERN criteria, but also boosted F1 performance by 0.05 compared to the same architecture without an attention mechanism.

Conclusions

Our research suggests that it is feasible to automate online health information quality assessment, which is an important step towards empowering patients to become informed partners in the healthcare process.

Exploring the Vast Choice of Question Prompt Lists Available to Health Consumers via Google: Environmental Scan

Background

There is increasing interest in shared decision making (SDM) in Australia. Question prompt lists (QPLs) support question asking by patients, a key part of SDM. QPLs have been studied in a variety of settings, and increasingly the internet provides a source of suggested questions for patients. Environmental scans have been shown to be useful in assessing the availability and quality of online SDM tools.

Objective

This study aimed to assess the number and readability of QPLs available to users via Google.com.au.

Methods

Our environmental scan used search terms derived from literature and reputable websites to search for QPLs available via Google.com.au. Following removal of duplicates from the 4000 URLs and 22 reputable sites, inclusion and exclusion criteria were applied to create a list of unique QPLs. A sample of 20 QPLs was further assessed for list length, proxy measures of quality such as a date of review, and evidence of doctor endorsement. Readability of the sample QPL instructions and QPLs themselves was assessed using Flesch Reading Ease and Flesch-Kincaid Grade Level scores.

Results

Our environmental scan identified 173 unique QPLs available to users. Lists ranged in length from 1 question to >200 questions. Of our sample, 50% (10/20) had a listed date of creation or update, and 60% (12/20) had evidence of authorship or source. Flesch-Kincaid Grade Level scores for instructions were higher than for the QPLs (grades 10.3 and 7.7, respectively). There was over a 1 grade difference between QPLs from reputable sites compared with other sites (grades 4.2 and 5.4, respectively).

Conclusions

People seeking questions to ask their doctor using Google.com.au encounter a vast number of question lists that they can use to prepare for consultations with their doctors. Markers of the quality or usefulness of various types of online QPLs, either surrogate or direct, have not yet been established, which makes it difficult to assess the value of the abundance of lists. Doctor endorsement of question asking has previously been shown to be an important factor in the effectiveness of QPLs, but information regarding this is not readily available online. Whether these diverse QPLs are endorsed by medical practitioners warrants further investigation.

Assessment of quality and readability of internet-based health information related to commonly prescribed angiotensin receptor blockers

Introduction

Hypertension is a global public health burden. Angiotensin receptor blockers (ARBs) have proven efficacy in the management of hypertension and related complications. The Internet has become a major source of health information for patients and healthcare professionals. The study aimed to assess the quality and readability of internet-based information related to selected Angiotensin Receptor Blockers (ARBs).

Methods

The three most widely used ARBs were identified from published literature, after which internet-based patient information was identified from the first five pages of three search engines (Google, Yahoo and Bing). Quality of identified websites were assessed using the DISCERN instrument, while readability was evaluated using the SMOG instrument and the Flesch-Kincaid readability algorithm. Final ratings were then calculated as described by the instruments developers. Further, inter-class correlation coefficients (ICC) were calculated using the Statistical Package for Social Sciences.

Results

The average overall DISCERN score in this study was 2.99 (SD±1.05). No website received an excellent rating, 15% were rated good, 66% as moderate and 19% as poor. The inter-class reliability was 0.804 for losartan and 0.695 for valsartan. The mean Flesch Reading Ease score for the websites was 48.87 (SD±16.12), mean Flesch-Kincaid Reading Grade Level was 9.29 (SD±1.98) while mean SMOG value was 11.29 (SD±1.70).

Conclusion

Overall, patient information on the reviewed ARBs websites was found to be of moderate quality and suboptimal readability. Content providers on websites should ensure that health information is of favorable quality and easy to read by patients with varying degree of health literacy.

Systematic Online Academic Resource (SOAR) Review: Renal and Genitourinary

BACKGROUND

Online resources for emergency medicine (EM) trainees and physicians have variable quality and inconsistent coverage of core topics. In this first entry of the Society for Academic Emergency Medicine Systematic Online Academic Resource (SOAR) series, we describe the application of a systematic methodology to comprehensively identify, collate, and curate online content for topic-specific modules.

METHODS

A list of module topics and related terms was generated from the American Board of Emergency Medicine's Model of the Clinical Practice of Emergency Medicine. The authors selected "renal and genitourinary" for the first module, which contained 35 terms; all MeSH headers and colloquial synonyms related to the topic and related terms were searched both within the 100 most impactful online educational websites per the Social Media Index and the FOAMsearch.net search engine. Duplicate entries, journal articles, images, and archives were excluded. The quality of each article was rated using the revised METRIQ (rMETRIQ) score.

RESULTS

The search yielded 13,058 online resources. After 12,717 items were excluded, 341 underwent quality assessment. All renal/genitourinary topics were covered by at least one resource. The median rMETRIQ score was 11 of 21 (interquartile range = 8-14). Calculus of urinary tract was most prominently featured with 60 posts. Thirty-four posts (10% of full-text screened FOAM articles) covering 12 core topics were identified as high quality (rMETRIQ ≥ 16).

CONCLUSIONS

We demonstrated the feasibility of systematically identifying and curating FOAM resources for a specific EM topic and identified an overrepresentation of some subtopics. This curated list of resources may guide trainees, teacher recommendations, and resource producers. Further entries in the series will address other topics relevant to EM.

Promotion and marketing of bioidentical hormone therapy on the internet: a content analysis of websites

OBJECTIVE

To evaluate the quality of information presented and claims made on websites offering bioidentical hormone therapy (BHT) products or services.

METHODS

A quantitative content analysis was completed on 100 websites promoting or offering BHT products or services. Websites were identified through Google search engine from September to October 2013. Search terms included "bioidentical hormone therapy" or "bioidentical progesterone," accompanied by "purchase or buy," "service," or "doctors." The Brief DISCERN instrument was used to determine the quality of the health information.

RESULTS

Websites were from Canada (59%), United States (38%), and other countries (3%). Almost half of the websites originated from medical clinics (47%), and healthcare professionals offering BHT services included physicians (50%), pharmacists (19%), and naturopaths (16%). Majority of websites promoted BHT as custom-compounded formulations (62%), with only 27% indicating that BHT is also commercially available. Websites overall claimed that BHT had less risk compared with conventional hormone therapy (62%). BHT was described as having less breast cancer risk (40%), whereas over a quarter of websites described BHT as "protective" for breast cancer. Websites mainly targeted women (99%), with males mentioned in 62% of websites. Product descriptors used to promote BHT included individualization (77%), natural (70%), hormone imbalance (56%), and antiaging (50%). The mean Brief DISCERN score was 15, indicating lower quality of information.

CONCLUSIONS

Claims made about BHT on the internet are misleading and not consistent with current professional organizations' recommendations. Understanding how BHT may be promoted on the internet can help healthcare professionals when educating patients.

Toward automated assessment of health Web page quality using the DISCERN instrument

Background

As the Internet becomes the number one destination for obtaining health-related information, there is an increasing need to identify health Web pages that convey an accurate and current view of medical knowledge. In response, the research community has created multicriteria instruments for reliably assessing online medical information quality. One such instrument is DISCERN, which measures health Web page quality by assessing an array of features. In order to scale up use of the instrument, there is interest in automating the quality evaluation process by building machine learning (ML)-based DISCERN Web page classifiers.

Objective

The paper addresses 2 key issues that are essential before constructing automated DISCERN classifiers: (1) generation of a robust DISCERN training corpus useful for training classification algorithms, and (2) assessment of the usefulness of the current DISCERN scoring schema as a metric for evaluating the performance of these algorithms.

Methods

Using DISCERN, 272 Web pages discussing treatment options in breast cancer, arthritis, and depression were evaluated and rated by trained coders. First, different consensus models were compared to obtain a robust aggregated rating among the coders, suitable for a DISCERN ML training corpus. Second, a new DISCERN scoring criterion was proposed (features-based score) as an ML performance metric that is more reflective of the score distribution across different DISCERN quality criteria.

Results

First, we found that a probabilistic consensus model applied to the DISCERN instrument was robust against noise (random ratings) and superior to other approaches for building a training corpus. Second, we found that the established DISCERN scoring schema (overall score) is ill-suited to measure ML performance for automated classifiers.

Conclusion

Use of a probabilistic consensus model is advantageous for building a training corpus for the DISCERN instrument, and use of a features-based score is an appropriate ML metric for automated DISCERN classifiers.

Availability

The code for the probabilistic consensus model is available at https://bitbucket.org/A_2/em_dawid/ .

Web-Based Information on the Treatment of Tobacco Dependence for Oral Health Professionals: Analysis of English-Written Websites

BACKGROUND

Studies have been conducted on the content and quality of Web-based information for patients who are interested in smoking cessation advice and for health care practitioners regarding the content of e-learning programs about tobacco cessation. However, to the best of our knowledge, there is no such information about the quality of Web-based learning resources regarding smoking cessation dedicated to oral health professionals.

OBJECTIVE

The aim of this study was to identify and evaluate the quality of the content of webpages providing information about smoking cessation for oral health care professionals.

METHODS

Websites were identified using Google and Health on Net (HON) search engines using the terms: smoking cessation OR quit smoking OR stop smoking OR 3As OR 5As OR tobacco counselling AND dentistry OR dental clinic OR dentist OR dental hygienist OR oral health professionals. The first 100 consecutive results of the 2 search engines were considered for the study. Quality assessment was rated using the DISCERN questionnaire, the Journal of the American Medical Association (JAMA) benchmarks, and the HON seal. In addition, smoking cessation content on each site was assessed using an abbreviated version of the Smoke Treatment Scale (STS-C) and the Smoking Treatment Scale-Rating (STS-R). To assess legibility of the selected websites, the Flesch Reading Ease (FRES) and the Flesch-Kinkaid Reading Grade Level (FKRGL) were used. Websites were also classified into multimedia and nonmultimedia and friendly and nonfriendly usability.

RESULTS

Of the first 200 sites selected (100 of Google and 100 of HON), only 11 met the inclusion criteria and mainly belonged to governmental institutions (n=8), with the others being prepared by Professional Associations (n=2) and nonprofit organizations (n=1). Only 3 were exclusively dedicated to smoking cessation. The average score obtained with the DISCERN was 3.0, and the average score in the FKRGL and FRES was 13.31 (standard deviation, SD 3.34) and 40.73 (SD 15.46), respectively. Of the 11 websites evaluated, none achieved all the four JAMA benchmarks. The mean score of STS-R among all the websites was 2.81 (SD 0.95) out of 5. A significant strong positive correlation was obtained between the DISCERN mean values and the STS-R (R=.89, P=.01).

CONCLUSIONS

The mean quality of webpages with information for oral health care professionals about smoking cessation is low and displayed a high heterogeneity. These webpages are also difficult to read and often lack multimedia resources, which further limits their usefulness.

Online self-help forums on cannabis: A content assessment

OBJECTIVE

To investigate online self-help forums related to cannabis users who were searching for help on the Internet.

METHODS

We analyzed the content of 717 postings by 328 users in three online forums in terms of fields of interest and self-help mechanisms. Only English-language forums that were free of charge and without registration were investigated.

RESULTS

The main self-help mechanisms were disclosure and symptoms, with relatively few posts concerning legal issues and social perceptions. The forums differed significantly in all fields of interest and self-help mechanisms except for social network and financial and vocational issues. Highly involved users more commonly posted on topics related to diagnosis, etiology/research, and provision of information and less commonly on those related to gratitude. Correlation analysis showed a moderate negative correlation between emotional support and illness-related aspects and between emotional support and exchange of information.

CONCLUSIONS

Cannabis forums share similarities with other mental health forums. Posts differ according to user involvement and the specific orientation of the forum.

PRACTICE IMPLICATIONS

The Internet offers a viable source of self-help and social support for cannabis users, which has potential clinical implications in terms of referring clients to specific forums.

The poor quality and reliability of information on periacetabular osteotomy on the internet in Japan

Although many patients use the internet to access health-related information, the quality and the reliability of the information is highly inconsistent. Periacetabular osteotomy (PAO) is one of the surgical procedures for hip dysplasia. However, medical information on PAO is limited on the internet. This study aims to evaluate the quality and reliability of information available on PAO on the internet in Japan. A web search was conducted on two search engines for the following terms: “hip osteotomy,” “pelvic osteotomy,” and “osteotomy for hip preservation” in Japanese. In total, we found 120 websites. To determine the quality and reliability of information on each website, we used the Health on the Net Foundation (HON) score, the Brief DISCERN score, and an osteotomy-specific content (OSC) score. After eliminating duplicate websites, we reviewed 49 unique websites. Only three websites (6.1%) had good reliability, as indicated by their HON scores. Twelve websites (24.4%) had good-quality information, as measured by their Brief DISCERN scores. As evaluated by their OSC scores, physician websites were found to be biased toward etiology and surgical indication and did not provide information on the complications of procedures. Non-physician websites were generally insufficient. The information about PAO on the internet is, therefore, unreliable and of poor-quality for Japanese patients.

Evaluating asthma websites using the Brief DISCERN instrument

PURPOSE

The primary purpose of this study was to examine the quality of sponsored and unsponsored asthma websites using the Brief DISCERN instrument and to evaluate whether the Health On the Net Code of Conduct (HONcode) logo was present, thereby indicating that the site met the criteria. The Internet is an important source of health information for patients and their families. The primary purpose of this study was to examine the quality of sponsored and unsupported asthma websites. A secondary aim was to determine the readability and reading ease of the materials for each website along with the grade level.

METHODS

We queried seven Internet search engines using the keyword "asthma." The websites were evaluated using the six-item Brief DISCERN instrument and by ascertaining whether the HONcode quality label was present. The websites were also evaluated for readability employing Flesch-Kincaid grade level and Flesch reading ease tools using Microsoft Office Word 2013 software.

RESULTS

A total of 22 unique websites were included in the study. Approximately 68% of the websites reviewed had a Brief DISCERN cutoff score of ≥16. The overall Brief DISCERN scores ranged from 6 to 30, and the mean score was 17.32 (SD =6.71). The Flesch-Kincaid grade level scores ranged from 2.9 to 15.4, and the average reading grade score was 9.49 (SD =2.7). The Flesch reading ease scores ranged from 17 to 82.7, with a mean reading ease score of 53.57 (SD =15.03). Sites with a HONcode quality label had significantly higher Brief DISCERN scores than those without one (t=2.3795; df=20; p=0.02).

CONCLUSION

Brief DISCERN scores revealed that there is quality asthma information for children and their families available on the Internet. The grade level ranged between 2.9 and 15.4 among the websites. However, the mean grade level scores were 9.3-9.89, which is high for the average consumer. Access to accurate information via the Internet, with appropriate readability, may enable pediatric asthma patients and their caregivers to better control and manage asthma.

The Quality of Online Health-Related Information – an Emergent Consumer Health Issue

The Internet has become one of the main means of communication used by people who search for health-related information. The quality of online health-related information affects the users’ knowledge, their attitude, and their risk or health behaviour in complex ways and influences a substantial number of users in their decisions regarding diagnostic and treatment procedures.

The aim of this review is to explore the benefits and risks associated with using the Internet as a source of health-related information; the relationship between the quality of the health-related information available on the Internet and the potential risks; the multiple conceptual components of the quality of health-related information; the evaluation criteria for quality health-related information; and the main approaches and initiatives that have been implemented worldwide to help improve users’ access to high-quality health-related information.

Psychometric properties of the 7-item game addiction scale among french and German speaking adults

BACKGROUND

The 7-item Game Addiction Scale (GAS) is a used to screen for addictive game use. Both cross cross-linguistic validation and validation in French and German is needed in adult samples. The objective of the study is to assess the factorial structure of the French and German versions of the GAS among adults.

METHODS

Two samples of men from French (N = 3318) and German (N = 2665) language areas of Switzerland were assessed with the GAS, the Major Depression Inventory (MDI), the Brief Sensation Seeking Scale, and the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ-50-cc). They were also assessed for cannabis and alcohol use.

RESULTS

The internal consistency of the scale was satisfactory (Cronbach α = 0.85). A one-factor solution was found in both samples. Small and positive associations were found between GAS scores and the MDI, as well as the Neuroticism-Anxiety and Aggression-Hostility subscales of the ZKPQ-50-cc. A small negative association was found with the ZKPQ-50-cc Sociability subscale.

CONCLUSION

The GAS, in its French and German versions, is appropriate for the assessment of game addiction among adults.

Readability and Understandability of Online Vocal Cord Paralysis Materials

Objective

Patients use several online resources to learn about vocal cord paralysis (VCP). The objective of this study was to assess the readability and understandability of online VCP patient education materials (PEMs), with readability assessments and the Patient Education Materials Evaluation Tool (PEMAT), respectively. The relationship between readability and understandability was then analyzed.

Study Design and Setting

Descriptive and correlational design.

Method

Online PEMs were identified by performing a Google search with the term “vocal cord paralysis.” After scientific webpages, news articles, and information for medical professionals were excluded, 29 articles from the first 50 search results were considered. Readability analysis was performed with 6 formulas. Four individuals with different educational backgrounds conducted understandability analysis with the PEMAT. Fleiss’s Kappa interrater reliability analysis determined consistency among raters. Correlation between readability and understandability was determined with Pearson’s correlation test.

Results

The reading level of the reviewed articles ranged from grades 9 to 17. Understandability ranged from 29% to 82%. Correlation analysis demonstrated a strong negative correlation between materials’ readability and understandability ( r = −0.462, P < .05).

Conclusion

Online PEMs pertaining to VCP are written above the recommended reading levels. Overall, materials written at lower grade levels are more understandable. However, articles of identical grade levels had varying levels of understandability. The PEMAT may provide a more critical evaluation of the quality of a PEM when compared with readability formulas. Both readability and understandability should be used to evaluate PEMs.

The Source and Credibility of Colorectal Cancer Information on Twitter

Despite the rapid penetration of social media in modern life, there has been limited research conducted on whether social media serves as a credible source of health information. In this study, we propose to identify colorectal cancer information on Twitter and assess its informational credibility. We collected Twitter messages containing colorectal cancer-related keywords, over a 3-month period. A review of sample tweets yielded content and user categorization schemes. The results of the sample analysis were applied to classify all collected tweets and users, using a machine learning technique. The credibility of the information in the sampled tweets was evaluated. A total of 76,119 tweets were analyzed. Individual users authored the majority of tweets (n = 68,982, 90.6%). They mostly tweeted about news articles/research (n = 16,761, 22.0%) and risk/prevention (n = 14,767, 19.4%). Medical professional users generated only 2.0% of total tweets (n = 1509), and medical institutions rarely tweeted (n = 417, 0.6%). Organizations tended to tweet more about information than did individuals (85.2% vs 63.1%; P < 0.001). Credibility analysis of medically relevant sample tweets revealed that most were medically correct (n = 1763, 84.5%). Among those, more frequently retweeted tweets contained more medically correct information than randomly selected tweets (90.7% vs 83.2%; P < 0.01). Our results demonstrate an interest in and an engagement with colorectal cancer information from a large number and variety of users. Coupled with the Internet's potential to increase social support, Twitter may contribute to enhancing public health and empowering users, when used with proper caution.

Quality of Smartphone Apps Related to Alcohol Use Disorder

BACKGROUND

Apps for smartphones are opening an important range of opportunities for improving the care of people with alcohol use disorders (AUDs).

AIMS

This study aimed to evaluate the quality of English language apps for AUDs and to compare paid and free apps. The keywords 'alcohol', 'alcohol addiction', 'alcohol help' and 'stop drinking' were entered into the iTunes Store search engine. Apps were evaluated using a standardized assessment designed to rate the quality of apps in terms of accountability, interactivity, self-help score and evidence-based content. The Brief DISCERN score and the criteria of the 'Health on the Net' label were also used as content quality indicators.

RESULTS AND CONCLUSIONS

Of the 137 unique apps identified, 52 met the inclusion criteria. Overall, the content quality and self-help scores of these AUD apps were poor. The main quality indicators were not linked to payment status. Multiple linear regressions showed that the Brief DISCERN score significantly predicted content quality. Poor content quality and self-help scores of AUD smartphone apps underline the gap between their potential promises and the overall quality of available products in stores. The quality indicators used in the present study may be used for further app developments.

Evaluation of Health on the Net seal label and DISCERN as content quality indicators for patients seeking information about thumb sucking habit

Aims:

The aim of this study was to evaluate the reliability of websites on the thumb sucking habit using DISCERN instrument and Health on the Net (HON) seal code at a single moment in time.

Subjects and Methods:

An Internet search engine (www.google.com) was used to identify websites comprising information on “thumb sucking habit.” Of over 204,000 links for thumb sucking habit, the first 100 were analyzed in detail. After excluding discussion groups, news and video feeds, and removing carbon copy sites, only 36 relevant websites remained, which were then assessed using the DISCERN instrument and HON seal code. Using the 16 questions of DISCERN for assessing the reliability and quality of the consumer information which were scored from 1 to 5, an appropriate index of the quality of the information was generated. All the assessed websites were also checked for presence or absence of HON seal code.

Results:

The maximum score attainable for an outstanding website is 80. Of the 36 websites that were scored the highest score obtained by one of the websites according to the DISCERN tool was 55 of 80, and the lowest score achieved was 16 of 80. The websites achieving the maximum and minimum score were children.webmd.com and thebehaviorsolution.com, respectively. The HON seal was displayed only in three websites, which were medicinenet.com, righthealth.com, and children.webmd.com.

Conclusions:

By directing patients to validated websites on the thumb sucking habit, clinicians can ensure patients find appropriate information.