Understanding online health information: Evaluation, tools, and strategies

Development and evaluation of health education resources for culturally and linguistically diverse populations: a systematic review

People from culturally and linguistically diverse (CaLD) backgrounds in Australia generally experience poorer health outcomes, explained in part by low levels of health literacy. We conducted a systematic review to examine the development and evaluation of health education resources designed for CaLD populations. Five electronic databases were searched for English language, peer-reviewed studies published between 1980 and 2020. Thirty-four studies met the inclusion criteria. Twenty-four different health education resources were described and broadly categorized into four types: media campaigns (n = 10), text-based materials (n = 5), films (n = 8) and radio (n = 1). Studies were assessed against domains adapted from a health literacy guideline incorporating: need, collaboration, audience, health literacy, theory, test and process and impact evaluation. All but one study met the majority of the domains. All studies reported positive evaluation outcomes; this may be due to studies involving community early in resource design and including health literacy considerations in their design. Reporting resource design and evaluation against standard practice controls is recommended to build a more robust evidence base for developing effective health education resources for use by audiences from CaLD backgrounds.

Evaluation of the Accuracy, Credibility, and Readability of Statin-Related Websites: Cross-Sectional Study

BACKGROUND

Cardiovascular disease (CVD) represents the greatest burden of mortality worldwide, and statins are the most commonly prescribed drug in its management. A wealth of information pertaining to statins and their side effects is on the internet; however, to date, no assessment of the accuracy, credibility, and readability of this information has been undertaken.

OBJECTIVE

This study aimed to evaluate the quality (accuracy, credibility, and readability) of websites likely to be visited by the general public undertaking a Google search of the side effects and use of statin medications.

METHODS

Following a Google web search, we reviewed the top 20 consumer-focused websites with statin information. Website accuracy, credibility, and readability were assessed based on website category (commercial, not-for-profit, and media), website rank, and the presence or absence of the Health on the Net Code of Conduct (HONcode) seal. Accuracy and credibility were assessed following the development of checklists (with 20 and 13 items, respectively). Readability was assessed using the Simple Measure of Gobbledegook scores.

RESULTS

Overall, the accuracy score was low (mean 14.35 out of 20). While side effects were comprehensively covered by 18 websites, there was little information about statin use in primary and secondary prevention. None of the websites met all criteria on the credibility checklist (mean 7.8 out of 13). The median Simple Measure of Gobbledegook score was 9.65 (IQR 8.825-10.85), with none of the websites meeting the recommended reading grade of 6, even the media websites. A website bearing the HONcode seal did not mean that the website was more comprehensive or readable.

CONCLUSIONS

The quality of statin-related websites tended to be poor. Although the information contained was accurate, it was not comprehensive and was presented at a reading level that was too difficult for an average reader to fully comprehend. As such, consumers risk being uninformed about this pharmacotherapy.

Evaluation of the Quality and Readability of Online Information about Alzheimer's Disease in China

Background

With the increasing popularity of the internet, a growing number of patients and their companions are actively seeking health-related information online.

Objective

The aim of this study was to assess the quality and readability of online information about Alzheimer's disease (AD) in China.

Methods

A total of 263 qualified AD-related web pages from different businesses, governments, and hospitals were obtained. The quality of the web pages was assessed using the DISCERN tool, and the readability of the web pages was assessed using a readability measurement website suitable for the Chinese language. The differences in readability and quality between different types of web pages were investigated, and the correlation between quality and readability was analyzed.

Results

The mean overall DISCERN score was 40.93±7.5. The government group scored significantly higher than the commercial and hospital groups. The mean readability score was 12.74±1.27, and the commercial group had the lowest readability score. There was a positive correlation between DISCERN scores and readability scores.

Conclusions

This study presents an evaluation of the quality and readability of health information pertaining to AD in China. The findings indicate that there is a need to enhance the quality and readability of web pages about AD in China. Recommendations for improvement are proposed in light of these findings.

The Effect of Mobile eHealth Education to Improve Knowledge, Skills, Self-Care, and Mobile eHealth Literacies Among Patients With Diabetes: Development and Evaluation Study

BACKGROUND

The promotion of mobile health (mHealth) and eHealth technologies as tools for managing chronic diseases, particularly diabetes mellitus, is on the rise. Nevertheless, individuals with diabetes frequently face a literacy gap that hinders their ability to fully leverage the benefits offered by these resources. Enhancing technology literacy to facilitate the adoption of mobile eHealth services poses a significant challenge in numerous countries.

OBJECTIVE

This study aims to develop an educational mobile eHealth literacy (eHL) program for patients with diabetes and to evaluate its effect on patients' outcomes.

METHODS

This study designed a mobile eHL education program comprising 2 modules specifically tailored for individuals with type 2 diabetes (T2D). These modules focused on guiding participants through the process of effectively navigating reliable health websites and utilizing diabetes-related apps. Using a pre- and posttest experimental design, the study featured an intervention group and a control group. Participants were recruited from 3 outpatient departments in hospitals, and assessments were conducted both before and after the intervention, along with a follow-up measure at the 3-month mark. The evaluation encompassed sociodemographic characteristics, computer and internet proficiency, mobile app usage, mobile eHL, and patient outcomes such as self-care behaviors and glycated hemoglobin (HbA1c) levels.

RESULTS

The analysis included a total of 132 eligible participants. Significant differences were observed in the mean scores of knowledge (P<.001) and skills (P<.001) related to computers, the web, and mobile devices at the initiation of the study and after the intervention. During the 3-month follow-up, the findings indicated a significant improvement in mobile eHL (t114=3.391, P=.001) and mHealth literacy (mHL, a subconcept of mobile eHL; t114=3.801, P<.001) within the intervention group, whereas no such improvement was observed in the control group. The chi-square values from the McNemar test underscored that individuals with uncontrolled diabetes (HbA1c≥7%) in the intervention group exhibited more improvement compared with the control group. The generalized estimating equations model unveiled a significant difference in the change of general mHL in the intervention group (β=1.91, P=.047) and self-care behavior in the control group from T0 to T2 (β=-8.21, P=.015). Despite being small, the effect sizes for mobile eHL (d=0.49) and HbA1c (d=0.33) in the intervention group were greater than those in the control group (d=0.14 and d=0.16, respectively).

CONCLUSIONS

The implementation of a mobile eHL education intervention demonstrates a positive influence on the familiarity of patients with T2D regarding health technology, leading to favorable glycemic outcomes. While additional studies are warranted for a more comprehensive understanding, this program emerges as a promising solution for enhancing patients' uptake of digital health technology.

A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: A comparative analysis between plastic surgeons and laypersons

OBJECTIVES

ChatGPT has gained significant popularity as a source of healthcare information among the general population. Evaluating the quality of chatbot responses is crucial, requiring comprehensive and qualitative analysis. This study aims to assess the answers provided by ChatGPT during hypothetical breast augmentation consultations across various categories and depths. The evaluation involves the utilization of validated tools and a comparison of scores between plastic surgeons and laypersons.

METHODS

A panel consisting of five plastic surgeons and five laypersons evaluated ChatGPT's responses to 25 questions spanning consultation, procedure, recovery, and sentiment categories. The DISCERN and PEMAT tools were employed to assess the responses, while emotional context was examined through ten specific questions. Additionally, readability was measured using the Flesch Reading Ease score. Qualitative analysis was performed to identify the overall strengths and weaknesses.

RESULTS

Plastic surgeons generally scored lower than laypersons across most domains. Scores for each evaluation domain varied by category, with the consultation category demonstrating lower scores in terms of DISCERN reliability, information quality, and DISCERN score. Plastic surgeons assigned significantly lower overall quality ratings to the procedure category compared to other question categories. They also gave lower emotion scores in the procedure category compared to laypersons. The depth of the questions did not impact the scoring.

CONCLUSIONS

Existing health information evaluation tools may not be entirely suitable for comprehensively evaluating the quality of individual responses generated by ChatGPT. Consequently, the development and implementation of appropriate evaluation tools to assess the appropriateness and quality of AI consultations are necessary.

Assessing Arabic youtube videos on herbal cancer treatment: Absence of health information quality

This study assesses the quality of the health information in Arabic YouTube videos regarding herbal cancer treatment. It also provides an overview of how the quality of video content shapes user awareness by assessing the users' engagement indicators. A simple Python tool was developed using YouTube API V3 to automate the YouTube search based on the recommendation of Google Trends. After applying inclusion and exclusion criteria, 110 YouTube videos were selected, of which 95% were uploaded by non-experts and had a total of 8,633,569 views. The analyzed videos presented more than 40 different herbals as sources of cancer treatment; for example, Ephedra, garden cress, Green tea, Ginseng, Rosemary, and Thyme. 32.7% of the videos provided information about a single herb, 41% about mixing herbals, and 26.3% provided testimonials and success stories without pointing to specific herbs. The videos were assessed by two experts using two reliable tools, DISCERN and PEMAT, which were produced mainly for assessing health information quality. DISCERN has evaluated the reliability and quality of health information. PEMAT has assessed the understandability and actionability. The qualitative and quantitative analyses of the videos represent bias and poor health information quality, with a total score of 27 out of 80 for DISCERN and 31 out of 100 for the PEMAT. The results also showed weak users' awareness regarding the content of videos with no association between user engagement indicators (likes, dislikes, VPI, views, comments) and the dimensions of the two tools. The study concludes that it is evident that YouTube, in its current form, is an inadequate Arabic source for herbal cancer treatment information. To overcome this, this study proposed the GAP framework for social media that integrated Governance, Awareness, and Proficiency.

Investigating #vapingcessation in Twitter

Evidence suggests that an increasing number of e-cigarette users report intentions and attempts to quit vaping. Since exposure to e-cigarette-related content on social media may influence e-cigarette and other tobacco product use, including potentially e-cigarette cessation, we aimed to explore vaping cessation-related posts on Twitter by utilizing a mixed-methods approach. We collected tweets pertaining to vaping cessation for the time period between January 2022 and December 2022 using snscrape. Tweets were scraped for the following hashtags: #vapingcessation, #quitvaping, and #stopJuuling. Data were analysed using Azure Machine Learning and Nvivo 12 software. Sentiment analysis revealed that vaping cessation-related tweets typically embody positive sentiment and are mostly produced in the U.S. and Australia. Our qualitative analysis identified six emerging themes: vaping cessation support, promotion of vaping cessation, barriers and benefits to vaping cessation, personal vaping cessation, and usefulness of peer support for vaping cessation. Our findings imply that improved dissemination of evidence-based vaping cessation strategies to a broad audience through Twitter may promote vaping cessation at the population level.

Contributions of digital social research to develop Telemedicine in Calabria (Southern Italy): identification of inequalities in post-COVID-19

The paper discusses the role that sociology and digital social research methods could play in developing E-health and Telemedicine, specifically after the COVID-19 pandemic, and the possibility of dealing with new pandemics. In this article, we will reflect on an interdisciplinary research pilot project carried out by a team of sociologists, medical doctors, and software engineers at The University of Calabria (Italy), to give a proof of concept of the importance to develop Telemedicine through the contribution of digital social research. We apply a web and app survey to administrate a structured questionnaire to a self-selected sample of the University Community. Digital social research has highlighted socioeconomic and cultural gaps that affect the perception of Telemedicine in the University Community. In particular, gender, age, educational, and professional levels influence medical choices and behaviors during Covid-19. There is often an unconscious involvement in Telemedicine (people use it but don't know it is Telemedicine), and an optimistic perception grows with age, education, professional, and income levels; equally important are the comprehension of digital texts and the effective use of Telemedicine. Limited penetration of technological advances must be addressed primarily by overcoming sociocultural and economic barriers and developing knowledge and understanding of digital environments. The key findings of this study could help direct public and educational policies to reduce existing gaps and promote Telemedicine in Calabria.

Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1

Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a "text preparation" mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor's ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.

Assessment of Stress Urinary Incontinence-Related Podcasts on Stitcher

OBJECTIVE

To assess the quality, understandability, actionability, accuracy and commercial bias of podcasts regarding stress urinary incontinence (SUI).

METHODS

The first 100 Stitcher podcasts from the search, "Stress urinary incontinence" were evaluated using the validated DISCERN criteria, PEMAT tool, and a Misinformation Likert Scale. The accuracy of information was assessed by comparison to the American Urological Association (AUA) SUI Guidelines.

RESULTS

Common publisher types included health/wellness channels (66%). Low-moderate quality, poor understandability and misinformation was seen in 67% (DISCERN ≤3), 69% (PEMAT understandability ≤75%) and 17% of podcasts, respectively. Treatments discussed included pelvic floor muscle training (PFMT) (89%) and surgery (47%). Health and wellness channels had the most evidence of commercial bias (44%). Podcasts that discussed surgery more often emphasized shared decision-making (DISCERN15).

CONCLUSION

SUI-related podcasts on Stitcher are mostly published by health/wellness channels and skewed toward discussing PFMT. Podcasts are often low-moderate quality, poorly understandable and often have commercial bias.

Promoting patient engagement in cancer genomics research programs: An environmental scan

Background: A national priority in the United States is to promote patient engagement in cancer genomics research, especially among diverse and understudied populations. Several cancer genomics research programs have emerged to accomplish this priority, yet questions remain about the meaning and methods of patient engagement. This study explored how cancer genomics research programs define engagement and what strategies they use to engage patients across stages in the conduct of research. Methods: An environmental scan was conducted of cancer genomics research programs focused on patient engagement. Research programs were identified and characterized using materials identified from publicly available sources (e.g., websites), a targeted literature review, and interviews with key informants. Descriptive information about the programs and their definitions of engagement, were synthesized using thematic analysis. The engagement strategies were synthesized and mapped to different stages in the conduct of research, including recruitment, consent, data collection, sharing results, and retention. Results: Ten research programs were identified, examples of which include the Cancer Moonshot Biobank, the MyPART Network, NCI-CONNECT, and the Participant Engagement and Cancer Genome Sequencing (PE-CGS) Network. All programs aimed to include understudied or underrepresented populations. Based on publicly available information, four programs explicitly defined engagement. These definitions similarly characterized engagement as being interpersonal, reciprocal, and continuous. Five general strategies of engagement were identified across the programs: 1) digital (such as websites) and 2) non-digital communications (such as radio broadcasts, or printed brochures); 3) partnering with community organizations; 4) providing incentives; and 5) affiliating with non-academic medical centers. Digital communications were the only strategy used across all stages of the conduct of research. Programs tailored these strategies to their study goals, including overcoming barriers to research participation among diverse populations. Conclusion: Programs studying cancer genomics are deeply committed to increasing research participation among diverse populations through patient engagement. Yet, the field needs to reach a consensus on the meaning of patient engagement, develop a taxonomy of patient engagement measures in cancer genomics research, and identify optimal strategies to engage patients in cancer genomics. Addressing these needs could enable patient engagement to fulfill its potential and accelerate the pace of cancer genomic discoveries.

Online public information about advance care planning: An evaluation of UK and international websites

Introduction

Healthcare information is increasingly internet-based. Standards require websites to be 'perceivable, operable, understandable and robust' with relevant content for citizens in appropriate language. This study examined UK and international websites offering public healthcare information on advance care planning (ACP) using current recommendations for website accessibility and content and informed by a public engagement exercise.

Methods

Google searches identified websites in English from health service providers, governmental or third sector organisations based in the UK and internationally. Target keywords that would be used by a member of the public informed the search terms. Data extraction used criterion-based assessment and web content analysis of the first two pages of each search result. Public patient representatives as key members of the multidisciplinary research team guided the development of the evaluation criteria.

Results

A total of 1158 online searches identified 89 websites, reduced to 29 by inclusion/exclusion criteria. Most sites met international recommendations for knowledge/understanding about ACP. Differences in terminology, lack of information about ACP limitations and non-adherence to recommended reading levels, accessibility standards and translation options were apparent. Sites targeting members of the public used more positive, non-technical language than those for both professional and lay users.

Conclusions

Some websites met accepted standards required to facilitate understanding and public engagement in ACP. Others could be improved significantly. Website providers have important roles and responsibilities in increasing people's understanding of their health conditions, future care options and ability to take an active role in planning for their health and care.

The Impact of Popular Science Articles by Physicians on Their Performance on Online Medical Platforms

The public demand for popular science knowledge regarding health is increasing, and physicians' popular science practices on online medical platforms are becoming frequent. Few studies have been conducted to address the relationship between specific characteristics of popular science articles by physicians and their performance. This study explored the impact of the characteristics of popular science articles on physicians' performance based on the elaboration likelihood model (ELM) from the central path (topic focus and readability) and the peripheral path (form diversity). Data on four diseases, namely, lung cancer, brain hemorrhage, hypertension, and depression, were collected from an online medical platform, resulting in relevant personal data from 1295 doctors and their published popular science articles. Subsequently, the independent variables were quantified using thematic analysis and formula calculation, and the research model and hypotheses proposed in this paper were verified through empirical analysis. The results revealed that the topic focus, readability, and form diversity of popular science articles by physicians had a significant positive effect on physicians' performance. This study enriches the research perspective on the factors influencing physicians' performance, which has guiding implications for both physicians and platforms, thereby providing a basis for patients to choose physicians and enabling patients to receive popular science knowledge regarding health in an effective manner.

Evidence-based health information about pulmonary embolism: Assessing the quality, usability and readability of online and offline patient information

Objective

Pulmonary embolism (PE) is the third most common cardiovascular disease worldwide. However, public awareness is considerably lower than for myocardial infarction or stroke. Patients suffering from PE complain about the lack of (understandable) information and express high informational needs. To uncover if reliable information is indeed scarce, this study evaluates the quantity and quality of existing patient information for tertiary prevention using an evidence-based health information paradigm.

Methods

We conducted a quantitative content analysis (n = 21 patient information brochures; n = 67 websites) evaluating content categories addressed, methodical quality, usability, and readability.

Results

Results show that there is not enough patient information material focusing on PE as a main topic. Existing patient information material is mostly incomplete, difficult to understand, and low in actionability as well as readability.

Conclusion

Our systematic analysis reveals the need for more high-quality patient information on PE as part of effective tertiary prevention.

Innovation

This is the first review analyzing content, methodical quality, readability, and usability of patient information on PE. The findings of this analysis are guiding the development of an innovative, evidence-based patient information on PE aiming to support patients' informational needs and their self-care behavior.

Assessment of Online Patient Education Material About Dysphagia

To examine quality, readability, understandability, and actionability of English-language online educational materials about dysphagia. A Google search of "dysphagia" and related terms was conducted. Web page quality and accountability were measured using HON and URAC certification seals, the DISCERN instrument, and JAMA benchmark criteria. Understandability and actionability were assessed with the Patient Education Materials Assessment Tool for Printed Material (PEMAT-P). Readability was assessed using the Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (F-KGL), Gunning Fog (FOG), and the Simple Measure of Gobbledygook (SMOG) scores using dedicated readability software. Fifty web pages were analyzed. Seventeen web pages displayed a HON or URAC seal. DISCERN scores ranged from 17 to 50 (Mdn = 25.00; IQR = 32.25-21.00). Of the JAMA benchmark criteria, 88% of web pages met the disclosure criterion, while only 22% met the authorship, 20% met the attribution, and 16% met the currency criteria. PEMAT-P understandability and actionability scores were 69.38% ± 11.14% and 28.58% ± 22.19%, respectively. Readability scores, on average, exceeded the recommended grade reading levels for health information (FRE 46.34 ± 13.59, F-KGL 10.26 ± 2.29, FOG 12.11 ± 2.08, and SMOG 12.38 ± 1.70). Online materials about dysphagia can be improved by obtaining quality certificates and by including content that is more readable and easier to understand and act upon.

Structural Analysis of the Health Literacy Facet Access to Information on General Health, COVID-19 Infection Prevention, and Early Childhood Allergy Prevention in Pregnant Women and Mothers of Infants

Abstract. The health literacy (HL) facet Access to health information is measured in the European Health Literacy Survey (HLS-EU-Q47) by 12 items. To assess Access, we developed adapted item formulations for COVID-19 infection prevention (COVID-19-IP) and early childhood allergy prevention (ECAP) in addition to the original 12 items on General Health (GH). N = 343 (expectant) mothers of infants answered the items in an online assessment. Confirmatory structural analyses for ordinal data were adopted (WLSMV-algorithm). Women’s item ratings varied significantly across domains (η2 = .017 – .552). Bi-factor models exhibited the best data fit (GH / COVID-19-IP / ECAP: CFI = .964 / .968 / .977; SRMR: .062 / .069 / .035): The general factor Access most strongly determined item information. Additionally, three subfactors contributed significantly (but rather weakly) to the item information in each domain. The overall score Access proved to be internally consistent (McDonald’s ωGH/COVID-19-IP/ECAP = .874 / .883 / .897) and was associated with socioeconomic state (McArthur scale; rGH/COVID-19-IP/ECAP = .218 / .210 / .146). Access correlated not or only weakly with the other HL facets Understand, Appraise, and Apply. The health domains GH, COVID-19-IP, and ECAP moderated both the difficulty and the dimensional structure of the 12 Access items. This suggests that in the HLS-EU Access reflects not only the search competence but also the availability of health information.

The complementary and substitutive value of online health information

The Internet plays a significant role in health information searching, sharing and emotional support. However, scholars have devoted little attention to the complementary and substitute value of online health information from diseases, especially chronic diseases, health insurance, barriers to health resources and their interaction effects with income. This research uses data from the 2020 Health Information National Trends Survey (HINTS 2020), the latest HINTS survey that includes seeking online health information questions critical to this research. This paper proposes that the factors contributing to seeking online health information can be categorized into two modalities - complementary and substitutive. Concerning the complementary value, I argue that individuals with certain health conditions use online health information as a complementary health resource in addition to traditional health resources such as doctors to understand their health issues better. Online health information also functions as substitute information sources for individuals who have experienced more barriers to typical health information resources.

Internet-Based Learning for Professionals in Addiction Psychiatry: A Scoping Review

BACKGROUND AND AIM

Distance learning techniques are gaining popularity as the internet today is a faster and more convenient source of spreading exhaustive information. We aim at conducting a scoping review on the utility of internet-based learning in addiction psychiatry.

METHODS

We performed a systematic search of databases such as Google Scholar, PubMed, and Cochrane using the following keywords: Distance education, substance related disorder, teaching, internet, along with the MeSH terms ("Internet" AND "Teaching" AND "Substance Related Disorders") OR ("Distance Education" AND "Substance Related Disorders"). The reference lists of articles were searched for other relevant literature.

RESULTS

A total of 34 studies were included. Most of the courses offered online were on specific therapies, for example, training on motivational interviewing, twelve-step facilitation, cognitive behavioral therapy, alcohol screening, and brief intervention. They were based on various aspects of distance learning such as the acquisition of knowledge or skills, feasibility, level of satisfaction of the trainees/students, the cost-effectiveness of the techniques, and the type of trainees who have access to the course. The majority of the studies showed positive findings in terms of the parameter being studied. However, the level of satisfaction among the students is questionable.

CONCLUSION

Internet-based teaching programs provide a good alternative to traditional educational methods in terms of effectiveness, feasibility, and acquisition of knowledge and skills. We emphasize the use of internet-based training in addiction psychiatry reinforced by actual practical experiences.

Online Presence of the Funeral Industry: The Example of the Quebec Federation of Funeral Cooperatives

The digitalization of modern societies has offered new tools for funeral industries to reach the communities they serve, ranging from using technologies in funeral planning, in commemoration of the dead, or to support the bereaved. The COVID-19 pandemic only pushed this need for online presence of the funeral industries further. We explore the digitalization of funeral industries through the example of Quebec (Canada), where many funeral institutions are regrouped under a federation of funeral cooperatives. We analyze how this influences the delivery of funeral services and allows the development of common services answering various needs of the population (e.g., a funding program for youth funerals, an ecological memorialization program, online grief support). Finally, we discuss how the federation's online presence contributes to its mission, and more largely whether it changes the perception of the industry.

Evaluation of Web-Based Health Information From the Perspective of Women With Eating Disorders: Thematic Analysis

Background

Users with experience of eating disorders use the internet as a source of information, whether for prorecovery activities (such as web-based treatment, looking for information, support, and sharing) or activities that promote eating disorder behavior as a desirable lifestyle choice (such as pro–eating disorder communities and reading and creating pro–eating disorder posts). Their assessment of web-based eating disorder–related information is crucial for understanding the context of the illness and for health professionals and their web-based interventions.

Objective

This study aimed to understand the criteria young women with the experience of eating disorders use in evaluating eating disorder–related web-based information and what eating disorder–related characteristics of these women are involved in their evaluation.

Methods

We analyzed 30 semistructured individual interviews with Czech women aged 16 to 28 years with past or present eating disorder experience using a qualitative approach. Thematic analysis was adopted as an analytical tool.

Results

The specifics of eating disorder phases (the disorder stage and the treatment process) emerged as important aspects in the process of information assessment. Other specific characteristics of respondents (eg, motivation, abilities, and resources) addressed how the respondents arrived at certain web-based information and how they evaluated it. In addition, the respondents described some content cues as features of information (eg, novelty and social information pooling). Another finding is that other users’ attitudes, experiences, activities, and personal features are involved in the information evaluation of these users and the information presented by them. Finally, the respondents evaluated the websites’ visual look and graphic components.

Conclusions

This study shows that web-based information evaluation reported by women with experience of eating disorders is a complex process. The assessment is influenced by current personal characteristics related to the illness (mainly the motivation for maintaining or curing the eating disorder) using cues associated with information content, other users, and website look. The study findings have important implications for health professionals, who should ask their clients questions about web-based communities and their needs to understand what information and sources they choose.

A comparative study of strategies for identifying credible sources of mental health information online: Can clinical services deliver a youth-specific internet prescription?

BACKGROUND

Adolescents and young adults undertake internet searches before and after attending mental health consultations, but they express concerns about how they can identify reliable online mental health information (OMHI). This action research project examines how clinicians might direct young people towards credible OMHI by analysing which search strategies are most helpful for developing an 'internet prescription' targeted at a youth audience.

METHODS

Recently qualified doctors undertook searches for OMHI about the diagnosis and treatment of seven mental health conditions. Rankings of websites in search results were compared with reliability and quality ratings derived from established evaluation instruments [the Brief DISCERN and the 12-item Health on the Net (HoN) tool].

RESULTS

Of 140 websites identified through seven Google searches, only 15% of those evaluated were categorized as high quality. About 17% of websites were selling a treatment or service and about 25% were regarded as untrustworthy. Also, higher rankings in the search results were not indicative of better quality OMHI (Wilcoxon signed rank test: Z = -5.28; p < .001). Although the HoN and DISCERN tools provided useful insights into the credibility of OMHI, investigators did not think either instrument could be recommended for independent use by youth.

CONCLUSIONS

Our findings suggest that clinical services should take responsibility for identifying a range of credible sources of OMHI. The next step in translating the research strategy into clinical practice involves collaboration with young people to select those platforms most orientated to their needs and to develop rating instruments that enable youth to evaluate OMHI.

Evaluating breast cancer websites targeting Arabic speakers: empirical investigation of popularity, availability, accessibility, readability, and quality

Background

Nowadays, patients have access to all types of health information on the internet, influencing their decision-making process. The Middle East and North Africa (MENA) region consists of 22 countries with an estimated population of around 600 million. Breast cancer is the highest diagnosed cancer in this region. Websites are commonly the go-to cancer information sources. A large population of the MENA region is only fluent in the Arabic language, thus access to Arabic websites is in more demand. However, little is known about breast cancer websites that cater to an Arabic-speaking audience. This study aims at evaluating Arabic breast cancer websites and offering recommendations to improve engagement and access to health information.

Methods

This study employed a cross-sectional analysis approach. Google trends was used to reveal the top searched topics across the MENA region, which in turn were used as search terms to identify the websites. To be included, a website had to be active, available in Arabic, and contain breast cancer information. The evaluation was based on a combination of automated and expert-based evaluation methods through five dimensions: Availability, Accessibility, Readability, Quality, and Popularity.

Results

Overall most of the websites performed poorly in the five dimensions and require careful reassessment concerning design, content, and readability levels; Only one website performed well in all dimensions, except for readability. Generally, the readability scores indicated that the websites were above the recommended level of reading. None of the websites passed the automated accessibility tests. The expert evaluation using the “Health on the Net” checklist showed good results for most websites.

Conclusions

Breast cancer rates are rising in the MENA region, therefore having comprehensive, accurate, trustworthy, and easy-to-understand health information in their native language is a must. The results from this study show a need for improving the accessibility to breast cancer information websites available to Arabic speakers. The search was limited to three search engines yielding 10 websites and only one tool was used per dimension. Future work is needed to overcome these limitations. Collaboration between multiple stakeholders is necessary to develop websites that contain easy-to-read and understand high-quality information.

The perceptions of social media users of digital detox apps considering personality traits

The purpose of this study was to investigate the perceptions of users about using digital detox applications and to display relationships among personality traits and technology-related variables. This study was designed using survey approach and employed Generalized Structured Component Analysis (GSCA). As such, 11 hypotheses were constructed and tested. The study recruited 263 participants who utilize detox applications to avoid social media distractions. Data were collected through Google Form and analyzed using GSCA Pro 1.1 to better understand whether the proposed conceptual model fits the data. The results of the study indicated that behavioral intention predicted usage behavior significantly; performance expectancy, effort expectancy, and social influence positively affected behavioral intention; in turn, agreeableness and extroversion positively influenced performance expectancy, and extroversion affected effort expectancy; finally, neuroticism had a statistically significant and negatively associated with effort expectancy of using social media detox apps. The significant exceptions were that facilitating conditions were not predictive of behavioral intention, openness to experience did not influence performance expectancy, and conscientiousness was not linked to effort expectancy. The proposed conceptual model explained 56.68% of the amount of variation, indicating that instructors, policy makers and software designers should consider personal factors for preparing practical intervention approaches to mitigate learning issues related to social media distraction.

Research to Clinical Practice-Youth seeking mental health information online and its impact on the first steps in the patient journey

BACKGROUND

Online searches about anxiety and depression are recorded every 3-5 s. As such, information and communication technologies (ICT) have enormous potential to enable or impair help-seeking and patient-professional interactions. Youth studies indicate that ICT searches are undertaken before initial mental health consultations, but no publications have considered how this online activity affects the first steps of the patient journey in youth mental health settings.

METHODS

State-of-the-art review using an iterative, evidence mapping approach to identify key literature and expert consensus to synthesize and prioritise clinical and research issues.

RESULTS

Adolescents and young adults are more likely to seek health advice via online search engines or social media platforms than from a health professional. Young people not only search user-generated content and social media to obtain advice and support from online communities but increasingly contribute personal information online.

CONCLUSIONS

A major clinical challenge is to raise professional awareness of the likely impact of this activity on mental health consultations. Potential strategies range from modifying the structure of clinical consultations to ensure young people are able to disclose ICT activities related to mental health, through to the development and implementation of 'internet prescriptions' and a youth-focused 'toolkit'.

Quality of information of websites dedicated to obesity: a systematic search to promote high level of information for Internet users and professionals

BACKGROUND

The Internet is increasingly used as a source of information. This study investigates with a multidimensional methodology the quality of information of websites dedicated to obesity treatment and weight-loss interventions. We compared websites in English, a language that it is used for the international scientific divulgation, and in Italian, a popular local language.

METHODS

Level of Evidence: Level I, systematic review search on four largely used search engines. Duplicated and unrelated websites were excluded. We checked: popularity with PageRank; technological quality with Nibbler; readability with the Flesch Reading Ease test or the Gulpease readability index; quality of information with the DISCERN scale, the JAMA benchmark criteria, and the adherence to the Health on the Net Code.

RESULTS

63 Italian websites and 41 English websites were evaluated. English websites invested more in the technological quality especially for the marketing, experience of the user, and mobile accessibility. Both the Italian and English websites were of poor quality and readability.

CONCLUSIONS

These results can inform guidelines for the improvement of health information and help Internet users to achieve a higher level of information. Users must find benefits of treatment, support to the shared decision-making, the sources used, the medical editor's supervision, and the risk of postponing the treatment.

Experiencia en el uso de videos para la promoción de ejercicio físico en el hogar en personas mayores en modalidad online en Chile durante la pandemia Covid19

Objetivo

Describir la usabilidad, la accionabilidad y la comprensión de videos para promover el ejercicio en el hogar de manera online durante la pandemia COVID-19, analizando las características de la población, su satisfacción, adherencia y barreras para realizar ejercicio físico.

Material y métodos

Estudio de diseño observacional descriptivo, no experimental y transversal, con enfoque cuantitativo. Se crearon videos de ejercicio físico y educación sobre dolor para personas mayores de 60 años, y fueron distribuidos en la Región Metropolitana (Chile) durante los meses de octubre a diciembre del 2020. La intervención consistió en el uso independiente en el hogar de estos videos, y tuvo una duración de 4 semanas. Luego, una muestra que recibió estos videos contestó una encuesta voluntariamente.

Resultados

Treinta y cuatro participantes evaluaron los videos como comprensibles y accionables. La usabilidad fue menor en las personas que percibían un nivel de manejo tecnológico menor. La adherencia promedio fue de 2 días a la semana durante 2,5 semanas. La principal barrera percibida fue la falta de voluntad, la cual mostró una mayor tendencia en las personas que realizaban ejercicio físico de manera grupal antes de las restricciones por la pandemia COVID-19.

Conclusiones

Los videos desarrollados fueron una herramienta bien evaluada por parte de la población mayor, sin embargo, la brecha tecnológica puede ser una limitante para favorecer un uso fácil y sencillo. Dentro de las estrategias se deberían considerar intervenciones que favorezcan la interacción social para promover la motivación interpersonal, en especial en las personas que prefieren modalidades grupales de ejercicio.

Does social media provide adequate health education for prevention of COVID-19? A case study of YouTube videos on social distancing and hand-washing

Social media offers an opportune platform for educating the public about the recommended interventions during global health emergencies. This case study evaluated information in the popular social media platform YouTube about two key interventions (namely, 'social distancing' and 'hand washing') recommended during coronavirus disease-2019. Using the keywords 'social distancing' and 'hand washing', 77 and 78 videos, respectively, were selected from YouTube through pre-defined criteria. The understandability, actionability and quality of information in these videos were assessed. Cumulatively, the social distancing videos received >9 million views and the hand-washing videos received >37 million views. Thirteen social distancing videos (16.9%) and 46 hand-washing videos (58.9%) provided understandable, actionable and good-quality information. The non-understandable, non-actionable or poor-quality videos had paradoxically more viewer engagements than the understandable, actionable or good-quality videos, respectively. Most social distancing videos came from news agencies (68.8%). Hand-washing videos were mostly uploaded by health agencies or academic institutes (52.6%). The videos were less likely to be understandable and actionable and to be of good quality when uploaded by sources other than health agencies or academic institutes. The paucity of adequate information and the limited representation of 'authoritative' sources were concerning. Strategies for harnessing social media as an effective medium for public health education are necessary during pandemics.

Assessing the Accuracy, Quality, and Readability of Information Related to the Surgical Management of Benign Prostatic Hyperplasia

Objectives: To assess the accuracy, quality, and readability of online educational health information in English related to the most common benign prostatic hyperplasia (BPH) guideline-approved surgical treatments. Methods: The terms "benign prostatic hyperplasia," "BPH," and all eight guideline-approved treatment modalities studied, were searched to retrieve the first five relevant websites and first two paid advertised websites related to the surgical treatment options for BPH. These modalities included transurethral resection of the prostate (TURP), GreenLight photovaporization, endoscopic enucleation of the prostate, Rezum, Urolift, Aquablation, open simple prostatectomy, and robotic simple prostatectomy (RSP). All relevant websites were assessed for their accuracy, quality, and readability using standardized scoring systems. Results: The mean accuracy score for each of the treatment modalities were all indicative of good accuracy, with 76%-99% of the information presented as being accurate. The median quality score was statistically different across the eight treatment modalities (p = 0.015). The median readability grade level was statistically different across the eight treatment modalities (p = 0.009). Websites that described TURP (median readability grade level, 9.00 [interquartile range (IQR) 8.00-10.80]) were significantly easier to read than those related to RSP (median readability grade level, 14.35 [IQR, 11.08-16.50]) (p = 0.011). No other statistically significant differences were found within the other treatment modality websites. Conclusions: The majority of websites retrieved were found to be of high accuracy, good quality, and poor readability. Additionally, it was found that none of the retrieved websites included descriptions for all the other included treatment modalities. Given these findings, the authors recommend the development of centralized resources with all guideline-approved treatment modalities and accurate, readable, and high-quality information related to the surgical treatment of BPH.

Sentiment Analysis for Necessary Preview of 30-Day Mortality in Sepsis Patients and the Control Strategies

This study was to preview the risk of 30-day mortality in sepsis patients using sentiment analysis. The clinical data of patients and nursing notes were collected from the Medical Information Mart for Intensive Care (MIMIC-III) database. The factors influencing 30-day mortality were analyzed using the Cox regression model. And, the prognostic index (PI) was estimated. The receiver operating characteristic (ROC) curve was used to determine the PI cut-off point and assess the prediction ability of the model. In total, 1844 of 3560 patients were eligible for the study, with a 30-day mortality of 37.58%. Multivariate Cox analysis showed that sentiment polarity scores, sentiment subjectivity scores, simplified acute physiology score (SAPS)-II, age, and intensive care unit (ICU) types were all associated with the risk of 30-day mortality (P < 0.05). In the preview of 30-day mortality, the area under the curve (AUC) of ROC was 0.78 (95%CI: 0.74–0.81,P < 0.001) when the cut-off point of PI was 0.467. The documented notes from nurses were described for the first time. Sentiment scores measured in nursing notes are associated with the risk of 30-day mortality in sepsis patients and may improve the preview of 30-day mortality.

Can We Go Online for Sports Injury Prevention? A Systematic Review of English-Language Websites with Exercise-Based Sports Injury Risk Reduction Programmes

Background

Preventing sports injuries is at the forefront of sports medicine. Although effective preventive strategies in scientific literature exist, their implementation is lagging behind. The Internet could support the translation of knowledge from the literature to end-users, but the quality of the online resources would have to be assured. This online-based systematic review is to assess availability, readability, quality, and content of the websites presenting exercise-based sports injury risk reduction (prevention) programmes. Moreover, the quality of reporting and contents of the exercise programmes were assessed.

Methods

Google, Yahoo, and Bing were searched on 2 September 2018. We used ‘sports injury prevention program*’ and ‘sports injury prevention warm-up’ as search phrases. The owners/authors of the included websites were asked for further recommendations on online resources. Search updates were run in DuckDuckGo on 15 May 2020 and 22 August 2021. Eligible websites were active, in English, and contained instructions for the exercise/s aiming at sports injury prevention. Two reviewers independently screened the links and previews and performed an in-depth appraisal of included websites. The website quality was assessed using JAMA framework criteria and Health on the Net Foundation Code of Conduct (HONcode) certification. The readability of websites was assessed using the Flesch-Kincaid Reading Ease score. The reporting appraisal of exercise programmes was done using the modified Consensus on Exercise Reporting Template (CERT).

Results

Among 480 websites screened, 16 were eligible with an additional four recommended and nine found in search updates (29 in total). None of the websites was certified by HONcode. The overall quality of websites was low 2.1 ± 1.0/4, but overall readability was high 67 ± 17/100. The average quality of reporting of exercise programmes was low 5.79 ± 3.1/12. Websites with community input had the lowest readability, but the highest quality, and vice versa websites run by businesses had the highest readability, but the lowest quality. Eight websites presented programmes tested for effectiveness.

Conclusions

Overall, the quality of the websites was low, but their readability was high. Improvements required are relatively easy to implement (i.e. including the date when the website was updated, applying for HONcode certification) and extremely important (e.g. providing resources on which the website’s content is based). There are some sports injury risk reduction programmes reported with high quality and effectiveness-tested available online for team sports, but none for individual sports.

Trial Registration This review has been registered in the PROSPERO (CRD42019107104).

Quality and readability of online information on idiopathic subglottic stenosis

OBJECTIVE

Idiopathic subglottic stenosis (ISS) is a chronic condition characterized by disease recurrence and multiple surgeries. These frustrated patients may utilize the internet to research their condition. The aim of this study was to determine the quality and readability of online ISS information.

METHODS

"Idiopathic subglottic stenosis" was entered into Google. The first 50 websites that met inclusion criteria were extracted. The DISCERN instrument, Flesch Reading Ease Score (FRES), and Flesch-Kincaid Grade Level (FKGL) assessed the quality and readability, respectively. Means, SDs, Pearson correlation coefficients, and two-tailed Student's t-test were calculated.

RESULTS

The 50 websites consisted of 17 patient-targeted and 33 professional-targeted websites, plus 30 major and 20 minor websites. The overall DISCERN, FRES, and FKGL scores were 2.81 ± 0.99, 27.75 ± 15.27, and 13.65 ± 2.79, respectively (mean ± SD). Patient-targeted websites had significantly lower quality (DISCERN [P < .00]) but were easier to read (lower FKGL [P < .00], higher FRES [P < .00]) than professional-targeted websites. Minor websites had a significantly lower quality (DISCERN [P < 0.00]) but were easier to read (lower FKGL [P < .00], higher FRES [P < .00]) than major websites. There was a positive correlation between overall quality and difficulty in readability.

CONCLUSION

The quality of online ISS information was suboptimal. Resources were too difficult to comprehend and readability scores were above AMA and NIH recommendations. Improved online information is required to properly educate this patient population.

LEVEL OF EVIDENCE

Level 4.

Factors Affecting User Intention to Pay via Online Medical Service Platform: Role of Misdiagnosis Risk and Timeliness of Response

With regard to platform performance and trust, we study the influencing factors of users’ intention to pay on an online medical service platform. Results of this paper will provide a new perspective for online medical service platform research in the context of the Internet.A questionnaire survey is administered to collect 312 effective sample data, and the data are analyzed by partial least squares structural equation modeling. Results showed that the information quality, system quality, and convenience of the platform significantly affect the perceived benefit (PB) of users. Users’ perceived reliability of the platform significantly positively affects doctor reliability (DR). Users’ PB significantly positively affects their ITP and DR, and DR significantly positively affects the users’ ITP. The misdiagnosis risk positively regulates the relationship between the users’ trust tendency and DR. The timeliness of the response of the platform positively adjusts the relationship between DR and users’ ITP.

Searching for Social Media Addiction: A Content Analysis of Top Websites Found through Online Search Engines

Disordered social media use, often referred to as “social media addiction”, has not been officially recognized by medical bodies such as the American Psychiatric Association or the World Health Organization. However, websites still present information to laypeople on how to treat and manage social media addiction, which can pose the risk of spreading low quality or incorrect information. As such, we aimed to assess how the most popular social media addiction websites present information across multiple metrics. We conducted an in-depth online search to identify the top social media addiction websites in November 2019 (N = 23). Websites were separated into four distinct classifications: (1) treatment/therapy/medical; (2) informational; (3) news article; and (4) blog/essay. Based on previous website analysis research, three trained coders evaluated these websites on six metrics: (1) design; (2) credibility; (3) accessibility; (4) literacy; (5) engagement; and (6) social media addiction content. Design features were the top-rated metric across all websites, followed by credibility. Websites scored the lowest for the engagement and social media addiction content metrics. Across website classifications, scores for social media addiction content varied greatly, with blog/essay websites ranking the lowest and informational websites ranking the highest. Our findings provide necessary information for both patients and healthcare providers, apprising these individuals and the field about the current online health information landscape for disordered social media use.

Quality of Carpal Tunnel Syndrome Patient Education Handouts Available on the Internet: A Systematic Analysis of Content and Design

OBJECTIVE

To evaluate the quality of carpal tunnel syndrome (CTS) patient education handouts and identify the best resources for patients and clinicians.

DESIGN

A document content analysis of handouts identified through a systematic internet search using 8 search terms on Google and Bing and a hand search of professional association websites.

SETTING

Not applicable.

PARTICIPANTS

Documents (N=56) were identified from the top 50 search results across 16 individual searches. Included documents provided general patient education for CTS; descriptive websites, videos, and research studies were excluded.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Content analysis was conducted using the Information Score (IS) tool to evaluate completeness of information (0%-100%) and misleading treatment recommendations were identified. Design analysis was conducted using the Patient Material Assessment Tool for Printable Materials (PEMAT-P) (0%-100%) and 2 widely used readability formulas, Flesch Reading Ease and Flesch Kincaid Grade Level. Using these results, all handouts were rated with a summative 12-point scale.

RESULTS

Of 805 unique search results, we included 56 CTS handouts. The average IS was 74.6%±17.9%, and 78.6% of the handouts mentioned non-evidence-based treatment recommendations. The average PEMAT-P score was 70.2%±10.9%, and the average readability grade level was 7.7±1.7. Only 3 handouts were identified as high quality based on the 12-point summative scores, 22 handouts had mixed quality, and 17 handouts had low quality on both content and design.

CONCLUSIONS

Findings of this study suggest a lack of high-quality and easily understandable CTS patient education handouts. Most handouts contained unreliable treatment information. Improvements are needed to ensure patients' ability to understand and manage this condition.

Women’s Preferences and Design Recommendations for a Postpartum Depression Psychoeducation Intervention: A User Involvement Study (Preprint)

Background

Objective

Methods

Results

Conclusions

Help or hinder? An assessment of the accessibility, usability, reliability and readability of disability funding website information for Australian mental health consumers

This study aimed to assess the accessibility, usability, reliability and readability of those websites most likely encountered by Australian mental health consumers when using the internet to find information regarding the National Disability Insurance Scheme (NDIS). Websites were systematically identified with 127 deemed relevant for assessment in 2018. The LIDA instrument, the Simple Measure of Gobbledygook and Flesch-Kincaid Grade Level were used to evaluate the quality of information provided to mental health consumers. The study identified mediocre results for the accessibility, usability, reliability and readability of websites as they pertain to the needs of mental health consumers. Furthermore, it was identified that mental health support websites did not provide a more appropriate online experience for mental health consumers than general information websites, despite their focus on this demographic. These findings suggest a lack of understanding regarding the needs of mental health consumers and their experience of the online environment, which may in turn affect their access to information, agency and, ultimately, their uptake of the NDIS. The establishment of guidelines around enhancements to the online environment for mental health consumers would provide an experience that instils confidence, returns dignity and aids this group in realising their personal recovery journey.

Predicting Writing Styles of Web-Based Materials for Children's Health Education Using the Selection of Semantic Features: Machine Learning Approach

BACKGROUND

Medical writing styles can have an impact on the understandability of health educational resources. Amid current web-based health information research, there is a dearth of research-based evidence that demonstrates what constitutes the best practice of the development of web-based health resources on children's health promotion and education.

OBJECTIVE

Using authoritative and highly influential web-based children's health educational resources from the Nemours Foundation, the largest not-for-profit organization promoting children's health and well-being, we aimed to develop machine learning algorithms to discriminate and predict the writing styles of health educational resources on children versus adult health promotion using a variety of health educational resources aimed at the general public.

METHODS

The selection of natural language features as predicator variables of algorithms went through initial automatic feature selection using ridge classifier, support vector machine, extreme gradient boost tree, and recursive feature elimination followed by revision by education experts. We compared algorithms using the automatically selected (n=19) and linguistically enhanced (n=20) feature sets, using the initial feature set (n=115) as the baseline.

RESULTS

Using five-fold cross-validation, compared with the baseline (115 features), the Gaussian Naive Bayes model (20 features) achieved statistically higher mean sensitivity (P=.02; 95% CI -0.016 to 0.1929), mean specificity (P=.02; 95% CI -0.016 to 0.199), mean area under the receiver operating characteristic curve (P=.02; 95% CI -0.007 to 0.140), and mean macro F1 (P=.006; 95% CI 0.016-0.167). The statistically improved performance of the final model (20 features) is in contrast to the statistically insignificant changes between the original feature set (n=115) and the automatically selected features (n=19): mean sensitivity (P=.13; 95% CI -0.1699 to 0.0681), mean specificity (P=.10; 95% CI -0.1389 to 0.4017), mean area under the receiver operating characteristic curve (P=.008; 95% CI 0.0059-0.1126), and mean macro F1 (P=.98; 95% CI -0.0555 to 0.0548). This demonstrates the importance and effectiveness of combining automatic feature selection and expert-based linguistic revision to develop the most effective machine learning algorithms from high-dimensional data sets.

CONCLUSIONS

We developed new evaluation tools for the discrimination and prediction of writing styles of web-based health resources for children's health education and promotion among parents and caregivers of children. User-adaptive automatic assessment of web-based health content holds great promise for distant and remote health education among young readers. Our study leveraged the precision and adaptability of machine learning algorithms and insights from health linguistics to help advance this significant yet understudied area of research.

Development and proposed evaluation of an eHealth learning tool for undergraduate university students in Ireland

Undergraduate university students are at a critical stage of development in terms of their academic, social, psychological and behavioural health. Patterns established during these formative years can last a lifetime. eHealth tools have the potential to be engaging, convenient and accessible to a wide range of students by providing health information and enhancing the uptake of positive health behaviours. The 'Healthy Trinity Online Tool' (H-TOT) was developed in collaboration with students and a transdisciplinary team with decades of experience between them in terms of research, clinical responsibility and service delivery. Developmental steps undertaken included: a literature review to formulate the topic content choices; a survey of students to check the relevance and suitability of topics identified; and, the tacit experience of the development team. This co-design model led to the development of content encompassing academic life, healthy eating, physical activity, mood, financial matters, alcohol, tobacco, drugs and relaxation. Qualitative focus groups were subsequently conducted for in-depth exploration of the usage and functionality of H-TOT. The theoretical underpinnings include the locus of control and social cognitive theory. Evidence-based behavioural change techniques are embedded throughout. During early pre-piloting of H-TOT, the team identified and solved content functionality problems. The tone of the content was also revised to ensure it was non-judgemental. To make the H-TOT as interactive as possible, video scenarios were included and all content was audio-recorded to allow playback for students with visual or learning difficulties. Evaluation plans for the pilot year of H-TOT are outlined.

Online Patient Resources for Breast Implant-Associated Anaplastic Large Cell Lymphoma: A Readability Analysis

INTRODUCTION

Breast implant-associated anaplastic large cell lymphoma (BIA-ALCL) is an emerging issue facing the medical community. Government organizations such as the US Food and Drug Administration and specialty groups including the American Society of Plastic Surgeons have published online resources about BIA-ALCL for patients. Given the complexity of the diagnosis, it is important that patients can easily read these resources. In this study, we examined the readability levels of online BIA-ALCL patient resources using multiple verified reading scores.

METHODS

"BIA-ALCL" and "breast implant-associated anaplastic large cell lymphoma" were entered into 3 Internet search engines. The top 20 results for each were filtered by resource type and intended audience (physician vs patient). Published scientific articles, online database physician resources, and Web sites requiring subscriptions or fees were excluded. We then examined the readability of each with multiple verified reading scores, including the Flesch-Kincaid, Gunning-Fog, Coleman-Liau, Simplified Measure of Gobbledygook, and Automated Readability Index indices. Obtained data were analyzed using descriptive statistics and t test for independent samples.

RESULTS

Fifteen Web sites qualified for further analysis. For all texts, the average readability level was measured between 12 and 13 years of education on each readability index or approximately 18 to 19 years old. The Flesch-Kincaid Reading Ease average was 43.16 ±10.9 on a scale of 1 to 100, corresponding to a "difficult" designation. When compared by search criteria (spelled-out vs abbreviated), the results for the abbreviation "BIA-ALCL" had higher education requirements than those with the condition spelled out. However, these differences were not statistically significant. There was also great variation in word and sentence measurements. Twelve of the 15 Web sites contained more than 15% complex words, having more than 3 syllables, with breastcancer.org having the lowest (11%) and plasticsurgery.org the highest (20%).

DISCUSSION

Since the initial announcement in 2014 by the National Cancer Comprehensive Network, the medical community has begun educating ourselves and our patients about BIA-ALCL. Unfortunately, this study suggests that online patient resources on BIA-ALCL may be too complex for most readers, exceeding that of the average US resident (eighth grade) and Medicare beneficiary (fifth grade). Although the goal of learning more about BIA-ALCL and counseling patients appropriately remains paramount, we should continue to improve patient education materials given their vital role in healthcare decision-making.

Use of Machine Learning Algorithms to Predict the Understandability of Health Education Materials: Development and Evaluation Study

BACKGROUND

Improving the understandability of health information can significantly increase the cost-effectiveness and efficiency of health education programs for vulnerable populations. There is a pressing need to develop clinically informed computerized tools to enable rapid, reliable assessment of the linguistic understandability of specialized health and medical education resources. This paper fills a critical gap in current patient-oriented health resource development, which requires reliable and accurate evaluation instruments to increase the efficiency and cost-effectiveness of health education resource evaluation.

OBJECTIVE

We aimed to translate internationally endorsed clinical guidelines to machine learning algorithms to facilitate the evaluation of the understandability of health resources for international students at Australian universities.

METHODS

Based on international patient health resource assessment guidelines, we developed machine learning algorithms to predict the linguistic understandability of health texts for Australian college students (aged 25-30 years) from non-English speaking backgrounds. We compared extreme gradient boosting, random forest, neural networks, and C5.0 decision tree for automated health information understandability evaluation. The 5 machine learning models achieved statistically better results compared to the baseline logistic regression model. We also evaluated the impact of each linguistic feature on the performance of each of the 5 models.

RESULTS

We found that information evidentness, relevance to educational purposes, and logical sequence were consistently more important than numeracy skills and medical knowledge when assessing the linguistic understandability of health education resources for international tertiary students with adequate English skills (International English Language Testing System mean score 6.5) and high health literacy (mean 16.5 in the Short Assessment of Health Literacy-English test). Our results challenge the traditional views that lack of medical knowledge and numerical skills constituted the barriers to the understanding of health educational materials.

CONCLUSIONS

Machine learning algorithms were developed to predict health information understandability for international college students aged 25-30 years. Thirteen natural language features and 5 evaluation dimensions were identified and compared in terms of their impact on the performance of the models. Health information understandability varies according to the demographic profiles of the target readers, and for international tertiary students, improving health information evidentness, relevance, and logic is critical.

The readability of general practice websites: a cross-sectional analysis of all general practice websites in Scotland

Background

General practice websites are an increasingly important point of interaction, but their readability is largely unexplored. One in four adults struggle with basic literacy, and there is a socioeconomic gradient. Readable content is a prerequisite to promoting health literacy.

Aim

To assess general practice website readability by analysing text and design factors, and to assess whether practices adapted their website text to the likely literacy levels of their populations.

Design and setting

Websites for all general practices across Scotland were analysed from March to December 2019, using a cross-sectional design.

Method

Text was extracted from five webpages per website and eight text readability factors were measured, including the Flesch Reading Ease and the Flesch-Kincaid Grade Level. The relationship between readability and a practice population’s level of deprivation, measured using the Scottish Index of Multiple Deprivation (SIMD), was assessed. Overall, 10 design factors contributing to readability and accessibility were scored.

Results

In total, 86.4% (n = 813/941) of Scottish practices had a website; 22.9% (n = 874/3823) of webpages were written at, or below, the government-recommended reading level for online content (9–14 years old), and the content of the remaining websites, 77.1% (n = 2949/3823), was suitable for a higher reading age. Of all webpages, 80.5% (n = 3077/3823) were above the recommended level for easy-to-understand ‘plain English’. There was no statistically significant association between webpage reading age and SIMD. Only 6.7% (n = 51/764) of websites achieved all design and accessibility recommendations.

Conclusion

Changes to practice websites could improve readability and promote health literacy, but practices will need financial resources and ongoing technical support if this is to be achieved and maintained. Failure to provide readable and accessible websites may widen health inequalities; the topic will become increasingly important as online service use accelerates.

Development and Functionality of an Internet-Based, Self-Managed Parent Training Program

Purpose This research note evaluated a self-managed, Internet intervention (i.e., Success With Stories) designed to facilitate parental reading practices and beliefs. Method Twelve reviewers were composed of parents of young children, speech-language pathologists, and experts in self-managed, Internet intervention. Satisfaction of usability, content, presentation, suitability, and exercises of Success With Stories was examined using an intervention satisfaction rating. Value and usefulness of content was evaluated via a survey. Results Reviewers found content to be valuable and useful for facilitating parental reading practices and beliefs. Results indicated high satisfaction across all reviewers. Conclusion Results point toward strengths and limitations of the protocol and future directions.

Getting the right message: a content analysis and application of the health literacy INDEX tool to online HIV resources in Australia

Health literacy is considered by some as a determinant of health. Research suggests that different levels of health literacy may be associated with human immunodeficiency virus (HIV) knowledge and behaviour, including willingness to test. This study assessed the health literacy demands of existing Australia HIV prevention resources available online using a health literacy assessment and content analysis. Two reviewers assessed 66 resources using the Health Literacy INDEX. Sixty-one (92%) scored below 50%, with an average score of 35.5%. A coding frame was developed to conduct a content analysis of the resources excluding videos, multiple webpages and booklets (n = 52). We coded for six categories of knowledge: HIV knowledge (100% of resources), transmission (96%), acquired immune deficiency syndrome (AIDS) knowledge (88%), testing (87%), consequence of infection (85%) and prevention (77%). We found that resources required a reading grade above grade 8, and very few resources considered audience appropriateness. There were missed opportunities to encourage HIV prevention or testing. Some resources used incorrect language to refer towards people living with HIV, and transmission and prevention messages were often inconsistent. Guidelines for developing HIV prevention resources are warranted to improve health literacy, accessibility and appropriateness of resources and ensure consistent messages and framing of HIV risk.

Quality and readability of internet information about stuttering

PURPOSE

We examined the quality and readability of English-language Internet information about stuttering and evaluated the results considering recommendations by experts in health literacy.

METHOD

A search of Internet websites containing information about stuttering was conducted. Three key words (i.e., stuttering, stammering, speech disfluency) were entered into five country-specific versions of the most commonly used Internet search engine. A total of 79 websites were assessed. Their origin (commercial, non-profit, government, personal or university), quality [Health On the Net (HON) certification and DISCERN scores], and readability [Flesch Reading Ease (FRE) score, Flesch-Kincaid Grade Level Formula (F-KGL), and Simple Measure of Gobbledygook (SMOG)] were assessed.

RESULTS

Of the 79 websites, 38 % were of commercial, 42 % were of nonprofit organization, 15 % were of government and 5% were of university origins, respectively. Only 13 % had obtained HON certification and the mean DISCERN scores was 3.10 in a 5-point scale. The mean reading grade levels were at 13th and 14th grade and 100 % of the websites exceeded the recommended 5th to 6th reading grade level for health information.

CONCLUSIONS

The quality of Internet-based health information about the treatment of stuttering is generally adequate, but actual usability of the sites examined in this study may be limited due to poor readability levels. This is problematic in persons with poor literacy skills. Since the Internet can be readily accessed as a valuable consumer information resource, speech-language pathologists and other healthcare professionals have an opportunity to direct consumers to websites that provide readable information of good quality.

Analysis of Pelvic Organ Prolapse-related Content on Pinterest

OBJECTIVES

Our study is a cross-sectional, qualitative study analyzing pelvic organ prolapse-related content on Pinterest, a social media platform that also functions as a visual search engine. Primary outcomes assessed include quality, understandability, and actionability of content. Secondary outcomes include levels of misinformation and commercial bias. Our study further characterizes sources of prolapse content and treatment modalities discussed.

METHODS

A Pinterest search was conducted using the term "pelvic organ prolapse," analyzing the first 100 relevant results. We examined the publisher of each pin and the writer of linked content. Using validated tools including the Patient Education Materials Assessment Tool and DISCERN criteria, we assessed understandability and actionability of content. We further compared content to professional guidelines for discussing and treating prolapse, evaluating for misinformation. We also evaluated for commercial bias and subjectively rated pin quality.

RESULTS

Of pins evaluated, low understandability was found in 16, low actionability in 30, and low-to-moderate quality in 69. The most common publisher of pins was health and wellness groups, and the most common writer of content was physical therapists. Pelvic floor muscle training (PFMT) is the treatment modality discussed most frequently on Pinterest.

CONCLUSIONS

Pinterest content discussing pelvic organ prolapse shows good levels of understandability for patients at all levels of health literacy. However, the number of pins showing poor actionability raises concern that content may not encourage readers to take direct steps toward prolapse assessment and management. In all, a majority of prolapse-related content on Pinterest was of low-to-moderate quality.

Awareness, Information-Seeking Behavior, and Information Preferences About Early Childhood Allergy Prevention Among Different Parent Groups: Protocol for a Mixed Methods Study

Background

In early childhood allergy prevention (ECAP), parents act on behalf of their children. Parental health literacy and the availability of high-quality information, both online and offline, are crucial for effective ECAP. Recent research highlights three main points. First, parents need sufficient health literacy to discriminate between high-quality and low-quality information. Second, ECAP information behaviors may vary between phases of childhood development and according to individual circumstances. Third, to strengthen user-centeredness of available services, a better overview of parents’ information practices and needs and how they handle uncertainties is required.

Objective

This study aims to explore why, how, and when parents search for and apply ECAP-specific health information and which individual (eg, understanding of advice) and organizational challenges (eg, information services, information complexity, and changing recommendations) they perceive and how they handle them. This study also aims to assess the needs and preferences that parents express for future information formats and contents. The findings should inform the practical design of ECAP information as well as formats and channels specific to different parent groups.

Methods

The above-named issues will be explored with parents in four German cities as one element in our efforts to cover the spectrum of perspectives. Based on a mixed methods design, including qualitative and quantitative assessments, the first year serves to prepare focus groups, a piloted focus group guide, a short standardized survey adapted from the European Health Literacy Project, recruitment channels, and the recruitment of participants. After conducting 20 focus groups in the second year, data will be analyzed via a constant comparison method in the third year. Based on this, practice implications on channels (ie, Where?), formats (ie, How?), and contents (ie, What?) of ECAP-specific information will be derived and discussed with parents and associated project partners before its dissemination to relevant ECAP actors (eg, childcare institutions and pediatricians).

Results

The study began with preselection of recruitment channels, drafting of recruitment and study information for potential participants, and agreement on a first full version of the guideline. Then, a detailed contact list was compiled of health professionals, administrative and social institutions, and relevant social media channels (N=386) to be approached for assistance in contacting parents. The recruitment was postponed due to COVID-19 and will start in January 2021.

Conclusions

ECAP is a relevant example for assessing how users (ie, parents) handle not only health information but the various and continuous changes, uncertainties, and controversies attached to it. So far, it is unclear how parents implement the respective scientific recommendations and expert advice, which is why this study aims to inform those who communicate with parents about ECAP information.

International Registered Report Identifier (IRRID)

PRR1-10.2196/25474

Practical Advice Regarding the Reliability of the Patient Educational Materials Assessment Tool for Health Educators

The quality of patient education materials is an important issue for health educators, clinicians, and community health workers. We describe a challenge achieving reliable scores between coders when using the Patient Educational Materials Assessment Tool (PEMAT) to evaluate farmworker health materials in spring 2020. Four coders were unable to achieve reliability after three attempts at coding calibration. Further investigation identified improvements to the PEMAT codebook and evidence of the difficulty of achieving traditional interrater reliability in the form of Krippendorff's alpha. Our solution was to use multiple raters and average ratings to achieve an acceptable score with an intraclass correlation coefficient. Practitioners using the PEMAT to evaluate materials should consider averaging the scores of multiple raters as PEMAT results otherwise may be highly sensitive to who is doing the rating. Not doing so may inadvertently result in the use of suboptimal patient education materials.