Readability assessment of Internet-based patient education materials related to acoustic neuromas

Quality, Readability, and Suitability of Hearing Health-Related Materials: A Descriptive Review

Objectives The objective of this descriptive review was to determine the quality, readability, and suitability of ear and hearing health information and materials for patients and their significant others. Method A literature search was conducted between August 2018 and April 2019 in the databases CINAHL Complete, MEDLINE, and PsychInfo. Inclusion and exclusion criteria were used to shortlist studies. Data regarding quality, suitability, and readability were extracted from the included studies. Data were assessed qualitatively. Results There were 34 studies included in this review. Of those, eight examined quality, 33 assessed readability, and four investigated the suitability of materials. The range of materials assessed included diagnostic reports, patient education materials (PEMs), patient-reported outcome measures, and websites. Quality elements were examined in studies focusing on website information. Findings indicated that most websites were of poor quality. Suitability was examined in studies focusing on PEMs such as hearing aid user guides. Findings indicated that most of the existing materials were not suitable for the intended populations. The reading grade level of information across all four categories was found to be higher than the recommended fifth or sixth reading grade level for health-related materials. Revisions of some diagnostic reports and PEMs showed that improvements are possible. Conclusions This review suggests that ear- and hearing-related materials generally have lower quality and suitability with higher readability (more difficult to read). Development of materials that are suitable, of high quality, and at the appropriate readability levels is required to improve accessibility of ear- and hearing-related materials.

Differences in Perceptions of Health Information Between the Public and Health Care Professionals: Nonprobability Sampling Questionnaire Survey

Background

In the new media age, the public searches for information both online and offline. Many studies have examined how the public reads and understands this information but very few investigate how people assess the quality of journalistic articles as opposed to information generated by health professionals.

Objective

The aim of this study was to examine how public health care workers (HCWs) and the general public seek, read, and understand health information and to investigate the criteria by which they assess the quality of journalistic articles.

Methods

A Web-based nonprobability sampling questionnaire survey was distributed to Israeli HCWs and members of the public via 3 social media outlets: Facebook, WhatsApp, and Instagram. A total of 979 respondents participated in the online survey via the Qualtrics XM platform.

Results

The findings indicate that HCWs find academic articles more reliable than do members of the general public (44.4% and 28.4%, respectively, P<.001). Within each group, we found disparities between the places where people search for information and the sources they consider reliable. HCWs consider academic articles to be the most reliable, yet these are not their main information sources. In addition, HCWs often use social networks to search for information (18.2%, P<.001), despite considering them very unreliable (only 2.2% found them reliable, P<.001). The same paradoxes were found among the general public, where 37.5% (P<.001) seek information via social networks yet only 8.4% (P<.001) find them reliable. Out of 6 quality criteria, 4 were important both to HCWs and to the general public.

Conclusions

In the new media age where information is accessible to all, the quality of articles about health is of critical importance. It is important that the criteria examined in this research become the norm in health writing for all stakeholders who write about health, whether they are professional journalists or citizen journalists writing in the new media.

Quality and readability of online patient information regarding sclerotherapy for venous malformations

BACKGROUND

Patients often use the internet as a source of information about their condition and treatments. However, this information is unregulated and varies in quality.

OBJECTIVE

To evaluate the readability and quality of online information for pediatric and adult patients and caregivers regarding sclerotherapy for venous malformations.

MATERIALS AND METHODS

"Venous malformation sclerotherapy" was entered into Google, and results were reviewed until 20 sites that satisfied predefined inclusion criteria were identified. Scientific and non-patient-focused web pages were excluded. Readability was assessed using the Flesch Reading Ease Score and American Medical Association reading difficulty recommendations and quality was assessed using Journal of the American Medical Association standards and assessing if the site displayed HONcode (Health on the Net Code) certification. Assessment of the breadth of relevant information was made using a predefined checklist.

RESULTS

Forty-nine search engine results were reviewed before 20 sites were identified for analysis. Average Flesch Reading Ease Score was 44 (range: 24.2-70.1), representing a "fairly difficult" reading level. None of the sites had a Flesch Reading Ease Score meeting the American Medical Association recommendation of 80-90. Only one site met all four Journal of the American Medical Association quality criteria (average: 2.1). None of the sites displayed a HONcode seal. The information most frequently found was: sclerotherapy is performed by radiologists, multiple treatments may be needed and surgery is an alternative treatment.

CONCLUSION

Online information regarding sclerotherapy for venous malformations is heterogeneous in quality and breadth of information, and does not meet readability recommendations for patient information. Radiologists should be aware of and account for this when meeting patients.

Patients' preferences for labor analgesic counseling: A qualitative analysis

BACKGROUND

The decision to use, or not use, neuraxial labor analgesia is complex, with both maternal and fetal considerations. Fears and concerns about neuraxial analgesia may influence analgesic decision-making. Little is known about patients' preferences for analgesic counseling. Therefore, the objectives of this qualitative study were to evaluate the sources of information used by patients, the timing and content of antepartum counseling about labor analgesia, and patients' preferences for such counseling.

METHODS

A semi-structured interview guide was developed. Survey domains included the sources of information used by patients, the timing and content of antenatal counseling, patients' perceptions of intrapartum analgesic counseling, and their preferences for analgesic counseling. Interviews occured on postpartum day 1. Interview transcripts were analyzed using content analysis.

RESULTS

The interviews of 45 women were analyzed. The Internet was the most frequently mentioned source of information (44%). The majority of women (78%) discussed analgesic options with their obstetric providers before delivery. All women were counseled intrapartum by an anesthesiologist, but several women commented on the difficulty of concentrating while laboring. Seventy-three percent of women stated that they would want counseling on analgesic options from their obstetric provider during the second or third trimester.

CONCLUSIONS

Women prefer to be counseled about labor analgesia by their obstetric providers, as they have an established relationship. Patients would prefer the counseling to occur during pregnancy, before the onset of labor. However, given the frequent use of the Internet, the content and quality of online materials should also be evaluated.

Quality and readability assessment of websites related to recurrent respiratory papillomatosis

Objective

Recurrent respiratory papillomatosis (RRP) is a rare disease for which a limited number of information sources for patients exist. The role of the Internet in the patient–physician relationship is increasing. More and more patients search for online health information, which should be of good quality and easy readable. The study aim was to investigate the quality and readability of English online health information about RRP.

Study Design

Quality and readability assessment of online information.

Methods

Relevant information was collected using three different search engines and seven different search terms. Quality was assessed with the DISCERN instrument. The Flesch Reading Ease Score (FRES) and average grade level (AGL) were determined to measure readability of the English websites.

Results

Fifty‐one English websites were included. The mean DISCERN score of the websites is 28.1 ± 9.7 (poor quality); the mean FRES is 41.3 ± 14.9 (difficult to read); and the mean AGL is 12.6 ± 2.3.

Conclusion

The quality and readability of English websites about RRP is alarmingly poor.

Level of Evidence

NA. Laryngoscope, 127:2293–2297, 2017

Understanding online health information: Evaluation, tools, and strategies

OBJECTIVE

Considering the status of the Internet as a prominent source of health information, assessing online health material has become a central issue in patient education. We describe the strategies available to evaluate the characteristics of online health information, including readability, emotional content, understandability, usability.

METHODS

Popular tools used in assessment of readability, emotional content and comprehensibility of online health information were reviewed. Tools designed to evaluate both printed and online material were considered.

RESULTS

Readability tools are widely used in online health material evaluation and are highly covariant. Assessment of emotional content of online health-related communications via sentiment analysis tools is becoming more popular. Understandability and usability tools have been developed specifically for health-related material, but each tool has important limitations and has been tested on a limited number of health issues.

CONCLUSION

Despite the availability of numerous assessment tools, their overall reliability differs between readability (high) and understandability (low). Approaches combining multiple assessment tools and involving both quantitative and qualitative observations would optimize assessment strategies.

PRACTICE IMPLICATIONS

Effective assessment of online health information should rely on mixed strategies combining quantitative and qualitative evaluations. Assessment tools should be selected according to their functional properties and compatibility with target material.

Are Online Zenker’s Diverticulum Materials Readable and Understandable?

Objective

Patients use a multitude of resources to learn about Zenker’s diverticulum (ZD). The objectives of this study were to assess the readability and understandability of online materials on ZD, evaluate them against the existing criteria, and investigate the relationship between readability and understandability.

Methods

The first 50 webpages from an online search for ZD were analyzed. Twenty-one webpages had materials intended for patients and were included in the study. The patient education materials (PEMs) were analyzed using 6 readability tools. Four individuals used the Patient Education Materials Evaluation Tool (PEMAT) to assess the understandability. Fleiss κ interrater reliability analysis determined consistency among the raters. Finally, Pearson correlation coefficient analyzed the relationship between readability and understandability.

Results

The reading grade level of the materials reviewed ranged from 10th to 16th grade while the understandability ranged from 31% to 74%. Correlation analysis demonstrated a strong negative correlation between readability and understandability ( r = −0.62, P < .05). Fleiss’ κ interrater reliability for the raters demonstrated substantial agreement between the 4 raters (κ = 0.64).

Conclusion

Online PEMs pertaining to ZD are written well above the recommended reading level. Materials written at a lower reading level are more understandable. A wide range of understandability exists among materials with identical reading grade levels.

Implications for Practice

Health care providers need to create new PEMs for ZD that are available online that are both readable and understandable. The PEMAT and readability formulas can provide a framework for authors to create these materials.

Readability of Educational Materials to Support Parent Sexual Communication With Their Children and Adolescents

Sexual communication is a principal means of transmitting sexual values, expectations, and knowledge from parents to their children and adolescents. Many parents seek information and guidance to support talking with their children about sex and sexuality. Parent education materials can deliver this guidance but must use appropriate readability levels to facilitate comprehension and motivation. This study appraised the readability of educational materials to support parent sexual communication with their children. Fifty brochures, pamphlets, and booklets were analyzed using the Flesch-Kincaid, Gunning Fog, and Simple Measure of Gobbledygook (SMOG) index methods. Mean readability grade-level scores were 8.3 (range = 4.5-12.8), 9.7 (range = 5.5-14.9), and 10.1 (range = 6.7-13.9), respectively. Informed by National Institutes of Health-recommended 6th to 7th grade levels and American Medical Association-recommended 5th to 6th grade levels, percentages falling at or below the 7.0 grade level were calculated as 38%, 12%, and 2% and those falling at or below the 6.0 grade level were calculated as 12%, 2%, and 0% based on the Flesch-Kincaid, Gunning Fog, and SMOG methods, respectively. These analyses indicate that the majority of educational materials available online to support parents' communication with their children about sex and sexuality do not meet the needs of many or most parents. Efforts to improve the accessibility of these materials are warranted.

Information quality and dynamics of patients' interactions on tonsillectomy web resources

Information technologies have drastically altered the way patients gather health-related information. By analysing web resources on tonsillectomy, we expose information quality and dynamics of patients' interactions in the online continuum. Readability was assessed using Flesch Reading Ease (FRE), Flesch Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning Fog Index (GFI). Comprehensibility and actionability were assessed using the Patient Education Materials Assessment Tool (PEMAT). Metrics of forums included author characteristics (level of disclosure, gender, age, avatar image, etc.), posts' motive (community support vs. medical information) and content (word count, emoticon use, number of replies, etc.). Analysis of 6 professional medical websites, of 10 health information portals, and of 3 discussion forums totalizing 1369 posts on 358 threads, from January 1, 2007 to December 31, 2014, reveals that online resources exceed understandability recommendations. Women were more present on online health forums (68.2% of authors disclosing their gender) and invested themselves more in their avatar. Authors replying were significantly older than authors of original posts (39.7 ± 0.8 years vs. 29.2 ± 0.9 years, p < 0.001). The degree of self-disclosure was inversely proportional to the requests for medical information (p < 0.001). Men and women were equally seeking medical information (men: 74.0%, women: 77.0%) and community support (men: 65.7%, women: 70.4%), however women responded more supportively (women 86.2%, men 59.1%, p < 0.001). The dynamics of patients' interactions used to overcome accessibility difficulties encountered is complex. This work outlines the necessity for comprehensible medical information to adequately answer patients' needs.

Readability and Understandability of Online Vocal Cord Paralysis Materials

Objective

Patients use several online resources to learn about vocal cord paralysis (VCP). The objective of this study was to assess the readability and understandability of online VCP patient education materials (PEMs), with readability assessments and the Patient Education Materials Evaluation Tool (PEMAT), respectively. The relationship between readability and understandability was then analyzed.

Study Design and Setting

Descriptive and correlational design.

Method

Online PEMs were identified by performing a Google search with the term “vocal cord paralysis.” After scientific webpages, news articles, and information for medical professionals were excluded, 29 articles from the first 50 search results were considered. Readability analysis was performed with 6 formulas. Four individuals with different educational backgrounds conducted understandability analysis with the PEMAT. Fleiss’s Kappa interrater reliability analysis determined consistency among raters. Correlation between readability and understandability was determined with Pearson’s correlation test.

Results

The reading level of the reviewed articles ranged from grades 9 to 17. Understandability ranged from 29% to 82%. Correlation analysis demonstrated a strong negative correlation between materials’ readability and understandability ( r = −0.462, P < .05).

Conclusion

Online PEMs pertaining to VCP are written above the recommended reading levels. Overall, materials written at lower grade levels are more understandable. However, articles of identical grade levels had varying levels of understandability. The PEMAT may provide a more critical evaluation of the quality of a PEM when compared with readability formulas. Both readability and understandability should be used to evaluate PEMs.

Informed consent conversations and documents: A quantitative comparison

BACKGROUND

Informed consent for clinical research includes 2 components: informed consent documents (ICDs) and informed consent conversations (ICCs). Readability software has been used to help simplify the language of the ICD, but to the authors' knowledge is rarely used to assess the language used during the ICC, which may influence the quality of informed consent. The current analysis was performed to determine whether length and reading levels of transcribed ICCs are lower than their corresponding ICDs for selected clinical trials, and to assess whether investigator experience affected the use of simpler language and comprehensiveness.

METHODS

The current study was a prospective study in which ICCs were audiorecorded at 6 institutions when families were offered participation in pediatric phase I oncology trials. Word count, Flesch-Kincaid Grade Level (FKGL), and Flesch Reading Ease score (FRES) of the ICCs were compared with corresponding ICDs, including the frequency with which investigators addressed 8 prespecified critical consent elements during the ICC.

RESULTS

Sixty-nine unique physician/protocol pairs were identified. Overall, ICCs contained fewer words (4677 vs 6364 words; P = .0016) and had a lower FKGL (6 vs 9.7; P ≤ .0001) and a higher FRES (77.8 vs 56.7; P<.0001) compared with their respective ICDs, but were more likely to omit critical consent elements, such as voluntariness (55%) and dose-limiting toxicities (26%). Years of investigator experience was not correlated with reliably covering critical elements or decreased linguistic complexity.

CONCLUSIONS

Clinicians use more understandable language during ICCs than the corresponding ICD, but appear to less reliably cover elements critical to fully informed consent. Efforts focused at providing communication training for clinician-investigators should be made to optimize the synergy between the ICD and the ICC.

Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students?

OBJECTIVE

To evaluate accuracy of content and readability level of English Wikipedia articles on cardiovascular diseases, using quality and readability tools.

METHODS

Wikipedia was searched on the 6 October 2013 for articles on cardiovascular diseases. Using a modified DISCERN (DISCERN is an instrument widely used in assessing online resources), articles were independently scored by three assessors. The readability was calculated using Flesch-Kincaid Grade Level. The inter-rater agreement between evaluators was calculated using the Fleiss κ scale.

RESULTS

This study was based on 47 English Wikipedia entries on cardiovascular diseases. The DISCERN scores had a median=33 (IQR=6). Four articles (8.5%) were of good quality (DISCERN score 40-50), 39 (83%) moderate (DISCERN 30-39) and 4 (8.5%) were poor (DISCERN 10-29). Although the entries covered the aetiology and the clinical picture, there were deficiencies in the pathophysiology of diseases, signs and symptoms, diagnostic approaches and treatment. The number of references varied from 1 to 127 references; 25.9±29.4 (mean±SD). Several problems were identified in the list of references and citations made in the articles. The readability of articles was 14.3±1.7 (mean±SD); consistent with the readability level for college students. In comparison, Harrison's Principles of Internal Medicine 18th edition had more tables, less references and no significant difference in number of graphs, images, illustrations or readability level. The overall agreement between the evaluators was good (Fleiss κ 0.718 (95% CI 0.57 to 0.83).

CONCLUSIONS

The Wikipedia entries are not aimed at a medical audience and should not be used as a substitute to recommended medical resources. Course designers and students should be aware that Wikipedia entries on cardiovascular diseases lack accuracy, predominantly due to errors of omission. Further improvement of the Wikipedia content of cardiovascular entries would be needed before they could be considered a supplementary resource.

Readability of Internet Information on Hearing: Systematic Literature Review

PURPOSE

This systematic literature review asks the following question: “ What is the readability of Internet information on hearing that people with hearing impairment and their significant others can access in the context of their hearing care?”

METHOD

Searches were completed in three databases: CINAHL, PubMed, and Scopus. Seventy-eight records were identified and systematically screened for eligibility: 8 records were included that contained data on the readability of Internet information on hearing that people with hear ing impairment and their significant others can access in the context of their hearing care.

RESULTS

Records reported mean readability levels from 9 to over 14. In other words, people with hearing impairment and their significant others need 9 to 14 years of education to read and understand Internet information on hearing that they access in the context of their hearing care.

CONCLUSION

The poor readability of Internet information on hearing has been well documented; it is time to focus on valid and sustainable initiatives that address this problem.

Sexuality Education Websites for Adolescents: A Framework-Based Content Analysis

The web has unique potential for adolescents seeking comprehensive sexual health information. As such, it is important to understand the nature, scope, and readability of the content and messaging provided by sexuality educational websites. We conducted a content analysis of 14 sexuality education websites for adolescents, based on the 7 essential components (sexual and reproductive health and HIV, relationships, sexual rights and sexual citizenship, pleasure, violence, diversity, and gender) of the International Planned Parenthood Framework for Comprehensive Sexuality Education. A majority of content across all sites focused on sexual and reproductive health and HIV, particularly pregnancy and STI prevention, and other information about STIs and HIV. No other topic comprised more than 10% of content coverage across a majority of sites. The authors found little discussion of gender issues, sexual rights, sexual diversity, or sexual violence. Most sites provided brief references to sexual pleasure, generally moderated with cautionary words. Language used implied a heterosexual female audience. Reading levels for most sites were above the 9th-grade level, with several at the college level. These findings have implications for enhancing online sexuality education and broadening the coverage of essential topics.

Readability Assessment of Internet-Based Patient Education Materials Related to Parathyroid Surgery

Objectives:

Patient education is critical in obtaining informed consent and reducing preoperative anxiety. Written patient education material (PEM) can supplement verbal communication to improve understanding and satisfaction. Published guidelines recommend that health information be presented at or below a sixth-grade reading level to facilitate comprehension. We investigate the grade level of online PEMs regarding parathyroid surgery.

Methods:

A popular internet search engine was used to identify PEM discussing parathyroid surgery. Four formulas were used to calculate readability scores: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Gunning Frequency of Gobbledygook (GFOG), and Simple Measure of Gobbledygook (SMOG).

Results:

Thirty web-based articles discussing parathyroid surgery were identified. The average FRE score was 42.8 (± 1 standard deviation [SD] 16.3; 95% confidence interval [CI], 36.6-48.8; range, 6.1-71.3). The average FKGL score was 11.7 (± 1 SD 3.3; 95% CI, 10.5-12.9; range, 6.1-19.0). The SMOG scores averaged 14.2 (± 1 SD 2.6; 95% CI, 13.2-15.2; range, 10.7-21.9), and the GFOG scores averaged 15.0 (± 1 SD 3.5; 95% CI, 13.7-16.3; range, 10.6-24.8).

Conclusion:

Online PEM on parathyroid surgery is written above the recommended sixth-grade reading level. Improving readability of PEM may promote better health education and compliance.