Online Patient Education Materials for Orthognathic Surgery Fail to Meet Readability and Quality Standards

Large language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources

BACKGROUND

The readability of online bariatric surgery patient education materials (PEMs) often surpasses the recommended 6th grade level. Large language models (LLMs), like ChatGPT and Bard, have the potential to revolutionize PEM delivery. We aimed to evaluate the readability of PEMs produced by U.S. medical institutions compared to LLMs, as well as the ability of LLMs to simplify their responses.

METHODS

Responses to frequently asked questions (FAQs) related to bariatric surgery were gathered from top-ranked health institutions. FAQ responses were also generated from GPT-3.5, GPT-4, and Bard. LLMs were then prompted to improve the readability of their initial responses. The readability of institutional responses, initial LLM responses, and simplified LLM responses were graded using validated readability formulas. Accuracy and comprehensiveness of initial and simplified LLM responses were also compared.

RESULTS

Responses to 66 FAQs were included. All institutional and initial LLM responses had poor readability, with average reading levels ranging from 9th grade to college graduate. Simplified responses from LLMs had significantly improved readability, with reading levels ranging from 6th grade to college freshman. When comparing simplified LLM responses, GPT-4 responses demonstrated the highest readability, with reading levels ranging from 6th to 9th grade. Accuracy was similar between initial and simplified responses from all LLMs. Comprehensiveness was similar between initial and simplified responses from GPT-3.5 and GPT-4. However, 34.8% of Bard's simplified responses were graded as less comprehensive compared to initial.

CONCLUSION

Our study highlights the efficacy of LLMs in enhancing the readability of bariatric surgery PEMs. GPT-4 outperformed other models, generating simplified PEMs from 6th to 9th grade reading levels. Unlike GPT-3.5 and GPT-4, Bard's simplified responses were graded as less comprehensive. We advocate for future studies examining the potential role of LLMs as dynamic and personalized sources of PEMs for diverse patient populations of all literacy levels.

What Is Health Literacy Among Orthognathic Surgery Patients?

BACKGROUND

Health literacy of orthognathic surgery patients has not been thoroughly evaluated.

PURPOSE

The purpose of this study was to estimate health literacy and identify risk factors associated with inadequate health literacy in orthognathic surgery patients.

STUDY DESIGN, SETTING, SAMPLE

A cross-sectional study was implemented utilizing patients ages 14-80 years who presented for orthognathic surgery evaluation between September 2021 and December 2022. Subjects were excluded from the study if they did not complete the orthognathic surgery evaluation, were not between the ages of 14-80 years old, or did not complete the Brief Health Literacy Screening Tool (BRIEF) questionnaire during intake. Subjects who have not undergone orthognathic surgery but completed the initial evaluation for orthognathic surgery were included in the study.

PREDICTOR VARIABLES

The predictor variables were a set of risk factors for inadequate health literacy: age, sex, primary language, race, estimated household income, and diagnosis.

MAIN OUTCOME VARIABLE

The main outcome variable was health literacy assessed using the BRIEF questionnaire. During intake, subjects completed the BRIEF questionnaire consisting of four questions scored on an ordinal scale of 1-5. Inadequate health literacy was defined as a BRIEF score ≤16.

COVARIATES

Not applicable.

ANALYSES

Bivariate and multivariate analyses were performed. P < .05 was considered statistically significant.

RESULTS

Of 150 patients presenting for orthognathic surgery, fifteen percent of patients had inadequate health literacy via the BRIEF test. The mean age of those with adequate health literacy was 27.9 years (standard deviation, ±12.5) compared to 18.5 years (standard deviation, ±5.7) for those with inadequate health literacy (P = <.001). After adjusting for sex, language, race, estimated household income, and diagnosis via multivariate analysis, increasing age was associated with decreased odds of inadequate health literacy (adjusted odds ratio = 0.81; confidence interval, 0.72-0.92; P = <.001).

CONCLUSION AND RELEVANCE

In the complex process of orthognathic surgery, it is essential to identify patients with inadequate health literacy that may require additional health literacy interventions. Ultimately, 15% of orthognathic surgery subjects had inadequate health literacy, and younger patients were the most susceptible as the odds of inadequate health literacy decreased with increasing age.

Quality assessment of available Internet information on early orthodontic treatment

BACKGROUND

This study aimed to evaluate the content, reliability, quality and readability of information on Internet websites about early orthodontic treatment.

METHODS

The "early orthodontic treatment" search term was individually entered into four web search engines. The content quality and reliability were reviewed with DISCERN, Journal of American Medical Association (JAMA), and Health on the Net code (HONcode) tools using the contents of websites meeting predetermined criteria. The readability of websites was evaluated with Flesch Reading Facilitate Score (FRES) and Flesch-Kincaid Grade Level (FKGL).

RESULTS

Eighty-six websites were suitable for inclusion and scoring of the 200 websites. 80.2% of websites belonged to orthodontists, 15.1% to multidisciplinary dental clinics and 4.7% to professional organizations. The mean DISCERN score of all websites (parts 1 and 2) was 27.98/75, ranging between 19 and 67. Professional organization websites had the highest scores for DISCERN criteria. Moreover, 45.3% of websites were compatible with JAMA's disclosure criterion, 7% with the currency criterion, 5.8% with the authorship criterion and 5.8% with the attribution criterion. Only three websites met all JAMA criteria, and these websites belonged to professional organizations. None of the websites had the HONcode logo. Mean FRES and FKGL were 47.6 and 11.6, respectively.

CONCLUSIONS

The quality of web-based information about early orthodontic treatment is poor, and readability is insufficient. More accurate and higher quality Internet sources are required on the web.

Evaluation of readability levels of online patient education materials for female pelvic floor disorders

Most women hesitate to seek help from healthcare providers as they find it difficult to share complaints of involuntary leakage or vaginal prolapse. Hence, they often refer to the websites of national and/or international bodies' patient education materials (PEMs), which are considered the most reliable sources. The crucial factor that determines their usefulness is their readability level, which makes them "easy" or "difficult" to read, and is recommended, not to exceed the sixth grade level. In this study, we aimed to assess the readability levels of Turkish translated PEMs from the websites of the International Urogynecological Association and the European Association of Urology and the PEMs originally written in Turkish from the website of the Society of Urological Surgery in Turkey. All the PEMs (n = 52) were analyzed by online calculators using the Atesman formula, Flesch-Kincaid grade level, and Gunning Fog index. The readability parameters, number of sentences, words, letters, syllables, and readability intervals of these methods were compared among the groups using the Kruskal-Wallis test, or ANOVA test, with post hoc comparisons where appropriate. The readability level of all PEMs is at least at an "averagely difficult" interval, according to both assessment methods. No significant differences were found among the PEM groups in terms of readability parameters and assessment methods (P > .05). Whether original or translated, international or national societies' PEMs' readability scores were above the recommended level of sixth grade. Thus, the development of PEMs needs to be revised accordingly by relevant authorities.

Qualitative assessment of available online patient resources for pulmonary embolism and deep vein thrombosis

OBJECTIVE

This study seeks to evaluate the quality and readability of freely available online patient information resources for deep vein thrombosis (DVT) and pulmonary embolism (PE).

METHODS

Internet searches were performed for five DVT and PE search terms in July 2020 across three search engines and two metasearch engines. Qualitative content analysis was performed. Readability was assessed using four validated instruments.

RESULTS

Two hundred fifty websites were identified of which 62 websites met inclusion criteria.Website structure and content were satisfactory (>50% overall score), accountability was mixed between DVT (47%) and PE (56%) sites, while interactivity was poor (<30%). On qualitative content analysis, anticoagulation (95.2%) was the most discussed treatment while the most discussed procedures were IVC filter placement for DVT and thrombolysis for PE. Overall readability was difficult with median level suitable for ages 14-18 years.

CONCLUSION

Freely available online DVT and PE patient information resources publish appropriate content but have very variable accountability and poor readability for the average patient.

Readability of patient education materials for bariatric surgery

INTRODUCTION

Bariatric surgery is a successful treatment for obesity, but barriers to surgery exist, including low health literacy. National organizations recommend patient education materials (PEM) not exceed a sixth-grade reading level. Difficult to comprehend PEM can exacerbate barriers to bariatric surgery, especially in the Deep South where high obesity and low literacy rates exist. This study aimed to assess and compare the readability of webpages and electronic medical record (EMR) bariatric surgery PEM from one institution.

METHODS

Readability of online bariatric surgery and standardized perioperative EMR PEM were analyzed and compared. Text readability was assessed by validated instruments: Flesch Reading Ease Formula (FRE), Flesch Kincaid Grade Level (FKGL), Gunning Fog (GF), Coleman-Liau Index (CL), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI), and Linsear Write Formula (LWF). Mean readability scores were calculated with standard deviations and compared using unpaired t-tests.

RESULTS

32 webpages and seven EMR education documents were analyzed. Webpages were overall "difficult to read" compared to "standard/average" readability EMR materials (mean FRE 50.5 ± 18.3 vs. 67.4 ± 4.2, p = 0.023). All webpages were at or above high school reading level: mean FKGL 11.8 ± 4.4, GF 14.0 ± 3.9, CL 9.5 ± 3.2, SMOG 11.0 ± 3.2, ARI 11.7 ± 5.1, and LWF 14.9 ± 6.6. Webpages with highest reading levels were nutrition information and lowest were patient testimonials. EMR materials were sixth to ninth grade reading level: FKGL 6.2 ± 0.8, GF 9.3 ± 1.4, CL 9.7 ± 0.9, SMOG 7.1 ± 0.8, ARI 6.1 ± 1.0, and LWF 5.9 ± 0.8.

CONCLUSION

Surgeon curated bariatric surgery webpages have advanced reading levels above recommended thresholds compared to standardized PEM from an EMR. This readability gap may unintentionally contribute to barriers to surgery and affect postoperative outcomes. Streamlined efforts are needed to create materials that are easier to read and comply with recommendations.

Patient education information material assessment criteria: A scoping review

BACKGROUND

Patient education information material (PEIM) is an essential component of patient education programs in increasing patients' ability to cope with their diseases. Therefore, it is essential to consider the criteria that will be used to prepare and evaluate these resources.

OBJECTIVE

This paper aims to identify these criteria and recognize the tools or methods used to evaluate them.

METHODS

National and international databases and indexing banks, including PubMed, Scopus, Web of Science, ProQuest, the Cochrane Library, Magiran, SID and ISC, were searched for this review. Original or review articles, theses, short surveys, and conference papers published between January 1990 and June 2022 were included.

RESULTS

Overall, 4688 documents were retrieved, of which 298 documents met the inclusion criteria. The criteria were grouped into 24 overarching criteria. The most frequently used criteria were readability, quality, suitability, comprehensibility and understandability.

CONCLUSION

This review has provided empirical evidence to identify criteria, tools, techniques or methods for developing or evaluating a PEIM. The authors suggest that developing a comprehensive tool based on these findings is critical for evaluating the overall efficiency of PEIM using effective criteria.

A critical appraisal of web-based information on shoulder pain comparing biomedical vs. psychosocial information

BACKGROUND

Websites have become a primary way for patients to access health-related information, which allows patients to not only understand their condition better but also to engage in better decision making with their health care provider. However, this can be a double-edged sword, as information patients access may not be of high quality, easily readable, or could be biased based on website authorship. This study examines the readability and content quality of common websites about shoulder pain, with a specific focus on biomedical vs. psychosocial information.

METHODS

The Flesch reading-ease score (FRES) and Flesch-Kincaid grade level (FKGL) instruments were used to assess website readability. Health on the net code of conduct (HONcode) certification and the DISCERN tool were used to evaluate the reliability and quality of information. Lastly, shoulder-specific content quality and focus was gauged using a Shoulder-specific Website Appraisal Tool (SWAT) created for this study. A systematic search protocol was used to identify popular shoulder injury websites. The websites were identified using 5 search terms (shoulder pain, shoulder muscle pain, shoulder impingement, shoulder rotator cuff pain, and shoulder pain diagnosis) across 5 English-speaking regions (United States, Canada, United Kingdom, Australia, and New Zealand) using the Google search engine. The top 10 websites for each term and region combination were included and combined, yielding 41 original websites for appraisal; several (6) websites were omitted as duplicates, behind paywalls, or nontext (video) for a total of 35 websites appraised.

RESULTS

On average, the FRES shoulder pain websites readability was 55.37, which is categorized as "fairly difficult" to read, and a seventh- or eighth-grade reading level based on the FKGL. For trustworthiness and bias, 57% (20 of 35) of the websites were HONcode certified. The quality of health care information using the DISCERN score averaged 50.92%. Examining shoulder-specific content quality, the average SWAT score was 10.54 out of 14 possible points, with only 1 website reporting information on psychosocial aspects of shoulder pain.

CONCLUSIONS

This study suggests that shoulder pain websites present information that is at too high of a reading level for public consumption, has a strong bias toward biomedical causes of shoulder pain, and would benefit from implementation of website screening tools to improve evidence-based content.

How readable are orthognathic surgery consent forms?

BACKGROUND/OBJECTIVE

The valid consent process for medical intervention requires the disclosure of information in a format that is easily understandable by the patient. The aim of this investigation was to assess the readability of orthognathic surgery informed consent forms (OSICFs).

METHODS

An online search methodology was conducted to identify OSICFs for analysis. The forms that satisfied inclusion/exclusion criteria were evaluated according to a standardised protocol. The readability of the content was assessed using three validated tools: the Simple Measure of Gobbledegook (SMOG) score, Flesch-Kincaid Grade-Level (FKGL) score and Flesch Reading Ease (FRE) score.

RESULTS

Most of the 26 evaluated OSICFs were sourced from websites within the United States (69.2%) and from oral and maxillo-facial surgery practices (76.9%). Two of the assessed forms were template OSICFs available from oral and maxillo-facial professional societies to its members. The scores from the three tools found that the content of 84.6% to 92.3% of the forms were "difficult" to read. The mean (SD) SMOG score for all evaluated OSICFs was 12.31(2.22) [95% CI: 11.42 to 13.21]. The SMOG and FKGL scores were closely correlated (r=0.99, P < 0.0001; 95% CI: 0.9864 to 0.9973). There was no association between SMOG scores and the number of words contained within each consent form (r=-0.047;95% CI: -0.44 to 0.36).

CONCLUSIONS

The OSICFs surveyed in this investigation failed to meet recommended readability levels. A significant number of patients are not likely to understand the information contained within the forms. Orthodontists are advised that poor literacy skills of their patients may preclude them from validly consenting to orthognathic surgery treatment procedures.

Infodemiology for oral health and disease: A scoping review

INTRODUCTION

Increasing affordability, accessibility and penetration of internet services worldwide, have substantially changed the ways of gathering health-related information. This has led to the origin of concept infodemiology that allows the information to be collected and analysed in near real time. Globally, oral diseases affect nearly 3.5 billion people; thus, volume and profile of oral health searches would help in understanding specific community dental needs and formulation of pertinent oral health strategies.

AIM

To review the published literature on infodemiological aspects of oral health and disease.

METHODOLOGY

This scoping review was conducted in accordance with PRISMA-ScR guidelines. Electronic search engines (Google Scholar) and databases (PubMed, Web of science, Scopus) were searched from 2002 onwards.

RESULTS

Thirty-eight articles were included in this review. The infodemiological studies for oral health and disease were mainly used in two domains. Out of 38 articles, 24 accessed the quality of available online information and 15 studied online oral health-related information seeking behaviour.

CONCLUSION

The most commonly searched oral diseases were toothache, oral cancer, dental caries, periodontal disease, oral maxillofacial surgical procedures and paediatric oral diseases. Most of the studies belonged to developed countries and Google was the most researched search engine.

Quality-Based Evaluation of Patient-Facing Online Education Materials Regarding Facial Filler Procedures

Injectable facial fillers have become tremendously more popular in recent years, and the Internet offers a proportional amount of consumer-facing educational material. This study sought to explore the quality of these online materials. The top 20 Web sites offering educational materials about facial filler were identified via Google search and sorted by source: Medical Professional Boards, Hospitals and Providers, Medical News and Reference, and Fashion. The materials were assessed for overall quality with the validated DISCERN instrument. The authors also assessed understandability and actionability (Patient Education Material Assessment Tool - PEMAT), accuracy, comprehensiveness, and readability (Flesch-Kincaid Grade Level and Flesch Reading Ease). The mean DISCERN score was 46.9 ± 7.6, which is considered "fair" quality educational material; above "poor," but below "good" and "excellent." Understandability and actionability scores were low, particularly with respect to visual aids. The materials were generally accurate (76-99%), but scored poorly in comprehensiveness, as 15% failed to mention any risks/adverse effects and only 35% mentioned cost. On average, readability was at an 11th grade level, far more complex than ideal (< 6th grade level). Information disseminated from seemingly reputable sources such as professional boards and hospitals/providers were not of higher quality or superior in any of the above studied domains. In conclusion, online educational materials related to injectable facial fillers are of subpar quality, including those from academic and professional organizations. Visual aids were particularly weak. The facial rejuvenation community should make a concerted effort to set a higher standard for disseminating such information.

An assessment of the treatment information contained within the websites of direct-to-consumer orthodontic aligner providers

BACKGROUND

The use of direct-to-consumer (DTC) orthodontic aligners is growing. The purpose of this study was to evaluate the quality of information contained within the websites of DTC orthodontic aligner providers.

METHODS

Websites that satisfied inclusion criteria were assessed for the presence of specific treatment-related features. Four validated quality of information instruments and readability tools were used to evaluate the content of the websites. Descriptive statistical analyses and intra-examiner reliability tests were performed.

RESULTS

Twenty-one websites were evaluated. Few websites referred to the need for pre-treatment dental health (38.1%) and indefinite post-treatment retention (23.8%). Most websites (95.2%) were categorized as either 'poor' or 'very poor' according to their DISCERN scores. DISCERN items regarding information bias and treatment risks scored particularly poorly. Cohen's kappa intra-examiner reliability testing for DISCERN scores was 0.81-0.89. Three websites honoured two out of the four Journal of the American Medical Association benchmarks and 18 honoured one benchmark only. The readability scores indicated 'difficult to read' content.

CONCLUSIONS

The quality of information contained within the websites of DTC orthodontic aligner providers is poor. Patient consent for DTC aligner treatment based solely on the information contained within the websites is likely to be invalid.

Readability of online patient education material for the novel coronavirus disease (COVID-19): a cross-sectional health literacy study

Objectives

The internet has become one of the most important resources for the general population when searching for healthcare information. However, the information available is not always suitable for all readers because of its difficult readability. We sought to assess the readability of online information regarding the novel coronavirus disease (COVID-19) and establish whether they follow the patient educational information reading level recommendations.

Study design

This is a cross-sectional study.

Methods

We searched five key terms on Google and the first 30 results from each of the searches were considered for analysis. Five validated readability tests were utilized to establish the reading level for each article.

Results

Of the 150 gathered articles, 61 met the inclusion criteria and were evaluated. None (0%) of the articles met the recommended 5th to 6th grade reading level (of an 11-12-year-old). The mean readability scores were Flesch Reading Ease 44.14, Flesch-Kincaid Grade Level 12.04, Gunning-Fog Index 14.27, Simple Measure of Gobbledygook SMOG Index 10.71, and Coleman-Liau Index 12.69.

Conclusions

Online educational articles on COVID-19 provide information too difficult to read for the general population. The readability of articles regarding COVID-19 and other diseases needs to improve so that the general population may understand health information better and may respond adequately to protect themselves and limit the spread of infection.

•

Online health information regarding COVID-19 is too difficult to read and understand.

•

Based on past research, health articles too difficult to understand may cause misinformation to spread and public panic.

•

The readability of COVID-19 needs to improve so that the general population may respond adequately to protect themselves and limit the spread of infection.

Quality assessment of patient information on orthognathic surgery on the internet

OBJECTIVE

The aim of this study was to assess the quality of online information for patients on orthognathic surgery.

MATERIALS AND METHODS

A selection of search terms specific for orthognathic surgery was chosen and 150 websites were identified using the Internet search engines Google, Yahoo and Bing. Irrelevant websites were excluded. The remaining websites were assessed with a modified Ensuring Quality Information for Patients (EQIP) tool. EQIP evaluates the quality of medical patient information by measuring the three key aspects of content, structure, and identification data.

RESULTS

48 relevant websites were identified. EQIP values ranged between 2 and 28 (median 13.65). While 37 of the 48 websites described details of the surgical procedures, only 13 mentioned possible risks and complications of the surgery. No differences were found between the websites of private practices, dentists and public hospitals, universities, or others (p = 0.66). Websites found by Google had a significantly lower EQIP score compared with Yahoo and Bing (11.12 vs. 16.60 for Yahoo and 16.23 for Bing; p = 0.012). The better the rank of the website, the higher the EQIP score (r = -0.411, p = 0.004).

CONCLUSIONS

The results of this study reflected a large variation of quality of information on orthognathic surgery on the Internet. Therefore, surgeons must be aware that they might be confronted with unrealistic expectations of patients, who may underestimate the potential risks and drawbacks of orthognathic surgery.

Evaluation of the Quality of Printed Online Education Materials in Cosmetic Facial Plastic Surgery

Importance: In the modern-day health care environment, it is expected that a large proportion of patients will utilize the Internet to gather health care-related information. However, online sources are often not subjected to adequate quality control measures. Objective: To evaluate the quality of printed online education materials (POEMs) pertaining to elective facial cosmetic surgery. Design, Setting, and Participants: This is a descriptive correlational study. Independent Internet searches for the terms "rhinoplasty," "rhytidectomy," "blepharoplasty," and "browlift" were carried out using the Google search engine. The top 20 web resources for each term were analyzed. Main Outcomes and Measures: All web resources were analyzed for HONcode certification. The DISCERN instrument was then used to determine reliability and specificity of information, whereas readability was assessed using the Flesch Reading Ease (FRE) score and the Flesch-Kincaid Grade Level (FKGL). Results: Of the resulting 80 web resources, 30% (n = 24) were HONcode certified. None of the surgical society web resources (n = 12) were HONcode certified, and only 42% (n = 8) of web resources associated with academic institutions were HONcode certified. Out of a maximum DISCERN score of 80, the mean (standard deviation [SD]) scores were 48 (11) for rhinoplasty, 50 (11) for rhytidectomy, 47 (11) for blepharoplasty, and 45 (10) for browlift. The mean DISCERN scores of the HONcode-certified web resources did not differ significantly from scores of uncertified web resources (p = 0.069). There was no significant difference in overall mean DISCERN scores for the different web resource categories. The mean (SD) FRE score was 45.7 (11.9), whereas the mean FKGL score was 10.7 (1.9). HONcode-certified web resources had significantly higher FRE scores (p = 0.028) and significantly lower FKGL scores (p = 0.0003). Conclusions and Relevance: A significant proportion of POEMs regarding elective cosmetic facial plastic surgery remains uncertified and of inadequate quality. This can negatively impact the informed decision-making process and increase the risk of poor outcomes and lower satisfaction rates.

Evaluation of quality and reliability of websites about orthognathic surgery using Google Trends™ application

Objectives:

Orthognathic surgery is the only treatment option for some cases in orthodontics. Two main goals were aimed in this study. The first goal was to determine the top three keywords that Google searchers look for “orthognathic surgery” using the Google Trends™ application. The second was to assess the quality and reliability of the webpages yielded by searching those four keywords on Google.

Materials and Methods:

Google Trends was searched using the keyword “Orthognathic Surgery.” Then, Google was searched using keywords: “Orthognathic surgery” and the other first three keywords. An oral surgeon and an orthodontist analyzed the quality and reliability of the websites using the DISCERN assessment tool.

Results:

The correlation between evaluators was low. Forty-six of the 70 websites were the websites of different oral and maxillofacial clinics and orthodontic clinics. The percentage of websites was 11% for personal doctor websites, 10% for academic journals, 7% for institutes, 6% for orthodontic clinics, and 1.4% for dental clinics. Overall rating scores of the websites for researchers were 2.41±1.35 for oral surgeon and 1.99±1.49 for orthodontist.

Conclusions:

The reliability and quality of the information in all of the websites was very low. The data on the websites did not have any scientific review processes such as review-peer review.

Reading Level of Online Patient Education Materials From Major Obstetrics and Gynecology Societies

OBJECTIVE

To assess whether the readability of online patient education materials provided by eight nationally recognized obstetrics and gynecology societies is in accordance with the recommended 6th-grade reading level outlined by the American Medical Association, National Institute of Health, and United States Department of Health and Human Services.

METHODS

An analysis of 410 online patient education materials from the American Association of Gynecologic Laparoscopists, the American College of Obstetricians and Gynecologists, the American Society for Reproductive Medicine, the American Urogynecologic Society, the Association of Reproductive Health Professionals, the Society of Gynecologic Oncology, the Society for Maternal-Fetal Medicine, and Voices for Pelvic Floor Disorders was completed, and the readability scores using the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Scale, and Simple Measure of Gobbledygook readability scales were calculated. These scales are used to assess print and web-based written material across a wide range of medical specialties and are endorsed by the National Institutes of Health. All four scales are computer-based assessments of readability that consider word count, number of syllables, and length of sentences when calculating a score that corresponds to grade level.

RESULTS

Majority of the patient education materials across all eight obstetrics and gynecology societies had readability scores above the recommended 6th-grade reading level. The average reading level for the 69 obstetrics-related articles ranged from 9th to 12th grade. The mean grade level for the 341 gynecology articles had a similar range across the four readability scales.

CONCLUSION

Online patient education materials provided by major obstetrics and gynecology societies do not currently adhere to recommended readability guidelines. Continued efforts to provide accessible and informative patient education materials is recommended to help improve health literacy for women.

A Revised Model of Trust in Internet-Based Health Information and Advice: Cross-Sectional Questionnaire Study

BACKGROUND

The internet continues to offer new forms of support for health decision making. Government, charity, and commercial websites increasingly offer a platform for shared personal health experiences, and these are just some of the opportunities that have arisen in a largely unregulated arena. Understanding how people trust and act on this information has always been an important issue and remains so, particularly as the design practices of health websites continue to evolve and raise further concerns regarding their trustworthiness.

OBJECTIVE

The aim of this study was to identify the key factors influencing US and UK citizens' trust and intention to act on advice found on health websites and to understand the role of patient experiences.

METHODS

A total of 1123 users took part in an online survey (625 from the United States and 498 from the United Kingdom). They were asked to recall their previous visit to a health website. The online survey consisted of an updated general Web trust questionnaire to account for personal experiences plus questions assessing key factors associated with trust in health websites (information corroboration and coping perception) and intention to act. We performed principal component analysis (PCA), then explored the relationship between the factor structure and outcomes by testing the fit to the sampled data using structural equation modeling (SEM). We also explored the model fit across US and UK populations.

RESULTS

PCA of the general Web trust questionnaire revealed 4 trust factors: (1) personal experiences, (2) credibility and impartiality, (3) privacy, and (4) familiarity. In the final SEM model, trust was found to have a significant direct effect on intention to act (beta=.59; P<.001), and of the trust factors, only credibility and impartiality had a significant direct effect on trust (beta=.79; P<.001). The impact of personal experiences on trust was mediated through information corroboration (beta=.06; P=.04). Variables specific to electronic health (eHealth; information corroboration and coping) were found to substantially improve the model fit, and differences in information corroboration were found between US and UK samples. The final model accounting for all factors achieved a good fit (goodness-of-fit index [0.95], adjusted goodness-of-fit index [0.93], root mean square error of approximation [0.50], and comparative fit index [0.98]) and explained 65% of the variance in trust and 41% of the variance in intention to act.

CONCLUSIONS

Credibility and impartiality continue to be key predictors of trust in eHealth websites. Websites with patient experiences can positively influence trust but only if users first corroborate the information through other sources. The need for corroboration was weaker in the United Kingdom, where website familiarity reduced the need to check information elsewhere. These findings are discussed in relation to existing trust models, patient experiences, and health literacy.

A Readability Analysis of Online Cardiovascular Disease-Related Health Education Materials

Background:

Online cardiovascular health materials are easily accessible with an Internet connection, but the readability of its content may limit practical use by patients.

Objective:

The goal of our study was to assess the readability of the most commonly searched Internet health education materials for cardiovascular diseases accessed via Google.

Methods:

We selected 20 commonly searched cardiovascular disease terms: aneurysm, angina, atherosclerosis, cardiomyopathy, congenital heart disease, coronary artery disease, deep vein thrombosis, heart attack, heart failure, high blood pressure, pericardial disease, peripheral arterial disease, rheumatic heart disease, stroke, sudden death, valvular heart disease, mini-stroke, lower extremity edema, pulmonary embolism, and exertional dyspnea. Terms were selected on Google and selected up to 10 results in order of presentation in the search results by reviewing a maximum of 15 pages of Google search results specifically providing education toward patients to yield 196 total patient education articles.

Key Results:

All readability measures assessing grade level measures found the 196 articles were written at a mean 10.9 (SD = 1.8) grade reading level. Moreover, 99.5% of the articles were written beyond the 5th- to 6th-grade level recommended by the American Medical Association.

Conclusions:

Given the prominent use of online patient education material, we consider readability as a quality metric that should be evaluated prior to online publication of any health education materials. Further study of how to improve the readability of online materials may enhance patient education, engagement, and health outcomes. [HLRP: Health Literacy Research and Practice. 2019;3(2):e74–e80.]

Plain Language Summary:

Patients often use Google as a tool for understanding their medical conditions. This study examined the readability of articles accessed via Google for commonly searched cardiovascular diseases and found all articles were written above reading grade levels appropriate for patients. We hope this study will promote the importance of ensuring that online patient education articles are written at appropriate reading levels.