Assessing the Quality of YouTube's Incontinence Information after Cancer Surgery: An Innovative Graphical Analysis

BACKGROUND

Prostate and colorectum cancers rank among the most common cancers, and incontinence is a significant postsurgical issue affecting the physical and psychological well-being of cancer survivors. Social media, particularly YouTube, has emerged as a vital source of health information. While YouTube offers valuable content, users must exercise caution due to potential misinformation.

OBJECTIVE

This study aims to assess the quality of publicly available YouTube videos related to incontinence after pelvic cancer surgery.

METHODS

A search on YouTube related to "Incontinence after cancer surgery" was performed, and 108 videos were analyzed. Multiple quality assessment tools (DISCERN, GQS, JAMA, PEMAT, and MQ-VET) and statistical analyses (descriptive statistics and intercorrelation tests) were used to evaluate the characteristics and popularity, educational value, quality, and reliability of these videos, relying on novel graphical representation techniques such as Sankey and Chord diagrams.

RESULTS

Strong positive correlations were found among quality rating scales, emphasizing agreement. The performed graphical analysis reinforced the reliability and validity of quality assessments.

CONCLUSIONS

This study found strong correlations among five quality scales, suggesting their effectiveness in assessing health information quality. The evaluation of YouTube videos consistently revealed "high" quality content. Considering the source is mandatory when assessing quality, healthcare and academic institutions are reliable sources. Caution is advised with ad-containing videos. Future research should focus on policy improvements and tools to aid patients in finding high-quality health content.

Quality of patient decision aids to support the public making COVID-19 decisions: An online environmental scan

OBJECTIVE

To identify and appraise the quality of COVID-19 patient decision aids (PtDAs).

METHODS

We conducted an environmental scan of online publicly available COVID-19 PtDAs. Two reviewers independently searched and extracted data. We calculated median International Patient Decision Aid Standards (IPDAS) scores and proportion scoring > 70% on Patient Education Materials Information Tool (PEMAT) adequate for understandability and actionability.

RESULTS

Of 876 resources identified, 12 were PtDAs. Decisions focused on initial COVID-19 vaccination series (n = 9), location of care for elderly (n = 2), and social distancing (n = 1). All 12 PtDAs were written materials and two had accompanying videos. The median IPDAS score minimizing risk of biased decisions was 4 of 6 items (IQR 1, range 2-4). For PEMAT, 92% had adequate for understandability and none for actionability.

CONCLUSIONS

We identified few online publicly available COVID-19 PtDAs and none were about COVID-19 vaccination boosters or treatment. PtDAs scored poorly on actionability and none met all IPDAS criteria for minimizing risk of biased decisions.

PRACTICE IMPLICATIONS

PtDA developers for COVID-19 and future pandemics should ensure their PtDAs meet all IPDAS criteria for minimizing risk of bias, have adequate scores for actionability, and are disseminated in the A to Z inventory.

Designing Dietary Education Materials for People With Chronic Kidney Disease: Recommendations for Improving the Quality of Resources

OBJECTIVE

The aim of this study is to quantify the readability, actionability, understandability, and overall quality of dietary education resources designed for patients and published in the Journal of Renal Nutrition.

DESIGN AND METHODS

All patient education materials published in the "Patient Education" section of the journal from 2011 to 2021 were included. The readability, health literacy demand, and quality were evaluated using the Hemingway editor, Patient Education Materials Assessment Tool, and the Centers for Disease Control and Prevention Clear Communication Index (CDC CCI) respectively. Good quality materials were those with a reading grade level of ≤8; a Patient Education Materials Assessment Tool score of >70% (indicating materials were understandable and actionable), and a CDC CCI score >90%.

RESULTS

A total of 42 resources were evaluated. Most materials (92%) were written at an appropriate level of readability (median grade 5, interquartile range [IQR: 5-7). The median understandability score was 71% (IQR: 60-81); however, only half (52%) of the materials met the 70% benchmark. Materials published performed poorly for actionability with the median actionability score of 37% (IQR: 20-83), and only 29% met the benchmark score. Overall quality was scored as low, with a median CDC CCI score of 65%, and only 10% of materials met the benchmark score. Areas for improvement were identified including providing a clear purpose, and summary of important points, explaining numbers and how to perform calculations, and including at least one action to take. Future efforts to improve actionability need to use the active voice, directly address readers, explain how to act, and describe the steps required.

CONCLUSION

Patient education materials that are attentive to health literacy principles beyond readability may enhance patient engagement, confidence, and empowerment, and improve adherence to the kidney diet.

The reliability, functional quality, understandability, and actionability of fall prevention content in YouTube: an observational study

Background

Falls are common but dangerous in the elderly. More and more seniors are searching for healthcare information online. YouTube has become the world’s most popular video streaming platform. Albeit thousands of fall prevention videos are available on YouTube, their reliability, functional quality, understandability, and actionability have not been verified.

Methods

The top 300 watched videos on YouTube related to fall prevention were retrieved. After exclusion, all qualified sample videos were evaluated by three validated assessment instruments (the PEMAT scale, the HONCode scale, and the DISCERN instrument) regarding their reliability, functional quality, understandability, and actionability. Each video’s length, number of views/likes/comments, forms of expression, and the uploader’s profile were collected as well. The Wilcoxon rank sum test was performed for further analysis from the perspective of expression forms and uploaders’ identities.

Results

One hundred thirty-seven videos (45.67%) were qualified as sample videos, and individuals/organizations with medical backgrounds posted 54.01% of them. Most of the excluded videos (n = 163) were irrelevant (n = 91, 55.83%), and commercial (n = 52, 31.90%). The median video length for sample videos was 470 seconds. The DISCERN instrument indicated that 115 videos (83.94%) were of moderate to high overall quality. Medical practitioners and organizations gained the highest scores in functional quality and reliability (P < 0.05), while they also tended to use technical terms more often (mean = 3.15). The HONCode scale suggested a lack of traceability was common. The most popular and actionable form of expression was workout (n = 58, median score = 86.90, P < 0.05), while monolog and keynote presentations scored the highest in understandability (no significant difference between them). The PEMAT scale suggested videos uploaded by medical teams were the easiest to be understood (P = 0.011 and P < 0.001, respectively), whereas they were less actionable than those made by fitness trainers (P = 0.039 and P < 0.001, respectively).

Conclusions

Cooperation between the medical team and fitness trainers is expected for better health promotion. Plain language is advised, and sources should be provided. As for expression form, monolog or keynote presentations, plus workout clips, might be the most effective.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03330-x.

Assessment of patient education materials for age-related macular degeneration

Purpose

Age‐related macular degeneration (AMD) is a leading cause of vision loss. It is helpful for patients living with AMD to understand the prognosis, risk factors and management of their condition. Online education materials are a popular and promising channel for conveying this knowledge to patients with AMD. However, the quality of these materials—particularly with respect to qualities such as ‘understandability’ and ‘actionability’—is not yet known. This study assessed a collection of online materials about AMD based on these qualities of ‘understandability’ and ‘actionability’.

Methods

Online education materials about AMD were sourced through Google from six English‐speaking nations: Australia, New Zealand, USA, UK, Ireland and Canada. Three Australian/New Zealand trained and registered optometrists participated in the grading of the ‘understandability’ and ‘actionability’ of online education materials using the Patient Education Materials Assessment Tool (PEMAT).

Results

This study analysed a total of 75 online materials. The mean ‘understandability’ score was 74% (range: 38%–94%). The ‘understandability’ PEMAT criterion U11 (calling for a summary of the key points) scored most poorly across all materials. The mean ‘actionability’ score was 49% (range: 0%–83%). The ‘actionability’ PEMAT criterion A26 (using ‘visual aids’ to make instructions easier to act on) scored most poorly across all materials.

Conclusion

Most education materials about AMD are easy to understand, but difficult to act on, because of a lack of meaningful visual aids. We propose future enhancements to AMD education materials—including the use of summaries, visual aids and a habit tracker—to help patients with AMD improve their understanding of disease prognosis, risk factors and eye assessment schedule requirements.

Quality evaluation of patient educational resources for catheter ablation treatment of atrial fibrillation

AIMS 

The prevalence of atrial fibrillation (AF) is increasing rapidly with the growing utilization of catheter ablation (CA) as a treatment strategy. Education for individuals undertaking this procedure is diverse, with varying degrees of information provided and little standardization. Many individuals utilize the internet as an educational resource. However, there is limited regulation of online patient information. To evaluate the quality of web-based patient education resources for patients undergoing CA for AF.

METHODS AND RESULTS 

A cross-sectional observational study was performed to obtain all freely accessible online educational resources about CA for AF from inception until 1 October 2019. Search engines used: Google, Yahoo!, and Bing. The Patient Education Materials Assessment Tool (PEMAT) was used to evaluate the quality of web-based patient education materials and printable tools. The PEMAT score objectively measures both the understandability and actionability of educational material. A total of 17 websites and 15 printable sources were included in the analysis. Non-government organizations developed 19% of materials and 75% were created by private or university hospitals. Nineteen sources (59.4%) were rated as highly understandable: 9 websites (52.9%) and 10 printable tools (66.7%). Seven sources (21.9%) were rated as highly actionable: 6 (35.3%) websites and 1 (6.7%) printable tool.

CONCLUSION 

The overall understandability of educational CA material was high, whilst improvement of actionability is warranted. The addition of summaries, visual aids, and tools, such as checklists may improve quality. These findings have significant implications for the development of patient educational material for CA in AF.

A Multimetric Evaluation of Online Patient Educational Materials for Breast Implant-associated Anaplastic Large Cell Lymphoma

OBJECTIVE

To assess understandability, actionability, and readability of online patient educational materials (OPEM) related to breast implant-associated anaplastic large cell lymphoma (BIA-ALCL).

METHODS

Search volumes for query terms related to BIA-ALCL including "breast implant associated lymphoma," "breast implant associated anaplastic large cell lymphoma," and "BIA-ALCL" were analyzed in one-week increments and normalized to total Google search volume. The same terms were then queried using an online search engine to identify commonly accessed OPEM on this topic. Understandability and actionability of OPEM were evaluated using the Patient Education Materials Assessment Tool. Grade-level readability was determined using generalized estimating equations, with observations nested within readability metrics from each website. All interval estimates were calculated for 95% confidence.

RESULTS

Overall, 24 websites were identified based on search parameters. Of these websites, 11 (45.8%) met criteria for understandability, and 1 (4.2%) met criteria for actionability. Overall, readability ranged from 10.2 to 17.3 for all websites with an average grade level readability of 12.4; 0 websites were written at or below a sixth-grade reading level. Government websites had the highest average grade reading level at 14.0, followed by commercial websites at 13.2, nonprofit websites at 12.0, and then academic/hospital-based websites at 11.5.

CONCLUSION

The quality of available OPEM on BIA-ALCL is limited. Future development of OPEM should be designed with the goal of improving both comprehension and actionability to help reduce patient anxiety and unnecessary clinical appointments related to this disease.

YouTube as a source of information on penile prosthesis

The purpose of this study is to evaluate the quality of information on YouTube regarding penile prosthesis. We searched the term 'penile prosthesis' on YouTube recording the first 100 video links. Each video was analysed by three evaluators through the validate Patient Education Materials Assessment Tool (PEMAT), the DISCERN quality criteria for consumer health information and a misinformation assessment tool specifically created for this subject. All the videos were viewed 10'011'232 times. Median duration time was 5'13″ (IQR = 7'25″-1'58″). Sixty-eight per cent were nonsurgical videos and 32% of them contained surgical procedures. Most of the videos were uploaded by Private Companies (51%). Videos were primarily directed at patients (73 of 100 videos). The mean PEMAT understandability score was 57.8% ± 19.2%, the actionability score was 0% (IQR = 33%-0%). The median DISCERN score was 26 (IQR = 30-21). None of the videos provided a complete patients' information and the overall mean misinformation score was 3.21 ± 2.24. Based on our data, video's quality according to PEMAT and DISCERN score is mostly poor. Furthermore popularity does not correlate with the quality of the videos, and most of them fail to provide adequate patient's information.

Development and evaluation of an integrated diabetes-periodontitis nutrition and health education module

BACKGROUND

A good understanding of the bi-directional relationship between diabetes and periodontitis is essential to ensure the successful management of both diseases. This study aimed to develop and evaluate an integrated diabetes-periodontitis nutrition and health education module.

METHODS

The module was developed as an iterative and review process by five experts in nutrition and dietetics, periodontics, and dental public health. It consisted of three phases: (i) needs assessment on module contents and characteristics, (ii) module development and (iii) module evaluation by experts. Twelve healthcare professionals aged between 30 and 53 years (average 13.5 years of working experience) validated the module contents and its comprehensibility using the Patient Education Materials Assessment Tool for printable materials (PEMAT-P) and audio-visual materials (PEMAT-A/V). Scores of 0 (disagree) or 1 (agree) were given for sets of understandability and actionability statements and presented as a total percentage.

RESULTS

Seventeen infographic-flip charts and 13 short-videos were developed in the Malay language and grouped into four topics: (i) Introduction to Diabetes and Periodontitis, (ii) Diabetes and Periodontitis Care, (iii) Lifestyle Modification, and (iv) Myths and Facts. Flip charts were rated between 76-100% for understandability and 80-100% for actionability, while videos rated between 90-100% for understandability and 100% for actionability, respectively.

CONCLUSION

Overall, the newly developed module ranked high median scores for understandability and actionability. This finding reflects positive acceptance of the integrated module among the various healthcare professionals involved in managing patients with diabetes and periodontitis.

Use of Machine Learning Algorithms to Predict the Understandability of Health Education Materials: Development and Evaluation Study

BACKGROUND

Improving the understandability of health information can significantly increase the cost-effectiveness and efficiency of health education programs for vulnerable populations. There is a pressing need to develop clinically informed computerized tools to enable rapid, reliable assessment of the linguistic understandability of specialized health and medical education resources. This paper fills a critical gap in current patient-oriented health resource development, which requires reliable and accurate evaluation instruments to increase the efficiency and cost-effectiveness of health education resource evaluation.

OBJECTIVE

We aimed to translate internationally endorsed clinical guidelines to machine learning algorithms to facilitate the evaluation of the understandability of health resources for international students at Australian universities.

METHODS

Based on international patient health resource assessment guidelines, we developed machine learning algorithms to predict the linguistic understandability of health texts for Australian college students (aged 25-30 years) from non-English speaking backgrounds. We compared extreme gradient boosting, random forest, neural networks, and C5.0 decision tree for automated health information understandability evaluation. The 5 machine learning models achieved statistically better results compared to the baseline logistic regression model. We also evaluated the impact of each linguistic feature on the performance of each of the 5 models.

RESULTS

We found that information evidentness, relevance to educational purposes, and logical sequence were consistently more important than numeracy skills and medical knowledge when assessing the linguistic understandability of health education resources for international tertiary students with adequate English skills (International English Language Testing System mean score 6.5) and high health literacy (mean 16.5 in the Short Assessment of Health Literacy-English test). Our results challenge the traditional views that lack of medical knowledge and numerical skills constituted the barriers to the understanding of health educational materials.

CONCLUSIONS

Machine learning algorithms were developed to predict health information understandability for international college students aged 25-30 years. Thirteen natural language features and 5 evaluation dimensions were identified and compared in terms of their impact on the performance of the models. Health information understandability varies according to the demographic profiles of the target readers, and for international tertiary students, improving health information evidentness, relevance, and logic is critical.

The Patient Perspective: Evaluating the Accessibility of Transoral Robotic Surgery Online Resources

PURPOSE

The varied treatment options available to patients with HPV-positive oropharyngeal squamous cell carcinoma (OPSCC) can cause significant patient confusion. In particular, transoral robotic surgery (TORS) has become widely used for treatment of HPV-positive OPSCC. As patients commonly refer to the internet for additional information, we aim to evaluate the quality of online patient educational materials for TORS in comparison to other otolaryngology surgical procedures.

METHODS

The terms "transoral robotic surgery," "glossectomy," "thyroidectomy," and "neck dissection" were searched on Google. Flesch reading ease, Flesh-Kincaid Grade Level, MD review rates, and PEMAT understandability and actionability were assessed for each search term. Google trends was used to determine search interest for each term between May 2015 and May 2020.

RESULTS

Of the 30 TORS websites that met inclusion criteria, the average FRE and FKGL scores were 40.74 and 11.60 (that of an average high school senior). The FRE and FKGL scores for TORS were all statistically significantly lower than those of all comparator search terms (P < .05). Only 1 out of 30 TORS sites were MD reviewed, representing the lowest MD review rate across all search terms. Understandability and actionability scores for TORS were also the lowest across all search terms at 77.71% and 2.66%, respectively. TORS had the lowest search interest.

CONCLUSIONS

In comparison to other common otolaryngology procedures, TORS websites are the least patient friendly. As TORS becomes a more widely used procedure, it is critical that TORS websites become higher quality resources that patients can reliably reference.

Evaluating the readability, understandability, and quality of online materials about chest pain in children

An increasing number of individuals use the Internet to obtain health information. However, online health information is unregulated and highly variable. We aimed to assess the readability, understandability, and quality of online information available for "chest pain in children." This analysis was performed in January 2020, by inputting the search term "chest pain in children" into Google. The 180 search results were evaluated/categorized. The readability was assessed using the Flesch reading ease score, the Gunning FOG readability score, the Flesch-Kincaid grade level, the Coleman-Liau score, the Simple Measure of Gobbledygook readability score, the Fry readability score, and the automated readability index (ARI). The quality was assessed through the Journal of the American Medical Association (JAMA) benchmark criteria. The understandability was evaluated by the Patient Education Materials Assessment Tool (PEMAT) for this study. Sixty-five websites were analyzed (academic and hospital websites (n = 30), physicians and health information websites (n = 35)). Among all websites, the average reading grade level was 9.99. There was no statistical difference between the two groups for the average readability level (p: 0.645). The mean PEMAT score for all websites was 65.09%. There was no statistical difference between the two groups for the average PEMAT score (p: 0.945). For both groups, the understandability score was below 70%. The average JAMA benchmark score was 2.43 ± 1.06, with a statistically significant difference between the academic and hospital websites (2.07 ± 0.91) and physician and health information websites (2.74 ± 1.09, p: 0.009).Conclusion: The readability of online materials available for patients regarding "chest pain in children" was significantly higher than the grade 6 recommended by the National Institutes of Health. The current online health information related to pediatric chest pain may be too difficult for the average reader to read. The quality and understandability were not good for both groups. Improving the readability, understandability, and quality of pediatric health-related online materials has the potential to reduce parental anxiety, improve baseline medical knowledge, and even enhance the physician-parent alliance.

Search Trends and Quality of Online Resources Regarding Thyroidectomy

OBJECTIVE

Thyroidectomy is one of the most common procedures performed in head and neck surgery. The quality of online resources for thyroidectomy is unknown. We aim to evaluate search trends and online resource quality regarding thyroidectomy.

STUDY DESIGN

Cross-sectional analysis.

SETTING

Websites appearing on Google search.

METHODS

The first 30 Google websites for thyroidectomy were reviewed, excluding research, video, and restricted sites. Search patterns were obtained with Google Trends. Quality was measured by readability (Flesch Reading Ease and Flesch-Kinkaid Grade Level), understandability and actionability (Patient Education Materials Assessment Tool), and clinical practice guideline (CPG) compatibility. Fleiss kappa interrater reliability analysis was performed for 2 raters.

RESULTS

Twenty-one sites were evaluated. Search popularity for thyroidectomy has increased since 2004. Median reading ease was 42.2 (range, 15.4-62.7) on a scale from 1 to 100, with 100 indicating maximum readability. Median reading grade level was 12 (range, 7-16). Thyroidectomy resources were poorly understandable (median, 66%; range, 21%-88%) and actionable (median, 10%; range, 0%-60%). Median CPG compatibility was 4 out of 5 (range, 0-5). Interrater reliability ranged from substantial to moderate for understandability (0.78), actionability (0.57), and CPG compatibility (0.58), with P < .05 for all results.

CONCLUSION

Online resources about thyroidectomy vary in quality and reliability and are written at grade levels above the average reading level of the public. Providers should be aware of existing resources and work to create education resources that meet universal health literacy guidelines. The framework provided in this article may also serve as a guide and provide tangible steps that providers can take to help patients access care.

[Development and evaluation of educational material directed to fall prevention in pregnant women]

OBJECTIVE

One in four pregnant women has at least one fall during their pregnancy. The objective of this research was to develop a written educational material on fall prevention for pregnant women, promoting self-efficacy and the expectations of results based on the social cognitive theory. It was also intended to evaluate the validity of the content of the educational material, its understanding and actionability.

METHODS

This methodological study was carried out in two phases: the first was based on the development of educational material and its evaluation by two panels of experts (n=13). Secondly, the educational material was evaluated for organization, clarity, appearance, relevance, content, self-efficacy and expectations of results, using an evaluation sheet created by the researchers, as well as its understanding and actionability using the PEMAT instrument. Descriptive statistics are used for sociodemographic variables and the results of the evaluation instruments.

RESULTS

Recommendations on the organization, clarity of information, appearance in terms of image quality, relevance, self-efficacy, expectation of results and safety aspects were met. In the evaluation carried out by the content expert panel, an average of 87.8% and 91.6% was obtained in the areas of actionability and understanding of PEMAT-P, respectively.

CONCLUSIONS

This study develops an educational material with content validity, highly understandable and actionable on the falls incident, their causes and consequences, strategies to prevent them, the benefits of exercising and the recommendation of exercises to prevent falls during pregnancy.

Understandability, actionability, and readability of online patient education materials about diabetes mellitus

PURPOSE

The purpose of this study was to assess the understandability, actionability, and readability of online diabetes education materials.

SUMMARY

This was a descriptive study that identified printable diabetes education materials through an online search. Materials were included from the following sources: national organizations with materials approved by expert panels, corporations with materials subject to FDA approval, and not-for-profit organizations with inter-professional advisory boards to approve materials. Topics included were basic knowledge of diabetes, hypoglycemia, insulin, and blood sugar goals. Materials were excluded if they were non-printable, contained active links, had a publication date prior to January 2011, were greater than 2 pages in length, or were pediatric focused. Understandability and actionability of the patient education materials were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Descriptive statistics and inter-rater reliability analysis using the kappa statistic were utilized. Readability was assessed using the Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook (SMOG) formula. Pearson correlation coefficient was calculated to assess the relationship between reading grade level and PEMAT scores. In total, 25 websites were identified, 5 of which met the inclusion criteria; 13 patient education materials were included, PEMAT scoring revealed that 4 of these met the criteria for understandability and only 1 met the criteria for actionability. There was no correlation found between PEMAT scores and reading grade levels (Pearson correlation coefficient = -0.30, p = 0.325).

CONCLUSION

The majority of diabetes patient education materials reviewed scored poorly using the PEMAT. Future development of diabetes patient education materials should be designed with the goal of increasing understandability and actionability.

Implementation of an organisation-wide health literacy approach to improve the understandability and actionability of patient information and education materials: A pre-post effectiveness study

OBJECTIVE

Limited examples exist globally of coordinated, organisation-wide health literacy approaches to systematically improve the understandability and actionability of patient health information. Even fewer have been formally evaluated. The aim of this study was to use the Patient Education Materials Assessment Tool (PEMAT) to evaluate the effectiveness of an organisation-wide, evidence-based approach to improve the understandability and actionability of patient information materials in regional health service in New South Wales, Australia.

METHODS

Two independent raters (blinded to the document version) evaluated pre- and post-implementation versions of 50 randomly-selected patient information materials using the PEMAT, with differences in understandability and actionability analysed using paired samples tests.

RESULTS

Mean (±SD) overall scores for understandability increased significantly by 5% (95% CI 2-8; p = 0.002) up to 77%±10%, and mean actionability (±SD) increased significantly by 4% (95% CI 0-8; p = 0.046) up to 56%±22%.

CONCLUSION

These results demonstrate that organisation-wide approaches with standardised processes for staff to prepare, review and store written patient information and education materials can be successfully implemented to address the impacts and risks of low health literacy.

PRACTICE IMPLICATIONS

The success of this approach provides a framework for other health organisations to work in partnership with patients to make health information more understandable and actionable.

Assessing of the audiovisual patient educational materials on diabetes care with PEMAT

OBJECTIVE

The purpose of this study was to evaluate the understandability and actionability of audiovisual educational materials on diabetes in Korea using the Patient Education Materials Evaluation Tool (PEMAT), as well as determine the usefulness of these materials.

METHODS

A total of 85 audiovisual materials were collected from Korean websites of territory general hospitals, national health institutions, research associations, and major search engines relating to diabetes that were created between 2006 and 2015. Of these, 34 materials that met the inclusion and exclusion criteria were analyzed. Five trained researchers evaluated the materials independently.

RESULTS

More than half of the materials (58.8%) had been created by nongovernment organizations. Slightly more than half (n = 19) of the audiovisual materials were streaming-style animation. The average PEMAT score (58.5%) for these materials was moderate. Compared to "understandability" ratings (49.5%), "actionability" ratings were low (31.4%); indeed, fourteen materials had an actionability of 0%. The average usefulness score of the materials was 4.3 points out of a possible 7. There were few suitable audiovisual materials for patient education on diabetes.

CONCLUSIONS

These findings will be useful for developing new audiovisual educational materials for diabetes patients with high understandability, actionability, and usefulness.

A systematic evaluation of advance care planning patient educational resources

BACKGROUND

Advance care planning (ACP) conversations help individuals exercise autonomy and make informed decisions about their care. There are many ACP resources available to support the process of advance care planning, yet available resources not universally accepted and under-utilized in clinical practice.

OBJECTIVE

This study aims to systematically evaluate commonly available general ACP patient educational resources using a comprehensive approach to assess elements ranging from resource inclusion to health literacy.

DESIGN

Systematic evaluation of general ACP educational resources METHODS: ACP educational resources were systematically identified and evaluated for: Patient Education Materials Assessment Tool (PEMAT) score, Flesch-Kincaid reading ease and grade level, cost, languages available, and categories of content including Advance directive/Healthcare proxy, Physician Order for Life-Sustaining Treatment (POLST), Living will, values, preferences, and goals.

RESULTS

Educational resources (n = 20) were identified. These resources were classified as information resources, workbooks, interactive websites, or and activities/conversation guides. The average PEMAT understandability score was 86 (range 58-100) and the average actionability was 90 (range 40-100). Two resources met Flesch-Kincaid reading ease and grade level. Eight were available in English only, nine in two languages, and three in more than two languages. Three explicitly discussed completion of POLST forms. The majority of the resources adequately addressed the content of advanced directives and health care proxy, values, goals, and preferences for future care. The level of time commitment required to use each resource also varied considerably.

CONCLUSION

We present a comprehensive evaluation of resources to support ACP. Using the Transtheoretical Model, we provide a guide for the use of these educational resources based on patient readiness. This review allows clinicians to identify aspects of each resource that may be helpful in preparing older adults for future healthcare decsions. Our analysis has identified the need to continue to refine these educational resources so that they can reach a broad population with varying needs. Advance care planning (ACP) conversations help individuals exercise autonomy and make informed decisions about their care. There are many ACP educational resources available to support the process of advance care planning, yet available resources are not universally accepted and under-utilized in clinical practice. We present a comprehensive evaluation of 20 educational resources to support ACP. Educational resources were identified and evaluated for: Patient Education Materials Assessment Tool (PEMAT) score, Flesch-Kincaid reading ease and grade level, cost, languages available, and categories of content. Using the Transtheoretical Model, we provide a guide for the use of these educational resources based on patient readiness. This review allows clinicians to identify aspects of each resource that may be helpful in preparing older adults for future healthcare decisions. Our analysis has identified the need to continue to refine these educational resources so that they can reach a broad population with varying needs.

Interrater reliability of the Patient Education Materials Assessment Tool (PEMAT)

OBJECTIVE

To assess the interrater reliability (IRR) and usability of the Patient Education Materials Assessment Tool (PEMAT) and the relationship between PEMAT scores and readability levels.

METHODS

One hundred ten materials (80 print, 30 audiovisual) were evaluated, each by two raters, using the PEMAT. IRR was calculated using Gwet's AC1 and summarized across items in each PEMAT domain (understandability and actionability) and by material type. A survey was conducted to solicit raters' experience using the PEMAT. Readability of each material was assessed using the SMOG Index.

RESULTS

The median IRR was 0.92 for understandability and 0.93 for actionability across all relevant items, indicating good IRR. Eight PEMAT items had Gwet's AC1 values less than 0.81. PEMAT and SMOG Index scores were inversely correlated, with a Spearman's rho of -0.20 (p=0.081) for understandability and -0.15 (p=0.194) for actionability. While 92% of raters agreed the PEMAT was easy to use, survey results suggested specific items for clarification.

CONCLUSION

While the PEMAT demonstrates moderate to excellent IRR overall, amendments to items with lower IRR may increase the usefulness of the tool.

PRACTICE IMPLICATIONS

The PEMAT is a useful supplement to reading level alone in the assessment of educational materials.

Information quality and dynamics of patients' interactions on tonsillectomy web resources

Information technologies have drastically altered the way patients gather health-related information. By analysing web resources on tonsillectomy, we expose information quality and dynamics of patients' interactions in the online continuum. Readability was assessed using Flesch Reading Ease (FRE), Flesch Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning Fog Index (GFI). Comprehensibility and actionability were assessed using the Patient Education Materials Assessment Tool (PEMAT). Metrics of forums included author characteristics (level of disclosure, gender, age, avatar image, etc.), posts' motive (community support vs. medical information) and content (word count, emoticon use, number of replies, etc.). Analysis of 6 professional medical websites, of 10 health information portals, and of 3 discussion forums totalizing 1369 posts on 358 threads, from January 1, 2007 to December 31, 2014, reveals that online resources exceed understandability recommendations. Women were more present on online health forums (68.2% of authors disclosing their gender) and invested themselves more in their avatar. Authors replying were significantly older than authors of original posts (39.7 ± 0.8 years vs. 29.2 ± 0.9 years, p < 0.001). The degree of self-disclosure was inversely proportional to the requests for medical information (p < 0.001). Men and women were equally seeking medical information (men: 74.0%, women: 77.0%) and community support (men: 65.7%, women: 70.4%), however women responded more supportively (women 86.2%, men 59.1%, p < 0.001). The dynamics of patients' interactions used to overcome accessibility difficulties encountered is complex. This work outlines the necessity for comprehensible medical information to adequately answer patients' needs.