Readability of patient education materials from professional societies in radiation oncology: are we meeting the national standard?

Validation testing of a language translation device for suitability in assisting Australian radiation therapists to communicate with Mandarin-speaking patients

Introduction

Clear, timely communication between practitioners and patients is key in ensuring equitable access to health services and optimal care. Australia's linguistically diverse population adds complexity to healthcare provision. This paper describes a validation study to assess clinical suitability of a language translation device, intended for use with Mandarin speaking patients undergoing radiotherapy (RT).

Materials and methods

After a comprehensive device selection process, common phrases used in RT practice were curated within one clinical center and translated by interpreters. Phrases were categorized by conversation type and readability (according to Flesch-Kincaid and FORCAST scores). Validation of device performance was undertaken by purposely selected radiation therapists (RTTs) who tested and evaluated the device using a survey with 5-point Likert scale responses. Statistical analysis was undertaken on Excel using Pearson's chi-square, z-test, interrater reliability/agreement and linear regression analyses.

Results

Six RTTs and two interpreters volunteered to participate in this study. 188 common phrases were spoken verbatim into the device and scored on a 5-point Likert scale, yielding an overall output accuracy of 66%. A z-test confirmed significance against prior comparative research and Linear regression analysis observed improved output between consecutive participants. 62.7% of interpreter scores were identical; a further 29.1% constituted a single point scoring variation. Poorer outcomes were observed with colloquial English and lower readability.

Conclusions

This study found the device produced suitable translation accuracy and identified language styles that should be avoided with use. Further research could consider clinical application, expanded languages and/or health disciplines, and development of a national RTT phrase list.

Communicating the Gynecologic Brachytherapy Experience (CoGBE): Clinician perceived benefits of a graphic narrative discussion guide

PURPOSE

Many current radiotherapy patient education materials are not patient-centered. An interprofessional team developed Communicating the Gynecologic Brachytherapy Experience (CoGBE), a graphic narrative discussion guide for cylinder, intracavitary, and interstitial high-dose-rate (HDR) gynecologic brachytherapy. This study assesses perceived clinical benefits, usability, and anxiety-reduction of CoGBE.

METHODS AND MATERIALS

An electronic survey was sent to members of the American Brachytherapy Society. Participants were assigned to assess one of the three modality-specific CoGBE versions using a modified Systems Usability Scale (SUS), modified state-trait anxiety index (mSTAI), and Likert-type questions. Free response data was analyzed using modified grounded theory.

RESULTS

Median modified SUS score was 76.3 (interquartile range [IQR], 71.3-82.5) and there were no significant differences between guide types. Median mSTAI was 40 (IQR, 40-43.3) for all guides collectively. The cylinder guide had a significantly higher median mSTAI than the intracavitary and interstitial guides (41.6 vs. 40.0 and 40.0; p = 0.04) suggesting the cylinder guide may have less impact on reducing anxiety. Most respondents reported that CoGBE was helpful (72%), would improve patient understanding (77%) and consultation memorability (82%), and was at least moderately likely to be incorporated into their practice (80%). Qualitative analysis themes included personalization and relatability (positive); generalizability (negative); illustrations (both).

CONCLUSIONS

Clinicians rate CoGBE as usable with potential to reduce patient anxiety, especially with more invasive treatment modalities including intracavitary or interstitial high-dose-rate. CoGBE has the potential to improve patient-clinician communication for a wider range of patients due to its accessible, adaptable, and patient-centered design.

Spanish Adaptation and Evaluation of Clinical Discussion Guides: Communicating the External Beam Radiotherapy Experience (CEBRE) en Español

PURPOSE

Cancer is the leading cause of death for the Hispanic/Latinx United States (US) community, which comprises 64% of the US population with limited English proficiency. Despite the common use of radiation therapy for cancer treatment, there is a dearth of radiation therapy educational materials-at appropriate reading levels-available in Spanish. To address the gap in patient-centered educational resources for communicating with Spanish-speaking patients about radiation therapy, we sought to linguistically and culturally adapt the Communicating the External Beam Radiotherapy Experience (CEBRE) clinical discussion guide series into Spanish.

METHODS AND MATERIALS

From January to December 2021, we developed and applied a stepwise methodology for Spanish adaptation of the discussion guides involving (1) professional translation; (2) interprofessional review for linguistic and cultural appropriateness and medical accuracy; (3) design review; and (4) evaluation for readability, understandability, and actionability using validated tools. We applied 4 indices for readability evaluation: Gilliam-Peña-Mountain, Läsbarhetsindex, Rate Index, and the Spanish Simple Measure of Gobbledygook. Two trained reviewers assessed understandability and actionability using the Patient Education Materials Assessment Tool.

RESULTS

After 2 revision rounds, 4 CEBRE en español discussion guides were produced through an interprofessional, iterative translation and linguistic/cultural adaptation process. Readability scores across the 4 guides ranged from 4.3 to 7.3 grade-level equivalents, thereby meeting the American Medical Association's 8th-grade standard. Patient Education Materials Assessment Tool analysis yielded near-perfect scores along understandability and actionability domains.

CONCLUSIONS

The stepwise linguistic/cultural adaptation process yielded a patient-centered guide that is appropriately readable, understandable, and actionable for Spanish-speaking patients receiving radiation therapy in the US. Future work should include an external evaluation of CEBRE en español by clinicians and patients. The methodology described can be applied to adapting resources for patient-centered communication in other fields of medicine and into other languages as part of an interprofessional approach to delivering equitable health care for all.

A review of patient questions from physicist-patient consults

Purpose

To provide insight into the types of questions asked to medical physicists by patients during one‐on‐one physicist–patient consults at one institution.

Materials and Methods

Medical physicists trained in patient communication techniques met with patients to provide an overview of the treatment planning and delivery processes, discuss the patient's treatment plan, and answer any technical questions. From August 2016 to December 2019, 152 physicist–patient consults were conducted. In the initial months of the study (August 2016—December 2017), following each physicist–patient consult, all patient questions were documented by the physicists. For the remaining time period (January 2018—December 2019), any newly encountered questions were periodically added to the list. The questions were compiled into a comprehensive list and organized into categories.

Results

There were a total of 88 unique patient questions. These questions fit into four topical categories. Fifty‐four questions (61.4%) were in the “Treatment Planning and Delivery Questions” category, 15 questions (17.1%) were in the “General Radiation Questions or Concerns” category, 13 questions (14.8%) were in the “Safety and Quality Assurance Questions” category, and 6 questions (6.8%) were in the “Medical Questions” category. Overall, patients were primarily concerned about how radiation works, the treatment planning and delivery processes, and what is being done to keep them safe throughout their treatment.

Conclusion

Physicist–patient consults provided an opportunity to address the technical aspects of radiation therapy with patients in greater detail. The fact that patient questions could be conveniently grouped into only four topical categories indicates that it may be straightforward for other medical physicists to prepare for effectively addressing technical questions during physicist–patient consults.

Evaluation of patient education materials for stereotactic radiosurgery from high-performing neurosurgery hospitals and professional societies

Background

Little is known about the readability and utility of patient education materials for stereotactic radiosurgery (SRS). Therefore, the goal of this investigation was to evaluate such materials from high-performing neurosurgery hospitals and professional societies through an analysis of readability and educational content.

Methods

In this cross-cross sectional study, 61 websites associated with the top 50 neurosurgery and neurology hospitals according to U.S. News & World Report (USNWR) and 11 predetermined professional medical societies were queried. Identified SRS education materials were analyzed by 6 readability indices. Educational content was assessed by 10 criteria based on surveys of patients' perspectives about SRS.

Results

Fifty-four materials were identified from the target population (45 from USNWR hospital websites and 9 from professional society websites). Mean readability of materials ranged from 11.7 to 15.3 grade level, far more difficult than national recommendations of sixth and eighth grade. Materials were found to have deficiencies in educational content. Compared with high-performing hospitals, materials from websites of professional societies were longer (P = .002), and more likely to discuss risks and benefits specific to SRS (P = .008), alternative treatment options (P = .05) and expected outcomes or postprocedure descriptions (P = .004). Hospital materials were also more likely to favor brand-specific terminology (eg, GammaKnife) over generic terminology (eg, radiosurgery; P = .019).

Conclusion

Publicly available online patient educational materials for SRS are written at reading levels above national recommendations. Furthermore, many lack information identified as important by patients. Reevaluation and improvement of online SRS educational materials on a national scale are warranted.

Communicating the External Beam Radiation Experience (CEBRE): Perceived Benefits of a Graphic Narrative Patient Education Tool

PURPOSE

Current radiation oncology patient education materials exceed national readability recommendations. A graphic narrative educational tool, the Communicating the External Beam Radiation Experience (CEBRE) discussion guide, was developed to facilitate patient-provider communication. A pilot study evaluated perceived benefits of CEBRE for patients and physicians.

METHODS AND MATERIALS

CEBRE was designed through a collaboration between physicians and designers. Designers interviewed patients, family members, and the clinical team. Interviews were coded for themes, leading to the design principles that drove the design of CEBRE, including a graphic narrative component. CEBRE explains the radiation therapy care path. Readability was measured using the Flesch-Kincaid test. Patients receiving radiation therapy or in follow-up and practicing radiation oncologists reviewed CEBRE and completed independent surveys. Each survey included modified versions of the Systems Usability Score (SUS) and Spielberger State-Trait Anxiety Inventory (STAI) short form, along with questions unique to CEBRE. Likert-type scores are reported as median (interquartile range).

RESULTS

CEBRE scores at a 5.4 Flesch-Kincaid grade level. Thirty-four patients and 15 radiation oncologists completed surveys. Patients had completed a high school/general equivalency degree (18%), a 2-year degree or some college (50%), or at least 4 years of college (32%). Patient and physician responses were concordant. On a scale of 1 to 5 for modified SUS and 1 to 4 for modified STAI ("strongly disagree" to "strongly agree") the SUS scores were 4 (4-5) and 4 (4-5) and STAI scores were 3 (3-4) and 3 (3-3.5) for patients and providers, respectively. This indicates CEBRE is usable and would decrease anxiety. Compared with text, the graphic narrative component of CEBRE was rated as "quite helpful" (4 [4-5]).

CONCLUSIONS

CEBRE, a graphic narrative education tool developed through a novel collaboration between designers and radiation therapy stakeholders, is accessible for patients and practical to facilitate patient-provider discussion. Perceived benefits demonstrating high usability and the potential to decrease patient anxiety warrant further prospective investigation of CEBRE in the clinical setting.

Readability of Patient Education Materials in Radiation Oncology-Are We Improving?

PURPOSE

Prior research suggests that patient education materials on radiation therapy from various professional groups are written at reading levels above national recommendations of sixth to eighth grade. Since publication of these initial findings, many materials from these sources have been updated or newly created. However, the extent to which readability was considered in the design of these new documents remains unknown. Therefore, the goal of this investigation was to evaluate readability of online education materials for radiation therapy, comparing readability of updated materials with those included in the previous study.

METHODS AND MATERIALS

Patient education materials related to radiation oncology were identified from websites of the 3 professional societies used in the original investigation (the American College of Radiology, the American Cancer Society, and the American Society for Radiation Oncology). The documents included in the first analysis were used as a comparator. To allow for accurate evaluation across groups, materials were matched by content; when necessary, documents were combined from the original cohort to create topically equivalent materials to reflect those currently available on society websites. Identified materials were analyzed using 7 validated readability indices, and results are reported in grade-level equivalents. Original materials were also reanalyzed with the same method.

RESULTS

American Cancer Society materials had the lowest average baseline levels (9.4-11.3 grade level) and showed improved readability in the more recent materials (8.0-10.5 grade level). By contrast, the American Society for Radiation Oncology materials had high average baseline readability levels (11.1-13.9 grade level) that actually worsened in the more recent materials (11.7-16.2 grade level). The number of documents meeting national recommendations did not improve in the updated cohort.

CONCLUSION

Despite evidence that readability levels of patient education materials in radiation oncology are unacceptably high, the readability of newly created materials is not consistently improving across professional societies. Although certain groups may incorporate readability in the design of educational documents, more consistent consideration across all organizations is needed.

Malignant websites? Analyzing the quality of prostate cancer education web resources

INTRODUCTION

Prostate cancer patients are using more web resources to inform themselves about their cancer. However, patients may receive out-of-date or inaccurate information due to lack of regulation. The current study looks to systematically analyze the quality of websites accessed by patients with prostate cancer.

METHODS

The term "prostate cancer" was searched in Google and the metasearch engines, Yippy and Dogpile, and the top 100 hits related to patient information were compiled from over 32 million hits. A standardized tool was used to examine 100 sites with respect to attribution, currency, usability, and content.

RESULTS

Of the top 100 websites relating to prostate cancer information, only 27% identified an author, of which 16% had their credentials displayed. The majority of websites disclosed ownership (97%). Over half of the websites did not include the date of the last update and of those that did, only 66% were current within two years. According to the Flesch Kincaid grade level tool for readability, the majority (87%) of sites were found to be at a high school level, while 6% were at university level. Finally, content varied among websites; 90% of sites provided information on detection and workup and treatments, but only 14% of sites included information on prognosis.

CONCLUSIONS

The reliability of websites presenting prostate cancer information is questionable. There were noted deficiencies in attribution, currency, and readability. While information on detection and treatment is well-covered, information related to prognosis is lacking.

Online patient information from radiation oncology departments is too complex for the general population

PURPOSE

Nearly two-thirds of cancer patients seek information about their diagnosis online. We assessed the readability of online patient education materials found on academic radiation oncology department Web sites to determine whether they adhered to guidelines suggesting that information be presented at a sixth-grade reading level.

METHODS AND MATERIALS

The Association of American Medical Colleges Web site was used to identify all academic radiation oncology departments in the United States. One-third of these department Web sites were selected for analysis using a random number generator. Both general information on radiation therapy and specific information regarding various radiation modalities were collected. To test the hypothesis that the readability of these online educational materials was written at the recommended grade level, a panel of 10 common readability tests was used. A composite grade level of readability was constructed using the 8 readability measures that provide a single grade-level output.

RESULTS

A mean of 5605 words (range, 2058-12,837) from 30 department Web sites was collected. Using the composite grade level score, the overall mean readability level was determined to be 13.36 (12.83-13.89), corresponding to a collegiate reading level. This was significantly higher than the target sixth-grade reading level (middle school, t (29) = 27.41, P < .001).

CONCLUSIONS

Online patient educational materials from academic radiation oncology Web sites are significantly more complex than recommended by the National Institutes of Health and the Department of Health and Human Services. To improve patients' comprehension of radiation therapy and its role in their treatment, our analysis suggests that the language used in online patient information should be simplified to communicate the information at a more appropriate level.

Radiation Oncology and Online Patient Education Materials: Deviating From NIH and AMA Recommendations

PURPOSE

Physicians encourage patients to be informed about their health care options, but much of the online health care-related resources can be beneficial only if patients are capable of comprehending it. This study's aim was to assess the readability level of online patient education resources for radiation oncology to conclude whether they meet the general public's health literacy needs as determined by the guidelines of the United States National Institutes of Health (NIH) and the American Medical Association (AMA).

METHODS

Radiation oncology-related internet-based patient education materials were downloaded from 5 major professional websites (American Society for Radiation Oncology, American Association of Physicists in Medicine, American Brachytherapy Society, RadiologyInfo.org, and Radiation Therapy Oncology Group). Additional patient education documents were downloaded by searching for key radiation oncology phrases using Google. A total of 135 articles were downloaded and assessed for their readability level using 10 quantitative readability scales that are widely accepted in the medical literature.

RESULTS

When all 10 assessment tools for readability were taken into account, the 135 online patient education articles were written at an average grade level of 13.7 ± 2.0. One hundred nine of the 135 articles (80.7%) required a high school graduate's comprehension level (12th-grade level or higher). Only 1 of the 135 articles (0.74%) met the AMA and NIH recommendations for patient education resources to be written between the third-grade and seventh-grade levels.

CONCLUSION

Radiation oncology websites have patient education material written at an educational level above the NIH and AMA recommendations; as a result, average American patients may not be able to fully understand them. Rewriting radiation oncology patient education resources would likely contribute to the patients' understanding of their health and treatment options, making each physician-patient interaction more productive and efficient.

Health Literacy and Education Predict Nutrient Quality of Diet of Socioeconomically Diverse, Urban Adults

BACKGROUND

Research has shown that health literacy may be a stronger predictor of health than age, employment status, education level, race, and income. Evidence supports a strong link between low health literacy and poor dietary management of chronic diseases.

OBJECTIVE

The aim was to evaluate the relationship of micronutrient quality of diet, health numeracy and health literacy in White and African American adults randomly selected from 13 Baltimore neighborhoods.

DESIGN

Cross-sectional analysis of Wave 3 (2009-2013) of the longitudinal Healthy Aging in Neighborhoods of Diversity across the Life Span (HANDLS) study initiated in 2004.

MAIN OUTCOME MEASURES

Health literacy was measured using the Rapid Estimate of Adult Literacy in Medicine (REALM). Health numeracy was measured using the numeracy subscale of the Test of Functional Health Literacy in Adults (TOFHLA). Nutrient-based diet quality was measured using Mean Adequacy Ratio (MAR-S) scores calculated from 17 micronutrients from diet plus dietary supplement intake.

STATISTICAL ANALYSES

The relationship of MAR-S scores to the health literacy measures were explored with multiple ordinary least square regression models, adjusting for a number of potential confounders.

RESULTS

REALM but not numeracy was associated with MAR-S; significant covariates included age, current cigarette smoking status, and energy intake. The interactions of race and educational attainment, and REALM and educational attainment were significant, with the relationship between REALM and MAR-S becoming stronger as education level increased.

CONCLUSION

There is a synergistic relationship between health literacy and educational attainment in predicting nutrient-based diet quality. Education was a stronger predictor for Whites compared to African Americans emphasizing the need for health professionals to focus on both education and literacy when creating and providing diet and health-related interventions and resources.

Assessment of online patient materials for breast reconstruction

BACKGROUND

Limited health literacy affects nearly half of American adults and adversely affects patient participation, satisfaction, health care costs, and overall outcomes. As patients increasingly search the Internet for health information, accessibility of online material is critical. Previous studies examining this topic have focused on the readability of these materials. This study evaluates online breast reconstruction resources with regard to reading level, however, adds new metrics to assess degree of complexity, and suitability for the intended audience.

METHODS

The 10 most popular patient Web sites for "breast reconstruction" were identified using the largest Internet search engine. The content of each site was assessed for readability using the simple measure of gobbledygook analysis, complexity using the PMOSE/iKIRSCH formula, and suitability using the suitability assessment of materials instrument. Resulting scores were analyzed overall and by Web site.

RESULTS

Readability analysis revealed an overall average grade level of 13.4 (range 10.7-15.8). All sites exceeded the recommended sixth grade level. Complexity evaluation revealed a mean PMOSE/iKIRSCH score of 6.3, consistent with "low" complexity and requiring an 8th-12th grade education; individual sites ranged from "very low" to "high" complexity. Suitability assessment overall produced a mean 41.2% score, interpreted as "adequate" for the intended patient audience. Five of the 10 sites were found to be "not suitable" when examined individually; the remaining five were "adequate."

CONCLUSIONS

Available online patient material for breast reconstruction is often too difficult for many patients to understand based on readability, complexity, and suitability metrics. Comprehensive assessment is needed to design appropriate patient material and minimize disparities related to limited health literacy.