Measuring Health Literacy Levels of a Patient Portal Using the CDC's Clear Communication Index

Integrating consumer perspectives into a large-scale health literacy audit of health information materials: learnings and next steps

BACKGROUND

Health information is less effective when it does not meet the health literacy needs of its consumers. For health organisations, assessing the appropriateness of their existing health information resources is a key step to addressing this issue. This study describes novel methods for a consumer-centred large-scale health literacy audit of existing resources and reflects on opportunities to further refine the method.

METHODS

This audit focused on resources developed by NPS MedicineWise, an Australian not-for-profit that promotes safe and informed use of medicines. The audit comprised 4 stages, with consumers engaged at each stage: 1) Select a sample of resources for assessment; 2) Assess the sample using subjective (Patient Education Materials Assessment Tool) and objective (Sydney Health Literacy Lab Health Literacy Editor) assessment tools; 3) Review audit findings through workshops and identify priority areas for future work; 4) Reflect and gather feedback on the audit process via interviews.

RESULTS

Of 147 resources, consumers selected 49 for detailed assessment that covered a range of health topics, health literacy skills, and formats, and which had varied web usage. Overall, 42 resources (85.7%) were assessed as easy to understand, but only 26 (53.1%) as easy to act on. A typical text was written at a grade 12 reading level and used the passive voice 6 times. About one in five words in a typical text were considered complex (19%). Workshops identified three key areas for action: make resources easier to understand and act on; consider the readers' context, needs, and skills; and improve inclusiveness and representation. Interviews with workshop attendees highlighted that audit methods could be further improved by setting clear expectations about the project rationale, objectives, and consumer roles; providing consumers with a simpler subjective health literacy assessment tool, and addressing issues related to diverse representation.

CONCLUSIONS

This audit yielded valuable consumer-centred priorities for improving organisational health literacy with regards to updating a large existing database of health information resources. We also identified important opportunities to further refine the process. Study findings provide valuable practical insights that can inform organisational health actions for the upcoming Australian National Health Literacy Strategy.

eHealth Literacy and Patient Portal Use and Attitudes: Cross-sectional Observational Study

BACKGROUND

Throughout the COVID-19 pandemic, patient portals have become more widely used tools of patient care delivery. However, not all individuals have equivalent access or ability to use patient portals.

OBJECTIVE

The aim of this study is to evaluate the relationships between eHealth literacy (eHL) and patient portal awareness, use, and attitudes among hospitalized patients.

METHODS

Inpatients completed patient portal surveys; eHL was assessed (eHealth Literacy Scale). Multivariable logistic regression analyses adjusted for age, self-reported race, gender, and educational attainment were completed with significance at P<.006 (Bonferroni correction).

RESULTS

Among 274 participants, most identified as Black (n=166, 61%) and female (n=140, 51%), mean age was 56.5 (SD 16.7) years, and 178 (65%) reported some college or higher educational attainment. One-quarter (n=79, 28%) had low eHL (mean 27, SD 9.5), which was associated with lower odds of portal access awareness (odds ratio 0.11, 95% CI 0.05-0.23; P<.001), having ever used portals (odds ratio 0.19, 95% CI 0.10-0.36; P<.001), less perceived usefulness of portals (odds ratio 0.20, 95% CI 0.10-0.38; P=.001), and lower likelihood of planning to use portals in the coming years (odds ratio 0.12, 95% CI 0.06-0.25; P<.001). As time through the COVID-19 pandemic passed, there was a trend toward increased perceived usefulness of patient portals (53% vs 62%, P=.08), but average eHL did not increase through time (P=.81).

CONCLUSIONS

Low eHL was associated with less awareness, use, and perceived usefulness of portals. Perceived usefulness of portals likely increased through the COVID-19 pandemic, but patients' eHL did not. Interventions tailored for patients with low eHL could ensure greater equity in health care delivery through the COVID-19 pandemic.

Interventions to increase patient portal use in vulnerable populations: a systematic review

BACKGROUND

More than 100 studies document disparities in patient portal use among vulnerable populations. Developing and testing strategies to reduce disparities in use is essential to ensure portals benefit all populations.

OBJECTIVE

To systematically review the impact of interventions designed to: (1) increase portal use or predictors of use in vulnerable patient populations, or (2) reduce disparities in use.

MATERIALS AND METHODS

A librarian searched Ovid MEDLINE, EMBASE, CINAHL, and Cochrane Reviews for studies published before September 1, 2018. Two reviewers independently selected English-language research articles that evaluated any interventions designed to impact an eligible outcome. One reviewer extracted data and categorized interventions, then another assessed accuracy. Two reviewers independently assessed risk of bias.

RESULTS

Out of 18 included studies, 15 (83%) assessed an intervention's impact on portal use, 7 (39%) on predictors of use, and 1 (6%) on disparities in use. Most interventions studied focused on the individual (13 out of 26, 50%), as opposed to facilitating conditions, such as the tool, task, environment, or organization (SEIPS model). Twelve studies (67%) reported a statistically significant increase in portal use or predictors of use, or reduced disparities. Five studies (28%) had high or unclear risk of bias.

CONCLUSION

Individually focused interventions have the most evidence for increasing portal use in vulnerable populations. Interventions affecting other system elements (tool, task, environment, organization) have not been sufficiently studied to draw conclusions. Given the well-established evidence for disparities in use and the limited research on effective interventions, research should move beyond identifying disparities to systematically addressing them at multiple levels.

Challenges and solutions to employing natural language processing and machine learning to measure patients' health literacy and physician writing complexity: The ECLIPPSE study

OBJECTIVE

In the National Library of Medicine funded ECLIPPSE Project (Employing Computational Linguistics to Improve Patient-Provider Secure Emails exchange), we attempted to create novel, valid, and scalable measures of both patients' health literacy (HL) and physicians' linguistic complexity by employing natural language processing (NLP) techniques and machine learning (ML). We applied these techniques to > 400,000 patients' and physicians' secure messages (SMs) exchanged via an electronic patient portal, developing and validating an automated patient literacy profile (LP) and physician complexity profile (CP). Herein, we describe the challenges faced and the solutions implemented during this innovative endeavor.

MATERIALS AND METHODS

To describe challenges and solutions, we used two data sources: study documents and interviews with study investigators. Over the five years of the project, the team tracked their research process using a combination of Google Docs tools and an online team organization, tracking, and management tool (Asana). In year 5, the team convened a number of times to discuss, categorize, and code primary challenges and solutions.

RESULTS

We identified 23 challenges and associated approaches that emerged from three overarching process domains: (1) Data Mining related to the SM corpus; (2) Analyses using NLP indices on the SM corpus; and (3) Interdisciplinary Collaboration. With respect to Data Mining, problems included cleaning SMs to enable analyses, removing hidden caregiver proxies (e.g., other family members) and Spanish language SMs, and culling SMs to ensure that only patients' primary care physicians were included. With respect to Analyses, critical decisions needed to be made as to which computational linguistic indices and ML approaches should be selected; how to enable the NLP-based linguistic indices tools to run smoothly and to extract meaningful data from a large corpus of medical text; and how to best assess content and predictive validities of both the LP and the CP. With respect to the Interdisciplinary Collaboration, because the research required engagement between clinicians, health services researchers, biomedical informaticians, linguists, and cognitive scientists, continual effort was needed to identify and reconcile differences in scientific terminologies and resolve confusion; arrive at common understanding of tasks that needed to be completed and priorities therein; reach compromises regarding what represents "meaningful findings" in health services vs. cognitive science research; and address constraints regarding potential transportability of the final LP and CP to different health care settings.

DISCUSSION

Our study represents a process evaluation of an innovative research initiative to harness "big linguistic data" to estimate patient HL and physician linguistic complexity. Any of the challenges we identified, if left unaddressed, would have either rendered impossible the effort to generate LPs and CPs, or invalidated analytic results related to the LPs and CPs. Investigators undertaking similar research in HL or using computational linguistic methods to assess patient-clinician exchange will face similar challenges and may find our solutions helpful when designing and executing their health communications research.

Readability and understandability of clinical research patient information leaflets and consent forms in Ireland and the UK: a retrospective quantitative analysis

OBJECTIVES

The first aim of this study was to quantify the difficulty level of clinical research Patient Information Leaflets/Informed Consent Forms (PILs/ICFs) using validated and widely used readability criteria which provide a broad assessment of written communication. The second aim was to compare these findings with best practice guidelines.

DESIGN

Retrospective, quantitative analysis of clinical research PILs/ICFs provided by academic institutions, pharmaceutical companies and investigators.

SETTING

PILs/ICFs which had received Research Ethics Committee approval in the last 5 years were collected from Ireland and the UK.

INTERVENTION

Not applicable.

MAIN OUTCOME MEASURES

PILs/ICFs were evaluated against seven validated readability criteria (Flesch Reading Ease, Flesh Kincaid Grade Level, Simplified Measure of Gobbledegook, Gunning Fog, Fry, Raygor and New Dale Chall). The documents were also scored according to two health literacy-based criteria: the Clear Communication Index (CCI) and the Suitability Assessment of Materials tool. Finally, the documents were assessed for compliance with six best practice metrics from literacy agencies.

RESULTS

A total of 176 PILs were collected, of which 154 were evaluable. None of the PILs/ICFs had the mean reading age of <12 years recommended by the American Medical Association. 7.1% of PILs/ICFs were evaluated as 'Plain English', 40.3%: 'Fairly Difficult', 51.3%: 'Difficult' and 1.3%: 'Very Difficult'. No PILs/ICFs achieved a CCI >90. Only two documents complied with all six best practice literacy metrics.

CONCLUSIONS

When assessed against both traditional readability criteria and health literacy-based tools, the PILs/ICFs in this study are inappropriately complex. There is also evidence of poor compliance with guidelines produced by literacy agencies. These data clearly evidence the need for improved documentation to underpin the consent process.

Cross-cultural adaptation of the Clear Communication Index to Brazilian Portuguese

OBJECTIVE

To perform a cross-cultural adaptation of the Clear Communication Index instrument from the Centers for Disease Control and Prevention (CDC-CCI) from English to Brazilian Portuguese.

METHODS

This study comprised initial discussion about the conceptual equivalence of the instrument by a committee formed by experts on health education. We performed translations, synthesis of translations, back-translations, revision by the committee, and linguistic revision. Semantic equivalence was obtained by analyzing the referential and general meaning of each item by the committee, resulting in a pre-final version of the instrument. Subsequently, thirty professionals with health sciences degrees performed a pre-test. These professionals used the pre-final version of the instrument to assess a health education material. A questionnaire was applied to evaluate the acceptability of the instrument, the understanding of each of the 20 items, as well as the individual and professional variables. We analyzed the scores attributed to the health education material, the variables related to healthcare professionals, the proportions of the acceptability of the instrument, and the comprehension of each item.

RESULTS

After we obtained the conceptual equivalence of the instrument, the committee of experts, the instrument’s main author, and the linguist produced the pre-final version using two translations, a synthesis of the translations, and two back-translations. A general equivalence was maintained in 15 of the 20 items (75%), four of the items were slightly altered (20%), and one item was very altered (5%). Nineteen items presented referential equivalence or near equivalence (95%). We then carried out with the pre-test, in which the professionals used the pre-final version. Two items in the domains of “risks” and “main message” were unclear and needed to be revised.

CONCLUSION

The process of cross-cultural adaptation of the Clear Communication Index provided an adapted version to the Brazilian Portuguese language.

Designing Occupational Safety and Health Training Materials for Clear Communication

INTRODUCTION

Printed materials are an essential part of occupational safety and health programs. Public health professionals at the Centers for Disease Control and Prevention (CDC) have created a Clear Communication Index (CCI) to guide design of health education materials for the general public.

METHODS

We revised an existing handout on heat exposure hazards in construction using the CCI and tested the old and new versions of the handout with an audience of 425 construction apprentices and journey-level workers.

RESULTS

Some features recommended by the CCI-such as the use of subheadings, numbering, and other visual cues-strongly conditioned the readers' understanding of the main message.

CONCLUSIONS

Design and layout have a significant impact on the delivery of messages in written materials. A communications-based rubric such as the CCI can help writers preparing written occupational safety and health materials for workers and general audiences.

Users' acceptance of electronic patient portals in Lebanon

Background

Acceptance of Electronic patient portal (EPP) is instrumental for its success. Studies on users’ acceptance in the Middle East region are scarce. This study aims to use the TAM as a framework to quantitatively describe potential users, diabetic and chronic high blood pressure patients and their providers, intention to use and factors influencing the intention to use EPP at AUBMC-FMC We concurrently test the internal construct validity and the reliability of the TAM.

Methods

A cross-sectional survey design and the vignette approach were used. For validation, we needed a minimum of 180 patients; all 35 attending physicians and 11 registered nurses were targeted. We used descriptive statistics to calculate the intention to use EPP and its determinants based on the TAM constructs. Exploratory factor analysis (EFA) and structural equation modeling (SEM) were employed to estimate significant path coefficients for patients only as the sample size of providers was too small.

Results

We had 199 patients, half intended to use EPP; 73% of providers (N = 17) intended to use EPP. Perceived ease of use and privacy concerns were significantly higher among providers than patients (Mean (M) = 0.77 vs M = 0.42 (CI: − 0.623; − 0.081)) and (M = 3.67 vs M = 2.13, CI: − 2.16; − 0.91) respectively; other constructs were not significantly different. Reliability of TAM revealed a Cronbach Alpha of α=.91. EFA showed that three components explained 73.48% of the variance: Behavioral Intention of Use (14.9%), Perceived Ease of Use (50.74%), Perceived Usefulness (7.84%). SEM found that perceived ease of use increased perceived usefulness (standardized regression weight = 0.49); perceived usefulness (0.51) had more predictive value than perceived ease of use (0.27) to explain the behavioral intention of use of the EPP.

Conclusions

We found that providers valued the usefulness of EPP and were mostly intending to use it. This finding has yet to be tested in future studies testing actual use as intention and actual use may not be concordant. The intention to use among patients was lower than those reported in developed countries. We identified two factors that we need to address to increase use, namely perceived ease and usefulness, and proposed practical implications to address them; future research directions were also discussed.

Is research on patient portals attuned to health equity? A scoping review

OBJECTIVE

Our scoping review examined how research on patient portals addresses health equity. Questions guiding our review were: 1) What health equity concepts are addressed in patient portal research-both explicitly and implicitly? 2) What are the gaps? 3) Is the potential for ehealth-related health inequities explicitly acknowledged in studies on patient portals? 4) What novel approaches and interventions to reduce health inequities are tested in patient portal research?

MATERIALS AND METHODS

We searched 4 databases. Search terms included "patient portal" in combination with a comprehensive list of health equity terms relevant in ehealth context. Authors independently reviewed the papers during initial screening and full-text review. We applied the eHealth Equity Framework to develop search terms and analyze the included studies.

RESULTS

Based on eHealth Equity Framework categories, the main findings generated from 65 reviewed papers were governance structures, ehealth policies, and cultural and societal values may further inequities; social position of providers and patients introduces differential preferences in portal use; equitable portal implementation can be supported through diverse user-centered design; and intermediary strategies are typically recommended to encourage portal use across populations.

DISCUSSION

The predominant focus on barriers in portal use may be inadvertently placing individual responsibility in addressing these barriers on patients already experiencing the greatest health disparities. This approach may mask the impact of the socio-technical-economic-political context on outcomes for different populations.

CONCLUSION

To support equitable health outcomes related to patient portals we need to look beyond intermediary initiatives and develop equitable strategies across policy, practice, research, and implementation.

Using the Clear Communication Index to Improve Materials for a Behavioral Intervention

Ensuring that written materials used in behavioral interventions are clear is important to support behavior change. This study used the Clear Communication Index (CCI) to assess the original and revised versions of three types of written participant materials from the SIPsmartER intervention. Materials were revised based on original scoring. Scores for the entire index were significantly higher among revised versions than originals (57% versus 41%, p < 0.001); however, few revised materials (n = 2 of 53) achieved the benchmark of ≥90%. Handouts scored higher than worksheets and slide sets for both versions. The proportion of materials scored as having "a single main message" significantly increased between versions for worksheets (7% to 57%, p = 0.003) and slide sets (33% to 67%, p = 0.004). Across individual items, most significant improvements were in Core, with four-items related to the material having a single main message. Findings demonstrate that SIPsmartER's revised materials improved after CCI-informed edits. They advance the evidence and application of the CCI, suggesting it can be effectively used to support improvement in clarity of different types of written materials used in behavioral interventions. Implications for practical considerations of using the tool and suggestions for modifications for specific types of materials are presented.

Participatory development and pilot testing of iChoose: an adaptation of an evidence-based paediatric weight management program for community implementation

BACKGROUND

To describe the identification, adaptation, and testing of an evidence-based pediatric weight management program for a health disparate community.

METHODS

A community advisory board (CAB) of decision-makers and staff from local health care, public health, and recreation organizations engaged with academic partners to select an evidence-based program (EBP) for local implementation. Three EBPs were identified (Traffic Light, Bright Bodies, Golan and colleagues Home Environmental Model) and each EBP was rated on program characteristics, implementation and adaptation, and adoptability. Following selection of the EBP that was rated highest, the POPS-CAB made adaptations based on the program principles described in peer-reviewed publications. The adapted intervention, iChoose, was then pilot tested in 3 iterative phases delivered initially by research partners, then co-delivered by research and community partners, then delivered by community partners. The RE-AIM framework was used to plan and evaluate the iChoose intervention across all waves with assessments at baseline, post program (3 months), and follow-up (6 months).

RESULTS

Bright Bodies rated highest on program characteristics and adoptability (p's < 0.05), while Home Environmental Model rated highest on implementation factors (p < 0.05). Qualitatively, the selection focused on important program characteristics and on matching those characteristics to the potential to fit within the community partner services. The adapted program-iChoose-had 18% reach and with participants that were representative of the target population on age, gender, ethnicity, and race. Effectiveness was demonstrated by modest, but significant reductions in BMI z-scores at post-program compared to baseline (MΔ = - 0.047; t = - 2.11, p = 0.046). This decrease returned to values similar to baseline 3 months (MΔ = 0.009) after the program was completed. Implementation fidelity was high and implementation fidelity did not differ between community or research delivery agents.

CONCLUSION

The process to help organizations identify and select evidence-based programs appropriate for their community led to consensus on a single EBP. While iChoose was successful in initiating changes in BMI z-scores, could be implemented in a low resource community with fidelity, it was insufficient to lead to sustained child BMI z-scores. In response to these data, maintenance of program effects and delivery are the current focus of the CBPR team.

Assessing the Needs and Perspectives of Patients With Asthma and Chronic Obstructive Pulmonary Disease on Patient Web Portals: Focus Group Study

BACKGROUND

As accessibility to the internet has increased in society, many health care organizations have developed patient Web portals (PWPs), which can provide a range of self-management options to improve patient access. However, the available evidence suggests that they are used inefficiently and do not benefit patients with low health literacy. Asthma and chronic obstructive pulmonary disease (COPD) are common chronic diseases that require ongoing self-management. Moreover, patients with COPD are typically older and have lower health literacy.

OBJECTIVE

This study aimed to obtain and present an overview of patients' perspectives of PWPs to facilitate the development of a portal that better meets the needs of patients with asthma and COPD.

METHODS

We performed a focus group study using semistructured interviews in 3 patient groups from the north of the Netherlands who were recruited through the Dutch Lung Foundation. Each group met 3 times for 2 hours each at a 1-week interval. Data were analyzed with coding software, and patient descriptors were analyzed with nonparametric tests. The consolidated criteria for reporting qualitative research were followed when conducting the study.

RESULTS

We included 29 patients (16/29, 55% male; mean age 65 [SD 10] years) with COPD (n=14), asthma-COPD overlap (n=4), asthma (n=10), or other respiratory disease (n=1). There was a large variation in the internet experience; some patients hardly used the internet (4/29, 14%), whereas others used internet >3 times a week (23/29, 79%). In general, patients were positive about having access to a PWP, considering access to personal medical records as the most important option, though only after discussion with their physician. A medication overview was considered a useful option. We found that communication between health care professionals could be improved if patients could use the PWP to share information with their health care professionals. However, as participants were worried about the language and usability of portals, it was recommended that language should be adapted to the patient level. Another concern was that disease monitoring through Web-based questionnaire use would only be useful if the results were discussed with health care professionals.

CONCLUSIONS

Participants were positive about PWPs and considered them a logical step. Today, most patients tend to be better educated and have internet access, while also being more assertive and better informed about their disease. A PWP could support these patients. Our participants also provided practical suggestions for implementation in current and future PWP developments. The next step will be to develop a portal based on these recommendations and assess whether it meets the needs of patients and health care providers.

Assessing the value of the Meaningful Use Clinical Summary for patients and families with pediatric asthma

Electronic clinical summaries are innovations supported by the Electronic Health Record Incentive Program, known as "Meaningful Use" (MU). The MU clinical summary documents the shared understanding of the plan of care for patients and assists families in managing asthma-related health care. The purpose of this analysis was to identify the communicative value of the summaries to patients and families. Readability measurements, content analysis, and descriptive statistics were employed in a review of twenty clinical summaries and compared with provider encounter notes. The average age of the patients from whom we collected clinical summaries was six years old. The average reading level of the summaries was ninth grade. Neither summaries nor health education contained visual images. There was a total of nine different asthma diagnoses. A full list of diagnoses was present in 45% of summaries. The average medications per patient was 5.75, and there were multiple medication changes noted (dosage adjustment, add, discontinued). Allergies, vital signs, and smoking status were reliably reported (99-100%). Provider orders present included medication, follow up, and return to the clinic instructions. The plan of care was replicated on 45% of summaries. There was variable reporting of various asthma guidelines. Opportunities to improve the clinical summary include using plain language to promote readability, action, understanding, and health literacy, training providers to standardize their documentation and include asthma action plans, and configuring EHR settings to ensure diagnoses and plan of care is carried over from provider notes to the summary.