An evaluation of the suitability, readability, quality, and usefulness of online resources for family caregivers of patients with cancer

In-Home Respite Care Services Available to Families With Palliative Care Needs in Quebec: Novel Digital Environmental Scan

BACKGROUND

Caregiving dyads in palliative care are confronted with complex care needs. Respite care services can be highly beneficial in alleviating the caregiving burden, supporting survivorship and dying at home. Yet, respite care services are difficult to locate and access in the province of Quebec, Canada, particularly when navigating ubiquitous sources of online health information of varying quality.

OBJECTIVE

This project aimed to (1) compile a list of at-home palliative respite care services in Quebec, Canada; (2) describe key accessibility features for each respite care service; (3) identify accessibility gaps and opportunities; and (4) describe a novel method for conducting environmental scans using internet search engines, internet-based community health databases, and member checking.

METHODS

A novel environmental scan methodology using 2 internet-based targeted databases and 1 internet search engine was conducted. Results were screened and data were extracted, descriptively analyzed, and geographically schematized.

RESULTS

A total of 401 services were screened, and 52 at-home respite care services specific to palliative populations were identified, compiled, and analyzed. These respite care services were characterized by various types of assistance, providers, fees, and serviced geographical regions. Accessibility was explored through the lens of service amenability, availability, eligibility, and compatibility. The data revealed important barriers to accessing respite care services, such as a lack of readily available information on service characteristics, limited availability, and a time-consuming, technical search process for potential respite care users and clinicians to identify appropriate services.

CONCLUSIONS

Both methodological and contextual knowledge have been gained through this environmental scan. Few methodologies for conducting internet-based environmental scans have been clearly articulated, so we applied several learnings from other scans and devised a methodology for conducting an environmental scan using the mixed methods of internet search engines, internet-based community health databases, and member checking. We have carefully reported our methods, so that others conducting community health environmental scans may replicate our process. Furthermore, through this scan, we identified assorted respite care services and pinpointed needs in the provision of these services. The findings highlighted that more easily accessible and centralized information about respite care services is needed in Quebec. The data will enable the creation of a user-friendly tool to share with community support services across Quebec and ultimately help alleviate the added burden caregivers and clinicians face when looking for respite care services in fragmented and complex digital spaces.

Assessment of electronic patient education materials for adolescent bariatric surgery candidates: An environment scan

Objective

Adolescents who opt for metabolic and bariatric surgery (MBS) will use the internet to learn about the procedure. The objective of this study is to assess the suitability of electronic patient education materials (ePEM) of North American centers that perform adolescent bariatric surgery.

Methods

Canadian and American bariatric centers that perform adolescent MBS were identified from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program and Google web-based searches. Suitability of ePEM for the adolescent readership was evaluated using the Suitability Assessment of Materials (SAM).

Results

Sixty-five centers were evaluated from June to July 2020 with 41% citing adolescent specific material. Six percent of the ePEM were evaluated as 'not suitable', 69% were evaluated as 'adequate', and 25% were evaluated as 'superior'.

Conclusion

Adequate ePEM scoring was obtained, but centers had little resources tailored to adolescent patients. Further research is needed to evaluate all the resources provided to adolescents (i.e., resources provided by the health team) to ensure the tools are appropriate for the adolescent readership.

Innovation

This environmental scan provided insights to ePEM available for adolescents considering MBS.

An Integrative Review of the Support Needs of Informal Caregivers of Hematological Cancer Patients in the Period Immediately Following Discharge From Inpatient to Outpatient Care

BACKGROUND

Preparing informal caregivers for a patient's transition to outpatient care is an important component of safe, quality hematological cancer care. The development of many novel therapies and emerging treatments has created opportunities to address the needs of informal caregivers following the discharge of patients from inpatient settings.

OBJECTIVE

To review and synthesize the literature on the needs of informal caregivers of patients with a hematological malignancy postdischarge from inpatient care.

INTERVENTIONS/METHODS

Integrative review methodology was used to explore the body of evidence available. This included a quality appraisal of qualitative, quantitative, and mixed-methods research findings, subsequent data extraction, and inductive thematic synthesis.

RESULTS

One thousand eight articles were screened with 10 included in the review. Key insights into the needs of caregivers entering the outpatient setting were identified and grouped into key subheadings: Encountering complex emotions knowing what to know, little time for yourself, and collateral impact.

CONCLUSION

Findings convey the complex and multiple needs of informal caregivers of hematological cancer patients. With a growing population of people with hematological malignancies and innovations in outpatient cancer therapies, there is a pressing need to codesign interventions to support their caregivers.

IMPLICATIONS FOR PRACTICE

This review has identified a need for more robust research to coproduce interventions in collaboration with caregivers. In addition, interventions developed from further research should be tested in quality implementation science studies to determine their feasibility, sustainability, and impact on outcomes that matter to hematological cancer caregivers.

An examination of the content of diabetes websites targeting Arabic speakers

BACKGROUND

There has been an increase in the numbers of people in the Middle East and North Africa (MENA) region seeking online information about diabetes, as the Internet is considered a helpful tool for finding information about disease prevention and treatment. It is important that the sources available are reliable, trustworthy and easily accessible. However, the quality and readability of such information is questionable. This study examines the content of Arabic diabetes websites and offers recommendations to improve engagement and access to health information.

METHODS

This study employed a cross-sectional analysis approach to assess diabetes websites targeting Arabic speakers in the MENA countries. Google trends was utilized to reveal the top searched topics across the MENA region, which were used as search terms to identify the websites in search engines, resulting in the examination of 26 websites. The analysis combined automated tools with expert evaluation methods and involved five dimensions: quality, readability, engagement, popularity, and availability.

RESULTS

Overall, the majority of the websites performed poorly in the five dimensions and thus require careful reassessment of their content, design, and readability; no website performed well in all dimensions. Only two websites (7.6%) were HONcode certified, and none of the websites achieved a high score in the DISCERN analysis. The readability scores indicated that most websites were above the recommended level of reading. The results also revealed that all websites had poor engagement. Half of the websites had good ranking, and 58% of the websites covered all of the target topics. Of the websites examined, the non-commercial and non-professional sites (classified as "others") seem to have greater readability, higher bounce rates and longer visit duration than the commercial sites.

CONCLUSIONS

The results from this study show a need for improving the content of diabetes information websites available to Arabic speakers. In order to develop websites that contain easy-to-read and understand high-quality information, collaboration between multiple stakeholders is necessary.

Evaluation of the usability, content, readability and cultural appropriateness of online alcohol and other drugs resources for Aboriginal and Torres Strait Islander Peoples in New South Wales, Australia

OBJECTIVES

This study aimed to analyse the usability, content, readability and cultural appropriateness of alcohol and other drugs (AODs) resources for Aboriginal and Torres Strait Islander Peoples in New South Wales (NSW), Australia.

OUTCOME MEASURES

The content of 30 AOD resources for Aboriginal and Torres Strait Islander Peoples was analysed according to the following criteria: general characteristics; elements of graphical design and written communication; thoroughness and content; readability (Flesch-Kincaid grade level (FKGL), Gunning Fog index (Fog), Simplified Measure of Gobbledygook and Flesch Reading Ease); and cultural appropriateness.

RESULTS

Most resources displayed good usability, depicted by the use of headings and subheadings (n=27), superior writing style (n=19), relevant visuals (n=19) and use of colour support (n=30). However, some resources used at least one professional jargon (n=13), and many did not provide any peer-reviewed references (n=22). During content analysis, 12 resources were categorised into the alcohol group and 18 resources in the other drugs group. Impact of alcohol during pregnancy and breast feeding (n=12) was the most common included topics in the resources related to alcohol, while the physical impact of drugs (n=15) was the most discussed topics among the other drugs group. Based on the FKGL readability score, 83% of resources met the recommended reading grade level of 6-8 by NSW Health. Many resources (n=21) met at least half of the cultural appropriateness elements of interest. However, less than one-third were developed in collaboration with the local community (n=9), used local terms (n=5), targeted the local community (n=3), included an Aboriginal voice (n=2) and addressed the underlying cause (n=1).

CONCLUSIONS

Many AOD resources are developed specifically for Aboriginal and Torres Strait Islander Peoples, but their usability, content and readability differed, and they were not culturally appropriate for all communities. Development of a standardised protocol for resource development is suggested.

An evaluation of the suitability, readability, quality, and usefulness of online resources for patients with cancer receiving immunotherapy

OBJECTIVE

To evaluate the suitability, readability, quality, and usefulness of available online resources about the side effects of immunotherapy.

METHODS

Resources were identified through online searches (Google, Microsoft Bing, and Yahoo search engines) and evaluated using the Suitability Assessment Materials (SAM), three readability formulae, the DISCERN tool (quality), and a checklist of symptom management strategies and self-management skills (usefulness). Hierarchical cluster analysis identified resources scoring highly for suitability and quality. Factors associated with the clusters were identified using Fischer's exact tests.

RESULTS

The 51 resources retained for evaluation were of low suitability and quality and exceeded recommended reading levels. Resources were lacking key features to support learning (e.g., illustrations) as well as information about the risks of treatments and the effect of treatments on quality of life. Few resources addressed key self-management skills of problem-solving and taking action. The cluster analysis identified "low" and "moderate" clusters for suitability and quality, and pamphlets and booklets were more likely to be in the moderate cluster as compared to webpages.

CONCLUSION

There is a lack of high-quality resources to support patients' learning to self-manage side effects of immunotherapy.

PRACTICE IMPLICATIONS

Results identify the best available online resources for self-managing side effects of immunotherapy and gaps for future work. Based on findings, a template for patient education materials was developed.

Evaluation of quality, readability, suitability, and usefulness of online resources available to cancer survivors

PURPOSE

The aim of this study was to evaluate the quality, readability, suitability, and usefulness of resources publicly available to adult cancer survivors (aged 18 +) who have completed primary treatment.

METHODS

Resources were identified in July 2021 through Google. Search completeness was verified using Yahoo, Bing, and MedlinePlus. Retrieved resources were assessed for quality using the DISCERN, readability, suitability using the Suitability Assessment Measure (SAM), and usefulness based on a list of unmet needs and self-management skills derived from the literature. Descriptive analyses were conducted, and a cluster analysis identified the highest-scoring resources.

RESULTS

Forty-five resources were included. The mean DISCERN score was fair at 63.3% (SD 13.7%) with low-rated items being sources, publication date, and risks and mechanisms of treatment. The mean reading grade level was 11.19 (SD 1.61, range 8-16) with only one resource scoring an 8. The mean SAM score was in the adequate range at 48.2% (SD 10.6%), with graphics being the lowest-rated section. On average, included resources addressed 57.7% (SD 27.3%) of the unmet needs and 48.4% (SD 20.9%) of the self-management skills, the least addressed being problem-solving.

CONCLUSION

Quality and suitability were fair, whereas readability exceeded recommended levels. Only one resource had a superior score in both quality and suitability.

IMPLICATIONS FOR CANCER SURVIVORS

The most pressing need is to develop resources for cancer survivors that address their unmet needs and are accessible in terms of literacy. Study findings outline the highest-scoring resources currently available to survivors, families, and clinicians.

Development of Educational Print Materials for Physical Activity in Cancer: Evaluation of Readability and Suitability

Educational health materials may be important tools to increase physical activity in cancer patients. Nevertheless, most of the available resources regarding physical activity for cancer patients were found not suitable, had a low grade of readability, and thus, represent a significant barrier to behavior change. To date, little data about development criteria and evaluation of physical activity resources for cancer before their spread exist. The purposes of this study were (i) to describe the development of a physical activity guidebook designed for cancer patients and (ii) to test its readability and suitability. The guidebook was developed through multi-step passages, including group discussions, a literature review, identification of a motivational theory, and using previous research on exercise preferences, barriers, and facilitators to target the information. Two validated formulae were used to assess the readability, whereas thirty-four judges completed the Suitability of Assessment Materials questionnaire to evaluate the suitability of the guidebook. The guidebook was found readable for patients having at least a primary education, and the judges scored it as "superior" material. Our guidebook, following a rigorous method in the development phase, was considered to be suitable and readable. Further evaluations through clinical trials could investigate its effectiveness for behavior change and its impact on cancer patients.

Evaluating breast cancer websites targeting Arabic speakers: empirical investigation of popularity, availability, accessibility, readability, and quality

Background

Nowadays, patients have access to all types of health information on the internet, influencing their decision-making process. The Middle East and North Africa (MENA) region consists of 22 countries with an estimated population of around 600 million. Breast cancer is the highest diagnosed cancer in this region. Websites are commonly the go-to cancer information sources. A large population of the MENA region is only fluent in the Arabic language, thus access to Arabic websites is in more demand. However, little is known about breast cancer websites that cater to an Arabic-speaking audience. This study aims at evaluating Arabic breast cancer websites and offering recommendations to improve engagement and access to health information.

Methods

This study employed a cross-sectional analysis approach. Google trends was used to reveal the top searched topics across the MENA region, which in turn were used as search terms to identify the websites. To be included, a website had to be active, available in Arabic, and contain breast cancer information. The evaluation was based on a combination of automated and expert-based evaluation methods through five dimensions: Availability, Accessibility, Readability, Quality, and Popularity.

Results

Overall most of the websites performed poorly in the five dimensions and require careful reassessment concerning design, content, and readability levels; Only one website performed well in all dimensions, except for readability. Generally, the readability scores indicated that the websites were above the recommended level of reading. None of the websites passed the automated accessibility tests. The expert evaluation using the “Health on the Net” checklist showed good results for most websites.

Conclusions

Breast cancer rates are rising in the MENA region, therefore having comprehensive, accurate, trustworthy, and easy-to-understand health information in their native language is a must. The results from this study show a need for improving the accessibility to breast cancer information websites available to Arabic speakers. The search was limited to three search engines yielding 10 websites and only one tool was used per dimension. Future work is needed to overcome these limitations. Collaboration between multiple stakeholders is necessary to develop websites that contain easy-to-read and understand high-quality information.

An Evaluation of Publicly Available Smartphone Apps to Support Unpaid Cancer Caregivers

CONTEXT

With advancements in mobile technology and increased access to smartphones, the use of Mobile Health applications (apps) has surged. These apps provide an innovative avenue for supporting cancer caregivers who face increasing burden and lack formal support; however, the quality of these apps has not been formally evaluated.

OBJECTIVES

Evaluate the quality, usefulness, therapeutic potential, and security of publicly available apps to support unpaid cancer caregivers in managing their roles and responsibilities.

METHODS

Cancer caregiving apps were identified through a search of the Apple and Google Play stores in October 2020. Two authors evaluated the apps using 1) the Mobile App Rating Scale (MARS) tool for quality, 2) complementary sections of Enlight for therapeutic persuasiveness and alliance, security and privacy, and 3) an unmet needs checklist to assess usefulness. Analyses were undertaken to identify high-scoring apps.

RESULTS

Overall, 24 apps were evaluated by two authors (MB, SW). Across the sample, the mean quality score (MARS) was adequate at 65.7% (3.3/5.0, SD = .5, range: 2.3-4.2). The combined score for therapeutic persuasiveness and alliance (Enlight) was fair at 60.7% (3.0/5.0, SD = .8, range: 1.1-4.5), and the privacy and security checklists yielded means of 79.3% (6.3/8.0, SD = 1.4, range: 50.0%-100.0%) and 41.3% (1.7/4.0, SD = 1.4, range: 0.0-100.0%), respectively. The unmet needs checklist was 43.2% (SD = 5.3, range: 9.4%-69.7%). A hierarchical cluster analysis identified 12 high scoring apps.

CONCLUSION

Superior cluster apps scored acceptably for quality and privacy and low for security and usefulness. Findings will assist clinicians, caregivers, and families identify apps to support cancer caregivers.

Predictors of unhealthy physical and mental days among informal cancer caregivers: results from the 2019 Behavioral Risk Factor Surveillance System survey

BACKGROUND

Patients with cancer and cancer survivors commonly rely upon family members and friends to act as caregivers to help manage cancer treatment and the late effects of that treatment. Informal caregivers provide a variety of supportive functions for cancer patients, including emotional, informational, and functional support, and practical assistance with skilled care activities.

OBJECTIVE

We examined predictors of unhealthy physical and mental days among informal cancer caregivers. Unhealthy days are an estimate of the overall number of days during the previous 30 days when the respondent felt that either his or her physical or mental health was not good.

METHODS

Data were used from the 2019 Behavioral Risk Factor Surveillance System (BRFSS), a cross-sectional, population-based study. The participants were adults aged ≥ 18 years who provided regular care or assistance to a friend or family member who had cancer in the past 30 days.

RESULTS

On average, caregivers who had a household income of less than $25,000 per year reported more (p < 0.05) unhealthy physical and mental days (during the last 30 days). Average reported number of mental unhealthy days was the highest for those who provided care for 2 or more years and 40 or more hours per week. Caregivers of Hispanic ethnicity had a greater risk of reporting unhealthy physical and mental days among those who provided care for 2 or more years. Among those who were providing care for 40 + h a week, caregivers from less wealthy households (income less than $50,000) were at greater risk of experiencing a larger number of unhealthy mental days.

CONCLUSIONS

Informal cancer caregiving can be a stressful experience with potentially negative consequences for both psychological and physical health. Of particular concern are caregivers who are low-income or have limited financial resources.

Codesigning a supportive online resource for Australian cancer carers: a thematic analysis of informal carers' and healthcare professionals' perspectives about carers' responsibilities and content needs

OBJECTIVE

To gather preliminary qualitative data that will assist in the codesign and development of a new informational and supportive website to assist informal cancer carers in Australia.

DESIGN AND SETTING

Utilising a previously tested codesign process, informal carers' experiences and perspectives, including those of healthcare professionals', were examined via focus groups and/or interviews. Data were analysed via thematic analysis.

PARTICIPANTS

Rural (n=9) and urban (n=11) carers', and healthcare professionals' (n=8) perspectives were collected. Carers participated in a focus group (n=9) or telephone interview (n=11). Healthcare professionals completed an interview (n=6) or online survey (n=2).

RESULTS

Rural and urban carers typically felt ill prepared for their multitudinal caregiving responsibilities. Supporting patient-to-healthcare professional liaisons could especially challenge. Carers' biopsychosocial and fiscal strains were affected by patients' hardships and available informal supports. Rural carers described greater social support than urban carers. Both rural and urban carers also described discontentment related to a carer neglecting healthcare system. Both carers and healthcare professionals endorsed the need for a user-friendly, carer-specific website encompassing practical information and resources, peer-driven advice and evidence-based illness information, tailored to the Australian context.

CONCLUSIONS

Carers and healthcare professionals recognise the pressing need for an Australian, cancer carer-specific online resource. Findings will inform the next phase, where a resource will be designed, developed and tested.

Predicting Writing Styles of Web-Based Materials for Children's Health Education Using the Selection of Semantic Features: Machine Learning Approach

BACKGROUND

Medical writing styles can have an impact on the understandability of health educational resources. Amid current web-based health information research, there is a dearth of research-based evidence that demonstrates what constitutes the best practice of the development of web-based health resources on children's health promotion and education.

OBJECTIVE

Using authoritative and highly influential web-based children's health educational resources from the Nemours Foundation, the largest not-for-profit organization promoting children's health and well-being, we aimed to develop machine learning algorithms to discriminate and predict the writing styles of health educational resources on children versus adult health promotion using a variety of health educational resources aimed at the general public.

METHODS

The selection of natural language features as predicator variables of algorithms went through initial automatic feature selection using ridge classifier, support vector machine, extreme gradient boost tree, and recursive feature elimination followed by revision by education experts. We compared algorithms using the automatically selected (n=19) and linguistically enhanced (n=20) feature sets, using the initial feature set (n=115) as the baseline.

RESULTS

Using five-fold cross-validation, compared with the baseline (115 features), the Gaussian Naive Bayes model (20 features) achieved statistically higher mean sensitivity (P=.02; 95% CI -0.016 to 0.1929), mean specificity (P=.02; 95% CI -0.016 to 0.199), mean area under the receiver operating characteristic curve (P=.02; 95% CI -0.007 to 0.140), and mean macro F1 (P=.006; 95% CI 0.016-0.167). The statistically improved performance of the final model (20 features) is in contrast to the statistically insignificant changes between the original feature set (n=115) and the automatically selected features (n=19): mean sensitivity (P=.13; 95% CI -0.1699 to 0.0681), mean specificity (P=.10; 95% CI -0.1389 to 0.4017), mean area under the receiver operating characteristic curve (P=.008; 95% CI 0.0059-0.1126), and mean macro F1 (P=.98; 95% CI -0.0555 to 0.0548). This demonstrates the importance and effectiveness of combining automatic feature selection and expert-based linguistic revision to develop the most effective machine learning algorithms from high-dimensional data sets.

CONCLUSIONS

We developed new evaluation tools for the discrimination and prediction of writing styles of web-based health resources for children's health education and promotion among parents and caregivers of children. User-adaptive automatic assessment of web-based health content holds great promise for distant and remote health education among young readers. Our study leveraged the precision and adaptability of machine learning algorithms and insights from health linguistics to help advance this significant yet understudied area of research.

Evaluation of online, publicly available cancer-related educational and self-management resources for symptom management

OBJECTIVE

The aims of this study were to evaluate the readability, suitability, and quality of publicly available online self-management (SM) resources for people with cancer.

METHODS

Resources were identified using two strategies: (1) a targeted search of 20 Canadian organizations and (2) a Google search. These were evaluated using the Suitability Assessment of Materials (SAM), the DISCERN tool for quality, and readability indices. The SM skills (e.g., problem-solving) and symptom management strategies addressed by each resource were also assessed. Descriptive and hierarchical cluster analyses were performed to identify resources of the highest suitability and quality as well as resource characteristics associated with higher quality and suitability.

RESULTS

A total of 92 resources were evaluated. The mean reading grade level for English resources was 10.29 (SD = 1.64, range of 7.05 to 15.09) and 12.62 for French resources (SD = 2.27, range of 10.12 to 15.65). The mean SAM score across the sample was 50.4% (SD = 10.6%), or 'adequate', and the mean DISCERN score was 61.1% (SD = 11.8%), or 'fair'. The cluster analysis indicated that 10 resources scored highly on both the SAM and the DISCERN. In total, 91 symptom management strategies were identified. On average, resources addressed 2.73 SM skills (SD = 1.58).

CONCLUSIONS

There is a need for plain language resources for people with lower reading ability and resources that incorporate more SM skills. Study findings will help healthcare professionals, patients, and their families identify optimal resources to address cancer-related symptoms.

The quality, suitability, content and readability of online health-related information regarding sexual dysfunction after rectal cancer surgery

AIM

Patients are not well informed about sexual dysfunction after rectal cancer surgery and often turn to the Internet for information. The purpose of this study was to assess online information for patients on sexual dysfunction after rectal cancer surgery.

METHODS

An online search of Google, Yahoo and Bing was performed using specific (e.g., rectal cancer surgery and vaginal pain) and general (e.g., rectal cancer surgery and sex) search terms. Inclusion criteria were websites in English, designed for patients, and including content regarding sexual dysfunction after rectal cancer surgery. Websites were assessed for readability (nine standardized tests), quality (DISCERN tool), suitability (Suitability Assessment of Materials tool), and content.

RESULTS

Of 5040 websites identified, 99 unique websites met inclusion criteria. Three (3%) websites fulfilled the American Medical Association recommendation of a 6th-grade reading level. Using the DISCERN instrument, two (2%) websites were assigned good/excellent quality, nineteen (19%) referenced their sources of information, and thirty-one (31%) fully discussed the impact of sexual dysfunction on quality of life. Using the SAM instrument, three (3%) websites were classified as highly suitable for rectal cancer patients, sixty-five (66%) were adequate, and thirty-one (31%) were inadequate. With regards to content, nine (9%) websites fully discussed the impact of sexual dysfunction on patients partners and fifty-one (52%) websites did not cover prognosis.

CONCLUSION

Online health information available to patients on sexual dysfunction after rectal cancer surgery is suboptimal. Websites are not suitable, lack important content, and are written at too complex a reading level for patients.

Evaluating the Quality of Health-Related WeChat Public Accounts: Cross-Sectional Study

Background

As representatives of health information communication platforms accessed through mobile phones and mobile terminals, health-related WeChat public accounts (HWPAs) have a large consumer base in the Chinese-speaking world. However, there is still a lack of general understanding of the status quo of HWPAs and the quality of the articles they release.

Objective

The aims of this study were to assess the conformity of HWPAs to the Health on the Net Foundation Code of Conduct (HONcode) and to evaluate the suitability of articles disseminated by HWPAs.

Methods

The survey was conducted from April 23 to May 5, 2019. Based on the monthly (March 1-31, 2019) WeChat Index provided by Qingbo Big Data, the top 100 HWPAs were examined to evaluate their HONcode compliance. The first four articles published by each HWPA on the survey dates were selected as samples to evaluate their suitability. All materials were assessed by three raters. The materials were assessed using the HONcode checklist and the Suitability Assessment of Materials (SAM) score sheet. Data analysis was performed with SPSS version 17.0 (SPSS Inc, Chicago, IL, USA) and Excel version 2013 (Microsoft Inc, Washington DC, USA).

Results

A total of 93 HWPAs and 210 of their released articles were included in this study. For six of the eight principles, the 93 HWPAs nearly consistently did not meet the requirements of the HONcode. The HWPAs certified by Tencent Corporation (66/93, 71%) were generally slightly superior to those without such certification (27/93, 29%) in terms of compliance with HONcode principles. The mean SAM score for the 210 articles was 67.72 (SD 10.930), which indicated “adequate” suitability. There was no significant difference between the SAM scores of the articles published by certified and uncertified HWPAs (P=.07), except in the literacy requirements dimension (t_df=97=–2.418, P=.02).

Conclusions

The HWPAs had low HONcode conformity. Although the suitability of health information released by HWPAs was at a moderate level, there were still problems identified, such as difficulty in tracing information sources, excessive implicit advertisements, and irregular usage of charts. In addition, the low approval requirements of HWPAs were not conducive to improvement of their service quality.