Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites

Quality of Smartphone Apps Related to Panic Disorder

Quality of smartphone apps related to panic: smartphone apps have a growing role in health care. This study assessed the quality of English-language apps for panic disorder (PD) and compared paid and free apps. Keywords related to PD were entered into the Google Play Store search engine. Apps were assessed using the following quality indicators: accountability, interactivity, self-help score (the potential of smartphone apps to help users in daily life), and evidence-based content quality. The Brief DISCERN score and the criteria of the "Health on the Net" label were also used as content quality indicators as well as the number of downloads. Of 247 apps identified, 52 met all inclusion criteria. The content quality and self-help scores of these PD apps were poor. None of the assessed indicators were associated with payment status or number of downloads. Multiple linear regressions showed that the Brief DISCERN score significantly predicted the content quality and self-help scores. Poor content quality and self-help scores of PD smartphone apps highlight the gap between their technological potential and the overall quality of available products.

Quality of websites of obstetrics and gynecology departments: a cross-sectional study

Background

The internet has become an easily accessible and widely used source of healthcare information. There are, however, no standardized or commonly accepted criteria for the quality of Obstetrics and Gynecology websites. In this study, we aimed to evaluate the quality of websites of Obstetrics and Gynecology departments in German-speaking countries and to compare websites nationally and internationally.

Methods

We scored 672 websites from Germany (n = 566), Austria (n = 57), and Switzerland (n = 49) using the objective criteria: Google search rank (2 items), technical aspects (11 items), navigation (8 items), and content (6 items) for a 26 point score. Scores were compared nationally and internationally. Multivariable regression models assessed good quality scores (≥50% of maximum) as the dependent variables and country, academic affiliation, being member of a healthcare consortium, confessional affiliation, and content management system (CMS) use as independent variables.

Results

The mean score of websites was 13.8 ± 3.3. 4.2% were rated as good (≥75% of maximum), 61.8% as fair (≥50% of maximum). German (14.0 ± 3.2) and Swiss (13.8 ± 4.0) websites scored significantly higher compared to Austrian websites (11.6 ± 2.5) (P < 0.001 and P = 0.005, respectively). Within Germany, academic had higher scores than non-academic departments (14.9 ± 3.2 vs. 13.7 ± 3.1, P < 0.001). Single institutions had higher scores compared to healthcare consortium institutions (14.1 ± 3.2 vs. 13.2 ± 2.6, P = 0.003). Departments in Northern and Southern states had higher scores compared to Eastern states (14.4 ± 3.2 and 14.2 ± 3.2 vs. 13.0 ± 3.0, P < 0.001). In multivariate regression models, all subscores (all: P < 0.001) independently predicted a website’s reaching a good quality score, with navigation subscore as strongest predictor. Affiliations were predictors for some good individual subscores, but not for others. High content subscore was associated with good Google search rank, technical aspects, and navigation subscores.

Conclusions

The quality of websites of Obstetrics and Gynecology departments varies widely. We found marked differences depending on country, affiliation, and region.

Electronic supplementary material

The online version of this article (doi:10.1186/s12884-015-0537-9) contains supplementary material, which is available to authorized users.

Evaluating the theoretical content of online physical activity information for people with multiple sclerosis

BACKGROUND

Physical activity can aid people with multiple sclerosis (MS) in managing symptoms and maintaining functional abilities. The Internet is a preferred source of physical activity information for people with MS and, therefore, a method for the dissemination of behavior change techniques. The purpose of this study was to examine the coverage and quality of physical activity behavior change techniques delivered on the Internet for adults with MS using Abraham and Michie's taxonomy of behavior change techniques.

METHODS

Using the taxonomy, 20 websites were coded for quality (ie, accuracy of information) and coverage (ie, completeness of information) of theoretical behavior change techniques.

RESULTS

RESULTS indicated that most websites covered a mean of 8.05 (SD 3.86, range 3-16) techniques out of a possible 20. Only one of the techniques, provide information on behavior-health link and consequences, was delivered on all websites. The websites demonstrated low mean coverage and quality across all behavior change techniques, with means of 0.64 (SD 0.67) and 0.62 (SD 0.37) on a scale of 0 to 2, respectively. However, coverage and quality improved when websites were examined solely for the techniques that they covered, as opposed to all 20 techniques.

CONCLUSIONS

This study, which examined quality and coverage of physical activity behavior change techniques described online for people with MS, illustrated that the dissemination of these techniques requires improvement.

Factor Structure of the Internet Addiction Test in Online Gamers and Poker Players

BACKGROUND

The Internet Addiction Test (IAT) is the most widely used questionnaire to screen for problematic Internet use. Nevertheless, its factorial structure is still debated, which complicates comparisons among existing studies. Most previous studies were performed with students or community samples despite the probability of there being more problematic Internet use among users of specific applications, such as online gaming or gambling.

OBJECTIVE

To assess the factorial structure of a modified version of the IAT that addresses specific applications, such as video games and online poker.

METHODS

Two adult samples-one sample of Internet gamers (n=920) and one sample of online poker players (n=214)-were recruited and completed an online version of the modified IAT. Both samples were split into two subsamples. Two principal component analyses (PCAs) followed by two confirmatory factor analyses (CFAs) were run separately.

RESULTS

The results of principal component analysis indicated that a one-factor model fit the data well across both samples. In consideration of the weakness of some IAT items, a 17-item modified version of the IAT was proposed.

CONCLUSIONS

This study assessed, for the first time, the factorial structure of a modified version of an Internet-administered IAT on a sample of Internet gamers and a sample of online poker players. The scale seems appropriate for the assessment of such online behaviors. Further studies on the modified 17-item IAT version are needed.

Medical information on the internet: a tool for measuring consumer perception of quality aspects

BACKGROUND

Most of adult Internet users have searched for health information on the Internet. The Internet has become one of the most important sources for health information and treatment advice. In most cases, the information found is not verified with a medical doctor, but judged by the "online-diagnosers" independently. Facing this situation, public health authorities raise concern over the quality of medical information laypersons can find on the Internet.

OBJECTIVE

The objective of the study was aimed at developing a measure to evaluate the credibility of websites that offer medical advice and information. The measure was tested in a quasi-experimental study on two sleeping-disorder websites of different quality.

METHODS

There were 45 survey items for rating the credibility of websites that were tested in a quasi-experimental study with a random assignment of 454 participants to either a high- or a low-quality website exposure. Using principal component analysis, the original items were reduced to 13 and sorted into the factors: trustworthiness, textual deficits of the content, interferences (external links on the Web site), and advertisements. The first two factors focus more on the provided content itself, while the other two describe the embedding of the content into the website. The 45 survey items had been designed previously using exploratory observations and literature research.

RESULTS

The final scale showed adequate power and reliability for all factors. The loadings of the principal component analysis ranged satisfactorily (.644 to .854). Significant differences at P<.001 were found between the low- and high-quality groups. Advertisements on the website were rated as disturbing in both experimental conditions, meaning that they do not differentiate between good and bad information.

CONCLUSIONS

The scale reliably distinguished high- and low-quality of medical advice given on websites.

Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument

Background

Validated Internet-based screening tools for cannabis use and abuse are needed. The present study aimed to establish equivalence between the previously validated Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) as a paper-and-pencil (PaP)-administered questionnaire and its online use.

Methods

Two groups of cannabis users took part in this study and the results were analyzed using structural equation modeling. One group consisted of 150 participants and was assessed with the ASSIST PaP questionnaire in a face-to-face interview (the PaP group). They were recruited from three settings: a primary health care outpatient clinic, a general psychiatric facility, and an ambulatory specialized addiction treatment facility. The other group (the Web group) comprised 1382 persons who answered the online version of the same questionnaire. This sample was drawn from people who naturalistically visited a website dedicated to helping people with cannabis addiction.

Results

The internal consistency was good for the online questionnaire (0.74) and high for the already validated PaP questionnaire (0.91). The Web group, however, had higher scores on cannabis use than did the PaP group. The results show support for configural invariance, meaning that the one-factor structure was preserved across groups, although measurement equivalence between these two survey modes was not achieved. However, when the Web group was split into two random subsamples, measurement invariance was demonstrated between them by cross-validation.

Conclusions

Measurement equivalence was not achieved between the two survey modes. Nonetheless, subanalyses of the Web group demonstrated that the cannabis screening questions of the ASSIST can be used for online screening. Differences in ASSIST scores between samples may be due to the sensitive nature of the information surveyed, with possible underreporting in face-to-face interviews, or to the different characteristics of the Web group because of the specialized nature of the website.

Psychenet.de: Development and process evaluation of an e-mental health portal

BACKGROUND

E-mental health interventions can have a positive impact on patient-reported and clinical outcomes. The purpose of this project was to develop a user-centered e-mental health portal.

METHODS

The development of the portal www.psychenet.de included mixed-methods techniques for needs assessment to identify user-relevant content. Furthermore, user-centered design techniques were applied by utilizing individual usability testing with cognitive task analysis. First, a basic version of the portal was created and introduced to the public by means of a media campaign. After the development of module-specific content, exposure and use of the portal was investigated as part of a process evaluation.

RESULTS

Relevant content identified by needs assessment covered both, overarching and diagnosis-specific topics. Results of the process evaluation showed a highly accessed website. During the first 18 months, 119,423 visits were tracked. The portal was predominantly accessed by Google searches (73.9%), while 17.6% of visits were related to direct traffic.

DISCUSSION

Serving as a complement to face-to-face consultations, www.psychenet.de attempts to inform about mental disorders, and engage patients in the course of their treatment. Results of the process evaluation confirm the high relevance and potential of the portal and can be used for further improvements and extensions in the future.

Low health literacy and older adults: meanings, problems, and recommendations for social work

Many older adults struggle to manage their health care problems. Low health literacy exacerbates such struggles and contributes to a variety of adverse health behaviors and outcomes. Addressing how health literacy impinges on the lives of older adults is a neglected area of social work practice and knowledge. This article explores seven areas: defining health literacy, the problem and prevalence of low health literacy among older adults, health inequalities and health literacy, a brief literature review, neglected issues in the literature, suggestions for macro and micro social work interventions to improve health literacy for older adult populations, and conclusion.

Evaluation of Autism-Related Health Information on the Web

BACKGROUND

The Internet is a frequently accessed source of information for parents of a child with autism. To help parents make informed decisions about treatment options, websites should contain accurate information. This study aimed to evaluate the quality of information in a sample of autism-relevant websites.

MATERIALS AND METHODS

Autism-related keywords were entered into three widely used search engines in April 2013 and the 20 most frequently appearing sites identified. Website quality was rated, by two independent raters, using the DISCERN tool. Websites were also coded according to the type of references/sources provided to support the intervention content presented.

RESULTS

The mean DISCERN score was 46.5 (range 23-67.5), of a possible 80. Information about treatment risks and no treatment as an option was rarely described. Only six (30%) websites provided research references when describing intervention options.

CONCLUSIONS

Many websites did not meet criteria for quality health information and failed to cite evidence supporting described interventions. Implications of these findings are discussed.

A systematic evaluation of websites offering information on chronic kidney disease

In this study, we described the content and characteristics of 40 non-proprietary websites offering information about chronic kidney disease (CKD) and evaluated their information quality using the DISCERN scale and readability using Flesch Reading Ease and Flesch-Kincaid grade level. The areas in which the websites scored the lowest on the DISCERN scale were whether the website discussed knowledge gaps, presented balanced information, and was clear about the information source. Websites that rated higher quality on the DISCERN scale were more difficult to read. The quality and readability of many websites about CKD to be used as meaningful educational resources for patients who desire to learn more about CKD and treatment options remain inadequate.

The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output

Background

During the past 2 decades, the Internet has evolved to become a necessity in our daily lives. The selection and sorting algorithms of search engines exert tremendous influence over the global spread of information and other communication processes.

Objective

This study is concerned with demonstrating the influence of selection and sorting/ranking criteria operating in search engines on users’ knowledge, beliefs, and attitudes of websites about vaccination. In particular, it is to compare the effects of search engines that deliver websites emphasizing on the pro side of vaccination with those focusing on the con side and with normal Google as a control group.

Method

We conducted 2 online experiments using manipulated search engines. A pilot study was to verify the existence of dangerous health literacy in connection with searching and using health information on the Internet by exploring the effect of 2 manipulated search engines that yielded either pro or con vaccination sites only, with a group receiving normal Google as control. A pre-post test design was used; participants were American marketing students enrolled in a study-abroad program in Lugano, Switzerland. The second experiment manipulated the search engine by applying different ratios of con versus pro vaccination webpages displayed in the search results. Participants were recruited from Amazon’s Mechanical Turk platform where it was published as a human intelligence task (HIT).

Results

Both experiments showed knowledge highest in the group offered only pro vaccination sites (Z=–2.088, P=.03; Kruskal-Wallis H test [H₅]=11.30, P=.04). They acknowledged the importance/benefits (Z=–2.326, P=.02; H₅=11.34, P=.04) and effectiveness (Z=–2.230, P=.03) of vaccination more, whereas groups offered antivaccination sites only showed increased concern about effects (Z=–2.582, P=.01; H₅=16.88, P=.005) and harmful health outcomes (Z=–2.200, P=.02) of vaccination. Normal Google users perceived information quality to be positive despite a small effect on knowledge and a negative effect on their beliefs and attitudes toward vaccination and willingness to recommend the information (χ²₅=14.1, P=.01). More exposure to antivaccination websites lowered participants’ knowledge (J=4783.5, z=−2.142, P=.03) increased their fear of side effects (J=6496, z=2.724, P=.006), and lowered their acknowledgment of benefits (J=4805, z=–2.067, P=.03).

Conclusion

The selection and sorting/ranking criteria of search engines play a vital role in online health information seeking. Search engines delivering websites containing credible and evidence-based medical information impact positively Internet users seeking health information. Whereas sites retrieved by biased search engines create some opinion change in users. These effects are apparently independent of users’ site credibility and evaluation judgments. Users are affected beneficially or detrimentally but are unaware, suggesting they are not consciously perceptive of indicators that steer them toward the credible sources or away from the dangerous ones. In this sense, the online health information seeker is flying blind.

How can research keep up with eHealth? Ten strategies for increasing the timeliness and usefulness of eHealth research

Background

eHealth interventions appear and change so quickly that they challenge the way we conduct research. By the time a randomized trial of a new intervention is published, technological improvements and clinical discoveries may make the intervention dated and unappealing. This and the spate of health-related apps and websites may lead consumers, patients, and caregivers to use interventions that lack evidence of efficacy.

Objective

This paper aims to offer strategies for increasing the speed and usefulness of eHealth research.

Methods

The paper describes two types of strategies based on the authors’ own research and the research literature: those that improve the efficiency of eHealth research, and those that improve its quality.

Results

Efficiency strategies include: (1) think small: conduct small studies that can target discrete but significant questions and thereby speed knowledge acquisition; (2) use efficient designs: use such methods as fractional-factorial and quasi-experimental designs and surrogate endpoints, and experimentally modify and evaluate interventions and delivery systems already in use; (3) study universals: focus on timeless behavioral, psychological, and cognitive principles and systems; (4) anticipate the next big thing: listen to voices outside normal practice and connect different perspectives for new insights; (5) improve information delivery systems: researchers should apply their communications expertise to enhance inter-researcher communication, which could synergistically accelerate progress and capitalize upon the availability of “big data”; and (6) develop models, including mediators and moderators: valid models are remarkably generative, and tests of moderation and mediation should elucidate boundary conditions of effects and treatment mechanisms. Quality strategies include: (1) continuous quality improvement: researchers need to borrow engineering practices such as the continuous enhancement of interventions to incorporate clinical and technological progress; (2) help consumers identify quality: consumers, clinicians, and others all need to easily identify quality, suggesting the need to efficiently and publicly index intervention quality; (3) reduce the costs of care: concern with health care costs can drive intervention adoption and use and lead to novel intervention effects (eg, reduced falls in the elderly); and (4) deeply understand users: a rigorous evaluation of the consumer’s needs is a key starting point for intervention development.

Conclusions

The challenges of distinguishing and distributing scientifically validated interventions are formidable. The strategies described are meant to spur discussion and further thinking, which are important, given the potential of eHealth interventions to help patients and families.

Quality comparison of websites related to developmental disabilities

The Internet is commonly used to seek health-related information, but little is known about the quality of websites on developmental disabilities. Therefore, we sought to evaluate the characteristics and quality of websites located by searching ten common terms related to developmental disabilities and explore relations between website characteristics and website quality in order to make recommendations on ways to ensure locating good online information. We located 208 unique websites in our November 2012 US searches of Google and Bing. Two independent coders evaluated 10 characteristics of the websites and two different coders assessed the quality of the websites. From the 208 websites, 104 (50%) provided relevant information about the disability being searched. Of these 104 websites, those found to be of highest quality were least likely to be a sponsored result, contain advertisements, be from a for-profit company, and did contain references to peer-reviewed publications or had a top-level domain of .gov or .org. Individuals with developmental disabilities and their family members who choose to obtain disability-related information online should remain vigilant to ensure that they locate high-quality and accurate information and should not replace information obtained from health-care professionals and educational specialists with information found online.

An analysis of online health information on schizophrenia or related conditions: a cross-sectional survey

BACKGROUND

Around 20% of those who seek health information online, search specifically for mental health. However, little is known about the nature of the online health information offered by two European countries, Finland and Greece, which are characterized by markedly differing levels of Internet access and online health information seeking. This study aims to assess, describe and compare websites, written in two European, non-English languages (Finnish and Greek) that appear first after performing an online search concerning schizophrenia or related conditions.

METHODS

The first 20 results from four search terms (searched in Finnish and Greek) in the Web search engine 'Google' were screened. A total of 160 websites were retrieved (80 Finnish, 80 Greek) and evaluated using a preformulated coding system which consisted of websites' indicators, such as: types, characteristics, accountability, interactivity, aesthetics and content. Differences between websites were evaluated with Chi-Square or Fisher's Exact tests for categorical data and independent t-tests for parametric data.

RESULTS

Twenty-four Finnish and thirty-four Greek websites (36% in total) were included. Almost two-thirds (62%, n=36) were owned by an organization, compared to 17% (n=10) by an individual. In both countries, aesthetics had the highest score (possible range 0-4, mean = 2.6, SD = .62), while interactivity the lowest (range 0-5, mean = 1.79, SD = .87). There were no statistically significant differences among the accountability, interactivity and aesthetics scores of the Finnish and Greek websites.

CONCLUSIONS

All assessed indicators suggest there is a need to improve Finnish and Greek online information about schizophrenia or related conditions. The poor website interactivity is of particular concern given the challenges faced by the target group. The findings can be used to guide the development and dissemination of online mental health information aimed at Finnish and Greek online health-seekers.

Survey of quality, readability, and social reach of websites on osteosarcoma in adolescents

OBJECTIVE

Little is known about Internet resources for adolescent patients. This study assessed the quality, readability, and social reach of websites on an illustrative adolescent cancer diagnosis, osteosarcoma.

METHODS

The top 50 results from four queries in two search engines were screened. Quality and readability were determined using standard DISCERN tool, Flesch Reading Ease and Flesch-Kinkaid Grade. Social reach was gauged by social networking links, global website traffic, and a pilot adolescent-specificity measure.

RESULTS

Of 400 websites assessed, 56 (14%) met inclusion criteria. Websites' mean quality was fair (49.8 on 75-point scale; range 31.0-66.0, poor to excellent); 86% failed readability standards (Grade>8); 75% offered at least one social networking link; and 34% offered site-specific social media. More than 60% received over 50,000 visits in the past month. Only 12.5% included adolescent-specific content. Of the 10 websites ranked highest for quality, only one achieved both readability targets and adolescent-specific content.

CONCLUSIONS

Although some patient-oriented websites on osteosarcoma are of acceptable quality, most failed readability targets, and few appeared to address adolescents.

PRACTICE IMPLICATIONS

Better awareness of Internet health resources and social media for adolescents with cancer is needed to address gaps, promote health literacy and facilitate patient-provider communication.

Quality of Web-based information on obsessive compulsive disorder

BACKGROUND

The Internet is increasingly used as a source of information for mental health issues. The burden of obsessive compulsive disorder (OCD) may lead persons with diagnosed or undiagnosed OCD, and their relatives, to search for good quality information on the Web. This study aimed to evaluate the quality of Web-based information on English-language sites dealing with OCD and to compare the quality of websites found through a general and a medically specialized search engine.

METHODS

Keywords related to OCD were entered into Google and OmniMedicalSearch. Websites were assessed on the basis of accountability, interactivity, readability, and content quality. The "Health on the Net" (HON) quality label and the Brief DISCERN scale score were used as possible content quality indicators. Of the 235 links identified, 53 websites were analyzed.

RESULTS

The content quality of the OCD websites examined was relatively good. The use of a specialized search engine did not offer an advantage in finding websites with better content quality. A score ≥16 on the Brief DISCERN scale is associated with better content quality.

CONCLUSION

This study shows the acceptability of the content quality of OCD websites. There is no advantage in searching for information with a specialized search engine rather than a general one.

PRACTICAL IMPLICATIONS

The Internet offers a number of high quality OCD websites. It remains critical, however, to have a provider-patient talk about the information found on the Web.

Quality of Internet information in pediatric otolaryngology: a comparison of three most referenced websites

OBJECTIVE

Patients commonly refer to Internet health-related information. To date, no quantitative comparison of the accuracy and readability of common diagnoses in Pediatric Otolaryngology exist.

STUDY AIMS

(1) identify the three most frequently referenced Internet sources; (2) compare the content accuracy and (3) ascertain user-friendliness of each site; (4) inform practitioners and patients of the quality of available information.

METHODS

Twenty-four diagnoses in pediatric otolaryngology were entered in Google and the top five URLs for each were ranked. Articles were accessed for each topic in the three most frequently referenced sites. Standard rubrics were developed to include proprietary scores for content, errors, navigability, and validated metrics of readability.

RESULTS

Wikipedia, eMedicine, and NLM/NIH MedlinePlus were the most referenced sources. For content accuracy, eMedicine scored highest (84%; p<0.05) over MedlinePlus (49%) and Wikipedia (46%). The highest incidence of errors and omissions per article was found in Wikipedia (0.98±0.19), twice more than eMedicine (0.42±0.19; p<0.05). Errors were similar between MedlinePlus and both eMedicine and Wikipedia. On ratings for user interface, which incorporated Flesch-Kinkaid Reading Level and Flesch Reading Ease, MedlinePlus was the most user-friendly (4.3±0.29). This was nearly twice that of eMedicine (2.4±0.26) and slightly greater than Wikipedia (3.7±0.3). All differences were significant (p<0.05). There were 7 topics for which articles were not available on MedlinePlus.

CONCLUSIONS

Knowledge of the quality of available information on the Internet improves pediatric otolaryngologists' ability to counsel parents. The top web search results for pediatric otolaryngology diagnoses are Wikipedia, MedlinePlus, and eMedicine. Online information varies in quality, with a 46-84% concordance with current textbooks. eMedicine has the most accurate, comprehensive content and fewest errors, but is more challenging to read and navigate. Both Wikipedia and MedlinePlus have lower content accuracy and more errors, however MedlinePlus is simplest of all to read, at a 9th Grade level.

HON label and DISCERN as content quality indicators of health-related websites

Content quality indicators are warranted in order to help patients and consumers to judge the content quality of health-related on-line information. The aim of the present study is to evaluate web-based information on health topics and to assess particular content quality indicators like HON (Health on the Net) and DISCERN. The present study is based on the analysis of data issued from six previous studies which assessed with a standardized tool the general and content quality (evidence-based health information) of health-related websites. Keywords related to Social phobia, bipolar disorders, pathological gambling as well as cannabis, alcohol and cocaine addiction were entered into popular World Wide Web search engines. Websites were assessed with a standardized proforma designed to rate sites on the basis of accountability, presentation, interactivity, readability and content quality (evidence-based information). "Health on the Net" (HON) quality label, and DISCERN scale scores were used to verify their efficiency as quality indicators. Of 874 websites identified, 388 were included. Despite an observed association with higher content quality scores, the HON label fails to predict good content quality websites when used in a multiple regression. Sensibility and specificity of a DISCERN score >40 in the detection of good content quality websites were, respectively, 0.45 and 0.96. The DISCERN is a potential quality indicator with a relatively high specificity. Further developments in this domain are warranted in order to facilitate the identification of high-quality information on the web by patients.

Health Literacy INDEX: development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials

There is no consensus on how best to assess the health literacy demands of health information materials. Comprehensive, reliable, and valid assessment tools are needed. The authors report on the development, refinement, and testing of Health Literacy INDEX, a new tool reflecting empirical evidence and best practices. INDEX is comprised of 63 indicators organized into 10 criteria: plain language, clear purpose, supporting graphics, user involvement, skill-based learning, audience appropriateness, user instruction, development details, evaluation methods, and strength of evidence. In a sample of 100 materials, intercoder agreement was high: 90% or better for 52% of indicators, and above 80% for nearly all others. Overall scores generated by INDEX were highly correlated with average ratings from 12 health literacy experts (r = 0.89, p < .0001). Additional research is warranted to examine the association between evaluation ratings generated by INDEX and individual understanding, behaviors, and improved health. Health Literacy INDEX is a comprehensive tool with evidence for reliability and validity that can be used to evaluate the health literacy demands of health information materials. Although improvement in health information materials is just one aspect of mitigating the effects of limited health literacy on health outcomes, it is an essential step toward a more health literate public.

The quality of mental disorder information websites: a review

OBJECTIVE

This paper reviews studies assessing the quality of websites providing information about mental disorders.

METHODS

The review included 31 articles identified by searching research databases in March 2010. Topics covered included affective disorders, anxiety disorders, eating disorders, substance use disorders and schizophrenia/psychosis.

RESULTS

The largest number of articles (13) reported studies assessing affective disorder information quality. Methodologies varied in site selection and rating methods, with some of limited validity. Most concluded that quality was poor, although quality of affective disorder sites may be improving.

CONCLUSION

There is currently very little understanding of the influence of website quality on user behaviour. Future quality assessments might use the criteria informed by key behaviour change theories.

PRACTICE IMPLICATIONS

A possible approach to research on websites and user behaviour might be to develop an evaluation framework incorporating strategies from behaviour change models, key mental health literacy elements and health outcomes relevant to mental health promotion.

Quality of Internet information related to the Mediterranean diet

Objective

The present study aimed to evaluate the quality of Internet information on the Mediterranean diet and to determine the relationship between the quality of information and the website source.

Design

Website sources were categorized as institutional, pharmaceutical, non-pharmaceutical commercial, charitable, support and alternative medicine. Content quality was evaluated using the DISCERN rating instrument, the Health On the Net Foundation's (HON) code principles, andJournal of the American Medical Association(JAMA) benchmarks. Readability was graded by the Flesch Reading Ease score and Flesch–Kincaid Grade Level score.

Setting

The phrase ‘Mediterranean diet’ was entered as a search term into the six most commonly used English-language search engines.

Subjects

The first thirty websites forthcoming by each engine were examined.

Results

Of the 180 websites identified, thirty-two met our inclusion criteria. Distribution of the website sources was: institutional,n8 (25 %); non-pharmaceutical commercial,n12 (38 %); and support,n12 (38 %). As evaluated by the DISCERN, thirty-one of the thirty-two websites were rated as fair to very poor. Non-pharmaceutical commercial sites scored significantly lower than institutional and support sites (P= 0·002). The mean Flesch Reading Ease score and mean Flesch–Kincaid Grade Level were 55·9 (fairly difficult) and 7·2, respectively. The Flesch–Kincaid Grade Level score determines the difficulty of material by measuring the length of words and sentences and converting the results into a grade level ranging from 0 to 12 (US grade level).

Conclusions

Due to the poor quality of website information on the Mediterranean diet, patients or consumers who are interested in the Mediterranean diet should get advice from physicians or dietitians.

A follow-up study on the quality of alcohol dependence-related information on the web

In order to evaluate the one-year evolution of web-based information on alcohol dependence, we re-assessed alcohol-related sites in July 2007 with the same evaluating tool that had been used to assess these sites in June 2006. Websites were assessed with a standardized form designed to rate sites on the basis of accountability, presentation, interactivity, readability, and content quality. The DISCERN scale was also used, which aimed to assist persons without content expertise in assessing the quality of written health publications. Scores were highly stable for all components of the form one year later (r = .77 to .95, p < .01). Analysis of variance for repeated measures showed no time effect, no interaction between time and scale, no interaction between time and group (affiliation categories), and no interaction between time, group, and scale. The study highlights lack of change of alcohol-dependence-related web pages across one year.

Assessment of the quality and variability of health information on chronic pain websites using the DISCERN instrument

BACKGROUND

The Internet is used increasingly by providers as a tool for disseminating pain-related health information and by patients as a resource about health conditions and treatment options. However, health information on the Internet remains unregulated and varies in quality, accuracy and readability. The objective of this study was to determine the quality of pain websites, and explain variability in quality and readability between pain websites.

METHODS

Five key terms (pain, chronic pain, back pain, arthritis, and fibromyalgia) were entered into the Google, Yahoo and MSN search engines. Websites were assessed using the DISCERN instrument as a quality index. Grade level readability ratings were assessed using the Flesch-Kincaid Readability Algorithm. Univariate (using alpha = 0.20) and multivariable regression (using alpha = 0.05) analyses were used to explain the variability in DISCERN scores and grade level readability using potential for commercial gain, health related seals of approval, language(s) and multimedia features as independent variables.

RESULTS

A total of 300 websites were assessed, 21 excluded in accordance with the exclusion criteria and 110 duplicate websites, leaving 161 unique sites. About 6.8% (11/161 websites) of the websites offered patients' commercial products for their pain condition, 36.0% (58/161 websites) had a health related seal of approval, 75.8% (122/161 websites) presented information in English only and 40.4% (65/161 websites) offered an interactive multimedia experience. In assessing the quality of the unique websites, of a maximum score of 80, the overall average DISCERN Score was 55.9 (13.6) and readability (grade level) of 10.9 (3.9). The multivariable regressions demonstrated that website seals of approval (P = 0.015) and potential for commercial gain (P = 0.189) were contributing factors to higher DISCERN scores, while seals of approval (P = 0.168) and interactive multimedia (P = 0.244) contributed to lower grade level readability, as indicated by estimates of the beta coefficients.

CONCLUSION

The overall quality of pain websites is moderate, with some shortcomings. Websites that scored high using the DISCERN questionnaire contained health related seals of approval and provided commercial solutions for pain related conditions while those with low readability levels offered interactive multimedia options and have been endorsed by health seals.

Quality of Web-based information on depression

BACKGROUND

Although some studies have already been done on the quality of websites on depression, they have mainly focused on content quality, and were mostly carried out about a decade ago. The main aim of this work was thus to examine the quality of current websites on depression through the use of several indicators.

METHODS

Three keywords on depression were entered into a popular search engine. Websites were evaluated on accountability, interactivity, esthetics, readability, and content quality. Moreover, the presence of the "Health on the Net" (HON) quality label and the website's affiliation were checked, and the Brief DISCERN was used as a potential content quality indicator for general consumers.

RESULTS

A total of 45 websites were analyzed. The content quality of the depression-related websites was good. Websites with the HON label had significantly higher accountability, content quality, and Brief DISCERN scores. Moreover, websites with a high score on the Brief DISCERN scored significantly higher for accountability and content quality. Finally, the content quality of websites was significantly and positively correlated with the Brief DISCERN score, the presence of the HON label, and affiliation of the websites.

CONCLUSION

The overall quality of websites on depression seems to be rather good.