Developers and evaluation of interactive health communication applications. The Science Panel on Interactive Communications and Health

Analysis of Internet-Based Health Assessments

Health-assessment tools assist practitioners in developing interventions to promote health. An increasing number of online health assessments have evolved recently, creating a need to evaluate for usefulness and effectiveness. This study reviewed 42 online health assessments based on recommendations developed by the Scientific Panel on Interactive Communication and Health. Four types of health assessments were reviewed, with Health Status/Positive Well-Being types, as a whole, demonstrating the highest scores. At least 75% of all sites received maximum scores for items related to background of developers and sponsors, purpose of site, and site design. Fewer than 1 of 5 sites received a maximum score for content issues, confidentiality, and evaluation results and effectiveness. Particularly poor scores were received for the evaluation criteria. Developers of online health assessments need to modify sites to incorporate evaluation results and effectiveness. Practitioners need to be cautious in using sites with no reported evaluation efforts.

Public Health Informatics: The Nature of the Field and Its Relevance to Health Promotion Practice

The purpose of this article is to introduce the relatively new field of public health informatics and provide examples of how informatics is currently being applied to health promotion activities. Additionally, the article illustrates how informatics is well positioned to play a central role and is likely to be central to disease prevention, health education, and health promotion as it will be practiced in the future.

Internet Cognitive Behavioral Therapy for Women With Postnatal Depression: A Randomized Controlled Trial of MumMoodBooster

BACKGROUND

There are few published controlled trials examining the efficacy of Internet-based treatment for postnatal depression (PND) and none that assess diagnostic status (clinical remission) as the primary outcome. This is despite the need to improve treatment uptake and accessibility because fewer than 50% of postnatally depressed women seek help, even when identified as depressed.

OBJECTIVE

In a randomized controlled trial (RCT), we aimed to test the efficacy of a 6-session Internet intervention (the MumMoodBooster program, previously evaluated in a feasibility trial) in a sample of postnatal women with a clinical diagnosis of depression. The MumMoodBooster program is a cognitive behavioral therapy (CBT) intervention, is highly interactive, includes a partner website, and was supported by low-intensity telephone coaching.

METHODS

This was a parallel 2-group RCT (N=43) comparing the Internet CBT treatment (n=21) to treatment as usual (n=22). At baseline and at 12 weeks after enrollment, women's diagnostic status was assessed by telephone with the Standardized Clinical Interview for DSM-IV (SCID-IV) and symptom severity with the Beck Depression Inventory (BDI-II). Depression symptoms were measured repeatedly throughout the study period with the Patient Health Questionnaire (PHQ-9).

RESULTS

At the end of the study, 79% (15/19) of women who received the Internet CBT treatment no longer met diagnostic criteria for depression on the SCID-IV (these outcome data were missing for 2 intervention participants). This contrasted with only 18% (4/22) remission in the treatment as usual condition. Depression scores on the BDI-II showed a large effect favoring the intervention group (d=.83, 95% CI 0.20-1.45). Small to medium effects were found on the PHQ-9 and on measures of anxiety and stress. Adherence to the program was very good with 86% (18/21) of users completing all sessions; satisfaction with the program was rated 3.1 out of 4 on average.

CONCLUSIONS

Our results suggest that our Internet CBT program, MumMoodBooster, is an effective treatment option for women clinically diagnosed with PND. This is one of only two controlled evaluations of specialized online psychological treatment among women clinically diagnosed with PND. MumMoodBooster appears to be a feasible, effective treatment option, which is potentially accessible to large numbers of women in metropolitan, rural, and remote areas. Future work might be focused profitably on establishing comparability with face-to-face treatments and purely self-guided delivery. We have commenced a larger RCT comparing MumMoodBooster with face-to-face CBT.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry (ANZCTR): ACTRN12613000113752; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=363561 (Archived by WebCite® at http://www.webcitation.org/6f64kuyLf).

Notes from the field: planting, nurturing, and watching things grow

Editors' Note: We invited a sample of AJPM's Editorial Board, supplement guest editors, and longtime colleagues and contributors to share their experiences and impressions of the impact of AJPM on their respective domains. We were particularly fond of Ken Powell's analogy to being "tenders of the planted seed"--as planters, weeders, and harvesters--watching and nurturing the growth of AJPM, and have elected to lead with his commentary.

Developing and validating a model to predict the success of an IHCS implementation: the Readiness for Implementation Model

OBJECTIVE

To develop and validate the Readiness for Implementation Model (RIM). This model predicts a healthcare organization's potential for success in implementing an interactive health communication system (IHCS). The model consists of seven weighted factors, with each factor containing five to seven elements.

DESIGN

Two decision-analytic approaches, self-explicated and conjoint analysis, were used to measure the weights of the RIM with a sample of 410 experts. The RIM model with weights was then validated in a prospective study of 25 IHCS implementation cases.

MEASUREMENTS

Orthogonal main effects design was used to develop 700 conjoint-analysis profiles, which varied on seven factors. Each of the 410 experts rated the importance and desirability of the factors and their levels, as well as a set of 10 different profiles. For the prospective 25-case validation, three time-repeated measures of the RIM scores were collected for comparison with the implementation outcomes.

RESULTS

Two of the seven factors, 'organizational motivation' and 'meeting user needs,' were found to be most important in predicting implementation readiness. No statistically significant difference was found in the predictive validity of the two approaches (self-explicated and conjoint analysis). The RIM was a better predictor for the 1-year implementation outcome than the half-year outcome.

LIMITATIONS

The expert sample, the order of the survey tasks, the additive model, and basing the RIM cut-off score on experience are possible limitations of the study.

CONCLUSION

The RIM needs to be empirically evaluated in institutions adopting IHCS and sustaining the system in the long term.

Advanced practice registered nurse usability testing of a tailored computer-mediated health communication program

This study tested the usability of a touch-screen-enabled Personal Education Program with advanced practice RNs. The Personal Education Program is designed to enhance medication adherence and reduce adverse self-medication behaviors in older adults with hypertension. An iterative research process was used, which involved the use of (1) pretrial focus groups to guide the design of system information architecture, (2) two different cycles of think-aloud trials to test the software interface, and (3) post-trial focus groups to gather feedback on the think-aloud studies. Results from this iterative usability-testing process were used to systematically modify and improve the three Personal Education Program prototype versions-the pilot, prototype 1, and prototype 2. Findings contrasting the two separate think-aloud trials showed that APRN users rated the Personal Education Program system usability, system information, and system-use satisfaction at a moderately high level between trials. In addition, errors using the interface were reduced by 76%, and the interface time was reduced by 18.5% between the two trials. The usability-testing processes used in this study ensured an interface design adapted to APRNs' needs and preferences to allow them to effectively use the computer-mediated health-communication technology in a clinical setting.

Usability testing by older adults of a computer-mediated health communication program

Failure to adhere to an antihypertensive regimen and interactions between antihypertensives and other medicines represent serious health threats to older adults. This study tested the usability of a touch-screen-enabled personal education program (PEP). Findings showed that older adults rated the PEP system usability, system usefulness, and system-use satisfaction at a moderately high level for prototype-1 and at an exceptionally high level for prototype-2. A 201.91% reduction in interface errors and a 31.08% decrease in interface time also were found between the two trials. This participatory usability design was highly successful in tailoring its program interface design to accommodate older users to enhance their health communication and technology use efficacy.

User acceptance of HIV TIDES--Tailored Interventions for Management of Depressive Symptoms in persons living with HIV/AIDS

OBJECTIVE

The Tailored Interventions for management of DEpressive Symptoms (TIDES) program was designed based on social cognitive theory to provide tailored, computer-based education on key elements and self-care strategies for depressive symptoms in persons living with HIV/AIDS (PLWHAs).

DESIGN AND MEASUREMENT

Based on an extension of the Technology Acceptance Model (TAM), a cross-sectional design was used to assess the acceptance of the HIV TIDES prototype and explore the relationships among system acceptance factors proposed in the conceptual model.

RESULTS

Thirty-two PLWHAs were recruited from HIV/AIDS clinics. The majority were African American (68.8%), male (65.6%), with high school or lower education (68.7%), and in their 40s (62.5%). PARTICIPANTS spent an average of 10.4 minutes (SD = 5.6) using HIV TIDES. The PLWHAs rated the system as easy to use (Mean = 9.61, SD = 0.76) and useful (Mean = 9.50, SD = 1.16). The high ratings of behavior intention to use (Mean = 9.47, SD = 1.24) suggest that HIV TIDES has the potential to be accepted and used by PLWHAs. Four factors were positively correlated with behavioral intention to use: perceived usefulness (r = 0.61), perceived ease of use (r = 0.61), internal control (r = 0.59), and external control (r = 0.46). Computer anxiety (r = -0.80), tailoring path (r = 0-.35) and depressive symptoms (r = -0.49) were negatively correlated with behavioral intention to use.

CONCLUSION

The results of this study provide evidence of the acceptability of HIV TIDES by PLWHAs. Individuals are expected to be empowered through participating in the interactive process to generate their self-care plan. HIV TIDES enables information sharing about depression prevention and health promotion and has the potential to reframe the traditional patient-provider relationship.

Design and evaluation in eHealth: challenges and implications for an interdisciplinary field

Much has been written about insufficient user involvement in the design of eHealth applications, the lack of evidence demonstrating impact, and the difficulties these bring for adoption. Part of the problem lies in the differing languages, cultures, motives, and operational constraints of producers and evaluators of eHealth systems and services. This paper reflects on the benefits of and barriers to interdisciplinary collaboration in eHealth, focusing particularly on the relationship between software developers and health services researchers. It argues that the common pattern of silo or parallel working may be ameliorated by developing mutual awareness and respect for each others’ methods, epistemologies, and contextual drivers and by recognizing and harnessing potential synergies. Similarities and differences between models and techniques used in both communities are highlighted in order to illustrate the potential for integrated approaches and the strengths of unique paradigms. By sharing information about our research approaches and seeking to actively collaborate in the process of design and evaluation, the aim of achieving technologies that are truly user-informed, fit for context, high quality, and of demonstrated value is more likely to be realized. This may involve embracing new ways of working jointly that are unfamiliar to the stakeholders involved and that challenge disciplinary conventions. It also has policy implications for agencies commissioning research and development in this area.

Pedometer-determined physical activity among multiethnic low-income housing residents

PURPOSE

We sought to characterize pedometer-determined physical activity among a predominantly racial and ethnic minority sample of adults residing in low-income housing.

METHODS

Data were collected from 433 participants at baseline in a randomized colon cancer prevention intervention trial conducted within low-income housing communities. Using random effects models to control for clustering within housing sites, we examined variation in daily steps by several sociodemographic characteristics.

RESULTS

Participants recorded a mean of 5326 (+/- 3871 SD) daily steps over a 5-d sampling period. Significantly lower levels of pedometer-determined physical activity were found among older-aged participants (P < 0.0001), women (P = 0.02), those who were overweight and obese (P = 0.03), those reporting no weekly exercise (P = 0.04), as well as among nonworking individuals (P < 0.0001). No significant differences were found by education or income. In multivariable analyses, age, gender, body mass index, and employment status remained significantly associated with steps.

CONCLUSIONS

These findings suggest a high prevalence of physical inactivity among low-income housing residents. These data, derived from a well-characterized sample, provide useful estimates for the investigation of pedometers as measures of total accumulated physical activity among lower-income, racial and ethnic minority populations.

Internet-based patient education and support interventions: a review of evaluation studies and directions for future research

The Internet provides patients, families, and health providers with unparalleled opportunities to learn, inform, and communicate with one another. This paper is a review of studies that have evaluated the impact of professionally facilitated Internet-based programs for diverse clinical populations on health outcomes, utilization, and user satisfaction. The findings suggest that some outcomes in certain groups can be moderately improved and that user satisfaction has been generally positive. Given the increasing need to provide timely and effective patient-centered care, a stronger evidence base for eHealth applications must be established before broader deployment of such programs will take place.

Meeting patients' needs with patient information systems: potential benefits of qualitative research methods

This article reports on our pilot evaluation of an electronic patient information system for children with amblyopia and their parents. The aim was to investigate whether the information system would be able to improve the quality of care, as indicated by an improvement in the effectiveness and efficiency of care, and in an increase in patient satisfaction. In the pilot evaluation, we used qualitative research methods, exploring the impact of the information system on children and their parents, with the aim to find suitable indicators for a potential further, quantitative study. Yet we found that the system was little used and had marginal effects on the quality of care for children with amblyopia and their parents. It appeared that the main problem underlying this patient information system was that the needs of those people who actually would be using the system had never really been investigated. The designers had built their assumptions about these needs into the system. These appeared to be mistaken at so many levels that the system could not become a success. As a result of this pilot evaluation, the patient information project was thoroughly transformed. This study makes clear that a thorough exploration of user needs before building the system, using qualitative research methods, may be crucial because it can prevent mismatches and maximizes the chance that the eventual information system meets its most important aim: to enhance patient empowerment and improve the quality of care.

More than just a pamphlet: development of an innovative computer-based education program for cancer patients

Patients with cancer continue to lack practical information regarding their illness and report low levels of awareness and use of patient services. The challenge to educators is to find cost-effective and timely ways to deliver a complex mix of interesting and high-quality information and expertise to this large and diverse audience, while still tailoring the content to individual needs and situations. In the present article, we describe the Princess Margaret Hospital (PMH) computer-based patient education program. The program is aimed toward empowering those dealing with cancer and provides comprehensive medical information and support via an interactive Intranet web site containing information about cancer (the Oncology Interactive Education Series), library resources, Internet links, information about PMH services, and a hospital calendar of events. Preliminary evaluation results have provided valuable direction for on-going program development and suggest that the program is easy to use, informative, and enjoyable for patients, families, volunteers, and health professionals.

The content and accessibility of sex education information on the Internet

The objective of this study was to describe Web sites with sex education material and assess the accessibility of specific information on the Internet. First, the authors conducted a review of Web sites using specific sex education keywords. Second, 27 undergraduate students were asked to locate information on proper condom use and sexually transmitted disease (STD) symptoms. The time, number of search attempts, and number of clicks needed to identify each piece of information were recorded. The authors identified 41 sites with sex education material from almost 6 million pages yielded by the keywords. Sixty-three percent of the 1,556 most compatible pages were categorized as pornography. The students found the information on condom use and STD symptoms in an average of 4 minutes, using fewer then six clicks and two searches. The authors concluded that general information on sex education is difficult to locate on the Internet and often lacks essential elements, but accurate and useful information on specific topics can be more easily obtained.