Usability evaluation of a web-based support system for people with a schizophrenia diagnosis

Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation

BACKGROUND

Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients' experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM.

OBJECTIVE

This study examined the usability of an electronic real-time measurement system to monitor surgical patients' experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction.

METHODS

We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients' experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction).

RESULTS

A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access.

CONCLUSIONS

A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients' experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2023-079155.

Co-Production of a Flexibly Delivered Relapse Prevention Tool to Support the Self-Management of Long-Term Mental Health Conditions: Co-Design and User Testing Study

BACKGROUND

Supported self-management interventions, which assist individuals in actively understanding and managing their own health conditions, have a robust evidence base for chronic physical illnesses, such as diabetes, but have been underused for long-term mental health conditions.

OBJECTIVE

This study aims to co-design and user test a mental health supported self-management intervention, My Personal Recovery Plan (MyPREP), that could be flexibly delivered via digital and traditional paper-based mediums.

METHODS

This study adopted a participatory design, user testing, and rapid prototyping methodologies, guided by 2 frameworks: the 2021 Medical Research Council framework for complex interventions and an Australian co-production framework. Participants were aged ≥18 years, self-identified as having a lived experience of using mental health services or working in a peer support role, and possessed English proficiency. The co-design and user testing processes involved a first round with 6 participants, focusing on adapting a self-management resource used in a large-scale randomized controlled trial in the United Kingdom, followed by a second round with 4 new participants for user testing the co-designed digital version. A final round for gathering qualitative feedback from 6 peer support workers was conducted. Data analysis involved transcription, coding, and thematic interpretation as well as the calculation of usability scores using the System Usability Scale.

RESULTS

The key themes identified during the co-design and user testing sessions were related to (1) the need for self-management tools to be flexible and well-integrated into mental health services, (2) the importance of language and how language preferences vary among individuals, (3) the need for self-management interventions to have the option of being supported when delivered in services, and (4) the potential of digitization to allow for a greater customization of self-management tools and the development of features based on individuals' unique preferences and needs. The MyPREP paper version received a total usability score of 71, indicating C+ or good usability, whereas the digital version received a total usability score of 85.63, indicating A or excellent usability.

CONCLUSIONS

There are international calls for mental health services to promote a culture of self-management, with supported self-management interventions being routinely offered. The resulting co-designed prototype of the Australian version of the self-management intervention MyPREP provides an avenue for supporting self-management in practice in a flexible manner. Involving end users, such as consumers and peer workers, from the beginning is vital to address their need for personalized and customized interventions and their choice in how interventions are delivered. Further implementation-effectiveness piloting of MyPREP in real-world mental health service settings is a critical next step.

How should patient decision aids for schizophrenia treatment be designed? - A scoping review

Despite the clear rationale for applying shared decision-making in the context of the preference sensitive decision for or against antipsychotics and the upswing of patient decision aids (pDAs) to support this process, there is still a lack of knowledge regarding which key features are crucial for pDAs in schizophrenia treatment. A scoping review according to the PRISMA-SRc was conducted to inform on crucial key features and quality indicators. The review focussed on the following seven aspects for investigating pDAs: (1) Types of decision aids, (2) Values, (3) Decision Guidance, (4) Output of the decision aid, (5) Target group, (6) Effectiveness according to publication and (7) Decision aid evaluation. Eleven studies which addressed six unique decision aids met the eligibility criteria. There were major differences in the design as well as in the development of the decision aids. Three aspects emerged that should be given special consideration in the design of such tools for antipsychotics: the evidence used by the decision aid, the algorithm for translating evidence into a decision aid and finally the presentation of the evidence. We recommend the use of data with a high level of evidence and to combine it with individualized treatment by taking into account patient preferences and previous experiences as well as comparing them with clinical assessments. Fully computerized decision aids that use complicated algorithms, for example, by merging treatment effects with patient characteristics to suggest an appropriate treatment at the end, tend to be paternalistic and thus not appropriate for SDM, in our view. In addition, possible cognitive deficits need to be considered when presenting the output of decision aids for antipsychotics.

Trends, challenges, and priorities for shared decision making in mental health: The first umbrella review

BACKGROUND

Shared decision making (SDM) is a health communication model promoting patient-centered care that has not been routinely utilized in mental health. Inconsistent definitions, models, measurement tools, and lack of sufficient evidence for the effectiveness of SDM interventions are potential contributors to the limited use of SDM in mental health.

AIMS

(1) Provide the first systematic analysis of global development trends and challenges of SDM research; (2) clarify the meaning, role, and measurement of SDM in mental health; (3) create a theoretical framework for key effective SDM components to guide future development and implementation of SDM interventions.

METHODS

A comprehensive search strategy was conducted in CINAHL, PubMed, Scopus, MEDLINE, EMBASE, Cochrane Library, Web of Science, Scopus, and PsycInfo. Included reviews focused on SDM interventions for prevention and/or treatment of mental illness in adults. A narrative synthesis was performed to capture the range of interventions, populations, measurement tools, comparisons, and outcomes.

RESULTS

10 systematic reviews of SDM in mental health were included with 100 nested studies spanning from 2006 to 2020. All reviews focused on dyadic and psychopharmacological decision-making. Primary outcomes of SDM in mental health interventions include treatment satisfaction, medication adherence, symptom severity, quality of life, and hospital readmissions. Participant-related factors unique to SDM in mental health, such as stigma and mental capacity, were not reported.

CONCLUSIONS

The current landscape of SDM in mental health is overwhelmingly disconnected from the needs and experiences of potential end-users; clients, clinicians, and family members. Most SDM interventions and tools were adapted from physical health and are mainly geared to psychopharmacological decision-making. The SDM in Mental Health Framework (SDM-MH), developed here, expands the scope of decisions to non-psychopharmacological discussions, diversifies the pool of SDM participants and settings, and offers potential primary target outcomes of SDM in mental health to reduce heterogeneity across studies.

Attitudes of Nurses Towards Searching Online for Medical Information for Personal Health Needs: Cross-Sectional Questionnaire Study

BACKGROUND

Use of online clinical health care information has become part of the skill set required by medical teams. Nurses believe that information quality and availability affect nursing care and methods. However, nurses tend not to exploit professional medical databases for evidence-based medical information for their personal needs. This phenomenon has received little research attention.

OBJECTIVE

This study aimed to address the knowledge gap around nurses' attitudes towards searching online for medical information for their personal needs (ie, for themselves and their families) by (1) evaluating the level of exposure to medical information and the effect on attitudes towards the use of online search options, (2) assessing the effect of the choice of a primary means of searching for medical information on the attitudes towards the use of online search options, and (3) gauging the influence of sociodemographic data and health status on nurses' attitudes towards searching online for medical information.

METHODS

Nurses employed in general departments in a general hospital (34/210, 16.2%), nursing home (42/200, 21.0%), and geriatric medical center (45/180, 25.0%) in Israel were invited to complete the eHealth Impact Questionnaire (alpha=.95). Questionnaires were distributed by nurses in charge of the general hospitalization wards. The data collection period was February to March 2018. The response rate was 40.3% (121/300).

RESULTS

Nurses tended to search for medical information for personal needs on social media (24/121, 19.8%) and TV (eg, health programs, health news; 23/121, 19.0%). Nurses who chose social media as their primary means of receiving general information had a positive attitude about using the online environment as a source for medical information compared to nurses who found information through other means (t119=4.44, P<.001). Nurses exposed to medical information via social media had a positive attitude towards the use of the internet to find medical information compared to nurses who were not exposed to social media (t119=3.04, P=.003). The attitudes of nurses towards the utility of online medical information for personal needs increased with better participant health status (F2,118=3.63, P=.03). However, the attitudes of participants with a chronic disease did not differ from those of healthy participants.

CONCLUSIONS

Nurses in Israel are less likely to use their professional skills and knowledge to search in professional databases for evidence-based medical information for their personal needs. Instead, they prefer medical information that is easy to access and not evidence-based, such as that on social media and TV. However, these search patterns for personal use may affect their clinical role, impair quality of care, and lead to incorrect medical decisions for their patients in the health care system. Therefore, during nursing education, training for searching skills, retrieval skills, and online search techniques for evidence-based medical information is vital for evidence-based practice.

A web application to involve patients in the medication reconciliation process: a user-centered usability and usefulness study

Objective

Medication reconciliation (MedRec) can improve patient safety by resolving medication discrepancies. Because information technology (IT) and patient engagement are promising approaches to optimizing MedRec, the SEAMPAT project aims to develop a MedRec IT platform based on two applications: the "patient app" and the "MedRec app." This study evaluates three dimensions of the usability (efficiency, satisfaction, and effectiveness) and usefulness of the patient app.

Methods

We performed a four-month user-centered observational study. Quantitative and qualitative data were collected. Participants completed the system usability scale (SUS) questionnaire and a second questionnaire on usefulness. Effectiveness was assessed by measuring the completeness of the medication list generated by the patient application and its correctness (ie medication discrepancies between the patient list and the best possible medication history). Qualitative data were collected from semi-structured interviews, observations and comments, and questions raised by patients.

Results

Forty-two patients completed the study. Sixty-nine percent of patients considered the patient app to be acceptable (SUS Score ≥ 70) and usefulness was high. The medication list was complete for a quarter of the patients (7/28) and there was a discrepancy for 21.7% of medications (21/97). The qualitative data enabled the identification of several barriers (related to functional and non-functional aspects) to the optimization of usability and usefulness.

Conclusions

Our findings highlight the importance and value of user-centered usability testing of a patient application implemented in "real-world" conditions. To achieve adoption and sustained use by patients, the app should meet patients' needs while also efficiently improving the quality of MedRec.

A technology-assisted life of recovery from psychosis

Developments in digital health technologies have the potential to expedite and strengthen the path towards recovery for people with psychosis. This perspective piece provides a snapshot of how a range of digital technologies can be deployed to support a young adult’s efforts to cope with schizophrenia-spectrum illness. In conjunction with a day in the life of this individual, we provide examples of innovations in digital health research designed for this clinical population, as well as brief summaries of the evidence supporting the usability, feasibility, or effectiveness of each approach. From early detection to ongoing symptom management and vocational rehabilitation, this day-in-the-life vignette provides an overview of the ways in which digital health innovations could be used in concert to augment, scaffold, and enhance schizophrenia-spectrum illness management and recovery.

Implementation and Use of a Client-Facing Web-Based Shared Decision-Making System (MyCHOIS-CommonGround) in Two Specialty Mental Health Clinics

Electronic shared-decision making programs may provide an assistive technology to support physician-patient communication. This mixed methods study examined use of a web-based shared decision-making program (MyCHOIS-CommonGround) by individuals receiving specialty mental health services, and identified qualitative factors influencing adoption during the first 18 months of implementation in two Medicaid mental health clinics. T-tests and χ2 analyses were conducted to assess differences in patient use between sites. Approximately 80% of patients in both clinics created a MyCHOIS-CommonGround user profile, but marked differences emerged between clinics in patients completing shared decision-making reports (79% vs. 28%, χ2(1) = 109.92, p < .01) and average number of reports (7.20 vs. 3.60, t = - 3.64, p < .01). Results suggest high penetration of computer-based programs in specialty mental health services is possible, but clinic implementation factors can influence patient use including leadership commitment, peer staff funding to support the program, and implementation strategy, most notably integration of the program within routine clinical workflow.

A User-Centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network

BACKGROUND

Medication discrepancies, which are a threat to patient safety, can be reduced by medication reconciliation (MedRec). MedRec is a complex process that can be supported by the use of information technology and patient engagement. Therefore, the SEAMPAT project aims to develop a MedRec IT platform based on two applications. The application for the professionals is called: the "MedRec app".

OBJECTIVE

In the present study, we aimed to describe the development and usability testing of the MedRec app, reporting results of a three iterations user-centered usability evaluation.

METHODS

We used a three phase iterative user-centered study spread over 16 months. At each phase, the usability evaluation included several methods (observations, questionnaires, and follow-up discussions with participants) to collect quantitative and qualitative data in order to improve the current prototype and evolve to the next prototype.

RESULTS

In total, 48 healthcare professionals (25 general practitioners and 23 hospital clinicians) participated to the MedRec app evaluation. There were 14, 32 and 5 participants for phases 1, 2 and 3 respectively. At each phase, many design modifications were done to strengthen usability. Concerning usability, participants considered the prototypes as an acceptable interface with a median System Usability Score of 73 at phase 2 and 75 at phase 3. Participants emphasized the need for improvements concerning workflow integration, usefulness and interoperability.

CONCLUSION

The MedRec app was perceived as being useful, usable and satisfying. However, further improvements are required in several usability aspects. Our study demonstrates the importance of conducting usability assessments before investing time and resources in a large study evaluating the effect of an eMedRec approach on clinical outcomes. Our findings may also increase the chances of acceptability and sustained use over time by clinicians.

Potential Applications of Digital Technology in Assessment, Treatment, and Self-help for Hallucinations

The field of digital mental health is rapidly expanding with digital tools being used in assessment, intervention, and supporting self-help. The application of digital mental health to hallucinations is, however, at a very early stage. This report from a working group of the International Consortium on Hallucinations Research considers particular synergies between the phenomenon of hallucinations and digital tools that are being developed. Highlighted uses include monitoring and managing intermittently occurring hallucinations in daily life; therapeutic applications of audio and video media including virtual and augmented reality; targeting verbal aspects of hallucinations; and using avatars to represent hallucinatory voices. Although there is a well-established Internet-based peer support network, digital resources for hallucinations have yet to be implemented in routine practice. Implementation may benefit from identifying how to market resources to the broad range of populations who experience hallucinations and identifying sustainable funding models. It is envisaged that digital tools will contribute to improved self-management and service provision for people experiencing hallucinations.

A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach

BACKGROUND

Relatives of people experiencing bipolar mood episodes or psychosis face a multitude of challenges (eg, social isolation, limited coping strategies, and issues with maintaining relationships). Despite this, there is limited informational and emotional support for people who find themselves in supporting or caring roles. Digital technologies provide us with an opportunity to offer accessible tools, which can be used flexibly to provide evidence-based information and support, allowing relatives to build their understanding of mental health problems and learn from others who have similar experiences. However, to design tools that are useful to relatives, we first need to understand their needs.

OBJECTIVE

The aim of this study was to use a user-centered design approach to develop an accessible Web-based intervention, based on the Relatives Education And Coping Toolkit (REACT) booklet, to support the informational and emotional needs of relatives of people experiencing psychosis or bipolar disorder.

METHODS

We engaged relatives of people with experiences of bipolar disorder or psychosis in workshops to identify their needs and design requirements for developing a Web-based version of a paper-based toolkit. We used a 2-phase qualitative approach to explore relatives' views on content, design, and functionalities, which are considered to be engaging and useful in a Web-based intervention. In phase 1, we consulted 24 relatives in 2 workshops to better understand their existing support infrastructure, their barriers for accessing support, unmet needs, and relatives' views on online support. On the basis of the results of these workshops, we developed a set of design considerations to be explored in a smaller workshop. Workshop 3 then involved working with 2 digitally literate relatives to design a usable and acceptable interface for our Web-based toolkit. Finally, in phase 2, we conducted a heuristic evaluation to assess the usability of the toolkit.

RESULTS

Our findings indicated that relatives require technologies that (1) they can place their trust in, particularly when discussing a highly sensitive topic, (2) enable learning from the lived experiences of others while retaining confidentiality, and (3) they can work through at their own pace in a personalized manner.

CONCLUSIONS

Our study highlights the need for providing a trustworthy, supportive tool where relatives can engage with people who have similar experiences to their own. Our heuristic evaluation showed promise in terms of perceived usability of the REACT Web-based intervention. Through this work, we emphasize the need to involve stakeholders with various characteristics, including users with limited computer literacy or experience in online support.

Emotionally Oriented Analysis of the Experiences of Visually Impaired People on Facebook

With technological advancements, there has been a vast increase in the number of companies that fight over their market share. In search of a differentiating factor, companies are investing more and more in their products’ emotional designs. This researched work has evaluated the affects that are caused in visually impaired people when using Facebook's features and then compared them with the experiences of sighted users. To do that, these two types of Facebook users were subjected to a questionnaire that was based on the PANAS affect scale. Once the information was collected, statistics were employed so as to evaluate both users’ feelings. The results have shown that there were significant statistical differences between the sighted and the visually impaired users when the “affects” were evaluated by using the PANAS tool. The five “negative affects” that were selected (Irritability, Uselessness, Frustration, Sadness, and Confusion) were largely more relevant for the blind people in most of the evaluated features. This has indicated some serious accessibility problems. However, a high frequency of the five “positive affects” that were considered (Satisfaction, Pleasantness, Surprise, Excitement, Interest, and Determination) were additionally observed for both of these two groups. These results were interpreted as feelings of both social inclusion and social exclusion, indicating the possibility of exploring technological devices that were unavailable not long ago. After analyzing their experiences in their usage of the Facebook features, the findings have also highlighted the many differing emotions that are felt by the visually impaired and the sighted users. The resulting outcomes have indicated that there are some issues that are still open to problems and difficulties. Moreover, these issues involve human-computer interactions. Nevertheless, fortunately, there is light at the end of the tunnel, as will be revealed.

Preferences Related to the Use of Mobile Apps as Dental Patient Educational Aids: A Pilot Study

PURPOSE

Numerous patient education apps have been developed to explain dental treatment. The purpose of this study was to assess perceptions and preferences regarding the use of apps in dental settings.

MATERIALS AND METHODS

Four patient education apps describing fixed partial dentures were demonstrated to participants (N = 25). Questions about each app were asked using a semi-structured interview format to assess participants' opinions about each app's content, images, features, and use. Sessions were analyzed via note-based methods for thematic coding.

RESULTS

Participants believed that apps should be used in conjunction with a dentist's explanation about a procedure. They desired an app that could be tailored for scope of content. Participants favored esthetic images of teeth that did not show structural anatomy, such as tooth roots, and preferred interactive features.

CONCLUSIONS

Patient education apps may be a valuable tool to enhance patient-provider communication in dental settings. Participants exhibited varying preferences for different features among the apps and expressed the desire for an app that could be personalized to each patient. Additional research is needed to assess whether the use of apps improves oral health literacy and informed consent among patients.

Transitions from biomedical to recovery-oriented practices in mental health: a scoping review to explore the role of Internet-based interventions

Background

The Internet is transforming mental health care services by increasing access to, and potentially improving the quality of, care. Internet-based interventions in mental health can potentially play a role in transitions from biomedical to recovery-oriented research and practices, but an overview of what this may entail, current work, and issues that need addressing, is lacking. The objective of this study is to describe Internet-based recovery-oriented interventions (referred to as e-recovery) and current research, and to identify gaps and issues relevant to advancing recovery research and practices through opportunities provided by the Internet.

Methods

Five iterative stages of a scoping review framework were followed in searching and analyzing the literature. A recovery framework with four domains and 16 themes was used to deductively code intervention characteristics according to their support for recovery-oriented practices. Only Internet-based interventions used in conjunction with ongoing care were included.

Results

Twenty studies describing six e-recovery interventions were identified and originated in Australia, Finland, the Netherlands, Norway and USA. The domain supporting personal recovery was most clearly reflected in interventions, whereas the last three domains, i.e., promoting citizenship, organizational commitment and working relationship were less evident. Support for the formulation and follow-up of personal goals and preferences, and in accessing peer-support, were the characteristics shared by most interventions. Three of the six studies that employed a comparison group used randomization, and none presented definitive findings. None used recovery-oriented frameworks or specific recovery outcome measures. Four of the interventions were specific to a diagnosis.

Conclusion

Research about how technologies might aid in illuminating and shaping recovery processes is in its formative stages. We recommend that future e-recovery research and innovation attend to four dimensions: evidence-supported interventions, new knowledge about personal recovery, values-based approaches and Internet as a facilitator for organizational transformation. The incremental changes facilitated by e-recovery may help propel a shift in mental health care toward recovery-oriented practices.

How Do People Experiencing Schizophrenia Spectrum Disorders or Other Psychotic Disorders Use the Internet to Get Information on Their Mental Health? Literature Review and Recommendations

BACKGROUND

Studies show that the Internet has become an influential source of information for people experiencing serious psychiatric conditions such as schizophrenia spectrum disorders or other psychotic disorders, among which the rate of Internet users is growing, with rates ranging from 33.3% to 79.5% given the country. Between 20.5% and 56.4% of these Internet users seek mental health information.

OBJECTIVE

Focusing on this population's Web searches about their mental health, this paper examines what type of content they look for and what could be the benefits and disadvantages of this navigation.

METHODS

We conducted a literature review through medical and psychological databases between 2000 and 2015 using the keywords "Internet," "Web," "virtual," "health information," "schizophrenia," "psychosis," "e-mental health," "e-support," and "telepsychiatry."

RESULTS

People experiencing schizophrenia spectrum disorders or other psychotic disorders wish to find on the Internet trustful, nonstigmatizing information about their disease, flexibility, security standards, and positive peer-to-peer exchanges. E-mental health also appears to be desired by a substantial proportion of them. In this field, the current developments towards intervention and early prevention in the areas of depression and bipolar and anxiety disorders become more and more operational for schizophrenia spectrum disorders and other psychotic disorders as well. The many benefits of the Internet as a source of information and support, such as empowerment, enhancement of self-esteem, relief from peer information, better social interactions, and more available care, seem to outbalance the difficulties.

CONCLUSIONS

In this paper, after discussing the challenges related to the various aspects of the emergence of the Internet into the life of people experiencing schizophrenia spectrum disorders or other psychotic disorders, we will suggest areas of future research and practical recommendations for this major transition.

An eHealth information technology platform to help the treatment of mental disorders

For this project, we have used new technologies to create a new channel of communication between doctors and patients in the treatment of mental disorders. We have created a web application using an adaptable design accessible from any mobile device, which allows doctors to adapt their patients' therapy to real-time knowledge of their current condition. In turn, patients can express their mood state with respect to the component elements of their therapy.

Supporting shared decision making beyond consumer-prescriber interactions: Initial development of the CommonGround fidelity scale

Shared decision-making has become a central tenet of recovery-oriented, person-centered mental health care, yet the practice is not always transferred to the routine psychiatric visit. Supporting the practice at the system level, beyond the interactions of consumers and medication prescribers, is needed for successful adoption of shared decision-making. CommonGround is a systemic approach, intended to be part of a larger integration of shared decision-making tools and practices at the system level. We discuss the organizational components that CommonGround uses to facilitate shared decision-making, and we present a fidelity scale to assess how well the system is being implemented.

Routine outcome measurement in the Netherlands - A focus on benchmarking

Routine outcome measurement (ROM) is a 'hot' topic in the Netherlands. Over recent years the Netherlands have developed a centralized monitoring system for all reimbursed mental health interventions, in an attempt to improve the quality of care. The Foundation for Benchmarking Mental Health (SBG) is an independent knowledge centre for mental health providers and insurance companies. It was founded to organize and manage the countrywide ROM initiative. A Dutch countrywide ROM initiative is appealing, and the procedures in the Netherlands are described. However, the national ROM system was oversold. Arguments are discussed. It would have been a far better strategy if insurance companies and authorities had not focused on a national system but stimulated local data collection and requested a managerial plan-do-check-act (PDCA) cycle to stimulate service improvements from year to year. Within the same service, chances are higher that the same kind of clientele is served from year to year and therefore it will be easier to interpret the data. The ROM should regain its clinical focus. Mobile ROM systems using smartphones that collect sampled experiences could be an interesting future solution.

Acceptability of Interventions Delivered Online and Through Mobile Phones for People Who Experience Severe Mental Health Problems: A Systematic Review

Background

Psychological interventions are recommended for people with severe mental health problems (SMI). However, barriers exist in the provision of these services and access is limited. Therefore, researchers are beginning to develop and deliver interventions online and via mobile phones. Previous research has indicated that interventions delivered in this format are acceptable for people with SMI. However, a comprehensive systematic review is needed to investigate the acceptability of online and mobile phone-delivered interventions for SMI in depth.

Objective

This systematic review aimed to 1) identify the hypothetical acceptability (acceptability prior to or without the delivery of an intervention) and actual acceptability (acceptability where an intervention was delivered) of online and mobile phone-delivered interventions for SMI, 2) investigate the impact of factors such as demographic and clinical characteristics on acceptability, and 3) identify common participant views in qualitative studies that pinpoint factors influencing acceptability.

Methods

We conducted a systematic search of the databases PubMed, Embase, PsycINFO, CINAHL, and Web of Science in April 2015, which yielded a total of 8017 search results, with 49 studies meeting the full inclusion criteria. Studies were included if they measured acceptability through participant views, module completion rates, or intervention use. Studies delivering interventions were included if the delivery method was online or via mobile phones.

Results

The hypothetical acceptability of online and mobile phone-delivered interventions for SMI was relatively low, while actual acceptability tended to be high. Hypothetical acceptability was higher for interventions delivered via text messages than by emails. The majority of studies that assessed the impact of demographic characteristics on acceptability reported no significant relationships between the two. Additionally, actual acceptability was higher when participants were provided remote online support. Common qualitative factors relating to acceptability were safety and privacy concerns, the importance of an engaging and appealing delivery format, the inclusion of peer support, computer and mobile phone literacy, technical issues, and concerns about the impact of psychological state on intervention use.

Conclusions

This systematic review provides an in-depth focus on the acceptability of online and mobile phone-delivered interventions for SMI and identified the need for further research in this area. Based on the results from this review, we recommend that researchers measure both hypothetical and actual acceptability to identify whether initial perceptions of online and mobile phone-delivered interventions change after access. In addition, more focus is needed on the potential impact of demographic and clinical characteristics on acceptability. The review also identified issues with module completion rates and intervention use as measures of acceptability. We therefore advise researchers to obtain qualitative reports of acceptability throughout each phase of intervention development and testing. Further implications and opportunities for future research are discussed.

Effectiveness of Social Media Interventions for People With Schizophrenia: A Systematic Review and Meta-Analysis

Background

Recent studies have shown that people with serious mental disorders spend time online for the purposes of disclosure, information gathering, or gaming. However, coherent information on the effects of social media on treatment for people with schizophrenia is still lacking.

Objective

Our aim was to determine the effects of social media interventions for supporting mental health and well-being among people with schizophrenia.

Methods

A systematic review and meta-analysis were undertaken to determine the effects of social media interventions for supporting mental health and well-being among people with schizophrenia. Ten databases were searched, while search parameters included English-only manuscripts published prior to June 25, 2015. Study appraisals were made independently by 2 reviewers, and qualitative and quantitative syntheses of data were conducted.

Results

Out of 1043 identified records, only two randomized studies of moderate quality (three records, total N=331, duration 12 months) met the inclusion criteria. Participants were people with schizophrenia spectrum or an affective disorder. Social media was used as part of Web-based psychoeducation, or as online peer support (listserv and bulletin board). Outcome measures included perceived stress, social support, and disease-related distress. At 3 months, participants with schizophrenia in the intervention group reported lower perceived stress levels ( P=.04) and showed a trend for a higher perceived level of social support ( P=.06). However, those who reported more positive experiences with the peer support group also reported higher levels of psychological distress ( P=.01).

Conclusions

Despite using comprehensive searches from 10 databases, we found only two studies, whereas numerous reports have been published citing the benefits of social media in mental health. Findings suggest the effects of social media interventions are largely unknown. More research is needed to understand the effects of social media, for users with and without mental illness, in order to determine the impact on mental well-being ofsocial media use as well as its risks.

Snapshot of the supports of communication used by patients at a French psychiatric hospital: a digital or social division?

BACKGROUND

The role of information and communications technology is becoming increasingly prevalent in daily life and in the organization of medical care: are some people being left out?

AIMS

To evaluate access to and the uses of communication resources by psychiatric patients, focusing on the means of communication (e.g. mobile phones and computers), access and frequency of internet use.

METHODS

A questionnaire was distributed, over a period of 1 week, to inpatients or day hospitalised patients aged over 12 years in all care units.

RESULTS

Access to and the uses of modern communication resources were lower than in the general population. Among places and means of internet consultation, the personal computer was most often cited, but only by 34%, and the use of mobile phones is still not widespread. Finally, day hospitalised subjects, the elderly, or subjects being treated in the psychosis care sector use internet and technology the least.

CONCLUSIONS

Some differences exist between this population with mental illness and the general population on the use of new communication technologies. The possibility of integrating these techniques in individualized psychiatric care requires prior equipment and/or updates.

A systematic review of mixed methods research on human factors and ergonomics in health care

This systematic literature review provides information on the use of mixed methods research in human factors and ergonomics (HFE) research in health care. Using the PRISMA methodology, we searched four databases (PubMed, PsycInfo, Web of Science, and Engineering Village) for studies that met the following inclusion criteria: (1) field study in health care, (2) mixing of qualitative and quantitative data, (3) HFE issues, and (4) empirical evidence. Using an iterative and collaborative process supported by a structured data collection form, the six authors identified a total of 58 studies that primarily address HFE issues in health information technology (e.g., usability) and in the work of healthcare workers. About two-thirds of the mixed methods studies used the convergent parallel study design where quantitative and qualitative data were collected simultaneously. A variety of methods were used for collecting data, including interview, survey and observation. The most frequent combination involved interview for qualitative data and survey for quantitative data. The use of mixed methods in healthcare HFE research has increased over time. However, increasing attention should be paid to the formal literature on mixed methods research to enhance the depth and breadth of this research.

Horizon 2020 priorities in clinical mental health research: results of a consensus-based ROAMER expert survey

Within the ROAMER project, which aims to provide a Roadmap for Mental Health Research in Europe, a two-stage Delphi survey among 86 European experts was conducted in order to identify research priorities in clinical mental health research. Expert consensus existed with regard to the importance of three challenges in the field of clinical mental health research: (1) the development of new, safe and effective interventions for mental disorders; (2) understanding the mechanisms of disease in order to be able to develop such new interventions; and (3) defining outcomes (an improved set of outcomes, including alternative outcomes) to use for clinical mental health research evaluation. Proposed actions involved increasing the utilization of tailored approaches (personalized medicine), developing blended eHealth/mHealth decision aids/guidance tools that help the clinician to choose between various treatment modalities, developing specific treatments in order to better target comorbidity and (further) development of biological, psychological and psychopharmacological interventions. The experts indicated that addressing these priorities will result in increased efficacy and impact across Europe; with a high probability of success, given that Europe has important strengths, such as skilled academics and a long research history. Finally, the experts stressed the importance of creating funding and coordinated networking as essential action needed in order to target the variety of challenges in clinical mental health research.

Automated telehealth for managing psychiatric instability in people with serious mental illness

BACKGROUND

Serious mental illness (SMI) with psychiatric instability accounts for disproportionately high use of emergency room visits and hospitalizations.

AIM

To evaluate the effectiveness of an automated telehealth intervention supported by nurse health care management for improving psychiatric illness management and reducing acute service use among individuals with SMI and psychiatric instability.

METHODS

Thirty-eight individuals with SMI received the automated telehealth intervention for 6 months. Psychiatric symptoms, illness self-management, and self-reported service use (emergency room visits and hospital admissions) were collected at baseline, 3- and 6-months. Measures of quality of life, health indicators, and subjective health status were also collected.

RESULTS

Participants demonstrated improvements in self-reported psychiatric symptoms and illness self-management skills, an 82% decrease in hospital admissions (from 76 to 14 hospitalizations, p < 0.001) and a 75% decrease in emergency room visits (from 63 to 16 visits, p < 0.001). Improvements were also observed in quality of life, severity of depressive symptoms, and mental health status.

CONCLUSION

These highly promising findings support the use of an automated telehealth device monitored by a nurse care manager for people with SMI, and highlight the potential for cost savings through reductions in acute health care utilization.

Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions

BACKGROUND

Internet and mobile-based interventions provide a unique opportunity to deliver cost-effective, accessible, time-unlimited support to people with psychosis. The aims of this study were to systematically compile and analyze the evidence on the acceptability, feasibility, safety and benefits of online and mobile-based interventions for psychosis.

METHODS

Systematic review of peer-reviewed studies examining the usability, acceptability, feasibility, safety or efficacy of user-led, Internet or mobile-based interventions, with at least 80% of participants diagnosed with schizophrenia-spectrum disorders.

RESULTS

Of 38 potentially relevant articles, 12 were eligible for inclusion. Interventions included web-based psycho-education; web-based psycho-education plus moderated forums for patients and supporters; integrated web-based therapy, social networking and peer and expert moderation; web-based CBT; personalized advice based on clinical monitoring; and text messaging interventions. Results showed that 74-86% of patients used the web-based interventions efficiently, 75-92% perceived them as positive and useful, and 70-86% completed or were engaged with the interventions over the follow-up. Preliminary evidence indicated that online and mobile-based interventions show promise in improving positive psychotic symptoms, hospital admissions, socialization, social connectedness, depression and medication adherence.

CONCLUSIONS

Internet and mobile-based interventions for psychosis seem to be acceptable and feasible and have the potential to improve clinical and social outcomes. The heterogeneity, poor quality and early state of current research precludes any definite conclusions. Future research should investigate the efficacy of online and mobile interventions through controlled, well-powered studies, which investigate intervention and patient factors associated with take-up and intervention effects.

A web-based tool to support shared decision making for people with a psychotic disorder: randomized controlled trial and process evaluation

Background

Mental health policy makers encourage the development of electronic decision aids to increase patient participation in medical decision making. Evidence is needed to determine whether these decision aids are helpful in clinical practice and whether they lead to increased patient involvement and better outcomes.

Objective

This study reports the outcome of a randomized controlled trial and process evaluation of a Web-based intervention to facilitate shared decision making for people with psychotic disorders.

Methods

The study was carried out in a Dutch mental health institution. Patients were recruited from 2 outpatient teams for patients with psychosis (N=250). Patients in the intervention condition (n=124) were provided an account to access a Web-based information and decision tool aimed to support patients in acquiring an overview of their needs and appropriate treatment options provided by their mental health care organization. Patients were given the opportunity to use the Web-based tool either on their own (at their home computer or at a computer of the service) or with the support of an assistant. Patients in the control group received care as usual (n=126). Half of the patients in the sample were patients experiencing a first episode of psychosis; the other half were patients with a chronic psychosis. Primary outcome was patient-perceived involvement in medical decision making, measured with the Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE). Process evaluation consisted of questionnaire-based surveys, open interviews, and researcher observation.

Results

In all, 73 patients completed the follow-up measurement and were included in the final analysis (response rate 29.2%). More than one-third (48/124, 38.7%) of the patients who were provided access to the Web-based decision aid used it, and most used its full functionality. No differences were found between the intervention and control conditions on perceived involvement in medical decision making (COMRADE satisfaction with communication: F_1,68=0.422, P=.52; COMRADE confidence in decision: F_1,67=0.086, P=.77). In addition, results of the process evaluation suggest that the intervention did not optimally fit in with routine practice of the participating teams.

Conclusions

The development of electronic decision aids to facilitate shared medical decision making is encouraged and many people with a psychotic disorder can work with them. This holds for both first-episode patients and long-term care patients, although the latter group might need more assistance. However, results of this paper could not support the assumption that the use of electronic decision aids increases patient involvement in medical decision making. This may be because of weak implementation of the study protocol and a low response rate.

Trial Registration

Dutch Trial Register (NTR) trial number: 10340; http://www.trialregister.nl/trialreg/admin/rctsearch.asp?Term=10340 (Archived by WebCite at http://www.webcitation.org/6Jj5umAeS).

An analysis of online health information on schizophrenia or related conditions: a cross-sectional survey

BACKGROUND

Around 20% of those who seek health information online, search specifically for mental health. However, little is known about the nature of the online health information offered by two European countries, Finland and Greece, which are characterized by markedly differing levels of Internet access and online health information seeking. This study aims to assess, describe and compare websites, written in two European, non-English languages (Finnish and Greek) that appear first after performing an online search concerning schizophrenia or related conditions.

METHODS

The first 20 results from four search terms (searched in Finnish and Greek) in the Web search engine 'Google' were screened. A total of 160 websites were retrieved (80 Finnish, 80 Greek) and evaluated using a preformulated coding system which consisted of websites' indicators, such as: types, characteristics, accountability, interactivity, aesthetics and content. Differences between websites were evaluated with Chi-Square or Fisher's Exact tests for categorical data and independent t-tests for parametric data.

RESULTS

Twenty-four Finnish and thirty-four Greek websites (36% in total) were included. Almost two-thirds (62%, n=36) were owned by an organization, compared to 17% (n=10) by an individual. In both countries, aesthetics had the highest score (possible range 0-4, mean = 2.6, SD = .62), while interactivity the lowest (range 0-5, mean = 1.79, SD = .87). There were no statistically significant differences among the accountability, interactivity and aesthetics scores of the Finnish and Greek websites.

CONCLUSIONS

All assessed indicators suggest there is a need to improve Finnish and Greek online information about schizophrenia or related conditions. The poor website interactivity is of particular concern given the challenges faced by the target group. The findings can be used to guide the development and dissemination of online mental health information aimed at Finnish and Greek online health-seekers.

A type 2 diabetes prevention website for african americans, Caucasians, and mexican americans: formative evaluation

BACKGROUND

The majority of Americans now access the Internet, thereby expanding prospects for Web-based health-related education and intervention. However, there remains a digital divide among those with lower income and education, and among Spanish-speaking populations in the United States. Additional concerns are the low eHealth literacy rate among these populations and their interest in Internet-delivered interventions with these components. Given these factors, combined with the prevalence of type 2 diabetes among low socioeconomic status and Spanish-speaking Americans, strides need to be taken to reach these populations with online tools for diabetes prevention and management that are at once accessible and efficacious.

OBJECTIVE

Using a formative evaluation of an eHealth diabetes prevention and control website, we tested the extent to which African Americans, Caucasians, and Mexican Americans at risk for type 2 diabetes gained knowledge and intended to modify their dietary intake and physical activity subsequent to viewing the website. We also examined their general Internet use patterns related to type 2 diabetes.

METHODS

A mixed methods approach was undertaken. The diabetes prevention and control website provided educational and behavioral change information in English and Spanish. For this study, eligible participants (1) completed a prequantitative survey, (2) interacted with the website, (3) completed a qualitative interview, and (4) completed a postquantitative survey.

RESULTS

After finding a significant differences in posttest diabetes knowledge scores (P<.001), a regression analysis controlling for pretest score, health literacy, ethnicity, Transtheoretical Model Stage for exercise and fruit and vegetable consumption, and Internet literacy was conducted. Internet literacy score (P=.04) and fruit and vegetable consumption stage (P<.001) were significantly associated with posttest scores indicating that those in precontemplation stage and with low Internet literacy scores were less likely to show improved diabetes knowledge scores. We found significant difference in posttest intention to eat a healthy diet each day in the next 2 months after controlling for pretest score, health literacy, ethnicity, Transtheoretical Model Stage for fruit and vegetable consumption and Internet literacy. Those in the Action stage of the Transtheoretical model for exercise were significantly less likely (P=.023) to improve the posttest score for intention to eat a healthy diet compared to those in the Preparation stage for exercise. We also found that health information is sought commonly across ethnic groups, but that diabetes-related information is less commonly sought even among those at risk. Other specific ethnic usage patterns were identified in the qualitative data including content sought on Web searches and technology used to access the Internet.

CONCLUSIONS

This study provides in-depth qualitative insight into the seeking, access, and use of Web-based health information across three ethnic groups in two languages. Additionally, it provides evidence from pre-post measures of exposure to Web-based health content and related changes in diabetes knowledge and intention to eat a healthy diet.

Generating personalized advice for schizophrenia patients

The results of routine patient assessments in psychiatric healthcare in the Northern Netherlands are primarily used to support clinicians. We developed Wegweis, a web-based advice platform, to make this data accessible and understandable for patients.

OBJECTIVE

We show that a fully automated explanation and interpretation of assessment results for schizophrenia patients, which prioritizes the information in the same way that a clinician would, is possible and is considered helpful and relevant by patients. The goal is not to replace the clinician but rather to function as a second perspective and to enable patient empowerment through knowledge.

METHODS

We have developed and implemented an ontology-based approach for selecting and ranking information for schizophrenia patients based on their routine assessment results. Our approach ranks information by severity of associated schizophrenia-related problems and uses an ontology to decouple problems from advice, which adds robustness to the system, because advice can be inferred for problems that have no exact match.

RESULTS

We created a problem ontology, validated by a group of experts, to combine and interpret the results of multiple schizophrenia-specific questionnaires. We designed and implemented a novel ontology-based algorithm for ranking and selecting advice, based on questionnaire answers. We designed, implemented, and illustrated Wegweis, a proof of concept for our algorithm, and, to the best of our knowledge, the first fully automated interpretation of assessment results for patients suffering from schizophrenia. We evaluated the system vis-à-vis the opinions of clinicians and patients in two experiments. For the task of identifying important problems based on MANSA questionnaires (the MANSA is a satisfaction questionnaire commonly used in schizophrenia assessments), our system corresponds to the opinion of clinicians 94% of the time for the first three problems and 72% of the time, overall. Patients find two out of the first three advice topics selected by the system to be relevant and roughly half of the advice topics overall.

CONCLUSIONS

Our findings suggest that an approach that uses problem severities to identify important problems for a patient corresponds closely to the way a clinician thinks. Furthermore, after applying a severity threshold, the majority of advice units selected by the system are considered relevant by the patients. Our findings pave the way for the development of systems that facilitate patient-centered care for chronic illnesses by automating the sharing of assessment results between patient and clinician.

Active assistance technology for health-related behavior change: an interdisciplinary review

Background

Information technology can help individuals to change their health behaviors. This is due to its potential for dynamic and unbiased information processing enabling users to monitor their own progress and be informed about risks and opportunities specific to evolving contexts and motivations. However, in many behavior change interventions, information technology is underused by treating it as a passive medium focused on efficient transmission of information and a positive user experience.

Objective

To conduct an interdisciplinary literature review to determine the extent to which the active technological capabilities of dynamic and adaptive information processing are being applied in behavior change interventions and to identify their role in these interventions.

Methods

We defined key categories of active technology such as semantic information processing, pattern recognition, and adaptation. We conducted the literature search using keywords derived from the categories and included studies that indicated a significant role for an active technology in health-related behavior change. In the data extraction, we looked specifically for the following technology roles: (1) dynamic adaptive tailoring of messages depending on context, (2) interactive education, (3) support for client self-monitoring of behavior change progress, and (4) novel ways in which interventions are grounded in behavior change theories using active technology.

Results

The search returned 228 potentially relevant articles, of which 41 satisfied the inclusion criteria. We found that significant research was focused on dialog systems, embodied conversational agents, and activity recognition. The most covered health topic was physical activity. The majority of the studies were early-stage research. Only 6 were randomized controlled trials, of which 4 were positive for behavior change and 5 were positive for acceptability. Empathy and relational behavior were significant research themes in dialog systems for behavior change, with many pilot studies showing a preference for those features. We found few studies that focused on interactive education (3 studies) and self-monitoring (2 studies). Some recent research is emerging in dynamic tailoring (15 studies) and theoretically grounded ontologies for automated semantic processing (4 studies).

Conclusions

The potential capabilities and risks of active assistance technologies are not being fully explored in most current behavior change research. Designers of health behavior interventions need to consider the relevant informatics methods and algorithms more fully. There is also a need to analyze the possibilities that can result from interaction between different technology components. This requires deep interdisciplinary collaboration, for example, between health psychology, computer science, health informatics, cognitive science, and educational methodology.

Clinicians' perspectives on a Web-based system for routine outcome monitoring in old-age psychiatry in the Netherlands

BACKGROUND

In health care, the use of physical parameters to monitor physical disease progress is common. In mental health care, the periodic measurement of a client's functioning during treatment, or routine outcome monitoring, has recently become important. Online delivery of questionnaires has the potential to reduce clinicians' resistance to the implementation of routine outcome monitoring. Online delivery enables clinicians to receive results on a questionnaire in a graphic directly after data entry. This gives them insight into the progress of a client at a single glance.

OBJECTIVE

To explore clinicians' perspectives on a routine outcome monitoring procedure where questionnaires and feedback on scores were delivered online. Questionnaires could also be filled out on paper and then entered into the online system by a research assistant.

METHODS

In 2009 we sent an online survey, consisting of five yes-or-no questions and six open-ended questions, to all clinicians in the 14 mental health care organizations working with the routine outcome monitoring system in the Netherlands. Of the 172 clinicians contacted, 80 (47%) opened the link and 70 of these 80 (88%) clinicians completed the survey.

RESULTS

Clinicians seldom used the graphical feedback from the Web-based system, which indicates that direct feedback on scores did not enhance the implementation of routine outcome monitoring. Integration into the electronic patient record and more training on interpretation and implementation of feedback in daily practice were seen as the primary points for further improvement. It was mainly the availability of a research assistant that made the routine outcome monitoring procedure feasible.

CONCLUSIONS

Without a research assistant and training in the interpretation of outcomes, software programs alone cannot ensure effective implementation of monitoring activities in everyday practice.