Informing evaluation of a smartphone application for people with acquired brain injury: a stakeholder engagement study

How can patients shape digital medicine? A rapid review of patient and public involvement and engagement in the development of digital health technologies for neurological conditions

INTRODUCTION

Patient and Public Involvement and Engagement (PPIE) involves working 'with' or 'by' patients and the public, rather than 'to,' 'about,' or 'for' them, and is integral to neurological and digital health research. This rapid review examined PPIE integration in the development and implementation of digital health technologies for neurological conditions.

METHODS

Key terms were input into six databases. Included articles were qualitative studies or PPIE activities involving patient perspectives in shaping digital health technologies for neurological conditions. Bias was evaluated using the NICE qualitative checklist, with reporting following PRISMA guidelines.

RESULTS

2,140 articles were identified, with 28 included. Of these, 25 were qualitative studies, and only three were focused PPIE activities. Patient involvement was mostly limited to one-off consultations during development.There was little evidence of PPIE during implementation, and minimal reporting on its impact.

CONCLUSIONS

PPIE has been inconsistently reported in this research area, highlighting the need for more guidance and best-practice examples This review used a UK-based definition of PPIE, which may have excluded relevant activities from other countries. Future reviews should broaden terminology to capture PPIE integration globally.

Current Implementation of Digital Health in Chronic Disease Management: Scoping Review

BACKGROUND

Approximately 1 in 3 adults live with multiple chronic diseases. Digital health is being harnessed to improve continuity of care and management of chronic diseases. However, meaningful uptake of digital health for chronic disease management remains low. It is unclear how these innovations have been implemented and evaluated.

OBJECTIVE

This scoping review aims to identify how digital health innovations for chronic disease management have been implemented and evaluated: what implementation frameworks, methods, and strategies were used; how successful these strategies were; key barriers and enablers to implementation; and lessons learned and recommendations shared by study authors.

METHODS

We used the Joanna Briggs Institute methodology for scoping reviews. Five databases were searched for studies published between January 2015 and March 2023: PubMed, Scopus, CINAHL, PsycINFO, and IEEE Xplore. We included primary studies of any study design with any type of digital health innovations for chronic diseases that benefit patients, caregivers, or health care professionals. We extracted study characteristics; type of digital health innovation; implementation frameworks, strategies, and outcome measures used; barriers and enablers to implementation; lessons learned; and recommendations reported by study authors. We used established taxonomies to synthesize extracted data. Extracted barriers and enablers were grouped into categories for reporting. Descriptive statistics were used to consolidate extracted data.

RESULTS

A total of 252 studies were included, comprising mainly mobile health (107/252, 42.5%), eHealth (61/252, 24.2%), and telehealth (97/252, 38.5%), with some studies involving more than 1 innovation. Only 23 studies (23/252, 9.1%) reported using an implementation science theory, model, or framework; the most common were implementation theories, classic theories, and determinant frameworks, with 7 studies each. Of 252 studies, 144 (57.1%) used 2 to 5 implementation strategies. Frequently used strategies were "obtain and use patient or consumer feedback" (196/252, 77.8%); "audit and provide feedback" (106/252, 42.1%); and piloting before implementation or "stage implementation scale-up" (85/252, 33.7%). Commonly measured implementation outcomes were acceptability, feasibility, and adoption of the digital innovation. Of 252 studies, 247 studies (98%) did not measure service outcomes, while patient health outcomes were measured in 89 studies (35.3%). The main method used to assess outcomes was surveys (173/252, 68.7%), followed by interviews (95/252, 37.7%). Key barriers impacting implementation were data privacy concerns and patient preference for in-person consultations. Key enablers were training for health care workers and personalization of digital health features to patient needs.

CONCLUSIONS

This review generated a summary of how digital health in chronic disease management is currently implemented and evaluated and serves as a useful resource for clinicians, researchers, health system managers, and policy makers planning real-world implementation. Future studies should investigate whether using implementation science frameworks, including how well they are used, would yield better outcomes compared to not using them.

Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review

BACKGROUND

The field of eHealth is growing rapidly and chaotically. Health care professionals need guidance on reviewing and assessing health-related smartphone apps to propose appropriate ones to their patients. However, to date, no framework or evaluation tool fulfills this purpose.

OBJECTIVE

Before developing a tool to help health care professionals assess and recommend apps to their patients, we aimed to create an overview of published criteria to describe and evaluate health apps.

METHODS

We conducted a systematic review to identify existing criteria for eHealth smartphone app evaluation. Relevant databases and trial registers were queried for articles. Articles were included that (1) described tools, guidelines, dimensions, or criteria to evaluate apps, (2) were available in full text, and (3) were written in English, French, German, Italian, Portuguese, or Spanish. We proposed a conceptual framework for app evaluation based on the dimensions reported in the selected articles. This was revised iteratively in discussion rounds with international stakeholders. The conceptual framework was used to synthesize the reported evaluation criteria. The list of criteria was discussed and refined by the research team.

RESULTS

Screening of 1258 articles yielded 128 (10.17%) that met the inclusion criteria. Of these 128 articles, 30 (23.4%) reported the use of self-developed criteria and described their development processes incompletely. Although 43 evaluation instruments were used only once, 6 were used in multiple studies. Most articles (83/128, 64.8%) did not report following theoretical guidelines; those that did noted 37 theoretical frameworks. On the basis of the selected articles, we proposed a conceptual framework to explore 6 app evaluation dimensions: context, stakeholder involvement, features and requirements, development processes, implementation, and evaluation. After standardizing the definitions, we identified 205 distinct criteria. Through consensus, the research team relabeled 12 of these and added 11 more-mainly related to ethical, legal, and social aspects-resulting in 216 evaluation criteria. No criteria had to be moved between dimensions.

CONCLUSIONS

This study provides a comprehensive overview of criteria currently used in clinical practice to describe and evaluate apps. This is necessary as no reviewed criteria sets were inclusive, and none included consistent definitions and terminology. Although the resulting overview is impractical for use in clinical practice in its current form, it confirms the need to craft it into a purpose-built, theory-driven tool. Therefore, in a subsequent step, based on our current criteria set, we plan to construct an app evaluation tool with 2 parts: a short section (including 1-3 questions/dimension) to quickly disqualify clearly unsuitable apps and a longer one to investigate more likely candidates in closer detail. We will use a Delphi consensus-building process and develop a user manual to prepare for this undertaking.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42021227064; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021227064.

Exploring the usability of a smartphone application to monitor fatigue and activity for people with acquired brain injury

Background

Fatigue after acquired brain injury (ABI) leads to detrimental changes in ABI survivors' daily activities and participation. There is a need to capture individual's experience of fatigue as it happens, to better support self-management of fatigue.

Study aims

To investigate the usability of a real-time tracker of fatigue and activity (using ecological momentary assessment delivered by a smartphone application) and the feasibility of capturing activity and environmental factors using phone sensors.

Methods

Participants wore an activity monitor and completed up to eight surveys a day on a smartphone app, for 6 days, completed the system usability scale (SUS) and were interviewed on their views of using the app. Interview transcripts were analysed using qualitative content analysis. Agreement between data from the phone's sensors and the activity monitor was analysed using Kappa statistics.

Results

Seven participants completed between 11 and 58 surveys. Mean score on the SUS indicated good perceived usability of the app. Phone sensors did not reliably capture physical activity or background noise. Participants found the app easy to use and perceived self-monitoring to help their understanding of fatigue. A fatigue-tracking app may be acceptable to ABI survivors and has potential to aid self-management of fatigue.

Early mobile app training proficiency predicts how well memory-impaired individuals learn to use digital memory aids in the real world

Functional memory impairment following acquired brain injury can lead to decreased independence. External memory aids such as smartphones can be highly effective compensation tools, but cognitive deficits may create barriers to implementation in daily life. The present study examined predictors of real-world use of mobile calendar applications for memory compensation in an acquired brain injury sample. A retrospective chart review was completed from an outpatient rehabilitation program, extending 15 years into the past, yielding data from 34 eligible participants. All participants demonstrated skill learning of the calendar function in their digital device and subsequently completed the generalization phase of training, which is focused on real-world implementation (measured through prospective memory tasks). The results showed that the length of time required for skill learning of mobile calendars (event entry or responding to alerts) was not predictive of the duration of generalization training. Initial training performance for responding to alerts, but not event entry, was a significant predictor of the duration of generalization training needed to complete the program. A secondary analysis with a subset of the data revealed that individuals with additional executive deficits took significantly longer to complete generalization training compared to those with a more focal memory impairment.

Development of a metric to evaluate the ergonomic principles of assistive systems, based on the DIN 92419

The DIN 92419 defines six principles for assistive systems' ergonomic design. There is, however, a lack of measurement tools to evaluate assistive systems considering these principles. Consequently, this study developed a measurement tool for the quantitative evaluation of the fulfilment of each principle for assistive systems. A systematic literature review was performed to identify dimensions belonging to the principles, identify how previous research evaluated these dimensions, and develop a measurement tool for assistive systems. Findings show that scales commonly used for evaluating assistive systems disregard several aspects highlighted as relevant by research, implying the need for considering the DIN 92419 principles. Based on established scales and theoretical findings, a questionnaire, and a checklist for evaluating assistive systems were developed. The work provides a grounding for measuring relevant aspects of assistive systems. Further development is needed to substantiate the reliability and validity of the proposed questionnaire scales and items. Practitioner Summary: Responding to the gap of a holistic measurement tool to evaluate assistive systems, a systematic literature review was performed considering the DIN 92419 principles. This resulted in a comprehensive summary of relevant aspects of assistive systems that were made numerically measurable, which proposes better criteria to assess assistive systems. Abbreviations: IoT: internet of things; RQ: research question; TAM: technology acceptance model; UTAUT: unified theory of acceptance and use of technology; AaaS: adaptivity as a service; SAR: socially assistive robots; SEEV: salience, effort, expectancy, and value; PRISMA: preferred reporting items for systematic reviews and meta-analyses; HMI: human-machine interaction; HRI: human-robot interaction; BCI: brain-computer interface; QUEST: Quebec user evaluation of satisfaction with assistive technology; SUS: system usability scale; NASA-TLX: NASA task load index; ATD PA: assistive technology device predisposition assessment; Wheel Con: wheelchair use confidence scale; CATOM: caregiver assistive technology outcome measure; CBI: caregiver burden inventory; RoSAS: robotic social attributes scale; WheelCon: wheelchair use confidence scale; IMI: intrinsic motivation inventory; ATD PA: assistive technology device predisposition assessment; UEQ: User experience questionnaire; USEUQ: usefulness satisfaction and ease of use questionnaire; USPW: usability scale for power wheelchairs; UES: user engagement scale; SUTAQ: service user technology acceptability questionnaire; QUEAD: questionnaire for the evaluation of physical assistive devices; FATCAT: functional assessment tool for cognitive assistive technology; SE-HRI: human-robot interaction scale; SART: situation awareness rating technique; TSQ;WT: tele-healthcare satisfaction questionnaire-wearable technology; PAIF: participants' assessment of the intervention's feasibility; SWAT: subjective workload assessment technique; MARS-HA: measure of audiologic rehabilitation self-efficacy for hearing aids; IOI-HA: International outcome inventory for hearing aids; FMA: functional mobility assessment; FBIS: familiarity and behavioural intention survey; CSQ: client satisfaction questionnaire; COPM: canadian occupational performance measure; ATCS: assistive technology confidence scale; ACC: acceptance; SSP: safety, security and privacy; OPT: optimisation of resultant internal load; CTRL: controllability; ADAPT: adaptability; P&I: perceptibility and identifiability; AAL: ambient assisted living; VR: virtual reality; AS: assistive system; WEIRD: Western, educated, industrialised, rich, and democratic; HEART: horizontal european activities of rehabilitation technology; AAATE: advancement of assistive technology in Europe's; GATE: global collaboration on assistive technology; ATA-C: assistive technology assessment toolkit.

Managing challenging behaviours in adults with traumatic brain injury: A scoping review of technology-based interventions

Challenging behaviours are one of the most serious sequelae after a traumatic brain injury (TBI). These chronic behaviours must be managed to reduce the associated burden for caregivers, and people with TBI. Though technology-based interventions have shown potential for managing challenging behaviours, no review has synthesised evidence of technology aided behaviour management in the TBI population. The objective of this scoping review was to explore what technology-based interventions are being used to manage challenging behaviours in people with TBI. Two independent reviewers analysed 3505 studies conducted between 2000 and 2023. Studies were selected from five databases using search strategies developed in collaboration with a university librarian. Sixteen studies were selected. Most studies used biofeedback and mobile applications, primarily targeting emotional dysregulation. These technologies were tested in a variety of settings. Two interventions involved both people with TBI and their family caregivers. This review found that technology-based interventions have the potential to support behavioural management, though research and technology development is at an early stage. Future research is needed to further develop technology-based interventions that target diverse challenging behaviours, and to document their effectiveness and acceptability for use by people with TBI and their families.

Optimizing Internet Use during Adolescence: eHealth Solutions

Internet offers an immense opportunity as a resource for education, training, and health promotion. This narrative review summarizes the opportunities for health promotion among adolescents through use of the Internet and technology (defined as eHealth). The details of technology and use of Internet for adolescent-health related topics such as nutrition, fitness, sexual health, adventure, and violence were searched through PubMed. The review reports few digital solutions to address key challenges during adolescence like promotion of nutrition and sexual reproductive health, prevention of noncommunicable diseases, substance abuse, and mental health issues. eHealth was concluded as a potential solution for preventive and promotional health practices during adolescence. However, concerns of Internet addiction, safety, privacy, mental health disorders, and misinformation need to be addressed and monitored during adolescence.

Brain-In-Hand technology for adults with acquired brain injury: A convergence of mixed methods findings

Introduction

Individuals with acquired brain injury may find it difficult to self-manage and live independently. Brain-in-Hand is a smartphone app designed to support psychological problems and encourage behaviour change, comprised of a structured diary, reminders, agreed solutions, and traffic light monitoring system.

Aim

To evaluate the potential use and effectiveness of Brain-in-Hand for self-management in adults with acquired brain injury.

Methods

A-B mixed-methods case-study design. Individuals with acquired brain injury ( n = 10) received Brain-in-Hand for up to 12 months. Measures of mood, independence, quality of life, cognition, fatigue, goal attainment, participation administered at baseline, 6 and 12 months. Semi-structured interviews conducted with acquired brain injury participants ( n = 9) and healthcare workers ( n = 3) at 6 months.

Results

Significant increase in goal attainment after 6 months use (t(7) = 4.20, p = .004). No significant improvement in other outcomes. Qualitative data suggested improvement in anxiety management. Contextual (personal/environmental) factors were key in influencing the use and effectiveness of Brain-in-Hand. Having sufficient insight, appropriate support and motivation facilitated use.

Conclusions

Brain-in-Hand shows potential to support acquired brain injury, but further work is required to determine its effectiveness. Context played a pivotal role in the effectiveness and sustained use of Brain-in-Hand, and needs to be explored to support implementation.

Factors Affecting the Delivery and Acceptability of the ROWTATE Telehealth Vocational Rehabilitation Intervention for Traumatic Injury Survivors: A Mixed-Methods Study

Background: Returning to work after traumatic injury can be problematic. We developed a vocational telerehabilitation (VR) intervention for trauma survivors, delivered by trained occupational therapists (OTs) and clinical psychologists (CPs), and explored factors affecting delivery and acceptability in a feasibility study. Methods: Surveys pre- (5 OTs, 2 CPs) and post-training (3 OTs, 1 CP); interviews pre- (5 OTs, 2 CPs) and post-intervention (4 trauma survivors, 4 OTs, 2 CPs). Mean survey scores for 14 theoretical domains identified telerehabilitation barriers (score ≤ 3.5) and facilitators (score ≥ 5). Interviews were transcribed and thematically analysed. Results: Surveys: pre-training, the only barrier was therapists’ intentions to use telerehabilitation (mean = 3.40 ± 0.23), post-training, 13/14 domains were facilitators. Interviews: barriers/facilitators included environmental context/resources (e.g., technology, patient engagement, privacy/disruptions, travel and access); beliefs about capabilities (e.g., building rapport, complex assessments, knowledge/confidence, third-party feedback and communication style); optimism (e.g., impossible assessments, novel working methods, perceived importance and patient/therapist reluctance) and social/professional role/identity (e.g., therapeutic methods). Training and experience of intervention delivery addressed some barriers and increased facilitators. The intervention was acceptable to trauma survivors and therapists. Conclusion: Despite training and experience in intervention delivery, some barriers remained. Providing some face-to-face delivery where necessary may address certain barriers, but strategies are required to address other barriers.

A systematic review of factors influencing attitudes towards and intention to use the long-distance caregiving technologies for older adults

PURPOSE

Long-distance caregiving (LDC) technologies play a significant role in enabling distant care and facilitating living-alone older adults to keep socially connected. However, there is scarce research exploring the older adults' attitudes towards and intention to use such technologies. This paper is based on a systematic review of existing literature to explore the multifarious factors influencing independent community-living older adults' attitudes towards and intention to use LDC technologies.

METHODS

Articles published in English between 2006 and 2020 were reviewed by searching electronic databases of PubMed, ProQuest, EBSCOhost. The inclusion criteria were limited to quantitative, qualitative, or mixed-methods studies that involved: 1) distant caregiving; 2) older adults aged 60 years or above, who were living alone or with only their spouse in the community (even though the samples might also involve other non-older adults); 3) technologies including ICT-based devices, systems, or programs enabling data transmission were used; 4), intention to use or behavioral usage in regard to the technologies were reported or discussed.

RESULTS

In total, 41 out of 8674 articles were included. Both determinants and moderators of affecting the use of the ICT-based LDC technologies were identified with theoretical guidance. To summarize, there are personal factors involved, such as personality, concerns regarding security and privacy, health conditions, requisite knowledge, financial conditions, and influence from significant others, encompassing formal and informal caregivers; and factors related to the devices, in terms of their user-friendliness and functionality.

CONCLUSION

This review highlights the importance of striking a good balance between functionality and privacy concerns, besides considering the direct and indirect cost to users. LDC technology education should be promoted at the societal level to facilitate older adults' better understanding of the device utilities by enhancing their technological literacy. Implications for various stakeholders to cope with the challenges of an aging population are also discussed.

Use of mHealth Technology for Patient-Reported Outcomes in Community-Dwelling Adults with Acquired Brain Injuries: A Scoping Review

The purpose of our scoping review was to describe the current use of mHealth technology for long-term assessment of patient-reported outcomes in community-dwelling individuals with acquired brain injury (ABI). Following PRISMA guidelines, we conducted a scoping review of literature meeting these criteria: (1) civilians or military veterans, all ages; (2) self-reported or caregiver-reported outcomes assessed via mobile device in the community (not exclusively clinic/hospital); (3) published in English; (4) published in 2015-2019. We searched Ovid MEDLINE(R) < 1946 to 16 August 2019, MEDLINE InProcess, EPub, Embase, and PsycINFO databases for articles. Thirteen manuscripts representing 12 distinct studies were organized by type of ABI [traumatic brain injury (TBI) and stroke] to extract outcomes, mHealth technology used, design, and inclusion of ecological momentary assessment (EMA). Outcomes included post-concussive, depressive, and affective symptoms, fatigue, daily activities, stroke risk factors, and cognitive exertion. Overall, collecting patient-reported outcomes via mHealth was feasible and acceptable in the chronic ABI population. Studies consistently showed advantage for using EMA despite variability in EMA timing/schedules. To ensure best clinical measurement, research on post-ABI outcomes should consider EMA designs (versus single time-point assessments) that provide the best timing schedules for their respective aims and outcomes and that leverage mHealth for data collection.

Feasibility of an Intervention for Patients with Cognitive Impairment Using an Interactive Digital Calendar with Mobile Phone Reminders (RemindMe) to Improve the Performance of Activities in Everyday Life

The aim of this study is to increase evidence-based interventions by investigating the feasibility of an intervention using an interactive digital calendar with mobile phone reminders (RemindMe) as support in everyday life. Qualitative and quantitative data were collected from participating patients (n = 8) and occupational therapists (n = 7) from three rehabilitation clinics in Sweden. The intervention consisted of delivering the interactive digital calendar RemindMe, receiving an individualized introduction, a written manual, and individual weekly conversations for two months with follow-up assessments after two and four months. Feasibility areas of acceptability, demand, implementation, practicality, and integration were examined. Patients expressed their interest and intention to use RemindMe and reported a need for reminders and individualized support. By using reminders in activities in everyday life their autonomy was supported. The study also demonstrated the importance of confirming reminders and the possible role of habit-forming. Occupational therapists perceived the intervention to be useful at the rehabilitation clinics and the weekly support conversations enabled successful implementation. This study confirmed the importance of basing and tailoring the intervention to patients' needs and thus being person-centered.

Ecological Momentary Assessment in Patients With an Acquired Brain Injury: A Pilot Study on Compliance and Fluctuations

Ecological Momentary Assessment (EMA) promises to be a suitable method for capturing the dynamics in self-assessments through repeated measurements in naturalistic environments using common mobile devices. Therefore, EMA could increase the power of neuropsychological assessment by obtaining a more fine-grained picture of symptoms, limitations, and strengths in patients with an acquired brain injury (ABI) in real-life situations. The present study examined 15 patients with an ABI with cognitive and motor impairments. Following a semirandomized high-frequency sampling plan to assess EMA's feasibility and applicability, data were collected across 7 days. At eight prompts per day, patients were asked about their current activities, the social context they were in, their current mood, performance judgments of their own functional status, and the frequency of self-reflections. The average compliance rate was 71.6%. The fluctuations in patients' responses were measured in terms of variance distributions within simple (intercept only) three-level models and root mean square of successive difference values. They were sufficient, as shown, for example, by the mean within-person variability of 44.9% across all of the items studied. There were no significant correlations between patients' age, severity of depressive symptoms, or their level of functioning and their compliance with study participation or the variability of their responses. The results support the feasibility and applicability of EMA as an assessment technique in patients with an ABI. There are, however, limitations that should be considered when planning an assessment of brain-injured patients using EMA.