Remote Cognitive Screening Of Healthy Older Adults for Primary Care With the MyCog Mobile App: Iterative Design and Usability Evaluation

Using Journey Mapping and Service Blueprinting to Design Digital Health Behavior Change Innovations: A Scoping Review

INTRODUCTION

Solutions to support disease self-management and health-related behavior changes require a deep understanding of patient experiences, needs, and challenges across the care journey. Journey mapping (JM) and service blueprinting (SB) are valuable tools for visualizing user experiences and system processes over time. This scoping review explores how JM/SBs have been applied to design digitally enabled interventions targeting health-related behaviors among patients and the public.

METHODS

The JBI reviewer manual was used to guide the review. Studies were sourced from Embase, Psych Info, PubMed, Medline, Web of Science, and Scopus. Inclusion criteria required studies to describe how JM/SBs informed the design of a digitally enabled health innovation that aimed to impact health or health care-related behaviors of patients or the public. A two-level screening process and iterative data extraction were applied.

RESULTS

A total of 28 studies met the inclusion criteria, with a majority published between 2019 and 2024. The JM/SBs rarely used behavioral science theory and were structured, organized, and presented in diverse ways. Most studies designed their digital health behavior change innovations by using JM/SB to identify relevant innovation touchpoints across the patient journey. Patients frequently participated in the digital health behavior change innovation design process, with JM/SBs often serving as sensemaking tools. Innovations tended to address multifaceted health service problems through multimodal, digitally enabled solutions.

CONCLUSIONS

JM/SBs are emerging as versatile tools to help digital health innovations to conduct user research, engage diverse partners, identify complex problems, and ideate creative solutions. However, limited integration of behavioral science theory indicates an area for future exploration.

Examining Challenges to Co-Design Digital Health Interventions With End Users: Systematic Review

BACKGROUND

Digital health interventions (DHIs) are changing the dynamic of health care by providing personalized, private, and instantaneous solutions to end users. However, the explosion of digital health has been fraught with challenges. The approach to co-design with end users varies across a diverse domain of stakeholders, often resulting in siloed approaches with no clear consensus. The concept of validating user experiences contrasts greatly between digital stakeholders (ie, user experience and retention) and health stakeholders (ie, safety and efficacy). Several methodologies and frameworks are being implemented to address this challenge to varying degrees of success.

OBJECTIVE

We aimed to broadly examine the advancements and challenges to co-design DHIs with end users over the last decade. This task was undertaken to identify the key problem areas at the domain level, with the ultimate goal of creating recommendations for better approaches to co-design DHIs with end users.

METHODS

We conducted a systematic search of key databases for co-design studies involving end users in DHIs. Searches were divided into 3 relevant streams: health behavior, user experience, and digital methodologies and frameworks. The eligibility criteria were guided by the PerSPEcTiF framework and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI design. The search was conducted in a diverse group of 6 bibliographical databases from January 2015 to May 2024: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. From the 13,961 studies initially screened for titles and abstracts, 489 (3.6%) were eligible for a full-text screening, of which 171 (1.2%) studies matched the inclusion criteria and were included in a qualitative synthesis.

RESULTS

Of the 171 studies analyzed across 52 journals, we found 5 different research approaches, spanning 8 different digital health solution types and 5 different design methodologies. These studies identified several core themes when co-designing with end users: advancements, which included participatory co-design; challenges, which included participatory co-design, environment and context, testing, and cost and scale; and gaps, which included a pragmatic hybridized framework and industry implementability.

CONCLUSIONS

This research supports a pragmatic shift toward using mixed methods approaches at scale, methods that are primed to take advantage of the emerging big data era of digital health co-design. This organic outlook should blend the vision of digital health co-designers with the pragmatism of Agile design methodology and the rigor of health care metrics.

TRIAL REGISTRATION

PROSPERO CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021238164.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/28083.

Usability of the Mayo Test Drive remote self-administered web-based cognitive screening battery in adults aged 35-100 with and without cognitive impairment

BACKGROUND

Mayo Test Drive (MTD): Mayo Test Development through Rapid Iteration, Validation and Expansion, is a web-based remote cognitive assessment platform for self-administered neuropsychological measures with previously demonstrated validity and reliability. We examined the usability of MTD and hypothesized that completion rates would be greater than 90%. We explored whether completion and participation rates differed by cognitive status and demographic factors.

METHODS

A total of 1,950 Mayo Clinic Study of Aging and Mayo Alzheimer's Disease Research Center participants (97% White, 99% Non-Hispanic) were invited to participate in this ancillary, uncompensated remote study. Most invitees were cognitively unimpaired (CU; n = 1,769; 90.7%) and 9.3% were cognitively impaired (CI; n = 181). Usability was objectively defined as the percentage of participants who completed a session after initiating a session for a given timepoint (i.e. completion rates).

RESULTS

Baseline session completion rates were 98.5% (n = 1199/1217 participants, mean age 71, SD = 12, range 35-100) and were comparable between CU (98.7%) and CI (95.0%) groups (p = .23). Completion rates did not significantly differ by age groups (p > .10) and remained high in individuals 80+ (n = 251, 97.3%). Participation rates were higher in the CU (n = 1142, 65.4%) versus CI (n = 57, 33.1%) group (p < .001); participants were younger and had more years of education (p's < .001). Adherence (i.e. retention) rate for a 7.5-month follow-up session was 89%. Average session duration was 16 min. Most participants used a personal computer (62.7%), followed by a smartphone (22.2%) or tablet (14.8%). Comments entered by 36.4% of participants reflected several themes including acceptability, face validity, usability, and comments informative for session context.

CONCLUSIONS

MTD demonstrated high usability as defined by completion rates in this research sample that includes a broad age range, though participation rates are lower in individuals with cognitive impairment. Results support good adherence at follow-up, feasibility through mean session duration, and acceptability based on qualitative analysis of participant comments.

Protocol for a construct and clinical validation study of MyCog Mobile: a remote smartphone-based cognitive screener for older adults

INTRODUCTION

Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet primary care settings face time constraints that present barriers to routine screening. A remote cognitive screener completed on a patient's personal smartphone before a visit has the potential to save primary care clinics time, encourage broader screening practices and increase early detection of cognitive decline. MyCog Mobile is a promising new remote smartphone-based cognitive screening app for primary care settings. We propose a combined construct and clinical validation study of MyCog Mobile.

METHODS AND ANALYSIS

We will recruit a total sample of 300 adult participants aged 65 years and older. A subsample of 200 healthy adult participants and a subsample of 100 adults with a cognitive impairment diagnosis (ie, dementia, mild cognitive impairment, cognitive deficits or other memory loss) will be recruited from the general population and specialty memory care centres, respectively. To evaluate the construct validity of MyCog Mobile, the healthy control sample will self-administer MyCog Mobile on study-provided smartphones and be administered a battery of gold-standard neuropsychological assessments. We will compare correlations between performance on MyCog Mobile and measures of similar and dissimilar constructs to evaluate convergent and discriminant validity. To assess clinical validity, participants in the clinical sample will self-administer MyCog Mobile on a smartphone and be administered a Mini-Cog screener and these data will be combined with the healthy control sample. We will then apply several supervised model types to determine the best predictors of cognitive impairment within the sample. Area under the receiver operating characteristic curve, accuracy, sensitivity and specificity will be the primary performance metrics for clinical validity.

ETHICS AND DISSEMINATION

The Institutional Review Board at Northwestern University (STU00214921) approved this study protocol. Results will be published in peer-reviewed journals and summaries provided to the study's funders.

Remote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App

BACKGROUND

Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians' ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics' time, encourage broader screening practices, and increase early detection of cognitive decline.

OBJECTIVE

The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile?

METHODS

A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items.

RESULTS

All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants' rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants' abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items.

CONCLUSIONS

Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.

An Initial Exploratory Examination of the Sensitivity and Specificity of MyCog Mobile Using the Mini-Cog as a Proxy for Suspected Cognitive Impairment

OBJECTIVES

To help promote early detection of cognitive impairment in primary care, MyCog Mobile was designed as a cognitive screener that can be self-administered remotely on a personal smartphone. We explore the potential utility of MyCog Mobile in primary care by comparing MyCog Mobile to a commonly used screener, Mini-Cog.

METHODS

A sample of 200 older adults 65+ years (mean age = 72.56 years), completed the Mini-Cog and MyCog Mobile, which includes 2 memory measures and 2 executive functioning measures. A logistic regression model was conducted to predict failing Mini-Cog scores (≤2) based on MyCog Mobile measures.

RESULTS

A total of 20 participants earned a Mini-Cog score ≤2. MyCog Mobile demonstrated an AUC of 0.83 (95% bootstrap CI [0.75, 0.95]), sensitivity of 0.76 (95% bootstrap CI [0.63, 0.97]), and specificity of .88 (95% bootstrap CI [0.63, 0.10]). The subtest Name Matching from MyFaces and MySorting were the only significant predictors of failed Mini-Cogs.

CONCLUSIONS

MyCog Mobile demonstrated sensitivity and specificity to identify participants who failed the Mini-Cog, and may show promise as a screening tool for cognitive impairment in older adults. Further research is necessary to establish the clinical utility of MyCog Mobile in a larger sample using documented clinical diagnoses.