Development and validation of an episodic memory measure in the Mobile Toolbox (MTB): Arranging Pictures

INTRODUCTION

Arranging Pictures is a new episodic memory test based on the NIH Toolbox (NIHTB) Picture Sequence Memory measure and optimized for self-administration on a personal smartphone within the Mobile Toolbox (MTB). We describe evidence from three distinct validation studies.

METHOD

In Study 1, 92 participants self-administered Arranging Pictures on study-provided smartphones in the lab and were administered external measures of similar and dissimilar constructs by trained examiners to assess validity under controlled circumstances. In Study 2, 1,021 participants completed the external measures in the lab and self-administered Arranging Pictures remotely on their personal smartphones to assess validity in real-world contexts. In Study 3, 141 participants self-administered Arranging Pictures remotely twice with a two-week delay on personal iOS smartphones to assess test-retest reliability and practice effects.

RESULTS

Internal consistency was good across samples (ρxx = .80 to .85, p < .001). Test-retest reliability was marginal (ICC = .49, p < .001) and there were significant practice effects after a two-week delay (ΔM = 3.21 (95% CI [2.56, 3.88]). As expected, correlations with convergent measures were significant and moderate to large in magnitude (ρ = .44 to .76, p < .001), while correlations with discriminant measures were small (ρ = .23 to .27, p < .05) or nonsignificant. Scores demonstrated significant negative correlations with age (ρ = -.32 to -.21, p < .001). Mean performance was slightly higher in the iOS compared to the Android group (MiOS = 18.80, NiOS = 635; MAndroid = 17.11, NAndroid = 386; t(757.73) = 4.17, p < .001), but device type did not significantly influence the psychometric properties of the measure. Indicators of potential cheating were mixed; average scores were significantly higher in the remote samples (F(2, 850) = 11.415, p < .001), but there were not significantly more perfect scores.

CONCLUSION

The MTB Arranging Pictures measure demonstrated evidence of reliability and validity when self-administered on personal device. Future research should examine the potential for cheating in remote settings and the properties of the measure in clinical samples.

Development and Validation of a Vocabulary Measure in the Mobile Toolbox

OBJECTIVE

We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies.

METHOD

Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones.

RESULTS

The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001).

CONCLUSION

The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Remote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App

BACKGROUND

Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians' ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics' time, encourage broader screening practices, and increase early detection of cognitive decline.

OBJECTIVE

The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile?

METHODS

A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items.

RESULTS

All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants' rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants' abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items.

CONCLUSIONS

Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.

The Paradox in Positive and Negative Aspects of Emotional Functioning Among Older Adults with Early Stages of Cognitive Impairment

Introduction: Emotional functioning in older adults is influenced by normal aging and cognitive impairment, likely heterogeneous across positive versus negative aspects of emotional functioning. Little is known about positive emotional experiences at the early stages of cognitive impairment. Methods: We assessed different aspects of emotional functioning among 448 participants aged 65+ (Normal Control (NC) = 276, Mild Cognitive Impairment (MCI) = 103, and mild dementia of the Alzheimer type (mild DAT) = 69) and tested moderators. Results: Compared to NC, older adults with MCI and mild DAT have maintained many positive aspects of emotional functioning, despite higher levels of negative affect, sadness, and loneliness. Among the oldest-old, the mild DAT group experienced higher fear and lower self-efficacy. Discussion: Older adults at early stages of cognitive impairment can experience positive aspects of emotional functioning, such as positive affect, purpose, and life satisfaction, all of which are important buildable psychological resources for coping.

Remote Cognitive Screening Of Healthy Older Adults for Primary Care With the MyCog Mobile App: Iterative Design and Usability Evaluation

BACKGROUND

Annual cognitive screening in adults aged >65 years can improve early detection of cognitive impairment, yet less than half of all cases are identified in primary care. Time constraints in primary care settings present a major barrier to routine screening. A remote cognitive screener completed on a patient's own smartphone before a visit has the potential to save primary care clinics time, encourage broader screening practices, and increase early detection of cognitive decline.

OBJECTIVE

We described the iterative design and proposed the implementation of a remote cognitive screening app, MyCog Mobile, to be completed on a patient's smartphone before an annual wellness visit. The research questions were as follows: What would motivate primary care clinicians and clinic administrators to implement a remote cognitive screening process? How might we design a remote cognitive screener to fit well with existing primary care workflows? What would motivate an older adult patient to complete a cognitive screener on a smartphone before a primary care visit? How might we optimize the user experience of completing a remote cognitive screener on a smartphone for older adults?

METHODS

To address research questions 1 and 2, we conducted individual interviews with clinicians (n=5) and clinic administrators (n=3). We also collaborated with clinic administrators to create user journey maps of their existing and proposed MyCog Mobile workflows. To address research questions 3 and 4, we conducted individual semistructured interviews with cognitively healthy older adults (n=5) and solicited feedback from a community stakeholder panel (n=11). We also tested and refined high-fidelity prototypes of the MyCog Mobile app with the older adult interview participants, who rated the usability on the Simplified System Usability Scale and After-Scenario Questionnaire.

RESULTS

Clinicians and clinic administrators were motivated to adopt a remote cognitive screening process if it saved time in their workflows. Findings from interviews and user journey mapping informed the proposed implementation and core functionality of MyCog Mobile. Older adult participants were motivated to complete cognitive screeners to ensure that they were cognitively healthy and saw additional benefits to remote screening, such as saving time during their visit and privacy. Older adults also identified potential challenges to remote smartphone screening, which informed the user experience design of the MyCog Mobile app. The average rating across prototype versions was 91 (SD 5.18) on the Simplified System Usability Scale and 6.13 (SD 8.40) on the After-Scenario Questionnaire, indicating above-average usability.

CONCLUSIONS

Through an iterative, human-centered design process, we developed a viable remote cognitive screening app and proposed an implementation strategy for primary care settings that was optimized for multiple stakeholders. The next steps include validating the cognitive screener in clinical and healthy populations and piloting the finalized app in a community primary care clinic.

Uses of the NIH Toolbox® in Clinical Samples: A Scoping Review

Background and Objectives

The NIH Toolbox® for the Assessment of Neurologic and Behavioral Function is a compilation of computerized measures designed to assess sensory, motor, emotional, and cognitive functioning of individuals across the life span. The NIH Toolbox was initially developed for use with the general population and was not originally validated in clinical populations. The objective of this scoping review was to assess the extent to which the NIH Toolbox has been used with clinical populations.

Methods

Guided by the Joanna Briggs Methods Manual for Scoping Reviews, records were identified through searches of PubMed MEDLINE, PsycINFO, ClinicalTrials.gov, EMBASE, and ProQuest Dissertations and Theses Global (2008-2020). Database searches yielded 5,693 unique titles of original research that used at least one NIH Toolbox assessment in a sample characterized by any clinical diagnosis. Two reviewers screened titles, abstracts, and full texts for inclusion in duplicate. Conflicts at each stage of the review process were resolved by a group discussion.

Results

Ultimately, 281 publication records were included in this scoping review (nJournal Articles = 104, nConference Abstracts = 84, nClinical Trial Registrations = 86, and nTheses/Dissertations = 7). The NIH Toolbox Cognition Battery was by far the most used of the 4 batteries in the measurement system (nCognition = 225, nEmotion = 49, nMotor = 29, and nSensation = 16). The most represented clinical category was neurologic disorders (n = 111), followed by psychological disorders (n = 39) and cancer (n = 31). Most (96.8%) of the journal articles and conference abstracts reporting the use of NIH Toolbox measures with clinical samples were published in 2015 or later. As of May 2021, these records had been cited a total of nearly 1,000 times.

Discussion

The NIH Toolbox measures have been widely used among individuals with various clinical conditions across the life span. Our results lay the groundwork to support the feasibility and utility of administering the NIH Toolbox measures in research conducted with clinical populations and further suggest that these measures may be of value for implementation in fast-paced clinical settings as part of routine practice.

Lessons from Detecting Cognitive Impairment Including Dementia (DetectCID) in Primary Care

Background: Cognitive impairment, including dementia, is frequently under-detected in primary care. The Consortium for Detecting Cognitive Impairment, including Dementia (DetectCID) convenes three multidisciplinary teams that are testing novel paradigms to improve the frequency and quality of patient evaluations for detecting cognitive impairment in primary care and appropriate follow-up. Objective: Our objective was to characterize the three paradigms, including similarities and differences, and to identify common key lessons from implementation. Methods: A qualitative evaluation study with dementia specialists who were implementing the detection paradigms. Data was analyzed using content analysis. Results: We identified core components of each paradigm. Key lessons emphasized the importance of engaging primary care teams, enabling primary care providers to diagnose cognitive disorders and provide ongoing care support, integrating with the electronic health record, and ensuring that paradigms address the needs of diverse populations. Conclusion: Approaches are needed that address the arc of care from identifying a concern to post-diagnostic management, are efficient and adaptable to primary care workflows, and address a diverse aging population. Our work highlights approaches to partnering with primary care that could be useful across specialties and paves the way for developing future paradigms that improve differential diagnosis of symptomatic cognitive impairment, identifying not only its presence but also its specific syndrome or etiology.

Designing a Visit Schedule for Longitudinal Studies in Pediatric Research

A challenge for longitudinal studies is combining individual assessments into visits that are scientifically logical, not burdensome for participants, well-choreographed, and operationally feasible. The visits then need to be sequenced and spaced to address the scientific goals and generate a data archive that is sufficiently robust and well-documented to support subsequent analyses. This paper summarizes comprehensive multi-disciplinary activities that were coordinated to design the content, format, and structure of the National Children's Study and concurrently serve as a model and resource for other studies.

Challenges in Participant Engagement and Retention using Mobile Health Apps: A Literature Review (Preprint)

Background

Mobile health (mHealth) apps are revolutionizing the way clinicians and researchers monitor and manage the health of their participants. However, many studies using mHealth apps are hampered by substantial participant dropout or attrition, which may impact the representativeness of the sample and the effectiveness of the study. Therefore, it is imperative for researchers to understand what makes participants stay with mHealth apps or studies using mHealth apps.

Objective

This study aimed to review the current peer-reviewed research literature to identify the notable factors and strategies used in adult participant engagement and retention.

Methods

We conducted a systematic search of PubMed, MEDLINE, and PsycINFO databases for mHealth studies that evaluated and assessed issues or strategies to improve the engagement and retention of adults from 2015 to 2020. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Notable themes were identified and narratively compared among different studies. A binomial regression model was generated to examine the factors affecting retention.

Results

Of the 389 identified studies, 62 (15.9%) were included in this review. Overall, most studies were partially successful in maintaining participant engagement. Factors related to particular elements of the app (eg, feedback, appropriate reminders, and in-app support from peers or coaches) and research strategies (eg, compensation and niche samples) that promote retention were identified. Factors that obstructed retention were also identified (eg, lack of support features, technical difficulties, and usefulness of the app). The regression model results showed that a participant is more likely to drop out than to be retained.

Conclusions

Retaining participants is an omnipresent challenge in mHealth studies. The insights from this review can help inform future studies about the factors and strategies to improve participant retention.

Recommendations for Assessment of Social, Emotional, and Behavioral Health for the National Children's Study

The Social Emotional Behavioral (SEB) Team of the National Children's Study (NCS) was tasked with making recommendations for assessment of important aspects of social-emotional health and function in children. This paper describes the constructs recommended for assessment along with the rationale for their assessment. These constructs, representing aspects of Social Relationships, Social Capital, Temperament, Negative Affect, Externalizing Behavior, Social Competence, Self-efficacy, Self-image, Psychological well-being, Ethnic/racial Socialization, Perceived Discrimination, Sexual Orientation, Religiosity, and Perceived Stress and Resilience were identified as being critical to the understanding of children's health and development from birth to age 21.

PROMIS® Pediatric Depressive Symptoms as a Harmonized Score Metric

OBJECTIVE

To conduct an evidence-based review of adolescent self-report depression measures and to demonstrate how various measures can be rescored onto a harmonized metric.

METHOD

Six widely used person-reported outcome measures (PROMs) were reviewed. Psychometric properties were evaluated using previously published guidance for PROMs. Next, two secondary data sources (from an outpatient behavioral health clinic and from the general population) were evaluated to harmonize scores across three of the measures. Both item response theory and equipercentile linking methods were used and compared.

RESULTS

All six PROMs demonstrated a high evidence base for widespread use depending on the purpose of the assessments. Adolescent involvement when developing the PROM for content validity and floor or ceiling effects were the least frequent available evidence. Three of the PROMs were linked to the PROMIS® Pediatric Depressive Symptoms v2.0 (PROMIS-PedDepSx) metric. The scales were highly correlated and essentially unidimensional when aggregated. All linking methods were broadly comparable. Group-level score conversions are recommended to minimize linking bias.

CONCLUSIONS

There are a number of strong, widely used PROMs for the evidence-based assessment (EBD) of adolescent depression. However, score comparability is a concern whenever there is a proliferation of measures. Harmonized score metrics support data aggregation and re-analysis. Using four PROMs, one of which served as the scoring metric, we demonstrated the possibility of harmonized depression scores. Future directions for EBD should evaluate whether harmonized PROMs for other pediatric health domains would be useful.

Characteristics and impacts of itch in children with inflammatory skin disorders

BACKGROUND

Itch is a cardinal feature of paediatric disorders and can impair quality of life. However, few studies have addressed symptoms and impacts of itch in paediatric patients.

OBJECTIVES

We focused on understanding the child's experience of itch and the impact of itch specifically on affected children, including comparison with the adult experience.

METHODS

Semistructured interviews (nine parents, 15 children with itch) explored concerns related to paediatric itch experiences and effects. Themes were compared with those of previous adult interviews. Literature was reviewed to identify the need for a more comprehensive measure of paediatric itch.

RESULTS

Itch quality, intensity, duration and environmental triggers (sweating, climate change, stress and certain fabrics) are important aspects of the child's itch experience. Skin disruption, physical function, concentration, emotional reactions, stigma and relationships/social effects are itch impact themes that emerged. No paediatric-specific scale comprehensively captures the paediatric patient itch experience. However, differences between child and adult reports of itch-related pain, functional limitations, fatigue and restlessness, emotional reactions to itch, and treatment effects emphasize the need for a paediatric-specific measurement tool.

CONCLUSIONS

Children and parents endorse the importance of capturing the paediatric-focused characteristics and impacts of itch in measuring disease severity and response to intervention.

The NIH Toolbox: Overview of Development for Use with Hispanic Populations

OBJECTIVE

Hispanics/Latinos are the largest and fastest-growing minority population in the United States. To facilitate appropriate outcome assessment of this expanding population, the NIH Toolbox for Assessment of Neurological and Behavioral Function® (NIH Toolbox®) was developed with particular attention paid to the cultural and linguistic needs of English- and Spanish-speaking Hispanics/Latinos.

METHODS

A Cultural Working Group ensured that all included measures were appropriate for use with Hispanics/Latinos in both English and Spanish. In addition, a Spanish Language Working Group assessed all English-language NIH Toolbox measures for translatability.

RESULTS

Measures were translated following the Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology for instances where language interpretation could impact scores, or a modified version thereof for more simplified translations. The Spanish versions of the NIH Toolbox Cognition Battery language measures (i.e., Picture Vocabulary Test, Oral Reading Recognition Test) were developed independently of their English counterparts.

CONCLUSIONS

The Spanish-language version of the NIH Toolbox provides a much-needed set of tools that can be selected as appropriate to complement existing protocols being conducted with the growing Hispanic/Latino population in the United States.

NIH Toolbox® Emotion Batteries for Children: Factor-Based Composites and Norms

The recently released National Institutes of Health Toolbox (NIHTB) batteries for neurological and behavioral function were designed to serve as standardized, common measures in clinical and epidemiological research. The current study aimed to examine constructs assessed by the self-report and parental proxy-report scales in the NIHTB Emotion Battery (NIHTB-EB) for Children by using factor analyses on data from the U.S. national normative sample of 2,916 English-speaking children. This battery contains 31 scales designed to assess both positive and negative aspects of social and emotional functioning that are considered developmentally relevant at each of three age ranges (3-7, 8-12, and 13-17 years). Results revealed four similar self-report factors for ages 8 to 12 years and 13 to 17 years. Proxy reports for ages 3 to 7 years revealed three factors, and for ages 8 to 12 years two factors. Based on the standardization sample data, age- and gender-corrected norms are presented for all NIHTB-EB individual scales and factor-based composites.

National Institutes of Health Toolbox Emotion Battery for English- and Spanish-speaking adults: normative data and factor-based summary scores

Background

The National Institutes of Health Toolbox Emotion Battery (NIHTB-EB) is a "common currency", computerized assessment developed to measure the full spectrum of emotional health. Though comprehensive, the NIHTB-EB's 17 scales may be unwieldy for users aiming to capture more global indices of emotional functioning.

Methods

NIHTB-EB was administered to 1,036 English-speaking and 408 Spanish-speaking adults as a part of the NIH Toolbox norming project. We examined the factor structure of the NIHTB-EB in English- and Spanish-speaking adults and developed factor analysis-based summary scores. Census-weighted norms were presented for English speakers, and sample-weighted norms were presented for Spanish speakers.

Results

Exploratory factor analysis for both English- and Spanish-speaking cohorts resulted in the same 3-factor solution: 1) negative affect, 2) social satisfaction, and 3) psychological well-being. Confirmatory factor analysis supported similar factor structures for English- and Spanish-speaking cohorts. Model fit indices fell within the acceptable/good range, and our final solution was optimal compared to other solutions.

Conclusion

Summary scores based upon the normative samples appear to be psychometrically supported and should be applied to clinical samples to further validate the factor structures and investigate rates of problematic emotions in medical and psychiatric populations.

Evolution of Patient-Reported Outcomes and Their Role in Multiple Sclerosis Clinical Trials

Patient-reported outcomes (PROs) are playing an increasing role in multiple sclerosis (MS) research and practice, and are essential for understanding the effects that MS and MS treatments have on patients' lives. PROs are captured directly from patients and include symptoms, function, health status, and health-related quality of life. In this article, we review different categories (e.g., generic, targeted, preference-based) of PRO measures and considerations in selecting a measure. The PROs included in MS clinical research have evolved over time, as have the measures used to assess them. We describe findings from recent MS clinical trials that included PROs when evaluating Food and Drug Administration-approved disease-modifying therapies (e.g., daclizumab, teriflunomide). Variation in the measures used in these trials makes it difficult to draw any conclusions from the data. We therefore suggest a standardized approach to PRO assessment in MS research and describe 2 generic, National Institutes of Health-supported measurement systems [Neuro-QoL and the Patient-Reported Outcomes Measurement Information System (PROMIS)] that would facilitate such an approach. The use of PROs in MS care and research is expanding beyond clinical trials, as is demonstrated by examples from comparative effectiveness and other patient-centered research. The importance of PRO assessment is expected to continue to grow in the future.

Computerized Adaptive Testing in Pediatric Brain Tumor Clinics

CONTEXT

Monitoring of health-related quality of life and symptoms of patients with brain tumors is needed yet not always feasible. This is partially due to lack of brief-yet-precise assessments with minimal administration burden that are easily incorporated into clinics. Dynamic computerized adaptive testing (CAT) or static fixed-length short forms, derived from psychometrically sound item banks, are designed to fill this void.

OBJECTIVE

This study evaluated the comparability of scores obtained from CATs and short forms.

METHODS

Patients (ages 7-22 years) were recruited from brain tumor clinics and completed Patient-Reported Outcome Measurement Information System CATs and short forms (Fatigue, Mobility, Upper Extremity, Depressive Symptoms, Anxiety, and Peer Relationships). Pearson correlations, paired t-tests, and Cohen's d were used to evaluate the relationship, significant differences, and the magnitude of the difference between these two scores, respectively.

RESULTS

Data from 161 patients with brain tumors were analyzed. Patients completed each CAT within 2 minutes. Scores obtained from CATs and short forms were highly correlated (r = 0.95-0.98). Significantly different CAT vs. short-form scores were found on 4 (of 6) domains yet with negligible effect sizes (|d| < 0.09). These relationships varied across patients with different levels of reported symptoms, with the strongest association at the worst or best symptom scores.

CONCLUSIONS

This study demonstrated the comparability of scores from CATs and short forms. Yet the agreement between these two varied across degrees of symptom severity which was a result of the ceiling effects of static short forms. We recommend CATs to enable individualized assessment for longitudinal monitoring.

A PROMIS Measure of Neuropathic Pain Quality

OBJECTIVES

Neuropathic pain (NP) is a consequence of many chronic conditions. This study aimed to develop an unidimensional NP scale with scores that represent levels of NP and distinguish between individuals with NP and non-NP conditions.

METHODS

A candidate item pool of 42 pain quality descriptors was administered to participants with osteoarthritis, rheumatoid arthritis, diabetic neuropathy, and cancer chemotherapy-induced peripheral neuropathy. A subset of pain quality descriptors (items) that best distinguished between participants with and those without NP conditions were identified. Dimensionality of pain descriptors was evaluated in a development sample and cross-validated in a holdout sample. Item responses were calibrated using an item response theory model, and scores were generated on a T-score metric. NP scale scores were evaluated in terms of the reliability, validity, and ability to distinguish between participants with and without conditions typically associated with NP.

RESULTS

Of the 42 initial items, 5 were identified for the Patient-Reported Outcome Measurement Information System (PROMIS) Neuropathic Pain Quality Scale. T scores exhibited good discriminatory ability on the basis of receiver-operator characteristic analysis. Score thresholds that optimize sensitivity and specificity were identified. Construct, criterion, and discriminant validity, and reliability of scale scores were supported.

CONCLUSIONS

The five-item Patient-Reported Outcome Measurement Information System (PROMIS PQ-Neuro) Neuropathic Pain Quality Scale is a short and practical measure that can be used to identify patients more likely to have NP and to distinguish levels of NP. The data collected will support future research that targets other unidimensional pain quality domains (e.g., nociceptive pain).

Neuro-QoL health-related quality of life measurement system: Validation in Parkinson's disease

INTRODUCTION

Neuro-QoL is a multidimensional patient-reported outcome measurement system assessing aspects of physical, mental, and social health identified by neurology patients and caregivers as important. One of the first neurology-specific patient-reported outcome measure systems created using modern test development methods, Neuro-Qol enables brief, yet precise, assessment and the ability to conduct both PD-specific and cross-disease comparisons. We present results of Neuro-QoL clinical validation using a sample of PD patients.

METHODS

A total of 120 PD patients recruited from academic medical centers were assessed at baseline, 1 week, and 6 months. Assessments included Neuro-QoL and general and PD-specific validity measures.

RESULTS

Participants were 62% male and 95% white (average age = 66); H & Y stages were 1 (16%), 2 (61%), 3 (18%), and 4 (5%). Internal consistency and test-retest reliability of Neuro-QoL ranged from Cronbach's alphas = 0.81 to 0.94 with intraclass correlation coefficients = 0.66 to 0.80. Pearson's correlations between Neuro-QoL and legacy measures were generally moderate and in expected directions. UPDRS Part 2 was moderately correlated with Neuro-QoL Upper Extremity and Mobility, respectively (r's = -0.44; -0.59). Parkinson's Disease Questionnaire-39 and Neuro-QoL measures of similar constructs showed strong-to-moderate correlations (r's = 0.70-0.44). Neuro-QoL measures of fatigue, mobility, positive emotion, and emotional/behavioral control showed responsiveness to self-reported change.

CONCLUSIONS

Neuro-QoL is valid for use in PD clinical research. Reliability for all but two measures is sufficient for group comparisons, with some evidence supporting responsiveness to change. Neuro-QoL possesses characteristics, such as brevity, flexibility in administration, and suitability, for cross-disease comparisons that may be advantageous to users in a variety of settings. © 2016 Movement Disorder Society.

Validating Neuro-QoL short forms and targeted scales with people who have multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is a chronic, progressive, and disabling disease of the central nervous system with dramatic variations in the combination and severity of symptoms it can produce. The lack of reliable disease-specific health-related quality of life (HRQL) measures for use in clinical trials prompted the development of the Neurology Quality of Life (Neuro-QOL) instrument, which includes 13 scales that assess physical, emotional, cognitive, and social domains, for use in a variety of neurological illnesses.

OBJECTIVE

The objective of this research paper is to conduct an initial assessment of the reliability and validation of the Neuro-QOL short forms (SFs) in MS.

METHODS

We assessed reliability, concurrent validity, known groups validity, and responsiveness between cross-sectional and longitudinal data in 161 recruited MS patients.

RESULTS

Internal consistency was high for all measures (α = 0.81-0.95) and ICCs were within the acceptable range (0.76-0.91); concurrent and known groups validity were highest with the Global HRQL question. Longitudinal assessment was limited by the lack of disease progression in the group.

CONCLUSIONS

The Neuro-QOL SFs demonstrate good internal consistency, test-re-test reliability, and concurrent and known groups validity in this MS population, supporting the validity of Neuro-QOL in adults with MS.

Validation of the Neuro-QoL measurement system in children with epilepsy

OBJECTIVE

Children with epilepsy often face complex psychosocial consequences that are not fully captured by existing patient-reported outcome (PRO) measures. The Neurology Quality of Life Measurement System "Neuro-QoL" was developed to provide a set of common PRO measures that address issues important to people with neurologic disorders. This paper reports Neuro-QoL (anxiety, depression, interaction with peers, fatigue, pain, cognitive function, stigma, and upper and lower extremity functions) validation in children with epilepsy.

METHOD

Patients (aged 10-18years) diagnosed with epilepsy completed Neuro-QoL and legacy measures at time 1 (initial study visit) and 6-month follow-up. Internal consistency reliability was also evaluated. Concurrent validity was assessed by comparing Neuro-QoL measures with more established "legacy" measures of the same concepts. Clinical validity was evaluated by comparing mean Neuro-QoL scores of patients grouped by clinical anchors such as disease severity. Responsiveness of the Neuro-QoL from time 1 (initial study visit) to 6months was evaluated using self-reported change as the primary anchor.

RESULTS

Sixty-one patients (mean age=13.4years; 62.3% male, 75.9% white) participated. Most patients (64.2%) had been seizure-free in the 3months prior to participation, and seizure frequency was otherwise described as follows: 17.8% daily, 13.3% weekly, 35.6% monthly, and 33.3% yearly. All patients were taking antiepileptic drugs. Patients reported better function/less symptoms compared to the reference groups. Internal consistency (alpha) coefficients ranged from 0.76 to 0.87. Patients with different seizure frequencies differed on anxiety (p<.01) and cognitive function (p<.05). Compared to patients on polytherapy, those on monotherapy had better upper extremity scores (p<.05). Compared to those with localized seizures, those experiencing generalized seizures reported worse stigma (p<.05). Depression, anxiety, lower extremity, fatigue, pain, interaction with peers, and stigma also significantly discriminated patients with different levels of quality of life (p≤.05). All Neuro-QoL measures were significantly correlated with other measures assessing similar domains. Stigma was related to self-reported change in several areas of functioning but in sometimes unexpected directions.

SIGNIFICANCE

The Neurology Quality of Life Measurement System is a valid and reliable assessment tool for children with epilepsy and can be used in research and clinical settings.

Depressive symptoms are more strongly related to executive functioning and episodic memory among African American compared with non-Hispanic White older adults

We examined whether the reserve capacity model can be extended to cognitive outcomes among older African Americans. Two hundred and ninety-two non-Hispanic Whites and 37 African Americans over age 54 participated in the normative study for the NIH Toolbox for the Assessment of Neurological and Behavioral Function. Multiple-group path analysis showed that associations between depressive symptoms and cognition differed by race, independent of age, education, reading level, income, health, and recruitment site. Depressive symptoms were associated with slowed processing speed among Whites and worse task-switching, inhibition, and episodic memory among African Americans. African Americans may be more vulnerable to negative effects of depression on cognition than non-Hispanic Whites. Further research is needed to explicate the psychological and neurobiological underpinnings of this greater vulnerability.

Identifying Motor, Emotional-Behavioral, and Cognitive Deficits that Comprise the Triad of HD Symptoms from Patient, Caregiver, and Provider Perspectives

BACKGROUND

The objective of this study was to identify important attributes associated with the triad of symptoms (cognition, emotional-behavioral, and motor) of Huntington's disease (HD) from patient, caregiver, and medical provider perspectives to facilitate development of a new disease-specific, health-related quality of life (HRQOL) instrument.

METHODS

We conducted a targeted literature review of HD and HRQOL instruments, expert surveys, and patient and caregiver phone-based interviews to extract information on the symptoms and issues most relevant to the HD symptom triad (HD triad). The data collected from these sources were used to generate themes and subdomains and to develop an integrated schema that highlights the key dimensions of the triad.

RESULTS

THE SEARCH IDENTIFIED THE FOLLOWING AREAS: emotional functioning/behavioral changes (e.g., positive emotions, sadness/depression); cognitive functioning (e.g., memory/learning, attention/comprehension); physical functioning (e.g., motor functioning, medication); social functioning (e.g., leisure, interpersonal relationships); end-of-life concerns/planning; and gene testing. Fifteen individuals diagnosed with HD and 16 HD caregivers, recruited from several Huntington's Disease Society of America support group networks, completed phone interviews. Nineteen US medical providers who specialize in HD completed the online survey. Twenty-six subdomains of the HD symptom triad (seven cognition, 12 emotional-behavioral, and seven motor) emerged relatively consistently across patient, caregiver, and provider samples. These included movements/chorea, memory impairment, depression, and anxiety.

DISCUSSION

Based on an integrated, mixed-methods approach, important HD triad symptom were identified and organized into a guiding schema. These patient-, caregiver-, and provider-triangulated data served as the basis for development of a HD-specific HRQOL instrument, the HD-PRO-TRIAD™.

HD-PRO-TRIAD™ Validation: A Patient-reported Instrument for the Symptom Triad of Huntington's Disease

Background

Few valid, disease-specific measures of health-related quality of life (HRQOL) capture the spectrum of symptoms associated with Huntington's disease (HD). The HD-PRO-TRIAD™ is a new, HD-specific, patient-reported outcome (PRO) instrument of the HD symptom triad (cognitive decline, emotional/behavioral dyscontrol, and motor dysfunction) designed for clinical research and practice. The objective was to validate the HD-PRO-TRIAD™ through a cross-sectional sample of individuals with HD and caregivers.

Methods

Development of the HD-PRO-TRIAD™ has been described elsewhere. A total of 132 individuals with HD and 40 HD caregivers, comprising 29 dyads, participated in the cross-sectional psychometric validation of this instrument. Participants provided responses to the HD-PRO-TRIAD™ and other HRQOL and disease severity instruments (EuroQOL 5D, Short Form 12, Neuro-QOL Item Banks, PROMIS Global Health, and self-reported Unified Huntington's Disease Rating Scale Total Functional Capacity and Independence Scales). Internal consistency, construct validity, and patient–caregiver proxy consistency were evaluated.

Results

Internal consistency of the three domains and overall HD-PRO-TRIAD™ instrument was supported by Cronbach's alpha values ≥0.94. Construct validity was supported by significant correlations between HD-PRO-TRIAD™ domain scores and other measures of the same domains (e.g., significant positive correlations between HD-PRO-TRIAD™ Anxiety with Neuro-QOL Anxiety), as well as slightly weaker but still strong correlations with other HRQOL instruments (e.g., HD-PRO-TRIAD™ Anxiety and UHDRS Independence; all p<0.01). Consistency between patient self-report and caregiver proxy report was supported by an intra-class correlation coefficient ≥0.92 for all three domains and the overall instrument.

Discussion

These data indicate that HD-PRO-TRIAD™ is a reliable and valid HRQOL instrument that captures the typical triad of HD symptoms.

I. NIH Toolbox Cognition Battery (CB): introduction and pediatric data

This monograph presents the pediatric portion of the National Institutes of Health (NIH) Toolbox Cognition Battery (CB) of the NIH Toolbox for the Assessment of Neurological and Behavioral Function. The NIH Toolbox is an initiative of the Neuroscience Blueprint, a collaborative framework through which 16 NIH Institutes, Centers, and Offices jointly support neuroscience-related research, to accelerate discoveries and reduce the burden of nervous system disorders. The CB is one of four modules that measure cognitive, emotional, sensory, and motor health across the lifespan. The CB is unique in its continuity across childhood, adolescence, early adulthood, and old age, and in order to help create a common currency among disparate studies, it is also available at low cost to researchers for use in large-scale longitudinal and epidemiologic studies. This chapter describes the evolution of the CB; methods for selecting cognitive subdomains and instruments; the rationale for test design; and a validation study in children and adolescents, ages 3-15 years. Subsequent chapters feature detailed discussions of each test measure and its psychometric properties (Chapters 2-6), the factor structure of the test battery (Chapter 7), the effects of age and education on composite test scores (Chapter 8), and a final summary and discussion (Chapter 9). As the chapters in this monograph demonstrate, the CB has excellent psychometric properties, and the validation study provided evidence for the increasing differentiation of cognitive abilities with age.

NIH toolbox for assessment of neurological and behavioral function

At present, there are many studies that collect information on aspects of neurologic and behavioral function (cognition, sensation, movement, emotion), but with little uniformity among the measures used to capture these constructs. Further, available measures are generally expensive, normed on homogenous nondiverse populations, not easily administered, do not cover the lifespan (or have easily linked pediatric and adult counterparts for the purposes of longitudinal comparison), and not based on the current thinking in the neuroscience community. There is also a paucity of measurement tools to gauge normal children in the motor and sensation domain areas, and many of these measures rely heavily on proxy reporting. Investigators have expressed the need for brief assessment tools that could address these issues and be used as a form of "common currency" across diverse study designs and populations. This ability to assess functionality along a common metric and "crosswalk" across measures is essential to the process of being able to pool data, which is often necessary when a large and diverse sample is needed. When individual studies employ unique assessment batteries, comparisons between studies and combining data from multiple studies can be problematic. The contract for the NIH Toolbox for the Assessment of Neurological and Behavioral Function (www.nihtoolbox.org) was initiated by the NIH Blueprint for Neuroscience Research (www.neuroscienceblueprint.nih.gov) to develop a set of state-of-the-art measurement tools to enhance collection of data in large cohort studies and to advance the biomedical research enterprise.

Input on NIH Toolbox inclusion criteria: surveying the end-user community

OBJECTIVE

The NIH Toolbox is intended to be responsive to the needs of investigators evaluating neurologic and behavioral function in diverse settings. Early phases of the project involved gathering information and input from potential end users.

METHODS

Information was collected through literature and instrument database reviews, requests for information, consensus meetings, and expert interviews and integrated into the NIH Toolbox development process in an iterative manner.

RESULTS

Criteria for instrument inclusion, subdomains to be assessed, and preferences regarding instrument cost and length were obtained. Existing measures suitable for inclusion in the NIH Toolbox and areas requiring new measure development were identified.

CONCLUSION

The NIH Toolbox was developed with explicit input from potential end users regarding many of its key features.

Cognition assessment using the NIH Toolbox

Cognition is 1 of 4 domains measured by the NIH Toolbox for the Assessment of Neurological and Behavioral Function (NIH-TB), and complements modules testing motor function, sensation, and emotion. On the basis of expert panels, the cognition subdomains identified as most important for health, success in school and work, and independence in daily functioning were Executive Function, Episodic Memory, Language, Processing Speed, Working Memory, and Attention. Seven measures were designed to tap constructs within these subdomains. The instruments were validated in English, in a sample of 476 participants ranging in age from 3 to 85 years, with representation from both sexes, 3 racial/ethnic categories, and 3 levels of education. This report describes the development of the Cognition Battery and presents results on test-retest reliability, age effects on performance, and convergent and discriminant construct validity. The NIH-TB Cognition Battery is intended to serve as a brief, convenient set of measures to supplement other outcome measures in epidemiologic and longitudinal research and clinical trials. With a computerized format and national standardization, this battery will provide a "common currency" among researchers for comparisons across a wide range of studies and populations.

Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology

OBJECTIVE

To address the need for brief, reliable, valid, and standardized quality of life (QOL) assessment applicable across neurologic conditions.

METHODS

Drawing from larger calibrated item banks, we developed short measures (8-9 items each) of 13 different QOL domains across physical, mental, and social health and evaluated their validity and reliability. Three samples were utilized during short form development: general population (Internet-based, n = 2,113); clinical panel (Internet-based, n = 553); and clinical outpatient (clinic-based, n = 581). All short forms are expressed as T scores with a mean of 50 and SD of 10.

RESULTS

Internal consistency (Cronbach α) of the 13 short forms ranged from 0.85 to 0.97. Correlations between short form and full-length item bank scores ranged from 0.88 to 0.99 (0.82-0.96 after removing common items from banks). Online respondents were asked whether they had any of 19 different chronic health conditions, and whether or not those reported conditions interfered with ability to function normally. All short forms, across physical, mental, and social health, were able to separate people who reported no health condition from those who reported 1-2 or 3 or more. In addition, scores on all 13 domains were worse for people who acknowledged being limited by the health conditions they reported, compared to those who reported conditions but were not limited by them.

CONCLUSION

These 13 brief measures of self-reported QOL are reliable and show preliminary evidence of concurrent validity inasmuch as they differentiate people based upon number of reported health conditions and whether those reported conditions impede normal function.

A qualitative study of quality of life after stroke: the importance of social relationships

BACKGROUND

Stroke is a leading cause of long-term disability in the USA; however, we have an incomplete understanding of how stroke affects long-term quality of life.

METHODS

We report here findings from focus groups with 9 long-term stroke survivors and 6 caregivers addressing patients' post-stroke quality of life.

RESULTS

Key themes identified by patients were: social support, coping mechanisms, communication, physical functioning and independence. Role changes in patients were important to caregivers. Much of the discussion with patients and caregivers described specific ways in which the stroke altered social relationships.

CONCLUSION

These findings are consistent with prior research indicating the importance of social factors to quality of life following stroke. Our findings suggest that measures of stroke-related quality of life should include assessment of social function and social support.

The impact of converting to an electronic health record on organizational culture and quality improvement

Implementing an information technology system can impact more than just quality of care and patient outcomes. The purpose of this 4-year, observational research project is to examine changes in organizational culture, quality improvement (QI) maturity, and quality of care following adoption of a single, electronic health record (EHR) system within an integrated healthcare network. The primary outcome measure, the Culture and Quality Questionnaire (CQQ), assesses the perceived culture of an organization and the degree of CQI maturity in seven quality management areas. Baseline surveys were distributed prior to conversion to the EHR. Subsequent data collection occurred at 12 months post "go live" and will occur at 24 and 36 months after the first hospital "go live". Secondary data were abstracted from routinely collected patient satisfaction measures and standard quality indicators. Contrary to expectation, our findings from the Baseline and 12-month follow-up data suggest that employees perceived the organizational culture as becoming more, rather than less, hierarchical. We also hypothesized that quality indicators would show improvement due to enhanced information flow and ease of information retrieval. This hypothesis was not supported by 1-year results. However, follow-up data from years two and three may provide different results.

Measuring quality of life in chronic illness: the functional assessment of chronic illness therapy measurement system

We focus on quality of life (QOL) measurement as applied to chronic illness. There are 2 major types of health-related quality of life (HRQOL) instruments-generic health status and targeted. Generic instruments offer the opportunity to compare results across patient and population cohorts, and some can provide normative or benchmark data from which to interpret results. Targeted instruments ask questions that focus more on the specific condition or treatment under study and, as a result, tend to be more responsive to clinically important changes than generic instruments. Each type of instrument has a place in the assessment of HRQOL in chronic illness, and consideration of the relative advantages and disadvantages of the 2 options best drives choice of instrument. The Functional Assessment of Chronic Illness Therapy (FACIT) system of HRQOL measurement is a hybrid of the 2 approaches. The FACIT system combines a core general measure with supplemental measures targeted toward specific diseases, conditions, or treatments. Thus, it capitalizes on the strengths of each type of measure. Recently, FACIT questionnaires were administered to a representative sample of the general population with results used to derive FACIT norms. These normative data can be used for benchmarking and to better understand changes in HRQOL that are often seen in clinical trials. Future directions in HRQOL assessment include test equating, item banking, and computerized adaptive testing.

Normative clinical relationships between orientation and memory: age as an important moderator variable

The present study examined the relationship between memory and orientation to time, place, and personal and general information, as moderated by age, education, and simple attentional ability. A heterogeneous sample of 312 clinical referrals was divided into four groups, according to delayed memory functioning. Patients with globally good, globally poor, poor visual, and poor auditory memory were at differential risk of being disoriented, with the globally poor memory patients having the greatest risk. Overall, poorly oriented patients were older and less educated, with worse recall of digits backward. Discriminant Function Analysis selected visual and auditory memory and age as predictors of orientation. Normative tables stratified by age and memory performance are presented.

Differential diagnosis of steroid-dependent nephrotic syndrome with psychosis: Practical and ethical issues

The present case illustrates practical and ethical issues that can be encountered by clinical psychologists providing consultation services in medical settings. The neuropsychological consultation service was asked to evaluate a 22-year-old male with psychosis, steroid-dependent nephrotic syndrome, and a family history of schizophrenia. MRI revealedmarked cortical atrophy. Clinical findings were consistent with (1) steroid inducedapparent atrophy, which has been shown to be reversible with withdrawal of steroids; (2) uremia secondary to steroid withdrawal; (3) cortical atrophy, found in some schizophrenics; or (4) an atypical, diffuse degenerative disorder. Clinical interview and psychological testing revealed significant thought disorder, prominent delusions, somatic hallucinations, and mood disturbance. Deterioration in social and academic functioning was also present. Except for impaired attention and concentration, neuropsychological evaluation showed no clear evidence of brain-based dysfunction. Further, neuropsychological results effectively ruled out a degenerative process and were not consistent with a steroid effects profile. A conclusive differential diagnosis of steroid induced psychosis versus severe psychopathology would require withdrawal from steroids and antipsychotics. In addition to the practical and ethical issues of withholding antipsychotics, steroid withdrawal would require either dialysis or renal transplant surgery. Decision making regarding dialysis dependency and the possibility of postsurgical psychosis secondary to true psychopathology were salient issues to both the patient and the treatment team.