Assessing Language in Unstructured Conversation in People With Aphasia: Methods, Psychometric Integrity, Normative Data, and Comparison to a Structured Narrative Task

Towards efficient, ecological assessment of interaction: A scoping review of co-constructed communication

BACKGROUND

The complexity of communication presents challenges for clinical assessment, outcome measurement and intervention for people with acquired brain injury. For the purposes of assessment or treatment, this complexity is usually managed by isolating specific linguistic functions or speech acts from the interactional context. Separating linguistic functions from their interactional context can lead to discourse being viewed as a static entity comprised of discrete features, rather than as a dynamic process of co-constructing meaning. The ecological validity of discourse assessments which rely on the deconstruction of linguistic functions is unclear. Previous studies have reported assessment tasks that preserve some of the dialogic features of communication, but as yet, these tasks have not been identified as a distinct genre of assessment. We suggest the term 'co-constructed communication' to describe tasks which are specifically designed to capture the dynamic, jointly produced nature of communication within a replicable assessment task.

AIMS

To identify and summarize how co-constructed communication has been assessed with individuals with non-progressive acquired communication disability regarding task design, measures and psychometric robustness.

METHODS

A scoping review methodology was used to identity relevant studies. Systematic database searches were conducted on studies published before July 2021. Studies in the yield were assessed against eligibility criteria, with 37 studies identified as eligible for inclusion.

MAIN CONTRIBUTION

This is the first time that co-constructed communication has been defined as a genre of discourse assessment for stroke and traumatic brain injury populations. Co-constructed communication has been assessed for 144 individuals with aphasia and 111 with cognitive-communication disability. Five categories of co-constructed communication tasks were identified, ranging in complexity. Variability exists in how these assessment tasks are labelled and measured. Assessment measures require further psychometric profiling, specifically regarding test-retest reliability and validity.

CONCLUSIONS

Co-constructed communication is a discourse genre which offers researchers and clinicians a replicable method to assess language and communication in an experimentally rigorous way, within an ecologically valid context, bridging the gap between experimental and ecological assessment approaches.

WHAT THIS PAPER ADDS

What is already known on this subject Standardized assessments of language skills and monologue offer reliable, replicable ways to measure language. However, isolating language from an interactional context fundamentally changes the behaviour under study. This raises questions about the ecological validity of the measures we routinely use to determine diagnoses, guide treatment planning and measure the success of treatment. What this study adds to the existing knowledge This review highlights studies that conceptualize, and often quantify, interaction by combining experimental rigour and aspects of everyday dialogue. This is the first time this genre of discourse assessment has been identified. We propose the term 'co-constructed communication' to describe this genre and provide an operational definition for the term. What are the practical and clinical implications of this study? Co-constructed communication assessment tasks require refinement, particularly regarding aspects of psychometric robustness. In the future, these tasks offer pragmatic, meaningful ways to capture the effect and impact of aphasia and cognitive-communication disability within interaction.

Pilot Results for ECoLoGiC-Tx: A New Conversation-Level Intervention Improving Language in People With Moderate to Severe Aphasia

PURPOSE

This study reports pilot data for a novel intervention, ECoLoGiC-Tx, delivered to four people with moderate to severe aphasia. ECoLoGiC-Tx addresses language and communication in unstructured, participant-led conversation. The speech-language pathologist (SLP) uses a framework to choose turns that facilitate a social interaction. When communication breakdown occurs, the SLP implements a least-to-most hierarchy to maximize the people with aphasia's (PWA's) independence in self-repair. ECoLoGiC-Tx draws its theoretical underpinnings from conversation analysis and theories of rehabilitation, including principles of complexity, neuroplasticity, and learning.

METHOD

Four PWA attended 60-min sessions twice weekly for 10 weeks. Assessment occurred at pretreatment, posttreatment, and 6-week maintenance. Outcomes included established discourse measures for conversation and monologue, tests of language and functional communication, and patient-/family-reported outcome measures (P/FROMs). Discourse samples were collected three times per assessment. Interrater reliability and fidelity for assessment and treatment procedures are reported.

RESULTS

Participants presented with Broca's aphasia (one moderate, one severe) or conduction aphasia (one moderate, one severe). Each demonstrated improvements in discourse, test batteries, and P/FROMs. They all demonstrated reduced aphasia severity measured by the Western Aphasia Battery-Revised at posttreatment or maintenance. Change in conversation and monologue was robust for three participants, but was mixed for one person (P1: moderate Broca's aphasia). P/FROMs indicated improvement at posttreatment and maintenance for all participants. Most treatment gains were maintained at 6-week follow-up.

CONCLUSIONS

This study provides promising results for ECoLoGiC-Tx to improve language function of people with chronic moderate to severe aphasia. Generalization occurred to tests, functional communication, spontaneous conversation, and structured monologue tasks.

Handwritten and Typed Discourse in People With Aphasia: Reference Data for Sequential Picture Description and Comparison of Performance Across Modality

PURPOSE

Writing can be completed by hand or by typing. Increasingly, functional and social activities are completed in the virtual domain, which often requires discourse level writing. Yet, there is a shortage of research on discourse writing in aphasia. The purpose of this study was to provide preliminary reference data for a sequential picture description task in writing by hand and typing for people with aphasia. Additionally, we examined individual modality differences when comparing handwritten and typed discourse.

METHOD

Fifteen people with mild-moderate aphasia participated in this study. They completed a sequential picture description task in handwriting and in typing. Discourse samples were coded for Correct Information Units (CIUs) and Complete Utterances. Measures of productivity were also evaluated (e.g., Total Words, Total Utterances). Participants completed a computer use questionnaire regarding their current and premorbid typing and computer use.

RESULTS

Preliminary reference data are reported. No significant differences were found at the group level for the measures evaluated. At the individual level, there was evidence of a modality effect for seven participants who demonstrated differences in the proportion of CIUs.

CONCLUSIONS

Although preliminary, these findings suggest that, at the group level, the handwritten and typed discourse produced by people with mild-moderate aphasia is similar. However, at the individual level, there is potential for modality differences. Consistent patterns of premorbid computer use, difficulty ratings, and individual differences in writing modes were not identified for the participants who demonstrated a modality effect. However, there was preliminary evidence that poststroke handedness may contribute to modality differences exhibited by some participants, which should be explored in future research. Additionally, these reference data are preliminary and further research is required from a more heterogeneous group of people with aphasia and to better establish assessment practices for discourse writing.

Analyzing Language in the Picnic Scene Picture and in Conversation: The Type of Discourse Sample We Choose Influences Findings in People With Aphasia

PURPOSE

People with aphasia express that improved conversational discourse is a primary rehabilitation goal. Discourse is usually assessed using monologue, such as a picture description task, but research shows that language in monologue varies from language in everyday conversation. Consequently, we investigated the relationship of language in unstructured conversation and in the picnic scene picture because it is a part of the most often used aphasia battery (Western Aphasia Battery-Revised) and thus is frequently used to inform therapy. Second, because previous research suggests people with severe aphasia may not demonstrate language production variability between types of monologue-level discourse, we evaluated the relationship of severity and the difference in scores between conversation and the picnic scene task.

METHOD

Thirty-four people with mild-to-severe aphasia described the picnic scene and provided a conversation sample. We measured language production and communicative success using seven measures with established psychometrics in conversation/monologue. We conducted correlations to answer the research questions.

RESULTS

Correlations were moderate and weaker for the measures in the two conditions. A strong negative relationship was demonstrated between aphasia severity and global coherence. All other relationships were moderate and weaker for the remaining measures when correlated with aphasia severity (also negative).

CONCLUSIONS

Results are consistent with other studies indicating that language varies in different types of discourse. We conclude that for accurate, meaningful assessment, discourse sampling needs to include the specific type of discourse the individual wishes to address in therapy, because discourse samples and their findings are not interchangeable.

Test-Retest Reliability of Microlinguistic Information Derived From Spoken Discourse in Persons With Chronic Aphasia

PURPOSE

The purpose of this study was to characterize test-retest reliability of discourse measures across a battery of common tasks in individuals with aphasia and prospectively matched adults without brain damage.

METHOD

We collected spoken discourse during five monologue tasks at two timepoints (test and retest; within 2 weeks apart) in an aphasia group (n = 23) and a peer group with no brain damage (n = 24). We evaluated test-retest reliability for percentage of correct information units, correct information units per minute, mean length of utterance, verbs per utterance, noun/verb ratio, open/closed class word ratio, tokens, sample duration (seconds), propositional idea density, type-token ratio, and words per minute. We explored reliability's relationship with sample length and aphasia severity.

RESULTS

Rater reliability was excellent. Across tasks, both groups demonstrated discourse measures with poor, moderate, and good reliability, with the aphasia group having measures demonstrating excellent test-retest reliability. When evaluating measures within each task, test-retest reliability again ranged from poor to excellent for both groups. Across groups and task, measures that appeared most reliable appeared to reflect lexical, informativeness, or fluency information. Sample length and aphasia severity impacted reliability, and this differed across and by task.

CONCLUSIONS

We identified several discourse measures that were reliable across and within tasks. Test-retest statistics are intimately linked to the specific sample, emphasizing the importance of multiple baseline studies. Task itself should be considered an important variable, and it should not be assumed that discourse measures found to be reliable across several tasks (averaged) are likewise reliable for a single task.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23298032.

Evaluating circumlocution in naming as a predictor of communicative informativeness and efficiency in discourse

Background

Discourse analyses yield quantitative measures of functional communication in aphasia. However, they are historically underutilized in clinical settings. Confrontation naming assessments are used widely clinically and have been used to estimate discourse-level production. Such work shows that naming accuracy explains moderately high proportions of variance in measures of discourse, but proportions of variance remain unexplained. We propose that the inclusion of circumlocution productions into predictive models will account for a significant amount more of the variance. Circumlocution productions at the naming-level, while they may not contain the target word, are similar to the content that contributes to discourse informativeness and efficiency. Thus, additionally measuring circumlocution may improve our ability to estimate discourse performance and functional communication.

Aim

This study aimed to test whether, after controlling for naming accuracy, the addition of a measure of circumlocution into predictive models of discourse-level informativeness and efficiency would account for a significant amount more of the variance in these discourse-level outcomes.

Methods & Procedures

Naming and discourse data from 43 people with poststroke aphasia were analyzed. Naming data were collected using 120 pictured items and discourse data were collected using two picture description prompts. Data scoring and coding yielded measures of naming accuracy, incorrect response type, communicative informativeness, and efficiency. We used robust hierarchical regression to evaluate study predictions.

Outcomes & Results

After controlling for naming accuracy, the inclusion of circumlocution into predictive models accounted for a significant amount more of the variance in both informativeness and efficiency. The subsequent inclusion of other response types, such as real word and nonword errors, did not account for a significant amount more of the variance in either outcome.

Conclusions

In addition to naming accuracy, the production of circumlocution during naming assessments may correspond with measures of informativeness and efficiency at the discourse-level. Reducing the burden of estimating patients' functional communication will increase our ability to estimate functional communication using tools that are easy to administer and interpret.

Transactional Success in the Texting of Individuals With Aphasia

PURPOSE

Text messaging is a pervasive form of communication in today's digital society. Our prior research indicates that individuals with aphasia text, but they vary widely in how actively they engage in texting, the types of messages they send, and the number of contacts with whom they text. It is reported that people with aphasia experience difficulties with texting; however, the degree to which they are successful in conveying information via text message is unknown. This study describes the development of a rating scale that measures transactional success via texting and reports on the transactional success of a sample of 20 individuals with chronic aphasia. The relationships between texting transactional success and aphasia severity, texting confidence, and texting activity are explored.

METHOD

Performance on a texting script was evaluated using a three-category rating in which turns elicited from participants with aphasia received a score of 0 (no transaction of message), 1 (partial transaction), or 2 (successful transaction). Internal consistency was assessed using Cronbach's alpha. Interrater reliability was determined using intraclass correlation coefficient and Krippendorff's alpha.

RESULTS

Although preliminary, results suggest adequate internal consistency and strong interrater reliability. Texting transactional success on the script response items was significantly correlated with overall aphasia severity and severity of reading and writing deficits, but there was no relationship between transactional success and texting confidence or overall texting activity.

CONCLUSIONS

This study describes initial efforts to develop a rating scale of texting transactional success and to evaluate the validity of scores derived from this measure. Information from a texting transactional success measure could inform treatment that aims to improve electronic messaging in people with aphasia.

Two-Year Longitudinal Evaluation of Community Aphasia Center Participation on Linguistic, Functional Communication, and Quality of Life Measures Across People With a Range of Aphasia Presentations

PURPOSE

The purpose of this study was to evaluate potential changes on a hierarchy of language tasks and measures of functional communication and quality of life in a group of people with aphasia (PWA) who attended a community aphasia center for 2 years. A secondary purpose was to determine whether there were any predictors of change.

METHOD

Twenty-seven PWA who attended Brooks Rehabilitation Aphasia Center (BRAC) were evaluated on an aphasia battery, confrontation naming, and structured discourse in addition to completing self-reported measures of functional communication and quality of life at three time points: before attending BRAC and after 1 (N = 27) and 2 (N = 20) years of BRAC participation. Twenty-six communication partners who communicated regularly with the PWA completed a questionnaire about their functional communication at the same time points. A mixed linear model was conducted for all dependent variables to determine change over time. Tau-b correlations were conducted between demographic and aphasia-related variables and difference scores for outcome measures that exhibited significant improvements.

RESULTS

At 1-year testing, significant improvements were observed on the aphasia battery, object and action naming, and all self- and communication partner-reported measures. At 2-year testing, all improvements were maintained except for the self-reported measure of functional communication. Structured discourse showed increases in average number of words, percentage of meaningful words and utterances, and efficiency of meaningful word production after 2 years. No significant correlations were observed between predictor variables and difference scores.

CONCLUSIONS

Participation in aphasia centers can result in significant changes in language, functional communication, and quality of life in people with chronic aphasia. These findings support the importance of aphasia centers in the continuum of care for PWA.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21313689.

Validation of an Automated Procedure for Calculating Core Lexicon From Transcripts

PURPOSE

The aim of this study was to advance the use of structured, monologic discourse analysis by validating an automated scoring procedure for core lexicon (CoreLex) using transcripts.

METHOD

Forty-nine transcripts from persons with aphasia and 48 transcripts from persons with no brain injury were retrieved from the AphasiaBank database. Five structured monologic discourse tasks were scored manually by trained scorers and via automation using a newly developed CLAN command based upon previously published lists for CoreLex. Point-to-point (or word-by-word) accuracy and reliability of the two methods were calculated. Scoring discrepancies were examined to identify errors. Time estimates for each method were calculated to determine if automated scoring improved efficiency.

RESULTS

Intraclass correlation coefficients for the tasks ranged from .998 to .978, indicating excellent intermethod reliability. Automated scoring using CLAN represented a significant time savings for an experienced CLAN user and for inexperienced CLAN users following step-by-step instructions.

CONCLUSIONS

Automated scoring of CoreLex is a valid and reliable alternative to the current gold standard of manually scoring CoreLex from transcribed monologic discourse samples. The downstream time saving of this automated analysis may allow for more efficient and broader utilization of this discourse measure in aphasia research. To further encourage the use of this method, go to https://aphasia.talkbank.org/discourse/CoreLexicon/ for materials and the step-by-step instructions utilized in this project.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.20399304.

"If You Just Stay With Me and Wait…You'll Get an Idea of What I'm Saying": The Communicative Benefits of Time for Conversational Self-Repair for People With Aphasia

PURPOSE

This study investigated the communicative benefits of self-repair during conversation for persons with aphasia (PWAs). Self-repair of trouble sources is an interactional priority that emphasizes autonomy and competence. Of equal importance, conversationalists desire to minimize silences and work together to ensure forward movement (progressivity) of conversation. Simultaneously achieving progressivity and self-repair is challenging in aphasia, and PWAs and their partners often make trade-off decisions between these two activities. Conversation-level aphasia interventions usually focus on supportive techniques that promote participation while maintaining progressivity, effectively favoring progressivity over self-repair. This study evaluates the benefits of an alternative approach that shifts the emphasis to self-repair, thereby highlighting potential trade-off costs of routinely forgoing self-repair to achieve progressivity.

METHOD

Ten people with mild-to-moderate aphasia each held two conversations with two different partners. When trouble sources characterized by silent and/or filled pauses occurred, partners maintained a supportive and engaged stance, allowing PWAs time to self-repair. We analyzed language produced during these "edited turns" using three paradigms considering form, content, and use.

RESULTS

The data yielded 311 edited turns. For form, on average, each edited turn resulted in 3.72 words; for content, most edited turns contained autobiographical information; for use, approximately 40% of edited turns introduced new information, and 40% added to the ongoing topic. The remainder were either ambiguous or comments such as, "I can't think of it."

CONCLUSIONS

When given engaged support and time to self-repair, PWAs contributed meaningful personal information to conversations for approximately 80% of edited turns. Importantly, self-repair often resulted in self-expression that directed the conversation, which is a communicative role critical for empowering agency and identity. This research opens a dialogue about benefits and limitations of approaches that prioritize either progressivity or self-repair and how to balance the two to optimize therapeutic benefits for each individual.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19379738.

Toward Empowering Conversational Agency in Aphasia: Understanding Mechanisms of Topic Initiation in People With and Without Aphasia

PURPOSE

This study examined topic initiation (TI) in conversations involving people with aphasia (PWA), matched people without aphasia (M-PWoA), and speech-language pathologists who were their conversation partners (SLP-Ps). For each speaker type, we analyzed patterns of distribution of typical mechanisms of TI and patterns of simultaneous use of multiple TI mechanisms. Lastly, we examined associations between use of simultaneous TI mechanisms and communicative success.

METHOD

Twenty PWA and 20 M-PWoA each participated in two conversations with SLP-Ps. Conversation samples were analyzed for TI locations and mechanisms, with results tallied for each speaker type following a published typology. A measure of communicative success was applied to TI utterances. Rank-order correlations were conducted to evaluate the patterns of distribution of the TI mechanisms between speaker types and the patterns of multiple mechanism usage between speaker types. Descriptive analysis was conducted to provide additional insight to the TI behaviors of each speaker type and to evaluate the relationship between multiple TI mechanisms and communicative success.

RESULTS

All speaker types used cohesion most often to achieve TI. PWA used an abrupt method of TI (noncoherent TI) more often than other speaker types. A single mechanism of TI was used most often by all speaker types, except for SLP-Ps when they were in conversations with PWA. In this case, SLP-Ps most often used two or more layered mechanisms of TI. SLP-Ps also used a highly salient TI mechanism with greater frequency when speaking with PWA than observed between other speaker types. When PWA layered mechanisms of TI, they appeared to be more likely to achieve better communicative success.

CONCLUSIONS

Specific, teachable behaviors such as favoring certain TI mechanisms and using multiple TI mechanisms may improve communicative success during TI for PWA. Furthermore, findings suggest that SLP-Ps modify their TI behaviors when speaking to PWA.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.17699423.