Percent Grammatical Utterances Between 4 and 9 Years of Age for the Edmonton Narrative Norms Instrument: Reference Data and Psychometric Properties

Should We Stop Using Lexical Diversity Measures in Children's Language Sample Analysis?

PURPOSE

Prior work has identified weaknesses in commonly used indices of lexical diversity in spoken language samples, such as type-token ratio (TTR) due to sample size and elicitation variation, we explored whether TTR and other diversity measures, such as number of different words/100 (NDW), vocabulary diversity (VocD), and the moving average TTR would be more sensitive to child age and clinical status (typically developing [TD] or developmental language disorder [DLD]) if samples were obtained from standardized prompts.

METHOD

We utilized archival data from the norming samples of the Test of Narrative Language and the Edmonton Narrative Norms Instrument. We examined lexical diversity and other linguistic properties of the samples, from a total of 1,048 children, ages 4-11 years; 798 of these were considered TD, whereas 250 were categorized as having a language learning disorder.

RESULTS

TTR was the least sensitive to child age or diagnostic group, with good potential to misidentify children with DLD as TD and TD children as having DLD. Growth slopes of NDW were shallow and not very sensitive to diagnostic grouping. The strongest performing measure was VocD. Mean length of utterance, TNW, and verbs/utterance did show both good growth trajectories and ability to distinguish between clinical and typical samples.

CONCLUSIONS

This study, the largest and best controlled to date, re-affirms that TTR should not be used in clinical decision making with children. A second popular measure, NDW, is not measurably stronger in terms of its psychometric properties. Because the most sensitive measure of lexical diversity, VocD, is unlikely to gain popularity because of reliance on computer-assisted analysis, we suggest alternatives for the appraisal of children's expressive vocabulary skill.

Diagnostic Accuracy of Grammaticality and Utterance Length in Bilingual Children

PURPOSE

The purpose of this study was to investigate the diagnostic accuracy of two measures derived from spontaneous language samples, mean length of utterance in words (MLUw) and percentage of grammatical utterances (PGU), in identifying developmental language disorder (DLD) in Spanish-English bilingual children. We examined two approaches: best language and total language.

METHOD

The participants in this study included 74 Spanish-English bilingual children with (n = 36) and without (n = 38) DLD. Language samples were elicited through a story retell and story generation task using Frog wordless picture books in English and Spanish. Stories were transcribed and coded using the Systematic Analysis of Language Samples (Miller & Iglesias, 2020) to extract MLUw and PGU in both languages.

RESULTS

Logistic regression analyses suggested that a model that included PGU, MLUw, and age achieved the best diagnostic accuracy in predicting group membership. Both approaches, best language and total language, had fair diagnostic accuracy.

CONCLUSIONS

In combination, PGU and MLUw seem to be useful diagnostic tools to differentiate bilingual children with and without DLD. Clinical implications and usability are discussed.

Validating the Use of Percent Grammatical Utterances for Assessing Mandarin-Speaking Children's Grammatical Skill: Evidence From 3-Year-Olds

PURPOSE

The study examined the use of percent grammatical utterances (PGUs) for assessing grammatical skills in Mandarin-speaking 3-year-old children.

METHOD

Participants were 30 Mandarin-speaking 3-year-olds with typical development. Language samples were collected in two visits for each child using a picture description task. Children were asked to talk about 16 pictures in response to questions and prompts at each visit. Pictures for the language sample collection were identical across the visits. PGUs were computed, and the grammatical errors that children produced in the task were coded and tallied for error types at each visit. Test-retest reliability, split-half reliability, and concurrent criterion validity of PGUs were evaluated.

RESULTS

The mean PGU level was approximately 78% at Visit 1 and 81% at Visit 2, both of which were significantly below the mastery level (i.e., 90%). The correlation coefficient for test-retest reliability of PGU was large (r = .70, p < .01); the correlation coefficient for split-half reliability was medium at Visit 1 (r = .47, p < .01) and large (r = .65, p < .01) at Visit 2. In addition, the correlation coefficient for concurrent criterion validity of PGU was medium for both visits (rs ≥ .35, ps ≤ .03). The ranking and proportion of each error type were similar between the visits.

CONCLUSION

The initial evidence from psychometric properties suggests that PGU computed from the picture description task is a reliable and valid measure for evaluating grammatical skills in Mandarin-speaking 3-year-old children.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25395499.

Indices of Narrative Language Associated with Disability

Narratives skills are associated with long-term academic and social benefits. While students with disabilities often struggle to produce complete and complex narratives, it remains unclear which aspects of narrative language are most indicative of disability. In this study, we examined the association between a variety of narrative contents and form indices and disability. Methodology involved drawing 50 K-3 students with Individual Education Programs (IEP) and reported language concerns from a large diverse sample (n = 1074). Fifty typically developing (TD) students were matched to the former group using propensity score matching based on their age, gender, grade, mother's education, and ethnicity. Narrative retells and generated language samples were collected and scored for Narrative Discourse and Sentence Complexity using a narrative scoring rubric. In addition, the number of different words (NDW), subordination index (SI), and percentage of grammatical errors (%GE) were calculated using computer software. Results of the Mixed effect model revealed that only Narrative Discourse had a significant effect on disability, with no significant effect revealed for Sentence Complexity, %GE, SI, and NDW. Additionally, Narrative Discourse emerged as the sole significant predictor of disability. At each grade, there were performance gaps between groups in the Narrative Discourse, Language Complexity, and SI. Findings suggest that difficulty in Narrative Discourse is the most consistent predictor of disability.

Developmental language disorder in Chinese children: A systematic review of research from 1997 to 2022

Developmental language disorder (DLD) is one of the most common neurodevelopmental disorders. The influences of DLD on language development have been delineated in detail in English. The same is not true for Chinese, a group of Sinitic languages with distinct typological features that may modify the profile of DLD crosslinguistically. We conducted a systematic search of English and Chinese journal databases and reviewed 59 studies on the manifestations of DLD in Chinese. Methodological quality appraisal of the literature revealed several areas of improvement to enhance transparency and replicability. A bibliometric analysis indicated a steep growth trajectory of this literature. Examination of the participant selection and diagnostic criteria revealed limitations and calls for the development of assessment tools and increased knowledge of evidenced-based diagnostic practice. Areas of deficits demonstrated by Chinese children with DLD were synthesized qualitatively and discussed in light of the literature on clinical markers of DLD in English.

Speech-Language Pathologists' Endorsement of Speech, Language, and Literacy Myths Reveals Persistent Research-Practice Gap

PURPOSE

We explore the extent to which speech-language pathologists (SLPs) are operating under assumptions about speech, language, and literacy that are not supported by evidence or are contradicted by evidence.

METHOD

SLPs (N = 106) marked the degree to which they endorsed or rejected four true and 11 myth (i.e., false) statements on a visual analog scale via an online survey. We analyzed the degree to which participants agreed or disagreed with these statements related to speech, language, and literacy development and impairment.

RESULTS

Based on results of one-sample t tests, participants as a group correctly rejected seven myth statements and correctly endorsed three true statements. Participants as a group provided equivocal responses to the remaining four myth statements and one true statement. Scores for each statement spanned all or nearly all of the possible range. Even when participants overall showed relatively strong disagreement with a particular myth statement, at least a few participants endorsed the myth statement.

CONCLUSIONS

The findings indicate areas of relative strength for SLPs' knowledge of current evidence for augmentative and alternative communication, bilingualism, and language input that supports language development. Identified areas of relative weakness include knowledge of the expected grammatical and speech production skills of children with typical development and the influence of ear infections in childhood on language impairment. Additional research is needed to evaluate efforts to enhance SLPs' knowledge and use of evidence-based practices.

Evaluating Different Scoring Systems for a Picture Description Task Among Preschool Children Who Speak African American English

PURPOSE

Measures of grammatical accuracy are effective measures of children's language skills. However, many measures, such as percent grammatical utterances, were developed for children who speak General American English (GAE) and, therefore, may not be appropriate for students who speak other dialects. This study examines different scoring systems for a picture description task to explore the impact of different systems for children who speak African American English (AAE).

METHOD

Eighteen preschool-age children who speak AAE completed a play-based language sample and a picture description task. The Index of Productive Syntax (IPSyn) was calculated for the play-based language samples and used as the reference measure. The picture description task was scored using four scoring systems: an expansive AAE scoring system, a GAE scoring system, and two strategic scoring systems. Scores were compared for each scoring system, and correlations between IPSyn scores and picture description scores were conducted.

RESULTS

Scores on the picture description task were highest in the expansive AAE scoring system, followed by scores in the strategic scoring systems, all of which were higher than scores in the GAE scoring system. There was a significant correlation between IPSyn scores and picture description scores when using the GAE scoring system and the strategic scoring systems, but not when using the expansive AAE scoring system.

CONCLUSIONS

Different scoring systems affect AAE-speaking preschoolers' scores on measures of grammatical accuracy, and the use of an expansive AAE scoring system, based on lists of nonmainstream features, may diminish the ability to differentiate between children with different ability levels. Future research is needed to refine scoring systems and to explore the validity of different scoring systems for detecting differences between preschoolers who speak AAE, with and without developmental language disorder.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21498618.

Sharpening Our Tools: A Systematic Review to Identify Diagnostically Accurate Language Sample Measures

PURPOSE

This systematic review provides a comprehensive summary of the diagnostic accuracy of English language sample analysis (LSA) measures for the identification of developmental language disorder.

METHOD

An electronic database search was conducted to identify English publications reporting empirical data on the diagnostic accuracy of English LSA measures for children aged 3 years or older.

RESULTS

Twenty-eight studies were reviewed. Studies included between 18 and 676 participants ranging in age from 3;0 to 13;6 (years;months). Analyzed measures targeted multiple linguistic domains, and diagnostic accuracy ranged from less than 25% to greater than 90%. Morphosyntax measures achieved the highest accuracy, especially in combination with length measures, and at least one acceptable measure was identified for each 1-year age band up to 10 years old.

CONCLUSION

Several LSA measures or combinations of measures are clinically useful for the identification of developmental language disorder, although more research is needed to replicate findings using rigorous methods and to explore measures that are informative for adolescents and across diverse varieties of English.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21183247.

Investigating Narrative Performance in Children With Developmental Language Disorder: A Systematic Review and Meta-Analysis

PURPOSE

Narrative assessment is one potentially underutilized and inconsistent method speech-language pathologists may use when considering a diagnosis of developmental language disorder (DLD). However, narration research encompasses many varied methodologies. This systematic review and meta-analysis aimed to (a) investigate how various narrative assessment types (e.g., macrostructure, microstructure, and internal state language) differentiate children with typical development (TD) from children with DLD, (b) identify specific narrative assessment measures that result in greater group differences, and (c) evaluate participant and sample characteristics that may influence performance differences.

METHOD

Electronic databases (PsycINFO, ERIC, and PubMed) and ASHAWire were searched on July 30, 2019, to locate studies that reported oral narrative language measures for both DLD and TD groups between ages 4 and 12 years; studies focusing on written narration or other developmental disorders only were excluded. We extracted data related to sample participants, narrative task(s) and assessment measures, and research design. Group differences were quantified using standardized mean differences. Analyses used mixed-effects meta-regression with robust variance estimation to account for effect size dependencies.

RESULTS

Searches identified 37 eligible studies published between 1987 and 2019, including 382 effect sizes. Overall meta-analysis showed that children with DLD had decreased narrative performance relative to TD peers, with an overall average effect of -0.82 SD, 95% confidence interval [-0.99, -0.66]. Effect sizes showed significant heterogeneity both between and within studies, even after accounting for effect size-, sample-, and study-level predictors. Across model specifications, grammatical accuracy (microstructure) and story grammar (macrostructure) yielded the most consistent evidence of TD-DLD group differences.

CONCLUSIONS

Present findings suggest some narrative assessment measures yield significantly different performance between children with and without DLD. However, researchers need to improve consistency of inclusionary criteria, descriptions of sample characteristics, and reporting of correlations between measures to determine which assessment measures reliably distinguish between groups.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21200380.

The Reliability of Short Conversational Language Sample Measures in Children With and Without Developmental Language Disorder

PURPOSE

Language sample analysis (LSA) represents an ecologically valid method for diagnosing, identifying goals, and measuring progress in children with developmental language disorder (DLD). LSA is, however, time consuming. The purpose of this study was to determine the length of sample needed to obtain reliable LSA measures for children in kindergarten and first grade with typical language (TL) and DLD using automated analyses from the Systematic Analysis of Language Transcripts software.

METHOD

Play-based conversational language samples collected on kindergarten to first-grade children with TL (n = 21) and DLD (n = 21) from a community-based sample were analyzed. Eight LSA measures were calculated from 1-, 3-, 5-, 7-, and 10-min sample cuts and compared to 20-min samples for reliability.

RESULTS

Reliability estimates were similar for the TL and DLD groups except for errors and omissions, which showed overall higher levels of reliability in the DLD group and reached acceptable levels at 3 min. Percent grammatical utterances were reliable at 7 min in the DLD group and not reliable in shorter samples in the TL group. The subordination index was reliable at 10 min for both groups. Number of different words reached acceptable reliability at the 3-min length for the DLD group and at the 10-min length for the TL group. Utterances and words per minute were reliable at 3 min and mean length of utterance at 7 min in both groups.

CONCLUSIONS

Speech-language pathologists can obtain reliable LSA measures from shorter, 7-min conversational language samples from kindergarten to first-grade children with DLD. Shorter language samples may encourage increased use of LSA.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19529287.

Tell or Retell? The Role of Task and Language in Spanish-English Narrative Microstructure Performance

PURPOSE

This study examined performance of dual language learners (DLLs) on Spanish- and English-language narrative story retells and unique tells. Transcription and analysis focused on comparisons of common microstructural language sample measures in Spanish and English across tasks. Each language sample measure was evaluated for its possible convergence with norm-referenced standardized assessments for DLL children.

METHOD

Spanish-English DLLs (N = 133) enrolled in English-only kindergarten or first-grade classrooms completed two-language sample tasks (one in each language), which were transcribed and analyzed using Systematic Analysis of Language Transcripts (Miller & Iglesias, 2017) for measures of syntactic complexity (mean length of utterance [MLU] in words), lexical diversity (number of different words [NDW]), and grammaticality (percent grammatical utterances [PGU]). Students also completed a norm-referenced sentence repetition task (Peña et al., 2014) and expressive vocabulary assessment (Martin, 2013).

RESULTS

Comparison of story retells and unique stories revealed similar performance on MLU, NDW, and PGU across elicitation techniques, with one exception: NDW in Spanish was higher in the story retell condition. Predictive models revealed several differences in the relations between the microstructure measures and norm-referenced language measures by elicitation technique, although neither context demonstrated a consistent advantage across all metrics.

CONCLUSIONS

Measures derived from story retells and unique tells offer practical findings for speech-language pathologists and other educators to use in assessment of early grade DLLs. This work increases knowledge of procedural differences across narrative assessments and their influence on language variables, supporting school-based speech-language pathologists in making assessment decisions for DLLs on their caseload.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19191278.

Predicting Language Performance From Narrative Language Samples

PURPOSE

Analysis of narrative language samples is a recommended clinical practice in the assessment of children's language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3.

METHOD

Participants were a sample of 284 children who were administered a narrative retell task in kindergarten and a battery of vocabulary and grammar measures in kindergarten through Grade 3. Language samples were analyzed for number of different words, mean length of utterance, and a relatively new metric, percent grammatical utterances (PGUs). Structural equation models were used to estimate the concurrent and longitudinal relationships.

RESULTS

The narrative language sample metrics were consistently correlated with the individual vocabulary and grammar measures as well as the language factor in each grade, and also consistently and uniquely predicted the language factor in each grade. Standardized path estimates in the structural equation models ranged from 0.20 to 0.39.

CONCLUSIONS

This study found narrative language sample metrics to be predictive, concurrently and longitudinally, of a latent factor of language from kindergarten through Grade 3. These results further validate the importance of collecting and analyzing narrative language samples, to include PGU along with more traditional metrics, and point to directions for future research.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.17700980.

The Complex Role of Utterance Length on Grammaticality: Multivariate Multilevel Analysis of English and Spanish Utterances of First-Grade English Learners

PURPOSE

This study examined the relationship between utterance length, syntactic complexity, and the probability of making an error at the utterance level.

METHOD

The participants in this study included 830 Spanish-speaking first graders who were learning English at school. Story retells in both Spanish and English were collected from all children. Generalized mixed linear models were used to examine within-child and between-children effects of utterance length and subordination on the probability of making an error at the utterance level.

RESULTS

The relationship between utterance length and grammaticality was found to differ by error type (omission vs. commission), language (Spanish vs. English), and level of analysis (within-child vs. between-children). For errors of commission, the probability of making an error increased as a child produced utterances that were longer relative to their average utterance length (within-child effect). Contrastively, for errors of omission, the probability of making an error decreased when a child produced utterances that were longer relative to their average utterance length (within-child effect). In English, a child who produced utterances that were, on average, longer than the average utterance length for all children produced more errors of commission and fewer errors of omission (between-children effect). This between-children effect was similar in Spanish for errors of commission but nonsignificant for errors of omission. For both error types, the within-child effects of utterance length were moderated by the use of subordination.

CONCLUSION

The relationship between utterance length and grammaticality is complex and varies by error type, language, and whether the frame of reference is the child's own language (within-child effect) or the language of other children (between-children effect).

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.17035916.

The Contribution of Socioeconomic Status to Children's Performance on Three Grammatical Measures in the Edmonton Narrative Norms Instrument

Purpose The purpose of this study was to evaluate the relative contribution of socioeconomic status (SES) on three grammatical measures-finite verb morphology composite (FVMC), percent grammatical utterances (PGU), and clausal density-in children between the ages of 4 and 9 years. Method Data for this study were from the normative sample in the Edmonton Narrative Norms Instrument. For 359 children, hierarchical linear regression was performed to evaluate the amount of variance in FVMC, PGU, and clausal density that was uniquely explained by SES after accounting for child chronological age and language status (typical, impaired). Results After child age and language status were controlled, SES was a significant predictor of PGU and clausal density scores, but not of FVMC scores. SES uniquely accounted for 0.5% of variance in PGU scores and 0.8% of variance in clausal density scores. Conclusions Consistent with maturational accounts of children's development of tense markers, results of this study offer evidence that, among grammatical measures, FVMC is uniquely robust to variation in SES. Although significant, the variance of PGU and clausal density scores uniquely accounted for by SES was close to minimum. Clinicians can therefore include these three grammatical measures for assessing children of different socioeconomic backgrounds. Supplemental Material https://doi.org/10.23641/asha.14810484.

Properties of Lexical Diversity in the Narratives of Children With Typical Language Development and Developmental Language Disorder

Purpose We examined four measures of lexical diversity in the narratives of children with typical language development (TLD) and developmental language disorder (DLD) that comprised the normative sample of the Edmonton Narrative Norms Instrument (Schneider et al., 2005). The purpose was to document the properties of each measure with respect to variations in utterance and sample length, developmental trends, and group differences. Method The sample consisted of 377 picture-elicited, story generation transcripts from children with TLD (n = 300) and DLD (n = 77) aged 4-9 years. We extracted the moving-average type-token ratio (MATTR) and the number of different words from the full sample, from samples equated for the number of utterances, and from samples equated for the total number of words. Results MATTR was the only measure to show no relationships to utterance or sample length. All measures showed significant positive growth with age and significant groupwise differences between children with TLD and DLD. However, the magnitude of age effects and differentiation between groups varied considerably across measures. Across measures, there were significant differences in the number of children with DLD who were identified with low lexical diversity relative to their same-age peers in the TLD group. Conclusion The results of this study support the view that different measures of lexical diversity may be appropriate for different clinical purposes. It is important for clinicians to understand how measures of lexical diversity function in order to make educated choices among measures and ensure appropriate interpretation.

Clausal Density Between Ages 4 and 9 Years for the Edmonton Narrative Norms Instrument: Reference Data and Psychometric Properties

Purpose This study provided reference data and examined psychometric properties for clausal density (CD; i.e., number of clauses per utterance) in children between ages 4 and 9 years from the database of the Edmonton Narrative Norms Instrument (ENNI). Method Participants in the ENNI database included 300 children with typical language (TL) and 77 children with language impairment (LI) between the ages of 4;0 (years;months) and 9;11. Narrative samples were collected using a story generation task, in which children were asked to tell stories based on six picture sequences. CD was computed from the narrative samples. The split-half reliability, concurrent criterion validity, and diagnostic accuracy were evaluated for CD by age. Results CD scores increased significantly between ages 4 and 9 years in children with TL and those with LI. Children with TL produced higher CD scores than those with LI at each age level. In addition, the correlation coefficients for the split-half reliability and concurrent criterion validity of CD scores were all significant at each age level, with the magnitude ranging from small to large. The diagnostic accuracy of CD scores, as revealed by sensitivity, specificity, and likelihood ratios, was poor. Conclusions The finding on diagnostic accuracy did not support the use of CD for identifying children with LI between ages 4 and 9 years. However, given the attested reliability and validity for CD, reference data of CD from the ENNI database can be used for evaluating children's difficulties with complex syntax and monitoring their change over time. Supplemental Material https://doi.org/10.23641/asha.13172129.

Finite Verb Morphology Composite Between Age 4 and Age 9 for the Edmonton Narrative Norms Instrument: Reference Data and Psychometric Properties

Purpose

The purpose of this study was to provide reference data and evaluate the psychometric properties for the finite verb morphology composite (FVMC) measure in children between 4 and 9 years of age from the database of the Edmonton Narrative Norms Instrument (ENNI; Schneider, Dubé, & Hayward, 2005 ).

Method

Participants included 377 children between age 4 and age 9, including 300 children with typical language and 77 children with language impairment (LI). Narrative samples were collected using a story generation task. FVMC scores were computed from the samples. Split-half reliability, concurrent criterion validity, and diagnostic accuracy for FVMC were further evaluated.

Results

Children's performance on FVMC increased significantly between age 4 and age 9 in the typical language and LI groups. Moreover, the correlation coefficients for the split-half reliability and concurrent criterion validity of FVMC were medium to large ( r s ≥ .429, p s < .001) at each age level. The diagnostic accuracy of FVMC was good or acceptable from age 4 to age 7, but it dropped to a poor level at age 8 and age 9.

Conclusion

With the empirical evidence, FVMC is appropriate for identifying children with LI between age 4 and age 7. The reference data of FVMC could also be used for monitoring treatment progress.

Supplemental Material

https://doi.org/10.23641/asha.10073183