Perceptual Restoration of Temporally Distorted Speech in L1 vs. L2: Local Time Reversal and Modulation Filtering

The common limitations in auditory temporal processing for Mandarin Chinese and Japanese

The present investigation focused on how temporal degradation affected intelligibility in two types of languages, i.e., a tonal language (Mandarin Chinese) and a non-tonal language (Japanese). The temporal resolution of common daily-life sentences spoken by native speakers was systematically degraded with mosaicking (mosaicising), in which the power of original speech in each of regularly spaced time-frequency unit was averaged and temporal fine structure was removed. The results showed very similar patterns of variations in intelligibility for these two languages over a wide range of temporal resolution, implying that temporal degradation crucially affected speech cues other than tonal cues in degraded speech without temporal fine structure. Specifically, the intelligibility of both languages maintained a ceiling up to about the 40-ms segment duration, then the performance gradually declined with increasing segment duration, and reached a floor at about the 150-ms segment duration or longer. The same limitations for the ceiling performance up to 40 ms appeared for the other method of degradation, i.e., local time-reversal, implying that a common temporal processing mechanism was related to the limitations. The general tendency fitted to a dual time-window model of speech processing, in which a short (~ 20-30 ms) and a long (~ 200 ms) time-window run in parallel.

Perceptual restoration of locally time-reversed speech: Non-native listeners' performance in their L2 vs. L1

Nonnative listeners are generally not as good as native listeners in perceptually restoring degraded speech and understand what was being said. The current study investigates how nonnative listeners of English (namely, native Japanese speakers who learned English as a second language) perceptually restore temporally distorted speech in their L2 English as compared with native English listeners (L1 English) reported in Ishida et al. (Cognition, 151, 68–75, 2016), and as compared with the listeners’ native tongue (L1 Japanese). In the experiment, listeners listened to locally time-reversed words and pseudowords in their L2 English and L1 Japanese where every 10, 30, 50, 70, 90, or 110 ms of speech signal was flipped in time—these stimuli contained either many fricatives or stops. The results suggested that the intelligibility of locally time-reversed words and pseudowords deteriorated as the length of reversed segments increased in both listeners’ L2 English and L1 Japanese, while listeners understood locally time-reversed speech more in their L1 Japanese. In addition, lexical context supported perceptual restoration in both listeners’ L1 Japanese and L2 English, while phonemic constituents affected perceptual restoration significantly only in listeners’ L1. On the other hand, locally time-reversed words and pseudowords in L1 Japanese were much more intelligible than those in L1 English reported in Ishida et al. It is possible that the intelligibility of temporally distorted lexical items depends on the structure of basic linguistic units in each language, and the Japanese language might have a unique characteristic because of its CV and V structure.

Phonemic restoration of interrupted locally time-reversed speech : Effects of segment duration and noise levels

Intelligibility of temporally degraded speech was investigated with locally time-reversed speech (LTR) and its interrupted version (ILTR). Control stimuli comprising interrupted speech (I) were also included. Speech stimuli consisted of 200 Japanese meaningful sentences. In interrupted stimuli, speech segments were alternated with either silent gaps or pink noise bursts. The noise bursts had a level of - 10, 0 or + 10 dB relative to the speech level. Segment duration varied from 20 to 160 ms for ILTR sentences, but was fixed at 160 ms for I sentences. At segment durations between 40 and 80 ms, severe reductions in intelligibility were observed for ILTR sentences, compared with LTR sentences. A substantial improvement in intelligibility (30-33%) was observed when 40-ms silent gaps in ILTR were replaced with 0- and + 10-dB noise. Noise with a level of - 10 dB had no effect on the intelligibility. These findings show that the combined effects of interruptions and temporal reversal of speech segments on intelligibility are greater than the sum of each individual effect. The results also support the idea that illusory continuity induced by high-level noise bursts improves the intelligibility of ILTR and I sentences.

Intelligibility of chimeric locally time-reversed speech

The intelligibility of chimeric locally time-reversed speech was investigated. Both (1) the boundary frequency between the temporally degraded band and the non-degraded band and (2) the segment duration were varied. Japanese mora accuracy decreased if the width of the degraded band or the segment duration increased. Nevertheless, the chimeric stimuli were more intelligible than the locally time-reversed controls. The results imply that the auditory system can use both temporally degraded speech information and undamaged speech information over different frequency regions in the processing of the speech signal, if the amplitude envelope in the frequency range of 840-1600 Hz was preserved.

Irrelevant speech effects with locally time-reversed speech: Native vs non-native language

Irrelevant speech is known to interfere with short-term memory of visually presented items. Here, this irrelevant speech effect was studied with a factorial combination of three variables: the participants' native language, the language the irrelevant speech was derived from, and the playback direction of the irrelevant speech. We used locally time-reversed speech as well to disentangle the contributions of local and global integrity. German and Japanese speech was presented to German (n = 79) and Japanese (n = 81) participants while participants were performing a serial-recall task. In both groups, any kind of irrelevant speech impaired recall accuracy as compared to a pink-noise control condition. When the participants' native language was presented, normal speech and locally time-reversed speech with short segment duration, preserving intelligibility, was the most disruptive. Locally time-reversed speech with longer segment durations and normal or locally time-reversed speech played entirely backward, both lacking intelligibility, was less disruptive. When the unfamiliar, incomprehensible signal was presented as irrelevant speech, no significant difference was found between locally time-reversed speech and its globally inverted version, suggesting that the effect of global inversion depends on the familiarity of the language.