A two-microphone dual delay-line approach for extraction of a speech sound in the presence of multiple interferers

Neuroethology of sound localization in anurans

Albert Feng pioneered the study of neuroethology of sound localization in anurans by combining behavioral experiments on phonotaxis with detailed investigations of neural processing of sound direction from the periphery to the central nervous system. The main advantage of these studies is that many species of female frogs readily perform phonotaxis towards loudspeakers emitting the species-specific advertisement call. Behavioral studies using synthetic calls can identify which parameters are important for phonotaxis and also quantify localization accuracy. Feng was the first to investigate binaural processing using single-unit recordings in the first two auditory nuclei in the central auditory pathway and later investigated the directional properties of auditory nerve fibers with free-field stimulation. These studies showed not only that the frog ear is inherently directional by virtue of acoustical coupling or crosstalk between the two eardrums, but also confirmed that there are extratympanic pathways that affect directionality in the low-frequency region of the frog's hearing range. Feng's recordings in the midbrain also showed that directional information is enhanced by cross-midline inhibition. An important contribution toward the end of his career involved his participation in neuroethological research with a team of scientists working with frogs that produce ultrasonic calls.

Azimuthal sound localization using coincidence of timing across frequency on a robotic platform

An algorithm for localizing a sound source with two microphones is introduced and used in real-time situations. This algorithm is inspired by biological computation of interaural time difference as occurring in the barn owl and is a modification of the algorithm proposed by Liu et al. [J. Acoust. Soc. Am. 110, 3218-3231 (2001)] in that it creates a three-dimensional map of coincidence location. This eliminates localization artifacts found during tests with the original algorithm. The source direction is found by determining the azimuth at which the minimum of the response in an azimuth-frequency matrix occurs. The system was tested with a pan-tilt unit in real-time in an office environment with signal types ranging from broadband noise to pure tones. Both open loop (pan-tilt unit stationary) and closed loop experiments (pan-tilt unit moving) were conducted. In real world situations, the algorithm performed well for all signal types except pure tones. Subsequent room simulations showed that localization accuracy decreases with decreasing direct-to-reverberant ratio.

Binaural segregation in multisource reverberant environments

In a natural environment, speech signals are degraded by both reverberation and concurrent noise sources. While human listening is robust under these conditions using only two ears, current two-microphone algorithms perform poorly. The psychological process of figure-ground segregation suggests that the target signal is perceived as a foreground while the remaining stimuli are perceived as a background. Accordingly, the goal is to estimate an ideal time-frequency (T-F) binary mask, which selects the target if it is stronger than the interference in a local T-F unit. In this paper, a binaural segregation system that extracts the reverberant target signal from multisource reverberant mixtures by utilizing only the location information of target source is proposed. The proposed system combines target cancellation through adaptive filtering and a binary decision rule to estimate the ideal T-F binary mask. The main observation in this work is that the target attenuation in a T-F unit resulting from adaptive filtering is correlated with the relative strength of target to mixture. A comprehensive evaluation shows that the proposed system results in large SNR gains. In addition, comparisons using SNR as well as automatic speech recognition measures show that this system outperforms standard two-microphone beamforming approaches and a recent binaural processor.

Performance of time- and frequency-domain binaural beamformers based on recorded signals from real rooms

Extraction of a target sound source amidst multiple interfering sound sources is difficult when there are fewer sensors than sources, as is the case for human listeners in the classic cocktail-party situation. This study compares the signal extraction performance of five algorithms using recordings of speech sources made with three different two-microphone arrays in three rooms of varying reverberation time. Test signals, consisting of two to five speech sources, were constructed for each room and array. The signals were processed with each algorithm, and the signal extraction performance was quantified by calculating the signal-to-noise ratio of the output. A frequency-domain minimum-variance distortionless-response beamformer outperformed the time-domain based Frost beamformer and generalized sidelobe canceler for all tests with two or more interfering sound sources, and performed comparably or better than the time-domain algorithms for tests with one interfering sound source. The frequency-domain minimum-variance algorithm offered performance comparable to that of the Peissig-Kollmeier binaural frequency-domain algorithm, but with much less distortion of the target signal. Comparisons were also made to a simple beamformer. In addition, computer simulations illustrate that, when processing speech signals, the chosen implementation of the frequency-domain minimum-variance technique adapts more quickly and accurately than time-domain techniques.

Speech segregation based on sound localization

At a cocktail party, one can selectively attend to a single voice and filter out all the other acoustical interferences. How to simulate this perceptual ability remains a great challenge. This paper describes a novel, supervised learning approach to speech segregation, in which a target speech signal is separated from interfering sounds using spatial localization cues: interaural time differences (ITD) and interaural intensity differences (IID). Motivated by the auditory masking effect, the notion of an "ideal" time-frequency binary mask is suggested, which selects the target if it is stronger than the interference in a local time-frequency (T-F) unit. It is observed that within a narrow frequency band, modifications to the relative strength of the target source with respect to the interference trigger systematic changes for estimated ITD and IID. For a given spatial configuration, this interaction produces characteristic clustering in the binaural feature space. Consequently, pattern classification is performed in order to estimate ideal binary masks. A systematic evaluation in terms of signal-to-noise ratio as well as automatic speech recognition performance shows that the resulting system produces masks very close to ideal binary ones. A quantitative comparison shows that the model yields significant improvement in performance over an existing approach. Furthermore, under certain conditions the model produces large speech intelligibility improvements with normal listeners.