Efficient coding of timerelative structure using spikes

1 Introduction

Nonstationary and timerelative acoustic structures such as transients, timing relations among acoustic events, and harmonic periodicities provide essential cues for many types of auditory processing. In sound localization, human subjects can reliably detect interaural time differences as small as 10 µs, which corresponds to a binaural sound source shift of about 1 degree (Blauert, 1997). In comparison, the sampling interval for an audio CD sampled at 44.1 kHz is 22.7 microseconds. Auditory grouping cues, such as common onset and offset, harmonic comodulation, and sound source location, all rely on accurate representation of timing and periodicity (Slaney and Lyon, 1993). Timerelative structure is also crucial for the recognition of consonants and many types of transient, nonstationary sounds. Neurophysiological research in the auditory brainstem of mammals has found cells capable of conveying precise phase information up to 4 kHz or of tracking the quickly varying envelope of a highfrequency sound (Oertel, 1999).The importance of these acoustic cues has long been recognized, but extracting them from natural signals still poses many challenges because the problem is fundamentally illposed. In natural acoustic environments, with multiple sound sources and background noises, acoustic events are not directly observable and must be inferred using numerous ambiguous cues.

Another reason for the difficulty in obtaining these cues is that most approaches to signal representation are blockbased, i.e. the signal is processed piecewise in a series of discrete blocks. Transients and nonstationary periodicities in the signal can be temporally smeared across blocks. Large changes in the representation of an acoustic event can occur depending on the arbitrary alignment of the processing blocks with events in the signal. Signal analysis techniques such as windowing or the choice of the transform can reduce these effects, but it would be preferable if the representation was insensitive to signal shifts.

Shiftinvariance alone, however, is not a sufficient constraint on designing a general sound processing algorithm. Another important constraint is coding efficiency or, equivalently, the ability of the representation to capture underlying structure in the signal. A desirable code should reduce the information rate from the raw signal so that the underlying structures are more directly observable. Signal processing algorithms can be viewed as a method for progressively reducing the information rate until one is left with only the information of interest. We can make a distinction between the observable information rate, i.e. the rate of the observable variables and the intrinsic information rate, or the rate of the underlying structure of interest. In speech, the observable information rate of the waveform samples is about 50,000 bits per second, but the intrinsic rate of the underlying words is only around 200 bits per second (Rabiner and Levinson, 1981). Information reduction can be achieved either by selecting only the desired information (and discarding everything else) or by removing redundancy, e.g., the temporal correlations between samples. This reduces the observable information rate while preserving the intrinsic information.

In this paper, we investigate algorithms for fitting an efficient, shiftinvariant representation to natural sound signals. The outline of the paper is as follows. The next section describes the motivations behind this approach and illustrates some of the shortcomings of current methods. After defining the model for signal representation, we present different algorithms for signal decomposition and contrast their complexity. Next we illustrate the properties of the representation on various types of speech sounds. We then present a measure of coding efficiency and compare these algorithms to traditional methods for signal representation. Finally, we discuss the relevance of the computational issues discussed here to spike coding and signal representation at the auditory nerve.

2 Representing Nonstationary Acoustic Structure

Encoding the acoustic signal is the first step in any algorithm for performing an auditory task. There are numerous approaches to this problem which differ in both their computational complexity and in what aspects of signal structure are extracted. Ultimately, the choice about what the representation encodes depends on the tasks that need to be performed. In the ideal case, the encoding process extracts only that information which is necessary to perform the task and suppresses noise or unrelated information. A “generalist” approach, like that taken by most mammalian auditory systems, requires a representation which is efficient for a wide range of signals. As natural sounds contain both relatively stationary harmonic structure (e.g. animal vocalizations) as well as nonstationary transient structure (e.g. crunching leaves and twigs), this generalist approach requires a code capable of efficiently representing these disparate sound classes (Lewicki, 2002a).Here we seek an auditory representation that is useful for a variety of different tasks.