42nd IEEE Conference on Decision and Control, Dec 9-12, 2003 A Biomimetic Apparatus for Sound Source Localization (A System Identification Problem from Lord Rayleigh) Amir A. Handzel¹, Sean B. Andersson², Martha Gebremichael³ and P. S. Krishnaprasad University of Maryland, College Park Department of Electrical and Computer Engineering & Institute for Systems Research 42nd IEEE Conference on Decision and Control, Dec 9-12, 2003 ¹ Beyond Genomics Inc. ² Division of Engineering and Applied Sciences, Harvard ³ Microsoft Research

Lord Rayleigh Around 1876, Rayleigh was creating the theory of sound. He was interested in the physics of sound propagation and the perception of sound. 1842-1919 “The problem of the whispering gallery”, Philosophical Magazine, pp 1001-1004, 1910

Sound following

Outline Sound Source Localization Model-free approaches A model and an identification problem Ambiguity and determining elevation References Demonstration Thanks to Shihab Shamma for inspiration. Vinay Shah did recent measurements and demos.

References A. A. Handzel and P. S. Krishnaprasad. “Bio-mimetic sound source localization”, IEEE Sensors Journal, 2(6), 607-617, 2002 A. A. Handzel, S. B. Andersson, M. Gebremichael, and P. S. Krishnaprasad. “A bio-mimetic apparatus for sound source localization”, Proc. 42nd IEEE Conf. on Decision and Control, December, 2003. S. B. Andersson, A.A. Handzel, V. Shah and P. S. Krishnaprasad. “Robot phonotaxis with dynamic sound source localization”, submitted (November 2003).

Barn Owl and Robot Can we capture the barn owl’s auditory acuity in a binaural robot? 2 degrees ~ microseconds resolution

Sound Localization in Nature Localization: spatial aspect of auditory sense Sensory organ arrangement: Vision -- spatial “topographic” Audition -- tonotopic, transduction to sound pressure in frequency bands special computation required, performed in dedicated brainstem circuits and cortex

Acoustic Cues for Localization Binaural/Inter-aural Level/Intensity Difference (ILD/IID) Time/Phase Difference (ITD/IPD) On-set difference/precedence effect Monaural: spectral-directional filtering by Pinna, mostly for elevation

Place Theory (L. Jeffress) J. Comp. Physiol. & Psychol Jeffress model and schematic of brainstem auditory circuits for detection of interaural time (ITD) differences; from Carr & Amagai (1996)

Stereausis (S. Shamma et. al.) J. Acoust. Soc. Am. (1989) 86:989-1006 AVCN Ipsi- center contra- lateral lateral Yj Characteristic frequency Ckk +1 Ckk Ckk -1 Xi Cij Characteristic frequency Ipsi-lateral cochlea AVCN Sound Contara- lateral cochlea or

Initial Motivation The above approaches are static, and do not take account of motion. But psychophysical experiments inspired by early suggestions of Hans Wallach (1940), show active horizontal head rotations improve localization, break inter-aural symmetry, and thus provide information on elevation (Perret & Noble 1997, Wightman & Kistler 1999). Formulating this as an identification problem leads to insight and algorithms. We will show how (head) movement can be helpful is resolving ambiguities. Applications arise in guiding a robot towards an acoustic source. First, for a model, we turn to Rayleigh.

Lord Rayleigh and Binaural Perception ILD and ITD both needed for azimuth. (What about elevation?) Rayleigh set up the problem of sound propagation around an acoustically hard sphere. He introduced head-related transfer functions (HRTF). HRTF computed from sound pressure field generated at a point on the sphere by a point source located at (θ, φ). For a sphere, one has to solve the Helmholtz equation. See section 385 of The Theory of Sound Edition

Coordinate Systems - Azimuthal q - Polar f - Elevation f - Azimuth Microphones at poles on horizontal plane

Pressure field proportional to Static Solution Pressure field proportional to Does not depend on azimuthal angle (f) Head Related Transfer Function (HRTF) Numerical (e.g. FMP), and empirical methods for non-spherical heads

A representation of HRTF information

Some Related Work on HRTFs Blauert, J. (1997). Spatial Hearing (Revised Edition) (MIT Press, Cambridge, MA). C. P. Brown and R. O. Duda (1998). “A structural model for binaural sound synthesis”, IEEE Trans. Speech and Audio Proc., 6(5):476-488. Duda, R. O. (1995). “Estimating azimuth and elevation from the interaural head related transfer function,” in Binaural and Spatial Hearing, R. Gilkey and T. Anderson, Eds. (Lawrence Erlbaum Associates, Hillsdale, N.J.) R. O. Duda

Feature Plane (cylinder) and Signatures ILD & IPD constitute an intermediate computational space for localization. At each frequency a source gives rise to a point in the ILD-IPD plane (cylinder). A (broadband) point source imprints a signature curve on this feature plane (cylinder) according to its location.

Symmetry of Static Localization Sound pressure and resulting inter-aural functions depend only on polar angle; azimuth invariant -- SO(2) symmetry Sources on same circle of directions have identical signatures. Hence the localization confusion. Introduce distance measures.

Symmetry and Rotations - Azimuthal q - Polar f - Elevation f - Azimuth

Breaking the Symmetry Azimuthal invariance, but polar rotations do change the localization functions. Key mathematical step: infinitesimal rotations act as derivative operator -- generate vector fields on signatures. Derivatives ‘modulated’ by Cos(f) -- thus elevation extracted from horizontal rotation! (Head movement helps)

Experimental Results Broad band source - sum of pure tones 43 Hz – 11 KHz in steps of 43Hz. Passed through anti-aliasing filter and sampled at 22KHz. Knowles FG-3329 microphones used on head of 22.6 cm maximum diameter. To determine ILD and IPD, each 512 point segment (23 ms) of data was passed through an FFT. Measured IPD and ILD were smoothed by a nine-point moving average. This yields empirically determined (discrete) signature curves on ILD-IPD space. Localization computations based on minimizing distance functions. Implementation of this step on mobile robot achieved as a table lookup.

Pumpkin head side-view (left) and top view (right). Minimum diameter 19 cm and maximum diameter 22.6 cm.

Plot on left displays smoothed ILD against theoretical ILD for source at 17.5 degrees in horizontal plane. Plot on right shows smoothed IPD against theoretical IPD for same source.

Plot on left shows distance functions for source at 15 deg and 17.5 deg. Plot on right shows distance functions for source at 72.5 deg and 75 deg.

Performance plots for IPD-ILD algorithm (left) and traditional ITD algorithm (right)

Implications and Applications Psychophysics: auditory displays, auditory component of virtual environments and hearing aids. Bio-mimetic active robot head References: A. A. Handzel and P. S. Krishnaprasad. “Bio-mimetic sound source localization”, IEEE Sensors Journal, 2(6), 607-617, 2002 A. A. Handzel, S. B. Andersson, M. Gebremichael, and P. S. Krishnaprasad. “A bio-mimetic apparatus for sound source localization”, Proc. 42nd IEEE Conf. on Decision and Control, Dec. 2003. S. B. Andersson, A.A. Handzel, V. Shah and P. S. Krishnaprasad. “Robot phonotaxis with dynamic sound source localization”, submitted (November 2003).

Front Back Demo Without front-back distinction With front-back distinction