1. Boston University--Harvard University--University of Illinois--University of Maryland Distributed Sensor Fields and Uncertainty: Bio-mimetic Methods for Acoustic Source Localization P. S. Krishnaprasad University of Maryland, College Park Department of Electrical and Computer Engineering & Institute for Systems Research http://www.isr.umd.edu/~krishna ------------ Center for Communicating Networked Control Systems ------------ ARO-MURI01 Review, Boston University October 20-21, 2003

2. Boston University--Harvard University--University of Illinois--University of Maryland Sensor Field - Motivation Dynamic Sound Source Localization Outline Problems and Models Technical Approach References Demonstration This is joint work with Amir Handzel, Sean Andersson, and Martha Gebremichael. Also thanks to Shihab Shamma for inspiration. Vinay Shah did recent measurements and demos.

3. Boston University--Harvard University--University of Illinois--University of Maryland Sensor Field A sensor field is heterogeneous (acoustic, seismic, thermal, RF, magnetic, optical…) and often mobile on various platforms (e.g. UGV, UAV…), which are networked and in contact with key gateway nodes - (Eicke/Lavery (1999); Srour (discussions 1998, 2000); NRC (2000) NMAB-495; Emmerman (discussions 2000, 2001); Scanlon/Young (discussions 2003))

4. Boston University--Harvard University--University of Illinois--University of Maryland Photo: courtesy of Michael Scanlon, ARL

5. Boston University--Harvard University--University of Illinois--University of Maryland Control over noisy, limited bandwidth, communication channels Intelligent Servosystems Laboratory (ISL)

6. Boston University--Harvard University--University of Illinois--University of Maryland Dynamic Sound Source Localization - or why we need to move our head? Biologically inspired algorithms

7. Boston University--Harvard University--University of Illinois--University of Maryland Barn Owl and Robot Can we capture the barn owl’s auditory acuity in a binaural robot?

8. Boston University--Harvard University--University of Illinois--University of Maryland Sound Localization in Nature Localization: spatial aspect of auditory sense Sensory organ arrangement: Vision -- spatial “topographic” Audition -- tonotopic, transduction to sound pressure in frequency bands special computation required, performed in dedicated brainstem circuits and cortex

9. Boston University--Harvard University--University of Illinois--University of Maryland Acoustic Cues for Localization Binaural/Inter-aural: Level/Intensity Difference (ILD) Time/Phase Difference (IPD) On-set difference/precedence effect Monaural: spectral-directional filtering by Pinna, mostly for elevation

10. Boston University--Harvard University--University of Illinois--University of Maryland Place Theory (L. Jeffress) J. Comp. Physiol. & Psychol., (1948) 41:35-39 Jeffress model and schematic of brainstem auditory circuits for detection of interaural time (ITD) differences; from Carr & Amagai (1996)

11. Boston University--Harvard University--University of Illinois--University of Maryland Stereausis (S. Shamma et. al.) J. Acoust. Soc. Am. (1989) 86:989-1006 Ipsi-lateral cochlea Characteristic frequency Sound Characteristic frequency Contara- lateral cochlea AVCN Ipsi- center contra- lateral C kk +1 C kk C kk -1 YjYj C ij XiXi or

12. Boston University--Harvard University--University of Illinois--University of Maryland -45 deg (left) Stereausis shifts from the main diagonal according to the source location. 45 deg (right)0 deg center Incoming sound: a pure tone Stereausis scheme (courtesy Shihab Shamma, UMd)

13. Boston University--Harvard University--University of Illinois--University of Maryland Lord Rayleigh and Binaural Perception ILD and ITD both needed for azimuth (the concept of HRTF). What about elevation? 1842-1919 See section 385 of The Theory of Sound 1945 edition

14. Boston University--Harvard University--University of Illinois--University of Maryland Initial Motivation All the above are static, but real life usually dynamic, and psychophysical experiments show active horizontal head rotations improve localization, break inter-aural symmetry, and thus provide information on elevation (Perret & Noble 1997, Wightman & Kistler 1999). One can explain the above theoretically. Understanding such effects would matter in guiding robots towards acoustic source.

15. Boston University--Harvard University--University of Illinois--University of Maryland Coordinate Systems  zimuthal  Polar  Elevation  zimuth Microphones at poles on horizontal plane

16. Boston University--Harvard University--University of Illinois--University of Maryland Static Solution Pressure field proportional to Does not depend on azimuthal angle (  Head Related Transfer Function (HRTF) Numerical (e.g. FMP), and empirical methods for non-spherical heads

17. Boston University--Harvard University--University of Illinois--University of Maryland

18. Feature Plane (cylinder) and Signatures ILD & IPD constitute an intermediate computational space for localization At each frequency a source gives rise to a point in the ILD-IPD plane A (broadband) point source imprints a signature curve on this feature plane (cylinder) according to its location

19. Boston University--Harvard University--University of Illinois--University of Maryland

20. Symmetry of Static Localization Sound pressure and resulting inter-aural functions depend only on polar angle; azimuth invariant -- SO(2) symmetry Sources on same circle of directions have identical signatures. Hence the localization confusion

21. Boston University--Harvard University--University of Illinois--University of Maryland

23. Symmetry and Rotations  zimuthal  Polar  Elevation  zimuth

24. Boston University--Harvard University--University of Illinois--University of Maryland Breaking the Symmetry Azimuthal invariance, but polar rotations do change the localization functions Key mathematical step: infinitesimal rotations act as derivative operator -- generate vector fields on signatures. Derivatives ‘modulated’ by Cos(  -- thus elevation extracted from horizontal rotation!

25. Boston University--Harvard University--University of Illinois--University of Maryland

28. Experimental Results Broad band source - sum of pure tones 43 Hz – 11 KHz in steps of 43Hz. Passed through anti-aliasing filter and sampled at 22KHz. Knowles FG-3329 microphones used on head of 22.6 cm maximum diameter. To determine ILD and IPD, each 512 point segment (23 ms) of data was passed through an FFT. Measured IPD and ILD were smoothed by a nine-point moving average. This yields empirically determined (discrete) signature curves on ILD-IPD space. Localization computations based on minimizing distance functions. Implementation of this step on mobile robot achieved as a table lookup.

29. Boston University--Harvard University--University of Illinois--University of Maryland Pumpkin head side-view (left) and top view (right). Minimum diameter 19 cm and maximum diameter 22.6 cm.

30. Boston University--Harvard University--University of Illinois--University of Maryland Plot on left displays smoothed ILD against theoretical ILD for source at 17.5 degrees in horizontal plane. Plot on right shows smoothed IPD against theoretical IPD for same source.

31. Boston University--Harvard University--University of Illinois--University of Maryland Plot on left shows distance functions for source at 15 deg and 17.5 deg. Plot on right shows distance functions for source at 72.5 deg and 75 deg.

32. Boston University--Harvard University--University of Illinois--University of Maryland Performance plots for IPD-ILD algorithm (left) and traditional ITD Algorithm (right)

33. Boston University--Harvard University--University of Illinois--University of Maryland New experiments in summer 2003 yielded raw data for further investigation of HRTF dependence on elevation. Front-back ambiguity resolution via dynamic IPD-ILD algorithm implemented on robot. (See demo.) Plans to use soldier-helmet from ARL. Photo: Courtesy of Michael Scanlon, ARL

34. Boston University--Harvard University--University of Illinois--University of Maryland Accomplishments First theoretical analysis and derivation of localization under rotation (no pinnae) Showed analytically that information on elevation can be extracted from active horizontal rotation (in particular front- back) binaurally, with omni-directional sensors. Demonstration in acoustically cluttered environment

35. Boston University--Harvard University--University of Illinois--University of Maryland Implications and Applications Psychophysics: auditory displays, auditory component of virtual environments and hearing aids. Bio-mimetic active robot head References: A. A. Handzel and P. S. Krishnaprasad, “Bio-mimetic Sound Source Localization”, IEEE Sensors Journal, 2(6), 607-617, 2002. A. A. Handzel, S. B. Andersson, M. Gebremichael, and P. S. Krishnaprasad. “A Bio-mimetic Apparatus for Sound Source Localization”, Proc. 42nd IEEE Conf. on Decision and Control, Dec. 2003 (in press).

36. Boston University--Harvard University--University of Illinois--University of Maryland Sound following

37. Boston University--Harvard University--University of Illinois--University of Maryland Front Back Demo Without front-back distinctionWith front-back distinction