1. Intelligent and Noise- Robust Interfaces for MEMS Acoustic Sensors: Smart Microphone

2. UNIVERSITY OF MARYLAND Electrical and Computer Engineering & Psychology Departments Baras, Horiuchi, Krishnaprasad, Moss, Shamma THE JOHNS HOPKINS UNIVERSITY Electrical and Computer Engineering Department Andreou, Cauwenberghs, Etienne-Cummings UNIVERSITY OF SIDNEY Electrical Engineering Department van Schaik SIGNAL SYSTEMS CORPORATION Riddle, Murray COLLABORATIONS Institute for Neuroinformatics, ETH Army Research Laboratory

5. Acoustic and Ultrasonic Transducers (AGA, REC) Prototype and Evaluate Various Types of MEMS Microphones/Speakers Custom Microphones/SpeakersCommercial Transducers

6. Integrated MEMS Acoustic and Ultrasonic Arrays (AGA, REC) Prototype and Evaluate Various Types of MEMS Microphone/Speaker Arrays 2D Piezo Arrays Ceramics and Polymers 2D MEMS Arrays Capacitive Micro-Membranes

7. Vision: A small, low power microphone interface for acoustic sensors that reduces turbulence and vibration induced noise on military platforms such as battlefield robotics Polyurethane foam windscreen Mounting plate 1/2” 1” Primary Microphone Port Connector Secondary Sensor Port for Wind Sensing Microphones Preamps aVLSI noise reduction circuitry

8. MEMS/VLSI Integration and Prototyping (REC,AGA) Develop Integrated Processing Electronics for Transducers Integrated Transducers and ElectronicsTransduction Electronics

9. Approach: –Utilize multi-channel adaptive filtering modules based on aVLSI biomimetic technology Analog filter banks with Independent Component Analysis (ICA) and Least Mean Squares (LMS) adaptation –Incorporate low noise preamps; acoustic and vibration sensors –Develop specification and prototypes –Demonstrate in acoustic duct and installed on unmanned land vehicle Multi-Resolution Adaptive Filter Low noise acoustic signals Acoustic Sensors Wind Noise Sensor Noise and Vibration Sensors

10. Cochlear Frequency Analysis We will design a new silicon cochlea with the following features: –Increased robustness due to 2D design –Integrated Inner Hair Cell Model –Reproducible settings of the parameters Magnitude ResponsePhase Response

11. Stochastic Resonance Exploit stochastic resonance (noise-induced enhancement of spectral power amplification SPA) in conjunction with auditory-inspired (e.g. cochlear) sensor signal processing architectures + External World Band-pass filter Controlled noise generator circuit MEMS heater K Threshold detector

12. Adaptive Filtering and Blind Source Separation (GC, AGA) Dynamic ICA Array Processor Adaptive Cell Static and Dynamic ICA (Independent Component Analysis) Adaptive Noise and Wind Cancellation without Need for Isolated Reference

14. Cochlear Feature Extraction (AGA, GC) cochlea 15 channels hair cells auditory nerve AM FM Time T ZC BM in BM out Energy (single channel) Neuromorphic implementation with asynchronous “spiking” outputs

16. The second filter models the multiscale processing of the signal that happens in the auditory cortex A Ripple Analysis Model, using a ripple filter bank, acts on the output of the inner ear to give multiscale spectra of the sound timbre (Wavelet Transform) Multiresolution Preprocessor: Auditory Filtering Upward Moving Downward Moving Slow Rate Coarse Scale Slow Rate Fine Scale Fast Rate Fine Scale Fast Rate Coarse Scale Fast Rate Fine Scale Multiresolution cortical filter outputs Fast Rate Coarse Scale Slow Rate Fine Scale Slow Rate Coarse Scale

17. Auditory frequency scale 20406080100120 70 60 50 40 30 20 10 0 Example of multi-resolution representation from cortical module Auditory Processing of Vehicle Signals: Cortex

18. Wavelet TSVQ Applied to Acoustic Vehicle Classification Objective: a prototype vehicle acoustic signal classification system with low classification error and short search time Biologically motivated feature extraction models: cochlear filter banks and A1-cortical wavelet transform Vector Quantization (VQ) based classification algorithm. Including learning VQ (LVQ) and tree structured VQ (TSVQ) Feature extraction system Classification Result Acoustic Recording Preprocessing Peripheral auditory processing model VQ based Classification Algorithm Cortical processing model Algorithm Flowchart

19. Acoustic Transient Time-Frequency Analysis (GC, w/ APL) Segmenter audio in Continuous Wavelet Filterbank 32 ( freq ) X 64 ( time ) Time-Frequency Template Correlator ATP... 16 (12 used) Digital Postprocessor 16 (templ.) 32 (freq) “Shelf” (class 10)“Tub” (class 11) Models “Ripple” Dynamics of Cells Recorded in Auditory Cortex (Shamma)

20. Cochlear filters NAVLVp NM/NL (ITD processing) (ILD processing) ICcICx Cochlear filters NAVLVp NM/NL (ITD processing) (ILD processing) ICcICx BINAURAL LOCALIZATION ABL

21. Stereausis: A Biologically Plausible Binaural Network. A binaural sound localization system will be developed using 2 silicon cochleas and an aVLSI implementation for ILD and ITD detection.

24. BATS

25. Steerable Range Gauging and Echolocation (REC) Develop Ranging Signal Processing Algorithms

26. Massively Parallel Kernel Learning “Machines” (GC) Parallel vector quantizer Kernel “machines” subsume LVQ, RBF and SVM classifiers Locally adaptive, distributed memory Scalable and modular Factor 100-10,000 more efficient than CPU or DSP

27. Integrate-and-Fire Address-Event VLSI Neural Networks (GC & AGA) Integrate-and-Fire Array Address-Event Transceiver Chip Scalable, multi-chip architecture for “neural” computations Address-event routing circuit provides for arbitrary interconnection topologies Analog-valued synaptic weights are implemented by probabilistically transmitting address-events

28. Address-Event Asynchronous Communication and Computation (AGA, REC, GC) The multi-chip modular, scalable approach to system integration

29. New design: An AER cochlea chip Currently in fabrication 128 output channels Both for sonar and audio New silicon process (0.35um minimum feature size versus 2.0um) AER makes inter-chip communication possible. AER allows manipulation of output such that projective fields are readily implemented.

31. 1-D Address-Event Transceiver with Diode-Capacitors Integrators (AGA) 1-D Address-Event Transceiver Chip Address Event (AE) transceiver circuit is a modular element for future multi- dimensional communication between neuromorphic chips. Input AE data is processed by the sigmoid function of the nonlinear diode-capacitor integrators. The data is retransmitted using the AE protocol with an arbitrated queueing communication system.

32. Impact: –Enable effective acoustic surveillance for Future Combat Systems –Increase by 20 dB the turbulence induced noise rejection of acoustic sensors relative to passive treatments of the same size, using active noise control –Increase by 20 dB the platform noise rejection of acoustic sensors over existing methods Demo III Experimental Unmanned Vehicle

33. The Robots Microphones Sonar sensors Touch sensors Wireless camera Speakers