1. Audio Measurements Su, Amit, David, Muthu

2. Outline  Microphone  Introduction to Win CE  Audio data collecting with iPAQ  Audio data analysis

3. Introduction to microphones  What is microphone?  Microphone is a transducer -- an energy converter.  It senses acoustic energy (sound) and translates it into equivalent electrical energy.  How it works?  Dynamic Microphones  Good  reliability, need little maintenance  fairly good signal-to- noise ratio  Bad  no "tailored" response

4. How it works?  Condenser Microphones  Good: high-quality performance  Ability to respond to transient sounds  extended high-frequency response  weigh less smaller  Bad  sensitive to mechanical noise  Other Types of Microphones  Ribbon microphone  Phantom Power

5. How to choose microphone  Microphone specifications  Decibel (dB) scale  Measures how sensitive the microphone is.  Frequency Response  “Bandwidth“ of microphone  Multiple frequency response  “Bandwidths“ for sound coming from different directions  On-axis response  Response to sound coming directly to the microphone  Off-axis responses  Response to sound coming from all angles

6. Microphone specifications  Diffuse field response  Response to sound coming from reflections  Polar Response  how certain frequencies are reproduced when they enter the microphone from a circle  Equivalent noise level  noise from microphone itself (good if <15db)  Sensitivity  what voltage a microphone will produce at a certain sound pressure level  SPL handling capability (Sound pressure level)  Where a certain Total Harmonic Distortion (THD) occurs.  Where the signal from the microphone will clip, that is the waveforms will become squares.

7. Outline  Microphone  Introduction to Win CE  Audio data collecting with iPAQ  Audio data analysis

8. Windows CE Architecture  Windows CE Design Principles  Small Memory  Modular Approach  Processor Portability  Win32 Compatibility  Comprehensive Development Tool Support  Connectivity  Real Time Processing  Win32 Programming Model  Utilises a large subset of the Win32 API (No Win16 support)  Supports MFC, VC and VB (eMbedded)

9. Windows CE Architecture OEM Hardware Embedded Shell Applications WIN32 APIs COREDLL, WINSOCK, OLE, COMMCTRL, COMMDLG, WININET, TAPI Windows CE Shell Services Remote Connectivity KernelLibrary IrDA GWES Device Manager FileManager TCP/IP OALBootloaderDrivers Device drivers File drivers Microsoft OEM ISV, OEM

10. Developer Issues  Windows CE Memory Model  Protected Address Space  Virtual Memory  Memory Allocation  Stack  Heap  Virtual Memory (VirtualAlloc)  Memory mapped files  Processes and Threads  No process priority classes  Threads with the same priority run in a round-robin fashion  Number of threads only limited by available memory Reserved for system 64KB Guard Memory Mapped Files Process Slot 1(32MB) Process Slot 0 (32MB) Process slot 32 (32MB) 4GB 3GB 2GB 1GB 0GB...... Process Slot 2 (32MB)......

11. Developer Issues  File System  No Concept of Current Directory  No Support for Overlapped I/O  Support for Installable and Remote File Systems  Power Issues  Porting Win32 Applications  Unicode  GDI differences  User interface issues – e.g. no mouse  Tool Support  eMbedded Visual C++  eMbedded VB  Visual Studio.NET

12. Outline  Microphone  Introduction to Win CE  Audio data collecting with iPAQ  Audio data analysis

13. My own experience  Life cycle on data analysis  Background  Difficulties  Achievement  Demo Clean Data Analysis Data Build Application Collect Data

14. Audio data  What is audio data  To human: something you can hear  To computer: digital signals  What is audio data features  Energy  zero-crossing  Spectrum  ……

15. Where audio data being used?  Engineering Acoustics  Acoustic signal processing  musical sounds synthesis and composition  Physical Acoustics  Ultrasonics and infrasonics  Propagation of sound through the atmosphere, fluids, and fluid- filled materials  Psychological and Physiological Acoustics  Speech Recognition and Generation  Physiology and biophysics of the ear, the auditory nerve, and higher neural centers  Others  Acoustical Oceanography  Architectural Acoustics

16. Data collecting procedure  Tools used in our data collecting  iPAQ build in mirophone  Microsoft embedded C++  How?  On iPAQ  Record, compress, send  On server  Receive, unzip, concat

17. Difficulties  Which recorder is better?  Windows build in recorder control vs. self-developed wav recorder  Why choosing self-developed wav recorder?  Guessing …

18. Measure accuracy  Channels – one or two data stream  Mono  Stereo  Bit per sample – how good each sample is  8 bit  16 bit  Sample rate – how many samples are taken each second?  8.0 kHz (telephone quality)  11.025 kHz  22.05 kHz ( FM radio quality)  44.1 kHz ( CD quality)  File size  Channel * Bit/sample * Sample rate * sample time

19. Procedure  Record  Prepare  Open a connection with the device using this handle  Allocate a buffer for incoming data  Reading data  Write to wave file  Compress/Uncompress  Standard zip/unzip  Send/Receive  Sockets similar to ftp

20. Achievements  Let us do the demo now…

21. Future Improvements  Better headset  Better Compression  More efficient algorithm?  Online zipping  Make data streaming  Weakness  Each file length is limited by iPAQ memory  Total recording depends on wireless link  Your own file format  What if wireless link is broken?

22. Outline  Microphone  Introduction to Win CE  Audio data collecting with iPAQ  Audio data analysis

23. Data Cleaning and Analysis  What is noise?  From textbook  Sound - the occurrence of an audible event  Noise – nonperiodic sound  To us  Sound – signal data we are interested in  Noise – signal data that is useless to us  How to remove noise?  Example 1- data are mixed. pick up certain people’s voice while he is talking with a group.  Example 2 – data are sparse. Is there any cell phone rings during a 3 hours meeting?

24. Data Processing  Given voice samples, what can we get from it?  Volumn  Picth  Spectrum  …  What can we do with it?

25. Audio data from Telcordia

26. What is pattern inside?

27. Future Work  On-line processing  Server side  Client side - Fat sensor?  Fusing network

28. Future Work  Data Annotation?

29. Applications  Location detector  Scenario 1: Prof. Muthu is sleeping on the train, but he does not worry about missing New Brunswick…  Smart filter  Scenario 2: Prof. Muthu preparing his lecture notes on the train. And David will call him around that time. He does not want to be interrupted except that call…

30. A little test for fun  Given voice samples from David, Amit and Su. Can you tell them apart?

31. A little hint

32. Result  1- Amit  2 – David  3 - Su