1. Audio Systems Survey of Methods for Modelling Sound Propagation in Interactive Virtual Environments Ben Tagger Andriana Machaira

2. Overview Part 1 Audio Systems and Virtual Environments Basics of Real & VR World Acoustics Basics of the Human Hearing Auralization Room Effects

3. Overview Part 2 Some Basic Sound Theory Characteristics of Sound Methods of simulating the propagation of Sound through an Environment Spatialisation Demo

4. The need of audio in VR Additional channel of communication Formation of spatial information Localisation of objects Data-driven sound Simulation of ‘coctail party effect’ Sound enhances the presence in the VR

5. Basics of Real World Acoustics The sound source → Object that emits sound waves The acoustic environment → absorbtion, reflection, refraction and diffraction of sound waves The listener → From the arriving waves the listener extract information about the sound sources and the environment.

6. Basics of VR World Acoustics The auditory actor →Entity emitting sounds through its interface. The auditory space → The environment that has to be modeled. An auditory space object models the geometry of the enclosures in the world. The listener

7. Basics of Human Hearing Interaural intensity difference (IID) → a sound is louder at the ear that it is closer to Interaural time difference (ITD). → a sound will arrive earlier at one ear than the other. Ear pinna → key to accurately localizing sounds in space for wavelengths in the centimeter range or smaller.

8. Basics of Human Hearing acoustic transmission pathway into the ear [2]

9. Head Related Transfer Function HRTF HRTF representation [2]

10. 3D interactive Sound System Sounds are projected → in all three dimensions → in real-time and interactive rates → with the less coloring (tonal changes) introduced by processing

11. Auralization. Auralization is the process of rendering audio data by digital means to achieve a 3D sound space. Principle → Binaural human hearing  Extract information about the location of sound sources.

12. Auralization Processing Pipeline Basic Auralization Pipeline [4]

13. Modeling sound propagation Sound propagation paths from a source (A) to a receiver (R) [4]

14. Part II Overview Some Basic Sound Theory Characteristics of Sound Methods of simulating the propagation of Sound through an Environment Numerical Solutions High Frequency Approximations Perceptually-based Statistical Models Spatialisation Demo

15. Basic Sound Theory “No one can hear you scream in Space.” Hit a Tuning fork Tuning fork vibrates and hits the air molecules next to it These air molecules hit the ones next to them And so on… Doesn’t work in a vacuum.

16. Characteristics of Sound Wavelength Speed Dynamic Range Latency and Update Rate

17. Wavelength Wavelengths range - between 0.02 and 17 meters (20 KHz and 20 Hz respectively) Reflections are largely specular for large flat surfaces (i.e., walls) Diffraction of sound occurs around obstacles of the same size as the wavelength (i.e., tables) Small objects have little effect on the sound field (for all but the highest wavelengths.

18. Speed 343 MSec -1 Far slower than light Propagation delays are perceptible to humans. Sound arrives at the receiver at different times

19. Dynamic Range/ Latency & Update Rate Sensitivity of the Human Ear The effects of late sound reverberation are much more significant than for illumination Timing requirements System latency and update rates can have a significant impact on perceived quality of the environment.

20. Some Approaches Computational methods for simulating the propagation of sound through an environment. Numerical Solutions to Wave Equations High Frequency approximation based on geometric propagation paths Perceptually-based statistical models

21. Numerical Solutions – Finite and Boundary Element Methods

22. Boundary/Finite Element Methods Subdivide space into elements Elements are small compared to a wavelength Each element provides a linear equation Equations are solved with large amounts of Maths The acoustic field is calculated at various points of interest.

23. Mental Image Where am I? Sounds Elements Magic Box of Maths Your Sound

24. Geometric Methods Model acoustic effects with computations based on ray theory. Assume that sound wavelengths are significantly smaller than the size of obstacles. Algorithm finds ray paths along which a sound can travel. Mathematical models are used to approximate filters.

25. Propagation Paths

26. Artificial Reverberation Models

27. Spatialisation Demo

28. References References [1] http://www.acoustics.hut.fi/research/aurilization.html [2] http://www.headwize.com/tech/sibbald_tech.htm [3] http://alumnus.caltech.edu/~franko/thesis/Chapter1.html [4] Survey of Methods for modeling Sound Propagation in interactive Virtual Environment Systems, T. Funkhouser, interactive Virtual Environment Systems, T. Funkhouser, N. Tsingos, J.-M. Jot (2004), accepted for publication in N. Tsingos, J.-M. Jot (2004), accepted for publication in Presence, 2004 Presence, 2004