Audio for Telepresence and Virtual Reality Colin S. Harrison

Objective is to enhance sense of hearing for Telepresence and integrate with Virtual Environment Laboratory Replicate feeling of being there Replicate feeling of being there Hear a sound – turn to look at it Hear a sound – turn to look at it Replicate human hearing Replicate human hearing Localise azimuth, elevation, range, properly externalised Localise azimuth, elevation, range, properly externalised Use audio to augment video for improved sensation of presence Use audio to augment video for improved sensation of presence

Human 3D Spatial Hearing Differences in Time (ITD) Differences in Time (ITD) Differences in Intensity (IID) Differences in Intensity (IID) Movable Head Movable Head Pinnae Filtering (Head Related Transfer Function) Pinnae Filtering (Head Related Transfer Function)

Implementation Binaural sound / Simulated head Binaural sound / Simulated head (HRTF) for Headphones Integrate with Telepresence Projects Integrate with Telepresence Projects B-Format for loudspeakers B-Format for loudspeakers Integrate with VE aspects: ABACUS suite Integrate with VE aspects: ABACUS suite Lake DSP for Headtracked Binaural Simulation and Loudspeaker reproduction Lake DSP for Headtracked Binaural Simulation and Loudspeaker reproduction

Originality Binaural -Two eared technique Binaural -Two eared technique Ear/sensor assembly Ear/sensor assembly Mechatronic head Mechatronic head World first systemWorld first system with physical binaural sensors with physical binaural sensors B-Format approach for Telepresence ( new Application) B-Format approach for Telepresence ( new Application) No other UK VE site has installed realistic sound

Virtual Environment Integration Add realistic sound to VE presentations of architectural projects Add realistic sound to VE presentations of architectural projects Change sound as position and room type changes Change sound as position and room type changes Link to existing Virtual Environment Laboratory Link to existing Virtual Environment Laboratory Publications Conference Paper: (Mechatronics 98)Conference Paper: (Mechatronics 98) Journal Paper:(by invitation Mechatronics)Journal Paper:(by invitation Mechatronics)

Industrial Relevance Hostile human environments Hostile human environments Nuclear Chemical -Chernobyl Nuclear Chemical -Chernobyl Surveillance (Big Brother hears you too!) Surveillance (Big Brother hears you too!) Entertainment Entertainment Architectural walkthroughs Architectural walkthroughs Remote Military Intervention Remote Military Intervention

And Finally Telepresence: seamlessly represent senses: consider a Turing Test Telepresence: seamlessly represent senses: consider a Turing Test Accurate sound triggers sense memories Accurate sound triggers sense memories