1. Spatial Sound in X3D/X3DOM Athanasios G. Malamos Multimedia Lab Dept. of Informatics Engineering TEI of Crete, Heraklion, Crete, Greece (amalamos@ie.teicrete.gr) 18/06/2015

2. FUTURE The initial question is, does X3D have to interface to the W3C API or just leave engineers to use it in the application level….. ?? (provocation!! ) A Hint…… The presence of virtual reality hardware interfaces (ex. VR Headsets) increases the impact of spatial sound….. 19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY AUGUST 8-10 – VANCOUVER, CANADA

3. X3D v3.4 and sound In V3.4 could be included some neighbor to v3.4 features of W3C API ………………. And perhaps in v.4 to reform the X3D sound components more drastically considering also OpenAL, FMOD, Creative's EAX, Microsoft's XACT Audio, etc…. 19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY AUGUST 8-10 – VANCOUVER, CANADA

4. we decided to start with a test-bed where we could make all the compatibility tests http://medialab.teicrete.gr/minipages/x3domAudio/ index.html

5. Overview  Sound Nodes in X3D  Sound Nodes in X3DOM Framework  Test bed demo  Test bed methodology 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

6. Sound Nodes in X3D (1) Abstract types:  X3DSoundNode (the base for all sound nodes)  X3DSoundSourceNode (this derive node types that can emit audio data) Node reference:  AudioClip is derived from the X3DSoundSourceNode. It specifies audio data that can be referenced by the Sound node.  Sound is derived from the X3DSoundNode. It is designed for the description of the X3D scene sounds. Specifically, it determines both the location and the behavior of the sound. 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

7. Sound Nodes in X3D (2) Inheritance diagram of X3D AudioClip Node 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

8. Sound Nodes in X3D (3) Inheritance diagram of X3D Sound Node 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

9. Sound Nodes in X3DOM (1)  X3DSoundNode, X3DSoundSourceNode, AudioClip and Sound nodes have been registered in X3DOM, but without the most of the properties of the respective X3D nodes.  Specifically, an X3DOM scene could include an audio file for playing, without distinguished spatial characteristics.  The attribute “SFBool enabled” has been extra added to X3DOM in comparison with X3D, which specifies whether the clip is enabled or not. 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

10. Sound Nodes in X3DOM (2) AudioClip Node in X3D – X3DOM 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

11. Sound Nodes in X3DOM (3) Sound Node in X3D – X3DOM 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

12. Test Bed Methodology We investigated the combination of X3D sound nodes and the structure of Web Audio API, in order to upgrade the X3DOM framework with audio spatial abilities for a 3D web scene. Investigated Method 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

13. Why Web Audio API  It is open source and be supported from the most browsers.  It allows the web developers to manage audio using JavaScript.  It offers high-level sound abilities as filters, delay lines, amplifiers and spatial effects (such as panning).  Audio channels can have 3D distribution according to the position, speed or direction of the viewer and the sound source. 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

14. Analysis of test bed methodology The attributes of AudioClip (X3D node) and Sound (X3D node) have been re-introduced in X3DOM with the use of :  Two enhanced custom nodes AudioSound AudioSource  An Added Value set of Web Audio API nodes 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

15. Test Bed at a glance AudioSource: X3DOM:X3DSoundSource SFBool loop SFNode metadata X3D: X3DSoundSource NodeSFString description MFString url SFTime pauseTime SFFloat pitch SFTime resumeTime SFTime startTime SFTime stopTime AudioSource: X3DOM:X3DSoundSource SFBool loop SFNode metadata X3D: X3DSoundSource NodeSFString description MFString url SFTime pauseTime SFFloat pitch SFTime resumeTime SFTime startTime SFTime stopTime Test bed 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

16. X3D to test bed matching 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

17. X3D to Test bed matching (2) 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece

18. All layers overview of the matching (1) 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece Custom AudioSound Node

19. All layers overview of the matching (2) 20th International Conference on 3D Web Technology June 18-21 –Heraklion,Crete,Greece Custom AudioSound Node

20. Future work To test more technologies and get the best performance To cooperate more with members of the community that are interested to extend and enhance X3D audio api.

21. End Of Presentation THANK YOU!!! End Of Presentation THANK YOU!!!