1. Virtual Museum Ramón Oliva, Miguel Pasenau & Eugeni Casadesús 2012, January 24th Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC)

2. 2012, January 24th / 2 Contents Goal Implementation Results Conclusions

3. 2012, January 24th / 3 Contents Goal Implementation Results Conclusions

4. 2012, January 24th / 4 Goal Virtual Museum: 3 rooms User can move around: – Collision detection Objects exposed, selectable for inspection

5. 2012, January 24th / 5 Environment & tools Cave or Power-Wall: – Limited space – User gestures to move around scene  Kinect VRJuggler: – Targets Cave, Power-Wall and desktop computer as well Kinect interface: – Natural user gestures – NITE + OpenNI + FAAST Bullet: collision detection

6. 2012, January 24th / 6 Used tools gMeshSim ( PFC-EI) GiD 10.1.9b VRJuggler 3.0.0 Bullet 2.78 openNI 1.5.2.23 Nite 1.5.2.21 FAAST 0.10.0

7. 2012, January 24th / 7 Contents Goal Implementation Results Conclusions

8. 2012, January 24th / 8 Museum floor plan 13.964 triangles

9. 2012, January 24th / 9 Objects 1.024 triangles 28.688 triangles 69.451 triangles 111.658 triangles 54.931 triangles 51.925 triangles 51.132 triangles97.425 triangles 99.877 triangles

10. 2012, January 24th / 10 User movements Start & finish navigation: left arm up Forward, backward, left, right: – Segway: lean forward / backward: moves fw / bw – Rotate torso left / right: rotates left / right – Left arm up / down: starts / ends movement Selection – Using wand – Stretching right arm to ‘grasp’ object in front Inspection: – Zoom in / out Help – Raising right hand

11. 2012, January 24th / 11 VRJuggler approach User stays fixed, scene moves Init(): – Initialize devices – Build scene: read ply models: museum walls, pedestals & objects – Create bullet physics: Museum, objects and pedestals containers

12. 2012, January 24th / 12 VRJuggler approach preFrame(): – Update physics – Gets and process kinect and other events data Draw(): – Navigation mode: draw museum and objects – Inspection mode: draw selected object zoomed

13. 2012, January 24th / 13 Actual state Museum, pedestals and objects are rendered Collision detection is enabled User movements are enabled: – Forward and backward, left and right turning – Grasp object for inspection Help mode Inspection mode: – Scale object up and down – Automatic rotation Kinect integration: – Gestures recognition and application responds

14. 2012, January 24th / 14 Actual state Museum, pedestals and objects are rendered Collision detection is enabled User movements are enabled: – Forward and backward, left and right turning – Grasp object for inspection Help mode Inspection mode: – Scale object up and down – Automatic rotation Kinect integration: – Gestures recognition and application responds

15. 2012, January 24th / 15 VRJuggler issues No x64 alone build & install Only mixed x32 and x64 Linux build and install Using x32 build: – https://github.com/rpavlik/vrjuggler-windows-binaries https://github.com/rpavlik/vrjuggler-windows-binaries Units in feet and inches, not meters User centred, scene moving No restrictions or definition of movements No event filtering possible No seamless platform transition

16. 2012, January 24th / 16 Bullet + VRJuggler issues User centred, scene moving Translations  whole scene is translated the hard way Rotations  whole scene is rotated the hard way Collisions: – User: forward ray – Scene: bbox and cylinders moving with scene – Object selection = collision wand ray - object

17. 2012, January 24th / 17 Kinect issues Unstable FAAST 0.10 Using.jconf: lot of noise Gesture recognition using FAAST: – Send keys / events to window with focus – Depends on sensor position / orientation Some delay

18. 2012, January 24th / 18 Contents Goal Implementation Results Conclusions

19. 2012, January 24th / 19 Results: navigation mode

20. 2012, January 24th / 20 Results: inspection mode

21. 2012, January 24th / 21 Results: help

22. 2012, January 24th / 22 Results Source code at: http://code.google.com/p/museum-navigator/

23. 2012, January 24th / 23 Contents Goal Implementation Results Conclusions

24. 2012, January 24th / 24 Conclusions Developing a RV application: – Hard to do – Still a juggling art VRJuggler: – Hard to build and install: documentation so-so – Only x32 is known to work Bullet: easy to install and to start with Kinect: difficult to incorporate to VRJuggler: – Gesture analysis and interpretation – Complicated device and proxies structure – “Easy” using FAAST patch

25. 2012, January 24th / 25 Future work Solve problems with power-wall + kinect: – Inverse camera parameters – Inverse movements Use transformations instead of modifying coordinates Avoid noise depending on sensor position and orientation Interactive museum floor plan editor

26. 2012, January 24th / 26 References Bullet documentation VRJuggler documentation FAAST documentation http://www.keyboardmods.com/2010/12/howto-kinect-openninite- skeleton.html http://www.keyboardmods.com/2010/12/howto-kinect-openninite- skeleton.html https://github.com/avin2/SensorKinect http://www.cs.unc.edu/Research/vrpn/ http://projects.ict.usc.edu/mxr/faast/

27. Virtual Museum Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC) Ramón Oliva, Miguel Pasenau & Eugeni Casadesús 2012, January 24th