1. Christoph AnthesRoland Landertshamer Developing VR Applications with the inVRs Framework at IEEE VR’10 Part III – Going Immersive

2. 20.03.10inVRs - Tutorial II - Going Immersive2 Overview TutorialTutorial –Overview –Step 1 – Wrapping Functionality –Step 2 – Immersive Displays –Step 3 – Using the Input Interface –Step 4 – Working With Avatars –Step 5 – Coordinate Systems OutlookOutlook AcknowledgementsAcknowledgements

3. 20.03.10inVRs - Tutorial II - Going Immersive3 Tutorial Structure As the previous part of the tutorial this section is based on many copy and paste operations in order to safe timeAs the previous part of the tutorial this section is based on many copy and paste operations in order to safe time Four basic file classes are needed for this tutorialFour basic file classes are needed for this tutorial –GoingImmersive.cpp Contains the application parts from where the development can startContains the application parts from where the development can start –Configuration Files Stored in the config subdirectoryStored in the config subdirectory Have to be altered through snippetsHave to be altered through snippets –CodeFile - Snippets Contain code snippets which should be copied into the GoingImmersive.cpp source fileContain code snippets which should be copied into the GoingImmersive.cpp source file Can be found under the subdirectory snippetsCan be found under the subdirectory snippets –ConfigFile - Snippets Contain XML configuration which has to be copied in the actual configuration filesContain XML configuration which has to be copied in the actual configuration files Can be found under the subdirectory snippetsCan be found under the subdirectory snippets Open the file GoingImmersive.cpp with the editor of your choiceOpen the file GoingImmersive.cpp with the editor of your choice

4. 20.03.10inVRs - Tutorial II - Going Immersive4 Where wrapper classes are introduced to speed up application development. Chapter 1 Wrapping Functionality

5. 20.03.10inVRs - Tutorial II - Going Immersive5 Wrapping Functionality Initially in the Medieval Town Tutorial a fair bit of set-up operations have been performed in the codeInitially in the Medieval Town Tutorial a fair bit of set-up operations have been performed in the code –They can be wrapped up by the help of so called application bases –To use an application base an own class has to be derived from an ApplicationBase The developer has the possibility to derive from one of the already provided application bases in order to create an own application baseThe developer has the possibility to derive from one of the already provided application bases in order to create an own application base Or the developer can use the basic functionality of the already derived application bases (currently only implemented for OpenSG)Or the developer can use the basic functionality of the already derived application bases (currently only implemented for OpenSG) The basic class ApplicationBase is an abstract classThe basic class ApplicationBase is an abstract class It provides the following functions which have to be implemented:It provides the following functions which have to be implemented: –bool preInit(const CommandLineArgumentWrapper& args) Called right after the constructors as a first initialization stepCalled right after the constructors as a first initialization step –void initCoreComponentCallback(CoreComponents comp) Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification

6. 20.03.10inVRs - Tutorial II - Going Immersive6 Wrapping Functionality –void initInputInterfaceCallback(ModuleInterface* moduleInterface) Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification –void initOutputInterfaceCallback(ModuleInterface* moduleInterface) Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification –void initModuleCallback(ModuleInterface* module) Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification –void bool disableAutomaticModuleUpdate() Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification –void manualModuleUpdate(float dt) Called at core component initialization, it has to be overwritten for component notificationCalled at core component initialization, it has to be overwritten for component notification

7. 20.03.10inVRs - Tutorial II - Going Immersive7 Wrapping Functionality Other functions have to be implemented by the application developer:Other functions have to be implemented by the application developer: –std::string getConfigFile(const CommandLineArgumentWrapper& args) Must return url to main configuration file (e.g. general.xml )Must return url to main configuration file (e.g. general.xml ) –bool init(const CommandLineArgumentWrapper& args) Initializes application, called after initialization of inVRs componentsInitializes application, called after initialization of inVRs components –void run() Main application loop, ApplicationBase::globalUpdate() must be called every iterationMain application loop, ApplicationBase::globalUpdate() must be called every iteration –void display(float dt) Called by ApplicationBase::globalUpdate(), used to update application, dt contains elapsed time in ms since last method call.Called by ApplicationBase::globalUpdate(), used to update application, dt contains elapsed time in ms since last method call. –void cleanup() System cleanupSystem cleanup

8. 20.03.10inVRs - Tutorial II - Going Immersive8 Wrapping Functionality Other methods which have to be called by the application developer:Other methods which have to be called by the application developer: –bool start(int argc, char** argv) Starts the application, should be called out of main method when application instance was createdStarts the application, should be called out of main method when application instance was created –void globalUpdate() Updates inVRs components forwards update-command to display method. Must be called out of main application loopUpdates inVRs components forwards update-command to display method. Must be called out of main application loop –void globalCleanup() Cleans up inVRs components, must be called by application before finishing, forwards call to cleanup methodCleans up inVRs components, must be called by application before finishing, forwards call to cleanup method

9. 20.03.10inVRs - Tutorial II - Going Immersive9 Wrapping Functionality Besides the provided methods several member variables can be used when inheriting from the ApplicationBase:Besides the provided methods several member variables can be used when inheriting from the ApplicationBase: –SceneGraphInterface* sceneGraphInterface Points to the used SceneGraphInterface, if no SceneGraphInterface is used the pointer value is NULLPoints to the used SceneGraphInterface, if no SceneGraphInterface is used the pointer value is NULL –ControllerManagerInterface* controllerManager Pointer to the ControllerManager, NULL if no ControllerManager is usedPointer to the ControllerManager, NULL if no ControllerManager is used –NetworkInterface* networkModule Points to the Network module, NULL if no Network module is usedPoints to the Network module, NULL if no Network module is used –NavigationInterface* navigationModule Pointer to the Navigation Module, NULL if no ControllerManager is usedPointer to the Navigation Module, NULL if no ControllerManager is used –InteractionInterface* interactionModule Points to the Interaction module, NULL f no Interaction module is loadedPoints to the Interaction module, NULL f no Interaction module is loaded –User* localUser This variable points to the User object for the local user.This variable points to the User object for the local user. –CameraTransformation* activeCamera This variable is a pointer to the camera transformation object of the local camera.This variable is a pointer to the camera transformation object of the local camera.

10. 20.03.10inVRs - Tutorial II - Going Immersive10 Wrapping Functionality Writing an inVRs application using the ApplicationBaseWriting an inVRs application using the ApplicationBase –Class declaration: Inherit from OpenSGApplicationBase Declaring class membersDeclaring class members –Define variable for url to configuration file, initialize it in constructor GoingImmersive.cpp GoingImmersive.cpp

11. 20.03.10inVRs - Tutorial II - Going Immersive11 Wrapping Functionality Configure path to Plugins in general.xmlConfigure path to Plugins in general.xml Implement destructorImplement destructor –Call OpenSGApplicationBase::globalCleanup() in order to free all memory reserved by inVRs components general.xml GoingImmersive.cpp

12. 20.03.10inVRs - Tutorial II - Going Immersive12 Wrapping Functionality Implement getConfigFile()Implement getConfigFile() –Must return url to main inVRs configuration file (e.g. general.xml) –Default config file can already be stored in member variable –Url can be passed via command line Use CommandLineArgumentWrapperUse CommandLineArgumentWrapper Implement initialize() methodImplement initialize() method –Called automatically after all inVRs components were initialized –Set root node for scene graph as well as initial user transformation –Use variables sceneGraphInterface and localUser, which are inherited from ApplicationBase GoingImmersive.cpp

13. 20.03.10inVRs - Tutorial II - Going Immersive13 Wrapping Functionality Implement initialize() methodImplement initialize() method GoingImmersive.cpp

14. 20.03.10inVRs - Tutorial II - Going Immersive14 Wrapping Functionality Empty display and cleanup methodEmpty display and cleanup method –Don't have to update or cleanup anything Create instance of application class in main() methodCreate instance of application class in main() method GoingImmersive.cpp GoingImmersive.cpp

15. 20.03.10inVRs - Tutorial II - Going Immersive15 Wrapping Functionality

16. 20.03.10inVRs - Tutorial II - Going Immersive16 Where the use of inVRs in combination with immersive displays is introduced. Chapter 2 Immersive Displays

17. 20.03.10inVRs - Tutorial II - Going Immersive17 Immersive Displays Many types of Immersive Displays are availableMany types of Immersive Displays are available inVRs focuses on stereoscopic multi-display installationsinVRs focuses on stereoscopic multi-display installations –CAVE –HMD –Curved Screens –Powerwalls inVRs uses the OpenSG multi-display functionality for setting up immersive displaysinVRs uses the OpenSG multi-display functionality for setting up immersive displays –Thus in general a variety of other displays is possible Multi-display functionality abstracted and handled by an external tool, the CAVE Scene ManagerMulti-display functionality abstracted and handled by an external tool, the CAVE Scene Manager Other multi-modal output not covered yetOther multi-modal output not covered yet –E.g. haptic displays, motion platforms, etc.

18. 20.03.10inVRs - Tutorial II - Going Immersive18 Immersive Displays Setup examples from the JKUSetup examples from the JKU

19. 20.03.10inVRs - Tutorial II - Going Immersive19 Immersive Displays Using the CAVE Scene ManagerUsing the CAVE Scene Manager –Consists of 4 Header files: –OSGCAVESceneManager.h Contains main functionality and user API for scene manager, use similar like SimpleSceneManagerContains main functionality and user API for scene manager, use similar like SimpleSceneManager –OSGCAVEConfig.h Responsible for parsing, loading and setting scene manager configurationResponsible for parsing, loading and setting scene manager configuration –OSGCAVEWall.h Deals with setup of wall displaysDeals with setup of wall displays –appctrl.h Contains functionality for starting and shutting down display serversContains functionality for starting and shutting down display servers Configuration very similar to the CAVELib configurationConfiguration very similar to the CAVELib configuration

20. 20.03.10inVRs - Tutorial II - Going Immersive20 Immersive Displays Using the CAVE Scene ManagerUsing the CAVE Scene Manager –Keywords WallsWalls –Used to define list of display panes WallDisplayWallDisplay –Detailed wall configuration: name, display, resolution, offset ProjectionDataProjectionData –Alignment of different projection panes DisplayModeDisplayMode –Monoscopic (mono) or stereoscopic (stereo) OriginOrigin –Location of coordinate origin in physical space InterocularDistanceInterocularDistance –Eye seperation CAVEWidth and CAVEHeightCAVEWidth and CAVEHeight –Width and height in case of CAVE-like displays

21. 20.03.10inVRs - Tutorial II - Going Immersive21 Immersive Displays Using the CAVE Scene ManagerUsing the CAVE Scene Manager –Keywords UnitsUnits –Meters, centimeters or foot CAVE Scene Manager has to be enabled in the application baseCAVE Scene Manager has to be enabled in the application base Additional rendering servers are neededAdditional rendering servers are needed –server-mono Provides monoscopic renderingProvides monoscopic rendering –server-stereo Provides stereoscopic renderingProvides stereoscopic rendering Rendering servers are automatically started by CAVE Scene ManagerRendering servers are automatically started by CAVE Scene Manager –Typically one per display

22. 20.03.10inVRs - Tutorial II - Going Immersive22 Immersive Displays Using the CAVE Scene ManagerUsing the CAVE Scene Manager –Has to be enabled in the application base by setting useCSM to true –Define CSM configuration file –Define automatic startup of render servers –Define relation between world coordinates and units in real world –Set background image for control window –Set path to CAVE Scene Manager configuration –Set path for loading images Snippet 1-1  general.xml Snippet 1-2  general.xml Snippet 1-3  general.xml

23. 20.03.10inVRs - Tutorial II - Going Immersive23 Immersive Displays Using the CAVE Scene ManagerUsing the CAVE Scene Manager –Either set path for server-mono or copy the binary into working directory Typically located at inVRs/external/CAVESceneManager- 1.0/buildTypically located at inVRs/external/CAVESceneManager- 1.0/build –Application will show two windows: Window for render serverWindow for render server Control windowControl window –Set background image for window color: Build and start applicationBuild and start application –Console output of CAVE Scene Manager should be displayed –If render servers could be started Snippet 1-1

24. 12.06.09inVRs - Tutorial II - Going Immersive24 Immersive Displays

25. 20.03.10inVRs - Tutorial II - Going Immersive25 Where the implementation of the use of own input devices is explained. Chapter 3 Using the Input Interface

26. 20.03.10inVRs - Tutorial II - Going Immersive26 Using the Input Interface Vast amount of input devices is available in VRVast amount of input devices is available in VR inVRs tries to be totally technology agnosticinVRs tries to be totally technology agnostic –Abstraction of input devices –Whole input data reduced to ButtonsButtons AxesAxes SensorsSensors

27. 20.03.10inVRs - Tutorial II - Going Immersive27 Using the Input Interface Configuring your input devicesConfiguring your input devices –Abstract Controller configured inside InputInterface –Mapping between devices and abstract Controller

28. 20.03.10inVRs - Tutorial II - Going Immersive28 Using the Input Interface Accessing an Abstract ControllerAccessing an Abstract Controller int getButtonValue(int idx)int getButtonValue(int idx) –With this function call the current value of the button can be requested. On the button additional callbacks can be registered which are trigged depending on the button state. float getAxisValue(int idx)float getAxisValue(int idx) –Returns the value of the axis with the given index. Positive and negative floating point values are valid. SensorData getSensorValue(int idx)SensorData getSensorValue(int idx) –Provides a SensorData object containing transformation data about the sensor with the given index. int getNumberOfButtons()int getNumberOfButtons() –Returns the amount of registered buttons on the abstract controller. int getNumberOfAxes()int getNumberOfAxes() –Returns the amount of registered axes on the abstract controller. int getNumberOfSensors()int getNumberOfSensors() –Returns the amount of registered sensors on the abstract controller.

29. 20.03.10inVRs - Tutorial II - Going Immersive29 Using the Input Interface Writing own Devices for the Abstract ControllerWriting own Devices for the Abstract Controller New device inherits from InputDeviceNew device inherits from InputDevice Methods that have to be implemented:Methods that have to be implemented: –int getNumberOfButtons() This method must return the number of buttons which are provided by this deviceThis method must return the number of buttons which are provided by this device –int getNumberOfAxes() Must return the amount of available axes provided by the deviceMust return the amount of available axes provided by the device –int getNumberOfSensors() Must return the amount of available sensors provided by the deviceMust return the amount of available sensors provided by the device –int getButtonValue(int idx) Must return the value of the button with the passed index. If device does not provide button with this index it must return 0Must return the value of the button with the passed index. If device does not provide button with this index it must return 0 –float getAxisValue(int idx) Must return the value of the axis with the passed index. If the device does not provide an axis with this index it must return 0.Must return the value of the axis with the passed index. If the device does not provide an axis with this index it must return 0.

30. 20.03.10inVRs - Tutorial II - Going Immersive30 Using the Input Interface Writing own Devices for the Abstract ControllerWriting own Devices for the Abstract Controller Methods that have to be implemented:Methods that have to be implemented: –SensorData getSensorValue(int idx) Must return value of sensor (translation and orientation) with passed index. If the device does not provide this sensor, IdentitySensorData must be returnedMust return value of sensor (translation and orientation) with passed index. If the device does not provide this sensor, IdentitySensorData must be returned –void update() Called by Controller class once a frame to update value of input device.Called by Controller class once a frame to update value of input device. Methods that can be called:Methods that can be called: –void acquireControllerLock() Call if values for input device are updated outside the update() method or from another thread.Call if values for input device are updated outside the update() method or from another thread. –void releaseControllerLock() Release lock after acquireControllerLock()Release lock after acquireControllerLock() –void sendButtonChangeNotification(int buttonIndex, int newButtonValue ) Called when state of button changesCalled when state of button changes

31. 20.03.10inVRs - Tutorial II - Going Immersive31 Using the Input Interface Writing own Devices for the Abstract ControllerWriting own Devices for the Abstract Controller –VrpnDevice provided (tools/libraries/VRPNDevice) –VrpnDeviceFactory Used during loading ControllerManager configurationUsed during loading ControllerManager configuration Creates VrpnDeviceCreates VrpnDevice Provides single methodProvides single method create(std::string className, ArgumentVector* args)create(std::string className, ArgumentVector* args) –Creates class with className 'VrpnDevice' and parameters args parsed form configuration file or NULL if another className is passed.

32. 20.03.10inVRs - Tutorial II - Going Immersive32 Using the Input Interface Interconnect own device with applicationInterconnect own device with application –ControllerManager must be aware of new device –Factories must be registered Include headersInclude headers Ifdef statements needed to avoid use of specific VRPN or trackD datatypesIfdef statements needed to avoid use of specific VRPN or trackD datatypes Caution: Only insert snippet if VRPN device is availableCaution: Only insert snippet if VRPN device is available Snippet 2-1

33. 20.03.10inVRs - Tutorial II - Going Immersive33 Using the Input Interface Interconnect own device with applicationInterconnect own device with application –Register device at ControllerManager inside initInputInterfaceCallback() –This method is called before ControllerManager is configured to allow creation of device instance as soon as configuration is loaded –Caution: only insert this snippet if you have a VRPN device you want to use Snippet 2-2

34. 20.03.10inVRs - Tutorial II - Going Immersive34 Using the Input Interface Interconnect own device with applicationInterconnect own device with application –ControllerManager can now create TrackdDevices and VrpnDevices –Configure ControllerManager –Define VrpnController Provided in VrpnController.xmlProvided in VrpnController.xml Consists of single device of type VrpnDeviceConsists of single device of type VrpnDevice Provides 3 buttons, 2 axes, 2 sensorsProvides 3 buttons, 2 axes, 2 sensors –Change ControllerManager configuration file in InputInterface –Caution: only insert this snippet if you have a VRPN device you want to use –Alter Navigation configuration to use Controller effectively –Caution: only insert this snippet if you have a VRPN device you want to use –After starting the application navigation is performed with VrpnDevice Snippet 2-1  inputinterface.xml Snippet 2-2  modules.xml

35. 20.03.10inVRs - Tutorial II - Going Immersive35 Where animated avatars represent the user. Chapter 4 Working With Avatars

36. In Medieval Town Tutorial static avatars were usedIn Medieval Town Tutorial static avatars were used –Implemented in class SimpleAvatar Other types of avatars availableOther types of avatars available –Provided by external package Avatara Developed by Helmut and Martin GarstenauerDeveloped by Helmut and Martin Garstenauer –Make additional use of gathered tracking data –Provide visual approximation of actual user pose –External modeling tool needed for creating avatar –Scripts and tools for Avatara package for exporting avatars exist for 3D Studio Max3D Studio Max MAYAMAYA BlenderBlender Detailed information on modeling and using avatars is provide in the Avatara manualDetailed information on modeling and using avatars is provide in the Avatara manual Avatars can be used as well without inVRs with OpenGL or OpenSGAvatars can be used as well without inVRs with OpenGL or OpenSG 20.03.10inVRs - Tutorial II - Going Immersive36

37. 20.03.10inVRs - Tutorial II - Going Immersive37 Working With Avatars Modeling AvatarsModeling Avatars –Use mesh skinning –Geometry interconnected with bones of avatar –Possible to define animation sequences –Gives access to head bone, spine bone, hand position/orientation

38. 20.03.10inVRs - Tutorial II - Going Immersive38 Working With Avatars Using AvataraUsing Avatara –Offers 3 different functionalities: Starting, stopping of animation phasesStarting, stopping of animation phases Moving head boneMoving head bone Setting hand position and orientationSetting hand position and orientation –Configuration file for Avatara avatar undead.xml

39. 20.03.10inVRs - Tutorial II - Going Immersive39 Working With Avatars Integrating an Avatara avatarIntegrating an Avatara avatar –WorldDatabase needs AvataraAvatarFactory to create AvataraAvatar –Include header containing factory –Register factory at WorldDatabase in initCoreComponetCallback() –Avatar configuration is referenced by UserDatabase although the World Database is responsible for loading the avatar –Checks for callback for the UserDatabase initialization –Define configuration for avatar in UserDatabase configuration Snippet 3-1 Snippet 3-2 Snippet 3-1  userDatabase.xml

40. 20.03.10inVRs - Tutorial II - Going Immersive40 Working With Avatars Integrating an Avatara avatarIntegrating an Avatara avatar –Update avatar transformation Define 2 variablesDefine 2 variables –Pointer to avatar –Defining initial transformation –Initialize these variables in constructor –Obtain pointer to avatar from localUser in initialize() Snippet 3-3 Snippet 3-4 Snippet 3-5

41. 20.03.10inVRs - Tutorial II - Going Immersive41 Working With Avatars Integrating Avatara avatarIntegrating Avatara avatar –Update transformation of avatar –Request transformation of user relative to tracking system center –Default tracking system has at least one sensor for head transformation, one for hand –This does not provide correct position of user relative to tracking system center –UserTransformationModel calculates this position –Default: HeadPositionUserTransformationModel Subtracts height value from head transformationSubtracts height value from head transformation –Update transformation once per frame Snippet 3-6 Snippet 3-7

42. 20.03.10inVRs - Tutorial II - Going Immersive42 Working With Avatars Test avatar without tracking systemTest avatar without tracking system –Use GlutSensorEmulatorDevice –Simulates tracking system –ControllerManagerConfiguration: MouseKeybSensorController.xml GlutMouseDeviceGlutMouseDevice GlutKeyboardDeviceGlutKeyboardDevice GlutSensorEmulatorDeviceGlutSensorEmulatorDevice –Simulates 2 sensors –Switch between normal navigation mode and sensor emulation with keyboard button 1 –In sensor emulation mode switch between sensors by pressing left mouse button –Switch between manipulation of translation and rotation with middle mouse button –Switch between manipulation along X and Y axis by pressing right mouse button

43. 12.06.09inVRs - Tutorial II - Going Immersive43 Working With Avatars

44. 20.03.10inVRs - Tutorial II - Going Immersive44 Where an introduction into and a visualization of inVRs' coordinate systems is provided. Chapter 5 Coordinate Systems

45. 20.03.10inVRs - Tutorial II - Going Immersive45 Coordinate Systems inVRs coordinate systems are similar to OpenGLinVRs coordinate systems are similar to OpenGL –X-axis: point to the right –Y-axis: points to top –Z-axis: towards the observer out of the screen User CoordinatesUser Coordinates 1.) Origin of the VE 2.) Navigated transformation 3.) User transformation 4.) Hand transformation 5.) Head transformation

46. 20.03.10inVRs - Tutorial II - Going Immersive46 Coordinate Systems User CoordinatesUser Coordinates –Navigated transformation Result of navigation process provided by navigation moduleResult of navigation process provided by navigation module Avatar is placed at origin of this coordinate system if no other tracking information providedAvatar is placed at origin of this coordinate system if no other tracking information provided Considered the origin of the VEConsidered the origin of the VE –User transformation When tracking system available, tracked position added to navigated position to obtain avatar transformationWhen tracking system available, tracked position added to navigated position to obtain avatar transformation If no tracking system head tracking data set to identity matrixIf no tracking system head tracking data set to identity matrix Can be represented either as:Can be represented either as: –Tracked User Transformation »Relative to center of tracking system –World User Transformation »Relative to origin of VE »worldUserTrans = navigatedTrans ∗ trackedUserTrans

47. 20.03.10inVRs - Tutorial II - Going Immersive47 Coordinate Systems User CoordinatesUser Coordinates –Hand transformation Controller index 1Controller index 1 Relative to tracking system centerRelative to tracking system center –Transformation is provided by hand sensor Relative to user transformationRelative to user transformation –userHandTrans = trackedUserTrans ⁻ ¹ ∗ trackedHandTrans Relative to origin of VERelative to origin of VE –worldHandTrans = navigatedTrans ∗ trackedHandTrans –Head transformation Controller index 0Controller index 0 Relative to tracking system centerRelative to tracking system center –Transformation is provided by head sensor Relative to user transformationRelative to user transformation –userHeadTrans = trackedUserTrans ⁻ ¹ ∗ trackedHeadTrans Realtive to origin of VERealtive to origin of VE –worldHeadTrans = navigatedTrans ∗ trackedHeadTrans

48. 20.03.10inVRs - Tutorial II - Going Immersive48 Coordinate Systems User CoordinatesUser Coordinates –Sensor transformation Transformation for any additional sensorTransformation for any additional sensor Relative to userRelative to user –userSensorTrans = trackedUserTrans ⁻ ¹ ∗ trackedSensorTrans Relative to worldRelative to world –worldSensorTrans = navigatedTrans ∗ trackedSensorTrans –Cursor transformation Relative to origin of virtual worldRelative to origin of virtual world Different models provided by inVRsDifferent models provided by inVRs –Camera transformation Corresponds to navigated transformation of user by defaultCorresponds to navigated transformation of user by default Calculated and constantly updated in TransformationManagerCalculated and constantly updated in TransformationManager

49. 20.03.10inVRs - Tutorial II - Going Immersive49 Coordinate Systems Visualizing TransformationsVisualizing Transformations –Visualized using 3D models in form of coordinate system Tracked user transformationTracked user transformation Tracked sensor transformation for each sensorTracked sensor transformation for each sensor –Number of entities to create corresponds to number of sensors provided by Controller –Store number of sensors Snippet 4-1

50. 20.03.10inVRs - Tutorial II - Going Immersive50 Coordinate Systems Visualizing TransformationsVisualizing Transformations –Obtain value of numberOfSensors in initialize() –Request controller form ControllerManager –Store number of sensors of controller –Create entities for each sensor by calling WorldDatabase::createEntity() First parameter ID of EntityType, defined in entities.xmlFirst parameter ID of EntityType, defined in entities.xml Second parameter ID of EnvironmentSecond parameter ID of Environment Snippet 4-2

51. 20.03.10inVRs - Tutorial II - Going Immersive51 Coordinate Systems Visualizing TransformationsVisualizing Transformations –Update transformation of coordinate system entities –Method called updateCoordinateSystems() –Obtain pointer to EntityType from WorldDatabase –Request all instances (coordinate system entities) from EntityType –Set transformation of entity representing tracked user transformation to transformation obtained from local User and add position offset to platform center –Iterate over available sensors Update tracked sensor transformationsUpdate tracked sensor transformations Request transformation form local UserRequest transformation form local User Add to platform centerAdd to platform center

52. 20.03.10inVRs - Tutorial II - Going Immersive52 Coordinate Systems Visualizing TransformationsVisualizing Transformations Snippet 4-3

53. 20.03.10inVRs - Tutorial II - Going Immersive53 Coordinate Systems Visualizing TransformationsVisualizing Transformations –Call updateCoordinateSystems() continuously –Add call in update() Snippet 4-4

54. 12.06.09inVRs - Tutorial II - Going Immersive54 Coordinate Systems

55. 20.03.10inVRs - Tutorial II - Going Immersive55 Outlook

56. 20.03.10inVRs - Tutorial II - Going Immersive56 Outlook We have learned how toWe have learned how to –Use the ApplicationBase in order to wrap functionality for faster application development –Use the configuration of the CAVE Scene Manger to ease access to OpenSG multi-display functionality –Interconnect arbitrary (own) input devices with inVRs –Make use of articulated avatars –Work with the different coordinate systems

57. 20.03.10inVRs - Tutorial II - Going Immersive57 Outlook Variety of additional tools are available (some of the internal)Variety of additional tools are available (some of the internal) –Modules Physics support (2D and 3D Physics)Physics support (2D and 3D Physics) Animation and camera pathAnimation and camera path –External Tools Collaborative editorCollaborative editor –Scene Graph Specific Enhancements (OpenSG) Avatars (with export from 3D Max, MAYA, and Blender)Avatars (with export from 3D Max, MAYA, and Blender) CAVE Scene ManagerCAVE Scene Manager Particle WrapperParticle Wrapper SkyboxSkybox 3D Menu System3D Menu System Coming soonComing soon –inVRs port to OpenSG 2 (April - thanks to Georg Stevenson) –AR input and output (Spring) –Variety of example projects (Summer)

58. 20.03.10inVRs - Tutorial II - Going Immersive58 Outlook Novel conceptsNovel concepts –Clear distinction in the area of communication patterns –Events and transformations are handled independently –Transformation management allows for concurrent object manipulation –Composeable and clearly defined interaction and navigation –Automatic data distribution mechanism –Full abstraction from graphics, input, network, etc.; the developer should be able to focus on the application logic –Development and integration of networked physics module (Roland Landertshamer) Does not provide yetDoes not provide yet –Scripting support –Logic support

59. 20.03.10inVRs - Tutorial II - Going Immersive59 Acknowledgements ContributorsContributors –Architecture and Main Development Christoph Anthes, Helmut Bressler, Roland Landertshamer, Marina Lenger –Tools Helmut Garstenauer, Martin Garstenauer, Adrian Haffegee, Marlene Hochrieser, Roland Hopferwieser, Robert Owen, Stephan Reiter, Thomas Weberndorfer, Christian Wressnegger, Johannes Zarl

60. Questions ?

61. 20.03.10inVRs - Tutorial II - Going Immersive61 Bibliography Christoph Anthes A Collaborative Interaction Framework for Networked Virtual Environments. PhD Thesis at Institute of Graphics and Parallel Processing at JKU Linz, Austria, August 2009.Christoph Anthes A Collaborative Interaction Framework for Networked Virtual Environments. PhD Thesis at Institute of Graphics and Parallel Processing at JKU Linz, Austria, August 2009. Christoph Anthes, Philipp Aumayr, Clemens Birklbauer, Roland Hackl, Marlene Hochrieser, Roland Hopferwieser, Simon Opelt, Christoph Payrhuber, Mika Satomi, Stefan Simmer, Georg Stevenson, Roland Landertshamer, Bernhard Lehner, Marina Lenger, Martin Lenzelbauer, Clemens Mock, and Alexander Wilhelm. Space Trash – Development of a Networked Immersive Virtual Reality Installation. Technical Report at Institute of Graphics and Parallel Processing, Johannes Kepler University Linz, June 2009.Christoph Anthes, Philipp Aumayr, Clemens Birklbauer, Roland Hackl, Marlene Hochrieser, Roland Hopferwieser, Simon Opelt, Christoph Payrhuber, Mika Satomi, Stefan Simmer, Georg Stevenson, Roland Landertshamer, Bernhard Lehner, Marina Lenger, Martin Lenzelbauer, Clemens Mock, and Alexander Wilhelm. Space Trash – Development of a Networked Immersive Virtual Reality Installation. Technical Report at Institute of Graphics and Parallel Processing, Johannes Kepler University Linz, June 2009. Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Space Trash – An Interactive Networked Virtual Reality Installation. In Proceedings of the 11th Virtual Reality International Conference (VRIC ’09), pages 107-118, Laval, France, April 2009.Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Space Trash – An Interactive Networked Virtual Reality Installation. In Proceedings of the 11th Virtual Reality International Conference (VRIC ’09), pages 107-118, Laval, France, April 2009. Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Design and Setup of Space Trash – A Collaborative Virtual Reality Installation using Physical Interfaces. Installation description in Proceedings of the 11th Virtual Reality International Conference (VRIC ’09) ReVolution, pages 355-362, Laval, France, April 2009.Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Design and Setup of Space Trash – A Collaborative Virtual Reality Installation using Physical Interfaces. Installation description in Proceedings of the 11th Virtual Reality International Conference (VRIC ’09) ReVolution, pages 355-362, Laval, France, April 2009.

62. 20.03.10inVRs - Tutorial II - Going Immersive62 Bibliography Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Developing VR Applications for the Grid. In 1st International Workshop on Real–Time Online Interactive Applications (ROIA) on the GRID at the 14th International Euro-Par Conference European Conference on Parallel and Distributed Computing (Euro-Par ’08), pages 343–351, Las Palmas de Gran Canaria, Spain, August 2008.Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Developing VR Applications for the Grid. In 1st International Workshop on Real–Time Online Interactive Applications (ROIA) on the GRID at the 14th International Euro-Par Conference European Conference on Parallel and Distributed Computing (Euro-Par ’08), pages 343–351, Las Palmas de Gran Canaria, Spain, August 2008. Eva Pajorová, Ladislav Hluchý, and Christoph Anthes. 3D Geovisualization Service for Grid-oriented applications of Natural Disasters. Poster in the 16th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG '08), Plzen, Czech Republic, February 2008.Eva Pajorová, Ladislav Hluchý, and Christoph Anthes. 3D Geovisualization Service for Grid-oriented applications of Natural Disasters. Poster in the 16th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG '08), Plzen, Czech Republic, February 2008. Christoph Anthes, Alexander Wilhelm, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Net’O’Drom – An Example for the Development of Networked Immersive VR Applications. In International Conference on Computational Science (ICCS '07), pages 752-759, Beijing, China, May 2007.Christoph Anthes, Alexander Wilhelm, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Net’O’Drom – An Example for the Development of Networked Immersive VR Applications. In International Conference on Computational Science (ICCS '07), pages 752-759, Beijing, China, May 2007. Christoph Anthes, Roland Landertshamer, and Jens Volkert. Physically-Based Interaction for Networked Virtual Environments. In International Conference on Computational Science (ICCS '07), pages 776-783, Beijing, China, May 2007.Christoph Anthes, Roland Landertshamer, and Jens Volkert. Physically-Based Interaction for Networked Virtual Environments. In International Conference on Computational Science (ICCS '07), pages 776-783, Beijing, China, May 2007.

63. 20.03.10inVRs - Tutorial II - Going Immersive63 Bibliography Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Managing Transformations and Events in Networked Virtual Environments. In ACM Multimedia Modeling (MMM '07), pages 722-729, Singapore, January 2007.Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert. Managing Transformations and Events in Networked Virtual Environments. In ACM Multimedia Modeling (MMM '07), pages 722-729, Singapore, January 2007. Helmut Bressler, Roland Landertshamer, Christoph Anthes, and Jens Volkert. An efficient Physics Engine for Virtual Worlds. In medi@terra, pages 152-158, Athens, Greece, October 2006.Helmut Bressler, Roland Landertshamer, Christoph Anthes, and Jens Volkert. An efficient Physics Engine for Virtual Worlds. In medi@terra, pages 152-158, Athens, Greece, October 2006. Christoph Anthes and Jens Volkert. inVRs - A Framework for Building Interactive Networked Virtual Reality Systems. In International Conference on High Performance Computing and Communications (HPCC '06), pages 894-904, Munich, Germany, September 2006.Christoph Anthes and Jens Volkert. inVRs - A Framework for Building Interactive Networked Virtual Reality Systems. In International Conference on High Performance Computing and Communications (HPCC '06), pages 894-904, Munich, Germany, September 2006. Christoph Anthes, Adrian Haffegee, Paul Heinzlreiter, and Jens Volkert. A Scalable Network Architecture for Closely Coupled Collaboration. In Journal of Computing and Informatics, 24(1):31-51, August 2005.Christoph Anthes, Adrian Haffegee, Paul Heinzlreiter, and Jens Volkert. A Scalable Network Architecture for Closely Coupled Collaboration. In Journal of Computing and Informatics, 24(1):31-51, August 2005. Christoph Anthes and Jens Volkert. A Toolbox Supporting Collaboration in Networked Virtual Environments. In International Conference on Computational Science (ICCS '05), pages 383-390, Atlanta, GA, USA, May 2005.Christoph Anthes and Jens Volkert. A Toolbox Supporting Collaboration in Networked Virtual Environments. In International Conference on Computational Science (ICCS '05), pages 383-390, Atlanta, GA, USA, May 2005.

64. 20.03.10inVRs - Tutorial II - Going Immersive64 Bibliography Adrian Haffegee, Ronan Jamieson, Christoph Anthes, and Vassil N. Alexandrov. Tools For Collaborative VR Application Development. In International Conference on Computational Science (ICCS '05), pages 350-358, Atlanta, GA, USA, May 2005.Adrian Haffegee, Ronan Jamieson, Christoph Anthes, and Vassil N. Alexandrov. Tools For Collaborative VR Application Development. In International Conference on Computational Science (ICCS '05), pages 350-358, Atlanta, GA, USA, May 2005. Christoph Anthes, Paul Heinzlreiter, Adrian Haffegee, and Jens Volkert. Message Traffic in a Distributed Virtual Environment for Close-coupled Collaboration. In Parallel and Distributed Computing and Systems (PDCS '04), pages 484- 490, San Francisco, CA, USA, September 2004.Christoph Anthes, Paul Heinzlreiter, Adrian Haffegee, and Jens Volkert. Message Traffic in a Distributed Virtual Environment for Close-coupled Collaboration. In Parallel and Distributed Computing and Systems (PDCS '04), pages 484- 490, San Francisco, CA, USA, September 2004. Christoph Anthes, Paul Heinzlreiter, Gerhard Kurka, and Jens Volkert. Navigation models for a flexible, multi-mode VR Navigation Framework. In ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04), pages 476-479, Singapore, June 2004.Christoph Anthes, Paul Heinzlreiter, Gerhard Kurka, and Jens Volkert. Navigation models for a flexible, multi-mode VR Navigation Framework. In ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04), pages 476-479, Singapore, June 2004. Christoph Anthes, Paul Heinzlreiter, and Jens Volkert. An adaptive Network Architecture for Close-coupled Collaboration in Distributed Virtual Environments. In ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04), pages 382-385, Singapore, June 2004.Christoph Anthes, Paul Heinzlreiter, and Jens Volkert. An adaptive Network Architecture for Close-coupled Collaboration in Distributed Virtual Environments. In ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04), pages 382-385, Singapore, June 2004.