1. Integrating 3D Geodata in Service-Based Visualization Systems Jan Klimke, Dieter Hildebrandt, Benjamin Hagedorn, and Jürgen Döllner Computer Graphics Systems Group

2. Context & Scope 3D Geovirtual Environments such as virtual 3D city models virtual 3D landscape models serve as enabling technology for applications in anInternet of Things with respect to communication spatial information. Integration of 3D georeferenced data is required to build application-specific and task-specific solutions. Data source examples: Points of Interest (OpenLS Services) Sensor data (SWE services) Thematic feature data (WFS) Georeferenced data from the social web (Facebook, Foursquare, Latitude …) Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 2

3. Concept for Service-based Interactive Clients 1/2 Service-based 3D visualization system: High-quality visualization of complex, massive geodata Interactive exploration of 3D geovirtual environments Thin client applications on mobile, respectively low-end devices. Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 3 Key element: Portrayal service for 3D geovirtual environments based on server-side 3D rendering

4. Concept for Service-based Interactive Clients 2/2 3D Server: Management, integration, processing, and rendering of massive 3D geodata (Web View Service (WVS) ) Creates image representations (G-Buffers) 3D Client consumes rendered G- Buffers of 3D model data Interactive reconstruction of 3D scene from G-Buffers Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 4 WVSClient G-Buffer 3D Model Data Network Color Depth Object ID Normals G-Buffers

5. 3D Client Nested Geovisualization Pipeline for Image-based Interactive 3D Clients Visualization pipeline model: Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 Computer Graphics Primitives (Quads, Meshes, Points) Rendering 3D Server FilteringMapping G-Buffer Creation 3D Model Data Selected Model Data Image Post- processing G-Buffers Computer Graphics Primitives Mapping (Scene Reconstruction) 5 G-Buffer

6. The Running System Demonstration Video Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 6

7. Properties of the Image-based Visualization System 3D Server: Copes with raising complexity of 3D rendering techniques and raising diversity of 3D graphics hardware and software Reduces complexity for processing and rendering of massive, complex 3D geodata in client applications Enables high quality visualizations 3D Client: Minimal hardware and software requirements through partial reconstruction of 3D GeoVE from image data Enables interactive exploration of 3D GeoVEs Reduced and predictable network traffic Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 7

8. Research Question How to integrate georeferenced data from an Internet of Things into service-based 3D GeoVEs Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 8

9. Data Integration Strategies Aspects: Performance and scalability Flexibility Configurable display elements for entities Shape, color, size … Configure sources for 3D data Visual quality Can advanced visualization techniques (lighting, focus & context, illustrative rendering) be applied ? Dynamic data handling Support for visualization of changing data (e.g., moving objects) Interaction with information representations Access to underlying data Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 9

10. Data Integration Strategies (3D Server) Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 WVSClient G-Buffer 3D Model Data Network IoT Raw Data WFS, OpenLS, Sensor Data Renderables from W3DS Rendered image representations from WVS 10

11. Data Integration Strategies (3D Client) Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 11 WVSClient G-Buffer 3D Model Data Network IoT Raw Data WFS, OpenLS, Sensor Data Renderables from W3DS Rendered image representations from WVS

12. Server-side Integration Mapping 1/2 Include external data in mapping process of 3D model visualization External data is fetched, mapped and rendered by the portrayal server Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 3D Server FilteringMapping G-Buffer Creation 3D Model Data Selected Model Data Computer Graphics Primitives IoT Data for Selected Things Image Post- processing G-Buffers 12

13. Server-side Integration Mapping 2/2 Computer graphics representations of things through server-side mapping of input data Creating 3D objects for rendering Encapsulated in portrayal service instance Mapping -> computer graphics primitives can be encapsulated through W3DS Example primitives: Icons 3D models (e.g., vending machine, car, etc.) Projective textures Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 13

14. Server-side Integration Image Post Processing 1/2 Parallel instance of visualization pipeline for non-model 3D data Data integration through image post processing Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 3D Server Selected Model Data FilteringMapping G-Buffer Creation IoT SelectedThing Data Computer Graphics Primitives FilteringMapping G-Buffer Creation 3D Model Data Selected Model Data Image Post- processing G-Buffers Computer Graphics Primitives G-Buffers forThings 14

15. Server-side Integration – Image Post Processing 2/2 Combination of image representation in an image post processing step Use Information from G-Buffers (e.g., Depth, ObjectID) Specialized rendering pipeline / service for 3D model data Parallel rendering of 3D model data G-Buffers G-Buffers for external data Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 15

16. Client-side Integration Image Composition Analogue to server-side image post processing but on client-side Additional bandwidth requirements G-Buffers for 3D model and thing-data need to be transferred Specialized portrayal service instances can be used for item rendering Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 3D Client Computer Graphics Primitives (Quads, Meshes, Points) Rendering Mapping (Scene Reconstruction) City Model WVS Thing Visualization WVS G-Buffers of 3D Model Data G-Buffers forThings 16

17. Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 Client-side Integration Rendering of Thing Representations 1/2 3D Client Mapping (Scene Reconstruction) Computer Graphics Primitives (Quads, Meshes, Points) Rendering Filtering Mapping IoT Selected Thing Data Computer Graphics Primitives (Text, Icons, Models…) G-Buffers for 3D Model Data 3D Server Selected Model Data FilteringMapping G-Buffer Creation 3D Model Data Selected Model Data Image Post- processing G-Buffers Computer Graphics Primitives 17

18. Client-side Integration Rendering of Thing Representations Data of things is fetched by the client application Data handling has to be implemented on client side ! Integration of generated 3D computer graphics primitives into the reconstructed 3D scene Mapping on client side -> interactively configurable Limited amount of model data available, e.g., for item positioning ! Animation and direct interaction with data representations is possible Limited by client hardware/software capabilities Advanced rendering techniques demand for graphics capabilities on client side Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 18

19. Summary & Conclusions 3D GeoVE: Generic user interface for spatial information Service-based solution: 3D server + interactive 3D client Decomposition of visualization into separate components for each type of data (3D model data / thematic data) Usage of existing service definitions for portrayal of georeferenced data (WVS, W3DS) Georeferenced data can be seamlessly integrated: Desired/ feasible complexity of client applications as central point for selection of integration strategy Mapping, in particular 3D stylization, has to be investigated further Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 19

20. WVS www.webviewservice.org OGC 3D Portrayal Interoperability Experiment (3DPIE) http://www.opengeospatial.org/projects/initiatives/3dpie Contact Thank You ! Expanding GeoWeb to IoT| Jan Klimke | www.hpi3d.de | May 24th, 2011 20 Jan Klimke jan.klimke@hpi.uni-potsdam.de Computer Graphics Systems Group Prof. Dr. Jürgen Döllner www.hpi3d.de