Georgia Tech GVU Center Mobile Visualization in a Dynamic, Augmented Battlespace Mobile Visualization in a Dynamic, Augmented Battlespace William Ribarsky and Nickolas Faust GVU Center Georgia Institute of Technology

Georgia Tech GVU Center VGIS, a General Framework for Global Geospatial Data and Visualization It accepts and integrates all types of geospatial data into a global framework. It is scalable to very large data stores and to distributed environments. It provides an interactive (“real time”) visualization framework. It supports discovery of new information via navigation in context with unfolding detail. It supports heterogeneous detail management. Background:

Georgia Tech GVU Center General Data Framework Universal Data Structure One global hierarchy with multiple instances A common structure for fast paging & caching Common, image-based metrics for display Multiple levels of detail Evaluates Intent & Perception of user Background:

Georgia Tech GVU Center Global Data Model: Forest of Quadtrees 32 zones 8 levels per quadtree Background:

Georgia Tech GVU Center Global Data Model: Forest of Quadtrees 32 zones 8 levels per quadtree Separate coodinate system at each leaf node (256x256 total nested systems) 1 mm worst case resolution Background:

Georgia Tech GVU Center General Zonal Quadtree } tree structure Data-dependent, detailed, low-level representation Background:

Georgia Tech GVU Center } Terrain Zonal Quadtree 22 levels Regular Triangularization (1024 x 1024 grid points) Example:

Georgia Tech GVU Center Static Object Zonal Quadtree Buildings or other static objects –Static in organization –Uses hierarchy to around 100 meters,when closer: Uses bounding box detection Uses imposters for detail management Uses collision detection Example:

Georgia Tech GVU Center Time-dependent Volume Data Quadtree Stacked, time-sequenced volumes for fast visualization Example: Geospatial Quadtree Geolayered Volume Latitude Altitude Longitude Longitude cone wedge Latitude plane wedge i m e t

Georgia Tech GVU Center View-Dependent Detail Management This example is for buildings, but the procedure has been generalized to other types of data. Bounding dimension Eye Example:

Georgia Tech GVU Center World Atlanta, GA area Grand Canyon Rwanda Continuous Navigation Around the U.S. (or World) Results:

Georgia Tech GVU Center Atlanta NTC San Francisco Savannah Hierarchical Urban Construction: Cities Everywhere Results:

Georgia Tech GVU Center Goals and Directions Mobile and stationary visualization within a common framework Dynamic, universal data structures with fast updates Real-time, scalable visualization of all visual products Multimodal interaction in multiple environments

Georgia Tech GVU Center Technical Challenges Developing & evaluating an appropriate mobile visualization paradigm Hierarchical, scalable data structures with fast access that include time, uncertainty, and varied data products Methods for fast update of dynamic data structures Automated detail management schemes for uncertainty and other visual products Developing & evaluating multiple modes of interaction & navigation; matching them to different environments Intelligent retrieval and visualization of data products

Georgia Tech GVU Center Hierarchical, Scalable Data structures tree structure i m e t Uncertainty, Annotations, & Semantics Various low-level, detailed representations

Georgia Tech GVU Center Dynamic Data Structures & Real-time Data Insertion of geo-registered remote sensor data Skeleton quadtree Properties Dynamic Data Cache Specific Data Manager Skip Main Memory Prioritized

Georgia Tech GVU Center Dynamic Data Structures & Real-time Data Insertion of geo-registered remote sensor data Dynamic Data Cache Specific Data Manager Main Memory Distributed Hard Disk Specific Data Server Prioritized

Georgia Tech GVU Center Multimodal Interaction in Multiple Environments Environments Desktop Laptop Mobile (palm, HMD, larger handheld or wearable) Virtual Workbench Liveboard NAVE Others Interaction Modes Mouse Joystick or Spaceball Pen-based or touch screen Wired 3D or wireless 3D Head tracking, hand-tracking, gesture recognition Voice Others What are the environmental affordances? What are user needs? Is 3D interaction necessary? How best to navigate large virtual spaces (2D or 3D)? How to most effectively combine modes? How to evaluate?

Georgia Tech GVU Center Detail Management for Uncertainty and Other Visual Products Hierarchical fast clustering for 4D patterns Interactive switching on/off and focused, adaptive views Radar Icon

Georgia Tech GVU Center A Battlespace Visualization Testbed Mobile and stationary visualization within a common framework -for evaluation of concepts and tools -for integration and testing -for demonstrations

Georgia Tech GVU Center Agile, Mobile, Collaborative Testbed A networked, collaborative whole Earth terrain database linking workstations, large projected displays, and mobile handheld systems. Mobile users carry systems with handheld or augmented displays providing 3D terrain visualizations. Mobile users receive, record, and transmit information about the world. Users of stationary 3D displays collect and evaluate information provided by mobile users and route data and intelligence back to mobile users. Collaboration is through annotation of the virtual geospatial database.

Georgia Tech GVU Center Central Terrain Database NAVEVirtual Workbench Mobile Displays Collaboration Channels Wireless Terrain Datalink Terrain Datalink Position, Terrain Markup, Route Finding, Weather, Friendly/Foe advisories, etc. w/GPS, bearing, tilt sensors and wireless data. Local databases download and cache data according to bandwidth, movement, and rendering speed of each platform. Wireless Architecture: Agile, Mobile, Collaborative Testbed

Georgia Tech GVU Center Devices: Agile, Mobile, Collaborative Testbed Two mobile, wearable computers with different displays and different interfaces. The one on the left captures gestures with a camera while the one in the center uses a handheld "twiddler". On the right is the Virtual Workbench set up with a perceptual interface that captures 3D object shapes as well as hand and arm gestures. Multiscreen, immersive, and inexpensive NAVE environment for command post operations.

Georgia Tech GVU Center Mobile, Wireless Coverage Antenna Georgia Tech campus testbed