WP-8, ZIB WP-8: Data Handling And Visualization Review Meeting Report Felix Hupfeld, Andrei Hutanu, Andre Merzky, Thorsten Schütt, Brygg Ullmer Zuse-Institute-Berlin

WP-8, ZIB Overview Formal Matters Visualization Tangible Devices Remote File Access Replica Management / File Movement Metadata Service

WP-8, ZIB Formal Matters Deliverables: most right on time, some delivered earlier Milestone 8.2 (M24): Dynamic data access techniques are designed and implemented Replica catalog (Location independant file access) Remote File Access (Pitfalls and GridFTP) Visualization service Plan for the rest of this year: Further integration with other WPs Implement additional features (user management)

WP-8, ZIB Formal Matters (II) B. Ullmer, A. Hutanu, W. Benger, H.-C. Hege. (2003). Emerging Tangible Interfaces for Facilitating Collaborative Immersive Visualization. Workshop on Collaborative Virtual Reality and Visualization, October H.-C. Hege, A. Hutanu, R. Kähler, A. Merzky, T. Radke, E. Seidel, B. Ullmer. (2003). Progressive Retrieval and Hierarchical Visualization of Large Remote Data. Workshop on Adaptive Grid Middleware, September T. Schütt, A. Merzky, A. Hutanu, F. Schintke, Remote Partial File Access Using Compact Pattern Descriptions, IEEE CCGrid 2004.

WP-8, ZIB Visualization context Grid = “virtualization” of resources Broad progress toward virtualization of hardware resources (compute power, disk space, network) Less common: virtualization of graphics hardware, software “Virtualization” of visualization resources Scientific visualization has demanding hardware and software requirements, unavailable to many users Software is frequently complex and “heavy-weight” Our approach: Automation for core visualization operations Provision of grid-based “visualization services” Develop simplifying mobile services & devices

WP-8, ZIB Visualization and mobile devices Mobile devices Access to viz services from mobile phones, PDAs Development of new grid-enabled “mobile” viz devices Visualization Access to grid resources from existing viz applications Access to existing viz applications from the grid

WP-8, ZIB Visualization service 1D 2D 3D

WP-8, ZIB Mobile + viz = “viz artifacts” Motivation Advantages of stereo, collaborative interaction Importance of the meeting room, teleconferencing Simplifying interaction + gaining real estate by breaking out into the physical world

WP-8, ZIB Data cards Represent online data, user credentials, remote users, parameters, etc.

WP-8, ZIB Interaction pads “binding pad”“placement pad” “parameter pad”“control pad”

WP-8, ZIB Remote File Access (I) Needed for Visualization of remote files (intermediate results of large jobs) Job input files (extract and transmit only data of interest) … Existing mechanisms GridFTP remote partial file access (read, seek, read, seek,.. ) I/O wrapper for local and remote access methods (to be used in GAT) But does not fit Grid requirements, because too many small remote operations!

WP-8, ZIB Remote File Access (II)  Need to execute high-level operations on the server side File format specific approach(HDF5) HDF5 – structured binary file format. Datasets, Groups, Attributes Forwards HDF5 calls to the remote side Plugin for the Globus GridFTP server Generic approach (much better )  next slide

WP-8, ZIB Remote File Access (III) Generic approach Compact access pattern description: Pitfalls Speedup of several orders of magnitude achieved! Example: Access part of a Cube: (339, 65874, , (0, 511, 2048, 32, (0, 511, 2048, 32, (0, 3, 16, 32)))  just 1 (in words: one!) remote call

WP-8, ZIB Replica Management Components Replica catalog Supports hierarchical data organization POSIX filesystem like interface Uses C++ with SOAP/GSI interface Adaptor inside of GAT File movement service Used for replication Based on GridFTP Adaptive service (WP6) Ranks replica locations Optimizes GridFTP parameters

WP-8, ZIB StorageBox Metadata Service Unified storage for metadata of persistent Grid objects (archived files, job input/output, messages/notifications) Any number of namespaces, configurable via pluggable access policies as public or private archives, group collaboration spaces or private annotations for public data Public Sensor Data Archive User ‘A’: Sensor Data Annotations and Private Messages Project ‘X’: Collaboration Space User ‘B’: Sensor Data Annotations and Private Messages A B

WP-8, ZIB StorageBox Metadata Service Portlet as user interface Integrated with WP12’s MessageBox service SOAP web service interface Pluggable database backends with PostgreSQL implementation Portable C++ implementation Win32, Linux, Mac OS X Planned: Improved query language / processing Remote administration interface More access policies

WP-8, ZIB Continuing plans Visualization, mobile devices Further integration with portal Implement additional viz operations Deploy, test both services & devices w/ end users Data management Provide a flexible namespace for organizing files Remote File Access: Generic vs. File Format Dependant

WP-8, ZIB Questions?

WP-8, ZIB Visualization and mobile devices Mobile devices Access to viz services from mobile phones, PDAs Completed + demoable from mobile phones, iPAQ Completed + demoable from mobile phones, iPAQ Demoable: use of new devices to load remote content, activate and control video conference sessions Development of new grid-enabled “mobile” viz devices Demoable: use of new devices to load remote content, activate and control video conference sessions Visualization Access to grid resources from existing viz applications Completed: remote data access from Amira Completed: remote data access from Amira Access to existing viz applications from the grid Demoable: grid automation, presentation of reports integrating content from gnuplot, OpenDX, Amira; grid access to streaming Amira content Demoable: grid automation, presentation of reports integrating content from gnuplot, OpenDX, Amira; grid access to streaming Amira content

WP-8, ZIB Videoconferencing example , Berlin-Munich, surgical planning