VAPoR: A Discovery Environment for Terascale Scientific Data Sets Alan Norton & John Clyne National Center for Atmospheric Research Scientific Computing Division

Alan Norton VAPoR project overview l GOAL: Improve ability of earth sciences researchers to analyze and explore complex dynamics found in vast data sets resulting from high-resolution, 4D (3D space x time) numerical simulations of fluid flow in the oceans, atmosphere and sun l Open Source software development effort aimed at bridging gap between visualization research and application tool availability l Funded by the NSF ITR program, VAPoR is a collaborative effort between NCAR’s Visualization and Enabling Technology Section, U.C. Davis’ Institute for Data Analysis and Visualization, and Ohio State University’s, Department of Computer and Information Sciences

Alan Norton The numbers – an example from solar compressible convection l 512x512x2048 l 5 variables (u,v,w,rho,temp) l ~500 time steps saved l 9TBs storage l Six months compute time required on 112 IBM SP RS/6000 processors (blackforest) l Three months for post- processing l Data may be analyzed for several years Image courtesy of Joseph Mendoza, NCAR/SCD

Alan Norton Our approach l Exploit power of multiresolution data representation combined with progressive data access and efficient region sub-setting to enable user to make speed/quality tradeoffs [ Pascucci and Frank, SC2001; Clyne, VIIP2003] l Tightly couple traditional, non-visual (MatLab, IDL) analysis with highly interactive, advanced visualization [Clyne and Rast, VDA2005] l Develop domain-specific application for numerical simulation of fluid flow l Emphasize usability for scientists

Alan Norton A visual comparison between coarsened and original data

Alan Norton Combining visual and non-visual analysis tools l When visualization and quantitative tools are appropriately combined, result is a powerful environment for scientific discovery l Couple existing quantitative tools (e.g. IDL, Matlab, NCL) with VAPoR’s advanced visualization capabilities l Benefits of integration with existing analysis tools –Avoids duplication of effort –Lower development time –Leverage what scientists already know

Alan Norton Integrated analysis and visualization system Vtk High quality rendering VAPoR Interactive visual browsing IDL Data manipulation & analysis Multiresolution data access Disk Array

Alan Norton Software Design Priorities l Functional and usability requirements prioritized by scientific users l Cross-platform (Linux, Irix, Windows, Mac,…) l Enable side-by-side comparison of multiple time-varying visualizations –Unique approach to management of multiple parameter sets l Exploit high-performance graphics cards –GPU programming l Incorporate recent advances in data visualization –Volume rendering –Flow visualization –Isosurface generation –Color/transparency mapping l XML data description –Encourage wider usage of multi-resolution data representation

Alan Norton Demonstration l Load multiresolution data into VAPoR application l Combine global and local parameter settings l Edit transfer function(s) l Interactive region selection l Animation l Export to IDL l Use IDL for analysis l Visualize analysis results

Alan Norton Future Plans l Incorporate visualization techniques based on scientists’ needs –Nonuniform grids –Vector field visualization –Isosurfaces –Contour planes l Understand effect of lossy data compression –Error analysis and error visualization –Obtain bounds on degradation of analysis results –Measure the effect of performing analysis on lower-resolution data l Improve access to terabyte datasets –Multiresolution data output as a byproduct of the simulation

Alan Norton Summary l NSF funded collaboration with Davis and Ohio state l Domain specific application aimed at improving productivity of turbulence researchers (though other groups may benefit) l Scientists are target user group (not visualization specialists) l Multiresolution data representation exploited for handling large data l Quantitative and advanced visualization methods combined l Usability is key to adoption by scientists I can directly testify to the crucial importance of this [effort] to our numerical work... to develop interactive tools that can access and analyze the flow in its entirety, and visualize it in complex ways including parallelized algorithms of perspective volume rendering, superposition of scalar and vector fields, and multi-resolution access to the flows. Annick Pouquet, Director of ESSL