Presentation on theme: "The Big Picture Scientific disciplines have developed a computational branch Models without closed form solutions solved numerically This has lead to."— Presentation transcript:
The Big Picture Scientific disciplines have developed a computational branch Models without closed form solutions solved numerically This has lead to an explosion of data Simulation and analysis workloads are data-intensive Producing\scanning large amounts of data Management of these data represents a significant challenge Storage\archiving Query processing Visualization
Remote Immersive Analysis Formerly, analysis performed during the computation No data stored for subsequent examination Data-intensive computing breakthroughs have allowed for new interaction with scientific numerical simulations Turbulence Database Cluster Stores entire space-time evolution of the simulation Provides public access to world-class simulations Implements “immersive turbulence * ” approach Introduces new challenges * E. Perlman, R. Burns, Y. Li, and C. Meneveau. Data exploration of turbulence simulations using a database cluster. In Supercomputing, 2007.
Goals Develop data-driven query processing techniques Reduce I/O and computation costs Reduce or eliminate storage overhead Exploit domain knowledge and structure Provide user interfaces that are efficient and flexible Streamline the process of data ingest
128 Workload Over an order of magnitude improvement Sorting leads to a more sequential acces Join/Order By executes entire batch as a join I/O Streaming Each atom is read only once Effective cache usage
I/O Streaming alleviates I/O bottleneck Computation emerges as the more costly operation
Particle Tracking Web Server/Mediator DB Node 1 Distribute Points based on Computational Module Storage Layer Retrieve DB Node N Computational Module Storage Layer Retrieve x p (t m ) x * p (t m )
Particle Tracking Web Server/Mediator DB Node 1 Distribute Points based on Computational Module Storage Layer Retrieve DB Node N Computational Module Storage Layer Retrieve x * p (t m ) x p (t m+1 )
Summary and Future Work Extend I/O streaming technique to different decomposable kernel computations: Differentiation Spatial Interpolation Temporal interpolation Filtering and coarse-graining Provide a flexible user interface Allow for different filter functions Allow for new kernel computations Improve particle tracking routine Reduce communication between mediator and DB nodes Asynchronous processing Caching and pre-fetching
Questions Images courtesy of Kai Buerger (firstname.lastname@example.org)
Your consent to our cookies if you continue to use this website.