SCD User Forum 2005 Dynamically adaptive geophysical fluid dynamics simulation using GASpAR: Geophysics/Astrophysics Spectral-element Adaptive Refinement.

Slides:

Advertisements

Similar presentations

Numerical Simulation of Complex and Multiphase Flows 18 th – 22 nd April Porquerolles 1/24 Finite volumes and finite elements for the numerical simulation.

Advertisements

Modeling Electrical Systems With EMTP-RV

Trellis: A Framework for Adaptive Numerical Analysis Based on Multiparadigm Programming in C++ Jean-Francois Remacle, Ottmar Klaas and Mark Shephard Scientific.

Parallel Computation of the 2D Laminar Axisymmetric Coflow Nonpremixed Flames Qingan Andy Zhang PhD Candidate Department of Mechanical and Industrial Engineering.

Adaptive mesh refinement for discontinuous Galerkin method on quadrilateral non-conforming grids Michal A. Kopera PDE’s on the Sphere 2012.

A Bezier Based Approach to Unstructured Moving Meshes ALADDIN and Sangria Gary Miller David Cardoze Todd Phillips Noel Walkington Mark Olah Miklos Bergou.

Gravitational Collapse in Axisymmetry Collaborators: Matthew Choptuik, CIAR/UBC Eric Hircshmann, BYU Steve Liebling, LIU APS Meeting Albuquerque, New Mexico.

1 Internal Seminar, November 14 th Effects of non conformal mesh on LES S. Rolfo The University of Manchester, M60 1QD, UK School of Mechanical,

Parallel Mesh Refinement with Optimal Load Balancing Jean-Francois Remacle, Joseph E. Flaherty and Mark. S. Shephard Scientific Computation Research Center.

Prediction of Fluid Dynamics in The Inertial Confinement Fusion Chamber by Godunov Solver With Adaptive Grid Refinement Zoran Dragojlovic, Farrokh Najmabadi,

SSL (UC Berkeley): Prospective Codes to Transfer to the CCMC Developers: W.P. Abbett, D.J. Bercik, G.H. Fisher, B.T. Welsch, and Y. Fan (HAO/NCAR)

Adaptive MPI Chao Huang, Orion Lawlor, L. V. Kalé Parallel Programming Lab Department of Computer Science University of Illinois at Urbana-Champaign.

Chamber Dynamic Response Modeling Zoran Dragojlovic.

Efficient Parallelization for AMR MHD Multiphysics Calculations Implementation in AstroBEAR.

1/36 Gridless Method for Solving Moving Boundary Problems Wang Hong Department of Mathematical Information Technology University of Jyväskyklä

Application Performance Analysis on Blue Gene/L Jim Pool, P.I. Maciej Brodowicz, Sharon Brunett, Tom Gottschalk, Dan Meiron, Paul Springer, Thomas Sterling,

1 Parallel Simulations of Underground Flow in Porous and Fractured Media H. Mustapha 1,2, A. Beaudoin 1, J. Erhel 1 and J.R. De Dreuzy IRISA – INRIA.

Introduction to virtual engineering László Horváth Budapest Tech John von Neumann Faculty of Informatics Institute of Intelligent Engineering.

Computational Modeling of Turbulent Asymmetric Jet Flows Prof. Ed Akin Mechanical Engineering and Materials Science Rice University Houston, Texas Jon.

Parallel Adaptive Mesh Refinement Combined With Multigrid for a Poisson Equation CRTI RD Project Review Meeting Canadian Meteorological Centre August.

ParFUM Parallel Mesh Adaptivity Nilesh Choudhury, Terry Wilmarth Parallel Programming Lab Computer Science Department University of Illinois, Urbana Champaign.

LLNL-PRES-XXXXXX This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

SOME EXPERIMENTS on GRID COMPUTING in COMPUTATIONAL FLUID DYNAMICS Thierry Coupez(**), Alain Dervieux(*), Hugues Digonnet(**), Hervé Guillard(*), Jacques.

Scalable Algorithms for Structured Adaptive Mesh Refinement Akhil Langer, Jonathan Lifflander, Phil Miller, Laxmikant Kale Parallel Programming Laboratory.

Sub-grid Scale Terms in Plasma Turbulence Direct Numerical Simulations of Drift-Kinetic Turbulence in 4D Ravi Samtaney PSE and MCSE Divisions, KAUST.

Presented by High Productivity Language and Systems: Next Generation Petascale Programming Wael R. Elwasif, David E. Bernholdt, and Robert J. Harrison.

ParCFD Parallel computation of pollutant dispersion in industrial sites Julien Montagnier Marc Buffat David Guibert.

Supercomputing Center CFD Grid Research in N*Grid Project KISTI Supercomputing Center Chun-ho Sung.

Adaptive Mesh Modification in Parallel Framework Application of parFUM Sandhya Mangala (MIE) Prof. Philippe H. Geubelle (AE) University of Illinois, Urbana-Champaign.

Institute for Mathematics Applied to Geoscience Geophysical Statistics Project - GSP Data Assimilation Research Section - DAReS Turbulence Numerics Team.

SUPERCOMPUTING CHALLENGE KICKOFF 2015 A Model for Computational Science Investigations Oct 2015 © challenge.org Supercomputing Around.

Supercomputing ‘99 Parallelization of a Dynamic Unstructured Application using Three Leading Paradigms Leonid Oliker NERSC Lawrence Berkeley National Laboratory.

ARGONNE NATIONAL LABORATORY Climate Modeling on the Jazz Linux Cluster at ANL John Taylor Mathematics and Computer Science & Environmental Research Divisions.

Computational Aspects of Multi-scale Modeling Ahmed Sameh, Ananth Grama Computing Research Institute Purdue University.

1 1 What does Performance Across the Software Stack mean?  High level view: Providing performance for physics simulations meaningful to applications 

Parallel Solution of the Poisson Problem Using MPI

VAPoR: A Discovery Environment for Terascale Scientific Data Sets Alan Norton & John Clyne National Center for Atmospheric Research Scientific Computing.

Domain Decomposition in High-Level Parallelizaton of PDE codes Xing Cai University of Oslo.

Cracow Grid Workshop, November 5-6, 2001 Concepts for implementing adaptive finite element codes for grid computing Krzysztof Banaś, Joanna Płażek Cracow.

A Dirichlet-to-Neumann (DtN)Multigrid Algorithm for Locally Conservative Methods Sandia National Laboratories is a multi program laboratory managed and.

Present / introduce / motivate After Introduction to the topic

Partitioning using Mesh Adjacencies  Graph-based dynamic balancing Parallel construction and balancing of standard partition graph with small cuts takes.

Data Structures and Algorithms in Parallel Computing Lecture 7.

Presented by Adaptive Hybrid Mesh Refinement for Multiphysics Applications Ahmed Khamayseh and Valmor de Almeida Computer Science and Mathematics Division.

Distributed Resistances and Fan Models Chapter 4.

1 Data Structures for Scientific Computing Orion Sky Lawlor /04/14.

1 Rocket Science using Charm++ at CSAR Orion Sky Lawlor 2003/10/21.

ParMA: Towards Massively Parallel Partitioning of Unstructured Meshes Cameron Smith, Min Zhou, and Mark S. Shephard Rensselaer Polytechnic Institute, USA.

C OMPUTATIONAL R ESEARCH D IVISION 1 Defining Software Requirements for Scientific Computing Phillip Colella Applied Numerical Algorithms Group Lawrence.

Predictive Load Balancing Using Mesh Adjacencies for Mesh Adaptation  Cameron Smith, Onkar Sahni, Mark S. Shephard  Scientific Computation Research Center.

Hierarchical Load Balancing for Large Scale Supercomputers Gengbin Zheng Charm++ Workshop 2010 Parallel Programming Lab, UIUC 1Charm++ Workshop 2010.

A Parallel Hierarchical Solver for the Poisson Equation Seung Lee Deparment of Mechanical Engineering

Quality of Service for Numerical Components Lori Freitag Diachin, Paul Hovland, Kate Keahey, Lois McInnes, Boyana Norris, Padma Raghavan.

An Introduction to Computational Fluids Dynamics Prapared by: Chudasama Gulambhai H ( ) Azhar Damani ( ) Dave Aman ( )

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation,

1 Copyright by PZ Bar-Yoseph © Finite Element Methods in Engineering Winter Semester Lecture 7.

Application of Design Patterns to Geometric Decompositions V. Balaji, Thomas L. Clune, Robert W. Numrich and Brice T. Womack.

Unstructured Meshing Tools for Fusion Plasma Simulations

Chamber Dynamic Response Modeling

Programming Models for SimMillennium

Adaptive Grid Generation for Magnetically Confined Plasmas

Component Frameworks:

Milind A. Bhandarkar Adaptive MPI Milind A. Bhandarkar

GENERAL VIEW OF KRATOS MULTIPHYSICS

ArcEOR A posteriori error estimate tools to enhance the performance of

Low Order Methods for Simulation of Turbulence in Complex Geometries

Ph.D. Thesis Numerical Solution of PDEs and Their Object-oriented Parallel Implementations Xing Cai October 26, 1998.

CASA Day 9 May, 2006.

Presentation transcript:

SCD User Forum 2005 Dynamically adaptive geophysical fluid dynamics simulation using GASpAR: Geophysics/Astrophysics Spectral-element Adaptive Refinement Duane Rosenberg, Aimé Fournier, Paul Fischer, Annick Pouquet NCAR Institute for Mathematics Applied to Geosciences Turbulence Numerics Team

SCD User Forum 2005 Motivation: Methodology Requirements Variable (e.g., high) order method (Patera 1984) Must be non-dissipative for DNS Must also model small scales, high Re, complex boundaries Must support variety of equations, physical models, geometries  code extensibility Leverage existing NCAR expertise in science applications, methods and performance

SCD User Forum 2005 GASpAR Features Designed for turbulence studies Object oriented ( C++ ) –extensive use of inheritance, polymorphism, templates Hierarchical: Elements  Fields  Eq. solvers  SEM operators  Bases  GBLAS Adaptive Mesh (currently 2D unstructured) Parallelized

SCD User Forum 2005

Spectral element numerics Based on spectral-element operators Typically, use operator-vector products  basic matrix-matrix products (m_x_m)

SCD User Forum 2005 Handling Nonconformity SCD User Forum 2005

Connectivity uses “mortars” (color matched)

SCD User Forum 2005 Locating neighbors and constructing mortars: VDB ( Henderson 1993 )

SCD User Forum 2005 ‘Mortar’ Data Exchange Mortar nodes given unique Morton keys (Warren & Salmon 1993) Bin-sort keys and create node/processor map Do DSS

SCD User Forum 2005 Adaptation Rules Isotropic (for now) Refinement: –At most 2 child edges (4 child faces in 3D) –Edge bisection –h-refinement only (currently) –Max. number of refinement levels Coarsening: –Cannot coarsen root –All siblings must be tagged –Must not violate Refinement rules

SCD User Forum 2005 Refinement & Coarsening Criteria a-posteriori spectral estimator of Mavriplis (1990) gradient 2 magnitude open question for turbulent flows (esp. in various formulations of the equations)

SCD User Forum 2005

Development/Porting Strategy Initial development under Linux using gcc Test MPI using MPICH with 1 or 2 procs Locate bottlenecks with port to Lightning (use PG compilers with -Minfo=loop,inline) --Find C m_x_m will not vectorize, but Fortran will! --Factor of 10 speedup in m_x_m! Port to IBM SPs (VAC++, f77) --Use gprof, HPM --Debug with totalview and printf’s

SCD User Forum 2005 Advecting Gaussian ‘Lump’

SCD User Forum 2005 Advecting Gaussian ‘Lump’ Mesh

SCD User Forum 2005 Radial N-wave Solution for Nonlinear Burgers Eqn.

SCD User Forum 2005 Rotated Burgers Front

SCD User Forum 2005 Dynamical Systems: Large Scale Passive Advection of Small Scale Field

SCD User Forum 2005 Conclusion 2D adaptive algorithm is working  Must include dynamic load balancing/processor distribution. Complete 3D AMR, enhance performance based on 2D; debugging difficult Better performance monitoring/reporting on the SPs?

SCD User Forum 2005 Nonconforming Connectivity: non-Boolean gather operator Q maps global (j=0:18) to local (i=0:26) d.o.f. (after Fischer & Kruse 1997) Q 12,2 Qi,jQi,j

SCD User Forum 2005