Presentation is loading. Please wait.

Presentation is loading. Please wait.

Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project.

Similar presentations

Presentation on theme: "Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project."— Presentation transcript:

1 Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project E. Audit, D. Pomarède, R. Teyssier, B. Thooris CEA/DAPNIA - Saclay

2 2D. Pomarède, CEA/DAPNIAAstronum2006 Overview of numerical simulations in Saclay Cosmological structures formation : the RAMSES code Dynamics of the interstellar medium : the HERACLES code Numerical simulations software project  data handling  visualization Perspectives Outline

3 3D. Pomarède, CEA/DAPNIAAstronum2006 Numerical simulations in Saclay An extensive program of simulations of astrophysical plasmas is conducted at the Saclay/DAPNIA Laboratory COSMOLOGICAL STRUCTURES FORMATION RAMSES STELLAR EVOLUTION ASH DYNAMICS OF THE ISM HERACLES FORMATION OF PROTO-PLANETARY DISKS FARGO It covers a vast spectrum of challenging problems at various scales and has its foundations in a suite of independent numerical codes :

4 4D. Pomarède, CEA/DAPNIAAstronum2006 Numerical simulations in Saclay : overview Cosmological structures – The formation of the large structures in the Universe – Interplay of the dark matter component and the baryon gas – RAMSES code (R. Teyssier) : N-body and hydrodynamical simulations The Interstellar Medium – The formation and dynamics of molecular clouds – Radiation Hydrodynamics – HERACLES code (E. Audit et al.) : radiative transfer coupled to hydrodynamics Stellar evolution – MHD of the Sun – Participation (A.S. Brun) to ASH developments (J. Toomre et al.) Proto-planetary systems – Study of the disk-planet tidal interactions – FARGO code (F. Masset) : 2D & 3D hydrodynamics

5 5D. Pomarède, CEA/DAPNIAAstronum2006 Numerical simulations in Saclay : overview Computing resources –CCRT (CEA Supercomputing Center) : HP/OSF1 256 quadri-processors alpha nodes Linux 128 quadri-processors AMD Opteron “Horizon Project” resources (cosmology) –IDRIS (CNRS Supercomputing Center) 1024 processors Power4 cluster –MareNostrum (Barcelona Supercomputing Center) 2406 dual 64-bit processor nodes @ 2.2GHz, ~42 TeraFlops the most powerful supercomputer in Europe Software Project –Developments are now addressed in the context of a software engineering project aiming at the optimization and sharing of expertise for various components : Algorithms parallelization/optimization Data Handling Database Post-treatment Visualization

6 6D. Pomarède, CEA/DAPNIAAstronum2006 The RAMSES code A hybrid simulation code –N-body treatment to solve the Dark Matter (DM) dynamics –Hydrodynamical treatment of the baryonic component Position of the problem : –Objective is to study the structure formation in the Universe with high spatial resolution –DM is believed to be the dominant component in mass of the cosmological density field, with only a small fraction ~10% in baryons –At intermediate scales such as galaxy clusters DM stills dominates but a gaseous component is introduced (constraints from observations of hot intracluster medium) –At smaller scales, gas-cooling and fluid dynamics play a dominant role in the structure of galaxy-size objects at 1st order, baryons = hydrostatic ionized plasma trapped in DM gravitational potential wells complex hydrodynamical processes require accurate treatment :  shock heating  atomic radiation cooling  star formation

7 7D. Pomarède, CEA/DAPNIAAstronum2006 The RAMSES code Resolution requirements –a volume of 100 Mpc h -1 side with haloes simulated with 10 resolution elements requires a spatial resolution of 10 kpc h -1 dynamical range of 10 4 –100 particles/galaxy 256 3 particles in total Algorithms –F90 parallelized with MPI or OpenMP –Based entirely on Adaptive Mesh Refinement (AMR) technique –Tree based data structure allowing recursive refinements on a cell- by-cell basis –DM particles considered as a collisionless N-body system described by the Vlasov-Poisson equations –Hydrodynamical solver based on a 2 nd -order Godunov method –With 4.1  10 7 cells a formal resolution of 8192 3 is reached at the 14 th Level Ref : R. Teyssier, A&A 385, 337-364 (2002)

8 The RAMSES AMR level 2 level 3 level 5 level 9 level 14 basic element of AMR structure : group of 2 dim sibling cells called “octs” level 11

9 The RAMSES AMR level 9 to level 14 highest granularity reach 2 13 =8192 cells in each direction

10 RAMSES simulations of cosmological structures Recent results obtained with RAMSES : “The History of the Baryon Budget – Cosmic Logistics in a Hierarchical Universe”, Y. Rasera and R. Teyssier, accepted for publication in A&A, astro-ph/0505473 (23 May 2005) “Kinematic Dynamos using Constrained Transport with High Order Godunov Schemes and Adaptive Mesh Refinement”, R. Teyssier, S. Fromang, E. Dormy, accepted for publication in Journal of Computa- tional Physics, astro-ph/0601715 (31 Jan 06) extension to the modeling of magnetic field evolution (MHD) gas density zooming sequence : from large scale filamentary structures to spiral discs

11 11D. Pomarède, CEA/DAPNIAAstronum2006 The HERACLES code A 3-D RHD code written in F90 parallelized with MPI Implemented in Cartesian, cylindrical, and spherical coordinates In current implementation only regular mesh grids Solves the equations of radiative transfer coupled to hydrodynamics :  fluid evolution is determined by the classical conservations equations (mass, momentum, energy)  + source terms characterizing the momentum and energy exchanges between the fluid and the radiation The transfer equations are solved by a second order Godunov type method and integrated implicitly using iterative solvers Reference : “HERACLES: a three dimensional radiation hydrodynamics code”,M. González, E. Audit, and P. Huynh, to be published

12 12D. Pomarède, CEA/DAPNIAAstronum2006 HERACLES - simulations of ISM dynamics Current studies focus on the thermal fragmentation of turbulent flows of interstellar hydrogen  neutral atomic hydrogen (HI) represents more than 50% of the ISM  molecular clouds form through the condensation of a warm neutral phase (WNM) into a cold medium (CNM) Thermal processes :  heating : photo-electric effect on small grains and poly-aromatic hydrocarbons due to the far-ultraviolet galactic radiation  cooling by molecules and various processes Initial conditions :  collision between two turbulent streams (converging flow) of WNM triggers the formation of CNM structures Resolution requirements : 4 different spatial scales are involved 1. cooling length of the WNM typically ~ 10-20 pc 2. typical size of CNM fragments ~ 0.1 pc 3. conduction length in the 10 -1 -10 -3 pc range 4. size of the shocked layer ~ 10 -3 pc dynamic range of 10 4

13 13D. Pomarède, CEA/DAPNIAAstronum2006 HERACLES - simulations of ISM dynamics First studies conducted with 2D simulations Analysis of structure morphology, internal velocity dispersion, fraction of gas in the various states in E. Audit and P. Hennebelle, A&A 443,1-13 (2005) weakly turbulent forcingvery turbulent forcing Fragments of cold, high-density gas connected by thin layers of low-density gas. Sharp thermal fronts bound the structures and connect them to the warm surrounding medium. Complex density field. The different phases are highly interwoven with pockets of warm gas embedded in filaments of cooler gas.

14 New, high resolution 3D simulations density field volume of size 15 pc grid 1200 3

15 HERACLES simulations of ISM dynamics 2D slice the limited dynamical range leads to under- estimation of the highest density reached during supersonic collisions

16 16D. Pomarède, CEA/DAPNIAAstronum2006 Numerical simulations software project Software developments are managed in the context of an indepen- dent engineering project. The objective is to provide a core of software modules useable by the various simulations tools. This includes data handling, post-treatment, visualization, parallelization optimization,… RAMSES ASH HERACLES FARGO CORE SOFTWARE MODULES I/O, Graphics, Algorithms

17 17D. Pomarède, CEA/DAPNIAAstronum2006 Data Handling A unique format has been chosen for the data produced by all the simulation codes : HDF5, the Hierarchical Data Format developed by the NCSA National Center for Supercomputing Applications. HDF5 is a general purpose library and file format for storing scientific data Two primary objects are handled :  Datasets = multidimensional array of data elements  Groups = structures Efficient storage and I/O : the library is tuned and adapted to read and write data efficiently on parallel computing systems Free, open source software, including utilities (browser) Plenty of applications have interfaces to HDF5, in particular in the graphics/analysis area :  IDL, ParaView, MATLAB, …

18 browser of HDF5 files

19 Visualization and browsing of HDF5 files in IDL

20 20D. Pomarède, CEA/DAPNIAAstronum2006 Visualization Visualization is performed in the framework of IDL Baseline is to use the object-oriented programming offered by IDL’s Object Graphics The visualization tool is a graphical widget  users act on the objects to modify their attributes through menus, droplist, sliders, buttons & dialog fields Input data : scalar and vector fields on regular grids and on AMR tree, particles Once the data are loaded in memory, various implementations of the visualization objects are proposed based on different classes of the IDL library :  IDLgrPolygon : set of polygons and vertices, to hold an iso-surface  IDLgrVolume : mapping from a 3D array of data to a 3D array of voxel colors projected to two dimensions  IDLgrPolyline : useful for hedgehog displays of vector fields  IDLgrSurface, IDLgrImage, IDLgrPlot, IDLgrSymbol Basic graphics objects are inserted in Models that can be transformed (rotated, scaled and/or translated) : this forms the basis for spatial navigability around the 3D objects on display.  IDLgrLight objects represent sources of illumination : ambient, positional, directional, spotlight Two methods of rendering objects :  hardware accelerator using OpenGL, highly efficient on a local machine  software rendering applies when running on a distant computer

21 the graphical widget menus sliders buttons droplist dialog fields surface object axis object plot object image object navigation profile Select :  density  pressure  velocity  radiative energy  radiative flux  particles  custom variable

22 Visualization of regular grids and AMR data Max grid : regular 3D 1200x1200x1200 = 1.728 10 9 cells regular 2D 10000x10000 AMR up to level 10 projected in grid 1024x1024x1024 image of 2D 10000x10000 HERACLES simulation surface of a density slice on a restricted region of the RAMSES AMR (level 7 to level 11)

23 Visualization of iso-surfaces Simulation of ISM hydrogen turbulences density field - volume of size 15 pc grid 1200 3 Interactive setting of the contour value on the density histogram

24 Visualization of AMR data Volume Projection Box of size 100 h -1 Mpc

25 Visualization of 3D vector fields RAMSES cosmological simulation box of size 100 h -1 Mpc HERACLES simulation of ISM turbulences Hedgehog display of hydrodynamical velocity field Blue semi-transparent volume = iso-surface of the density

26 Visualization of particles particle cloud display of a dark matter sample yellow semi-transparent volume = iso-surface of the hydrodynami- cal density field DM haloes

27 27D. Pomarède, CEA/DAPNIAAstronum2006 Conclusions and outlooks An extensive program of simulations of astrophysical plasmas is pursued New developments are managed in the framework of a software engineering project Solutions to current limitations will be studied : –HERACLES : implementation of multiple grid algorithm to reach the effective 10 4 mesh resolution in 3D simulations –RAMSES : algorithm optimization to improve load balancing –Visualization : optimization of memory management to access larger data sets parallelization to improve speed (fastDL/mpiDL solutions from RSI) implementation of multiple grid New projects : –Extend the use of the core software modules to FARGO and ASH –Database : development of a Virtual Observatory for cosmological simulations (Horizon project) –Openings towards other domains : HERACLES application to laser fusion and plasma physics

Download ppt "Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project."

Similar presentations

Ads by Google