Download presentation
Presentation is loading. Please wait.
1
4/30/04LSU 2004 Cactus Retreat1 Toward Relativistic Hydrodynamics on Adaptive Meshes Joel E. Tohline Louisiana State University http://www.phys.lsu.edu/~tohline
2
4/30/04LSU 2004 Cactus Retreat2 Principal Collaborators Simulations to be shown today: –Shangli Ou – (LSU) –Mario D’Souza – (LSU) –Michele Vallisneri – (Caltech/JPL) Code development over the years: –John Woodward – (Valtech; Dallas, Texas) –John Cazes – (Stennis Space Center; Stennis, Mississippi) –Patrick Motl – (Colorado) Science: –Juhan Frank (LSU) –Lee Lindblom (Caltech) –Luis Lehner (LSU) –Jorge Pullin (LSU)
3
4/30/04LSU 2004 Cactus Retreat3 Show 3 Movies Nonlinear development of the r-mode in young neutron stars [w/ Lindblom & Vallisneri] http://www.cacr.caltech.edu/projects/hydrligo/rmode.html Nonlinear development of the secular bar-mode instability in rapidly rotating neutron stars [w/ Ou & Lindblom] http://paris.phys.lsu.edu/~ou/movie/fmode/new/fmode.b181.om4.2e5.mov Mass-transferring binary star systems [w/ D’Souza, Motl, & Frank] http://paris.phys.lsu.edu/~mario/models/q_0.409_no_drag_3.8orbs/movies/q_0.409_no_drag_3.8orbs_top.mov
![4/30/04LSU 2004 Cactus Retreat3 Show 3 Movies Nonlinear development of the r-mode in young neutron stars [w/ Lindblom & Vallisneri] Nonlinear development of the secular bar-mode instability in rapidly rotating neutron stars [w/ Ou & Lindblom] Mass-transferring binary star systems [w/ D’Souza, Motl, & Frank]](http://images.slideplayer.com/24/7362143/slides/slide_3.jpg "4/30/04LSU 2004 Cactus Retreat3 Show 3 Movies Nonlinear development of the r-mode in young neutron stars [w/ Lindblom & Vallisneri] Nonlinear development of the secular bar-mode instability in rapidly rotating neutron stars [w/ Ou & Lindblom] Mass-transferring binary star systems [w/ D’Souza, Motl, & Frank]")
4
4/30/04LSU 2004 Cactus Retreat4 Storyline Present Algorithm – has been producing publishable astrophysical results for over 20 years: –Entirely home-grown code outside of Cactus environment –Manual domain decomposition –Explicit message-passing using mpi –Visualizations on serial machines (generally, post-processing) Plans for this calendar year: –Move present algorithm into Cactus environment Over the next few years, modify algorithm to: –Follow relativistic hydrodynamical flows on adaptive mesh –Accept evolving space-time metric –Visualize results “in parallel” with dynamical evolution
5
4/30/04LSU 2004 Cactus Retreat5 Storyline Present Algorithm – has been producing publishable astrophysical results for over 20 years: –Entirely home-grown code outside of Cactus environment –Manual domain decomposition –Explicit message-passing using mpi –Visualizations on serial machines (generally, post-processing) Plans for this calendar year: –Move present algorithm into Cactus environment Over the next few years, modify algorithm to: –Follow relativistic hydrodynamical flows on adaptive mesh –Accept evolving space-time metric –Visualize results “in parallel” with dynamical evolution
6
4/30/04LSU 2004 Cactus Retreat6 Storyline Present Algorithm – has been producing publishable astrophysical results for over 20 years: –Entirely home-grown code outside of Cactus environment –Manual domain decomposition –Explicit message-passing using mpi –Visualizations on serial machines (generally, post-processing) Plans for this calendar year: –Move present algorithm into Cactus environment Over the next few years, modify algorithm to: –Follow relativistic hydrodynamical flows on adaptive mesh –Accept evolving space-time metric –Visualize results “in parallel” with dynamical evolution
7
4/30/04LSU 2004 Cactus Retreat7 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results
![4/30/04LSU 2004 Cactus Retreat7 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results](http://images.slideplayer.com/24/7362143/slides/slide_7.jpg "4/30/04LSU 2004 Cactus Retreat7 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results")
8
4/30/04LSU 2004 Cactus Retreat8 Principal Governing Equations
9
4/30/04LSU 2004 Cactus Retreat9 Principal Governing Equations
10
4/30/04LSU 2004 Cactus Retreat10 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results
![4/30/04LSU 2004 Cactus Retreat10 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results](http://images.slideplayer.com/24/7362143/slides/slide_10.jpg "4/30/04LSU 2004 Cactus Retreat10 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results")
11
4/30/04LSU 2004 Cactus Retreat11 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results
![4/30/04LSU 2004 Cactus Retreat11 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results](http://images.slideplayer.com/24/7362143/slides/slide_11.jpg "4/30/04LSU 2004 Cactus Retreat11 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results")
12
4/30/04LSU 2004 Cactus Retreat12 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Serial
![4/30/04LSU 2004 Cactus Retreat12 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Serial](http://images.slideplayer.com/24/7362143/slides/slide_12.jpg "4/30/04LSU 2004 Cactus Retreat12 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Serial")
13
4/30/04LSU 2004 Cactus Retreat13 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Parallel
![4/30/04LSU 2004 Cactus Retreat13 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Parallel](http://images.slideplayer.com/24/7362143/slides/slide_13.jpg "4/30/04LSU 2004 Cactus Retreat13 Present Algorithm Select grid structure and resolution Construct initial configuration Perform domain decomposition While t < t stop –Determine Newtonian gravitational accelerations –Push fluid around on the grid using Newtonian dynamics –If mod[ t, (orbital period/80) ] = 0 Dump 3-D dataset for later visualization –EndIf EndWhile Visualize results Parallel")
14
4/30/04LSU 2004 Cactus Retreat14 Parallel Code’s Chronological Evolution John Woodward: –8,192-processor MasPar @ LSU John Cazes: –CM5 @ NCSA; T3D/E @ SDSC Patrick Motl: –mpi on T3E @ SDSC; SP2/3 @ SDSC & LSU Michele Vallisneri: –HP Exemplar @ CACR Mario D’Souza & Shangli Ou: –SuperMike (1,024-proc Linux cluster) @ LSU Shangli Ou: –Tungsten (2,560-proc Linux cluster) @ NCSA Early 90’s Mid-90’s Late 90’s 2000 2002-03 2004
15
4/30/04LSU 2004 Cactus Retreat15 Select Grid Structure and Resolution Unigrid, cylindrical mesh Fixed in time Typical resolution –Single star: 66 x 128 x 130 (as shown on the left) –Binary system: 192 x 256 x 98
16
4/30/04LSU 2004 Cactus Retreat16 Select Grid Structure and Resolution
17
4/30/04LSU 2004 Cactus Retreat17 Need for Non-unigrid and Adaptive Meshes
18
4/30/04LSU 2004 Cactus Retreat18 Perform Domain Decomposition Grid resolution 192 x 256 x 96 64 processors Distribute 192 x 96 (R,Z) grid across 8 x 8 processor array Leave angular zones “stacked” in memory Result: Each processor has data arrays of size 24 x 256 x 12 I/O: Scramble and unscramble handled manually Z R
19
4/30/04LSU 2004 Cactus Retreat19 Determine Newtonian Gravitational Accelerations (Three-dimensional Elliptic PDE on cylindrical mesh)
20
4/30/04LSU 2004 Cactus Retreat20 Principal Governing Equations
21
4/30/04LSU 2004 Cactus Retreat21 Determine Newtonian Gravitational Accelerations (Three-dimensional Elliptic PDE on cylindrical mesh) Perform FFT (in memory) in azimuthal coordinate direction reduce to decoupled set of (256) two-dimensional Helmholtz equations. Use ADI (alternating direction implicit) to solve each 2-D equation: –Data transpose –1-D, in-memory ADI sweep –Data transpose –1-D, in-memory ADI sweep –Data transpose –Etc. Inverse FFT Z R
22
4/30/04LSU 2004 Cactus Retreat22 Determine Newtonian Gravitational Accelerations (Three-dimensional Elliptic PDE on cylindrical mesh) Perform FFT (in memory) in azimuthal coordinate direction reduce to decoupled set of (256) two-dimensional Helmholtz equations. Use ADI (alternating direction implicit) to solve each 2-D equation: –Data transpose –1-D, in-memory ADI sweep –Data transpose –1-D, in-memory ADI sweep –Data transpose –Etc. Inverse FFT Z m
23
4/30/04LSU 2004 Cactus Retreat23 Determine Newtonian Gravitational Accelerations (Three-dimensional Elliptic PDE on cylindrical mesh) Perform FFT (in memory) in azimuthal coordinate direction reduce to decoupled set of (256) two-dimensional Helmholtz equations. Use ADI (alternating direction implicit) to solve each 2-D equation: –Data transpose –1-D, in-memory ADI sweep –Data transpose –1-D, in-memory ADI sweep –Data transpose –Etc. Inverse FFT R m
24
4/30/04LSU 2004 Cactus Retreat24 Visualize Results Specify isodensity surface(s) Find vertices and polygons on each surface (using marching cubes algorithm) Write out vertices & polygons in “OBJ” format Delete 3-D dataset Utilize “Maya” to render nested surfaces (from pre-specified viewer orientation) Write out TIFF image (typically 640 x 480) Generate.mov
25
4/30/04LSU 2004 Cactus Retreat25 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn “Maya” rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results
![4/30/04LSU 2004 Cactus Retreat25 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn Maya rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results](http://images.slideplayer.com/24/7362143/slides/slide_25.jpg "4/30/04LSU 2004 Cactus Retreat25 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn Maya rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results")
26
4/30/04LSU 2004 Cactus Retreat26 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn “Maya” rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results
![4/30/04LSU 2004 Cactus Retreat26 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn Maya rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results](http://images.slideplayer.com/24/7362143/slides/slide_26.jpg "4/30/04LSU 2004 Cactus Retreat26 Future Algorithm Select grid structure and resolution and [preferred AMR thorn] [We] Construct initial configuration [Let Cactus] Perform domain decomposition While t < t stop –[Call GR Group’s thorn] Determine structure of space-time metric –[We (or Whisky thorn)] Push fluid around on the grid using Relativistic dynamics –If mod[ t, (orbital period/80) ] = 0 Generate vertices and polygons in parallel Spawn Maya rendering task on additional processor(s) –EndIf –[Call AMR thorn] Modify mesh, as necessary EndWhile [Use Cactus thorn] Write article and Publish results")
27
4/30/04LSU 2004 Cactus Retreat27 THE END
Similar presentations
