Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Framework for Particle Advection for Very Large Data Hank Childs, LBNL/UCDavis David Pugmire, ORNL Christoph Garth, Kaiserslautern David Camp, LBNL/UCDavis.

Similar presentations


Presentation on theme: "A Framework for Particle Advection for Very Large Data Hank Childs, LBNL/UCDavis David Pugmire, ORNL Christoph Garth, Kaiserslautern David Camp, LBNL/UCDavis."— Presentation transcript:

1 A Framework for Particle Advection for Very Large Data Hank Childs, LBNL/UCDavis David Pugmire, ORNL Christoph Garth, Kaiserslautern David Camp, LBNL/UCDavis Sean Ahern, ORNL Gunther Weber, LBNL Allen Sanderson, Univ. of Utah

2 Particle advection is a foundational visualization algorithm Advecting particles creates integral curves

3 Particle advection is the duct tape of the visualization world Advecting particles is essential to understanding flow and other phenomena (e.g. magnetic fields)!

4 Outline A general system for particle-advection based analysis Efficient advection of particles

5 Outline A general system for particle-advection based analysis Efficient advection of particles

6 Goal Efficient code for a variety of particle advection workloads and techniques Cognizant of use cases from 1 particle to >>10K particles. Need handling of every particle, every evaluation to be efficient. Want to support many diverse flow techniques: flexibility/extensibility is key. Fit within data flow network design (i.e. a filter)

7 Motivating examples of system FTLE Stream surfaces Streamline Dynamical Systems (e.g. Poincaré Maps) Statistics based analysis + more

8 Design PICS filter: parallel integral curve system Execution: Instantiate particles at seed locations Step particles to form integral curves Analysis performed at each step Termination criteria evaluated for each step When all integral curves have completed, create final output

9 Design Five major types of extensibility: How to parallelize? How do you evaluate velocity field? How do you advect particles? Initial particle locations? How do you analyze the particle paths?

10 Inheritance hierarchy avtPICSFilter Streamline Filter Your derived type of PICS filter avtIntegralCurve avtStreamlineIC Your derived type of integral curve We disliked the matching inheritance scheme, but this achieved all of our design goals cleanly.

11 #1: How to parallelize? avtICAlgorithm avtParDomIC- Algorithm (parallel over data) avtParDomIC- Algorithm (parallel over data) avtSerialIC- Algorithm (parallel over seeds) avtSerialIC- Algorithm (parallel over seeds) avtMasterSlave- ICAlgorithm

12 #2: Evaluating velocity field avtIVPField avtIVPVTKField avtIVPVTK- TimeVarying- Field avtIVPM3DC1 Field avtIVP- Higher Order-Field IVP = initial value problem

13 #3: How do you advect particles? avtIVPSolver avtIVPDopri5 avtIVPEuler avtIVPLeapfrog avtIVP- M3DC1Integrator IVP = initial value problem avtIVPAdams- Bashforth

14 #4: Initial particle locations avtPICSFilter::GetInitialLocations() = 0;

15 #5: How do you analyze particle path? avtIntegralCurve::AnalyzeStep() = 0; All AnalyzeStep will evaluate termination criteria avtPICSFilter::CreateIntegralCurveOutput( std::vector &) = 0; Examples: Streamline: store location and scalars for current step in data members Poincare: store location for current step in data members FTLE: only store location of final step, no-op for preceding steps NOTE: these derived types create very different types of outputs.

16 Putting it all together PICS Filter avtICAlgorithm avtIVPSolver avtIVPField Vector< avtIntegral-Curve> Vector< avtIntegral-Curve> Integral curves sent to other processors with some derived types of avtICAlgorithm. ::CreateInitialLocations() = 0; ::AnalyzeStep() = 0;

17 Outline A general system for particle-advection based analysis Efficient advection of particles

18 Advecting particles Decomposition of large data set into blocks on filesystem ? What is the right strategy for getting particle and data together?

19 Strategy: load blocks necessary for advection Decomposition of large data set into blocks on filesystem Go to filesystem and read block

20 Decomposition of large data set into blocks on filesystem Strategy: load blocks necessary for advection

21 Parallelize over Particles Basic idea: particles are partitioned over PEs, blocks of data are loaded as needed. Positives: Indifferent to data size Trivial parallelization (partition particles over processors) Negative: Redundant I/O (both across PEs and within a PE) is a significant problem.

22 Parallelize over data strategy: parallelize over blocks and pass particles PE1 PE2 PE4 PE3

23 Parallelize over Data Basic idea: data is partitioned over PEs, particles are communicated as needed. Positives: Only load data one time Negative: Starvation!

24 Contracts are needed to enable these different processing techniques Parallelize-over-seeds One execution per block Only limited data available at one time Parallelize-over-data One execution total Entire data set available at one time

25 Both parallelization schemes have serious flaws. Two approaches: Parallelizing OverI/OEfficiency DataGoodBad ParticlesBadGood Parallelize over particles Parallelize over data Hybrid algorithms

26 The master-slave algorithm is an example of a hybrid technique. Uses master-slave model of communication One process has unidirectional control over other processes Algorithm adapts during runtime to avoid pitfalls of parallelize-over-data and parallelize-over-particles. Nice property for production visualization tools. (Implemented in VisIt ) D. Pugmire, H. Childs, C. Garth, S. Ahern, G. Weber, Scalable Computation of Streamlines on Very Large Datasets. SC09, Portland, OR, November, 2009

27 Master-Slave Hybrid Algorithm Divide PEs into groups of N Uniformly distribute seed points to each group Master: - Monitor workload - Make decisions to optimize resource utilization Slaves: - Respond to commands from Master - Report status when work complete SlaveSlave SlaveSlave SlaveSlave MasterMaster SlaveSlave SlaveSlave SlaveSlave MasterMaster SlaveSlave SlaveSlave SlaveSlave MasterMaster SlaveSlave SlaveSlave SlaveSlave MasterMaster P0P1P2P3P0P1P2P3P4P5P6P7P4P5P6P7P8P9P10P11P8P9P10P11P12P13P14P15P12P13P14P15

28 Master Process Pseudocode Master() { while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand() for cmd in commands SendCommandToSlaves( cmd ) } What are the possible commands?

29 Commands that can be issued by master Master Slave Slave is given a particle that is contained in a block that is already loaded 1. Assign / Loaded Block 2. Assign / Unloaded Block 3. Handle OOB / Load 4. Handle OOB / Send OOB = out of bounds

30 Master Slave Slave is given a particle and loads the block Commands that can be issued by master 1. Assign / Loaded Block 2. Assign / Unloaded Block 3. Handle OOB / Load 4. Handle OOB / Send OOB = out of bounds

31 Master Slave Load Slave is instructed to load a block. The particle advection in that block can then proceed. Commands that can be issued by master 1. Assign / Loaded Block 2. Assign / Unloaded Block 3. Handle OOB / Load 4. Handle OOB / Send OOB = out of bounds

32 Master Slave Send to J Slave J Slave is instructed to send the particle to another slave that has loaded the block Commands that can be issued by master 1. Assign / Loaded Block 2. Assign / Unloaded Block 3. Handle OOB / Load 4. Handle OOB / Send OOB = out of bounds

33 Master Process Pseudocode Master() { while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand() for cmd in commands SendCommandToSlaves( cmd ) }

34 Driving factors… You dont want PEs to sit idle (The problem with parallelize-over-data) You dont want PEs to spend all their time doing I/O (The problem with parallelize-over-particles) you want to do least amount of I/O that will prevent excessive idleness.

35 Heuristics If no Slave has the block, then load that block! If some Slave does have the block… If that Slave is not busy, then have it process the particle. If that Slave is busy, then wait a while. If youve waited a long time, have another Slave load the block and process the particle.

36 Master-slave in action S0 S1 S2 S3 S4 IterationAction 0S0 reads B0, S3 reads B1 1S1 passes points to S0, S4 passes points to S3, S2 reads B0 0: Read 1: Pass 1: Read

37 Algorithm Test Cases - Core collapse supernova simulation - Magnetic confinement fusion simulation - Hydraulic flow simulation

38 Workload distribution in parallelize-over-data Starvation

39 Workload distribution in parallelize-over- particles Too much I/O

40 Workload distribution in master-slave algorithm Just right

41 ParticlesData Hybrid Workload distribution in supernova simulation Parallelization by: Colored by PE doing integration

42 Astrophysics Test Case: Total time to compute 20,000 Streamlines Seconds Number of procs Uniform Seeding Non-uniform Seeding Data HybridParticles

43 Astrophysics Test Case: Number of blocks loaded Blocks loaded Number of procs DataHybrid Uniform Seeding Non-uniform Seeding Particles

44 Summary: Master-Slave Algorithm First ever attempt at a hybrid algorithm for particle advection Algorithm adapts during runtime to avoid pitfalls of parallelize-over-data and parallelize-over- particles. Nice property for production visualization tools. Implemented inside VisIt visualization and analysis package.

45 Final thoughts… Summary: Particle advection is important for understanding flow and efficiently parallelizing this computation is difficult. We have developed a freely available system for doing this analysis for large data. Documentation: (PICS) (VisIt) Future work: UI extensions, including Python Additional analysis techniques (FTLE & more)

46 Acknowledgements Funding: This work was supported by the Director, Office of Science, Office and Advanced Scientific Computing Research, of the U.S. Department of Energy under Contract No. DE- AC02-05CH11231 through the Scientific Discovery through Advanced Computing (SciDAC) program's Visualization and Analytics Center for Enabling Technologies (VACET). Program Manager: Lucy Nowell Master-Slave Algorithm: Dave Pugmire (ORNL), Hank Childs (LBNL/UCD), Christoph Garth (Kaiserslautern), Sean Ahern (ORNL), and Gunther Weber (LBNL) PICS framework: Hank Childs (LBNL/UCD), Dave Pugmire (ORNL), Christoph Garth (Kaiserslautern), David Camp (LBNL/UCD), Allen Sanderson (Univ of Utah)

47 A Framework for Particle Advection for Very Large Data Hank Childs, LBNL/UCDavis David Pugmire, ORNL Christoph Garth, Kaiserslautern David Camp, LBNL/UCDavis Sean Ahern, ORNL Gunther Weber, LBNL Allen Sanderson, Univ. of Utah Thank you!!


Download ppt "A Framework for Particle Advection for Very Large Data Hank Childs, LBNL/UCDavis David Pugmire, ORNL Christoph Garth, Kaiserslautern David Camp, LBNL/UCDavis."

Similar presentations


Ads by Google