1. USC Progress Report: 3D EMPIC Simulations of Whistler Turbulence I. Modeling Nonlinear Evolution of Whistler Turbulence: Local Simulation II. Modeling Whistler Wave/Turbulence Propagation: Toward a Global Simulation Model Joseph Wang, Ouliang Chang Department of Astronautical Engineering University of Southern California Acknowledgement: S. Peter Gary, Los Alamos National Lab Guru Ganguli, Naval Research Lab

2. Summary of Recent Work Extension of previous study on ring-beam induced instabilities Cross comparison/validation: EMPIC simulations of lower hybrid wave/instability vs. Winske’s hybrid ES simulation 3D EMPIC simulations of whistler turbulence evolution Development of a global EMPIC simulation of whistler wave/turbulence propagation This talk

3. I. I. Modeling Nonlinear Evolution of Whistler Turbulence: Local Simulation

4. Introduction Two recent studies on the evolution of whistler turbulence: Saito et al (2008), Gary et al (2010): 2D EMPIC simulation showed the evolution is dominated by forward cascade from long to short wavelengths Ganguli et al (2010): Whistler turbulence is fundamentally a 3-D phenomena Pseudo-3D EMPIC simulation which includes the effect showed that the evolution is dominated by inverse cascade from short to long wavelengths. 2-D simulations may not allow the development of important nonlinear wave- wave processes Objective: To perform 3D EMPIC simulations of whistler turbulence and to investigate the “forward cascade vs inverse cascade” issue in a fully 3D setup

5. Previous Work [Ganguli et al., 2010] pseudo-3D; whistler instability launched by a ring beam. 0θ   Ongoing Work fully 3D; whistler instability launched by a prescribed spectrum of whistlers z ( B o, ||) x y Initial simulations focus on 3D vs. 2D using the Satio-Gary initial condition Larger simulations to determine whether the forward cascade or the inverse cascade is going to “win” are currently running Due to computer limitations: this talk addresses the intermediate questions: 1)How does the 3D forward cascade compare against the 2D forward cascade 2)Is there an inverse cascade in a fully 3D setup of Saito-Gary initial condition

6. Computational Challenge to Perform a Fully 3D EMPIC Simulation of Whistler Turbulence Simulation Parameters to extend Saito et al (2008) 2D setup to full 3D Memory Requirement Estimation: 8TB Particle per Cell: ~64 particles per cell per species Cell resolution: dx= dy=1 Debye length=0.1c/omega_pe Domain size: 1024X1024X1024>10 9 Total # of particles: >128X10 9 Computing Time Estimation on USC HPC: >35 days on 256 nodes dt=0.05*(omega_pe) -1 CPU Time/particle/step: ~2.75E-7s Total CPU time for 20000 steps (447Ω ce -1 ): : >8770 day USC High Performance Computing Center Parallel Supercomputer: typical node: quad-core/dual processor typical memory each node: 12-16GB theoretical peak speed: 83.29teraflops on 1460nodes processor availability: <256 node for each run

7. Parallel Simulation Code Development Code development, optimization, and validation completed Implementation using hybrid MPI+OpenMP New particle sorting algorithm to enhance data locality and speed up the computation Various initial loading of whistler waves for pre-conditioned simulation Code Validation against pervious simulations

8. Simulation Setup Initial condition: Initial waves: where 2D configuration: 42 models calculated from S. Peter Gary’s dispersion solver On a 2D grid containing the background magnetic field, lay out 42 modes with k_|| and k_perp calculated from dispersion solver. Put the appropriate amount of magnetic energy in each component for each mode based on dispersion solver. Each mode should have the same total magnetic energy density, by assumption. Use Faraday's Law to get the appropriate amount of electric field energy in each component for each mode. Use Ampere's Law to compute the components of the fluctuating electron velocities for each mode, which is imposed as a perturbation of the average velocity of the electron velocity distribution. 3D configuration: 84 modes rotate the plane containing the wave vectors 90 degrees about B o, and repeat the above process

9. Simulation Cases Case 0: code validation. based on Saito et al (2008) Physical parameters: 2D 102.4 λ e, ω pe /Ω ce =2.236, β e =0.1, Te=Tp, c/v te =10 Cell=1024X1024; 64particle/cell/species; total particle=134.2million Case 1: to compare 3D forward cascade vs. 2D forward cascade Parameters: 51.2 λ e, ω pe /Ω ce =0.707, β e =0.01, Te=Tp, c/v te =10 1a 2D:Cell=512X512; total particle=33million 1b 3D: Cell=512X512X512; 32particle/cell/species; total particle=8.59E9 Case 2: to determine whether inverse cascade exists in the Saito-Gary setup Parameters: 51.2 λ e, ω pe /Ω ce =0.707, β e =0.01, Te=Tp, c/v te =10 2a: 2D. Cell=512X512; total particle=33million 2b: 3D. Cell=512X512X512; 32particle/cell/species; total particle=8.59E9 Case 3: forward cascade vs. inverse cascade (ongoing) Parameters: 102.4 λ e, ω pe /Ω ce =0.707, β e =0.01, Te=Tp, c/v te =10 3D. Cell=1024X1024X1024; 64particle/cell/species; total particle=137E9 Mi/Me=1836

10. Simulation 0: Code Validation Gary et al 2008 Ω ce *t=2011 Saito et al 2008 Ω ce *t=447 K || λ e KyλeKyλe Log(δB^2/B 0 ^2)

11. Simulation 1: 2D Ω ce *t=1414Ω ce *t=0 Log(δB^2/B 0 ^2) Case 1a 2D: Ω ce *t=1414Ω ce *t=0Ω ce *t=1414Ω ce *t=0

12. Simulation 1: 3D Case 1b 3D: Y-Z Plane X-Z Plane X-Y Plane at Ω ce *t=141.4 Log(δB^2/B 0 ^2)

13. Ω ce *t=0 Log(δB^2/B 0 ^2) Case 2a 2D: Ω ce *t=1414Ω ce *t=0 Simulation 2: 2D Ω ce *t=141.4

14. Ω ce *t=0 Log(δB^2/B 0 ^2) Case 2b 3D: Simulation 2: 3D both forward cascade and inverse cascade seem to exist in the 3D Saito-Gary setup; however, which process is going to “win” under realistic magnetospheric parameters remain to be answered

15. II. II. Modeling Whistler Wave/Turbulence: Toward Global Simulation

16. Introduction All simulation studies on whistler turbulence so far are based on local simulations with periodic BC Local simulations do not allow the study of global characteristics of whistler turbulence associated with whistler emission and propagation Objective: To develop a global simulation model for studying anomalous absorption of whistler waves injected by a transmitter This talk: Development and testing of an EMPIC code with absorbing BC for whistler waves

17. Algorithm: Wave Absorption at Boundary Effective damping region Outgoing waves at simulation domain boundary are absorbed using a damping region scheme (Umeda, Comp. Phys. Comm., 2001) In the damping region, an amplitude damping factor is used to gradually reduce the amplitude (energy) of outgoing waves at each time step and a phase retarding factor is used to gradually reduce the propagation speed of outgoing waves (and plasmas).

18. Code Testing: 2D Perturbations in Maxwellian Plasma Initial Condition: 2D sinusoidal wave propagating in X direction. Ey(i,j,k)=Bo*sin(re/20*2*pi); Bz(i,j,k)=Bo*sin(rm/20*2*pi) re, rm are distance to center point of 2D domain plain wave length: 20cell Background Plasma: Background magnetic field in the Z direction Speed of light c =8 ve_th=1 vi_th=0.125 v_d=0 Domain and Wave Damping Region: damping region: 40 cell; Domain size: 280X280X1; Boundary Condition Field: Wave absorption Particle: injection and absorption Injection: one-sided flux of background distribution

19. 2D Perturbations in Maxwellian Plasma Bz animation) Oscillating Bz 2D Perturbation propagation in X direction

20. Initial Simulation Results: Whistler Propagation in Open Domain 2D open (absorbing) boundary condition, cell size 300*1*512 Open BC in X and Z, periodic BC in Y Absorbing region size: 100 cells in each side Background B in Z direction (in simulation plain) Y Z X BoBo 100 312

21. Simulation Parameters

22. Whistler Wave Initial whistler mode loaded in X-Y direction Dispersion relation Choose k, assuming o tkzj y t j x t j y t j x BEe E Be E jB ejEEeEE 101010 1 11 11 /./., )()( )()(     ωω ωω ηη

23. Whistler Propagation In Vacuum (No Plasma) Propagation of B x Field Energy Comparison Totoal time: t*omega_pe=40 Time interval: dt*omega_pe=0.8

24. Whistler Propagation In Vacuum (No Plasma) Energy Frequency Spectrum (B x )Energy Frequency Spectrum (E x ) Energy Wavevector Spectrum (B x ) Consistent with initial loaded fluctuation Wave number spectrum at t*omega_pe=20 Totoal time: t*omega_pe=40

25. By Field Energy Comparison Ex Totoal time: t*omega_pe=120 Time interval: dt*omega_pe=0.8

27. Energy Frequency Spectrum (B y ) 120 Energy Frequency Spectrum (E x ) Energy Wavevector Spectrum (B y ) Whistler mode Plasma mode Consistent with initial loaded fluctuation Magnetic field frequency and wavenumber match the whistler dispersion relation. Electric field is dominated by plasma mode. Wave number spectrum taken at t*omega_pe=50 Totoal time: t*omega_pe=120

28. By Field Energy Comparison Ex Totoal time: t*omega_pe=200 Time interval: dt*omega_pe=0.8

29. Energy Frequency Spectrum (B y ) 200 Energy Frequency Spectrum (E x ) Energy Wavevector Spectrum (B y ) Whistler mode Plasma mode Consistent with initial loaded fluctuation Magnetic field frequency and wavenumber match the whistler dispersion relation. Electric field shows both whistler mode and plasma mode. Totoal time: t*omega_pe=200 Wave number spectrum at t*omega_pe=50

30. By Field Energy Comparison Ex Magnetic and electric field propagation become turbulent in later time. Totoal time: t*omega_pe=150 Time interval: dt*omega_pe=0.8

31. Energy Frequency Spectrum (B y ) 150 Energy Frequency Spectrum (E x ) Energy Wavevector Spectrum (B y ) Whistler mode Plasma mode Consistent with initial loaded fluctuation Whistler mode Magnetic field frequency and wavenumber match the whistler dispersion relation. Electric field shows that whistler mode is dominating. Totoal time: t*omega_pe=150 Wave number spectrum at t*omega_pe=50

32. Summary and Conclusions local simulation: –new parallel empic code has been implemented and optimized for 3D simulations of whistler turbulence evolution –First fully 3D EMPIC simulation of whistler turbulence carried out –Initial results (run time: days) showed both forward and inverse cascade using the Saito-Gary initial condition –Larger scale simulations using more than 10 9 cells and 10 11 particles (run time: weeks) to resolve the “forward cascade vs. Inverse cascade” issue are ongoing global simulation: –new subroutines for wave absorption are developed and tested for simulation of whistler turbulence associated with whistler emission from a transmitter in open space –Future work will develop a whistler emission model and study whistler turbulence evolution within the context of emission/ propagation

33. Simulation Case 1a Case 1a 2D: Dash line: (f(v,t) – f(v,t=0))/f(0,t=0)

34. Simulation Case 2a Case 2a 2D: Dash line: (f(v,t) – f(v,t=0))/f(0,t=0)