1. SciDAC-II Compass SciDAC-II Compass 1 Vay - Compass 09 Boosted frame LWFA simulations J.-L. Vay, C. G. R. Geddes, E. Cormier-Michel Lawrence Berkeley National Laboratory Compass 2009 all hands meeting - Advanced Accelerators session Tech-X, Boulder, CO - Oct 5-6, 2009 SciDAC-II Compass SciDAC-II Compass

2. SciDAC-II Compass SciDAC-II Compass 2 Vay - Compass 09 space+time  x/t = ( L/l, T/  t)* 00 00 00 00 00 *J.-L. Vay, Phys. Rev. Lett. 98, 130405 (2007) Range of space and time scales spanned by two identical beams crossing each other space F 0 -center of mass frame x y x y F B -rest frame of “B”  is not invariant under the Lorentz transformation:  x/t  . There exists an “optimum” frame which minimizes it. Result is general and applies to light beams too.

3. SciDAC-II Compass SciDAC-II Compass 3 Vay - Compass 09 3D scaled simulations of a 10 GeV LWFA stage ( =0.8  m, a 0 =1, k p L=2, n e =10 19 cc, L p =1.5mm in lab) Average beam energy and CPU time vs position in lab frame 30,000s 400s speedup x75 % level agreement obtained between Warp simulations with  =1-10 instability observed at front of plasma for higher  3D Max  frame achieved in 2D and 3D limited by instability developing at front of plasma 2D full scale simulation 10 GeV LWFA stage (n e =10 17 cc,  =130)

4. SciDAC-II Compass SciDAC-II Compass 4 Vay - Compass 09 Low dispersion solver reduces instability growth but is not sufficient Longitudinal electric field from 2D simulations at plasma densities of 10 18 cc with  frame = 40 Yee field solverKarkkainen field solver Z Z x x E // (V) Instability spikes at fairly short wavelength: can we filter it out?

5. SciDAC-II Compass SciDAC-II Compass 5 Vay - Compass 09 Widening the band of the digital filter improves significantly filter applied to current density and gathered field, NOT on Maxwell EM field Smoothing 1 Z E // (GV/m) Smoothing 2 Z E // (GV/m) Smoothing 3 Z E // (GV/m) Smoothing 4 Z E // (GV/m) Result is only slightly affected, even with most aggressive filtering tested. wideband filter applied along longitudinal direction only

6. SciDAC-II Compass SciDAC-II Compass 6 Vay - Compass 09 Simulations in 1D/2D/3D at plasma densities of 10 19 cc, 10 18 cc and 10 17 cc show good agreement on (scaled) beam energy gain: Filtering allows max speedup for simulations of 10 GeV LWFA stage 2D n e = 10 19 cc  frame = 13  10 17 cc : max  frame = 130  speedup > 10,000 In 3D, ≈3h using 256 CPUs  > 1 year x 256 CPUs in lab frame!  10 16 cc : max  frame = 400  speedup > 100,000 In 2D, ≈4h using 24 CPUs  > 50 years x 24 CPUs in lab frame!

7. SciDAC-II Compass SciDAC-II Compass 7 Vay - Compass 09 Scaled 10 GeV case with beam loading and plasma taper Note: - beam not matched transversely and experienced significant losses, - => significant population of halo particles, - filtered out of the diagnostics using cutoffs in position and energy. Good agreement between runs in frames at  =1-10

8. SciDAC-II Compass SciDAC-II Compass 8 Vay - Compass 09 Started exploration of the application of mesh refinement to LWFA 2D Warp calculation of a scale 10 GeV stage in a boosted frame (  =10) level 0 level 1 level 2 level 3 E // (V/m) Test with up-to 3 levels of refinement, refining in transverse direction only (on 8 CPUs, 1D // decomposition) Transverse emittance history from single grid calculations well recovered by using coarse main grid and appropriate number of refinement patches to match single grid resolution.

9. SciDAC-II Compass SciDAC-II Compass 9 Vay - Compass 09 Warp parallel speedup - benchmarking with Osiris (2D) Superlinear speedup up to 32,768 cores on Franklin (NERSC) on 3D 512 3 problem with particles random load (8ppc) and periodic bc. Very good agreement was obtained between Warp and Osiris on 2D LWFA test case (a 0 =4).

10. SciDAC-II Compass SciDAC-II Compass 10 Vay - Compass 09 Transfer of techniques between SciDAC Compass groups Discussion between Osiris, Vorpal and Warp development teams has been very active and exceptionally open on a range of key topics: – macroparticles shape factors and smoothing of current densities and fields, – low dispersion (Karkkainen, Cowan) field solvers, – mesh refinement. This enabled faster progress than would have been independently attained, notably concerning the challenging boosted frame simulations. Benchmarking on LWFA problems has been performed: – in 2D and 3D between Osiris and Vorpal, – in 2D between Osiris and Warp.

11. SciDAC-II Compass SciDAC-II Compass 11 Vay - Compass 09 Plans Pursue detailed benchmarking of calculations in boosted frames against calculations in laboratory frame and/or using different models. Benchmark Osiris, Vorpal and Warp on boosted frame calculations. Continue exploration of the application of mesh refinement for LWFA. Assess gains of performing fluid/envelope calculations in boosted frames.

12. SciDAC-II Compass SciDAC-II Compass 12 Vay - Compass 09 Backup

13. SciDAC-II Compass SciDAC-II Compass Wideband low pass filter based on bilinear digital filter We use 4xbilinear (1/2) + compensation (-1/3) digital filtering along all dimensions. Using a stride (s) allows shifting the filtered band in the frequency domain.  j = w  j-1 +  j + w  j+1 1 + 2 w  j = w  j-s +  j + w  j+s 1 + 2 w Using a sequence of the filter with different strides allows for a low pass filter with adustable cutoff frequency.

14. SciDAC-II Compass SciDAC-II Compass Enlarged stencil* => no disp. in x,y,z * M. Karkkainen, et al., Proceedings of ICAP’06, Chamonix, France Implementation in Warp full amplitude odd-even oscillations for  t=  x/c 1-D simulation of current Heaviside step 1-D simulation of LWFA at  t=  x/c No filter 121 filter E H H E E E E E Issues for PIC: – source terms are not given, – odd-even oscillation when  t=  x/c. Stencil on H unchanged – E and H switched, => E push same as Yee, – exact charge conservation preserved in 2D & 3D with unmodified Esirkepov current deposition and implied enlarged stencil on div E. Is the instability due to Yee solver numerical dispersion errors? Implementation of a low-dispersion solver in Warp 1-2-1 filter needed at  t=  x/c

15. SciDAC-II Compass SciDAC-II Compass 15 Vay - Compass 09 2D scaled simulations of a 10 GeV class LWFA stage ( =0.8  m, a 0 =1, k p L=2, L p =1.5mm in lab) E // (GV/m) e- beam Plasma wake Laser imprint Average beam energy and CPU time vs position in lab frame 1,500s 20s speedup x75 Warp 2-D Lab frame E // (GV/m) Boosted frame (  =10)  and // electric field history in lab frame Snapshots of surface plot of // electric field in lab frame and boosted frame at  =10 Station z=0.154mmStation z=1.354mm 2D

16. SciDAC-II Compass SciDAC-II Compass 16 Vay - Compass 09 3D scaled simulations of a 10 GeV LWFA stage ( =0.8  m, a 0 =1, k p L=2, L p =1.5mm in lab) E // (GV/m) e- beam Plasma wake Laser imprint Average beam energy and CPU time vs position in lab frame 30,000s 400s speedup x75 Warp 2-D Lab frame E // (GV/m) Boosted frame (  =10)  and // electric field history in lab frame Snapshots of surface plot of // electric field in lab frame and boosted frame at  =10 Station z=0.3mmStation z=1.354mm 3D

17. SciDAC-II Compass SciDAC-II Compass 17 Vay - Compass 09 Other possible complication: inputs/outputs z’,t’=LT(z,t) frozen active Often, initial conditions known and output desired in laboratory frame –relativity of simultaneity => inject/collect at plane(s)  to direction of boost. Injection through a moving plane in boosted frame (fix in lab frame) –fields include frozen particles, –same for laser in EM calculations. Diagnostics: collect data at a collection of planes –fixed in lab fr., moving in boosted fr., –interpolation in space and/or time, –already done routinely with Warp for comparison with experimental data, often known at given stations in lab.

18. SciDAC-II Compass SciDAC-II Compass 18 Vay - Compass 09 Simulations in 1D/2D/3D at plasma densities of 10 19 cc, 10 18 cc and 10 17 cc show good agreement on (scaled) beam energy gain:  1D: max  frame = 130  speedup > 10,000  2D: max  frame = 40  speedup > 1,000  3D: max  frame = 30  speedup > 500  24h using 256 CPUs  more than one year x 256 CPUs in lab frame! Full scale simulations of a 10 GeV LWFA stage *similar work by Bruhwiler et al (Tech X), Martins et al (IST/UCLA) Max  frame achieved in 2D and 3D limited by instability developing at front of plasma Origin: numerical Cerenkov? 2D n e = 10 19 cc  frame = 13

19. SciDAC-II Compass SciDAC-II Compass 19 Vay - Compass 09 BELLA PW laser 40 J / 40 fs BELLA 40 J PW Laser – Components for a Laser Plasma Collider Injection + Staging 10 GeV stages Energy spread & Emittance preservation Simulating 10 GeV stages explicitly (PIC) in lab frame needs ~1G CPU  hours  impractical* Predictions have relied on theory, reduced models (fluid, envelope, quasistatic), scaling: –Energy gain  n -1 : 100 MeV at 10 19 /cc  10 GeV at 10 17 /cc –Length  n -3/2 : 1mm at 10 19 /cc  1m at 10 17 /cc –Gradient  n 1/2 : 100 GV/m at 10 19 /cc  10 GV/m at 10 17 /cc Can simulations of full scale 10 GeV stages be practical using a Lorentz boosted ref. frame? –difficulty: backward emitted radiation frequency upshifted in boosted frame, (noise, instabilities). * Cormier-Michel et al, Proc. AAC 2008; Geddes et al, Proc. PAC’09 Positron acceleration + PWFA expt.’s Radiation sources

20. SciDAC-II Compass SciDAC-II Compass 20 Vay - Compass 09 boosted frame 10km 10cm 10km/10cm=100,000. 10m 10m/1nm=10,000,000,000. 1nm 3cm 3cm/1  m=30,000. 1  m compaction x10 3 450m 4.5m frame  ≈ 22 450m/4.5m=100. 1nm compaction x3.10 7 2.5mm 4m4m 2.5mm/4  m=625. frame  ≈ 4000 compaction x560 1.6mm 30  m 1.6mm/30  m=53. frame  ≈ 19 Lorentz transformation => large level of compaction of scales range HEP accelerators (e-cloud) Laser-plasma acceleration Hendrik Lorentz Free electron lasers