SciDAC-II Compass SciDAC-II Compass 1 Vay - Compass 09 Boosted frame LWFA simulations J.-L. Vay, C. G. R. Geddes, E. Cormier-Michel Lawrence Berkeley National Laboratory Compass 2009 all hands meeting - Advanced Accelerators session Tech-X, Boulder, CO - Oct 5-6, 2009 SciDAC-II Compass SciDAC-II Compass

SciDAC-II Compass SciDAC-II Compass 2 Vay - Compass 09 space+time  x/t = ( L/l, T/  t)* 00 00 00 00 00 *J.-L. Vay, Phys. Rev. Lett. 98, (2007) Range of space and time scales spanned by two identical beams crossing each other space F 0 -center of mass frame x y x y F B -rest frame of “B”  is not invariant under the Lorentz transformation:  x/t  . There exists an “optimum” frame which minimizes it. Result is general and applies to light beams too.

SciDAC-II Compass SciDAC-II Compass 3 Vay - Compass 09 3D scaled simulations of a 10 GeV LWFA stage ( =0.8  m, a 0 =1, k p L=2, n e =10 19 cc, L p =1.5mm in lab) Average beam energy and CPU time vs position in lab frame 30,000s 400s speedup x75 % level agreement obtained between Warp simulations with  =1-10 instability observed at front of plasma for higher  3D Max  frame achieved in 2D and 3D limited by instability developing at front of plasma 2D full scale simulation 10 GeV LWFA stage (n e =10 17 cc,  =130)

SciDAC-II Compass SciDAC-II Compass 4 Vay - Compass 09 Low dispersion solver reduces instability growth but is not sufficient Longitudinal electric field from 2D simulations at plasma densities of cc with  frame = 40 Yee field solverKarkkainen field solver Z Z x x E // (V) Instability spikes at fairly short wavelength: can we filter it out?

SciDAC-II Compass SciDAC-II Compass 5 Vay - Compass 09 Widening the band of the digital filter improves significantly filter applied to current density and gathered field, NOT on Maxwell EM field Smoothing 1 Z E // (GV/m) Smoothing 2 Z E // (GV/m) Smoothing 3 Z E // (GV/m) Smoothing 4 Z E // (GV/m) Result is only slightly affected, even with most aggressive filtering tested. wideband filter applied along longitudinal direction only

SciDAC-II Compass SciDAC-II Compass 6 Vay - Compass 09 Simulations in 1D/2D/3D at plasma densities of cc, cc and cc show good agreement on (scaled) beam energy gain: Filtering allows max speedup for simulations of 10 GeV LWFA stage 2D n e = cc  frame = 13  cc : max  frame = 130  speedup > 10,000 In 3D, ≈3h using 256 CPUs  > 1 year x 256 CPUs in lab frame!  cc : max  frame = 400  speedup > 100,000 In 2D, ≈4h using 24 CPUs  > 50 years x 24 CPUs in lab frame!

SciDAC-II Compass SciDAC-II Compass 7 Vay - Compass 09 Scaled 10 GeV case with beam loading and plasma taper Note: - beam not matched transversely and experienced significant losses, - => significant population of halo particles, - filtered out of the diagnostics using cutoffs in position and energy. Good agreement between runs in frames at  =1-10

SciDAC-II Compass SciDAC-II Compass 8 Vay - Compass 09 Started exploration of the application of mesh refinement to LWFA 2D Warp calculation of a scale 10 GeV stage in a boosted frame (  =10) level 0 level 1 level 2 level 3 E // (V/m) Test with up-to 3 levels of refinement, refining in transverse direction only (on 8 CPUs, 1D // decomposition) Transverse emittance history from single grid calculations well recovered by using coarse main grid and appropriate number of refinement patches to match single grid resolution.

SciDAC-II Compass SciDAC-II Compass 9 Vay - Compass 09 Warp parallel speedup - benchmarking with Osiris (2D) Superlinear speedup up to 32,768 cores on Franklin (NERSC) on 3D problem with particles random load (8ppc) and periodic bc. Very good agreement was obtained between Warp and Osiris on 2D LWFA test case (a 0 =4).

SciDAC-II Compass SciDAC-II Compass 10 Vay - Compass 09 Transfer of techniques between SciDAC Compass groups Discussion between Osiris, Vorpal and Warp development teams has been very active and exceptionally open on a range of key topics: – macroparticles shape factors and smoothing of current densities and fields, – low dispersion (Karkkainen, Cowan) field solvers, – mesh refinement. This enabled faster progress than would have been independently attained, notably concerning the challenging boosted frame simulations. Benchmarking on LWFA problems has been performed: – in 2D and 3D between Osiris and Vorpal, – in 2D between Osiris and Warp.

SciDAC-II Compass SciDAC-II Compass 11 Vay - Compass 09 Plans Pursue detailed benchmarking of calculations in boosted frames against calculations in laboratory frame and/or using different models. Benchmark Osiris, Vorpal and Warp on boosted frame calculations. Continue exploration of the application of mesh refinement for LWFA. Assess gains of performing fluid/envelope calculations in boosted frames.

SciDAC-II Compass SciDAC-II Compass 12 Vay - Compass 09 Backup

SciDAC-II Compass SciDAC-II Compass Wideband low pass filter based on bilinear digital filter We use 4xbilinear (1/2) + compensation (-1/3) digital filtering along all dimensions. Using a stride (s) allows shifting the filtered band in the frequency domain.  j = w  j-1 +  j + w  j w  j = w  j-s +  j + w  j+s w Using a sequence of the filter with different strides allows for a low pass filter with adustable cutoff frequency.

SciDAC-II Compass SciDAC-II Compass Enlarged stencil* => no disp. in x,y,z * M. Karkkainen, et al., Proceedings of ICAP’06, Chamonix, France Implementation in Warp full amplitude odd-even oscillations for  t=  x/c 1-D simulation of current Heaviside step 1-D simulation of LWFA at  t=  x/c No filter 121 filter E H H E E E E E Issues for PIC: – source terms are not given, – odd-even oscillation when  t=  x/c. Stencil on H unchanged – E and H switched, => E push same as Yee, – exact charge conservation preserved in 2D & 3D with unmodified Esirkepov current deposition and implied enlarged stencil on div E. Is the instability due to Yee solver numerical dispersion errors? Implementation of a low-dispersion solver in Warp filter needed at  t=  x/c

SciDAC-II Compass SciDAC-II Compass 15 Vay - Compass 09 2D scaled simulations of a 10 GeV class LWFA stage ( =0.8  m, a 0 =1, k p L=2, L p =1.5mm in lab) E // (GV/m) e- beam Plasma wake Laser imprint Average beam energy and CPU time vs position in lab frame 1,500s 20s speedup x75 Warp 2-D Lab frame E // (GV/m) Boosted frame (  =10)  and // electric field history in lab frame Snapshots of surface plot of // electric field in lab frame and boosted frame at  =10 Station z=0.154mmStation z=1.354mm 2D

SciDAC-II Compass SciDAC-II Compass 16 Vay - Compass 09 3D scaled simulations of a 10 GeV LWFA stage ( =0.8  m, a 0 =1, k p L=2, L p =1.5mm in lab) E // (GV/m) e- beam Plasma wake Laser imprint Average beam energy and CPU time vs position in lab frame 30,000s 400s speedup x75 Warp 2-D Lab frame E // (GV/m) Boosted frame (  =10)  and // electric field history in lab frame Snapshots of surface plot of // electric field in lab frame and boosted frame at  =10 Station z=0.3mmStation z=1.354mm 3D

SciDAC-II Compass SciDAC-II Compass 17 Vay - Compass 09 Other possible complication: inputs/outputs z’,t’=LT(z,t) frozen active Often, initial conditions known and output desired in laboratory frame –relativity of simultaneity => inject/collect at plane(s)  to direction of boost. Injection through a moving plane in boosted frame (fix in lab frame) –fields include frozen particles, –same for laser in EM calculations. Diagnostics: collect data at a collection of planes –fixed in lab fr., moving in boosted fr., –interpolation in space and/or time, –already done routinely with Warp for comparison with experimental data, often known at given stations in lab.

SciDAC-II Compass SciDAC-II Compass 18 Vay - Compass 09 Simulations in 1D/2D/3D at plasma densities of cc, cc and cc show good agreement on (scaled) beam energy gain:  1D: max  frame = 130  speedup > 10,000  2D: max  frame = 40  speedup > 1,000  3D: max  frame = 30  speedup > 500  24h using 256 CPUs  more than one year x 256 CPUs in lab frame! Full scale simulations of a 10 GeV LWFA stage *similar work by Bruhwiler et al (Tech X), Martins et al (IST/UCLA) Max  frame achieved in 2D and 3D limited by instability developing at front of plasma Origin: numerical Cerenkov? 2D n e = cc  frame = 13

SciDAC-II Compass SciDAC-II Compass 19 Vay - Compass 09 BELLA PW laser 40 J / 40 fs BELLA 40 J PW Laser – Components for a Laser Plasma Collider Injection + Staging 10 GeV stages Energy spread & Emittance preservation Simulating 10 GeV stages explicitly (PIC) in lab frame needs ~1G CPU  hours  impractical* Predictions have relied on theory, reduced models (fluid, envelope, quasistatic), scaling: –Energy gain  n -1 : 100 MeV at /cc  10 GeV at /cc –Length  n -3/2 : 1mm at /cc  1m at /cc –Gradient  n 1/2 : 100 GV/m at /cc  10 GV/m at /cc Can simulations of full scale 10 GeV stages be practical using a Lorentz boosted ref. frame? –difficulty: backward emitted radiation frequency upshifted in boosted frame, (noise, instabilities). * Cormier-Michel et al, Proc. AAC 2008; Geddes et al, Proc. PAC’09 Positron acceleration + PWFA expt.’s Radiation sources

SciDAC-II Compass SciDAC-II Compass 20 Vay - Compass 09 boosted frame 10km 10cm 10km/10cm=100, m 10m/1nm=10,000,000,000. 1nm 3cm 3cm/1  m=30,  m compaction x m 4.5m frame  ≈ m/4.5m=100. 1nm compaction x mm 4m4m 2.5mm/4  m=625. frame  ≈ 4000 compaction x mm 30  m 1.6mm/30  m=53. frame  ≈ 19 Lorentz transformation => large level of compaction of scales range HEP accelerators (e-cloud) Laser-plasma acceleration Hendrik Lorentz Free electron lasers