1. LUSI Experiment Needs Yiping Feng
Fluctuations Diagnostic suite
Intensity
Spatial
Temporal
Spectral
Large 2-dim detectors Sophisticated DAQ
Data storage
Real time processing
Diagnostics used for all instruments
Motivations for constructing these devices, why are they important
Specifications and description
Diagnostics are X-ray diagnostics
Page

2. Electron Beam Characteristics
X-ray Free-Electron Laser (FEL) is fundamentally different from storage-ring based synchrotron sources. For the LCLS
photo-injection at 120 Hz at LCLS
Each macro electron bunch is different at origination
Different timing, length, density
After passing through the Linac including acceleration and compression
Added difference in timing, length, density
Additional difference in energy, orbit
Orbit correction less effective due to low repetition rate in contrast to synchrotron sources (revolution frequency at APS = 272 kHz)
Page

3. X-ray Beam Characteristics
X-ray amplification process based on self-seeding SASE*
Lasing starts from a random electron density distribution
Each X-ray pulse consists of a random time sequence of spikes of varying degrees of saturation
X-ray FEL exhibits inherent Intensity, spatial, temporal, and spectral fluctuations on pulse by pulse basis
*Self Amplification of Spontaneous Emission
Page

4. Expected Fluctuations of LCLS FEL pulses
Parameter
Value
Origin*
Pulse intensity fluctuation
~ 30 %
Varying # of FEL producing SASE spikes; 100% intensity fluctuation/per-spike; etc.
Position & pointing jitter (x, y, a, b)
~ 25 % of beam diameter
~ 25 % of beam divergence
Varying trajectory per pulse; Saturation at different locations of b-tron curvature
Source point jitter (z)
~ 5 m
SASE process reaching saturation at different z-points in undulator
X-ray pulse timing (arrival time) jitter
~ 1 ps FWHM
Timing jitter btw injection laser and RF; Varying e-energy per-pulse
X-ray pulse width variation
~ 15 %
Varying e-energy leading to varying path (compression) in bunch compressors
Center wavelength variation
~ 0.2 %
(comparable to FEL bandwidth)
Varying e-energy leading to varying FEL fundamental wavelength and higher order
1) Linear accelerator based x-ray sources are fundamentally much different from synchrotron based sources and are comparatively unstable. Therefore, even when fully commissioned and running at saturation, intensity fluctuations are expected to exceed 30%, spatial jitter in the beam pointing …, ….
2) Unless mitigated, the level of jitter in these parameters would limit the studies that would be successfully performed. Since one cannot control these parameters to better than these values, the solution is to construct diagnostics that can measure them on a pulse by pulse basis.
*To be discussed in details now
Page

5. Expected Fluctuations of LCLS FEL pulses
Parameter
Value
Origin*
Pulse intensity fluctuation
~ 30 %
Varying # of FEL producing SASE spikes; 100% intensity fluctuation/per-spike; etc.
Position & pointing jitter (x, y, a, b)
~ 25 % of beam diameter
~ 25 % of beam divergence
Varying trajectory per pulse; Saturation at different locations of b-tron curvature
Source point jitter (z)
~ 5 m
SASE process reaching saturation at different z-points in undulator
X-ray pulse timing (arrival time) jitter
~ 1 ps FWHM
Timing jitter btw injection laser and RF; Varying e-energy per-pulse
X-ray pulse width variation
~ 15 %
Varying e-energy leading to varying path (compression) in bunch compressors
Center wavelength variation
~ 0.2 %
(comparable to FEL bandwidth)
Varying e-energy leading to varying FEL fundamental wavelength and higher order
1) Linear accelerator based x-ray sources are fundamentally much different from synchrotron based sources and are comparatively unstable. Therefore, even when fully commissioned and running at saturation, intensity fluctuations are expected to exceed 30%, spatial jitter in the beam pointing …, ….
2) Unless mitigated, the level of jitter in these parameters would limit the studies that would be successfully performed. Since one cannot control these parameters to better than these values, the solution is to construct diagnostics that can measure them on a pulse by pulse basis.
*To be discussed in details now
Page

6. Transverse Jitter Steering coil power supply regulation
Quadrupole magnet transverse vibrations
Quadrupole magnet power supply regulation in presence of typical 200-mm transverse misalignment
RF structure wakefields with varying charge and typical 200-mm transverse misalignments
CSR in bunch compressor chicanes with varying bunch length
Page

7. Spatial Jitter Transverse Stability Pointing Stability Orbit-1a
Orbit-1b
e- beam
X-ray beam
Orbit-2a
Orbit-2b
Pointing Stability
Page

8. Expected Fluctuations of LCLS FEL pulses
Parameter
Value
Origin*
Pulse intensity fluctuation
~ 30 %
Varying # of FEL producing SASE spikes; 100% intensity fluctuation/per-spike; etc.
Position & pointing jitter (x, y, a, b)
~ 25 % of beam diameter
~ 25 % of beam divergence
Varying trajectory per pulse; Saturation at different locations of b-tron curvature
Source point jitter (z)
~ 5 m
SASE process reaching saturation at different z-points in undulator
X-ray pulse timing (arrival time) jitter
~ 1 ps FWHM
Timing jitter btw injection laser and RF; Varying e-energy per-pulse
X-ray pulse width variation
~ 15 %
Varying e-energy leading to varying path (compression) in bunch compressors
Center wavelength variation
~ 0.2 %
(comparable to FEL bandwidth)
Varying e-energy leading to varying FEL fundamental wavelength and higher order
1) Linear accelerator based x-ray sources are fundamentally much different from synchrotron based sources and are comparatively unstable. Therefore, even when fully commissioned and running at saturation, intensity fluctuations are expected to exceed 30%, spatial jitter in the beam pointing …, ….
2) Unless mitigated, the level of jitter in these parameters would limit the studies that would be successfully performed. Since one cannot control these parameters to better than these values, the solution is to construct diagnostics that can measure them on a pulse by pulse basis.
*To be discussed in details now
Page

9. Z-Jitter Dz
DR2 = DZ
DR2 R1 R2
DR2 = DR1(R2/R1)2
Page

10. Expected Fluctuations of LCLS FEL pulses
Parameter
Value
Origin*
Pulse intensity fluctuation
~ 30 %
Varying # of FEL producing SASE spikes; 100% intensity fluctuation/per-spike; etc.
Position & pointing jitter (x, y, a, b)
~ 25 % of beam diameter
~ 25 % of beam divergence
Varying trajectory per pulse; Saturation at different locations of b-tron curvature
Source point jitter (z)
~ 5 m
SASE process reaching saturation at different z-points in undulator
X-ray pulse timing (arrival time) jitter
~ 1 ps FWHM
Timing jitter btw injection laser and RF; Varying e-energy per-pulse
X-ray pulse width variation
~ 15 %
Varying e-energy leading to varying path (compression) in bunch compressors
Center wavelength variation
~ 0.2 %
(comparable to FEL bandwidth)
Varying e-energy leading to varying FEL fundamental wavelength and higher order
1) Linear accelerator based x-ray sources are fundamentally much different from synchrotron based sources and are comparatively unstable. Therefore, even when fully commissioned and running at saturation, intensity fluctuations are expected to exceed 30%, spatial jitter in the beam pointing …, ….
2) Unless mitigated, the level of jitter in these parameters would limit the studies that would be successfully performed. Since one cannot control these parameters to better than these values, the solution is to construct diagnostics that can measure them on a pulse by pulse basis.
*To be discussed in details now
Page

11. Expected Fluctuations of LCLS FEL pulses
Parameter
Value
Origin*
Pulse intensity fluctuation
~ 30 %
Varying # of FEL producing SASE spikes; 100% intensity fluctuation/per-spike; etc.
Position & pointing jitter (x, y, a, b)
~ 25 % of beam diameter
~ 25 % of beam divergence
Varying trajectory per pulse; Saturation at different locations of b-tron curvature
Source point jitter (z)
~ 5 m
SASE process reaching saturation at different z-points in undulator
X-ray pulse timing (arrival time) jitter
~ 1 ps FWHM
Timing jitter btw injection laser and RF; Varying e-energy per-pulse
X-ray pulse width variation
~ 15 %
Varying e-energy leading to varying path (compression) in bunch compressors
Center wavelength variation
~ 0.2 %
(comparable to FEL bandwidth)
Varying e-energy leading to varying FEL fundamental wavelength and higher order
1) Linear accelerator based x-ray sources are fundamentally much different from synchrotron based sources and are comparatively unstable. Therefore, even when fully commissioned and running at saturation, intensity fluctuations are expected to exceed 30%, spatial jitter in the beam pointing …, ….
2) Unless mitigated, the level of jitter in these parameters would limit the studies that would be successfully performed. Since one cannot control these parameters to better than these values, the solution is to construct diagnostics that can measure them on a pulse by pulse basis.
*To be discussed in details now
Page

12. Goals
X-ray diagnostics are required to measure these fluctuations since they can’t be eliminated
Integral parts of Instruments
Timing & intensity measurements for XPP experiments
Wave-front characterization for CXI experiments
Measurements made on pulse-by-pulse basis
Requiring real-time processing by controls/data systems
Commonalities in needs & specs
Standardized and used for all applicable instruments
Modularized for greater flexibility of deployment and placement
Critical diagnostics must be performed and data made available on pulse-by-pulse basis
Page

13. Fluctuations, 120 hz pulse rate drive DAQ requirements
The 120 Hz per-pulse data collection/reduction, high data rate, large data volume, and sub-ps timing control requirements of LCLS experiments go far beyond those at existing synchrotron sources, requiring considerable complexity and sophistication in controls and data systems’ design, implementation, and integration that are not feasible for individual experimental teams
LCLS/LUSI controls and data systems must
Provide standard controls to all instruments
Support diagnostic measurements
Provide standard data acquisition capabilities
Provide standard data storage and management capabilities
Provide certain standard data analysis capabilities
The WBS 1.6 for controls will provide controls for endstations presented so far,
Page

14. Data system requirements
Data acquisition
Real-time data processing
Quick view
Data management
On-line storage
Long term archiving/retrieval
Data analysis
Volume rendering
visualization
Page

15. 2D Detectors Plug-play with a common interface? Cornell (PAD)
BNL(XAMPS)
Technology
diode/ASIC
Architecture
Bump-bond
integrated
readout
pixel
column
size
190x190x32
1024x1024
Data rate
1.9Gb/s
1.5Gb/s
Resolution
14bit
12bit
Plug-play with a common interface?
Page

16. Data rates - CXI Data Rate/Volume of CXI Experiment
high peak rate & large volume comparable to high-energy physics experiments such as SLAC
Data Rate/Volume of CXI Experiment
(comparable to other experiments)
LCLS Pulse Rep Rate (Hz)
120
Detector Size (Megapixel)
1.2
Intensity Depth (bit) 14
Success Rate (%)
30%
Ave. Data Rate (Gigabit/s)
0.6
Peak Data Rate (Gigabit/s)
1.9
Daily Duty Cycle (%)
50%
Accu. for 1 station (TB/day)
3.1
Is it possible to perform real-time data analysis
to reduce the data rate?
require high performance and high capacity data acquisition and management system
Page

17. Long term data storage needs
Year
2009-
2012-
2015-
Rep Rate (Hz)
120
Detector Size
(Megapixel)
0.58
1.16
Projected 5.8
Intensity Depth (bit) 14
Success Rate (%)
10%
30%
50%
Ave. Data Rate (Gigabit/s)
0.1
4.9
Peak Data Rate (Gigabit/s)
0.97
1.94
9.8
Daily Duty Cycle (%)
25%
75%
Accu. for 1 station (TB/day)
0.26
3.14 39
Accu. for 3 stations (TB/day)
0.80
9.4
118
Yearly Uptime (%)
Accu. (Petabyte/year)
0.072
1.7 32
Duration/Lifetime (year) 3
Total Accu. (Petabyte)
0.22
5.2 97
Page

18. Overall data needs Per pulse data collection Raw data rate and volume
Experimental
Diagnostic – EO signal, e- and g beam parameters
Raw data rate and volume
2 Gb/sec or higher
On-line storage capacity - 20 TB/day
Timing/Triggering
EO timing measurement < 1 ps
Detector trigger < 1 ms
Real time analysis
Frame correction, quality control
To the extent possible - binning, sparsification, FFT
Quick view
Quasi real-time feedback, 5 frame/s
Alignment
Data Management
Unified data model
Archiving capacity – 5 PB/year
Analysis staging storage capacity – 20 TB
Offline Analysis
> 1000 node cluster
Page

19. Applications needs User programs
Endstation operation
Calibration
Alignment
Interface to SW for diffraction/scattering experiments
SPEC
Interface to instrumentation/analysis SW
MatLab
LabView
User tools
STRIP tool
ALARM Handler
Page

20. Pieces of the Puzzle Pulse-by-pulse info exchange High peak rate/
large volume
EO Timing & Triggering
Feedback
Data Archiving/Retrieval
Offline Analysis/Rendering
At a very high level, the control and data system will have two major subsystems: control subsystem, and a dedicated data acquisition and management subsystem.
The control subsystem is similar to and will be an integral part of the LCLS controls system, and will be base don EPICS. There will be stronger interaction than before
Between the LCLS and LUSI controls due to the fact that most experiments at LCLS will need to include beam parameters on a pulse-by-pulse basis. The high peak rate and large volume calls for a dedicated data acquisition and management system that will have standardized design and implementation across the board, and it will be integrated into the existing SLAC data farm and computer clusters for longer term storage and complex and computational intensive offline analysis.
Page

21. Page

22. LCLS FEL Parameters
*Courtesy of Z. Huang
Page

23. LCLS Accelerator Schematics*
SLAC linac tunnel
research yard
Linac-0
L =6 m
Linac-1
L 9 m
rf  -25°
Linac-2
L 330 m
rf  -41°
Linac-3
L 550 m
rf  0°
BC1
L 6 m
R56 -39 mm
BC2
L 22 m
R56 -25 mm
DL2
L =275 m
R56  0
DL1
L 12 m
R56 0
undulator
L =130 m
6 MeV
z  0.83 mm
  0.05 %
135 MeV
  0.10 %
250 MeV
z  0.19 mm
  1.6 %
4.30 GeV
z  mm
  0.71 %
13.6 GeV
  0.01 %
Linac-X
L =0.6 m
rf= -160
21-1
b,c,d
...existing
linac
L0-a,b rf
gun
21-3b
24-6d X
25-1a
30-8c
Commission in Jan. 2007
Commission in Jan. 2008
*Courtesy of P. Emma
Page

24. Micro-bunching & SASE Process
Micro-bunched
Shot-noise
*Courtesy of Z. Huang
Page

25. Temporal Characteristic
*Courtesy of Z. Huang
DT = fs
tc ~ 200 as
M = DT/tc ~
For ideal e-beam of equal bunch length and same energy
<DI/I> ~ 1/√M ~ 5%
In reality
<DI/I> ~ 1/√M ~ 30%
attosecond
Page

26. Magnetic Bunch Compression
DE/E z
DE/E z
‘chirp’
DE/E z sz
…or over-compression
under-compression
sz0
sE/E
Dz = R56DE/E
Path Length-Energy
Dependent Beamline
V = V0sin(wt)
RF Accelerating
Voltage
Page

27. Timing Jitter SLAC linac tunnel research yard Commission in Jan. 2007
L =6 m
Linac-1
L 9 m
rf  -25°
Linac-2
L 330 m
rf  -41°
Linac-3
L 550 m
rf  0°
BC1
L 6 m
R56 -39 mm
BC2
L 22 m
R56 -25 mm
DL2
L =275 m
R56  0
DL1
L 12 m
R56 0
undulator
L =130 m
6 MeV
z  0.83 mm
  0.05 %
135 MeV
  0.10 %
250 MeV
z  0.19 mm
  1.6 %
4.30 GeV
z  mm
  0.71 %
13.6 GeV
  0.01 %
Linac-X
L =0.6 m
rf= -160
21-1
b,c,d
...existing
linac
L0-a,b rf
gun
21-3b
24-6d X
25-1a
30-8c
Commission in Jan. 2007
Commission in Jan. 2008
*Courtesy of P. Emma
Page

28. X-ray Diagnostics Suite
Fluctuation Type
Diagnostic Device
Pulse intensity fluctuation
a) Pop-In Intensity Monitor
b) In-Situ BPM/Intensity Monitor
Position & pointing jitter
c) Pop-In Position/Profile Monitor
In-Situ BPM/Intensity Monitor
- Pointing determination from multiple BMP’s
Source point jitter
 Focal point jitter w/ focusing optics
d) Wave-front Sensor
- Back-propagating from radius of curvature measurement
X-ray pulse timing jitter
e) Electro-Optic Sampling (EOS) Device
- Relative timing btw e-bunch & ref. probe laser
X-ray pulse width variation
EOS Device
- Establishes upper limit
center wavelength variation
LCLS e-energy calibration
- X-ray wavelength cross-calibration is needed
Page

29. Specifications Diagnostic Item Purposes Specifications* Pop-in
intensity monitor
(moderate-resolution)
Coarse beam alignment/monitoring;
Destructive; Retractable;
Dynamic range 104;
Per-pulse operation at 120 Hz;
Relative accuracy < 10-2
position/profile monitor
Coarse beam alignment/monitoring
At 50 mm resolution
- 25x25 mm2 field of view;
At 10 mm resolution
- 5x5 mm2 field of view
In-situ
BPM/Intensity monitor
(high-resolution)
Per-pulse normalization of experimental signals;
High-resolution beam position monitoring
Transmissive (< 5% loss);
Dynamic range 106;
Relative accuracy < 10-3
Electro-optic sampling (EOS) device
Measure relative timing between electron bunch (thus co-propagating x-ray pulse) and a probe optical laser pulse
Non-intrusive to e-beam;
Non-destructive;
20 fs resolution;
Wave-front sensor
Characterization of wave-front;
Locating focal point of focused beam
Destructive;
0.15 nm < l < 0.3 nm
Technically more challenging
Two types:
Conventional/Straightforward, higher damage threshold from peak power loading
and the more Challenging ones, technical specs tighter, much more difficult to make them to work
* Must have high damage threshold
Page

30. Data Processing Goals System Specifications for Data System
Data Acquisition and Management
Data Analysis
XPP, CXI, and XCS Instruments
System Description
 Gunther Haller’s breakout session
My name is Yiping Feng, and I am also the physicist for the controls. Next I will presents our controls system in terms of:
The scope, requirements, system description, the WBS, technical risks and mitigation, cost estimates, the schedule by functionalities.
Page

31. Scope - XPP Instrument time resolved scattering
at < ps time resolution
Page

32. Scope - CXI Instrument structures of single molecules
at near atomic resolution
Pixelated detector
(Cornell detector)
Molecule
injection
Intelligent beam-stop
(wave-front sensor)
LCLS beam
(focused, possibly compressed)
Data Processing
Serves two purposes: as an illustrated guide to the agenda, and as the plan of the experiments
Each label here is a research program in its own right - and this is really the key to this whole workshop - defining the R&D challenges and coming up with a rough cost estimate of that research.
potential particle orientation beam
Readout & reconstruction
Optical & x-ray diagnostics
To mass spectrometer
Page

33. Scope - PCS Instrument dynamics of disordered systems
at < ns time & near atomic resolutions
Data Processing
Speckle Pattern
(Iron-Aluminum Alloy)
Page

34. Diagnostics Control - Hartman Wavefront Sensor
Image obtained from Imagine Optics, Ltd
Measurement made far from focal plane
Single shot operation
120 Hz with CCD modification
1.5 nm and 0.15 nm operation with customization
Page

35. Timing Control - Temporal Jitter
Master
Clock
Coax RF Distribution Network
Electron Gun
Accelerating
Elements
Experimental
Pump Laser
Sources of Short Term Jitter
Coax RF distribution Network
e-beam phase to RF phase
End Station Laser phase to RF phase
Limited to ~ 1 ps !
Page

36. Timing Control - Electro-optic Sampling
Stabilized Fiber Optic RF Distribution (10 fs)
LBNL
Electro-optic Sampling Laser
Pump-probe
Laser
Gun Laser
Sector 20
LTU
NEH
Temporal resolution is now limited by:
Our ability to phase lock the lasers to the RF
Intra-bunch SASE jitter
Page

37. Timing Control - SPPS Laser/X-ray Timing
100 consecutive shots
Single shot, Lorentzian fit
Page

38. Real-time Processing – Binning in XPP
Page

39. Real-time processing – binning in XPP
For XPP experiments using 2D detector,
Is it possible to perform real-time data analysis
to reduce the data rate?
10 Hz
Point Detector
Page

40. Real-time Processing – sorting in CXI
Diffraction from a single molecule:
noisy
diffraction
pattern of unknown orientation
single LCLS pulse
unknown orientation
Combine 105 to 107 measurements into 3D dataset:
Reconstruct by Oversampling phase retrieval
Classify/sort
Average
Alignment
Real-time?
The highest achievable resolution is limited by the ability to group patterns of similar orientation
Miao, Hodgson, Sayre, PNAS 98 (2001)
Gösta Huldt, Abraham Szöke, Janos Hajdu (J.Struct Biol, ERD-047)
Page

41. Computing hardware requirements
Real-time computing
Power: 10 Tera-FLOPS
1000 processor cluster
Memory: GByte RAM
Bandwidth: 100 Gbit/s
Integrated w/ detector or immediate downstream of detector output
Data Storage/Management
10 – 100 Gbit/s links
10 – 100 on-line capacity: RAID disks, or flash memory
10 – 100 staging capacity: RAID disks, or flash memory
5 Petabyte yearly capacity
ESNET connection for transferring to sister institutes
Offline Analysis
Total FLOPs: 1017
If analysis done in minutes: 2000 – processor cluster
Large volume set rendering: 109
Page