1. Lucretia Status Glen White LCWS2011 Granada, Sept. 2011

2. Overview Summary of existing Lucretia status and capabilities. New features – Matching – Distributed Lucretia (MC) – Parallel Lucretia Tracking (GPU) – Floodland (a.k.a. Flight Simulator) Interface between Lucretia and EPICS or AIDA control system Planned future features

3. Authorship Lucretia was originally written by PT, formally at SLAC. It is currently maintained and developed by myself and Mark Woodley at SLAC.

4. What is Lucretia? A Matlab-based toolkit for simulations of beam dynamics in single-pass electron beamlines – Most components are Matlab scripts or functions – A handful of compiled C functions which run under Matlab (“mexfiles”) Compute intensive activities like tracking – Tracks pointlike rays with user-defined charge R-matrix calculation, Twiss propagation, lattice verification – Share a lot of code with Tracking Designed with the ILC in mind – Somewhat useful for other applications – LCLS, XFEL – Designed from the ground up to be an all-in-one tool for LET studies

5. 27-Nov-2007 SLAC-ILC-AP Meeting Global Design Effort5 Other Lucretia Features An online manual in the form of a heavily cross-linked website: http://www.slac.stanford.edu/accel/ilc/codes/Lucretia/ A GUI for studying the beamline data structures Subversion control via googlecode: http://slac-lucretia.googlecode.com

6. FFS Optics Matching Lucretia/src/TuningAndAnalysis/Match.m Match class for first and higher-order beam matrix matching using tracking engine, fast n th -order polynomial fitting routines in conjunction with Matlab optimisation tools. – ‘polyfitn’ from “Matlab Central” calculates beam correlations from tracked macro-particles. – ‘lsqnonlin’ for first-order (Twiss) parameter matching – ‘fminsearch’ (Nelder-Mead) or other (fmincon, genetic,…) Control higer-order aberrations Match constraints (e.g. ATF2 FFS lattice) – Waist at IP with given beta functions – Waists at MFB1FF and (MFB2FF OR LW1FF) Ensures correct phase advances for FFS feedbacks Match variables – Matching quads + final doublet for initial Twiss match – All FFS quadrupole and sextupole strengths plus SK1FF strength, for final beam size optimisation.

7. Distributed Lucretia Lucretia/src/Distributed/distributedLucertia.m Class for setup & control of distributed Lucretia tasks. Uses “Matlab parallel computing toolbox”. – Plus optionally “Matlab distributed computing server”. Designed mainly for Monte Carlo types of analysis. – Setup and operate on N beamlines with varying error characteristics. – Other classes (e.g. Track class) designed to work with distributedLucretia class operate on distributed set of beamlines. Handles synchronous or asynchronous distribution of tasks – i.e. can command an operation on all beamlines and wait for response (synchronous) or send operation command and do something else whilst the tasks complete (asynchronous)

8. Matlab Parallel Computing Environment Builtin routines to distribute operation of Matlab code across multiple cores on local CPU(s) or GPU(s) or across a heterogeneous compute cluster via Matlab or third-party scheduler. e.g. at SLAC, used in conjunction with IT services LSF scheduler. Image courtesy of Mathworks: http://www.mathworks.com/products/parallel-computing/

9. Distributed Tracking Example >Use Matlab inbuilt tools to setup distributed environment using distributed computing tool (link to scheduler, how many and which worker nodes to use etc) >DL=distributedLucretia(false,100) >% start 100 distributed “worker” sessions in an asynchronous mode >T=Track(Beam,DL) % (in Lucretia/src/Tracking) >% generate Track object with attached Lucretia Beam structure >DL.latticeSyncVals(12).PS(53)=0.85; >% Change the value of Power Supply 53 on the 12 th worker node >DL.PSTrim(53) % Trim PS 53 (on all nodes) >T.trackThru % Perform tracking operation on all nodes >DL.asynWait % wait for tracking operations to complete >Beam1=T.beamOut{1}; Beam12=T.beamOut{12} >Get tracked beam objects etc on desired worker nodes

10. Parallel Lucretia Tracking Re-write of Lucretia main tracking engine to make use of GPU cores – Use CUDA API with Nvidia-based graphics cards (e.g. Tesla) – Re-write kernels of tracker to distribute rays for tracking across cores of GPU (e.g. Tesla 2050 has 448 compute cores). – Tricky as requires careful memory management of BEAMLINE and Beam elements Shared Matlab/CPU memory (host memory) GPU-resident DDR memory (3GB) L1 cache + shared memory (up to 64KB per 8 cores) Main application is speeding up of serial tracking tasks such as matching using the tracking engine plus optimisers. Currently re-written about 50% of tracking subroutines.

11. Floodland (Flight Simulator) Lucretia/src/Floodland Routines in this module allow for connections between the simulation world of Lucretia and real-World control system objects attached to a particle accelerator. Lots of existing code written for ATF2 to perform various beam-dynamics tasks (tuning, steering, feedback etc) This incarnation of Floodland rapidly became very ATF2- specific. Current incarnation in Lucretia software repository is the start of a re-write to make truly generic code applicable to any accelerator control system implementing either EPICS or AIDA. – Starting tests at NLCTA with feedback code.

12. Flight Simulator Goals Provide simple to use, beam dynamics oriented, portable control access framework for ATF2 tuning tasks. Simple and reversible transition from beam dynamics simulation to accelerator ready code. Ability for international collaborators to develop beam tuning tools without need for expert level knowledge of control systems. Flight simulator operates in simulation mode at external location in the same way as the production system deployed at ATF2.

13. Flight Simulator at ATF2

14. ATF2 Flight Simulator Applications Dispersion measurement and correction Coupling measurement and correction – Wirescanners or OTRs Model verification & matching Response matrix measurement Steering Orbit feedback Sextupole-knob-based IP beam size tuning BBA BPM-functions – Orbit plotting, calibration, resolution calculations, orbit bumps… Watchdogs – Monitor beam parameters (e.g. beam orbit in specific apertures) and alarm when parameters fall outside pre-determined range. Save/restore functions Archive interface

15. New Floodland Module Lucretia non-SimMode Floodland -> IOC via labCA or Aida Matlab functions Lucretia non-SimMode Floodland -> IOC via labCA or Aida Matlab functions EPICS or AIDA Control System PV’s Lucretia SimMode Floodland -> provides simulation of control system environment Lucretia SimMode Floodland -> provides simulation of control system environment Lucretia Tracking Simplify simMode- move simulation of IOC’s into Matlab Floodland module. No more server/client implementation for Floodland itself, just deal with multiple Floodland sessions as a possibility.

16. Missing Features List Insertion devices – Undulators, wigglers. Generic field map elements – Generic 3D E, B field maps to attach to other elements – Useful for realistic representation of detector solenoids and any other situation involving combining different magnetic elements. CSR Generic transfer-function element …