1. Outline 3  PWA overview Computational challenges in Partial Wave Analysis Comparison of new and old PWA software design - performance issues Maciej Swat @ Indiana University Recent progress in 3  Partial Wave Analysis of E852 data

2. PWA basics - isobar model- 3     - p p s=(p p +p p ) 2 t=(p p -p X ) 2 X      Measured by experiment CG coefficients and D lmm’ ’sUsually Breit Wigner propagator We know how to calculate decay amplitudes A b This is what we are looking for - production amplitudes iso

3. 3  PWA results Events/0.025 GeV/c 2 Mass [GeV/c 2 ] Intensity All waves a 2 (1320) a1a1  2 (1670) Typical PWA fit involves: ~1-3M events/’t’ bin ~5-10 ‘t’ bins ~80 mass bins ~30-40 waves

4. For each event we have to: 1. Find helicity frame decay angles -  1,  2 2. Calculate decay amplitudes (involves Wigner D functions, CG coefficients,BW propagators). These amplitudes are model dependent. In particular,the time to calculate a single decay amplitude depends on the model. 3.Find normalization integrals : PWA implementation - Normalization Integrals in the isobar model Data file - 10M experimental events Raw Monte Carlo file - 150M events Accepted Monte Carlo file - 40M events

5. Data sets Decay Amplitudes All data events All decay amplitudes Normalization Integrals Master Distributed data Gather partial results NI’s Slaves calculate amplitudes “on the fly” and evaluate partial contributions to normalization integrals OLDNEW PC-nodeDisk Storage Legend

6. Assume 10 ‘t’ bins OLDNEW Total time:~150 hours of computer time 150M40M10M Calculate masses, angles, invariants, amplitudes. Store amplitudes.Fill normalization integrals tables. One has to handle ~5 000 files ~300 GB disk space Every time one changes data cut another 50 hours of computer time is required. MC filesData files Total time:~45 minutes of computer time 150M40M10M Calculate masses, angles, invariants, amplitudes. Fill normalization integrals tables. One has to handle ~15 files ~30 MB disk space Every time one changes data cut another 10 minutes of CPU time is required. MC filesData files Most of the time is spent doing Input/Output operations Input/Output operations are reduced to necessary minimum Performance comparison

7. Decay Amplitudes Master Fitted parameters Gather likelihood contributions At every iteration of minimization routine master sends fitted parameters, slaves calculate likelihood and send the result back to the master OLDNEW bin 0 bin 1 bin 2 bin n PWA fits Decay Amplitudes... bin 0 - bin n Use final parameters from bin k as the starting parameters for bin k+1. Have to re- read decay amplitudes for every bin. Minuit runs only on master

8. OLDNEW Only two types of fits are possible as of now: 1. Bins are fitted independent from each other - fast, can use multiple CPU’s. 2 hours / t bin 2. Fit with parameters boot-strapping - has to be done on single machine, slow. 30 hours /t bin New fitter is scalable and one can do the following types of fits: 1. Bins are fitted independent from each other - fast, can use multiple CPU’s. 1-2 hours / t bin 2. Fit with parameters boot-strapping - has to be done on single machine, slow. 0.5 - 1 hours / t bin 3. Mass dependent PWA or any other type of fit where one would require to fit entire data sample at once (without dividing data set into bins). PWA fitter features comparison CAUTION: Slow network connection between master and slaves can significantly degrade performance of the PWA fitter.

9. Tips,Tricks, Conclusions Ways to achieve good performance: 1. Write parallel code using e.g. Message Passing Interface (MPI) - a simple and easy to use library 2. Avoid reading from secondary storage 3. Compute things once and avoid redundancy Until now we have used only isobar model. We have to study also other models, a lot of progress has been made in 70’s. Try to represent data in model independent fashion - study moments Theoretical analysis - choice of model issues