1. The ALICE Analysis -- News from the battlefield Federico Carminati for the ALICE Computing Project CHEP 2010 – Taiwan

2. The problem Data management is the major problem for our Grid infrastructure – Both because it is intrinsically difficult and because it took a long time to tackle it When data access is predictable, the problem is manageable – Simulation, calibration, reconstruction When data access is not, then we have a problem – Analysis Main problems – Data location may not follow data access pattern – Multiplication of data formats October 16, 2010fca@chep20102

3. Data location Send job to the data Be ready when the data is not where you expected it October 16, 2010fca@chep20103

4. Sending jobs to data 4 ALICE central services Job 1lfn1, lfn2, lfn3, lfn4 Job 2lfn1, lfn2, lfn3, lfn4 Job 3lfn1, lfn2, lfn3 Optimizer Submits job User ALICE Job Catalogue Registers output lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} ALICE File Catalogue Computing Agent Site Computing Agent Site Computing Agent Site Computing Agent Send results Fetch job Job 1.1lfn1 Job 1.2lfn2 Job 1.3lfn3, lfn4 Job 2.1lfn1, lfn3 Job 2.1lfn2, lfn4 Job 3.1lfn1, lfn3 Job 3.2lfn2 October 16, 2010fca@chep2010

5. LPM chains logic October 16, 2010fca@chep20105 Reco. 1job/chunk Reco. 1job/chunk QA 1job/chunk QA 1job/chunk QA merging QA merging Delete partial output Delete partial output Merge ROOT tags AOD 1job/chunk AOD 1job/chunk AOD Merging AOD Merging Delete partial output Resubmit error jobs Same mechanism is used also for MonteCarlo productions and analysis trains on MC and RAW data When complete, start in parallel Analysis ESD/AOD Analysis ESD/AOD Integrated with MonALISA

6. The access to the data 6 Application ALICE FC File GUID, lfn or MD SE & pfn & envelope lfn → guid → ( acl, size, md5) build pfn who has pfn? SE & pfn xrootd Tag catalogue Direct access to data via TAliEn/TGrid interface ev#guidTag1, tag2, tag3… ev#guidTag1, tag2, tag3… ev#guidTag1, tag2, tag3… ev#guidTag1, tag2, tag3…

7. Complete “pull” model? Caching or catalogue registration 7 cmsd xrootd Centre A cmsd xrootd … any other cmsd xrootd CERN cmsd xroot d xrootd global redirector all.role meta manager all.role manager But missing a file? Ask to the global metamgr Get it from any other collaborating cluster Local clients work normally Remote access 2-3 times slower than local one!

8. Opportunistic storage discovery A client-to-storage metric allows the automatic discovery of the closest (working) storage elements from every job October 16, 2010fca@chep20108 France Italy Nordic Countries Russia USA B ased on MonALISA includes topology information, continuous functional and SE occupancy status

9. Multiple data formats Use one format But make it flexible to adapt to user needs October 16, 2010fca@chep20109

10. Analysis train 10 AOD production will be organized in a ‘train’ of tasks –To maximize efficiency of full dataset processing –To optimize CPU/IO –Using the analysis framework AOD TASK 1TASK 2TASK 3TASK 4 ESD Kine Eff cor October 16, 2010fca@chep2010 example of AOD production

11. Analysis Framework Basics: make use of the main ROOT event loop initiated when processing a TChain of files with a TSelector AliESDs.root TChain TSelector Current file Current event AliAnalysis Selector Our framework uses the TSelector technology that defines three analysis stages: initialization, event processing and termination. The current file change is notified. Begin Process Terminate Notify The framework is steered by a manager class that gets called in the different selector stages… AliAnalysis Manager Defining a common interface to access input data files (MC, ESD, AOD) and to write AOD outputs AliVEvent Handler Input Output MC kine TrackRefs AOD AliVEvent … and the interface for the user analysis that follow the selector analysis stages. A train of such analysis tasks share the main event loop AND the input data while still hot in memory. AliAnalysis Task 0…n (s) October 16, 201011fca@chep2010

12. 1228/03/07fca @ RUM07 General idea AliAODEvent TList AliAODHeader AliAODEvent AliAODTrackAliAODVertexAliAODCluster AliAODJet UserInfo AliAODUser1AliAODUser2 … … Standard part User part Too simple to be true… use a list

13. 1328/03/07fca @ RUM07 Extending the AOD S-AOD GetList()->Add(Obj) … U1-AOD … S-AOD U1-AODS-AOD U1’-AOD GetList()->Add(Obj) … U1-AOD … S-AOD GetList()->Add(Obj) … U1-AOD … S-AOD U2-AOD … U1’-AOD U2-AOD You got the idea…

14. Get users into the system Convince the users that the system is usable October 16, 2010fca@chep201014

15. 15 Grid load October 16, 2010fca@chep2010

16. 16 Grid usage – chaotic October 16, 2010fca@chep2010 227 users, ~5% of Grid resources 6T1, 67T2, 50/50 contribution

17. 17 CAF lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} xrootd T0 WN PROOF XROOTD WN PROOF XROOTD WN PROOF XROOTD WN PROOF XROOTD WN PROOF XROOTD WN PROOF XROOTD PROOF master The whole CAF becomes a xrootd cluster Powerful and fast machinery – very popular with users Powerful and fast machinery – very popular with users Allows for any use pattern, however quite often leading to contention for resources Allows for any use pattern, however quite often leading to contention for resources Can load directly from the Grid Can load directly from the Grid October 16, 2010fca@chep2010 CAF - CERN AF 208 workers (26 x 8 core) 80 TB space 24 GB RAM on all machines SKAF - Slovak Kosice AF 60 workers (15 x 4 core) 53 TB space 24 GB of RAM master 8 GB of RAM on workers

18. 18 Prompt analysis - CAF Data volume – over 1PB User load – over 50 users average October 16, 2010fca@chep2010

19. Conclusions Analysis on the Grid “just works” – In spite of all our fears people are routinely doing analysis on the Grid This does not mean that things work well – A lot of work of optimisation could and should be done Again, commonality between experiments is large, and could be exploited better October 16, 2010fca@chep201019

20. 20October 16, 2010fca@chep2010