1. PROOF - Parallel ROOT Facility Kilian Schwarz Robert Manteufel Carsten Preuß GSI http://root.cern.ch Bring the KB to the PB not the PB to the KB

2. Introduction A step towards a solution: (Ali) ROOT + AliEn Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan ● ROOT is becoming most popular available physics analysis toolkit. ● Interactive analysis work in familiar C++ style syntax ● data visualisation, an object-oriented I/O system ● crucial role for the LCG project ?! ● is successfully used within the AliROOT framework of the ALICE experiment as an all in one solution ● PROOF extends workstation based concept of ROOT to the 'parallel ROOT facility'. ● user procedures are kept identical during an analysis session ● tasks are distributed automatically in the background ● AliEn as a GRID analysis platform provides two key elements: ● a global filesystem ➔ files are indexed and tagged in a virtual file catalogue and everywhere globally accessible ● a global queuesystem ➔ global job scheduling according to resource requirements

3. Parallel Analysis of Event Data root Remote PROOF Cluster proof TNetFile TFile Local PC $ root ana.C stdout/obj node1 node2 node3 node4 $ root root [0] tree.Process(“ana.C”) $ root root [0] tree.Process(“ana.C”) root [1] gROOT->Proof(“remote”) $ root root [0] tree.Process(“ana.C”) root [1] gROOT->Proof(“remote”) root [2] dset->Process(“ana.C”) ana.C proof proof = slave server proof proof = master server #proof.conf slave node1 slave node2 slave node3 slave node4 *.root TFile

4. PROOF - Scalability

5. GSI environment the prooflogin-script scanning user-parameters for errors processing user-parameters scanning LSF-Cluster for PROOF-jobs testing.rootrc building scripts (cleanup.sh, proofstarter.C and proofd.sh) getting / setting ROOT-version starting the wanted amount of PROOF-daemons building.proof.conf and.rootauthrc starting local rootd starting ROOT and executing proofstarter.C killing jobs and processes removing all builded files starting PROOF, uploading packages and starting the analysis

6. User-parameters /usr/local/bin/prooflogin -sslave-count -ttermination-time -vROOT-version -fROOT-files -lib library-files -par file-packages -mol starts the master on the localhost -? / -h / -help help for proof.sh optional : -file a text-file with all parameters written in

7. dedicated batch queue for PROOF only proof jobs are started in the dedicated Proof queue Quick Response Queue currently in test operation on 30 nodes of the GSI batch farm In this queue proof jobs have advantage. Non proof jobs will be set on hold

8. PROOF configuration files $HOME/.rootauthrc (e.g.) proofserv lxb108:kschwarz:0 lxb109:kschwarz:0 lxb110:kschwarz:0 $HOME/.proof.conf (e.g.) node lxb108 port=1095 usrpwd slave lxb108 port=1096 usrpwd slave lxb109 port=1095 usrpwd slave lxb109 port=1096 usrpwd slave lxb110 port=1095 usrpwd slave lxb110 port=1096 usrpwd

9. Interactive Analysis with PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan Basic requirements: ● Analysis data has to be stored as objects derived from TObject in ROOT trees ● proofds have to load extension libraries for user-specific objects to access the data members ● Analysis code has to be inserted in the automatically generated selector macro for the object to be analyzed: ->MakeSelector(); Receipe: -store your objects in trees -use the selector macro for analysis code

10. Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan ● interactive analysis with Proof steered by a data packetizer ● in a local cluster: ➔ cluster wide accessible data can be processed by all slaves ➔ packet takeover by all slaves! ● in a grid environment: ➔ site wide accessible data can be processed by all slaves ➔ packet takeover by all slaves within one site ! Work Distribution

11. Short explanation for creating an analysis script 4 steps: Open ROOT-file. make Selector-files (analysis files) edit header-file edit source-file

12. Open ROOT-file and create Selector files Open your file: TFile f(„/u/dvgamma/hitfile.root"); Create Selector files: hittree->MakeSelector(„Anaproof“); the ROOT-file contains a tree called „hittree“

13. Edit header file Add your branch: TBranch *b_myHit; Set branch address: Fchain->SetBranchAddress(„myHit“,&myHit); Add user defined objects and some data members (will be explained later)

14. Class TCounter Dummy class designed for collecting analysis results from slaves Keep the data till it‘s catched in SlaveTerminate()

15. Edit source file (1/2) Explanation for each function see on top Analysis is embedded in Anaproof::Process(Int_t entry) the analysis checks the hits in chambers and counts them the counterobject collects hitcounters In Anaproof::SlaveTerminate() add: fOutput->Add(counterobj);

16. Edit source file (2/2) First you get your counterobj as an TObject from the outList Convert it back before it can be used as an TCounter object

17. libraries and packages TMytrackerhit and TCounter are two user defined classes. To use them in a PROOF session you have to build a package that can be uploaded by a ROOT–demon. You need following dictionary/file mix: libTMytrackerhit/PROOF-INF/SETUP.C A look into Setup.C: Int_t SETUP() { gSystem->Load(„/u/dvgamma/projects/globus/onestep/ROOT/libTMytrackhit.so“); gSystem->Load(„/u/dvgamma/projects/globus/onestep/PROOF/TCounter/TCounter.so“); return 1; } To create a package: „tar –czf libTMytrackerhit.par libTMytrackerhit;“

18. Finally launch a PROOF-Session and start the analysis Start the PROOF-session via script or manually Inside of the session: Upload packages Enable packages Create TDSet Add file Start analysis

19. At least a screenshot

20. Jointventure of AliEn + ROOT Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan API Service AliEn Services DBI DB Prox y Catalogue DB AliEn C++ API gSOAP Client TGrid Class TAlien Plugin Plugin ● AliEn Services + Catalogue are accessible via TAlien(TGrid) class and global variable in ROOT ● TAlien uses a SOAP based AliEn C++ API Examples: ➔ TGrid::Connect(“alien://aliendb1:15000/?direct”,””) ; // inititate gGrid with AliEn plugin (API server at aliendb1, port 15000) ➔ gGrid->mkdir(“/alice/acat03”); // create directory in virtual file catalogue Client HostClient or Remote HostVO Service + DB Hosts

21. Executable = "root"; Packages = "ROOT::3.10.01"; Arguments = "Command::ROOT -x macro.C"; InputData= {"/alice/production/peters/*Tree.root”}; InputFile= {"LF:/alice/user/p/peters/macro.C"}; OutputFile= {"myhisto.root"}; Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan AliEn Job Description Example Running a ROOT macro on registered data Simple and readable !

22. Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan PROOF USER SESSION PROOF SLAVE SERVERS PROOF MASTER SERVER AliEn Grid Proof Setup: PROOF SLAVE SERVERS TcpRoute r ● Guaranteed site access through multiplexing TcpRouters

23. Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan Sample Session:Connect/Query

24. Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan Sample Session:connection to assigned proofds

25. Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan Sample Session:data processing

26. Unification of Batch & Interactive Analysis with AliEn + ROOT/PROOF Andreas J. Peters CERN/Geneva @ ACAT03 – Tokyo/Japan current implementation: ● datasets are represented by objects of the type TDSet in ROOT ● a GRID data query assigns data files to TDSet Objects ● the “process” method initiates the interactive processing on the assigned GRID proof cluster to come: ● the same “process” method initiates the batch processing of the same data set and automatic merging of results. ALICE will test the analysis facilities during the physics data challenge end 2004.

27. Thanks to Fons Rademakers Andreas Joachim Peters For their contributions to the transparencies

28. http://www-w2k.gsi.de/root/