2. Volunteer Clouds for the LHC experiments H. Riahi – 12/11/15 EGI User Forum 20152 Laurence Field Hassen Riahi CERN IT-SDC

3. Volunteer Computing A type of distributed computing Origins in mid 1990s Computer owners donate computing capacity To a cause or project Not necessarily only spare cycles on desktops Idle machines in data centers Home clusters Berkeley Open Infrastructure for Network Computing (BOINC) Started in 2002 Funded by the National Science Foundation (NSF) SETI@home: 12K Active Users, 16K Active Hosts In 2007, IBM World Community Grid switched to BOINC 67K Active Users, 1.8M Active Hosts 3 H. Riahi – 12/11/15 EGI User Forum 2015

4. Volunteer Perspective Download and run BOINC software Choose a project Enter an email address and password Or silent connection with a key Earn Credit 4 H. Riahi – 12/11/15 EGI User Forum 2015

5. Challenges For HEP The cost of using the free resources (>100K hosts) Initial integration requires investment Operations and maintenance Attracting and retention of volunteers Advertisement Engagement Low Level of Assurance Anyone can register as a volunteer Not the same level of trust as with Grid authentication 5 H. Riahi – 12/11/15 EGI User Forum 2015

6. BOINC with Virtualization 6 H. Riahi – 12/11/15 EGI User Forum 2015 Pioneered with Test4Theory and CernVM 2010-2011 Included into the mainstream BOINC code BOINC projects currently deploying Virtualisation: CERN (Theory, ATLAS, CMS, LHCb) Climateprediction.net … Vboxwrapper: http://boinc.berkeley.edu/trac/wiki/VboxApps http://boinc.berkeley.edu/trac/wiki/VboxApps

7. BOINC Service @CERN 7 CERN IT-PES H. Riahi – 12/11/15 EGI User Forum 2015 BOINC server cluster LHC@home servers LHC@home Sixtrack, Theory, ATLAS Test servers (CMS, LHCb, project with EPFL, Dev environments) BOINC server application support MySQL database server back-end BOINC server application configuration, updates and monitoring Handled by the project teams: Porting of applications to BOINC Application specific job management framework Communication with users about scientific projects Management of forums and portal content

8. BOINC Service Monitor 8 CERN IT-PES: http://cern.ch/go/9nRz http://cern.ch/go/9nRz Sixtrack: 100k parallel Jobs H. Riahi – 12/11/15 EGI User Forum 2015

9. SixTrack (LHC machine) 9 H. Riahi – 12/11/15 EGI User Forum 2015 Original classic BOINC project for beam simulations Calculates stability of proton orbits in the LHC accelerator Based on experience from the Compact Physics Screensaver (CPSS) Ran SixTrack on desktop computers at CERN Outreach project for CERN’s 50th anniversary 2004 Also Year of Physics (Einstein Year) 2005 Renewed effort for LHC upgrade studies (HL-LHC) 12K Active Users 19K Active Hosts 35 TeraFLOPS

10. Test4Theory Theoretical fitting of all past experimental data Including from the LHC Using Monte Carlo simulation based on Standard Model Launched 2011 In partnership with the Citizen Cyberscience Centre (CCC) Pioneered use of Virtualization with BOINC Uses recent developments from CERN’s PH-SFT Group CernVM CernVMFS Wide range of potential (physics) applications In 2014 changed name to Virtual LHC@home Total of 1.7 trillion events simulated since 2011 10 H. Riahi – 12/11/15 EGI User Forum 2015

11. BOINC For LHC Experiments Use CernVM and virtualization Belong to LHC@Home projectLHC@Home Follow the Vacuum model Beauty @home (LHCb) In development since 2012 Communicates directly with DIRAC ATLAS@home Started as pilot beginning of 2014 ARC CE used to interface with BOINC PanDA for job management Supports simulations 11 H. Riahi – 12/11/15 EGI User Forum 2015

12. CMS@home Started development in Summer 2014 The integration with Analysis framework (CRAB) is achieved Available from the production Analysis infrastructure Supports users’ private MC generation The integration with Production framework (WMAgent) is on-going 12 H. Riahi – 12/11/15 EGI User Forum 2015

13. The Vacuum Model Inline with the cloud approach Common approaches Reduce costs Both development and operation Untrusted resource Authentication Validation 13 CernVM Agent Experiment Framework H. Riahi – 12/11/15 EGI User Forum 2015

14. 14 Async stage-out CRAB3 DataBridge VM Volunteer Agent Wrapper VBoxwrapper Grid GET Proxy DynaFed Ceph VCCS Volunteer CA CRT Happy User Task Glidein WMS Jobs GET Glidein glidein Join Condor Pool PUT Output FTS CMS@home architecture WebFTS EGI User Forum 2015 H. Riahi – 12/11/15

15. CMS@home Usage 15 H. Riahi – 12/11/15 EGI User Forum 2015

16. EGIFedCloud integration in WLCG 16 Andrew McNab et al. - HEPiX Spring 2015 - “The Vacuum Model“The Vacuum Model for running jobs in VMs” H. Riahi – 12/11/15 EGI User Forum 2015

17. 17 CMS Analysis workflow H. Riahi – 12/11/15 EGI User Forum 2015

18. Cloud accounting for WLCG 18  Only some WLCG sites joined the EGIFedCloud  Need to work with sites/VOs to deliver pledged resources Laurence Field – GDB Nov. 2015 – “WLCG Resource Reporting”“WLCG Resource Reporting” H. Riahi – 12/11/15 EGI User Forum 2015

19. EGIFedCloud resources for CMS 4 CMS sites have joined so far Pilot site: INFN-PRISMA-BARI Great infrastructure support by Marica Antonacci (Thanks!) Only dozens of cores Resources are mainly used for testing Pilot infrastructure for the vacuum model in CMS Used now for CERN Commercial Cloud/CMS@Home Consumer side accounting: cms-gangliacms-ganglia Daily log: DailyLogCMSDailyLogCMS CMS SAM tests: RAL-CMS-SAMRAL-CMS-SAM 19 H. Riahi – 12/11/15 EGI User Forum 2015

20. EGIFedCloud integration status for Analysis Vcycle instances at CERN It supports multiple providers (tenants) and VOs (contextualization) CRAB3 submission Submission to CRABServer@prod Job type: Analysis/PrivateMC Registration in CMS information system Job execution The job reads the input data, executed and stage-out the outputs Data is read via XrootD and the stage-out is done directly or via ASO CMS Job Monitoring Dashboard reporting 20 ✔ ✗ ✔ ✔ ✔ H. Riahi – 12/11/15 EGI User Forum 2015

21. Results 21 H. Riahi – 12/11/15 EGI User Forum 2015 Test of the execution of a privateMC workflow submitted with CRAB3 Generate minimum bias events with the Pythia MC generation 1000 jobs (100evt/job) Direct stage-out in a CMS Tier-2

22. Conclusions 22 Volunteer Computing can and is providing Significant additional computing resources Potentially O(100K) machines The experiments are trying to exploit this opportunistic resource Many @home projects exist or are in development The vacuum model has been adopted with success for exploiting both EGIFedCloud and volunteer clouds resources for LHC experiments Needs to attract WLCG cloud sites to integrate their private cloud in EGIFedCloud Work with them to update their usage report to join WLCG production environment H. Riahi – 12/11/15 EGI User Forum 2015

23. hassen.riahi@cern.ch laurence.field@cern.ch 23 H. Riahi – 12/11/15 EGI User Forum 2015