1. LHC@home Dr Jukka Klem CHEP06 1 Public Resource Computing at CERN – LHC@home Philippe Defert, Markku Degerholm, Francois Grey, Jukka Klem, Juan Antonio Lopez Perez, Eric Mcintosh, Jakob Pedersen, Ignacio Reguero, Frank Schmidt, Ben Segal, Christian Soettrup

2. LHC@home Dr Jukka Klem CHEP06 2 Outline Public Resource Computing and BOINC Public Resource Computing and BOINC LHC@home project and applications LHC@home project and applications Physics results with LHC@home Physics results with LHC@home Precision of numerical results Precision of numerical results Statistics Statistics BOINC and grid computing BOINC and grid computing

3. LHC@home Dr Jukka Klem CHEP06 3 Public Resource Computing Also called global computing, volunteer computing Also called global computing, volunteer computing Based on BOINC (Berkeley Open Infrastructure for Network Computing) Based on BOINC (Berkeley Open Infrastructure for Network Computing) Platform for distributed computing using volunteered computing resources Platform for distributed computing using volunteered computing resources Profit from unused CPU cycles for scientific computing Profit from unused CPU cycles for scientific computing

4. LHC@home Dr Jukka Klem CHEP06 4 BOINC Infrastructure Each project runs a server identified by a master URL (e.g. http://lhcathome.cern.ch). Project components: Each project runs a server identified by a master URL (e.g. http://lhcathome.cern.ch). Project components:http://lhcathome.cern.ch

5. LHC@home Dr Jukka Klem CHEP06 5 Client Screensaver

6. LHC@home Dr Jukka Klem CHEP06 6 Basic Principles Communication initiated by the client Communication initiated by the client Security: BOINC uses code signing to prevent distribution of malicious executables. Each project has a key pair for code signing and the private key kept on network-isolated machine. Security: BOINC uses code signing to prevent distribution of malicious executables. Each project has a key pair for code signing and the private key kept on network-isolated machine.

7. LHC@home Dr Jukka Klem CHEP06 7 Redundant Computing Results from different hosts are validated using application specific function Results from different hosts are validated using application specific function Credit: numeric measure of how much a user has contributed. Important in motivating users. Credit: numeric measure of how much a user has contributed. Important in motivating users.

8. LHC@home Dr Jukka Klem CHEP06 8 BOINC Applications Public appeal Public appeal Independent parallelism Independent parallelism Easiest for low data/compute ratio Easiest for low data/compute ratio Usually available at least for Windows and Linux (also for MacOS, Solaris,...) Usually available at least for Windows and Linux (also for MacOS, Solaris,...) Existing applications in e.g. C, C++, Fortran can run as BOINC applications with small modifications (BOINC API) Existing applications in e.g. C, C++, Fortran can run as BOINC applications with small modifications (BOINC API)

9. LHC@home Dr Jukka Klem CHEP06 9 Some BOINC Projects SETI@home: look for extraterrestrial life SETI@home: look for extraterrestrial life Climateprediction.net: study climate change Climateprediction.net: study climate change Einstein@home: search for gravitational signals Einstein@home: search for gravitational signals Predictor@home: protein-related diseases Predictor@home: protein-related diseases Rosetta@home: cures for human diseases Rosetta@home: cures for human diseases World Community Grid: IBM project World Community Grid: IBM project Africa@home: African humanitarian causes Africa@home: African humanitarian causes LHC@home: improve LHC particle accelerator LHC@home: improve LHC particle accelerator

10. LHC@home Dr Jukka Klem CHEP06 10 LHC@home applications Main application: SixTrack. Others prepared: ATLAS fast simulation, Garfield, Geant4 (poster presentation) Main application: SixTrack. Others prepared: ATLAS fast simulation, Garfield, Geant4 (poster presentation) SixTrack application simulates protons as they travel around the LHC ring SixTrack application simulates protons as they travel around the LHC ring Superconducting magnets generate unwanted multipole field errors, available phase space area (dynamic aperture) for stable particle motion limited Superconducting magnets generate unwanted multipole field errors, available phase space area (dynamic aperture) for stable particle motion limited Each job typically tracks 60 particles 10 5 or 10 6 turns in the LHC (1-10 hours on a modern PC) Each job typically tracks 60 particles 10 5 or 10 6 turns in the LHC (1-10 hours on a modern PC) SixTrack program is part of SPEC CPU2000 benchmark suite SixTrack program is part of SPEC CPU2000 benchmark suite

11. LHC@home Dr Jukka Klem CHEP06 11 LHC@home Physics Results Phase space images of stable particle motion (up) and unstable chaotic motion (down). Phase space images of stable particle motion (up) and unstable chaotic motion (down). Chaotic motion predicts that the particle will be lost from LHC. Chaotic motion predicts that the particle will be lost from LHC. Map out conditions under which particle motion is stable. Map out conditions under which particle motion is stable.

12. LHC@home Dr Jukka Klem CHEP06 12 LHC@home Physics Results Long range and head-on beam-beam interactions reduce dynamic aperture Long range and head-on beam-beam interactions reduce dynamic aperture Different beam crossing schemes and effect of triplet errors studied Different beam crossing schemes and effect of triplet errors studied Average stable phase space area (Dynamic Aperture, DA) for different tune values. Average stable phase space area (Dynamic Aperture, DA) for different tune values.

13. LHC@home Dr Jukka Klem CHEP06 13 LHC@home Physics Results These studies would not have been possible without LHC@home resources These studies would not have been possible without LHC@home resources Large number of parameters can be carefully studied Large number of parameters can be carefully studied Results used in LHC design to provide more interesting collisions for the experiments Results used in LHC design to provide more interesting collisions for the experiments

14. LHC@home Dr Jukka Klem CHEP06 14 Precision of Numerical Results (1/2) LHC@home is a heterogeneous distributed system: different processors and operating systems LHC@home is a heterogeneous distributed system: different processors and operating systems Redundant computing (each job sent to different computers) allows to find differences in results Redundant computing (each job sent to different computers) allows to find differences in results IEEE-754 standard for floating-point arithmetic helps but does not specify everything needed (logarithm, trigonometric functions etc.) IEEE-754 standard for floating-point arithmetic helps but does not specify everything needed (logarithm, trigonometric functions etc.) Getting correct results depends on processor, operating system, programming language, compiler Getting correct results depends on processor, operating system, programming language, compiler Systems often optimized for performance Systems often optimized for performance

15. LHC@home Dr Jukka Klem CHEP06 15 Precision of Numerical Results (2/2) If particle motion chaotic, small errors can lead to large differences in final results If particle motion chaotic, small errors can lead to large differences in final results Some PCs give consistently wrong results (e.g. 10 year old desktop PC and one Linux batch PC) Some PCs give consistently wrong results (e.g. 10 year old desktop PC and one Linux batch PC) Many small differences in results found (due to log and exp functions on different processors) Many small differences in results found (due to log and exp functions on different processors) Solution: link executable statically to a portable library crlibm (https://lipforge.ens-lyon.fr/projects/crlibm/). Provides correct results on different platforms with performance cost less than 10%. Solution: link executable statically to a portable library crlibm (https://lipforge.ens-lyon.fr/projects/crlibm/). Provides correct results on different platforms with performance cost less than 10%.https://lipforge.ens-lyon.fr/projects/crlibm/ Need to be very careful if consistent results expected from heterogenous resources Need to be very careful if consistent results expected from heterogenous resources

16. LHC@home Dr Jukka Klem CHEP06 16 Some Result Statistics LHC@home has about 14000 active users in 108 countries LHC@home has about 14000 active users in 108 countries Users contribute about 25000 host computers Users contribute about 25000 host computers >800 CPU years processed for the LHC (assuming 1 CPU = 1 KSfp2K = 2.8 GHz Xeon) >800 CPU years processed for the LHC (assuming 1 CPU = 1 KSfp2K = 2.8 GHz Xeon) BOINC projects combined: about 200.000 users and 350.000 hosts BOINC projects combined: about 200.000 users and 350.000 hosts Estimate: 1 billion PCs in operation (less than 0.05% participate) Estimate: 1 billion PCs in operation (less than 0.05% participate)

17. LHC@home Dr Jukka Klem CHEP06 17 BOINC and Grids Bridges between BOINC and grid have been built for LCG and NorduGrid/ARC grid middleware Bridges between BOINC and grid have been built for LCG and NorduGrid/ARC grid middleware Sending jobs from BOINC to grid easier, grid to BOINC more difficult (security) Sending jobs from BOINC to grid easier, grid to BOINC more difficult (security) BOINC has lightweight infrastructure, some limitations with applications BOINC has lightweight infrastructure, some limitations with applications

18. LHC@home Dr Jukka Klem CHEP06 18 Useful links LHC@home server: http://lhcathome.cern.ch/ Please join the project! LHC@home server: http://lhcathome.cern.ch/ Please join the project! http://lhcathome.cern.ch/ Some background information: http://athome.web.cern.ch/athome/LHCathome/whatis.html Some background information: http://athome.web.cern.ch/athome/LHCathome/whatis.html http://athome.web.cern.ch/athome/LHCathome/whatis.html BOINC web site: http://boinc.berkeley.edu/ BOINC web site: http://boinc.berkeley.edu/ http://boinc.berkeley.edu/ Unofficial BOINC Wiki: http://boinc-doc.net/boinc-wiki/ Unofficial BOINC Wiki: http://boinc-doc.net/boinc-wiki/ http://boinc-doc.net/boinc-wiki

19. LHC@home Dr Jukka Klem CHEP06 19 Summary Public resource computing projects built using BOINC platform Public resource computing projects built using BOINC platform Can obtain large resources with low cost Can obtain large resources with low cost Results very useful for the LHC Results very useful for the LHC Numerical results checked with redundant computing. Have to be careful when using heterogeneous resources Numerical results checked with redundant computing. Have to be careful when using heterogeneous resources