1. SA2.6 Task: EMI Testbeds Danilo Dongiovanni INFN-CNAF

2. EMI INFSO-RI-261611 10/17/2011 EMI AHM, Padova, Italy 2 Discussion Outline Intercomponent Testing Testbed –Nagios –EMI 2 and RC Continuous Integration –Multiplatform and Updates Timeline Large Scale Testing Testbed + EMI Preview

3. EMI INFSO-RI-261611 10/17/2011 EMI AHM, Padova, Italy 3 First Step in Basic Automatic (Integration) Testing: Nagios Status: –SAM Nagios probes monitoring services in place for dCache, most of gLite (not all), not yet for ARC, UNICORE, emiUI not yet supported WHO does WHAT ( EMI/EGI/NAGIOS meeting at EGI-TF Lyon) –EMI-PTs: develop probes + test them as a script on their resources –EMI-JRA1: provides meta-package listing the existing EMI probes and each probe will be responsible for handling its dependencies + provides info on probes configuration (frequency execution...) –EGI Nagios Team: WN framework, emiUI support, instance of the SAM framework, where EMI probes can be deployed and tested (support ARC, UNICORE and gLite info systems). –EMI-SA2.6: deploys Nagios server from EGI Nagios Team + probes (from Pts)

4. EMI INFSO-RI-261611 10/17/2011 EMI AHM, Padova, Italy 4 Nagios issues Info –EMI Infrastructure: https://twiki.cern.ch/twiki/bin/view/EMI/NagiosProbes –Nagios Probes specification: http://nagios.sourceforge.net/docs/nagioscore/3/en/pluginapi.html –EGI Nagios requirements: https://wiki.egi.eu/wiki/EMI_Nagios_probes –And the EGI Nagios Policies: https://tomtools.cern.ch/confluence/display/SAMDOC/Probes+Develop ment+Policy –Nagios-discuss list can be used to announce planned changes of the probes, new EGI requirements on the probes to EMI can go via RT Timing: PT write and test probes || Nagios team provides SAM Nagios Instance => SA2.6 integration testing –integration tests of nagios server + probes and services → difficult debug of source of problems (THIS needs cooperation from PT/NAGIOS!) –Probes status?

5. EMI INFSO-RI-261611 10/17/2011 EMI AHM, Padova, Italy 5 EMI 2 and RC Continuous Integration New Update policy (LINK → sec. 2.3), separation of Production / Component Release resources per Release:LINK –Production: RELEASE INTEGRATION CHECK → dedicated repo, updates triggered by Release Manager GGUS tkt, 1 week of SA2.6 manual tests + Nagios Monitoring IN PLACE FOR EMI-1 –RC: CONTINUOUS INTEGRATION → Nightly Build repo, updates triggered by PTs GGUS tkt, PTs tests + Nagios Monitoring TARGET FOR EMI-2 –New inventory (LINK): Resources grouped by Release/Prod/RC/Site + Nagios Status linkLINK Start ASAP with EMI-2 deployment: The AIM is to avoid deploying EMI-2 XXX times the week before EMI-2 Release Broadcast Testing Policy: Integration testing is on client products

6. EMI INFSO-RI-261611 10/17/2011 EMI AHM, Padova, Italy 6 EMI 2 and MULTIPLATFORM Multiplatform handling: –Testbed ready to deploy SL6 and Debian but not enough resources for 3 platform in parallel current: preEMI + EMI-1 * 2 ( prod + RC) soon: preEMI + EMI-1 * 2 ( prod + RC) + ( EMI-2 * 2 (prod+RC) ) * 3 (platforms!) + NAGIOS –1 at a time sequentially (switch on/off virtual boxes ? ) –Prioritize versions: EMI-1 Prod + EMI-2 RC + preEMI (keep just WNs ) –TESTS: just deployment tests on all platform + functionality on a default platform or do we need cross platforms Integration functionality testing ? (ex. Debian CREAM vs SL6 ARGUS) –Release Cycle must change accordingly, we need much more time to run same tests on 3 platforms

7. EMI INFSO-RI-261611 Large Scale / EMI Preview / External testing AIM: –Increase product quality: external (SITES/Users) testing is the best way to check things are working at production level –Increase HW resources –Increase “testing” coverage/effort 10/17/2011 EMI AHM, Padova, Italy 7

8. EMI INFSO-RI-261611 Large Scale / EMI Preview / External testing EMI Preview / Large Scale experience –We have a group of 16 sites/contacts which agreed (in april) to provide some resources / effort for preview. 5 already deployed services, –Unify access to available resources: a single testbed (TBD) –Actually run tests on it (TBD) –Problems: Intercomponent Level Testing and Large Scale testing effort underestimated Effort/HW demanding task Release process too short (we would need RC out for a while) Some sites do that for EGI RollOUT, not so useful to improve EMI before UMD –Possible strategies to improve: ASK A FRIEND: PTs suggest us friend sites with most relevant deployment scenarios / experience, sites wishing to shorten the release cycle (EMI- >UMD-> SITES) If not HW at least Effort: ask sites/users having testing scripts /suites to try them on EMI Testbed (testers VO) 10/17/2011 EMI AHM, Padova, Italy 8