1. EMI is partially funded by the European Commission under Grant Agreement RI-261611 Build and Test Services of the EMI project: Lessons Learned and Perspectives Andres Abad Rodriguez, CERN Duarte Bacelar de Begonha De Meneses, CERN Manchester, 10-04-2013

2. EMI INFSO-RI-261611 Situation Pre-EMI Challenges EMI Build and Test service ETICS infrastructure EMI metrics and reports Situation Post-EMI Benefits and lessons learned Conclusions Outline 10/04/2013EMI Tools2

3. EMI INFSO-RI-261611 Tools are needed for success climbing 10/04/2013EMI Tools3 EMI 1 Kebnekaise EMI 2 Matterhorn EMI 3 Monte Bianco Proper tools needed for success!

4. EMI INFSO-RI-261611 Situation Pre-EMI EMI Tools4 10/04/2013

5. EMI INFSO-RI-261611 Challenges EMI Tools510/04/2013 Different cultures, environments, requirements and priorities led to different tools difficult to merge Uniform the build environment and policies to provide a common software engineering workflow Obtain packages compliant with the main distributions guidelines (Fedora EPEL and Debian) Provide a comprehensive infrastructure able to satisfy the totality of needs Introduce QA metrics generation and reporting in the same tool-chain Support everybody on a new system

6. EMI INFSO-RI-261611 Situation in EMI EMI Tools6 10/04/2013 + Pbuilder Mock + Pbuilder Mock + Pbuilder Mock

7. EMI INFSO-RI-261611 Main EMI Build and Test service milestones 10/04/2013EMI Tools7 First year All Product Teams migrated to use the same set of tools and using the same platforms Metrics gathering and QA reports generation Second year Improve the system by adapting it to the new platforms and extending the infrastructure to use standard tools (Mock and Pbuilder) Provide more comprehensive QA reports Third year Consolidate the work done, specially with Mock and Pbuilder, and improve the general reliability and performance of the full system Build only from source packages

8. EMI INFSO-RI-261611 ETICS: EMI Build and Test Service EMI Tools810/04/2013 Worker Nodes

9. EMI INFSO-RI-261611 How it works All the jobs are executed in virtual nodes using an elastic infrastructure The nodes are reverted to the initial clean state after the jobs are finished Started on demand to manage high load in specific platforms Benefits Better use of hardware resources Standardization of platforms as all users are building and testing using the same images ETICS virtual node pool 9EMI Tools10/04/2013

10. EMI INFSO-RI-261611 First year SL5 (32 and 64 bits) Static pool of nodes (Fix number of nodes) 50 VMs, 2.340 jobs/month Second year SL5 and SL6 (32 and 64 bits) and Debian 6 (64 bits) Dynamic pool of nodes (Satisfy the demand with the same hardware) 290 VMs, 3.455 jobs/month Third year SL5 and SL6 (32 and 64 bits) and Debian 6 (64 bits) Dynamic pool of nodes (Improved load balancing) 310 VMs, 6.600 jobs/month Platforms and pool evolution 10/04/2013EMI Tools10 Pool of Nodes Jobs

11. EMI INFSO-RI-261611 EMI metrics and reports 10/04/2013EMI Tools11

12. EMI INFSO-RI-261611 First year Policies definition New QA plugins: SLOCCount, FindBugs, PMD, Checkstyle, PyLint, RPMLint, CPPCheck, etc... Development of the QA reporting tool Second year New plugins: RPMlint, CPPcheck, PyLint and IPv6 New reports: RPM compliance level Improved and more comprehensive QA reports Third year New plugin: Lintian New reports: DEB compliance level EMI metrics and reports evolution 10/04/2013EMI Tools12

13. EMI INFSO-RI-261611 Standardized tools and procedures Building from source (Mock and Pbuilder) A central build and test infrastructure is not needed anymore Some PTs have already selected their future tools. Among others: Situation Post-EMI 10/04/2013EMI Tools13

14. EMI INFSO-RI-261611 The system evolved together with the project to provide required functionality on time. Standardized software lifecycle process for all the PTs using standard tools. The products have been extended to new platforms. Generation of packages compliant with main distribution guidelines. Benefits and lessons learned 14 EMI Tools 10/04/2013

15. EMI INFSO-RI-261611 Questions? 10/04/2013EMI Tools15