Real World Use of Agile Software Development Methods C. Loomis (CNRS/LAL) Lancement du réseau LoOPS (LAL) 31 mai 2012

Agile Methologies (C. Loomis) Contents Biography StratusLab Project Adoption of Agile Adaptations to Reality Results & Statistics Towards DevOps Conclusions 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Biography Trained as physicist Worked at Fermilab (Chicago) and CERN (Geneva) Research engineer at LAL since 2001 European DataGrid (EDG), integration team leader EGEE-I, -II, -III, user community activity manager StratusLab, project coordinator Quattor Toolkit Developer of the pan language compiler 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) StratusLab Project Goal Create comprehensive, open-source, IaaS cloud distribution Demonstrate stability and production quality by supporting grid services Information 1 June 2010—31 May 2012 (2 years) 6 partners from 5 countries Budget : 3.3 M€ (2.3 M€ EC) Contacts Site web: http://stratuslab.eu/ Twitter: @StratusLab Support: support@stratuslab.eu CNRS (FR) UCM (ES) GRNET (GR) SIXSQ (CH) TID (ES) TCD (IE) 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Agile Methodologies Development process that is adaptable, rapid, and maintains quality. Many “theologies” and techniques: SCRUM, Kanban, XP, TDD, … Desire to manage human interactions as well as raw development 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) SCRUM Product owner: sets development priorities Sprint manager: ensures system is working correctly Team members: develop software/documentation Planning Meeting Demo Meeting Daily Standup 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Tools Git: Source code management JIRA: Issue tracking Maven: Uniform build framework Nexus: Build artifact repository Hudson: Continuous build & integration Hudson/SlipStream: Systems testing SlipStream/Claudia: Service deployment Communication: Telephone, Skype, EVO 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Results & Statistics 11 releases over 2 years (latest v2.0) Both academic and commercial deployments Variety of scientific domains using StratusLab Sprint Length: ~3 weeks No. Sprints: 26 Implemented Stories: 597 31 May 2012 Agile Methologies (C. Loomis)

Adaptations to Reality Academic project No real central control of personnel Agile requires commitment from all participants Developers not full-time on project Some constraints from EC project contract Requires high level of automation Need complete, integrated tool chain Must have automated unit and system tests 31 May 2012 Agile Methologies (C. Loomis)

Adaptations to Reality Developers distributed in 5 countries Teleconferencing for standup meetings Videoconferencing (EVO) for demo meetings JIRA provides distributed SCRUM board Hallway conversations via Skype Face-to-Face meetings essential Every 4-6 months to focus on integration tasks Improves human interactions and cooperation 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Problem Areas Release management Incomplete integration of release procedures into sprint procedures Certification included in latter part of project, but flow from developers to administrators not streamlined Tool support for what are essentially workflows is very poor and requires manual interventions 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Problem Areas Full system testing Bottleneck on system testing platform Only single deployment validated Difficult to trace failures to single change Certification not integrated into automated path 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) DevOps Integrate development and operations teams Ensure that operational concerns are met Release new features and fixes rapidly Fully automated tool chain Dashboard to monitor all aspects of process Deployment must be integrated into process Support for testing and deployment workflows Validate agile methods with DevOps 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Conclusions (I) Agile software development works: Series of solid releases with increasing functionality, deployed in both academic and commercial environments Hard-pressed to find similar productivity in other EC and non-EC project. Techniques must be tuned to the constraints of the project or development community 31 May 2012 Agile Methologies (C. Loomis)

Agile Methologies (C. Loomis) Conclusions (II) Towards DevOps: Must bring system administrators into development loop to ensure that their requirements are met Dashboard and fully automated development and deployment tool chain needed Looking forward to working on DevOps at LAL and potentially in a new EC project 31 May 2012 Agile Methologies (C. Loomis)