Confidential Continuous Integration Framework (CIF) 5/18/2004

Confidential 2 Agenda Introduction to Continuous Integration Framework (CIF) The “story” of CI at Sabre Benefits Implementing CI Auto build and test tools CI processes What does CI mean to you? Future Directions

Confidential 3 Quality Driven Goals “ Establish a framework for processes, procedures, and automation to support zero tolerance and quality initiatives for all projects in automated build, test, and deploy processes”

Confidential 4 Continuous Integration “A fully automated build and test process that allows a team to build and test their software many times a day.” – Martin Fowler “Service Support group extends the definition to include deployment of software.” Components of an Automated Daily Build Single storage for all source codes Access to anyone needing current or previous versions Automated build process for a single command build Automated testing with a solid/complete suite of tests possible with a single command Automated deployment and fallback based on successful execution of test criteria Maintains current executable (best executable so far)

Confidential 5 Successful Build & CI Successful Build Latest sources checked out of central repository All files compiled Resulting object files linked, versioned and packaged for execution (war, jar, or zip format) System deployed and suite of tests run against the system If all steps execute without error or human intervention and all test passes, we have a successful build! CI characteristics Single source point Automated build scripts Self testing code Master build Good checking in policies “Effort of integration is exponentially proportional to the amount of time between integrations” –

Confidential 6 CIF Benefits Drive product/application quality: Daily build/compile Source code cross-referencing Code style checking Documentation Code coverage Dependency analysis Unit testing Load testing Regression testing System deployment/fallback using automated end to end CIF

Confidential 7 CIF Benefits Reduces certain problems that come with team development Integration bugs manifest early. Early is GOOD! Reduces the scope of the search for the bug Catches enough bugs to be worth the cost AUTOMATION Less chance of accidental loss Helps support ever shrinking development cycles Published results

Confidential 8 Implementing Auto-builds Many choices for project build system – ANT, Centipede, CruiseControl Choices for various metrics tools – which ones to use? Resolving issues and logging Auto deployment integration Centipede structure with metrics tools integration Auto fallback integration

Confidential 9 Build and Test Tools Krysalis Centipede: project build system Easy, extensible and powerful Apache style cross project pollination Open source Java based Active developer community Source control: ClearCase Other tools: JUnit: Regression testing framework JellyUnit: Jelly based JUnit testing mechanism JDepend: Design quality metrics for each Java package JMeter: Load testing functional behavior & performance measurement Clover: JCoverage (open-source) UML Java docs Source code xref

Confidential 10 Confidential CIF Process – Automation! Get latest code from central repository Clean compile, JAR Deploy codes to target environments Instrument codes for code coverage Perform unit testing Perform regression/performance testing against deployed code Initiate fallback if necessary Generate code coverage report, UML Javadocs, JDepend report, source code xref report Publish results to Web site result to project personnel

Confidential 11 Confidential Current Achievements CIF implemented with COMPLETE automation for Java projects 20 projects implemented CI framework Automated deployment/fallback capability All latest open source tools integrated for maximum benefit, effective use, and company cost saving CI implementation time reduced from many weeks to few days for a typical Java project Support resources reduced 10-fold with standard CIF framework

Confidential 12 Confidential Current Achievements Extended CI framework for C++ Extended C++Unit for build and repository management Extended C++ unit and regression tests Extended volume and performance measurement tools Building cross Java/C++ testing bridges Currently use unit test using positive flow characteristics but error paths for cross integration are very hard to test Building cross Java/C++ load and volume tools

Confidential 13 Confidential Future Directions Multiple repository support Clearcase and CVS Diagnostics and quality metrics Rate of bugs per phase Complexity/hot spot predicator for code/system Build automated test cases concurrently with use cases and use tools (e.g. Rational) to create traceability matrices Extend CI framework for TPF Extending tools for build and repository management Extending unit and regression tests –Leveraging existing Winrunner and Loadrunner tools Extending volume and performance measurement tools

Confidential 14 Confidential Issues Lack of standardized tools for TPF Many of the standard test and tool features in C++ and Java do not exist in TPF or are not easily exploitable Many open source tools do not include and, moreover, are not easily interoperated with TPF –Could Rational Test Realtime be a solution? Cross platform development is difficult due to lack of standard configuration management (CM) processes for deployment and development: Version naming convention, release notes and release schedule Completion criteria prior to deployment to different environments Branch naming convention and criteria to branch creation Parallel development and code merges CM process for product change/enhancement/bug fixes

Confidential Questions?