Continuous Integration (CI) Report Test Version Control Build By Jim Rush 1

Agenda About Me What is Continuous Integration Components of Continuous Integration Commonly used Tools Tour through an actual Jenkins deployment Exploration of major components Advice 2

About Me Development Manager building enterprise products for Call Centers I work for Enghouse, which is a Canadian holding (acquisition) company. The local office was once known as Syntellect. In the last seven years we’ve built various types of products that are built and validated by CI systems: Telephony (call control) API (middleware) and User Interfaces (Winforms & SilverLight) Multiple development languages (Java, C#, C++) Different test tools and frameworks Team consists of those fresh out of school to the young at heart me@jimrush.net www.jimrush.net 3

What is Continuous Integration continuous integration (CI) implements continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development. ---Wikipedia 4

Periodically, Validate your Project More Simply Periodically, Validate your Project 5

Periodically, Validate your Project More Simply Periodically, Validate your Project Periodically – as often as you can afford to. Ideally, anytime something changes Validate – insure project is operating within a margin of expected results Project – Everything that constitutes the creation, validation and maintenance of your software. 6

Why Know when you break existing functionality. Issues are typically easier to address the sooner they are found. Issues are typically easier to address when the number of changes between working and failing is small. Visibility into health of project Fail fast – If there is a problem, discover and plan for it early. Don’t wait until the end of your project to realize it doesn’t work. 7

Common CI Components Software Repository (version control) Build Software Build scripts (Batch files, Ant Scripts, Make files, Visual Studio) Monitor and trigger process Software package that monitors software repositories for changes and triggers a build Identifies and reports failures/successes Automated Testing Unit, Integration, System, X-ability testing Metrics Collection and Reporting ESLOC, Test Counts, Code Coverage, performance data, … 8

Common CI Workflow Continuous Integration Manager Version Control Build Unit Test Integration Test System Test Reporting / Monitoring / Metrics 9

Continuous Integration Manager Manages the CI Workflow Monitors Version Control Handles CI workflow and dependencies Reporting Common Tools Cruise control Jenkins/Hudson Custom scripts and code http://en.wikipedia.org/wiki/List_of_build_automation_software 10

Why do I like Jenkins Reasonably friendly web based administration, but uses easy to change XML for persistence Flexible Enables distributed build and testing Provides a plugin for nearly every other available tool in the CI workflow Various artifact repositories Various triggers Authentication system, including not having one Very rich set of reporting components 11

Very Quick Tour of Jenkins 12

Our Main Page 13

Job Configuration 14

17

A Build Job (Compiles Project) 18

A Test Job 19

Custom Test Detail Report 20

Version Control System (VCS) If you aren’t using Version Control, why not? It can be free It’s fairly easy to setup and manage a basic repository Why to have it Backups Tracking changes to time, person, feature, … Support multiple versions of code including exploratory versions Common Solutions Subversion (SVN) Mercurial GIT … 21

Building A build is any set of steps that creates a distribution Compiles(if needed) are usually technology driven Common tools: Apache Ant Maven Batch files / Shell scripts Common Challenges Repositories or other intermediate build components Error reporting Dependency management Build efficiency 23

Build Advice Build a complete product including installation Invest in quality build scripts Well documented Good logging Good error handling and reporting Build all distribution flavors Multiple project members can maintain the build Apply static code analysis or other tools that can validate your code Profile your builds Design builds to support distribution 24

Testing Testing Types Common automation tools/frameworks Unit – modular, isolated Integration – not modular nor isolated System – Complete distribution and dependencies Common automation tools/frameworks Code testing: xUnit (Junit, NetUnit,…) UI Testing tools: Accessibility layers and/or commercial products Custom scripts, frameworks and simulators Testing Results Junit XML files are the most flexible Test count Code coverage 25

Testing Advice Write quality tests cases as they can become harder to maintain than the product Provide good feedback for working and failing steps Test failure recovery can be hard Integration and Testing frameworks can be more complicated and more difficult than the project you are creating 26

Metrics Collect what is important or useful Most data is only valuable as a trend There are no magic goals for some data 27

Metrics Advice Unless you have a tool that captures what you want, build your own. Simple code to collect and a database is all you need. Writing small applications that grab data and generate HTML isn’t hard. Linking graphs to data tables in Excel is easy. Periodically review the data 28

Metrics presented with Excel 29

Detail Test Report Showing last 10 test results 30

Code Coverage with Cobertura 31

Thinking about your CI Infrastructure It can be general purpose trigger and scheduling tool It can be a reporting / metrics hub It doesn’t have to be singular Multiple sites / teams Different versions or types of projects 32

Parting Advice Treat your CI environment as a core part of your project Multiple people should understand it your CI system Knowledge is power (metrics/project insight), but use it carefully Data doesn’t have to be pretty, focus on content and value Continuously Improve – Allocate time to improve Don’t let your process or metrics make you stupid; focus on your goals and how you are going to get there There isn’t enough time to do everything, prioritize It is both easier, and harder than it appears Your infrastructure needs to be reliable enough to be trusted 33

Questions ? 34