CI/CD for Digital Apps Streamlining the Dev, Test & Release Cycle in Enterprises Guy Arieli, CTO, Experitest

01 The digital application imperative and its challenges 02 Agenda Leveraging continuous testing to build digital quality 03 Continuous testing in action 04 Q&A

The times they are a changing Digital disruption is not a mere buzz. It is real and has a huge business impact. What is the common theme between these businesses? They were household names, but they did not respond in time to digital transformation. Some of them no longer exist – blockbusters, radio shack Some are bankrupt - Some downsizing – macy’s These companies did not address changing business models or did not provide the kind of experience that customers expect Customers are very vocal these days and have a choce, so that the retaliation on not keeping up is even faster

I run a technology company that happens to have a bank… CEO of a leading North American bank This is a recent quote from JP morgan – but I don’t believe we can mention it The point is that technology and applications is becoming an increasingly strategic activity, it is make or break for companies these days. Without compromising quality All these changes mean that development organizations are under pressure to release new functionality at a higher pace

Paradigm shift in software/ application development Focus on speed, business agility and risk reduction Agile DevOps All this has led in recent years to a complete change in the way applications are developed – all with the attempt to create digital services that meet customer expectations and beat the competition, and do so very in a quick an agile way. One of the first new methodologies is aptly named agile. It was focused on creating shorter development sprints with the attempt of making development more agile. Next came the DevOps concept with the attempt to steamline internall processes and avoid breakdown in communications between departments. Collaboration was the main issue here. CI CD focuses on creating an ongoing continuous processes both in the devlopemnt stages and in delivery stages. Finally shift left realizes that quality is an essential part of the development process and brings in quality and testing early into the development process both to reduce cost but also to imprve ownership and help improve overall quality.   CI/CD Shift Left

Short release cycle side effects Shorter cycles = More cycles Same or more features in app More cycles = More regression tests * The problem with more frequent release, is that it doesn’t change the number of regression tests that need to be done. So that now the testing organization has to conduct substantially more testing. Which causes growing tech, delays or low quality – unless you have extremely powerful automation Growing technical debt.

There can be no shortening of release cycles* without Continuous Testing Reasons – too much to test, need high scale automation Technical debt * And high quality

The estimated* impact of a shortened feedback loop R&D productivity will increase from 10% to 35% since more time will be invested on Dev-test activities * Data source: Experitest customers

Rapid feedback significantly increases R&D effectiveness Add 2 pie charts 9 * Data source: Experitest customers

01 The digital application imperative and its challenges 02 Agenda Leveraging continuous testing to build digital quality 03 Continuous testing in action 04 Q&A

The Continuous Testing process Load app to cloud 6 Trigger test execution 7 Consolidate test results 8 Continuous Integration Report Test Build Commit tests Test Engineer Pull test source 4 Build test project 5 Trigger build Pull app source 2 Commit app code 1 Build app 3 Development Engineer Version approved? 9 Debug / update test suite 10 NO YES

CI/CD best practices for Digital app testing Implement a Digital device lab allowing remote access to a wide selection of browsers, mobile devices, emulators and simulators Create a common test environment for developers and testers, complete with collaborative tools CI/CD best practices for Digital app testing Integrate your test environment with CI tools and case management systems for end-to end hands free automation Implement continuous testing with large scale test automation, covering all mobile app test cases Operate Develop Test Feedback Code Build Plan Release 3 Pilars of CI/CD: Automation Collaboration Data Develop on real devices Unit testing on devices and emulators in the cloud – all test cases and test data in the same place - early Test tool – environment is across lifecycle Integration with systems that manage application versions – for example airwatch?? = manage the entire process – continuous integration, continuous testing, continuous delivery Integration with JIRA and QC – test case inventory? Ask Tom Shared infrastructure for testing – mobile device sharing Mass application deployment on devices Reporting and root cause analysis Common environment: if developers use the same framework as testers, they are preparing the test cases as well as feeding in test result data early. It creates a common language. Also gives developers better tools to test themselves early on with automation – reducing unit testing efforts. detailed reports, shared device inventory, physical device sharing for troubleshooting, root-cause-analysis tools Large scale test automation executed directly from CI on all devices and OS Full coverage of all mobile app test cases – all app types, full device control, 3rd party apps and integrations, device conditions and network conditions, device vitals Collaborative tools for mobile developers, testers and ops – detailed reports, shared device inventory, physical device sharing for troubleshooting, root-cause-analysis tools Centralized management of all mobile development and assurance resources: physical devices, applications, tests and users; management dashboards and QA analytics tools Leverage end-to-end QA data analytics for rapid root cause analysis and constant QA status Centrally manage devices, applications, projects and users

01 The digital application imperative and its challenges 02 Agenda Leveraging continuous testing to build digital quality 03 Continuous testing in action 04 Q&A

Implementing continuous testing for rapid feedback

Demo

How the SeeTest platform support hands free continuous testing Epic Story thinning Prototype Acceptance Criteria Functional Spec Product Owner Analyst Product Owner Analyst Designer Analyst Tester Test Engineer Dev Engineer Dev-Test Feedback cycle – 1 hour Showcase / Demo Sign-off Acceptance testing Automated testing Build & Deploy Code (incl. White box Test) Product Owner Tester Dev & Test Engineer DevOps Engineer Dev & Test Engineer Digital assurance Lab (SeeTestCloud) Grid Execution Test development Test Analytics Manual Testing Dev & Debug RESTful API to mass deploy application on targeted devices

01 The digital application imperative and its challenges 02 Agenda Leveraging continuous testing to build digital quality 03 Continuous testing in action 04 Q&A

We look forward to seeing you at our booth