1. CI/CD for Digital Apps
Streamlining the Dev, Test & Release Cycle in Enterprises
Guy Arieli, CTO, Experitest

2. 01
The digital application imperative and its challenges 02
Agenda
Leveraging continuous testing to build digital quality 03
Continuous testing in action 04
Q&A

3. The times they are a changing
Digital disruption is not a mere buzz. It is real and has a huge business impact.
What is the common theme between these businesses?
They were household names, but they did not respond in time to digital transformation.
Some of them no longer exist – blockbusters, radio shack
Some are bankrupt -
Some downsizing – macy’s
These companies did not address changing business models or did not provide the kind of experience that customers expect
Customers are very vocal these days and have a choce, so that the retaliation on not keeping up is even faster

4. I run a technology company that happens to have a bank…
CEO of a leading
North American bank
This is a recent quote from JP morgan – but I don’t believe we can mention it
The point is that technology and applications is becoming an increasingly strategic activity, it is make or break for companies these days.
Without compromising quality
All these changes mean that development organizations are under pressure to release new functionality at a higher pace

5. Paradigm shift in software/ application development
Focus on speed, business agility and risk reduction
Agile
DevOps
All this has led in recent years to a complete change in the way applications are developed – all with the attempt to create digital services that meet customer expectations and beat the competition, and do so very in a quick an agile way.
One of the first new methodologies is aptly named agile. It was focused on creating shorter development sprints with the attempt of making development more agile.
Next came the DevOps concept with the attempt to steamline internall processes and avoid breakdown in communications between departments. Collaboration was the main issue here.
CI CD focuses on creating an ongoing continuous processes both in the devlopemnt stages and in delivery stages.
Finally shift left realizes that quality is an essential part of the development process and brings in quality and testing early into the development process both to reduce cost but also to imprve ownership and help improve overall quality.
 
CI/CD
Shift Left

6. Short release cycle side effects
Shorter cycles =
More cycles
Same or more features in app
More cycles =
More regression tests *
The problem with more frequent release, is that it doesn’t change the number of regression tests that need to be done.
So that now the testing organization has to conduct substantially more testing.
Which causes growing tech, delays or low quality – unless you have extremely powerful automation
Growing technical debt.

7. There can be no shortening of release
cycles* without Continuous Testing
Reasons – too much to test, need high scale automation
Technical debt
* And high quality

8. The estimated* impact of a shortened feedback loop
R&D productivity will increase from 10% to 35% since more time will be invested on Dev-test activities
* Data source: Experitest customers

9. Rapid feedback significantly increases R&D effectiveness
Add 2 pie charts 9
* Data source: Experitest customers

10. 01
The digital application imperative and its challenges 02
Agenda
Leveraging continuous testing to build digital quality 03
Continuous testing in action 04
Q&A

11. The Continuous Testing process
Load app to cloud 6
Trigger test execution 7
Consolidate test results 8
Continuous Integration
Report
Test
Build
Commit tests
Test Engineer
Pull test source 4
Build test project 5
Trigger build
Pull app source 2
Commit app code 1
Build app 3
Development Engineer
Version approved? 9
Debug / update test suite 10 NO
YES

12. CI/CD best practices for Digital app testing
Implement a Digital device lab allowing remote access to a wide selection of browsers, mobile devices, emulators and simulators
Create a common test environment for developers and testers, complete with collaborative tools
CI/CD best practices for Digital app testing
Integrate your test environment with CI tools and case management systems for end-to end hands free automation
Implement continuous testing with large scale test automation, covering all mobile app test cases
Operate
Develop
Test
Feedback
Code
Build
Plan
Release
3 Pilars of CI/CD:
Automation
Collaboration
Data
Develop on real devices
Unit testing on devices and emulators in the cloud – all test cases and test data in the same place - early
Test tool – environment is across lifecycle
Integration with systems that manage application versions – for example airwatch?? = manage the entire process – continuous integration, continuous testing, continuous delivery
Integration with JIRA and QC – test case inventory? Ask Tom
Shared infrastructure for testing – mobile device sharing
Mass application deployment on devices
Reporting and root cause analysis
Common environment: if developers use the same framework as testers, they are preparing the test cases as well as feeding in test result data early.
It creates a common language. Also gives developers better tools to test themselves early on with automation – reducing unit testing efforts.
detailed reports, shared device inventory, physical device sharing for troubleshooting, root-cause-analysis tools
Large scale test automation executed directly from CI on all devices and OS
Full coverage of all mobile app test cases – all app types, full device control, 3rd party apps and integrations, device conditions and network conditions, device vitals
Collaborative tools for mobile developers, testers and ops – detailed reports, shared device inventory, physical device sharing for troubleshooting, root-cause-analysis tools
Centralized management of all mobile development and assurance resources: physical devices, applications, tests and users; management dashboards and QA analytics tools
Leverage end-to-end QA data analytics for rapid root cause analysis and constant QA status
Centrally manage devices, applications, projects and users

13. 01
The digital application imperative and its challenges 02
Agenda
Leveraging continuous testing to build digital quality 03
Continuous testing in action 04
Q&A

14. Implementing continuous testing for rapid feedback

15. Demo

16. How the SeeTest platform support hands free continuous testing
Epic
Story thinning
Prototype
Acceptance Criteria
Functional Spec
Product Owner
Analyst
Product Owner
Analyst
Designer
Analyst
Tester
Test Engineer
Dev Engineer
Dev-Test Feedback cycle – 1 hour
Showcase / Demo Sign-off
Acceptance testing
Automated testing
Build & Deploy
Code (incl. White box Test)
Product Owner
Tester
Dev & Test Engineer
DevOps Engineer
Dev & Test Engineer
Digital assurance Lab (SeeTestCloud)
Grid Execution
Test development
Test Analytics
Manual Testing
Dev & Debug
RESTful API to mass deploy application on targeted devices

17. 01
The digital application imperative and its challenges 02
Agenda
Leveraging continuous testing to build digital quality 03
Continuous testing in action 04
Q&A

18. We look forward to seeing you at our booth