Mobile Apps Testing

1 Mobile is here to stay! “Not too long ago, PCs were a “fashion accessory” in mature markets with vendors linking themselves to fashion designers and even creating PCs specifically for women. The current ‘cool’ device is the smartphone, and now PCs will soon have to do battle with media tablets when they are launched in large numbers in the second quarter of 2011,” - Gartner "The mobile market is no longer just communication: It's Internet, music, video, Web, social networks, navigation, and everything else… The mobile ecosystem now embraces anything a consumer wishes to do either on the move or on a handheld device at home.” - Forrester

2 Mobility is everywhere…

3 Across all verticals… Media and Entertainment Banking & Financial Services Healthcare & Life Sciences Insurance Retail Technology Travel

4 And application features

5 Mobile market is growing in size…

6 App Revenue…

7 And complexity

8 The testing problem »Maintaining an inventory of handsets »Test coverage (due to complex test matrix) »Device makers, models and carriers »OS and browsers »Languages and test locations »Ability to use the service provider from remote location »Access to pre-commercial /enterprise devices »Test automation »Reusability of code across handsets »Ability to interact with hardware and use third party tools »Mobile service monitoring

9 Solution »Purchase popular devices for our lab »Using the right mix of emulators and devices »Device rental services »Renting lab time »Device cloud testing »Public »Private »Crowd Sourcing »Mobile and Automation COE

AppLabs Confidential 10 Tool and Platforms Automation ToolsPlatformsManufacturers Testquest CountdowniOSApple DeviceAnywhereAndroid OSRIM Perfecto MobileBlackberry OSNokia SquishWindows Phone 7Motorola UISpecWindows MobileHTC eggPlantSymbianSony Erricson SeleniumSamsung Fonemonkey LG TestComplete

11 Emulators vs Devices Testing on emulators can be a tempting, cost effective option to purchasing devices but miss out on issues: »Architecture and device specific features »Human interaction issues. »Accelerometer issues and multi-touch issues »Bandwidth and loading sequence »Wireless network behavior – Wifi and GSM signal drops »Data retention during signal drops »Device interrupts and multitasking »Unique localization related encoding issues

12 Device Cloud

13 Case Study Business Challenges »Critical launch window to gain market advantage »Minimize the cost involved in device compatibility testing »Reduce time for regression due to frequent updates »Creating an automation framework for testing across multiple mobile devices »Ensure quality of service in production environment Solution »Introduced early testing - Cycle 0 »Strengthened the Test Planning and introduced Masters for test case authoring »Developed an automation framework for regression »Provided 24x7 monitoring of the quality of service across various carries and locations »Automated common navigation patterns and created device independent reusable macros »Automated alerts for outages and slowdowns and weekly summary reports of service quality The client currently has an application that generates an extremely large number of customer transactions each day. The client was about to release it’s first set of native mobile applications for booking and had limited time before the scheduled release date. The new platform enables them to provide existing functionality as well as check-in functionality that will result in cost savings for the company. AppLabs rich experience in Airlines domain and mobile application testing helped achieve the targeted release date.

14 Case Study Benefits »Early defect detection and reduced defect leakage to Cycle 1 by 40% »Increased the test case authoring rate from 9-22 per day/engineer. »Time to market was reduced from 7 weeks to 3 weeks »Gained visibility into issues that impact end user experience »Third party validation of the service level performance »Warning of downtime to allow remedial action to minimize loss of revenue »Improved user experience through optimizations in infrastructure »Verification of scalability in order to sustain future growth »Growth in business volume due to increased customer satisfaction

AppLabs Confidential 15 Role of Mobile COE »Make recommendations regarding testing on device, emulators and/or device cloud »Keep a continuous watch on mobile market trends »Provide device recommendations and test matrix to maximize test coverage »Provide technology agnostic tool comparison and recommendation »Run pre-certification checklists to take care of testing nuances across various platforms.