Is it really possible to test my app? By Paweł Suszek Android testing Is it really possible to test my app? By Paweł Suszek

As simple as that? New Android project (ex. name „Abc”) Write some code New Android test project referencing our „Abc” project Write tests

Yes! See the examples! public class MyClass { public int multiply(int x, int y) { if (x > 99) { throw new IllegalArgumentException("X should be less than 100"); } // here is the error return x / y; public void testMultiply() { MyClass tester = new MyClass(); assertEquals("10 x 5 must be 50", 50, tester.multiply(10, 5));

Not really that simple… (I) Problem: functional/integration testing, UI testing, business logic testing Solution 1: Android specific classes that allows to manipulate UI, Activities flow Solution 2: Frameworks that extends the basic capabilities (triggering system events, scripts for UI testing in Python) Solution 3: Use Android SDK features just like in the application (ex. use threads, sleeping, locks waiting for callbacks, get system services like network/Wi-Fi manages, KeyGuard etc.)

Not really that simple… (I) Problem: functional/integration testing, UI testing, business logic testing Solution 1: Android specific classes that allows to manipulate UI, Activities flow Solution 2: Frameworks that extends the basic capabilities (triggering system events, scripts for UI testing in Python) Solution 3: Use Android SDK features just like in the application (ex. use threads, sleeping, locks waiting for callbacks, get system services like network/Wi-Fi manages, KeyGuard etc.)

Not really that simple… (I) Problem: functional/integration testing, UI testing, business logic testing Solution 1: Android specific classes that allows to manipulate UI, Activities flow Solution 2: Frameworks that extends the basic capabilities (triggering system events, scripts for UI testing in Python) Solution 3: Use Android SDK features just like in the application (ex. use threads, sleeping, locks waiting for callbacks, get system services like network/Wi-Fi manages, KeyGuard etc.)

Not really that simple… (I) Problem: functional/integration testing, UI testing, business logic testing Solution 1: Android specific classes that allows to manipulate UI, Activities flow Solution 2: Frameworks that extends the basic capabilities (triggering system events, scripts for UI testing in Python) Solution 3: Use Android SDK features just like in the application (ex. use threads, sleeping, locks waiting for callbacks, get system services like network/Wi-Fi manages, KeyGuard etc.)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Not really that simple… (II) Problem: Access modifiers may not allow to test certain methods/components Solution 1: test only public methods that use private ones Solution 2: use reflection (what if method name was changed in code? Test rewriting required) Solution 3: put test classes into class that needs to be tested (nested test class) Solution 4: give methods package access and create tests under same package tree Solution 5: write code in the way that allows testing (ex. utility classes, helper classes with public methods invoked both in the application code and in the test code)

Even more drawbacks… Android supports JUnit in version 3 (not 4 which is much more convenient to use) Android framework code has to run in the emulator or on a device (in the standard way, although we can use tools like Robolectric to bypass that): slows the test routine Dalvik VM doesn’t allow us to use some of the Java friendly frameworks (ex. JMock, EasyMock, PowerMock, BDD Behavior- driven development frameworks) Entry points for our applications are in the views (consider Activity as a view) what makes standard test routines more difficult to create as well as to design

What do we have here? Java Android JUnit 4 with annotations Parameterized tests Many frameworks JUnit 3 + Android-specific test classes Monkeyrunner (Python scripts) ExerciserMonkey (sends customizable sets of events to the device) JUnit4Android library to support JUnit 4 uiautomator Other frameworks (Robotium, Robolectic, Mockito, Espresso)

UI/App Exerciser Monkey Allows to test random set of events Can run a test that generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events Can run directly from ADB with a bunch of parameters (events repeat interval, events types) You can restrict the flow between classes or packages Example: adb shell monkey -p pl.pasu.circleandcross - -pct-touch 60 --pct-nav 5 --pct-majornav 15 --pct- appswitch 10 --pct-anyevent 10 -v -v -v --throttle 500 100

monkeyrunner Python program for testing app Can send keystrokes and touch events Testing on multiple devices at once Can simulate user interaction and take screenshots to compare with the screenshots that are known to be correct Can run ADB commands

uiautomator Part of Android SDK (since API 16 / Android 4.1) uiautomatorviewer – tool to view UI items hierarchy, can be used to analyze UI of the application (not only by developer) Events triggering (button events, touch events, gestures, screen manipulation) Screen manipulations (orientation changes, turn display on/off) Take screenshots (and save them to file) Waiting for updates (ex. UI update) Interaction with UI widgets/components UISelector to find UI widgets that satisfies specific criteria (scrollable, checkable, enabled etc.); also allows to create the collections of widgets UIWatcher to check specific conditions (ex. Dialog opened)

Robotium Simple as adding a JAR to the project Easy to learn and intuitive use Can manipulate UI, Activity’s lifecycle Widely used, good support and Javadoc Simple and extended API (through Solo class) Can use Android API as it is using Android test framework and classes Can take screenshots Focused on UI only

Robolectic Fast configuration and easy use Can run Android code on JVM (faster, but on the other hand does not run in the real Dalvik VM environment) Can invoke Activity’s lifecycle events Can manipulate views Uses JUnit 4 (annotation and parameterized tests) Customization with Shadow classes Allows to test applications built with RoboJuice (DI framework) Still can use Android API calls

Mockito Intuitive and quite simple Mocking objects: A lot of other possibilities (arguments matching, verifying number of invocations, @Mock annotation, spy to call real methods and many others) LinkedList mockedList = mock(LinkedList.class); //stubbing when(mockedList.get(0)).thenReturn("first"); when(mockedList.get(1)).thenThrow(new RuntimeException()); //following prints "first" System.out.println(mockedList.get(0));

Android testing capabilities (I) Integration tests with ActivityInstrumentationTestCase2 class Extended assertions with MoreAsserts and ViewAsserts classes Tests can trigger system events (ex. change connectivity status by turning on/off Wi-Fi) indirectly via Intents using a helper application that has permissions to change the system status Can use whole JUnit testing framework capabilities if class do not call Android API Can easily separate Test project from the actual project (does not mess in the code) with the Eclipse JUnit Test Project creator Allows to test Activity’s life cycle (and simulate life cycle events) with a use of instrumentation API (allows to hook into Android framework with a InstrumentationTestRunner) Can test Activities in isolation with an ActivityUnitTestCase class Allows to deliver touch/click events with TouchUtils class or Instrumentation methods (also disabling keyguard to allow events to be processed by Activity) Allows testing asynchronous processing (ex. with CountDownLatch object): allows to wait for the async processing to finish

Android testing capabilities (II) Interact with the view’s widgets/components, allows to verify UI components properties (ex. width/height, visibility) Verify Activities flow (monitor the Activities start with an ActivityMonitor object), input validation Deliver lifecycle events, deliver Intents to Activity, changing configuration events (orientation, language etc.) Allows testing different screen sizes and resolutions Allows using of special mock objects (see android.test.mock package): allows to run tests without interfering with real data (ex. without affecting the real application’s database) Allows to test Application object (using ApplicationTestCase class), services (with ServiceTestCase), content providers (with ProviderTestCase2, MockContentResolver) Allows to test UI (with a use of @UIThreadTest annotation or runTestOnUIThread method from Instrumentation class) Allows to divide tests into small/medium/large groups with the annotations

Total time spent

Sources http://developer.android.com/ http://www.vogella.com/tutorials/AndroidTesting/article.html http://www.vogella.com/tutorials/JUnit/article.html http://docs.mockito.googlecode.com/hg/org/mockito/Mockito.html http://code.google.com/p/robotium/wiki/Getting_Started http://code.google.com/p/android-test-kit/wiki/Espresso http://robolectric.org/index.html

Questions? Please be gentle, ask only the simple ones…