2 Introduction Dynodroid is a system for automatically generating relevant inputs to Android appsIt is capable of generating both UI inputs and system inputs.It also allows interleaving inputs from machine and humanDynodroid views an app as an event-driven program
3 Usage Dynodroid was evaluated with 50 open source android apps The same was evaluated using human exercising and Monkey toolDynodroid, humans, and Monkey covered 55%, 60%, and 53% java code, respectivelyMonkey took 20X more events on average than Dynodroid.Dynodroid found bugs in top google apps
4 Dynodroid was built on following criteria Robust - Does the system handle real-world apps?Black-box - Does the system forgo the need for app sources and the ability to decompile app binaries?Versatile - Is the system capable of exercising important app functionality?Automated - Does the system reduce manual effort?Efficient - Does the system generate concise inputs, i.e., avoid generating redundant inputs?
5 WorkingThe main principle underlying Dynodroid is an observe- select-execute cycle.observes which events are relevant to the app in the current state.selects one of those eventsexecutes the selected event to yield a new state in which it repeats this process.
6 Dynodroid handles two types of events : UI events : Example - Touch , Gesture System events : Incoming message , request from another app.Satisfying the desirable criteria in the presence of system events is challenging because. a. Number of possible events is large b. Many events have structured dataDynodroid addresses the problem of handling a very large number of possible system events by using the observation
7 System Architecture Algorithm 1 Overall algorithm of Dynodroid. INPUT: Number n > 0 of events to generate.OUTPUT: List L of n events. L := empty list e := event to install and start app under test s := initial program state for i from 1 to n do append e to L // Execute event e in current state s to yield updated state. s := executor(e, s) // Compute set E of all events relevant in current state s. E := observer(s) // Select an event e 2 E to execute in next iteration. e := selector(E) end for
9 Data flow DescriptionThe executor triggers a given event using Android Debug Bridge (adb)For UI events, the ADB host talks to the ADB daemon (adbd) on the emulator via the monkeyrunner tool.For system events, the ADB host talks to the Activity Manager tool (am) on the emulator.The observer computes the set of events that are relevant to the app under test and consists of two kinds of events a. Hierarchy Viewer tool for UI events - E1 b.Instrumented SDK for system events - E2provides the set of all relevant events E = E1 + E2 to the selector
10 EXECUTORAllows human and machine to generate input to combine benefits of human and machine The executor listens to commands from a console and starts in human mode. RESUME command is received from the console. At this point, the executor switches to machine mode. Generates events until the given bound n is reached or a PAUSE command is received from the console. When Pause command is received , the executor switches to human mode again.Finally, the STOP command from the console stops Dynodroid
12 OBSERVERThe observer computes the set of relevant events after an event is executed Note: an event relevant if triggering that event may result in executing code that is part of the app a.UI Events These are events generated by the SDK in response to interaction by users with the device's input mechanisms. Dynodroid supports two input mechanisms: touchscreen and navigation buttons The observer analyzes the app's current UI state in order to compute relevant UI events. The SDK provides two ways by which an app can react to inputs to a UI element: by overriding a method of the corresponding View object's class, or by registering a callback with it
13 The SDK dispatches each input on the touchscreen to the root node of the view hierarchy The observer obtains the view hierarchy from a service called ViewServer and considers View objects at leaf nodeExtracts two kinds of data from Objects : (a) the set of callback methods registered and the set of methods overridden by the app,for listening to inputs to this UI element, and (b) The location and size of the UI element on the touchscreen.
14 The observer uses the data in item (a) to compute which UI events are relevant. (b) to compute the parameters of each such event
15 System EventsThese are events generated by the SDK in response to non-UI inputs, such as an incoming phone call, a change in geolocation, etc.The SDK provides one of two means by which an app can receive each type of System event: broadcast receiver and system service Observer used the same mechanism for extracting relevant inputs.A system event becomes relevant when an app registers to receive it and, conversely, it becomes irrelevant when an app unregisters it
16 Broadcast receiver events Android provides two ways for an app to register for system events via a broadcast receiver. Dynamically or statically In Dynamic : receiver's lifetime is from when Context.registerReceiver() is called to either until Context.unregisterReceiver() is called or until the lifetime of the registering app component. In the static case, the receiver is specified in file AndroidManifest.xml, and has the same lifetime as the app
17 System Service EventsSystem services are a fixed set of processes that provide abstractions of different functionality of an Android device.These services are distinguished based on app provided data and Independent data and are called internally and externally triggeredExample : AlarmManager service is internally triggered, as it depends on the alarm duration given by anapp, but the LocationManager service is externally triggered, as it depends on the device's geo-location.Dynodroid only controls externally triggered services
18 SelectorThe selector selects an event for the executor to execute from the set of relevant events E computed by the observer.There are 3 different selection strategies : Frequency, UniformRandom, and BiasedRandom .Frequency : The Frequency strategy selects an event from E that has been selected least frequently by it so far . The idea is that infrequently selected events have a higher chance of exercising new app functionality. The drawback is its deterministic nature leads the app to the same state in repeated runs.
19 UniformRandom : This strategy circumvents the above problem by selecting an event from E uniformly at random.This is the same Strategy used in Monkey Fuzz tool but there are differencesA drawback of the UniformRandom strategy is that it does not take any domain knowledge into account , it does not distinguish between UI events and system events, nor between different contexts in which an event may occur, nor between frequent and infrequent events.Ex : an event that is always relevant (an incoming phone call event )stands to be picked disproportionately more often than an event that is relevant only in certain contexts (only on a particular screen of the app)
20 BiasedRandom Like the Frequency strategy, it maintains a history of how often each event has been selected in the past in a context sensitive mannerThe context for an event e at a particular instant is the set E of all relevant events at that instantThis history is recorded in global variable G that maps each pair (e;E) to a score.At any instant, the score for a pair (e;E) in the map is either -1, meaning event e is blacklisted in context E or it is a non-negative integer,
21 BiasedRandomAlgorithm 2 Event selection algorithm BiasedRandom. 1: var G : map from (event, set of events) pairs to int 2: G := empty map 3: INPUT: Set E of relevant events. 4: OUTPUT: An event in E. 5: for each (e in E) do 6: if ((e;E) is not in domain of G) then 7: // Initialize score of event e in context E. 8: G(e;E) := init score(e) 9: end if 10: end for 11: var L : map from events to int 12: L := map from each event in E to 0 13: while true do 14: e := event chosen uniformly at random from E
22 15: if (L(e) = G(e;E)) then 16: // Select e this time, but decrease its chance of being 17: // selected in context E in future calls to selector. 18: G(e;E) := G(e;E) : return e 20: else 21: // Increase chance of selecting e in next iteration in 22: // current call to selector. 23: L(e) := L(e) : end if 25: end while 26: procedure init score(e) : int 27: case (e) of 28: Text event: return -1 29: non-Text UI event: return 1 30: system event: return 2 31: end case
23 EMPIRICAL EVALUATIONDynodroid was evaluated on real world Android apps, and compared it to two state-of-the-art approaches for testing such apps: manual test-ing and automated fuzzing. Two studies we performed: measuring app source code coverage and Finding bugs in apps Measuring Coverage - app coverage metrics by using Emma a popular Java source code coverage tool. Emma generates detailed line coverage metrics to the granularity of branches, and provides coverage reports in different formats that assist in analysis and gathering statistics
24 Evaluated approaches : Following were eveluated Dynodroid using each of the three selection strategies eventsMonkey fuzz testing tool provided in the Android platform - 10,000 eventsManual testing conducted in a study involving ten users - unlimited eventsDynodroid runs 5X slower than Monkey primarily due to performance issuesOn the plus side Dynodroid achieves peak code coverage muchfaster than Monkey, requiring far fewer than even the2,000 events we generated
25 Dynodroid is an effective bug finding tool and is also robust Dynodroid exposed several bugs on the 1000 most popular free apps from Google play
26 LimitationsDynodroid needs significantly fewer number of events to converge than monkey but it is 5X times slower than monkey .Dynodroid restricts apps from communicating with other apps
27 ReferencesAravind MacHiry, Rohan Tahiliani, and Mayur Naik. Dynodroid: An Input Generation System for Android Apps