1. VanarSena: Automated App Testing

2. App Testing Test the app for – performance problems – crashes Testing app in the cloud – Upload app to a service – App is automatically executed in the cloud – Emulating different inputs and conditions – Feedback to the developer pinpointing performance problems and crashes

3. VanarSena Automated Monkey – Automatically navigate the app in the emulator – Maximize coverage and speed – Emulate impatient users Record and Replay – Record app interactions and execution – Transaction-level replay of traces – Control external inputs Emulation Framework – Emulate different network conditions – Newt – GPS and sensor inputs – Instrumentation – Device configuration – Emulator and Instrumentation Performance and Crash Analysis – Differential performance – Drill down, Critical path, Exception path analysis

4. Monkey Phone Emulator App Binary Instrumenter Runner Instrumented App Binary Feedback Navigate

5. Maximize Coverage Hit Test – Find Invokable controls – Turn-off execution of event handlers in code – Hit controls and monitor handlers being called Static Analysis – Select controls that maximize static reachability Dynamic transaction analysis – Dynamic pick controls with transaction graphs that maximize coverage

6. Maximizing Speed When to interact next? – Static timeouts can drastically slow down the monkey Processing Completed Event – Runtime transaction tracking ProcessingCompleted Next InteractionInteraction

7. Impatient Users Interact when a transaction is ongoing On edges slowed down by external inputs – For e.g. network Next Interaction Interaction

8. Record and Replay Track user interactions – UI controls, structure Monitor execution of the app – Transactions – Record timing of events Track external inputs – Network data, Location, – Random number Relay user interaction events – Interleaved in time by the execution of transactions – Replay external inputs – Approximate replay

9. Runner Modes Automated Runner Replay Runner Hybrid Runner Provide traces to go past certain UI controls – For e.g. username/password Learn automated runner from many recorded traces

10. Newt to emulate various network conditions Proxy to control and replay network data Interpose on system APIs to control inputs – GPS, Sensors etc. Configure emulator for device configurations Instrumentation to control – memory and isolated storage Emulation Framework

11. What inputs matter to the app? When is performance bad? Critical path analysis – When does the critical path change Focus development efforts on optimizing the critical path Performance Analysis

12. Tightly control external inputs – Does including the analytics/ad control affect the performance on other transactions? – Which JSON library is better? Given my app data – Has the performance improved? Differential Performance

13. Exception path of the crash – Sequence of interaction – External input Drill down analysis – More instrumentation along the exception path Crash Analysis