Presentation is loading. Please wait.

Presentation is loading. Please wait.

Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian.

Similar presentations


Presentation on theme: "Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian."— Presentation transcript:

1 Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

2 Value Why bother?

3 Are you working too hard, on the wrong things? We struggle to decide what to test, how much testing is enough, etc. 110% effort doesn’t cut it…

4 Most of our work is wasted effort How much of our testing is on target?

5 Most of our work is wasted effort How much of our testing is on target? Automated Tests 80% of reported bugs not addressed

6 Know your users Parallel Kingdom [2] Regular users generate 2.5x daily revenues Custom drink feature removed [1] => 1 star feedback ratings Logos © respective owners [1] example from App Quality book [2] example from Tale of Two Apps

7 Understand the effects Battery drain varied by 3x for similar hardware specifications Custom code added for Kindle Fire to reduce brightness – 40% less battery drain Higher network latencies reduced interactivity by 40% Users preferred Wi-Fi – 69% for Parallel Kingdom, 58% for StudyBlue Tablets 2x usage Pull-out keyboard also increased usage

8 Analytics can augment our work Help us to correct and improve what we do Reduce waste, reduce latency, Increase value How Insights into the app’s behaviour in-the-wild Feedback loops

9 Network profiling Volume Time (of day) Volume Time (of day) DiscoverUse

10 Network profiling Volume Time (of day) Volume Time (of day) Transform

11 Raw ingredients / context How’s it all work anyway? Why do I care? What’s involved? What’s special about mobile apps?

12 One ecosystem Web Server Some history Web server logs & analysis Web App Web Logs App Logs W3C format Graphs

13 What’s different about mobile? Logs isolated on the device Connection not guaranteed Many more sensors Much more variation Mobile App App Logs Common System Log

14 TOPOLOGY Overview of Mobile Analytics Each step may be delayed Data Collector Database Analytics WebServer Business view Filter(s) Mobile Apps sending Analytics data

15 Types of Events Mobile app Analytics Library Analytics Collector 1:1 App-initiated m:1 App-initiated Library-initiated E1E1 E1E1 E E E4E4 E4E4 … L L E E EaEa EaEa L L E E EaEa EaEa Analytics Database Internet connection Events

16 Analytical Questions: Past Trends, Defect Reports Trends, Defect Reports Software quality models, bottleneck analysis What’s Happened? (Reporting) How & why did it happen? (Factor analysis) How & why did it happen? (Factor analysis) Information Insight

17 Analytical Questions: Present Engineering Activity, Benchmarking, Testing Specification refinement, asset reallocation What’s Happened? (Alerts) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight

18 Analytical Questions: Future Extrapolation Failure prediction models What will Happen? (Forecasting) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight

19 Analytics for Software Development Engineering Activity, Benchmarking, Testing Trends, Defect Reports Trends, Defect Reports Extrapolation Software quality models, bottleneck analysis Software quality models, bottleneck analysis Specification refinement, asset reallocation Specification refinement, asset reallocation Failure prediction models What’s Happened? (Reporting) What’s Happened? (Alerts) What will Happen? (Forecasting) How and why did it happen? (Factor analysis) How and why did it happen? (Factor analysis) What is the next best action? (Recommendation) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight Past Present Future

20 Implementation Flow Feasible Practical Useful Design events Try various libraries Propose mappings Implement events Field tests Evaluate libraries Commercial evaluations Minimalist Acceptable?

21 Gaining confidence Accuracy & precision

22 Precision: repeatabilityAccuracy: on target Precision & accuracy https://en.wikipedia.org/wiki/Accuracy_and_precision

23 Add images here: precision And here: accuracy Precision & accuracy

24 Divergent answers increase doubt A tale of two three mobile analytics libraries (and what happens when bonuses are on the line…) Where were the testers (part one)?

25 The Dark Side Of Mobile Analytics

26 DO NO HARM TO A NEIGHBOUR Bad stuff happened; Location data collected Excessive traffic Where were the testers (part two)? A Study of Third-Party Tracking by Mobile Apps in the Wild ftp://ftp.cs.washington.edu/tr/2012/03/UW-CSE PDF

27 Reducing precision to protect privacy 10km x 10km squares From: Capturing Mobile Experience in the Wild: A Tale of Two Apps Figure © ACM

28 Beware the automation bias “Automation bias occurs in decision-making because humans have a tendency to disregard or not search for contradictory information in light of a computer- generated solution that is accepted as correct and can be exacerbated in time critical domains.” Automation Bias in Intelligent Time Critical Decision Support Systems “When presented with an automated solution 40% of pilots reasoned less or none at all”

29 Necessary but not sufficient Listening is a means to an end

30 Complementary Feedback System logs Crash logs App Store ratings In-App feedback Speaking to humans. Oh, and software testing Polychrome, richer tones while avoiding cacophony

31 Developer Console (Google Play)

32 Instrument the Ecosystem ApplicationAnalytics Ecosystem Change Me!

33 Observe the Behaviours Application Ecosystem Keys & GesturesNetwork Traffic

34 Correlation? Causation? Appropriate? From: User Interaction-based Profiling System for Android Application Tuning Figure © ACM

35 Breaking-up is hard to do From one thing, to another

36 Divorce can be messy, even for software Changing the code is the easy part… What about the data? And the systems & processes that rely on the data? Act in Haste, Repent at Leisure

37 Two ears to listen Are you willing to try?

38 Willing to try? Become “one” with the data [Rob Lambert] Instrumenting their code, need to learn how to understand and use the data [Rob Lambert] Bringing DevOps to Mobile Apps? Internet-of-Things is coming [Andy Stanford-Clark] Be aware of what can go wrong [Isabel Evans]

39 Further reading [4]http://www.howtomeasureanything.com/ [1,2]http://wip.org/ These books available at: [3]http://www.appqualitybook.com/

40 Academic References Capturing Mobile Experience in the Wild: A Tale of Two Apps User Interaction-based Profiling System for Android Application Tuning Automation Bias in Intelligent Time Critical Decision Support Systems A Study of Third-Party Tracking by Mobile Apps in the Wild ftp://ftp.cs.washington.edu/tr/2012/03/UW-CSE PDF All the papers are freely available online, e.g. using Google Scholar

41 Now? Later: Q&A Now? Later: Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.


Download ppt "Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian."

Similar presentations


Ads by Google