Presentation is loading. Please wait.

Presentation is loading. Please wait.

Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian.

Similar presentations


Presentation on theme: "Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian."— Presentation transcript:

1 Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by-sa/3.0/deed.en_US

2 Value Why bother?

3 www.eurostarconferences.com Are you working too hard, on the wrong things? We struggle to decide what to test, how much testing is enough, etc. 110% effort doesn’t cut it… www.eurostarconferences.com

4 Most of our work is wasted effort How much of our testing is on target?

5 www.eurostarconferences.com Most of our work is wasted effort How much of our testing is on target? Automated Tests 80% of reported bugs not addressed

6 www.eurostarconferences.com Know your users Parallel Kingdom [2] Regular users generate 2.5x daily revenues Custom drink feature removed [1] => 1 star feedback ratings Logos © respective owners [1] example from App Quality book [2] example from Tale of Two Apps

7 www.eurostarconferences.com Understand the effects Battery drain varied by 3x for similar hardware specifications Custom code added for Kindle Fire to reduce brightness – 40% less battery drain Higher network latencies reduced interactivity by 40% Users preferred Wi-Fi – 69% for Parallel Kingdom, 58% for StudyBlue Tablets 2x usage Pull-out keyboard also increased usage

8 www.eurostarconferences.com Analytics can augment our work Help us to correct and improve what we do Reduce waste, reduce latency, Increase value How Insights into the app’s behaviour in-the-wild Feedback loops

9 www.eurostarconferences.com Network profiling Volume Time (of day) Volume Time (of day) DiscoverUse

10 www.eurostarconferences.com Network profiling Volume Time (of day) Volume Time (of day) Transform

11 Raw ingredients / context How’s it all work anyway? Why do I care? What’s involved? What’s special about mobile apps?

12 www.eurostarconferences.com One ecosystem Web Server Some history Web server logs & analysis Web App Web Logs App Logs W3C format Graphs

13 www.eurostarconferences.com What’s different about mobile? Logs isolated on the device Connection not guaranteed Many more sensors Much more variation Mobile App App Logs Common System Log

14 www.eurostarconferences.com TOPOLOGY Overview of Mobile Analytics Each step may be delayed Data Collector Database Analytics WebServer Business view Filter(s) Mobile Apps sending Analytics data

15 www.eurostarconferences.com Types of Events Mobile app Analytics Library Analytics Collector 1:1 App-initiated m:1 App-initiated Library-initiated E1E1 E1E1 E E E4E4 E4E4 … L L E E EaEa EaEa L L E E EaEa EaEa Analytics Database Internet connection Events

16 www.eurostarconferences.com Analytical Questions: Past Trends, Defect Reports Trends, Defect Reports Software quality models, bottleneck analysis What’s Happened? (Reporting) How & why did it happen? (Factor analysis) How & why did it happen? (Factor analysis) Information Insight

17 www.eurostarconferences.com Analytical Questions: Present Engineering Activity, Benchmarking, Testing Specification refinement, asset reallocation What’s Happened? (Alerts) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight

18 www.eurostarconferences.com Analytical Questions: Future Extrapolation Failure prediction models What will Happen? (Forecasting) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight

19 www.eurostarconferences.com Analytics for Software Development Engineering Activity, Benchmarking, Testing Trends, Defect Reports Trends, Defect Reports Extrapolation Software quality models, bottleneck analysis Software quality models, bottleneck analysis Specification refinement, asset reallocation Specification refinement, asset reallocation Failure prediction models What’s Happened? (Reporting) What’s Happened? (Alerts) What will Happen? (Forecasting) How and why did it happen? (Factor analysis) How and why did it happen? (Factor analysis) What is the next best action? (Recommendation) What’s the best / worst that can happen?(Modeling / Simulation) Information Insight Past Present Future http://research.microsoft.com/pubs/136974/foser-2010-buse.pdf

20 www.eurostarconferences.com Implementation Flow Feasible Practical Useful Design events Try various libraries Propose mappings Implement events Field tests Evaluate libraries Commercial evaluations Minimalist Acceptable?

21 Gaining confidence Accuracy & precision

22 www.eurostarconferences.com Precision: repeatabilityAccuracy: on target Precision & accuracy https://en.wikipedia.org/wiki/Accuracy_and_precision

23 www.eurostarconferences.com Add images here: precision And here: accuracy Precision & accuracy

24 www.eurostarconferences.com Divergent answers increase doubt A tale of two three mobile analytics libraries (and what happens when bonuses are on the line…) Where were the testers (part one)?

25 The Dark Side Of Mobile Analytics

26 www.eurostarconferences.com DO NO HARM TO A NEIGHBOUR Bad stuff happened; Location data collected Excessive traffic Where were the testers (part two)? http://www.vam.ac.uk/users/node/1777 A Study of Third-Party Tracking by Mobile Apps in the Wild ftp://ftp.cs.washington.edu/tr/2012/03/UW-CSE-12-03-01.PDF

27 www.eurostarconferences.com Reducing precision to protect privacy 10km x 10km squares From: Capturing Mobile Experience in the Wild: A Tale of Two Apps Figure © ACM

28 www.eurostarconferences.com Beware the automation bias “Automation bias occurs in decision-making because humans have a tendency to disregard or not search for contradictory information in light of a computer- generated solution that is accepted as correct and can be exacerbated in time critical domains.” Automation Bias in Intelligent Time Critical Decision Support Systems http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.2634&rep=rep1&type=pdf “When presented with an automated solution 40% of pilots reasoned less or none at all”

29 Necessary but not sufficient Listening is a means to an end

30 www.eurostarconferences.com Complementary Feedback System logs Crash logs App Store ratings In-App feedback Speaking to humans. Oh, and software testing Polychrome, richer tones while avoiding cacophony

31 www.eurostarconferences.com Developer Console (Google Play)

32 www.eurostarconferences.com Instrument the Ecosystem ApplicationAnalytics Ecosystem Change Me!

33 www.eurostarconferences.com Observe the Behaviours Application Ecosystem Keys & GesturesNetwork Traffic

34 www.eurostarconferences.com Correlation? Causation? Appropriate? From: User Interaction-based Profiling System for Android Application Tuning Figure © ACM

35 Breaking-up is hard to do From one thing, to another

36 www.eurostarconferences.com Divorce can be messy, even for software Changing the code is the easy part… What about the data? And the systems & processes that rely on the data? Act in Haste, Repent at Leisure

37 Two ears to listen Are you willing to try?

38 www.eurostarconferences.com Willing to try? Become “one” with the data [Rob Lambert] Instrumenting their code, need to learn how to understand and use the data [Rob Lambert] Bringing DevOps to Mobile Apps? Internet-of-Things is coming [Andy Stanford-Clark] Be aware of what can go wrong [Isabel Evans]

39 www.eurostarconferences.com Further reading [4]http://www.howtomeasureanything.com/ [1,2]http://wip.org/ These books available at: [3]http://www.appqualitybook.com/

40 www.eurostarconferences.com Academic References Capturing Mobile Experience in the Wild: A Tale of Two Apps http://doi.acm.org/10.1145/2535372.2535391 User Interaction-based Profiling System for Android Application Tuning http://doi.acm.org/10.1145/2632048.2636091 Automation Bias in Intelligent Time Critical Decision Support Systems http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.2634&rep=rep1&type=pdf A Study of Third-Party Tracking by Mobile Apps in the Wild ftp://ftp.cs.washington.edu/tr/2012/03/UW-CSE-12-03-01.PDF All the papers are freely available online, e.g. using Google Scholar

41 Now? Later: julianharty@gmail.com Q&A Now? Later: julianharty@gmail.com Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by-sa/3.0/deed.en_US


Download ppt "Software Talks Are you listening? Julian Harty Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian."

Similar presentations


Ads by Google