Presentation on theme: "HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: Rev: 29 April 2013 Creative Commons."— Presentation transcript:
HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: firstname.lastname@example.org Rev: 29 April 2013 Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by-sa/3.0/deed.en_US
STATE OF AFFAIRS The industry is driving towards automated tests Manual testing is downgraded Agile projects need automated tests to provide fast & frequent feedback Development environments for Mobile Apps (SDKs) include Test Automation frameworks
AUTOMATED TESTS Unit Tests Continuous Builds UI Test Automation Scale out to run tests on multiple devices Virtual devices, remote devices, local devices Complex, sometimes fragile, ecosystems Challenges: device security models, limitations in frameworks, connectivity, immature domain, lack of support for multi-modal interfaces, etc.
BEYOND THE REACH OF AUTOMATED TESTS Test frameworks focus on code, API, or GUI, Not: Movement: acceleration, rotation, location Tactile, Haptic, audio Inputs or Outputs Aesthetics: look & feel, transitions User Experience (UX) Highly dynamic updates: video, audio output Complex inputs: Gestures, multi-touch, shaking Light: e.g. camera as an input device So how do we test these?
QOX Quality Of Experience (QOX) From the standpoint of a users holistic experience of using the mobile app. Essentially subjective http://en.wikipedia.org/wiki/Quality_of_experience UX Research, Evaluation, and Testing: strategies, tips and tools Mark J. Williams
IVE GOT FEELINGS TOO 6 hats of software testing The red hat – emotions, intuitions, feelings I've Got a Feeling: Emotions in Testing http://www.stickyminds.com/s.asp?F=S17922_COL_2 Here's an article I wrote a few years ago on the six hats of software testing http://archive.newsweaver.com/qualtech/newsweaver.ie/qualtech/e_article000666359.html http://archive.newsweaver.com/qualtech/newsweaver.ie/qualtech/e_article000666359.html and if you really have lots of time on your hands, here's a video of a talk I gave on the same topic at StarWest 2008 http://www.stickyminds.com/Media/Video/Detail.aspx?WebPage=117http://www.stickyminds.com/Media/Video/Detail.aspx?WebPage=117 An inconsistency between the product and the document might be surprising, frustrating, or annoying to that person, but what really matters is inconsistency between the product and the desire.
FAST, EFFICIENT INTERACTIVE TESTING Bugfests Move and rotate the phone Be mobile Bugfest: A group of people get together to test in parallel and in common. Developers on-hand for direct feedback.
LUNCH OUT TESTING A VIDEO APP Around 2 hours Around 10 testers Test the app while going out for lunch Examples of what the testers did Tourist trip on the top of a double-decker bus London Underground Out for a walk with sandwiches
ESTABLISH STRONG FOUNDATIONS Learn: The platforms, the norms, what users expect How the platform was designed to be used The application lifecycle The devices Common, and unusual problems
INSIGHTFUL TESTING RAPID software Heuristics Mnemonics to guide the testing Session-Based Test Management techniques Charter Time-boxed Simple, lightweight test reporting Debrief http://moolya.com/blog/2012/04/18/test-mobile-applications-with-cop-who-flung-gun/
CHALLENGES OF EXTENDED TESTING Loss of information Sensory deprivation Missing or incomplete data Loss of fidelity Lags, delays Loss of control How do we test rich interactive apps unless you have the device in your hand? In theory, theory and practice are the same. In practice, they are not. A device in your hand is worth 2 in the cloud, and 100 virtual devices.
Additional considerations Cultural norms & differences Generations Whats important to them? How can we empathize? Mobile Apps & Platforms International Culture? GSMA REVEALS FEARS OVER MOBILE PRIVACY ARE HOLDING BACK THE GROWTH OF MOBILE APPS IN LATIN AMERICA http://www.gsma.com/newsroom/gsma-reveals A South Korean woman using her mobile phone to buy a product at the virtual retail shop in Seoul. www.guardian.co.uk
EXAMPLES OF WHAT CAN GO WRONG What this means is that unlike the iPhone, which is most certainly the closest competitor on the market to this phone, the Storm's UI is not custom built for touch navigation -- touch navigation is added after the fact. Things which flow naturally on an iPhone -- flicking through lists, scrolling for a contact, moving around in a webpage or looking through photos -- feel inelegant and uncomfortable on the Storm. There's no inertia to movement, no assurance that your finger is the lynchpin to control of the device. The screen is sensitive enough, surely, but how its software reacts to those touches makes all the difference, and here the feeling is that you're never completely in charge of the phone. http://www.engadget.com/2008/11/19/blackberry-storm-review/ Image: http://worldwide.blackberry.com/gettingstarted/storm-9500.png
! XOR [*] PRACTICAL OUTCOMES [*] Not Exclusive OR
COMBINING STRENGTHS Automated Tests Fast feedback, technology and developer centric Interactive Tests Exploratory, rich interactions and dynamics Remote testing e.g. in other countries Combined Augmented, Cyborg testing e.g. automated preparation, data collection, etc. Learn to delegate & learn when to take control
SOME FURTHER READINGS http://www.wipconnector.com/download/GuideToTheParallelUniverse_3rdEdition.pdf http://www.enough.de/fileadmin/uploads/dev_guide_pdfs/Guide_12thEdition_WEB.pdf Testing and Test Automation for Mobile Apps Julian Harty Summer 2013 CRC Press Testing and Test Automation for Mobile Apps Julian Harty Summer 2013 CRC Press