Mobile Testing - Bug Report Portnov Computer School Copyright Ivette Doss 2013

Golden Bug Report Rules: 1. One Issue at the time 2. Determine if it is Reproducible: try to repeat the steps at least 3 times: type of bugs – bad code, bad data, degrade, intermittent… 3. Collect logs and screenshots 4. Clear Title and Description 5. Exact Steps to Reproduce 6. Device and Environment Details Copyright Ivette Doss 2013

The bugs that not reproducible immediately Some bugs may occur once out of three attempts. If you try to repeat it more times, and I may only be able to repeat it five times out of ten, or thirteen times out of twenty attempts. Then add the word “Intermittent” to the bug title and include those metrics. That just means that the programmer will have to try to repeat it a few times rather than once to be able to repeat the conditions and start fixing the bug. Copyright Ivette Doss 2013

Title and Description Brief description, usually based around a task or fulfilling a goal to put context around the bug. That helps the programmers figure out why they should fix the bug, and what sort of priority it should have. Here is an example: “While walking between Wi-Fi connections, I tried to submit a purchase, but the app crashed. This seems to happen when the Wi-Fi signal is weak.” Copyright Ivette Doss 2013

Exact Steps to Reproduce 1 Write down the steps, and practice it yourself. Also explore to see if there are any extra steps that you can remove and still see the problem. If you eliminate one or two steps and it no longer occurs, be sure to also note that in the report. This is useful information for diagnosing the source of the issue. Be sure to note anything in the environment when you are working with steps to reproduce bugs on mobile devices. Copyright Ivette Doss 2013

Exact Steps to Reproduce 2 Remember that movement activates sensors, different networks have better or worse performance, transitions between network types can cause problems, and low battery can cause the device to behave very differently. You may notice a problem when you are on the move, get back to your desk and be unable to reproduce it. I always take screenshots with the device (or take a picture with a different device) Copyright Ivette Doss 2013

Exact Steps to Reproduce 3 Once I work to repeat the bug, I will narrow things down: I had a low battery when the bug occurred. Does it also occur when the battery is charged? Maybe the Wi-Fi connection wasn’t at full connection. Can I reproduce it with full Wi-Fi? I noticed the bug while walking and moving the device, which gets sensors involved. Does it still occur when I am at my desk with the device sitting on it, without moving? In other cases, I might have been using a cellular network. Can I repeat it when I’m using a more powerful connection like Wi-Fi? Understanding these issues helps me create a much more accurate bug report. Copyright Ivette Doss 2013

What Should Have Happened Testers have great ideas on how apps can provide value to customers and users. We often use these motivations as we generate test ideas. We notice that there is a problem, but it may not be obvious to everyone else. Furthermore, the programmer may only have a vague idea of a better solution, so our ideas will help them as they solve the problem, or as they confer with designers and other developers. You can also find information in specifications, requirements documents, or in developer guidelines that are supplied with each mobile development framework program. This can help provide credibility to your bug reports. Copyright Ivette Doss 2013

Device and Environment Details At the minimum, always include the following: * Device manufacturer and model number (e.g. HTC One X) * Operating system version and sub-version (Android 4.0.3) http://www.softwaretestingmagazine.com/videos/how-real-tests-lead-to-real-progress/ http://www.softwaretestingmagazine.com/category/videos/ http://www.softwaretestingmagazine.com/videos/bad-software-testing-good-software-testing/ Copyright Ivette Doss 2013

Device and Environment Details You can find this information in the settings app on your device. Also be sure to include anything about the app, including: * The version of the software you are testing (Project Xena 1.01 Build 24) Here are some device details to consider including when relevant: * The date and time when you noticed the problem * Wireless network type * Low battery (can cause device to slow down) * Movement and sensor interaction * Using location services, such as GPS * Other activity on the device: other apps running, interruptions by OS, communication, etc. * Apps your app depends on, such as web browsers * Device heat (can cause device to slow down) Here are some environmental issues to include when relevant: * Network connection strength, or if a transition between two networks occurred * Light conditions and weather * User movement and travel speed Copyright Ivette Doss 2013

Supplementary Information The old saying that a picture says a thousand words holds true for bug reports. I may have a poor bug report, but an image or video capture demonstrating the problem may save the day. What I was unable to describe well I may be able to demonstrate perfectly. One of the most difficult pieces of supplementary information for beginners to gather are stack traces. Stack traces are complex error messages that are often suppressed from the user interface. You see a friendly message: “Oops, sorry, an error occurred” but in the background, more specific information is recorded and stored. Copyright Ivette Doss 2013

Golden Bug Report Rules: 1. One Issue at the time 2. Determine if it is Reproducible: try to repeat the steps at least 3 times: type of bugs – bad code, bad data, degrade, intermittent… 3. Collect logs and screenshots 4. Clear Title and Description 5. Exact Steps to Reproduce 6. Device and Environment Details 7. Supplementary Details: Screenshots and Logs Copyright Ivette Doss 2013