1. SoberIT Software Business and Engineering Institute HELSINKI UNIVERSITY OF TECHNOLOGY 1 T-76.115 Software Project T-76.613 Software Testing and Quality Assurance Bugzilla Tutorial Mikko Rusama Researcher, SoberIT mikko.rusama@hut.fi 14.10.2003 Updated 14.10.2005 by Lauri.Svan at hut dot fi: Each group has its own bugzilla, therefore no restrictions to bugzilla use

2. SoberIT Software Business and Engineering Institute 2 Contents  Introduction  What is Bugzilla?  Who is using Bugzilla?  Before entering a bug  How to enter a useful bug report into Bugzilla  Bug life-cycle  Querying existing bug reports  In this course students are allowed to...  Bugzilla advanced features

3. SoberIT Software Business and Engineering Institute 3 What is Bugzilla?  Bugzilla is a bug- or issue-tracking system.  Bug-tracking systems allow individual or groups of developers effectively to keep track of outstanding problems with their product.  An Open Source product  covered by the Mozilla Public License  Written in Perl, uses MySQL database  ”De-facto standard defect-tracking system against which all others are measured”  See: http://www.bugzilla.org

4. SoberIT Software Business and Engineering Institute 4 Who is using Bugzilla?  Netscape/AOL  Mozilla.org  NASA  Red Hat Software  SuSe Corp  The Horde Project  AbiSource  Real Time Enterprises, Inc  Eggheads.org  Strata Software RockLinux Creative Labs (makers of SoundBlaster) The Apache Foundation The Gnome Foundation Ximian Linux-Mandrake http://www.bugzilla.org/docs216/html/faq.html

5. SoberIT Software Business and Engineering Institute 5 Before entering a bug  Make sure the bug has not been previously reported!  Use the Bugzilla Query Form  For more information, see the tutorial of how to find reported bugs. http://www.mozilla.org/quality/help/beginning- duplicate-finding.htmlhttp://www.mozilla.org/quality/help/beginning- duplicate-finding.html  Next, be sure that you've reproduced your bug using the latest build released  Development process may produce new builds even daily, and the bug you've found may already have been fixed.

6. SoberIT Software Business and Engineering Institute HELSINKI UNIVERSITY OF TECHNOLOGY 6 How to enter a useful bug report into Bugzilla http://www.mozilla.org/quality/bug- writing-guidelines.html

7. SoberIT Software Business and Engineering Institute 7 A useful bug report has two qualities 1. Reproducible  If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it "WORKSFORME" or "INVALID", and move on to the next bug. Every relevant detail you can provide helps. 2. Specific  The quicker the engineer can isolate the issue to a specific problem, the more likely it'll be expediently fixed.  If a programmer or tester has to decipher a bug, they may spend more time cursing the submitter than solving the problem  In testing WWW pages, try to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible.

8. SoberIT Software Business and Engineering Institute 8 A useful bug report  Useful bug reports are ones that get bugs fixed!  Be non-judgmental in reporting bugs.  Bug reports need to be non-judgmental, non-personal and non- inflammatory.  Reports should be written against the product, not the person, and state only the facts.

9. SoberIT Software Business and Engineering Institute 9 Getting started – Creating A Bug Report  From the Bugzilla main page, choose "Enter a new bug report".  T-76.115  https://newclass.soberit.hut.fi/GROUP_ALIAS/  If you haven't logged into Bugzilla already, you'll need to enter your email address, password, and press the "Login" button.  Userid is your e-mail address (in most cases, your HUT unix account name + @cc.hut.fi)  Password is the same as in the course return system

12. Where did you find the bug?

13. How important is the bug?

14. Who will be following up on the bug?

15. What can you tell the engineer about the bug?

16. Who is allowed to view this bug?

17. Submit! Create a template?

18. SoberIT Software Business and Engineering Institute 18 Where did you find the bug?  Product - In which product did you find the bug?  Version - In which product version did you find the bug?  Component - In which component does the bug exist?  Platform - On which hardware platform did you find this bug?  OS - On which Operating System (OS) did you find this bug?

19. SoberIT Software Business and Engineering Institute 19 Where did you find the bug?  Product - In which product did you find the bug?  In this course, group name is same as the product name  Version - In which product version did you find the bug?  We're not yet using this field. Just leave the default value as you found it.  Component - In which component does the bug exist?  By default, your product has only the ”default” component  Click on the ’Component’ link to see descriptions of each component.  Platform - On which hardware platform did you find this bug?  e.g. Macintosh, SGI, Sun, PC,...  If you know the bug happens on all hardware platforms, choose 'All'. Otherwise, select the platform that you found the bug on, or "Other" if your platform isn't listed.  OS - On which Operating System (OS) did you find this bug?  e.g. Linux, Windows NT, Mac OS 8.5.  If you know the bug happens on all OSs, choose 'All'. Otherwise, select the OS that you found the bug on, or "Other" if your OS isn't listed.

20. SoberIT Software Business and Engineering Institute 20 Severity - How important is the bug?  Severity: How damaging is the bug? The impact of a bug.  This item defaults to ’unspecified'.  To determine the most appropriate severity for a particular bug, click on the Severity link for a full explanation of each choice, from ’Critical’ to ’Enhancement’.  You should specify what severities you are planning to use

21. SoberIT Software Business and Engineering Institute 21 Severity Values  Unspecified – the default value, severity not specified  Blocker - Blocks development and/or testing work  Critical - crashes, loss of data, severe memory leak  Major - major loss of function  Minor - minor loss of function, or other problem where easy workaround is present  Trivial - cosmetic problem like misspelled words or misaligned text  Enhancement - Request for enhancement, ideas

22. SoberIT Software Business and Engineering Institute 22 Priority  This field describes the importance and order in which a bug should be fixed.  Utilized by the managers and programmers or engineers to prioritize their work to be done.  The available priorities are:  P1 Most important  P2  P3  P4  P5 Least important

23. SoberIT Software Business and Engineering Institute 23 Who will be following up on the bug?  Assigned To - Which engineer should be responsible for fixing this bug?  Cc - Who else should receive e-mail updates on changes to this bug?  You would not normally change either of these fields from their default values!  In this course default component owner is ”randomly” chosen, so you may need to change the values.

24. SoberIT Software Business and Engineering Institute 24 Who will be following up on the bug?  Assigned To: Which engineer should be responsible for fixing this bug?  Bugzilla will automatically assign the bug to a default engineer upon submitting a bug report; the text box exists to allow you to manually assign it to a different engineer.  Default owner is one of the students (the first one listed in the CSV input file)  To see the list of default engineers for each component, click on the ’Component’ link.  Every time this field changes, the status changes to NEW to make it easy to see which new bugs have appeared on a person's list  Cc: Who else should receive e-mail updates on changes to this bug?  List the full e-mail addresses of other individuals who should receive an e-mail update upon every change to the bug report.  You can enter as many e-mail addresses as you'd like; e-mail addresses must be separated by commas, with no spaces between the addresses.

25. SoberIT Software Business and Engineering Institute 25 To change email settings  page footer ’prefs’ link  ’Email settings’

26. SoberIT Software Business and Engineering Institute 26 What can you tell about the bug?  URL - On what URL did you discover this bug?  Summary - How would you describe the bug, in approximately 60 or fewer characters?  Description - What else can you tell the engineer about this bug?

27. SoberIT Software Business and Engineering Institute 27 What can you tell about the bug?  URL - On what URL did you discover this bug?  If you encountered the bug on a particular URL, please provide it (or, them) here.  If you've isolated the bug to a specific HTML snippet, please also provide a URL for that, too or, preferably, return to the bug after you've submitted it and add the HTML snippet as an attachment.  Summary - How would you describe the bug, in approximately 60 or fewer characters?  A good summary should quickly and uniquely identify a bug report. Otherwise, developers cannot meaningfully query by bug summary, and will often fail to pay attention to your bug report when reviewing a 10 page bug list. Think of it as a "title".  A summary of "Drag-scrolling any web page crashes Mac builds" is a useful title. "Crash" or "Drag Crash" would be examples of a bad title.

28. SoberIT Software Business and Engineering Institute 28 What can you tell about the bug? - Description  What else can you tell the engineer about this bug?  Provide as detailed of a problem diagnosis in this field as possible, including as much as possible of the following information:  Overview Description - More detailed expansion of summary.  Steps to Reproduce - The minimal set of steps necessary to trigger the bug. Include any special setup steps.  Actual Results - What the application did after performing the above steps.  Expected Results - What the application should have done, were the bug not present.  Build Date & Platform - Date and platform of the build that you first encountered the bug in.  Additional Builds and Platforms - Whether or not the bug takes place on other platforms or browsers.  Additional Information - Minimized HTML snippets, Talkback crash IDs, and any other debugging information. (ATTACHEMENT)

29. SoberIT Software Business and Engineering Institute HELSINKI UNIVERSITY OF TECHNOLOGY 29 Bug life-cycle Status and Resolution

30. Bug Life-Cycle NewAssigned Reopened ResolvedVerified Closed Reassign Unconfirmed Open StatesEnd States Transition is allowed from any open state to the ”Resolved” state

31. SoberIT Software Business and Engineering Institute 31 Bug Status – Open States  NEW - This bug has recently been added to the assignee's list of bugs and must be processed.  Bugs in this state may be accepted, and become ASSIGNED, passed on to someone else, and remain NEW, or resolved and marked RESOLVED.  ASSIGNED - This bug is not yet resolved, but is assigned to someone who thinks they can fix it.  From here bugs can be given to another person and become NEW, or resolved and become RESOLVED.  REOPENED - The bug was once resolved, but the resolution was deemed incorrect.  For example, a WORKSFORME bug is REOPENED when more information shows up and the bug is now reproducible. From here bugs are either marked ASSIGNED or RESOLVED.

32. SoberIT Software Business and Engineering Institute 32 Bug Status – Unconfirmed State  UNCONFIRMED - Nobody has validated that this bug needs to be fixed.  Users who have the correct permissions may confirm this bug, changing its state to NEW.  A bug may be directly resolved and marked RESOLVED but usually a bug will be confirmed by the person to whom it is assigned.  Usually, an UNCONFIRMED bug will be left unconfirmed until someone has verified that the bug the reporter submitted actually occurs.  Bugzilla administrator may specify the number of votes a bug in this product needs to automatically get out of the UNCONFIRMED state.  THIS BUG STATE IS NOT USED IN THIS COURSE!

33. SoberIT Software Business and Engineering Institute 33 Bug Status – End States  RESOLVED - A resolution has been made, and it is awaiting verification by the QA.  From here bugs are either re-opened and become REOPENED, are marked VERIFIED, or are closed for good and marked CLOSED.  VERIFIED- QA has looked at the bug and the resolution and agrees that the appropriate action has been taken.  Bugs remain in this state until the product they were reported against actually ships, at which point they become CLOSED.  CLOSED - The bug is considered dead, the resolution is correct, and the product the bug has been reported against is terminated or shipped.  Any zombie bugs who choose to walk the earth again must do so by becoming REOPENED. This state is rarely ever used.  NOTE: Resolution values can only be specified for bugs being in one of the end states!

34. SoberIT Software Business and Engineering Institute 34 Resolution  The resolution field indicates what happened to this bug.  Only bugs in”Resolved” state will be marked with one of the resolutions.  All bugs which are in one of the ”Open” states have no associated resolution.

35. SoberIT Software Business and Engineering Institute 35 Resolution  FIXED - A fix for this bug is checked into the tree and tested.  INVALID - The problem described is not a bug.  WONTFIX - The problem described is a bug which will never be fixed.  LATER - The problem described is a bug which will not be fixed in this version of the product.  REMIND - The problem described is a bug which will probably not be fixed in this version of the product, but might still be.  DUPLICATE - The problem is a duplicate of an existing bug. Marking a bug duplicate requires the bug number of the duplicate and that number will be placed in the bug description.  WORKSFORME - All attempts at reproducing this bug were futile, reading the code produces no clues as to why this behavior would occur. If more information appears later, please re-assign the bug, for now, file it.

36. SoberIT Software Business and Engineering Institute HELSINKI UNIVERSITY OF TECHNOLOGY 36 Querying existing bug reports

37. SoberIT Software Business and Engineering Institute 37 Specify Product

38. SoberIT Software Business and Engineering Institute 38 Search Options

39. SoberIT Software Business and Engineering Institute 39 Hints about querying  You don't have to fill out any field on the query page you don't need.  On the list boxes, such as Status, you can Ctrl- Click to unselect an option.  You may save your queries  ’My bugs’ link

40. SoberIT Software Business and Engineering Institute 40 In this course students are  Allowed  Everything, including creating your own products, users etc.  Not allowed  None, each group has its own bugzilla instance

41. SoberIT Software Business and Engineering Institute HELSINKI UNIVERSITY OF TECHNOLOGY 41 Bugzilla Advanced Features Not Used In This Course!

42. SoberIT Software Business and Engineering Institute 42 Voting  Voting allows users to be given a pot of votes which they can allocate to bugs, to indicate that they'd like them fixed.  This allows developers to gauge user need for a particular enhancement or bugfix.  By allowing bugs with a certain number of votes to automatically move from "UNCONFIRMED" to "NEW", users of the bug system can help high- priority bugs garther attention so they don't sit for a long time awaiting triage.

43. SoberIT Software Business and Engineering Institute 43 Milestones  Milestones are "targets" that you plan to get a bug fixed by.  For example, you have a bug that you plan to fix for your 3.0 release, it would be assigned the milestone of 3.0.  Milestone options must be enabled by the Administrator  "usetargetmilestone" Param in the"Edit Parameters" screen.  To create new Milestones, set Default Milestones, and set the Milestone URL:  Select "Edit milestones" from the "Edit product"  Enter the name of the Milestone in the "Milestone" field. You can optionally set the "sortkey", which is a positive or negative number (- 255 to 255) that defines where in the list this particular milestone appears. This is because milestones often do not occur in alphanumeric order For example, "Future” might be after "Release 1.2". Select "Add".