1. Lecture 16 Formal Technical Reviews (FTRs) (also know as inspections) FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Don´t review in your spare time. Don´t let people get angry at the review meeting. Source: Software Formal Inspections Guidebook NASA Office of Safety and Mssion Guidance. NASA-GB-A302 1993. http://satc.gsfc.nasa.gov/Documents/fi/gdb/fitext.txt http://www.gpa.etsmtl.ca/cours/gpa777/pdf/fi.pdf

2. 19.9.2015Dr Andy Brooks2 A definition of an inspection. A static analysis technique that relies on visual examination of work products to detect defects, violations of development standards, and other problems. “... there is a fault here in Use Case 15...”

3. 19.9.2015Dr Andy Brooks3 A definition of an inspection. An inspection is a formal review of a work product by the work product owner and a team of peers looking for errors, omissions, inconsistencies, and areas of confusion in the work product. work products Software Requirement Specifications, use cases, UML class/state/sequence/activity/component diagrams, code, test cases,... Errors of omission can be difficult to detect... úrfelling

4. 19.9.2015Dr Andy Brooks4 Roles Author(s) Person or persons primarily responsible for creating a work product. The member of the inspection team that provides information about the work product during all stages of the inspection process and corrects defects during the rework stage. (Also known as (AKA) "Owner(s)".) Inspection Team A small group of peers who have a vested interest in the quality of the inspected work product and perform the inspection. This group usually ranges in size from 3 to 8 people and can be selected from various areas of the development life cycle (requirements, design, implementation, testing, quality assurance, user, etc.). Selected members of the inspection team fulfill the roles of moderator, author, reader, and recorder. Inspector A person whose responsibilities include reviewing work products created by others. All members should be considered inspectors in an inspection team.

5. 19.9.2015Dr Andy Brooks5 Roles Moderator Person who is primarily responsible for facilitating and coordinating an inspection. When there is no "Reader" in the inspection process, the moderator also controls the pace of review of the work product during the inspection meeting. Reader Person who guides the team during the inspection meeting by reading or paraphrasing the work product. The role of the reader is usually fulfilled by a member of the inspection team other than the author(s). All inspection methods don't use this role. Recorder Person who records, in writing, each defect found and its related information (location, severity, type, description) during the inspection meeting. (AKA "Scribe".)

6. 19.9.2015Dr Andy Brooks6 Do not criticise team members... Inspections are used to judge the quality of the work product,not the author of that work product. Managers should not participate in inspections. Results should be presented to managers statistically. If authors are criticised, they begin to find excuses for not participating. Inspectors can become reluctant to formally identify defects if they too are criticised.

7. 19/09/2015Dr Andy Brooks7 Shut up. This is not a fault. You are just stupid. Sit down. We are here to discuss things in a reasonable manner. This inspection meeting is not going very well. This is obviously a null pointer dereference! manual inspection meeting Such a poor meeting might not even report 20% of the defects... poor meeting/lélegur fundur

8. 19.9.2015Dr Andy Brooks8 Stages 1.Planning The period of time used to determine whether the work product to be inspected meets the entry criteria, plan the inspection, select the team, assign roles, schedule the inspection meeting, and prepare and distribute the inspection forms and materials. In addition, a determination is made on whether to hold an overview. If necessary the moderator divides up the work product into more manageable pieces. Inspection forms and materials include: the inspection announcement, the work product to be inspected, individual preparation logs, checklists, and inspection report form.

9. 19.9.2015Dr Andy Brooks9 Stages 2.Overview The optional stage in which the inspection team is provided with background information for the inspection. This stage may not be necessary if the team is already familiar with the work product being inspected. Overviews should be carried out if: the inspection team is not familiar with the product the work product is new or being inspected for the first time inspections are new to the project if novel techniques have been used in the work product.

10. 19.9.2015Dr Andy Brooks10 Stages 3.Preparation Key stage in which members of the inspection team prepare individually for the inspection by reviewing and finding potential defects in the work product being inspected. Potential defects found by individuals are discussed during the actual inspection meeting. Ad-hoc preparation is where inspectors simply use their own expertise to find and report potential defects. Focussed preparation may involve the use of special checklists or scenarios as inspection aids. Potential defects are recorded on individual preparation logs which should also record the time spent by each individual inspector. Ideally the moderator examines the individual preparation logs to check that everyone has prepared properly for the inspection meeting.

11. 19.9.2015Dr Andy Brooks11 Stages 4.Inspection Meeting Meeting where team members, as a group, review the work product to find, categorize, and record, but not resolve, defects. Moderator performs introductions and reminds everyone of roles. Reader begins a logical and orderly interpretation of the work product. Inspectors can interrupt at any time to report a potential defect. If consensus cannot be reached on whether or not a raised issue is a defect, an open issue should be recorded. Reading is resumed once the recorder has noted the potential defect (location, severity, type, description). Answers to questions: “Do we need a third hour?” “Do you we need to reinspect?”

12. 19.9.2015Dr Andy Brooks12 Inspection meeting failures Moderator dominates inspection –by talking too much or interrupting others Reader is too fast –by summarising too many paragraphs, diagrams, or lines of code at once Author is under attack –time spent making unnecessary comments directly at the author Author begins problem solving –time spent by the author verbalising possible solutions Recorder is too slow –inadequate preparation might mean the recorder does not understand the problem to be recorded

13. 19.9.2015Dr Andy Brooks13 Stages 5.Third Hour Optional additional time, apart from the inspection meeting, that can be used for discussion, possible solutions, or closure of open issues raised in the inspection meeting. 6.Rework Stage when the author corrects major defects found during the inspection. Minor defects are dealt with if the cost and schedule permit. Author receives a copy of the inspection report form and any relevant materials from the third hour. 7.Follow-up Short meeting between the moderator and author to determine if defects found during the inspection meeting have been corrected and to ensure that no additional defects have been introduced.

14. 19.9.2015Dr Andy Brooks14 Examples of entry criteria Clean compilation of code: –no errors or warnings All automatic tool checks have been performed: –static analysers –spelling and grammar checkers –CASE (Computer Aided Software Engineering) tools

15. 19.9.2015Dr Andy Brooks15 Examples of exit criteria All major defects have been corrected. All open issues have been resolved.

16. 19.9.2015Dr Andy Brooks16 Possible classifications of defect severity Major A defect, which if not identified and removed in a work product, could result in a test or customer/field reported problem. Minor All other defects. Critical A defect that might cause the system to crash or hang. Major A defect that could result in incorrect behaviour or results. Minor Defects that can be overlooked with no loss of system functionality.

17. 19.9.2015Dr Andy Brooks17 One classification of code defect types A. Variable naming standards B. Code layout standards C. Code documentation standards D. Missing or incorrect exception handling E. Misuse of variable types F. Unreachable code G. Inverted predicates H. Incomplete predicates I. Method lacking cohesion J. Improper sequencing of methods K. Incorrect access of array components L. Variables with unnecessary scope M. Infinite loop N. Initialization fault O. Inefficient code P. Failure to implement requirement If... then... else...

18. 19.9.2015Dr Andy Brooks18 Inspection rates: 1 page/hour? How many lines of code should be inspected assuming 90 minutes for individual preparation and 120 minutes for the inspection meeting? The faster the inspection rate (pages/hour) the smaller the defect density (defects/page). It is better to go slow and not try to do too much at once. Some argue for an inspection rate of 1 page/hour. Your teacher has carried out documentation reviews in industry (biotechnology software) and found it difficult to go faster than 1 page/hour (in individual preparation).