COMP8130 and 4130Adrian Marshall Verification & Validation INSPECTIONS Adrian Marshall

COMP8130 and 4130Adrian Marshall Inspections - 1 Requirements Design Code Unit Test Integration Test System Test Typical Software Defect Profile Delivered Defects by phase per KSLOC Rework Costs Thicker arrows mean more rework Software Inspection: An Industry Best Practice - IEEE CS 1996

COMP8130 and 4130Adrian Marshall Inspections - 2 Requirements Design Code Unit Test Integration Test System Test Typical Software Defect Profile with Inspections in early phases Reduced Defects by phase per KSLOC 5 (20) 10 (40) 15 (100) 7 (50) 3 (20) 1 (10) Reduced Rework Costs Software Inspection: An Industry Best Practice - IEEE CS 1996

COMP8130 and 4130Adrian Marshall Inspections - 3 Benefits of introducing early phase inspections Dramatic reduction in phase-phase defect transmission Order of magnitude reduction in delivered defects Significant gains in productivity (25% - 35% overall) Much less rework - 30% less overall cost & earlier finish Discovery of defects that testing doesn’t expose 30% to 100% net productivity increases 5 to 10 times reduction in test execution costs and time Reduction in maintenance costs of up to an order of magnitude Much lower defect correction backlash during integration Software Inspection: An Industry Best Practice - IEEE CS 1996

COMP8130 and 4130Adrian Marshall Inspections - 4 Dis-benefits of introducing early phase inspections Early investment of total development cost Isn’t “sexy” or “high tech” Not as simple as it seems Not always repeatable Software Inspection: An Industry Best Practice - IEEE CS 1996

COMP8130 and 4130Adrian Marshall Inspections - 5 Do’s and Don’ts of Inspections Don’t author bash Do provide adequate preparation time Do involve only those who are absolutely necessary in inspection sessions Software Inspections: An Effective Verification Process by Ackerman, Buchwald, Lewski - IEEE Software 1989

COMP8130 and 4130Adrian Marshall Inspections - 6 Informal Review by Associate(s) “Please take a look at this … ?” Usually no guidelines given to reviewer. Sometimes author wants certain sections reviewed as a priority. Often no format for returning comments - red pen is often the order of the day (which can be quite OK). Not very repeatable - separate reviews = different results.

COMP8130 and 4130Adrian Marshall Walkthroughs A Walkthrough Involves an author presenting his/her developed artefact to an audience of peers. Peers question and comment on the artefact as it is presented with the intent to identify defects. Mostly involves no prior preparation by the audience. Usually involves minimal documentation of either the process or any arising issues. Defect tracking is inconsistent.

COMP8130 and 4130Adrian Marshall Reviews Review Includes walkthrough-like processes. Artefact to be reviewed is distributed to a group of the author’s peers for constructive criticism. All feedback is collected and collated by some predefined deadline. A meeting is held once the collated material is ready. The meeting is a strictly moderated for delivery of results of review, and answering to criticisms or identified defects. Significant amounts of data can be collected. Virtually never repeated because of cost.

COMP8130 and 4130Adrian Marshall Fagan Inspections Five elements Six well-defined inspection steps Four well-defined inspection roles The formal collection of process and product data The intermediate/development product being inspected Supporting infrastructure Software Inspections: An Effective Verification Process by Ackerman, Buchwald, Lewski - IEEE Software 1989

COMP8130 and 4130Adrian Marshall Six Steps of Fagan Inspections - 1 Six well-defined inspection steps 1.Planning 2.Overview- for getting all inspectors up to speed 3.Preparation- to ready inspectors for meeting 4.Meeting- the main focus of time and effort 5.Rework- to resolve issues uncovered at meeting 6.Follow-up- to ensure issues have been resolved Software Inspections: An Effective Verification Process by Ackerman, Buchwald, Lewski - IEEE Software 1989

COMP8130 and 4130Adrian Marshall Six Steps of Fagan Inspections - 2 Six inspection steps Planning Inspection materials to meet entry criteria Arrange for appropriate participants Arrange meeting place Overview Educate participants on what is to be inspected Assign inspection roles to participants Preparation Participants learn material and prepare to fulfill their assigned roles Advances in Software Inspections by Fagan - IEEE Software Eng. Vol

COMP8130 and 4130Adrian Marshall Six Steps of Fagan Inspections - 3 Six inspection steps cont’d Meeting Find defects. No solution hunting! Rework Author reworks all defects Follow-up Verification by moderator (or whole inspection team) to assure that all proposed fixes have been effective and that no secondary defects have been introduced Advances in Software Inspections by Fagan - IEEE Software Eng. Vol

COMP8130 and 4130Adrian Marshall Four Roles of Fagan Inspections Four well-defined inspection roles Moderator Key participant. Functions as “player-coach” to bring out the best synergy among inspection team members Recorder Collects and records all defect information as it is produced during meeting. Can hold a dual role Reader (least understood) Provides perspective(s). For example - “If I were the implementer …” Producer / Author The person who produced the work being inspected Not always available Advances in Software Inspections by Fagan - IEEE Software Eng. Vol

COMP8130 and 4130Adrian Marshall Fagan Inspection Data The formal collection of process and product data Dates  Product distributed for inspection  Of inspection meeting  Rework was completed Type of inspection and whether initial or repeat Identity of product and inspectors Size of material inspected Duration times for preparation and inspection meeting Number, type (W / M / E) and severity (Min / Maj) of defects found Advances in Software Inspections by Fagan - IEEE Software Eng. Vol

COMP8130 and 4130Adrian Marshall Fagan Inspection Preliminaries The development product being inspected Entry criteria Existence of (inspected) product predecessors  Inspected, detailed design document exists for code module inspection Conformance to proforma standards for inspection  Page layout for ease of reading  Line and page numbering for referencing Satisfaction of automated checks Examples  Spelling and grammar checks done  CASE tool diagrams checked for consistency  Code is able to be compiled Advances in Software Inspections by Fagan - IEEE Software Eng. Vol

COMP8130 and 4130Adrian Marshall Fagan Inspection Support Supporting infrastructure Training of inspectors Applicability of inspections for type of project Updating inspection procedures/manuals Maintaining historical inspection database Analysis of historical inspection information for identification of improvements Inspection metrics Type-ing defects (Wrong, Missing, Extra) Advances in Software Inspections by Fagan - IEEE Software Eng. Vol