Peer Review and Testing

Scope What will you know by the end of this training. What are the different reviews and review process? How to choose the review type based on the work product. What are the different types of testing? How to create test cases for different types of testing. How to handle constraints during reviews and testing. How to do causal analysis.

What Is Quality? Producer’s view Meeting customers’ expectations Stated Implied User’s view Fitness for purpose Six sigma definition Customer satisfaction

What Is the Goal of a Software Project? Detect and eliminate defects Early Effectively Before delivering the products to the customer

What Is a Defect? Deviation from specifications Deviation from expectations

Defect Detection Methods Reviews (static) Testing (dynamic)

Cost of Quality Cost of prevention Cost of detection Cost of rework

Cost of Detection Cost of reviews Cost of testing Cost of detection increases exponentially as the project progresses Cost of testing will be costlier than cost of review

Reviews

Reviews - Purpose Point out improvements Confirm those parts of a product that do not require improvements Achieve uniform quality in order to make the process more manageable Also used as a supplementary communication mechanism

Reviews - Advantage Can be applied early during development stage Less expensive than tests

Reviews - Disadvantage Static Dynamic behaviors cannot be studied completely

Planning for Reviews Documented plan for reviews should be available Plan should address Objectives Items which need to be reviewed Type of review for each item Schedule Resources

Types of Reviews Informal reviews Walkthroughs Formal inspection Formal offline review (FOR)

Informal Reviews No set way - no formalities Friends look at the work product before strangers Results are not reported Usually used to ready the product for formal reviews Reviewer could act as the reviser along with author No formal role of a peer review leader

Walkthroughs Author guides the progress of the review Usually walks through with a set of user scenarios Step-by-step manipulation of a procedure (or line by line in a code) One (desk walkthrough) or multiple reviewers (structured walkthroughs) at the same time

Walkthroughs Cont… No advance preparation is needed by the reviewers The peer review leader controls the flow of the review process Responsibilities of a PRL is the same as in formal inspection process

Formal Inspection Provides reliable information about technical matters of a work product to the management A written report on the status of the product Active and open participation of everyone in the review group, following some traditions, customs, and written rules for review Full responsibility of all participants for the quality of the review - that is, for the quality of the information in the written report

Formal Offline Review (FOR) Done by a peer Author and reviewer need not be face to face during review No formal role for a PRL Review findings are reported formally

A Typical Peer Review Scene Scenario

Types of Review - Appropriateness Requirements, HLD - formal inspection Prototypes - walkthrough Complex LLD, code - combination of walkthrough and FOR LLD, UTP, code - FOR ITP, STP - formal inspection

Reviews - When to use what You have just written a piece of code implementing a complex algorithm. What type of review would you suggest for this. Project ABC is a development project having 40 modules. The HLD has to be prepared offshore and sent to customer for approval. What type of review would you suggest before sending this to customer?

Usage of Tools like checklist Keeps objectives clear Ensures coverage of scope Reduces bias Record observations

Entry Criteria for reviews Development is complete as per the author Self review is done Points mentioned in the checklist is ensured during self review

Review Process Input Item to be reviewed, Scope of review Input items (e.g for LLD review, HLD will be the input) Supporting items like checklist, standards Review based on the scope of review Order of review to be functionality and then standards Log the defects

Review Process (Contd.) Record review comments/observations to a reasonable granularity For example “improper usage of memory” is an improper review comment Review the defects and rework Verify the rework

Roles and Responsibilities Management Author Reviewer Recorder Leader

Defect Logging Defects to be logged along with. Description of the defect. Priority. Severity. Symptoms. Phase originated in. Phase found in. Date reported.

Defect Resolution/tracking Prioritize Schedule and assign all activities Execute activities to fix defect Analyse check-out Make changes/test changes Review Report/log status

Defect Metrics Product quality metrics / defect density metrics Defects /size of the work product Example Review efficiency Review defects/effort Review effectiveness Review defects/total defects Testing efficiency Testing defects/effort

Review Completion / Stop Track the rate of defects every hour Stop when the rate drops below N QC takes a decision on completion based on defect density

Configuration Control in Reviews and Testing Configurable items to be under configuration management (during review and after approval) Some of the items to be placed under configuration control Test plans Test procedures Test data Test results Components to be tested Test environment should be configuration controlled

Causal analysis - Why ? Identify root causes of ‘undesirable effects’ Initiate corrective action To prevent defects To improve process capability

Causal Analysis - When ? Plan for tackling ‘Suspected / Expected Causes’ right at the start Do a causal analysis after the first few rounds of activities

Causal Analysis - How ? Identify Primary Causes / Defect Types Brainstorm

Causal Analysis - How ? Some of the defect types are Unwanted Functionality Validation incorrect/not done Look and Feel related Boundary values not handled Improper data mapping Incorrect Assignment / Declaration Inefficient code/logic

Causal Analysis - How ? Draw a Pareto Chart Pareto charts are a type of bar chart in which the horizontal axis represents the categories. Ordered in the descending order. A cumulative percentage line helps to judge the added contribution of each category.

Causal Analysis - How ? Identify top n primary causes / defect types (should cover atleast more than 50 % of the defects) Can be derived from the Pareto chart

Tools of Analysis - Pareto Chart • • • • • # Of Occurances • # Of Days

Causal Analysis - How ? Do a root cause analysis - Drill down from the primary cause / defect type Brainstorm

Root Cause Analysis What Is It? Root Cause Analysis Allows The Team To Focus On The ‘Vital Few’ Objectives Of Root Cause Analysis Determine, With Reasonable Confidence, What Is Creating The Problem(s) In a Process Allows Development Of Focussed Permanent Solutions

Causal Analysis - How ? Fishbone diagram. Also known as cause-and-effect, or Ishikawa diagram. Helps to organize brainstorming information about potential causes of a problem. Represents relationships among potential causes.

Cause & Effect Diagrams Measurement Methods Machinery Effect Potential Causes Materials Mother Nature People

Causal Analysis - How ? Identify corrective action for root causes Implement the corrective actions

Post Causal Analysis Track results of corrective action taken Ensure sustenance of corrective action

After a period of time Top causes would have changed Hence, repeat the cycle

Causal Analysis - Some tips Best Done by a group Participation of PM, TL, QC in analysis and review is important Place in perspective while identifying certain causes - ‘Lack of skills’ Word your causes suitably

Case Study

Thank You