Estimating Defect Density

True genius resides in the capacity for evaluation of uncertain, hazardous and conflicting information Winston Churchill

Challenges in Testing How many defects did the development team inject? How do we know we have found enough defects? When do we stop testing?

Is this Theory or Practical! Pragmatic managers think defect density is for theorists Managers believe number of defects in software cannot be estimated up front So we tested the theory in real life Why don’t you test if estimating defect density will work –Will not take more than a an hour of effort –Will open your eyes to applicability of EDD –CAVEAT: Development projects only

Try this at work! (with the help of experts) Identify a completed software development project Use a code counting tool to count SLOC (Source Lines of Code) in application e.g. 48,500 SLOC or 48.5 KLOC Check how many defects (total) have been logged by the testing team e.g. 900 Defects / KLOC = Defect Density e.g. 900/48.5 = Go on. Do it now and find out if the prediction works on projects in your organization!

So how does this work? Developers typically inject 20 defects for every 1000 lines of code ~6/KLOC will be found in reviews ~6/KLOC will be found in Unit Testing ~2/KLOC will be found in Integration Testing ~ 4/KLOC will be found in System Testing ~1/KLOC will be leaked to customer Customers might or might not find all the defects leaked to them! 20 Defects Code Reviewed Unit Tested Integration Tested System Tested Code Developed 14 Defects 8 Defects 6 Defects 1 Defect

The Chakkilam Crystal Ball Projects with poor code reviews and unit tests had more than 15 defects per KLOC found in the System Testing Phase –How well are you doing in your organization? Projects with code reviews and unit tests found less than 10 defects per KLOC –How well are you doing in your organization? Early warning system to predict testing effort based on code growth of the project Testers know how many more defects are likely to be found based on code size, development team, process followed Improve specific process like static testing, unit testing, integration testing based on data

When will it not work? Small projects – sub 5,000 lines of code Very large projects – more than 500 KLOC Projects with one or two developers –This is an average for a team Package implementation or maintenance projects

Thank You Thank You