1. Software Defects Cmpe 550 Fall 2005

2. What leads to what?
Introduced by
ERROR
Observed by
FAULT
FAILURE

3. Product defects Defect = Fault or Failure
i.e. Defect is either the cause of failure or the failure itself (as the effect of the fault)

4. What constitutes failure
Failing product deadlines
Failing product budget (cost)
Failing product quality

5. Defect removal costs (taken and adapted from software development data in Germany in the 90s)
Require-
ments
Design
Coding
Component
testing
Integration
testing
System
use
50%
Fault
Source
40%
10%
50%
Fault
Detection
10% 7% 5%
25% 3%
KLm90
Per fault
Cost
KLm50
KLm25
Lm200
Lm200
Lm200

6. Availability
Availability is the probability of getting system service at any given point in time.
Availability =
sys_uptime/(sys_uptime + sys_downtime)
-or-
Failure_hazard/(failure_hazard + repair_hazard)

7. Reliability Many defs…
Musa et al.: “The probability that a system, or a capability of a system, functions without failure for a specific time or number of natural units”
Sommerville: “The probability of failure-free operation for a specified environment for a given purpose”

8. Reliability facts
The more failures are observed and corrected the more reliability improves
 System usage must be clearly modelled
Reliability depends on how a system is used
 Reliability growth must be modelled
System failures occur randomly
 Failure must be modelled probabilistically

9. H/W vs S/W Failure Rates
time
time

10. Some measures MTTF: Mean time to failure MTTR: Mean time to repair
The time between the system starts and failures
MTTR: Mean time to repair
The time a system is in repair
MTBF: Mean time between failures
The time between subsequent system failures

11. Our Research Aim to better allocate scarce resources in testing
establish an improved method for predicting software quality
identifying fault prone modules using machine-learning methods.

12. Our Research
constructed a two-step model that predicts potentially faulty modules by taking their metric data into consideration
datasets from NASA’s Metrics Data Program (MDP)
utilizes classification and regression type learning methods consecutively.
The results of the experiments show that the two-step model enhances the regression performance.

13. Machine Learning and Software Engineering
Machine learning algorithms have been proven to be practical in poorly understood problem domains that have changing conditions
Software quality problems can be formulated as learning processes and they can be classified by the characteristics of defect

14. Machine Learning and Software Engineering
possible to apply regular machine learning algorithms to come up with a probability distribution for error analysis
Decision trees, artificial neural networks, Bayesian belief network and clustering techniques
ML is also used to generate models of program properties that are known to cause errors
to classify and investigate the most relevant subsets of program properties
Support vector and decision tree learning tools
training and classification

15. Our Inspiration
The previous research tackled the problem of software defect prediction by either using classification or regression type methods.
Need to combine classification and regression type of prediction methods to find fault prone modules in software programs.

16. Our Model
two step model that takes historical software metric data as its input
In real life projects, we see that a dataset contains more defect free modules than defective ones
Machine learning methods fail to predict defect densities accurately when regression is applied in such datasets
our model first classifies the dataset as defective and defect free, and then, it applies regression.
experiment results showed that classification is a better approach when the dataset has more defect free modules than defective ones.