Software Defects Cmpe 550 Fall 2005

What leads to what? Introduced by ERROR Observed by FAULT FAILURE

Product defects Defect = Fault or Failure i.e. Defect is either the cause of failure or the failure itself (as the effect of the fault)

What constitutes failure Failing product deadlines Failing product budget (cost) Failing product quality

Defect removal costs (taken and adapted from software development data in Germany in the 90s) Require- ments Design Coding Component testing Integration testing System use 50% Fault Source 40% 10% 50% Fault Detection 10% 7% 5% 25% 3% KLm90 Per fault Cost KLm50 KLm25 Lm200 Lm200 Lm200

Availability Availability is the probability of getting system service at any given point in time. Availability = sys_uptime/(sys_uptime + sys_downtime) -or- Failure_hazard/(failure_hazard + repair_hazard)

Reliability Many defs… Musa et al.: “The probability that a system, or a capability of a system, functions without failure for a specific time or number of natural units” Sommerville: “The probability of failure-free operation for a specified environment for a given purpose”

Reliability facts The more failures are observed and corrected the more reliability improves  System usage must be clearly modelled Reliability depends on how a system is used  Reliability growth must be modelled System failures occur randomly  Failure must be modelled probabilistically

H/W vs S/W Failure Rates time time

Some measures MTTF: Mean time to failure MTTR: Mean time to repair The time between the system starts and failures MTTR: Mean time to repair The time a system is in repair MTBF: Mean time between failures The time between subsequent system failures

Our Research Aim to better allocate scarce resources in testing establish an improved method for predicting software quality identifying fault prone modules using machine-learning methods.

Our Research constructed a two-step model that predicts potentially faulty modules by taking their metric data into consideration datasets from NASA’s Metrics Data Program (MDP) utilizes classification and regression type learning methods consecutively. The results of the experiments show that the two-step model enhances the regression performance.

Machine Learning and Software Engineering Machine learning algorithms have been proven to be practical in poorly understood problem domains that have changing conditions Software quality problems can be formulated as learning processes and they can be classified by the characteristics of defect

Machine Learning and Software Engineering possible to apply regular machine learning algorithms to come up with a probability distribution for error analysis Decision trees, artificial neural networks, Bayesian belief network and clustering techniques ML is also used to generate models of program properties that are known to cause errors to classify and investigate the most relevant subsets of program properties Support vector and decision tree learning tools training and classification

Our Inspiration The previous research tackled the problem of software defect prediction by either using classification or regression type methods. Need to combine classification and regression type of prediction methods to find fault prone modules in software programs.

Our Model two step model that takes historical software metric data as its input In real life projects, we see that a dataset contains more defect free modules than defective ones Machine learning methods fail to predict defect densities accurately when regression is applied in such datasets our model first classifies the dataset as defective and defect free, and then, it applies regression. experiment results showed that classification is a better approach when the dataset has more defect free modules than defective ones.