Analysis of CK Metrics “Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults” Yuming Zhou and Hareton Leung, Member, IEEE Computer Society IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 32, NO. 10, OCTOBER 2006

Renaat Verbruggen Contribution of Paper 1 This paper makes a number of contributions. First, using a publicly available data set(NASA), they present new evidence indicating an association between OO design metrics and fault-proneness, thereby providing valuable data in an important area for which limited experimental data is available.

Renaat Verbruggen Contribution of Paper 2 Secondly, they validate the association between OO design metrics and fault- proneness of classes across fault severity. To the best of their knowledge, in spite of its importance, there has been no such previous research.

Renaat Verbruggen Contribution of Paper 3 Thirdly, on the methodological front, the fault- proneness of classes is analysed using not just the familiar method of logistical regression but also applied machine learning techniques.

Renaat Verbruggen Main findings This study makes two main findings. First, most of the OO design metrics in this study are statistically related to the fault-proneness of classes across fault severity. Second, the predictive abilities of these metrics strongly depend on the severity of faults. More specifically, the design metrics are better predictors of low severity faults in classes than of high severity faults.

Renaat Verbruggen

Descriptive Statistics of the Classes

Renaat Verbruggen Validation Results of the Hypotheses

Renaat Verbruggen Conclusions 1 The CBO, WMC, RFC, and LCOM metrics are statistically significant across fault severity, while DIT is not significant for any fault severity. NOC is statistically significant with regard to ungraded/low severity faults, but in an inverse direction, i.e., a class with a large NOC means a low fault-proneness. The significance of NOC with regard to high severity faults cannot be tested because of the quasi-complete separation problem. However, the results of machine learning methods indicate that the faultproneness prediction usefulness of NOC with regard to high severity faults is poor and, in this, is similar to that of DIT.

Renaat Verbruggen Conclusions 2 The fault-proneness prediction capabilities of these metrics differ greatly depending on the fault severity used. When applied to the classification of classes as fault-prone and not fault-prone in terms of ungraded/ low severity faults, the logistic regression and machine learning models based on these metrics achieve a performance comparable to previous studies. When applied to fault-proneness ranking of classes in terms of low severity faults, the logistic regression model based on these metrics outperforms the simple model based on class size. In both cases, however, the prediction usefulness of these metrics with regard to high severity faults is limited.