1. Presentation of the Quantitative Software Engineering (QuaSE) Lab, University of Alberta Giancarlo Succi Department of Electrical and Computer Engineering University of Alberta Honolulu, October 8, 2000 ISERN 2000 Meeting

2. October 8, 2000 G. Succi – QuaSE Lab – UoA2 The group James Miller Petr Musilek Marek Reformat Witold Pedrycz Giancarlo Succi 2 Visiting profs 1 PDF 12 Graduate students

3. October 8, 2000 G. Succi – QuaSE Lab – UoA3 Research interests Software metrics (definitions and tools) Advanced (Statistical and CI) models Sensitivity of cost models Certification of components and analysis of product lines Inspections Analysis of the nature of flexible methodologies … “Application of quantitative methods to software engineering”

4. October 8, 2000 G. Succi – QuaSE Lab – UoA4 Sponsors The University of Alberta The Alberta government ASRA NSERC Nortel CFI WaveRider Valmet

5. October 8, 2000 G. Succi – QuaSE Lab – UoA5 Brief overview of 2 projects Analysis of the ability of predicting defects using OO metrics Study of the occurrences of software service requests

6. October 8, 2000 G. Succi – QuaSE Lab – UoA6 Analysis of the ability of predicting defects using OO metrics quantify Investigate and quantify the impact of the object-oriented design on the defect- proneness of classes validate Empirically validate the ability of the object-oriented design metrics to identify classes with high number defects in commercial software applications evaluate Build and evaluate explanatory statistical models applicable for the count data

7. October 8, 2000 G. Succi – QuaSE Lab – UoA7 The adopted measures Object-oriented design metrics (Chidamber and Kemerer, 1991) Coupling Between Objects (CBO) Weighted Methods per Class (WMC/NOM) Lack of Cohesion in Methods (LCOM) WebMetrics Our tool: WebMetrics - metrics collection system Dependent variable – number of defects for a class Depth of Inheritance Tree (DIT) Number Of Children (NOC) Response For a Class (RFC)

8. October 8, 2000 G. Succi – QuaSE Lab – UoA8 Issues in the Statistical Analysis Distribution of the dependent variable? Count data Poisson Regression Equidispersion Equidispersion Negative Binomial Regression Gamma-distributed mean Underprediction Underprediction of zero values Zero-inflated Negative Binomial regression Two processes – different distributions

9. October 8, 2000 G. Succi – QuaSE Lab – UoA9 Building the models Ordinal Least Squares (OLS) vs. Maximum Likelihood (ML) ML - general solution for fitting model parameters Consistency: the probability that the ML estimator differs from the true parameter by an arbitrary small amount tends toward zero as the sample size grows Asymptotic efficiency: The variance of the ML estimator is the smallest possible Selection of predictors Stepwise regression based on the statistical significance Resulting models Univariate: RFC Bivariate: RFC and DIT

10. October 8, 2000 G. Succi – QuaSE Lab – UoA10 Comparison of the models Criticality prediction - Alberg diagram Goodness of fit and overdispersion

11. October 8, 2000 G. Succi – QuaSE Lab – UoA11 Study of the occurrences of software service requests Service Request demand for a modification of the software system behavior (early) life-cycle process attribute measure Our goal: Our goal: To define and validate a framework for pre-release SR analysis on three industrial datasets: effortresourcestime Predict effort, resources, and time to be allocated for a project number of SRs Predict the final number of SRs for a project basis for comparisonand assessment Provide a basis for comparison and assessment of different projects and development processes

12. October 8, 2000 G. Succi – QuaSE Lab – UoA12 SRs and Reliability Reliability failure-centric quality measure that views the software system as a whole from a customer perspective Software Reliability Growth Models (SRGM) models describing failure detection over time using calendar time, the number of tests run, or execution time The evaluated models: GO S-Shaped GO S-Shaped Goel-Okumoto Goel-Okumoto Gompertz Gompertz Hossain-Dahiya/GO Hossain-Dahiya/GO Logistic Logistic Weibull Weibull Weibull S-shaped Weibull S-shaped Yamada Exponential Yamada Exponential Yamada Raleigh Yamada Raleigh

13. October 8, 2000 G. Succi – QuaSE Lab – UoA13 Criteria Goodness of fit Accuracy of the final point Relative precision of fit Coverage of fit Predictive ability

14. October 8, 2000 G. Succi – QuaSE Lab – UoA14 Summary of Results

15. October 8, 2000 G. Succi – QuaSE Lab – UoA15 SRGM Sensitivity to Noise How sensitive are the models to the human factor in the SRs data recording? Monte Carlo analysis with normally distributed noise N(0,) added to the original data

16. October 8, 2000 G. Succi – QuaSE Lab – UoA16 Response Time and Gamma Analysis  When compared with other models, linear regression has the best performance

17. October 8, 2000 G. Succi – QuaSE Lab – UoA17 Future research plans Extend the analysis of software models with “more advanced” techniques Perform domain-specific studies, such as in product lines and extreme programming Analyse the evolution of the software market to determine driving forces of software development processes …

18. October 8, 2000 G. Succi – QuaSE Lab – UoA18 Netcraft survey: web server usage over the Internet

19. October 8, 2000 G. Succi – QuaSE Lab – UoA19 Expectation from ISERN In- and out- flow of ideas Sharing of experimental data and experimental protocols Exchange of visits Partnerships in projects (and funding proposals when possible) …