1. Supporting Release Management & Quality Assurance for Object-Oriented Legacy Systems - Lionel C. Briand Visiting Professor Simula Research Labs

2. 4/5/20012 Project Simula, Telenor Erik Arisholm, Valery Buzungu

3. 4/5/20013 Can we help plan and manage new releases of a legacy system? Can we help focus V&V activities on high risk parts of the legacy system? Can we help predict the fault correction effort of a release after delivery? Can we help assess the impact of process change? Can we predict the risk associated with a change? => Use error and change data, code analysis

4. 4/5/20014 Initial Study Fault-proneness model to focus V&V First one in the context of a legacy, OO system Realistic model evaluation Cost-effectiveness analysis

5. 4/5/20015 Vision Package in tool (tree maps) Deployment & Training Update fault prediction models Perform changes Focused V&V Feedback: Fault-prone components Corporate learning learning Release Release Record Release Change and Fault Correction Data

6. 4/5/20016 COS and XRadar COS – large telecom system, evolved over 5 years, 30- 60 developers, ~130K, Java, ~2000 application classes XRadar –Code structural metrics (e.g., control flow complexity) –Code quality (duplications, violations, style errors) –Change and fault correction data over releases 12 to 15.1 (5 releases) of COS –# changes and #fault corrections per class

7. 4/5/20017 Class Fault probability is a function of Size, complexity, inheritance, and coupling of classes (XRadar, JHawk) Amount of change performed on classes Code quality in terms of violations, style errors, duplication Skills and experience Class and fault history Interactions are also likely

8. 4/5/20018 Building & Assessing a Prediction Model Logistic regression Four releases (R1 to R4) with fault and change data Dependent variable: Fault corrections on R3 Explanatory variables: R2 measurements plus fault and change history for R1 The model is applied to R4, using R2 (history) and R3 measurements

9. 4/5/20019 Data Analysis Explanatory variables are log-transformed –Alleviate the outlier problem –Helps account for interactions PCA Univariate analysis Multivariate analysis (Stepwise) –Balanced modeling data set –82 observations to build the model Cross-validation (R3) Cost-effectiveness (R4)

10. 4/5/200110 PCA Results PC1: size, import coupling, violations, duplication, style errors, change counts PC2: number of releases, structural change measures PC3: Cohesion PC4: Fan-in PC5: Class ancestors …

11. 4/5/200111 Univariate Analysis Results PC1 measures very significant Change counts and fault corrections in previous release also significant No inheritance or cohesion measure is significant Nor are historic variables

12. 4/5/200112 Multivariate Analysis History variables selected Inheritance variables too Due to interactions? Variablep-value Ncss0.0069 Javadocs0.0088 Ccn0.0137 LCOM0.0817 HIER0.0297 INST0.0221 NSUP0.0654 NSUB0.0425 FaultCorrections0.0098 ChangeCount0.0225 n1FaultCorrections0.0099 n1ChangeCount0.0297

13. 4/5/200113 Cross-validation

14. 4/5/200114 Cost-Effectiveness

15. 4/5/200115 Cost-Effectiveness (no history)

16. 4/5/200116 Caveats and Problems Assumption: Most of the faults in release n are related to changes in release n-1 and to a lesser extent n-2 It is difficult, in general, to collect data about cause-effect relationships between changes and faults No change effort data: size measures as surrogate? Usually several concurrent version “streams” and merges are taking place No centralized defect tracking system

17. 4/5/200117 Conclusions CE analysis seems practical Model seems cost-effective (cost reduction ~ 29%) History variables are very important predictors Still much room for improvement Currently undertaking more extensive data collection through the configuration management system