Supporting Release Management & Quality Assurance for Object-Oriented Legacy Systems - Lionel C. Briand Visiting Professor Simula Research Labs

4/5/20012 Project Simula, Telenor Erik Arisholm, Valery Buzungu

4/5/20013 Can we help plan and manage new releases of a legacy system? Can we help focus V&V activities on high risk parts of the legacy system? Can we help predict the fault correction effort of a release after delivery? Can we help assess the impact of process change? Can we predict the risk associated with a change? => Use error and change data, code analysis

4/5/20014 Initial Study Fault-proneness model to focus V&V First one in the context of a legacy, OO system Realistic model evaluation Cost-effectiveness analysis

4/5/20015 Vision Package in tool (tree maps) Deployment & Training Update fault prediction models Perform changes Focused V&V Feedback: Fault-prone components Corporate learning learning Release Release Record Release Change and Fault Correction Data

4/5/20016 COS and XRadar COS – large telecom system, evolved over 5 years, developers, ~130K, Java, ~2000 application classes XRadar –Code structural metrics (e.g., control flow complexity) –Code quality (duplications, violations, style errors) –Change and fault correction data over releases 12 to 15.1 (5 releases) of COS –# changes and #fault corrections per class

4/5/20017 Class Fault probability is a function of Size, complexity, inheritance, and coupling of classes (XRadar, JHawk) Amount of change performed on classes Code quality in terms of violations, style errors, duplication Skills and experience Class and fault history Interactions are also likely

4/5/20018 Building & Assessing a Prediction Model Logistic regression Four releases (R1 to R4) with fault and change data Dependent variable: Fault corrections on R3 Explanatory variables: R2 measurements plus fault and change history for R1 The model is applied to R4, using R2 (history) and R3 measurements

4/5/20019 Data Analysis Explanatory variables are log-transformed –Alleviate the outlier problem –Helps account for interactions PCA Univariate analysis Multivariate analysis (Stepwise) –Balanced modeling data set –82 observations to build the model Cross-validation (R3) Cost-effectiveness (R4)

4/5/ PCA Results PC1: size, import coupling, violations, duplication, style errors, change counts PC2: number of releases, structural change measures PC3: Cohesion PC4: Fan-in PC5: Class ancestors …

4/5/ Univariate Analysis Results PC1 measures very significant Change counts and fault corrections in previous release also significant No inheritance or cohesion measure is significant Nor are historic variables

4/5/ Multivariate Analysis History variables selected Inheritance variables too Due to interactions? Variablep-value Ncss Javadocs Ccn LCOM HIER INST NSUP NSUB FaultCorrections ChangeCount n1FaultCorrections n1ChangeCount0.0297

4/5/ Cross-validation

4/5/ Cost-Effectiveness

4/5/ Cost-Effectiveness (no history)

4/5/ Caveats and Problems Assumption: Most of the faults in release n are related to changes in release n-1 and to a lesser extent n-2 It is difficult, in general, to collect data about cause-effect relationships between changes and faults No change effort data: size measures as surrogate? Usually several concurrent version “streams” and merges are taking place No centralized defect tracking system

4/5/ Conclusions CE analysis seems practical Model seems cost-effective (cost reduction ~ 29%) History variables are very important predictors Still much room for improvement Currently undertaking more extensive data collection through the configuration management system