1. Ray. A. DeCarlo School of Electrical and Computer Engineering Purdue University, West Lafayette, IN Aditya P. Mathur Department of Computer Science Friday September 30, 2004. COMPSAC 2004. Hong Kong. Modeling and Controlling the Incremental Software Test Process Scott Miller (CS) Department of Computer Science

2. The Problem What actions must the management take in order to adhere to process schedule?

3. Approaches 1.Use instinct and experience. 2.Use (1) supported by quantitative tools. (a)Use simulation: “forward” approach (b)Use (a) plus feedback control: “inverse” approach. Our approach..

4. Decision Support via Feedback Actual Process Parameter Estimation Progress Metrics Process Model Parameters Feedback Controller Estimated Future Schedule Deviation Suggested Decision Changes Mgmt. Control Management Decisions - Predicted Schedule   - + Desired Schedule Actual Schedule + Estimation Error

5. A Flow Model of Incremental Software Development/Test Test Specs Test Authoring Feat. Specs Test Code Coding Test Verification & Correction Reg. Cases Regression Testing Known Defects Code Debugging Project Code Latent Defects

6. Workforce Allocation Workforce allocated to particular tasks Effort is split across all active tasks

7. State-Model [Example] Equations System State Progress Feature Coding (fc) Code Debugging (dr) Test Authoring (ta) Test Debugging (td) Regression Testing (rr) Defect Model Development Testing

8. Variable Productivity Equation Human Productivity Workload Dependent (Csikszentmihalyi,’88) r b – Base Work rate c – Fractional size- dependent increase w c – Current workload size w n – Nominal workload size

9. The “Productivity” Eqns. Process Productivity (E.g. Feature Coding) Defect Introduction Defect Detection (Cangussu et al., ’02)

10. Control Strategy Model Predictive Control

11. Select Cost Functional E.g. Q 1,Q 2 := positive definite Calculate where S[x k, u k,k+P ]  x p k,k+P

12. Initial Study Data Collection Questionnaire Unavailable Data Estimated by Mgmt. Many “Linear Approximations” Many subjective estimates Results follow

13. Scheduled Simulated Expected Coding Progress Features Rel. 1 Features Rel. 2 Features Rel. 3 Inferred Actual

14. Expected Test Authoring Progress Scheduled Simulated Tests Rel. 1 Tests Rel. 2 Tests Rel. 3 Actual

15. Expected Test Debugging Progress Scheduled Actual Simulated Tests Rel. 1 Tests Rel. 2 Tests Rel. 3

16. Expected Regression Progress Scheduled Inferred Actual Simulated Regression Tests Rel. 1 Regression Tests Rel. 2 Regression Tests Rel. 3

17. Ongoing Study Data collection tool Objective parameter estimation Will reuse the tailored flow model Goal: Assess predictive accuracy Goal: Observe predictive accuracy over evolving process (i.e. training data growth)