Presentation is loading. Please wait.

Presentation is loading. Please wait.

Understanding the Human Estimator Gary D. Boetticher Univ. of Houston - Clear Lake, Houston, TX, USA

Similar presentations


Presentation on theme: "Understanding the Human Estimator Gary D. Boetticher Univ. of Houston - Clear Lake, Houston, TX, USA"— Presentation transcript:

1 Understanding the Human Estimator Gary D. Boetticher Univ. of Houston - Clear Lake, Houston, TX, USA 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop Nazim Lokhandwala Univ. of Houston - Clear Lake, Houston, TX, USA James C. Helm Univ. of Houston - Clear Lake, Houston, TX, USA

2 Introduction Chaos Chronicles [Standish03]  300 billion dollars  250,000 new projects  1.2 million dollars per project 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

3 Boehm’s 4X 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

4 Types of Estimation [Jorgenson04] % Human-Based % Algorithmic and Machine Learners 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

5 Research Focus Number of Papers On Software Estimation in IEEE [Jorgenson02]  Human-Based Estimation (17%)  Other (83%) 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

6 Statement of Problem How do human demographics affect human-based estimation? Can predictive models be constructed using human demographics? 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

7 Investigation Procedure Collect demographics from participants Request participants to estimate software components Build models (Estimates vs. Actuals) Survey 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

8 Which Demographics? Basic Demographics Academic Background Work Experience Domain Experience 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

9 The Survey 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

10 Competitive Procurement Software Buyer Admin Buyer 1 Buyer n... Buyer Software Distribution Server Supplier 1 Supplier 2 Supplier n : Supplier Software 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

11 Sample Estimation Screenshots 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

12 Survey Results Screenshots 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

13 Data Collection Invitations Filtered Incomplete Records 122 Final Records 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

14 Participant Educational Background Most of the participants hold Bachelors or Masters Degrees MeanMaximum Standard Deviation Computer Science Undergrad Courses Grad Courses Hardware Undergrad Courses Grad Courses Management Information Systems Undergrad Courses Grad Courses Project Management Undergrad Courses Grad Courses Software Engineering Undergrad Courses Grad Courses nd International Predictor Models in Software Engineering (PROMISE) Workshop

15 Participant Work Experience MeanMaximum Standard Deviation (Years) Years of Experience As Hardware Project Manager Software Project Manager No of Projects estimated Hardware Projects Software Projects nd International Predictor Models in Software Engineering (PROMISE) Workshop

16 Participant Domain Experience Process Industry Procurement and Billing Domain Experience Standard Deviation Maximum (Years) Mean (Years) 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

17 Data Preparation INPUT= 69% zeros…Needs Consolidation Courses, Workshops, Conferences, Programming Exp. 45 attributed reduced to 14 attributes Highest Degree Achieved…Need Transformation OUTPUT= MRE=Abs (Total Actual – Total Est.)/(Total Actual) 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

18 Build Models Linear Regression (Excel) Non-Linear Regression (DataFit) Genetic Programming (GDB_GP) 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

19 GP Configuration 3 Settings 1000 Chromosomes 50 Generations 512 Chromosomes 128 Generations 1000 Chromosomes 128 Generations 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop 20 Trials each

20 Results: All Demographic Factors 1.87E E-17T-test Mean Non-Linear Regression Genetic Programming Linear Regression Non-Linear Regression Std. Error R Squared Genetic Programming Linear Regression Best Values of R Squared with Min. Std. Error T-Test between Average R Square Values 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

21 Results: Educational Factors E-13T-test Mean Non-Linear Regression Genetic Programming Linear Regression Non-Linear Regression Std. Error R Squared Genetic Programming Linear Regression Best Values of R Squared with Min. Std. Error T-Test between Average R Square Values 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

22 Results: Work Experience 1.54E E-19T-test Mean Non-Linear Regression Genetic Programming Linear Regression Non-Linear Regression Std. Error R Squared Genetic Programming Linear Regression Best Values of R Squared with Min. Std. Error T-Test between Average R Square Values 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

23 Results: Domain Experience 4.55E E-23T-test Mean Non-Linear Regression Genetic Programming Linear Regression Non-Linear Regression Std. Error R Squared Genetic Programming Linear Regression Best Values of R Squared with Min. Std. Error T-Test between Average R Square Values 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

24 Summary of All Experiments R Square Values Linear Regression Best Case Genetic Prog. Avg. Case Genetic Prog. Non-Linear Regression All Factors Education Only Work Experience Only Domain Experience Only nd International Predictor Models in Software Engineering (PROMISE) Workshop

25 Best Equation: All Factors. r 2 = ((Log (TechGradCourses + (TechGradCourses ^ ((Log TotWShops)/(Cos (TechGradCourses ^ ((ProcIndExp + (Cos (TechGradCourses ^ ((ProcIndExp + (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Cos (Log (Log (Log SWProjEstExp))))))))))))) / (TechGradCourses ^ (Log SWProjEstExp)))))) / (((Cos (TechGradCourses ^ ((ProcIndExp + (Cos (TechGradCourses ^ ((ProcIndExp + (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Cos (TechGradCourses ^ ((ProcIndExp + (((ProcIndExp + (Log (Sin MgmtGradCourses)))/(Sin SWPMExp)) + (Sin ((Cos (TechGradCourses ^ ((ProcIndExp + (Cos (TechGradCourses ^ ((ProcIndExp + (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Sin SWPMExp)))))))))) / (TechGradCourses ^ (Log SWProjEstExp)))))) / (((Cos (TechGradCourses ^ ((Log SWProjEstExp) / (((Log (ProcIndExp + (Log (TechGradCourses ^ ((Log SWProjEstExp) / (Log SWProjEstExp)))))) - 3) / (ProcIndExp + (TechGradCourses ^ (Cos (TechGradCourses ^ ((ProcIndExp + (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Cos ((((Log SWProjEstExp) / ((ProcIndExp + (Log (TechGradCourses ^ (TechGradCourses ^ (Log SWProjEstExp))))) / (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Cos (Log (Log (Log SWProjEstExp)))))))))))))) / (Sin SWPMExp)) / (Sin SWPMExp)))))))))))) / (TechGradCourses ^ (Log SWProjEstExp))))))))))) - 3) / (TechGradCourses ^ (Log SWProjEstExp)))))) + ((Log SWProjEstExp) / (Log SWProjEstExp)))))) / (Log (Log (Log (TechGradCourses + (Cos (Log (Log (TechGradCourses ^ (Cos (((((Log SWProjEstExp) / (TechGradCourses ^ (Log SWProjEstExp))) / ((ProcIndExp + (Log (Sin MgmtGradCourses))) / ((Log SWProjEstExp) / (Log SWProjEstExp)))) / (Sin SWPMExp)) / (Sin SWPMExp))))))))))))))))))))))) / (TechGradCourses ^ (Log SWProjEstExp)))))) / (((Log ((((Log TotLangExp) / (Log SWProjEstExp)) / (Log SWProjEstExp)) / (Sin SWPMExp))) - 3) / (TechGradCourses ^ (Log SWProjEstExp)))))) - 3) / (TechGradCourses ^ (Log SWProjEstExp)))))))))) + (((((ProcIndExp + (Log (TechGradCourses ^ (Log (TechGradCourses + ((TechGradCourses ^ (TechGradCourses ^ (Cos (TechGradCourses ^ ((ProcIndExp + (Log (Log (TechGradCourses ^ (TechGradCourses ^ (Cos (Log (Log (TechGradCourses ^ (Cos ((((Log SWProjEstExp) / ((ProcIndExp + (Log (TechGradCourses ^ (Log (TechGradCourses + (Cos (Log (Log (TechGradCourses ^ (Cos (((((Log SWProjEstExp) / (TechGradCourses ^ (Log SWProjEstExp))) / ((ProcIndExp + (Log (Sin MgmtGradCourses))) / ((Log SWProjEstExp) / (Log SWProjEstExp)))) / (Sin SWPMExp)) / (Sin SWPMExp)))))))))))) / ((Log SWProjEstExp) / (Log SWProjEstExp)))) / (Sin SWPMExp)) / (Sin SWPMExp)))))))))))) / (TechGradCourses ^ (Log SWProjEstExp))))))) / (Sin SWPMExp))))))) / (TechGradCourses ^ (Log SWProjEstExp))) / (TechGradCourses ^ (Log SWProjEstExp))) / (TechGradCourses ^ (Log SWProjEstExp))) / (Sin SWPMExp))) 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop Too Much of a Good Thing?

26 Conclusions Viability of a human-based est. model Model assessment  Non-linear  GP Impact on Human Based Estimation 1) All Factors 2) Domain Experience  Work Experience 3) Education 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

27 Future Directions Equation Optimizer for GP Collect More Data  Further analysis without consolidation  Detailed Effect of Educational Factors Use other statistical indicators Build other models  Hybrid (Non-linear and GP)  Classifiers Impact of process on estimation 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop

28 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop Questions?

29 2 nd International Predictor Models in Software Engineering (PROMISE) Workshop Thank You !


Download ppt "Understanding the Human Estimator Gary D. Boetticher Univ. of Houston - Clear Lake, Houston, TX, USA"

Similar presentations


Ads by Google