Phase Distribution of Software Development Effort Ye Yang, Mei He, Mingshu Li, Qing Wang, Barry Boehm Institute of Software, Chinese Academy of Sciences (ISCAS) & USC-CSSE COCOMO Forum’08 October 28, 2008

Outline Background Subject and approach Results Discussions and Conclusions 11/28/2018 2 2

Background 1950’s: Norden observed Rayleigh Curve to be a good staffing level Approximation to most hardware development projects Main criticism: slow early build-up and long-tail effects COCOMO 81, the Detailed COCOMO Model Phase-sensitive cost multipliers enable more accurate, phase-wise estimation Allows to track down the effect of individual cost driver rating on phase distribution variation COCOMO 2000 Phase neutral cost multipliers due to lack of calibration data Two simplified schemes to facilitate phase/activity distribution: waterfall & RUP Challenges Lack of studies on causes of distribution variation Lot of estimation methods but lack of insightful allocation guidelines Our study Empirical analysis towards developing more in-depth understanding on factors impacting on the degree of intensity of different development phases/activities Cocomo 81 e.g. more focus on integration and test phase and less on code as the software size grows (i.e. 16% for small projects and 25% for large projects) 11/28/2018 3 3

Recent Studies Heijstek and Chaudron (2007) Milicic and Wholin (2004) Analyzed data from model-based development projects Confirmed the similarity with RUP hump-chart Milicic and Wholin (2004) Studied characteristics affecting estimating accuracy & proposed a lean approach to improve estimation based on distribution patterns of estimation errors Yiftachel et al. (2006) Proposed an economic model for optimal allocation of resources among development phases Lucia et al. (2003), Yang et al. (2008): High correlation between the effort of subsequent activities for maintenance projects 11/28/2018 4 4

Different Phase Definitions Estimation Model Phases covered COCOMO 81 Plan and requirement, preliminary design, detailed design (25%), code (33%), Integration & Test (25%) COCOMO II Option 1(Waterfall): Plan and requirement (7%), preliminary design (17%), detailed design (25%), code (33%), Integration & Test (25), deployment & maintenance (12%); Option 2(MBASE/RUP): Inception(6%), Elaboration(24%), Construction(76%), Transition (12%) RUP Inception(5%), Elaboration(20%), Construction(65%), Transition(10%) COCOTS Assessment, Tailoring, Gluecode & Integration. SLIM Concept Definition, Requirement & Design, Construct & Test, Perfective Maintenance SEER-SEM Early specification, design, development, delivery & maintenance 11/28/2018 5 5

Outline Background Subject and approach Results Discussions and Conclusions 11/28/2018 6 6

Subject CSBSG (China Software Benchmarking Standard Group) Established in 2006 Consistent with ISBSG Supported by the Government and software industry association of China 1012 projects from 141 organizations and 15 regions Phase/Activity definition in CSBSG database: Roughly waterfall; transforming guidelines were provided during data collection Phase Activities Included Plan Plan, Preliminary Requirement Analysis Requirement Requirement Analysis Design Product Design, Detailed Design Code Code, Unit Test, Integration Test System Test Transition Installation, Transition, Acceptance Test, User Training, Support 11/28/2018 7 7

Approach of the study Data Collection Data Selection and Cleaning Web-based questionnaire Sent out by CSBSG staff, completed by leading companies Simple automatic checking spelling and inconsistency errors Expert group check Determined for data quality and candidacy Data Selection and Cleaning Select the minimum set of attributes considered in the study Clean the data through several steps 11/28/2018 8 8

Summary of Attributes Considered Metric Unit Description Size SLOC Total Lines of Code Effort Person-Hour Summary Work Effort Plan phase effort Work Effort of plan phase Reqt’s phase effort Work Effort of reqt’s phase Design phase effort Work Effort of design phase Code phase effort Work Effort of code phase Test phase effort Work Effort of test phase Transition phase effort Work Effort of transition phase Development life cycle Nominal Waterfall, iterative, rapid prototyping Team Size # Person Maximum size of the development team Development Type New development, Enhancement, Re-development 11/28/2018 9 9

Data Selection Step ID # of proj. excluded # of proj. remained Reason for exclusion 1 115 Do not contain phased-related effort records 2 14 101 there are two or more phases’ effort data missing 3 100 the sum of its 5 recorded phased effort is greater than the total effort 4 if only 1 of the 5 phase effort value is missing, it is filled up by subtracting the other 4 phases from the total effort 5 25 75 one or more than one phase’s effort data recorded as zero 11/28/2018 10 10

Outline Background Subject and approach Results Discussions and Conclusions 11/28/2018 11 11

Summary of the project data Mean Median Min Max Size (KSLOC) 136.4 45.7 0.77 2340 Effort (Person-hours) 8969 4076 568 134840 11/28/2018 12 12

Dataset Comparison Mean Median Min Max CSBSG Size (KSLOC) 136.4 45.7 0.77 2340 Effort (Person-hours) 8969 4076 568 134840 COCOMO Size(KSLOC) 127 47 3 1293 Effort(Person-Months) 714 195 6 11400 Effort(Person-hours) 108528 29640 912 1732800 15.2 LOC/man-hour 1.2 LOC/man-hour 11/28/2018

Overall phase distribution pattern Overall phase distribution profile Comparison of CSBSG and COCOMO II Distribution 11/28/2018 14 14

Comparison by life cycle models Classifications: waterfall, iterative, rapid prototyping # of projects: 57, 15, 2 respectively Mean software size (KSLOC): Waterfall: 103 Iterative: 259 Rapid: 89 Explanation from the paper 11/28/2018

Comparison by Development type Enhancement projects have more focus on Test phase, Re-development has the greatest emphasis on Code phase 11/28/2018 16 16

Comparison by Development type Totally, New development has lower estimation error All phased effort for Code are underestimated, and for Transition, over-estimated Comparison of estimation accuracy RE = (Estimated phase effort – Actual phase effort)/Actual phase effort 11/28/2018 17 17

Comparison by Size scale Distribution of software size 11/28/2018 18 18

Comparison by Size scale Ascending trend for Code Descending trend for Test Comparison among difference software sizes scale in LOC 11/28/2018 19 19

Comparison by Team Size Average phase distribution for different team size 11/28/2018 20 20

Outline Background Subject and approach Results Discussions and Conclusions 11/28/2018 21 21

Factors influencing phased distribution ANOVA analysis is used to examine to what a degree the variance of each phase effort distribution percentage is explained by class variables Phase Distribution DevType Software size Team size P&R% 3.96% 14.80% 5.50% Design% 2.57% 3.67% 0.62% Code% 12.22% 7.93% 0.02% Test% 7.47% 7.16% 3.05% Trans.% 2.72% 9.36% 1.52% 11/28/2018 22 22

Guidelines for phased distribution Analysis on determining reasonable phase effort distribution quantities should be performed in the cost estimation process. CSBSG data analysis shows a waterfall-based phase distribution scheme as: 16.14% for plans and requirements phase, 14.88% for design phase, 40.36% for code phase, 21.57% for test phase, and the other 7.06% for transition phase. Software size and development type are two major factors to be considered when adjusting effort allocation. CSBSG dataset shows that for enhancement type of projects, percentage of development effort in code phase decreases by 10.67%, and that for test phase increases by 5.4%. CSBSG dataset indicates that as software size dramatically grows, distribution has an intensive focus on both code and test phases. CSBSG dataset indicates that team size is not a significant factor which may cause phase effort distribution variation. However, it shows a distinguishable emphasis on Test phase with the growth of team size 11/28/2018 23 23

Limitations Applicability of empirical findings Size metric All projects from domestic organizations in China Size metric SLOC vs. FP Selected influencing factors Correlation among factors not considered 11/28/2018 24 24

Conclusions and Future Work Empirical analysis towards developing more in-depth understanding on factors impacting on the phases/activities distribution Providing empirically-based suggestions for Chinese software industry Next steps Consider other influencing factors such as system complexity, number of concurrent users Strengthening the findings through more thorough statistical analysis 11/28/2018 25 25

Thank You! Contact info: ye@itechs.iscas.ac.cn 11/28/2018