1. University of Southern California Center for Systems and Software Engineering Software Cost Estimation Metrics Manual 26 th International Forum on COCOMO and Systems/Software Cost Modeling Ray Madachy, Barry Boehm, Brad Clark, Thomas Tan, Wilson Rosa November 2, 2011

2. University of Southern California Center for Systems and Software Engineering Project Background Goal is to improve the quality and consistency of estimating methods across cost agencies and program offices through guidance, standardization, and knowledge sharing. Project led by the Air Force Cost Analysis Agency (AFCAA) working with service cost agencies, and assisted by University of Southern California and Naval Postgraduate School We will publish the AFCAA Software Cost Estimation Metrics Manual to help analysts and decision makers develop accurate, easy and quick software cost estimates for avionics, space, ground, and shipboard platforms. 2

3. University of Southern California Center for Systems and Software Engineering Stakeholder Communities Research is collaborative across heterogeneous stakeholder communities who have helped us in refining our data definition framework, domain taxonomy and providing us project data. –Government agencies –Tool Vendors –Industry –Academia SLIM-Estimate ™ TruePlanning ® by PRICE Systems 3

4. University of Southern California Center for Systems and Software Engineering Research Objectives Make collected data useful to oversight and management entities –Provide guidance on how to condition data to address challenges –Segment data into different Application Domains and Operating Environments –Analyze data for simple Cost Estimating Relationships (CER) and Schedule Estimating Relationships (SER) within each domain –Develop rules-of-thumb for missing data Data Records for one Domain Cost (Effort) = a * Size b Schedule = a * Size b * Staff c Model-based CER/SER 4 Cost (Effort) = Size * (Size / PM) Productivity-based CER Data Conditioning and Analysis

5. University of Southern California Center for Systems and Software Engineering Data Source The DoD’s Software Resources Data Report (SRDR) is used to obtain both the estimated and actual characteristics of new software developments or upgrades. All contractors, developing or producing any software development element with a projected software effort greater than $20M (then year dollars) on major contracts and subcontracts within ACAT I and ACAT IA programs, regardless of contract type, must submit SRDRs. Reports mandated for 1.Initial Government Estimate 2.Initial Developer Estimate (after contract award) 3.Final Developer (actual values) 5

6. University of Southern California Center for Systems and Software Engineering Data Analysis Challenges Software Sizing –Inadequate information on modified code (only size provided) –Inadequate information on size change or growth, e.g. deleted –Size measured inconsistently (total, non-comment, logical statements) Development Effort –Inadequate information on average staffing or peak staffing –Inadequate information on personnel experience –Missing effort data for some activities Development Schedule –Replicated duration for multi-build components –Inadequate information on schedule compression –Missing schedule data for some phases Quality Measures –None (optional in SRDR) 6

7. University of Southern California Center for Systems and Software Engineering Data Conditioning Segregate data to isolate variations –Productivity Types (revised from Application Domains) –Operating Environments Sizing Issues –Developed conversion factors for different line counts to logical line count –Interviews with data contributors recovered modification parameters for 44% of records Missing effort data: use distribution by Productivity Type to fill in missing data (to possibly be improved using Tom Tan’s approach) Missing schedule data: use distribution by Productivity Type to fill in missing data

8. University of Southern California Center for Systems and Software Engineering Environments Productivity types can be influenced by their environment. Different Environments are grouped into 11 categories –Fixed Ground (FG) –Ground Surface Vehicles Manned (MGV) Unmanned (UGV) –Sea Systems Manned (MSV) Unmanned (USV) –Aircraft Manned (MAV) Unmanned (UAV) –Missile / Ordnance (M/O) –Spacecraft Manned (MSC) Unmanned (USC) 8

9. University of Southern California Center for Systems and Software Engineering 9 Productivity Types 1.Sensor Control and Signal Processing (SSP) 2.Vehicle Control (VC) 3.Real Time Embedded (RTE) 4.Vehicle Payload (VP) 5.Mission Processing (MP) 6.Command & Control (C&C) 7.System Software (SS) 8.Telecommunications (TEL) 9.Process Control (PC) 10.Scientific Systems (SCI) 11.Training (TRN) 12.Test Software (TST) 13.Software Tools (TUL) 14.Business Systems (BIS) 9 Different productivities have been observed for different software application types. Software project data is segmented into 14 productivity types to increase the accuracy of estimating cost and schedule

10. University of Southern California Center for Systems and Software Engineering CER/SER Reporting Operating Environments FGMGVUGVMSV…Mean Productivity Types SSP VC RTE VP MP C&C … Mean Mean productivity for each type Mean productivity for each environment Productivity for each type and environment

11. University of Southern California Center for Systems and Software Engineering 11 Current Status Completing the Software Cost Estimation Metrics Manual Initially available as an online wiki –Ability to print online version available Inviting Reviewer comments using the online wiki “Discussion” capability First version of manual ready Dec 31 st, 2011 11