Software Technology Support Center BE AMERICA’S BEST The Problems in Software Estimating Dr. Randall Jensen Software Technology Support Center Hill AFB, UT

O G D E N A I R L O G I S T I C S C E N T E R Outline Evolution of software estimating models Software problems Management is the issue Data collection concerns New models? BE AMERICA’S BEST It’s all about models – “Existence is elsewhere”

O G D E N A I R L O G I S T I C S C E N T E R Project Uncertainty Principle BE AMERICA’S BEST

O G D E N A I R L O G I S T I C S C E N T E R Software model genealogy Nordon IBM 1970 Putnam/ SLIM US Army/GE/ QSM 1976 Doty RADC 1977 JS1,2,3 CEI 1980 SEER-SEM GAI 1989 Seer II / Sage SEI 1995 COCOMO TRW 1981 COCOMO II USC 1995 Validation Seer HAC 1979 PRICE-S Price 1977 TRUE-S Price 2003 REVIC USAF 1991 Management

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Sage/SEER-SEM - Normal form where K = life cycle cost (person years), T d = development time (years) sloc

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Common estimate dilemma

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Software problems are not new Unreliable Late delivery Modification costs prohibitive Impossible to maintain Inadequate performance Product exceeds budget costs 1968 CS Conference, Munich, Germany KEY

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Historic note: “ More software projects have gone awry for lack of calendar time than for all other causes combined…” F. P. Brooks, Jr., Mythical Man Month

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Common technology claim If you use (new technology), Productivity will improve by an order of magnitude And Defects will reduce to zero Note: None of the history-based estimating models (tools) support this claim, not even close. Neither does history.

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST There is always HOPE Ada OOD Structured Analysis Structured Design Process Maturity PWB Structured Programming 3rd Generation Languages P R O D U C T I V I T Y lppm

O G D E N A I R L O G I S T I C S C E N T E R 3 Dimensions of development BE AMERICA’S BEST

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Impact of management on cost “Poor management can increase software costs more rapidly than any other factor… Despite this variation, COCOMO does not include a factor for management quality, but instead provides estimates which assume that the project will be well managed [ italics mine, RWJ ]” Boehm, B. W., Software Engineering Economics, (Englewood Cliffs, NJ: Prentice-Hall, Inc.), 1981, pg. 486 Note: On a scale of 0 to 10 well managed must rank approximately 3.4?

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Where would you focus effort? TOOLS PEOPLE SYSTEMS MANAGEMENT Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Biased research? ToolsPeopleSystems Managers SEI Papers Percent Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R Cost driver relative impact It takes people to implement processes and use technology. April 21 st, 2009

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST BIG productivity drivers Analyst capability Management style Motivation Use of team methods Working environment Problem solving skills Programmer capability Application domain experience Use of modern practices Automated tool support

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Capability shift (environment) Basic Technology Constant (1990)

O G D E N A I R L O G I S T I C S C E N T E R Historic data observations Models are made from historic data More data is better Adequate historic data takes time to collect (Lots of time) Repository vs Database Repository = a place where things may be stored for safekeeping Data stored without concern for a formal definition of the data placed in the repository SRDR is defined as a financial report that happens to contain technical information Database = a collection of data arranged for ease and speed of retrieval Requires consistency of entries Each data field must satisfy common definition DCARC is a repository as was SMC database Data definitions? BE AMERICA’S BEST18

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Problems?

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Popular estimating solution Let’s create a new software model Models are out of date Models are inaccurate Too difficult to use Oh, really! Are you sure?

O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST New estimating models? Lack of confidence in existing tools (current issues) Optimistic estimates Unacceptable estimates Aging tools (?) Culture is constant New models require validation New models -- No silver bullets Quality data (if, where, proprietary) New models require validation New (or better) estimators? Experience Skill Integrity

O G D E N A I R L O G I S T I C S C E N T E R Estimating Prediction is very hard, especially when it’s about the future