Some Experience With COSYSMOR At Lockheed Martin John Gaffney 301-721-5710 j.gaffney@lmco.com Lockheed Martin Systems & Software Resource Center (SSRC) Center for Process Improvement Excellence (CPIE) October 29, 2008 (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 Agenda What Is COSYSMOR ? Some Experience With COSYSMOR (c) Copyright, Lockheed Martin Corporation, 2008

COSYSMOR or COSYSMO Risk and Reuse COSYSMOR evolved from the USC Academic COSYSMO systems engineering estimation model/tool (Dr. Ricardo Valerdi) and LMCO implemented these enhancements in a tool, COSYSMOR, or COSYSMO Risk COSYSMOR is available on request It has been distributed to some organizations outside of Lockheed Martin already A major driver for the development of COSYSMOR was to get away from “single point” cost estimates in order to better recognize the uncertainty associated with effort, schedule, and cost estimates (c) Copyright, Lockheed Martin Corporation, 2008

Additional Functions Provided By COSYSMOR COSYSMOR provides four major additional functions beyond those provided by Academic COSYSMO: Estimation of Cost/Effort and Schedule Uncertainties/Risk and Confidence: Provides quantification of the impacts of uncertainties in the values of key model parameter values. Provides multiple cost and schedule values with associated probabilities. Risk=Prob [Actual Effort Will Be >Estimated Effort] Confidence=100%-Risk Representation of Multiple Types of Size Drivers: Provides for entering counts of: new, modified, reused, and deleted types for each of the four size driver categories. Labor Scheduling: Provides the spread of systems engineering labor for the five systems engineering activities and across four the development phases (time). Labor Allocation: Provides for the user to select the percentage allocations of the twenty activity/phase pairs or effort elements. (c) Copyright, Lockheed Martin Corporation, 2008

What Lockheed Martin Has Been Doing Been calibrating COSYSMOR in various organizations and applying it in several projects Have also developed an analogous model/tool for software resource estimation, COCOMOR We are quite interested in harmonizing the estimation of systems engineering effort and software engineering effort We have also developed a tailored version of COSYMOR, COSYSMOR-A for avionics This illustrates the general utility of this model/tool form (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 COSYSMOR-A COSYSMOR-A stands for COSYSMOR for Avionics COSYSMOR-A is a tailored version of COSYSMOR, to produce resource (effort) estimates for new avionics systems and upgrades produced by Lockheed Martin Aeronautics Company Mission Systems and Avionics The tailoring included changes in two of the size drivers and two of the cost drivers (of COSYSMO/COSYSMOR): Size Drivers: # of Hardware Components and # of Software Functions replace # of Algorithms and # of Operational Scenarios in COSYSMO/R, respectively Cost Drivers: Lab System Integration and Test Complexity and Flight Test Complexity replace Migration Complexity and Multisite Coordination in COSYSMO/R, respectively (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 COSYSMOR Data Entry-1 Three-point data entry for size and cost drivers. (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 COSYSMOR Data Entry-2 You indicate the uncertainty in size driver and cost driver values by entering Low, Likely and High values for each to indicate your assessment of the distribution for the easy, nominal and difficult size driver counts If you don’t believe that there is a range, but rather, there is the value, then you enter the same values for Low, Likely and High (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 COSYSMOR Data Entry-3 A Program User enters the percents of Modified, Reused and Deleted for each of the four size drivers; the tool computes the percent of New These percents are taken to apply to the Easy, Nominal and Difficult values for each of the four types of size drivers A Program User also enters the Relative Cost (relative to New) for the Modified, Reused and Deleted counts Note: If the Program User wants to designate different sets of percents and relative costs for Easy, Nominal and Difficult, then he will have to go the Reuse page of COSYSMOR to enter these values. (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 COSYSMOR Data Entry-4 An Administrative User enters the Low, Likely and High values for A, the Unit Effort Constant, and E, the Exponent, in the COSYSMOR model: K=A*SE* Di (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 Some COSYSMOR Outputs Principal outputs of the COSYSMOR tool are: Equivalent Size Risk Cost Driver Product Risk Systems Engineering Person Hours Risk Systems Engineering Person Hours Overrun Risk Percent of Total Systems Engineering Effort By Process Activity and By Phase Systems Engineering Person Hours Vs. Month Systems Engineering Schedule Risk (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 Systems Engineering Person Hours Risk Use this Graph to help understand labor (and cost) exposure for your choice of labor (say to be bid in a proposal). Example: The risk of exceeding 60000 labor hours is slightly less than 20%. Note: You might choose to present only the smooth curve fit rather than the discrete plot. (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 Systems Engineering Person Hours Overrun Risk This plot shows the overrun risk % for a target of 250000 of Systems Engineering Person Hours values. This plot is the tail of the Systems Engineering Person Hours Risk plot. Some persons might prefer to see this plot in which the “zero” value on the horizontal axis corresponds to the target value and the horizontal axis values are the (potential) overrun values relative to the target. (c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008

(c) Copyright, Lockheed Martin Corporation, 2008 Observations Better management of uncertainty starts with recognizing its existence and in estimating cost/schedule risk Use the risk information to make better management decisions during both business acquisition and program execution Serve as a basis for discussion of alternatives within the project as well as with the customer A key aspect of affordability Relate cost/schedule risk to risk register as part of normal risk management process Relate to potential program failure modes Need to harmonize SE and SW estimation (c) Copyright, Lockheed Martin Corporation, 2008