COCOTS Life Cycle Estimation: Some Preliminary Observations Chris Abts Betsy Clark USC Center for Software Engineering Software Metrics Inc. October 25, 2001 Copyright 2001 University of Southern California

Briefing Outline Background Maintenance Phase Modeling Conclusions Some Observations Conclusions

COTS Definition “Commercial Off the Shelf” Software sold, leased, licensed at advertised prices Source Code Unavailable Periodic releases with feature growth and fixes Eventual obsolescence, end of life

COTS Phenomena You have no control over a COTS product’s functionality or performance Most COTS products are not designed to interoperate with each other You have no control over a COTS product’s evolution COTS vendor behavior varies widely

COCOTS Model Status Development Phase Model Maintenance Phase Model Currently collecting data to calibrate Maintenance Model Limited sample collected to date but… Interesting patterns emerging Supports hypothesis suggested by Chris Abts based on workshop discussions from 1999 COCOMO forum

$ A COTS-Based System Economic Lifespan Model: The COTS-LIMO Model n+x No. of COTS in system n+3 n+2 $ n+1 Volatility effects just cancel increased integration experience n 5 Retire Volatility effects dominate increased integration experience 4 Cost of maintenance 3 2 1 Fn (synchronization, complexity of system, no. planned upgrades, etc.) Maintain Increased integration experience dominates volatility effects Time  1999 University of Southern California - Chris Abts

Caveat Observations based on interviews with four projects Three of the four projects have more than 40 COTS products

Observation - 1 “We can’t neglect a COTS-based system like we could a custom. We need a continual stream of funding.” All four projects mentioned a positive aspect of maintaining a CBS: “It forces us to bring new technologies to our users.”

Refresh: A Definition Periodic replacement of COTS products to sustain an indefinite life in a previously existing system Replacement may be with newer version of the same product or with a different product Necessary with COTS because products reach end-of-life (no longer vendor supported)

Observation - 2 Three of the four projects have removed COTS components because of maintenance complexity replaced with custom components “When you have more than one product, you exponentially complicate maintenance. We can get to market faster with COTS but maintenance has to be considered. We get complex very fast and complexity translates to cost.”

Projects were asked to rate their success from two perspectives Observations - 3 Projects were asked to rate their success from two perspectives Acquisition Maintenance Three of the projects rated themselves as very successful from an acquisition perspective but mixed in terms of maintenance success Initial delivery was on schedule But high maintenance cost, high complexity The remaining project rated itself as very successful from both perspectives

Observations - 3 What is the difference? Number of COTS products

Need for refresh prior to IOC Observations - 4 Need for refresh prior to IOC One of our projects stated that almost half of the components (46%) had already reached end-of-life before IOC

Observations - 5 Complexities stemming from multiple COTS products are made worse by multiple configurations One project has three software configurations in the field at any one time (prior, current, new) “Configuration management becomes very important in terms of coordinating what versions of what COTS products are in what system configurations.” Another project reported added complexity from different hardware configurations resulting from gradual hardware replacement across sites Complicates testing

Briefing Outline Background Maintenance Phase Modeling Conclusions Some Observations Conclusions

Multiple configurations makes this much worse Conclusions Initial observations suggest a non-linear impact of the sheer number of products on maintenance complexity Multiple configurations makes this much worse Strategies for managing complexity include: Rigorous CM Distinguishing between critical and non-critical components with focus on the former to avoid end-of-life Minimizing multiple configurations (hardware and software)

A plea for calibration data! Contact Betsy Clark Software Metrics Inc. (703) 754-0115 Betsy@software-metrics.com

Contact Information U S C e n t r f o w a E g i P s c - L A l Mr. Chris Abts (primary graduate researcher)…………...………. ……(213) 740-6470 Ms. Ladonna Pierce (CSE Office Administrator)…..……………….……(213) 740-5703 Dr. Barry W. Boehm (CSE Director)………………………………….….(213) 740-8163 USC Center for Software Engineering FAX line……..…………….……(213) 740-4927 COCOTS E-Mail………………………………………….…… cots-info@sunset.usc.edu World Wide Web page……….………..…… http://sunset.usc.edu/research/COCOTS/index.html d M , I . V ( D ) Dr. Betsy Clark.…………….……………..……………………….……...(703) 754-0115 FAX line…………………………………………………………….………(703) 754-3446 E-Mail………………………………………………………………..Betsy@Software-Metrics.com

Back up Slides

First Pass Maintenance Phase Straw Model CBS Maintenance Effort (for a Given Cycle Time TM) = COCOMO Application Maintenance + COTS Reassessment + COTS Retailoring + COTS Glue Code Evolution + COTS Volatility Effect on Application Effort + COTS Replacement CBS PMMaint Total = S(PMMaint-APP + PMRe-ASST + PMRe-TAIL + PMGLUE-Evol + PMVolEff-APP + PMCOTS-R ) Over all TM

Reassessment (per Refresh Cycle) PMRe-ASST = S [(# Releases in Cycle TM ) • PMDevASST • (Reasst Fractn)] (over all COTS classes) PMDevASST = Assessment Effort during development Reasst Fractn = Reassessment Fraction ; fraction of original assessment needing to be redone per release due to COTS changes (variable by COTS class, Release cycle) - represents a ratio to development experience; data is used to determine a reasonable value

Retailoring (per Refresh Cycle) PMRe-TAIL = S [(# Releases in Cycle TM ) • PMDevTAIL • (Retail Fractn)] (over all COTS classes) PMDevTAIL = Tailoring Effort during development Retail Fractn = Retailoring Fraction; fraction of original tailoring needing to be redone per release due to COTS changes (variable by COTS class, Release cycle) - represents a ratio to development experience; data is used to determine a reasonable value

Glue Code Evolution (per Refresh Cycle) PMGLUE-Evol = PMDevGLUE • [1+(SUGLUE/100 • UNFMGLUE)] • EAFM/ EAFD • S[(# Releases in Cycle TM) • (MCFGLUE/Release)] (over all COTS classses) PMDevGLUE = Glue Code Effort during development TM = No. of Months between COTS refresh efforts MCFGLUE = Maintenance Change Factor (similar to AAF) SUGLUE = Software Understanding (how well structured is code?) UNFMGLUE = Unfamiliarity of Maintainers with COTS Application EAF = Based on COCOTS Effort Multipliers

Effect of COTS Replacement (per Refresh Cycle) PMCOTS -R = S (PMASST + PMTAIL + PMGLUE) (over all COTS replaced) PMASST/TAIL/GLUE = Assessment & Tailoring & Glue Code Effort for replacement COTS components using the development COCOTS submodels For subsequent refresh cycles, identify changes in COTS life cycle cost drivers as appropriate within each submodel due to (presumably) improved COTS replacement components.