Costing Secure Systems Workshop Report Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Costing Secure Systems Workshop Report Edward Colbert Danni Wu {ecolbert, danwu}@cse.usc.edu 21st International Forum on COCOMO & Software Cost Modeling © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

In Case You Aren’t Sure That Security Is Important © 2002-6 USC-CSE 19 November 2018

Workshop Participants Ed Colbert, USC, Moderator (ecolbert@usc.edur) Danni Wu, Scribe (danwu@usc.edu) Don Reifer, Reifer Inc. (dreifer@earthlink.net) Martha Leonette (Martha.J.Leonettee@FAA.org) Anca-Jiliana Stoica (anca@dsv.su.se) Rita Creel (rc@sei.usc.edu) Ron Owens (Ronal.L.Owens@aero.org) Barry Boehm (boehm@usc.edu) © 2002-6 USC-CSE 19 November 2018

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Goal Of Workshop Review Research Draft model for early costing of system security Extensions to COCOMO II for development of secure software systems (“COSECMO”) Gather expert opinion Invite Data MetaH provides semantics & supporting tools UML provides graphic front-end © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Workshop Agenda Introduction Review Early Estimation Model Review COCOMO Security Extension ("COSECMO") Delphi © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Outline Project Schedule Early Estimation Model COCOMO Security Extension ("COSECMO") To Do © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Cost Model for System Security Increment 1 (Feb – July ’04) Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Cost Model for System Security Increment 1 (Feb – July ’04) Task Element Activities 1. Develop Early Estimation Model Prototype model 2. Sources of Cost Identify, define, scope sources of cost Relate sources of cost to FAA WBS Recommend type of CER for each 3. Secure Product Taxonomy Identify, define, scope product elements 4. COCOMO II Security Extensions Refine model form and data definitions 5. COCOTS Security Extensions Explore security aspects in COCOTS data collection © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Cost Model for System Security Increment 2 (Aug ’04 – Mar ’06) Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Cost Model for System Security Increment 2 (Aug ’04 – Mar ’06) Task Element Activities 1. Develop Early Estimation Model Experimental use & refinement 2. Sources of Cost Prioritize sources of cost needing CER’s Refine, prototype, experiment with top-priority CER’s Relate to scope of COCOMO II security extensions 3. Secure Product Taxonomy Experimental use, feedback, and refinement 4. COCOMO II Security Extensions Refine, scope, form, definitions based on results of Tasks 1-3 Experimentally apply to pilot projects, obtain usage feedback 5. COCOTS Security Extensions Develop initial scope, form, definitions based on results of Tasks 1-4 © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Cost Model for System Security Increment 3 (Mar ’06 – Feb ’07) Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 We are in middle of inc. Cost Model for System Security Increment 3 (Mar ’06 – Feb ’07) Task Element Activities 1. Develop Early Estimation Model Evolution; integration with other models 2. Sources of Cost Refine sources of cost, CER’s based on usage feedback Integrate with other models Address lower-priority CER’s as appropriate 3. Secure Product Taxonomy Monitor evolution 4. COCOMO II Security Extensions Baseline model definitions Collect project data Develop initially calibrated model; experiment and refine © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Cost Model for System Security Increment 4 (Apr ’07 – Mar ’08) Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Cost Model for System Security Increment 4 (Apr ’07 – Mar ’08) Task Element Activities 1. Develop Early Estimation Model Evolution Integration with other models 2. Sources of Cost Refine sources of cost, CER’s based on usage feedback Integrate with other models 3. Secure Product Taxonomy Monitor evolution 4. COCOMO II Security Extensions Collect project data Develop initially calibrated model Experiment & refine © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Outline Project Schedule Early Estimation Model COCOMO Security Extension (“COSECMO") Data Mining To Do © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Formula for Cost of System & Security Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Formula for Cost of System & Security Ctotal = CInitial/Mission Analysis + CInvestment Analysis + CSystem Engineering + CDev & Imp + CSys of Sys Integration + CInstall/Deployment + CO&M + CDisposal CDev & Imp = CDesign & Build HW + CDesign & Build SW + CPurchased Services + CCOTS-Sys + CEnv-Mods-design + CBus-Proc-Re-engineering Ctotal (Security) = Ctotal (with security) – Ctotal (without security) COTSYS  Commercial of the Shelf Systems O&M  operation & maintenance Env-Mods-Design  Design of Modifications to environment that needs to be implemented during installation/deployment (e.g. add steal-re-enforced cement barriers) Bus-Process Re-engineering  Re-engineering/Design of business processes that needs to be implemented during installation/deployment, operation & maintenance, or disposal C = Cost © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 5th Prototype Tool Screenshot#4 Advance Estimate — Cost Item by Parametric Models © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Outline Project Schedule Early Estimation Model COCOMO Security Extension (“COSECMO") Data Mining To Do © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Effect Of Security On COCOMO II Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Effect Of Security On COCOMO II Source lines of code (SLOC’s) increased Implementation of Security Functional Requirements (SFR’s) Effort to produce code increased by Security Assurance Requirements (SAR’s) A few Security Functional Requirements (SFR’S) Effort for “outer phases” of life–cycle (e.g. Inception, Transition) increased by Additional documents Additional activities e.g. definition of security roles, certification © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Effect Of Security Functional Requirements On SLOC’s & Computed Effort (cont.) Computation of total effort PMtotal = PMTSF + PMapplication + PMCertification/Validation/Accreditation TSF often developed at higher level of security © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

New Security Driver (SECU) (cont.) Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 New Security Driver (SECU) (cont.) 6 COCOMO levels ≈ 7 CC EAL’s (or equivalent activity) Treating EAL 1 as Nominal & EAL 2 as Nominal+50 (or High-50 ) Tailoring/Modification/Addition of SAR’s handled by increasing/decreasing base level Rating Level Estimated Scale Value Rating Scale (Refer to Supplement for details) Nominal (NOM) 0.00 No security requirements of added protection High (HI) 1.0 Informal security requirements, methodically tested and checked Very High (VH) 1.5 Methodically designed, tested and checked Extra High (XH) 2.0 Semi-formally designed and tested Super High (XXH) 5.0 Semi-formally verified designed and tested Ultra High (XXXH) 10.0 Formally verified designed and tested © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 COSECMO Estimation Trends Effort for Different Size Projects at Assurance Levels Plot of projects where only SECU & effort increasing drivers Efforts seem a little low based on values from Orange Book projects © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 COSECMO Estimation Trends Effort by Assurance Levels for Different Size Projects Plot of projects where only SECU & effort increasing drivers Efforts seem a little low based on values from Orange Book projects © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Barry’s Advice Follow KISS principle Keep It Simple, Stupid Or as Einstein said “Keep it as simple as possible, but no simpler” © 2002-6 USC-CSE 19 November 2018

Proposed Changes to COSECMO Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Proposed Changes to COSECMO Reviewing decision to make SECU driver a scale factor Calibration issues Reduce model complexity by Eliminating guide for other drivers & Re-integrate effect into SECU Discuss with customer simple ways to estimate Certification, Validation, Accreditation Eliminate “Validation” Define as percent of development cost? © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Outline Project Schedule Early Estimation Model COCOMO Security Extension (“COSECMO") Data Mining To Do © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 To Do Refine costing prototypes Get more feedback from security community Refine models Refine Delphi Collect & analyze data We need data! Write papers & Ph.D. thesis (theses?) © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE

Next Costing Secure Systems Workshop Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum 11/19/2018 Next Costing Secure Systems Workshop Date: 14 February 2007 Time: 8AM –5PM Location: USC’s CSSE Part of CSSE’s Annual Research Review & Executive Forum See csse.usc.edu & click on Events Cost: Workshop is free © 2002-6 USC-CSE 19 November 2018 © 2002-6 USC-CSE