FAA Information Technology- Information Systems Security R&D Workshop June 2015© USC-CSE Extending COCOMO II to Estimate the Cost of Developing Secure Software Edward Colbert, Murali Gangadharan, Donald Reifer, & Barry Boehm {ecolbert, murali,

23 June 2015© USC-CSE Outline  Why Extend COCOMO II for Security?  The COCOMO II & Modeling Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

33 June 2015© USC-CSE Why Extend COCOMO II for Security  Military projects have considered security in developing software since the early 1980s  Until recently commercial projects often gave it little weight  Threat to business-critical systems & private information has grown –Security can no longer be ignored  Few cost models (including COCOMO II) include security factors –Based 1980s military perspective (Orange Book) –Developing secure systems has changed dramatically

43 June 2015© USC-CSE Adding Security to Cost Models  Several approaches for addressing security –Modernize existing models Primarily by updating definitions of cost drivers –Add new cost drivers to cost models –Develop separate security model

53 June 2015© USC-CSE Adding Security to Cost Models  USC has taken intermediate approach –Add factor that addresses security from 3 viewpoints Development Operational Physical (Development Constraints) –Include factors as appropriate to all COCOMO II family cost models –Address both commercial & military projects regardless of Size Domain Level of maturity

63 June 2015© USC-CSE Outline  Why Extend COCOMO for Security?  The COCOMO II & Modeling Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

73 June 2015© USC-CSE Project scale factors: maturity, risk, flexibility, teamwork & precedentedness Software product, process, project & personnel cost drivers COCOMO II Refresher COCOMO II Model Software size estimate Software organization’s project data Effort & duration estimates Cost, schedule distribution by phase, activity, increment COCOMO II recalibrated to organization’s data Effort in Person Month SF: Scale Factors (5) EM: Effort Multipliers(17)

83 June 2015© USC-CSE COCOMO II Family of Cost Models COCOMO II Calibration Models Risk & Tradeoff Models Allocation Models Sizing Models Reuse Models COCOTS COQUALMO CORADMO COSYSMO OTHERS

93 June 2015© USC-CSE COCOMO II Modeling Methodology Analyze Existing Literature Perform Behavioral Analysis Determine Form of model &Identify relative significance of parameters Perform expert Judgment, Delphi Assessment Gather Project Data Determine Bayesian A Posteriori update Gather more data; Refine model

103 June 2015© USC-CSE Outline  Why Extend COCOMO for Security?  The COCOMO II & Modeling Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

113 June 2015© USC-CSE COCOMO II Security Driver (SECU)  Viewpoints –Design & Development for Security –Operational Security –Physical Security (Development Constraints)  Driver Ratings –Low –Nominal –High –Very High –Extremely High

123 June 2015© USC-CSE Security Factors  Design & Development for Security –Effect of design methodologies & processes for development & validation when security is factor  Operational Security –Effect of security policies, processes, tools & facilities that: Permit identification of security events Define subsequent actions to identify key elements Report pertinent information to appropriate individual, group, or process  Development Constraints –Constraints placed on development when protecting software facilities: From outside perimeter to inside office space Includes all of information system resources

133 June 2015© USC-CSE Design & Development for Security Rating : Low & Nominal  Low –No security requirements –No protection other than provided by execution environment  Nominal Requirements  Informal security requirements formulated for system Design  Analysis of security functions using –Informal functional & interface specification –Descriptive high-level design –Informal demonstration of corresponding pairs Testing  Developer tests implementation of requirements –Black box testing Life-cycle controls  Simple Configuration Management with version numbers

143 June 2015© USC-CSE Design & Development for Security Rating : High  Nominal + Requirements  Fully defined external interfaces  Informal security policy modeling Design  High-level design enforces security  Informal low-level design description Testing  Independent testing of all functional requirements  Inspection of COTS/OSS source code if available  Developer vulnerability analysis Life-cycle controls  Detailed delivery & installation procedures –with well-defined security defaults  Developer-defined life-cycle model –with well-defined development tools

153 June 2015© USC-CSE Design & Development for Security Rating: Very High  High+ Requirements  Semi-formal functional specifications  Semi-formal security policy modeling Design  Semi-formal high-level design  Semi-formal Correspondence demonstration  Modular implementation  Dynamic analysis for COTS/OSS Testing  Evidence of coverage for all developer test results  Testing of high-level design  Independent vulnerability analysis  Independent validation of analysis Life-cycle controls  Partial automation of CM –with authorization control, problem tracking, & detection of modification  Measurable life-cycle model –with compliance to implementation standards

163 June 2015© USC-CSE Design & Development for Security Rating: Extremely High  Very High + Requirements  Formal functional specification  Formal security policy modeling Design  Formal high level explanation  Formal Correspondence Demonstration  Structured implementation with minimization of complexity  Secure container for COTS & OSS Testing  Analysis of coverage of tests  Analysis & testing for insecure states  Ordered functional testing with tests of low-level design  Covert channel analysis Life-cycle controls  Compete automation of CM –with coverage for developer tools  Measurable life-cycle model –with supporting empirical data

173 June 2015© USC-CSE Operational Security Rating: Low & Nominal  Low –No organization-wide security policies –Ad-hoc security practices –Optional firewall & virus protection  Nominal Administrative controls  Basic Security policies –inc. Password & Virus Protection policy Network access & system use policy  Guidelines for administrators & users  Guidelines for regular backups of critical data Technical Controls  Reasonable practices for –Checksum Verification –Firewalls –Operating system logging  Simple password-based authentication schemes

183 June 2015© USC-CSE Operational Security Rating: High  Nominal + Administrative Controls  Security policies include –User administration policies –Access administration policies  Documentation & logging of all security incidents  Discretionary Access Control Policies for all resources  Security Screening of all staff on project –Initial background screening Technical Controls  Media access controls –Marking, logging & Integrity verification  Strong communication security implemented with reasonable practices for –Virtual Public Networks –Wireless Access –Active systems & network monitoring  Multi-factor authentication using passwords & smart-tokens

193 June 2015© USC-CSE Operational Security Rating: Very High  High + Administrative Controls  Security policies include – Incident response policy – Data classification policy  Incident response teams formed to handle security breaches  Mandatory Access Control Policies to all resources  Project staffing process considers –Separation of duties –Access with least privilege Technical Controls  Defense-in-Depth strategy to protect system with reasonable practices for –Layered system monitoring with Intrusion Detection Systems –Write protected audit trails  Digital certificates & signatures used for –Authentication –Message integrity –Non-repudiation

203 June 2015© USC-CSE Operational Security Rating : Extremely High  Very High + Administrative Controls  Comprehensive Security policies including – Business continuity plans – Disaster recovery plans  Strong role-based access control for all resources Technical Controls  Defense-in-depth strategy for protection is implemented with reasonable practices for –Protection against insider & outsider attacks –Honeypots for proactive defense  Independent vulnerability analysis & penetration tests  All communications are encrypted  Multi-factor authentication with biometrics

213 June 2015© USC-CSE Development Constraints Rating Description  Nominal –None  High –All source materials are locked up when not in active use  Very High –High + Audited security markings in code  Extremely High –Very High+ Multi-compartment developer communication constraints

223 June 2015© USC-CSE Outline  Why Extend COCOMO for Security?  The COCOMO II Modeling & Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

233 June 2015© USC-CSE Draft Security Effort Distribution With large Standard Deviation

243 June 2015© USC-CSE Outline  Why Extend COCOMO for Security?  The COCOMO II & Modeling Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

253 June 2015© USC-CSE Existing COCOMO Drivers Affected By Security  Proposed security cost driver (SECU) constrain existing COCOMO II cost drivers  Security functions add to project’s size  Increases project risk RELYRequired software reliability CPLXProduct complexity DOCUDocumentation match to life-cycle needs SITEMulti-site development TOOLUse of software tools

263 June 2015© USC-CSE Security Effect on Risk  Projects risk is increased when –Security driver rating is >= high & –Rating of following drivers are <= low or nominal ACAP : Analyst Capability PCAP : Programmer Capability APEX : Application Experience PLEX : Platform Experience LTEX : Language Experience

273 June 2015© USC-CSE Outline  Why Extend COCOMO for Security?  The COCOMO II & Modeling Methodology  New Security Cost Driver & its Factors  COCOMO II Cost Driver Values  Analysis of Security Impact On Other Drivers  Next Steps & Summary

283 June 2015© USC-CSE Next Steps  Reach consensus on cost drivers –FAA Workshop with AS400 on security, July 03 –Delphi run & calibration of factors in works  Initiate efforts to statistically validate accuracy of model –Survey available 160+ COCOMO project data –Perform initial calibration  Create enhanced COCOMO data collection forms –Gather security related efforts  Compare actual project data to expert opinions –Calibrate the model by weighting Actual data Expert opinions using Bayesian statistical techniques

293 June 2015© USC-CSE Summary  Proposed extensions to COCOMO for development of Secure Systems –Based on Common Criteria & DITSCAP –1 Driver: SECU –3 Factors: Development for Security Operational Security Physical Security (Development Constraints) –Affects on other COCOMO II Drivers RELY, CPLX, DOCU, SITE, TOOL –Affects on Size –Affects on project risk  Hopefully stimulated your interest & motivated you to participate by sharing project data