1. Costing Secure Systems Workshop
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Costing Secure Systems Workshop
Edward Colbert,
Danni Wu
Yue Chen
5th Workshop on Costing Secure Systems USC CSE Annual Research Review 2004
© USC-CSE
18 January 2019
© USC-CSE

2. Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Goal Of Workshop 
Review proposed
Model for costing development of secure systems
Extensions to COCOMO II for development of secure software systems
Feedback on behavior analysis
Validate proposed models
Identify research opportunities
Kick-off Delphi process
Review possible approaches & forms
Identify data sources  
© USC-CSE
18 January 2019
© USC-CSE

3. Participants Ed Colbert, USC, Moderator (ecolbert@cse.usc.com)
Barry Boehm, USC, Guru
Cheryl Jones, US Army
Clate Stansbury, MCR
Danni Wu, USC
David Seaver, Price Systems
Lee Bergstrom, Lockheed Martin Management & Data System
Paul Stelling, Aerospace Corp.
Sachin Shaw, USC usc.edu)
Sherman Paskett, General Dynamics Decision Systems
Winsor Brown, USC, Camera Man
Yue Chen, USC
© USC-CSE
18 January 2019

4. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

5. Cost Model for System Security Increment 1 (Feb – July ’04)
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Cost Model for System Security Increment 1 (Feb – July ’04)
Task Element
Activities
1. Develop Early Estimation Model
Prototype model
2. Sources of Cost
Identify, define, scope sources of cost
Relate sources of cost to FAA WBS
Recommend type of CER for each
3. Secure Product Taxonomy
Identify, define, scope product elements
4. COCOMO II Security Extensions
Refine model form and data definitions
5. COCOTS Security Extensions
Explore security aspects in COCOTS data collection
© USC-CSE
18 January 2019
© USC-CSE

6. Cost Model for System Security Increment 2 (Aug ’04 – July ’05)
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Cost Model for System Security Increment 2 (Aug ’04 – July ’05)
Task Element
Activities
1. Develop Early Estimation Model
Experimental use & refinement
2. Sources of Cost
Prioritize sources of cost needing CER’s
Refine, prototype, experiment with top-priority CER’s
Relate to scope of COCOMO II security extensions
3. Secure Product Taxonomy
Experimental use, feedback, and refinement
4. COCOMO II Security Extensions
Refine, scope, form, definitions based on results of Tasks 1-3
Experimentally apply to pilot projects, obtain usage feedback
5. COCOTS Security Extensions
Develop initial scope, form, definitions based on results of Tasks 1-4
© USC-CSE
18 January 2019
© USC-CSE

7. Cost Model for System Security Increment 3 (Apr ’05 – Sep ’06)
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Cost Model for System Security Increment 3 (Apr ’05 – Sep ’06)
Task Element
Activities
1. Develop Early Estimation Model
Evolution; integration with other models
2. Sources of Cost
Refine sources of cost, CER’s based on usage feedback
Integrate with other models
Address lower-priority CER’s as appropriate
3. Secure Product Taxonomy
Monitor evolution
4. COCOMO II Security Extensions
Baseline model definitions
Collect project data
Develop initially calibrated model; experiment and refine
5. COCOTS Security Extensions
Experimentally apply to pilot projects
Refine, baseline based on usage feedback
© USC-CSE
18 January 2019
© USC-CSE

8. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

9. Formula for Cost of System & of Security
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Formula for Cost of System & of Security
Etotal = EInitial/Mission Analysis + EInvestment Analysis + ESystem Engineering + EDev & Imp + ESys of Sys Integration + Einstall/deployment + EO&M + EDisposal
EDev & Imp = Edesign & build HW + Edesign & build SW + Epurchased services + ECOTS-Sys
Etotal (Security) = Etotal (with sec) – Etotal (without sec)
COTSYS  Commercial of the Shelf Systems
© USC-CSE
18 January 2019
© USC-CSE

10. Formula Elements & COCOMO Family
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Formula Elements & COCOMO Family
Formula Elements
COCOMO Family Member
ESystem Engineering
COSYSMO (new)
Edesign & build SW
COCOMO-II
COCOTS
ESys of Sys Integration
COSoSIMO (new)
© USC-CSE
18 January 2019
© USC-CSE

11. Draft Model of Cost Distribution System Purchase
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Draft Model of Cost Distribution System Purchase
Distribution depends on type system acquired
# systems
affects
Installation
O&M
Disposition
After 1st, costs are less
Example: $1M System
Numbers for show only
CORE of this model is for SW
© USC-CSE
18 January 2019
© USC-CSE

12. Draft Model of Cost Distribution System Development
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Draft Model of Cost Distribution System Development
Distribution depends on type system acquired
# systems
affects
Installation
O&M
Disposition
After 1st, costs are less
Example: $1M System
Numbers for show only
CORE of this model is for SW
© USC-CSE
18 January 2019
© USC-CSE

13. Draft Model of Cost Distribution System Development with COTS
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Draft Model of Cost Distribution System Development with COTS
Distribution depends on type system acquired
# systems
affects
Installation
O&M
Disposition
After 1st, costs are less
Example: $1M System
Numbers for show only
CORE of this model is for SW
© USC-CSE
18 January 2019
© USC-CSE

14. Draft Model of Cost Distribution Services
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Draft Model of Cost Distribution Services
Distribution depends on type system acquired
# systems
Affects
Implementation
O&M
Disposition
After 1st, costs are less
Example: $1M System
Numbers for show only
CORE of this model is for SW
© USC-CSE
18 January 2019
© USC-CSE

15. Cost Model for Secure System Approach
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Cost Model for Secure System Approach
Identify major sources of cost To
Develop
Own
Including
Facilities
Equipment
People
Acquired Systems
Services
© USC-CSE
18 January 2019
© USC-CSE

16. Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
How to Estimate Costs?
Costing Approaches
Activity Models
Unit Costing
Analogy Base
Parametric
For each source of cost, identify appropriate means
Cost Estimation Relation (CER)
© USC-CSE
18 January 2019
© USC-CSE

17. Cost Estimation Relations (CER) Example
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Cost Estimation Relations (CER) Example
Sample Activity
Preparation for Training
Classroom Training
Periodic Training on new procedures
Software Development
CER
Activity–based
Unit costing
Analogy-based
Parametric
Rule
10-20 hours for each Class Hour
N trainers total M trainees
It cost us $XXX last year,…
COCOMO II
© USC-CSE
18 January 2019
© USC-CSE

18. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

19. FAA Acquisition & Standard WBS
Analyzed FAA WBS to identify where security will affect activities
David Seaver pointed not clear where Analysis of Security Issues related to data management fits
Started study Standard WBS
© USC-CSE
18 January 2019

20. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

21. Map Security Objectives with Common Criteria
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Map Security Objectives with Common Criteria
Updates
© USC-CSE
18 January 2019
© USC-CSE

22. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

23. COCOMO II Security Driver (SECU)
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
COCOMO II Security Driver (SECU)
Viewpoints
Design & Development for Security
Operational Security
Physical Security (Development Constraints)
Driver Ratings
Nominal
High
Very High
Extremely High
Sky High
Stratospheric
Sky High (or Super High)
Stratospheric (or Ultra High)
New COCOMO
Levels
© USC-CSE
18 January 2019
© USC-CSE

24. Effect Of Security On COCOMO II
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Effect Of Security On COCOMO II
Security functional requirements add to project’s KSLOC’s
Many systems add
Security Manager or
Authorization– & Access-Controller
“Core Application” adds code to do support authorization & access checks
PMtotal = PMtrusted + PMapplication
© USC-CSE
18 January 2019
© USC-CSE

25. Example of COCOMO Security Extension
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Example of COCOMO Security Extension
Assume:
Application 1000 KSLOC SECU (App) = Nominal
Trusted SW 10 KSLOC SECU (trusted) = High
SF = 1
All Multipliers (except Security) = 1 (Nominal)
SECU Values
Nominal = 1
High = 2
VH = 3
EH = 4
SH = 6
Strat = 7
Then
PM (total) = 2.94 * (PM (Trusted) + PM (App))
= 2.94 * (10 * SECU(Trusted)) + (1000 * SECU(App))
= 2.94 * ((10 * 2) + (1000 * 1))
= 2.94 * 1020
Effort in Person Month
SF: Scale Factors (5) EM: Effort Multipliers(17)
© USC-CSE
18 January 2019
© USC-CSE

26. Number of Security Drivers Issue
Current model
Add 1 driver that addresses security from 3 viewpoints
Development
Operational
Physical (Development Constraints)
Alternative:
2 drivers
Security Functions or Objectives
Security Assurance
Need further research
© USC-CSE
18 January 2019

27. Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
Effect Of Security On COCOMO II (cont.) Relations to Existing Drivers Pre-Workshop
If Security driver rating is >= high & following drivers must be > Nominal
PREC Precedence (team done similar systems)
PMAT Process Maturity
TEAM Team Cohesion
RELY Required software reliability
CPLX Product complexity
DOCU Documentation match to life-cycle needs
SITE Multi-site development
TOOL Use of software tools
ACAP Analyst Capability
PCAP Programmer Capability
Etc.
© USC-CSE
18 January 2019
© USC-CSE

28. Effect Of Security On COCOMO II (cont
Effect Of Security On COCOMO II (cont.) Refined Relations to Existing Drivers
Attendees thought best to treat “Clashes” as risk
e.g. Precedence (PREC)
Security > High  Project = high risk if
PREC < High, and
ACAP, PCAP & APEX < High
Need further investigation for Security levels above Highs
© USC-CSE
18 January 2019

29. Effect Of Security On COCOMO II (cont
Effect Of Security On COCOMO II (cont.) Refined Relations to Existing Drivers
For Scale Factors:
Need to consider how much security drives entire project
Cannot easily assign different values to
Trusted
Non-trusted software
© USC-CSE
18 January 2019

30. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

31. COCOMO Security Extension Delphi Issues Different Stakeholders Needs
Customer: “how much will system cost me?”
Includes costs outside scope of developer
e.g. Independent Testing & Certification
Affects developer schedule
Developer: “how much effort to build system?”
Conclusion:
Separate Delphi for Customer’s costs/effort other then developer’s
Developer need to
Effort to support Independent Testing & Certification
Adjust schedule for time taken by Independent Testing & Certification
© USC-CSE
18 January 2019

32. COCOMO Security Extension Delphi Issues Level of Detail to Collect
Costing Secure Software, FAA-IT-ISS R&D Workshop
1/18/2019
COCOMO Security Extension Delphi Issues Level of Detail to Collect
Current draft based on Common Criteria
Functional Requirements
11 Classes
67 Families
138 Components
250 Elements
Assurance Requirements
10 Classes
42 Families
93 Components
 21 Classes x 7 levels
~4 questions each
Is this 2 much to Ask?
Couldn’t find better alternative
Did
Eliminated Question about schedule for some
Separate Customer assurance work from Developer
Created some examples
NTK
© USC-CSE
18 January 2019
© USC-CSE

33. COCOMO Security Extension Delphi Issues Example: Development Assurance Req. (ADV)
Activity
Nominal
High
Very High
Extremely High
Super High
Ultra High
Authentication
No explicit authentication requirement
Single authentication mechanism (FIA_UAU.4)
Multiple authentication mechanism (FIA_UAU.5)
Re-authenticating (FIA_UAU.6)
Authentication failure handling (FIA_AFL.1)
FIA_AFL.1
Protected authentication feedback (FIA_UAU.7)
Simple trusted acknowledgement between TSF (FPT_SSP.1)
Protected authentication feedback and unforgeable authentication (FIA_UAU.7, FIA_UAU.3)
Mutual trusted acknowledgement between TSF (FPT_SSP.2)
Trusted KSLOCS?
2-4
3-5
Application KSLOC Increase (%) 1%
1.1%
1.2%
Effort to Produce Kernel (%)
Additional Application Effort (%)
Rational / Comments
At higher levels, need to be more bug free, more effort for unit design, documentation & testing then predicted  need to have higher RELY & DOC
© USC-CSE
18 January 2019

34. COCOMO Security Extension Delphi Issues Example: Development Assurance Req. (ADV)
Activity
Nominal
High
Very High
Extremely High
Super High
Ultra High
Requirements Specification Standard
No explicit security requirements
Informal functional & interface specification (ADV_FSP.1)
Fully defined external interfaces (ADV_FSP.2)
Informal security policy modeling
(ADV_SPM.1)
Semi-formal functional specifications (ADV_FSP.3)
Semi-formal security policy modeling (ADV_SPM.2)
Semi-formal functional specification (ADV_FSP.3)
Formal security policy modeling (ADV_SPM.3)
Formal functional specification (ADV_FSP.4)
Trusted KSLOCS?
Application KSLOC Increase (%)
Effort to Produce Kernel (%)
10%      50
100
200
250
Additional Application Effort (%)
Rational / Comments
     
© USC-CSE
18 January 2019

35. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

36. Issues 1 or 2 Drivers? Length of Delphi Different Stakeholders Needs
Separate security assurance from functional requirements
1 driver for both
Length of Delphi
Different Stakeholders Needs
Customer: “how much will system cost me”
Developer: “how much effort for system”
What data
Do projects have?
Can we get projects to collect
© USC-CSE
18 January 2019

37. Highlights Project Schedule Early Estimation Model
Work-Breakdown Structure (WBS)
Common Criteria & Security Objectives
COCOMO Security Extension
Delphi Collection Approach
Issues
To Do
© USC-CSE
18 January 2019

38. To Do Create website Behavior Analysis Refine models Revise Delphi
Commercial
Security community
Refine models
Revise Delphi
Collect & analyze data
Write Ph.D. Thesis (theses?)
© USC-CSE
18 January 2019