1. 2006 Annual Research Review & Executive Forum
Costing Secure System
COSECMO Data Mining
Danni Wu
Edward Colbert
{danwu,
2006 Annual Research Review & Executive Forum
USC Center for Software Engineering
© 2006 USC-CSE
18 September 2018
© 2006 USC-CSE

2. 2006 Annual Research Review & Executive Forum
Costing Secure System
Goal Of Presentation
Review data mining of
COCOMO & COCOMO II data sets
Published Security Targets
How affects COCOMO II for development of secure software systems (“COSECMO”)
MetaH provides semantics & supporting tools
UML provides graphic front-end
© 2006 USC-CSE
18 September 2018
© 2006 USC-CSE

3. Outline Hidden Security Requirements in COCOMO data sets?
Security Assurance & Functional Requirements Relations
© 2006 USC-CSE
18 September 2018

4. Mining COCOMO Data Sets
Questions:
Do any of projects in COCOMO data set have security requirements?
Nobody asked when collected
Do we have any data that might support COSECMO
Behavior analysis
Calibration
validation
We have data
Document size for COCOMO 81 projects
Range of test data size (DATA) for COCOMO II
© 2006 USC-CSE
18 September 2018

5. Security, Reliability, & Document Size in COCOMO 81
E – A /A
Percentage estimation error of effort
Doesn’t match expectation
COCOMO will significantly under-estimate projects with RELY driver rating High or Very High
1 – Very Low, 2 – Low, 3 – Nominal, 4 – High, 5 – Very High
* Based on COCOMO ‘81 data set (50 projects)
© 2006 USC-CSE
18 September 2018

6. Security, Reliability, & Document Size in COCOMO 81 (cont.)
2006 Annual Research Review & Executive Forum
Costing Secure System
Security, Reliability, & Document Size in COCOMO 81 (cont.)
PP/TKDSI:
Pages of documentation per 1000 source instructions
Documentation increases when RELY rating goes up
Expected
3 project show “excessive” documentation compared to others with same RELY rating
Variance increase with RELY rating
Is discrepancy result of security or safety requirements?
1 – Very Low, 2 – Low, 3 – Nominal, 4 – High, 5 – Very High
* Based on COCOMO ‘81 data set (50 projects)
© 2006 USC-CSE
18 September 2018
© 2006 USC-CSE

7. Security, Reliability, & Document Size in COCOMO 81 (cont.)
COCOMO 81 under estimates effort of the 3 projects
Is documentation size factor?
Project
RELY
PP/KDSI
E-A / A (%) 1
High
291
-14 2
194
-18 3
Very High
241
-33
© 2006 USC-CSE
18 September 2018

8. No DOCU Driver in COCOMO 81
Security, Reliability, & Document Size in COCOMO 81 (cont.) What about DOCU Driver in COCOMO 2000?
No DOCU Driver in COCOMO 81
In COCOMO 2000, only asks if size “is/is not excessive for lifecycle need”
High reliability, security, & safety assurance routinely needs more documentation So
Extra documentation not excessive
DOCU would be set to nominal
© 2006 USC-CSE
18 September 2018

9. Security, Reliability, & Document Size in COCOMO 81 (cont
Security, Reliability, & Document Size in COCOMO 81 (cont.) RELY & DOCU in COCOMO 2000*
No clear trend indicates that DOCU rating increases when RELY rating increases
1 – Very Low, 2 – Low, 3 – Nominal, 4 – High, 5 – Very High
* Based on COCOMO 2000 data set (161 projects)
© 2006 USC-CSE
18 September 2018

10. New security driver will have to address
Security, Reliability, & Document Size in COCOMO 81 (cont.) Observations
Some high-reliability projects in COCOMO ’81 set are under estimated
Possibly due to unaccounted effect of security or safety requirement
Reliability requirements accounted for
At least in theory
DOCU driver doesn’t help
Only looks at size “is/is not excessive for lifecycle need”
New security driver will have to address
© 2006 USC-CSE
18 September 2018

11. Security Target File Analysis
Goals
Discover relationship between EAL & number of SFRs
Size estimation by security objectives
256 files from NIST website
Domains include
Access control devices & systems
Boundary protection devices & systems
Database Management Systems
Operating System
© 2006 USC-CSE
18 September 2018

12. Are Security Assurance & Functional Requirements Really Independent?
Common Criteria v2 treats as independent
Do more SFR’s always indicate higher EAL requirement?
© 2006 USC-CSE
18 September 2018

13. Mapping EAL with SFRs Data from 256 ST files in NIST website
Range of SFR: [3, 107]
Range of EAL: [1, 7]
© 2006 USC-CSE
18 September 2018

14. Estimating Size by Security Objectives
Current COSECMO:
SFR classes for size estimation
Security objectives are more intuitive to developer in early phase of SW life-cycle
© 2006 USC-CSE
18 September 2018

15. CC SFR Classes Correlations
Red => High correlation between two classes
© 2006 USC-CSE
18 September 2018

16. SFR Classes Correlations (cont.)
FDP (Data Protection) & FCS (Cryptographic Support)
Most relations cannot be discovered by this matrix
3 or more SFR classes may be needed to achieve 1 security objective
© 2006 USC-CSE
18 September 2018

17. Mapping Security Objectives with SFRs
* (X) indicates the corresponding SFR class is optional for a particular security objective
© 2006 USC-CSE
18 September 2018

18. How to Choose SFRs Step 1: Define project domain
Step 2: List main security objectives
Step 3: Selecting SFR classes based on security objectives
Step 4: Identify SFR’s that support selected SFR classes
© 2006 USC-CSE
18 September 2018

19. Case Study: Firewall Domain: Boundary Protection Devices & Systems
Primary security objectives:
Authentication
Accountability
Intrusion Detection & Response
Selected SFR classes:
Authentication – FDP, FIA, FMT, FPT, FTP
Accountability – FAU, FDP, FMT, FPT, FTP
Intrusion Detection & Response – FAU, FMT, FPT
FAU
FDP
FIA
FMT
FPT
FTP
© 2006 USC-CSE
18 September 2018

20. Regression Analysis Result
Response = TOTALSFR
Terms = (FAU FDP FIA FMT FPT FTP)
Coefficient Estimates
Label Estimate Std. Error t-value p-value
Constant
FAU
FDP
FIA
FMT
FPT
FTP
R Squared:
Sigma hat:
Number of cases:
© 2006 USC-CSE
18 September 2018

21. Case Study: Database Domain: Database Primary security objectives:
Authentication
Accountability
Availability
Integrity
Recoverability
Selected SFR classes:
Authentication – FDP, FIA, FMT, FPT
Accountability – FAU, FDP, FMT, FPT
Availability – FMT, FPT, FRU
Integrity – FDP, FMT, FPT
Recoverability - FDP, FMT, FPT, FTA
FAU
FDP
FIA
FMT
FTA
FPT
FRU
© 2006 USC-CSE
18 September 2018

22. Observation from ST File Analysis
Total 11 ST files in the domain
Seven SFR classes are used in Database domain
FAU, FDP, FIA, FMT, FPT, FRU, FTA
Availability: Achieved mainly by FRU class
FRU_RSA.1 Resource Allocation
© 2006 USC-CSE
18 September 2018

23. Conclusions
Potential data points for COSECMO could exist in current COCOMO data set
DOCU does not help in cost estimation for secure software
Trend exists that EAL increases when SFR increases
Prove mapping between SFR classes & security objectives
By ST files analysis
© 2006 USC-CSE
18 September 2018

24. Future Work
Further analysis on other COCOMO drivers using COCOMO 2000 data set
DOCU driver may not be the only one
DATA? TOOL?
Size estimation using security objectives
Collect expert opinions
Run Delphi
Get size data from ST files’ vendors
© 2006 USC-CSE
18 September 2018

25. Appendix: Common Criteria SFR classes
FAU – Security Audit
FCO – Communication
FCS – Cryptographic Support
FDP – User Data Protection
FIA – Identification and Authentication
FMT – Security Management
FPR – Privacy
FPT – Protection of the TSF (Trusted Security Function)
FRU – Resource Utilization
FTA – TOE Access
FTP – Trusted Path/Channels
© 2006 USC-CSE
18 September 2018