1. © 2004 Spire Security, LLC. All rights reserved. security i SPRE Security Measures & Metrics Pete Lindstrom, CISSP Research Director Spire Security, LLC www.spiresecurity.com petelind@spiresecurity.com

2. © 2004 Spire Security. All rights reserved. 2 Security Metrics I Security Metrics (Part 1): Building the Framework There are obvious benefits to charting and quantifying the success of your security program. But where do you begin? This session -- part 1 of a 2-part mini- workshop -- outlines a practical approach to security metrics that links standard business practices with security functions. Find out from Information Security magazine contributing editor, Pete Lindstrom, Research Director for Spire Security, how to build a rock-solid foundation based on a model known as the "Four Disciplines of Security Management." Then learn about the elements of a cohesive security metrics program from a functional and resource-usage perspective. Plus, you leave with a solid understanding of the relative utility metrics for productivity, process efficiency, cost effectiveness and risk management.

3. © 2004 Spire Security. All rights reserved. 3 What is the Four Disciplines Model?  A way to think about security oHigh-level without losing clarity oDetailed enough for technical folks oIdentifies relationships  A taxonomy of objectives, functions, activities, and products.  A framework for security measurement.

4. © 2004 Spire Security. All rights reserved. 4 Introducing the Four Disciplines Identity Mgt: Managing Users and other sources Threat Mgt: Monitoring activities and events Trust Mgt: Designing security policy and process Vuln. Mgt: Hardening the systems 3 4 2 1

5. © 2004 Spire Security. All rights reserved. 5 1. Harden Systems

6. © 2004 Spire Security. All rights reserved. 6 Vulnerability Mgt Functions  Evaluate and harden configurations oBy platform  Identify and remediate vulnerabilities oSoftware bugs  Configure firewalls / other access control  Reduce/filter anomalous traffic

7. © 2004 Spire Security. All rights reserved. 7 2. Identify/Manage Users

8. © 2004 Spire Security. All rights reserved. 8 Identity Management Functions  Validate user information  Create/modify user accounts and privileges  Disable/delete user accounts  Change/reset passwords  Validate sessions  Authorize access

9. © 2004 Spire Security. All rights reserved. 9 3. Design/Strengthen Processes

10. © 2004 Spire Security. All rights reserved. 10 Trust Management Functions  Create/modify user policies  Create/modify system policies - technical baselines  Design security architecture  Design/implement controls to prevent sniffing or copying data.  Design/implement controls to prevent modifying data.

11. © 2004 Spire Security. All rights reserved. 11 4. Monitor Environment

12. © 2004 Spire Security. All rights reserved. 12 Threat Management Functions  Identify anomalous activities oMonitor network and components oAggregate alerts and logs oCollect physical information  Manage/resolve incidents  Incident response - take corrective action  Conduct forensic analysis of systems/data

13. © 2004 Spire Security. All rights reserved. 13 Putting It All Together

14. © 2004 Spire Security. All rights reserved. 14 Q1: Most Important? Which Discipline is most important to a strong security program? 1.Vulnerability Management (firewalls, vuln assess, patch) 2.Identity Management (provision, acct mgt, authent.) 3.Trust Management (policies, tech guides, crypto) 4.Threat Management (monitor, incident, forensics)

15. © 2004 Spire Security. All rights reserved. 15 Q2: Most Time? Which Discipline does your organization spend the most time on? 1.Vulnerability Management (firewalls, vuln assess, patch) 2.Identity Management (provision, acct mgt, authent.) 3.Trust Management (policies, tech guides, crypto) 4.Threat Management (monitor, incident, forensics)

16. © 2004 Spire Security. All rights reserved. 16 Fundamental Security Elements People: Departments Admins Costs: Salaries, Consulting HW, SW, Maint. Time: Hr/Day Month/Yr Resources: User accts, systems, apps Activities: Four Disciplines

17. © 2004 Spire Security. All rights reserved. 17 Types of Metrics  Process Effectiveness – doing things right. (measure quality)  Staff Productivity – people doing more things. (measure volume)  Cycle Time – transaction time. (measure process efficiency)  Staff Efficiency – people doing things faster. (people / transaction / time)  Cost Effectiveness – transaction costs. (cost / activity)

18. © 2004 Spire Security. All rights reserved. 18 Process Effectiveness Metrics “doing things right” Key Elements: Activities errors Examples: Acct request errors Remediation errors False alarm rate Policy exceptions error rates

19. © 2004 Spire Security. All rights reserved. 19 Process Effectiveness  Measure quality by identifying error rates of activities  Identity Management oUser account request errors  Vulnerability Management oVulnerabilities not remediated  Threat Management oImproper incident management  Trust Management oPolicy violations

20. © 2004 Spire Security. All rights reserved. 20 Staff Productivity Metrics “people doing more things” Elements: People Activities Examples: Accts per person Vulns per person Patches per person

21. © 2004 Spire Security. All rights reserved. 21 Staff Productivity  Productivity and workload for all manual activities (activities/people)  Identity Management oRequests per administrator oAccount disablements per admin oPassword resets per admin  Vulnerability Management oVulnerabilities resolved per administrator  Threat Management oIncidents per person  Trust Management oPolicy changes per person

22. © 2004 Spire Security. All rights reserved. 22 Cycle Time Metrics avg “time to perform activity x” Elements: Time Activities Examples: Accts per month Vulns fixed per month Patches per month

23. © 2004 Spire Security. All rights reserved. 23 Cycle Time  Process efficiency  Identity Management oUser account request time to complete  Vulnerability Management oRemediation time to complete  Threat Management oIncident response time to complete  Trust Management oPolicy creation time to complete

24. © 2004 Spire Security. All rights reserved. 24 Staff Efficiency Metrics Admins by Department 2000 Hours per FTE “people doing things” quicker Elements: People Activities Time Examples: Accts per person/hr Vulns per person/hr Patches per person/hr

25. © 2004 Spire Security. All rights reserved. 25 Staff Efficiency  Combines staff productivity and cycle time metrics.  Identity Management oUser account requests completed per person per day/week/month  Vulnerability Management oVulnerabilities remediated per person per day/week/month  Threat Management oIncidents closed per person per day/week/month  Trust Management oPolicies reviewed per person per day/week/month

26. © 2004 Spire Security. All rights reserved. 26 Cost Effectiveness Metrics Cheaper transactions Elements: Activities Costs Examples: Cost per acct Cost per vuln fixed Cost per patch

27. © 2004 Spire Security. All rights reserved. 27 Cost Effectiveness  Dollars/activities; dollars/resources; dollars/demographics  Identity Management oCost per request oCost per password reset  Vulnerability Management oCost per vulnerability oCost per system setting  Threat Management oCost per incident  Trust Management oCost per policy oCost per project

28. © 2004 Spire Security. All rights reserved. 28 When to Use Metrics  Process Effectiveness oSix Sigma  Staff Productivity oROI / promotions  Cycle Time oBalanced Scorecard  Staff Efficiency oROI  Cost Effectiveness oActivity-based costing oROI/TCO

29. © 2004 Spire Security. All rights reserved. 29 Q3: Most Useful? Which metric type is most useful to your security program? 1.Process Effectiveness 2.Staff Productivity 3.Cycle Time 4.Staff Efficiency 5.Cost Effectiveness

30. © 2004 Spire Security. All rights reserved. 30 Conclusions  Security functions are spread throughout organizations.  You can’t improve security until you measure it.  Ultimately, security is a business operation that should be run like a business operation.

31. © 2004 Spire Security, LLC. All rights reserved. security i SPRE Pete Lindstrom petelind@spiresecurity.com www.spiresecurity.com Agree? Disagree?