Presentation is loading. Please wait.

Presentation is loading. Please wait.

Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron.

Similar presentations


Presentation on theme: "Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron."— Presentation transcript:

1 Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron Thomas Principal, Meritology Mini-Metricon, February 5, 2007 San Francisco, CA

2 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 2 Purpose of this Talk To introduce a new approach –Influence thought leaders, academic research, and professional practice –Stimulate your thinking and inspire hope Build productive bridges between business and IT –Show how key concepts of each can be made compatible –Take a stand on what will work and what wont To get your feedback –Is this on the right rack? Is it worth pursuing? –Does it fit with other approaches to security metrics? To recruit collaborators and advocates Non-purposes –Debate the devilish details –Debate politics –Debate acceptability in Mainstream and Late Adopter organizations It will take years, of course!

3 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 3 The Challenge Problem: Disconnect between business decision-makers and security specialists regarding value and risk of InfoSec* –Security directors appear to be politically isolated within their companies – They face a challenging search for allies when they need to gain support from upper management for new security initiatives. –Companies reported less alignment of security with long-range strategic objectives of the firm. –The results suggest that security remains a function that is mired in operations in the eyes of senior executives. Result: under-spending, over-spending, misallocation, burden-dumping, denial, and worse… –Fighting the last war –Failures of imagination –Unintended consequences * Conference Board Survey Oct. 2006: Navigating RiskThe Business Case for Security

4 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 4 The Simplistic Approach is a Blind Alley ROSI *, ALE **, and variants p(L|e i )LiLi i = 1 ^ i = incident types n Probability of loss given incident & exposure Expected loss value Why a blind alley? Laplaces Dream: If only we had more data… (see appendix) Why a blind alley? Laplaces Dream: If only we had more data… (see appendix) V = ROSI * = V / I * Return on Security Investment ** Annualized Loss Expectancy Security investment Loss of Economic Value Example reference: Calculated Risk - Guide to determining security ROI - CSO Magazine - December 2002

5 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 5 Two Viewpoints on Economic Risk #1 Rational Investor (Capital Asset Pricing, Discounted Cash Flow) #2 Insurance Actuary (Ruin Theory, Iceberg Risk) Ruin time change in value value random walk with avalanches random walk p(v) What matters: Mean, variance Fat part of the curve When: Quarterly EPS Earnings volatility Shorter time periods 99% What matters: Extreme events Tail of the curve When: Credit rating Solvency Reserve funds Longer time periods Normal distributions Fat Tailed and skewed distributions

6 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 6 The Core Idea: Three Costs Categories Annual Probability Total Cost of InfoSec mean 1x1x10x100x 1,000x Budgeted Self-insurance Catastrophic (borrowed from Value at Risk concept in Financial Services Risk Management) Idealized

7 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 7 Budgeted Costs Q: What is the expected (average) impact of security-related costs on EPS and earnings volatility (+/– budget)? The rule: costs must already be in the budget * somewhere –Defined to fit the budget and spending approval processes –Results in stable ratio-scale values –Theoretically and practically sound Applies Activity-based Costing methods Compatible with accounting practice (GAAP) Fits discounted cash flow assumptions for multi-year analysis –Good information available (in principle) –Simple Arithmetic Tractable and simple to understand –Composable across organization units and systems If you are claiming cost reductions, show me whose budget I should cut. If you are claiming revenue increases, show me whose sales quota I should raise. (Exec VP) * Includes both operating and capital budgets, but excludes cyber insurance or reserves

8 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 8 Calculating Budgeted Costs (1) Aggregate direct costs –Security staff, training, awareness, tools, services, technology, management, threat monitoring, assessments, etc. –Direct cost of predictable and expected loss events and remediation w/ portfolio effects Use cost driver models for indirect costs –Patch testing, installation, upgrades, etc. –Vendor support costs, 3 rd party support –Help desk –New employee screening and hiring process –Indirect costs of predictable and expected loss events with portfolio effects Negotiate cost allocation rules for bundled and overhead costs –Infrastructure software and hardware costs –Application software –Internal IT development –Legal dept. Identify costs from unintended consequences and business prevention –Its a judgment call how best to account for these, but they will win credibility! If possible, use incremental cost analysis, not just total costs –Compare to a base case (e.g. a barely legal budget)

9 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 9 Calculating Budgeted Costs (2) Modeling indirect costs using cost drivers: e.g. Desktop/Laptop Incidents and Remediation Cost #1: Provisioning Cost #2: Help Desk Method: 1.Identify cost drivers using security metrics combined with business operational metrics (e.g. number of new employees, turnover, etc.). 2.Aggregate and simplify where possible. 3.Only account for budgeted (forward-looking) costs. Use historical costs as a guide, if available. Illustrative Benefits: Simplicity – many fewer budget categories than incident types, scenarios, etc. Effectiveness – puts attention on the right levers Focus – most often, a few cost drivers dominate (80/20 rule).

10 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 10 Calculating Budgeted Costs (3) Modeling indirect costs using cost drivers: e.g. Indirect costs of predictable and expected loss events, with portfolio effects Asset: Customer DB attacks, breaches, incidents Damage, violations, etc. Abstracted and Aggregated Exposure, given defenses Risk Drivers Detection, remediation, etc.. Cost Categories: Staff (extra headcount) Customer Service (damage control) etc. Benefits: Simpler calculations More robust to varying assumptions Cost Drivers

11 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 11 Decision Framework for Budgeted Costs Differential Analysis #1 Total Budgeted Costs vs. benchmarks Barely legal Budget Direct Indirect Premium Budget Current Budget Time Current #4 Self-insurance Cost Implications Higher Same Lower #3 Lifetime #2 Budget Optimization

12 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 12 Self-Insurance Cost Q: How much money would you put aside each year into a reserve fund * to avoid a serious decline in credit rating due to low-probability/high-impact losses? The rule: an actuarially-sound self-insurance premium, given… –Budget-busting loss events Severe outage, delay in a key new product, loss of major sales contract, etc. Material to quarterly EPS (> 1% ) –Extreme loss events (short of bankruptcy) that threaten credit rating, etc. Long-lasting business interruption, executive fraud, earnings restatement, regulatory action, punitive damages, etc. –Interdependencies, correlations (avalanche effects), and portfolio effects –Parameters: Maximum risk threshold and time horizon set by top management –Mark to Model approach, calibrated by history & wisdom of the crowds A betting mans judgment: The race doesnt always go to the swiftest, but thats how you bet. *Analogous to the concept of Economic Capital in financial services

13 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 13 Calculating Self-Insurance Cost (1) 99 th Percentile threshold 1 2 Budget threshold Self-insurance pool (Value at Risk) 3 Time period * 5 4 Fund solvency * Shape of the curve Annual premium Pool ÷ (Time Period) Estimation Parameters * Policy decisions by top management Modeling: Distribution curves from parameters Monte Carlo simulation of self-insurance pool with funding parameters, interest rates, etc. to calculate annual premium Dominated by largest losses 2 6 Interest rates Cost distribution curve Magnitude of costs (if time period is long enough)

14 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 14 Calculating Self-Insurance Cost (2) How: A Competitive Marketplace for Models time Consensus Estimates Prediction Markets Delphi Technique Qualitative Reasoning (e.g. Inference to the Best Explanation, Reasoning about Uncertainty, etc.) Bayesian Networks Statistical analysis of historical loss data External data bases, benchmarks parameter Parameter values change with new information Assessments, Scorecards Simulations

15 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 15 Ways to Make Self-Insurance Cost Real Link it to real cyber insurance policies Set up a real self-insurance fund via Finite Risk program* or tradable subordinated debt Use it as the glue for multi-firm risk sharing pools –Focused on information sharing and mutual assistance, with incentive instruments Link to performance management and incentive compensation –Subdivide Self-Insurance Cost into a Risk Budget for each org. unit, or –Use it as a risk adjustment factor for other performance metrics Create incentive instruments tied to self-insurance costs or cost drivers for… –Security outsource vendors –Supply chain partners –Channel partners –Customers –Alliance partners Public disclosure –SEC filings, other regulatory filings –Stakeholder reports –Credit rating agencies –Cap and Trade markets *See appendix

16 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 16 Catastrophic Costs Q: How much confidence should we have that the firm can survive InfoSec catastrophes? The rule: prioritized loss scenarios above a significance threshold that cover the space of possibilities. –Use for business continuity preparation agility and robustness –Avoid failures of imagination and fighting the last war –Root out unintended consequences –Categorize and prioritize – dont waste time on precision estimates –Strategic scenario analysis, war gaming, etc. –Focus on discovery, out of the box, and reframing –Challenge conventional wisdom! Its not what we dont know that will kill us. Its what we know that aint so.

17 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 17 Risk Management Decisions Budgeted Costs Self-insurance Costs Catastrophic Costs Prudence Gambling

18 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 18 A Simple Example – Earthquake Preparation ProbabilitiesMin Prep.Max Prep.Benefits Quake2% No Quake98% Mod. | Quake88%94%46%lower cost of moderate damage Severe | Quake10%5%50%reduction in probability of severe damage Death | Quake2%1%50%reduction in probability of death (catastrophe) #1: Minimum Preparation#2: Maximum Preparation ProbabilityCostALEProbabilityCostALE Preparation costs98% $ 60 $ 5998% $ 1,500 $ 1,470 Mod. Damage1.76% $ 57,060 $ 1, % $ 31,500 $ 592 Severe Damage0.20% $ 500,060 $ 1, % $ 501,500 $ 502 Death + Severe0.04% $ 2,500,060 $ 1, % $ 2,501,500 $ 500 ALE $ 3,063 ALE $ 3,064 Mean* $ 2,887 Mean* $ 3,087 Spend an extra $1,440 per year over 30 years for earthquake loss reduction? ALE same for both Simple average says no to extra spending * from Monte Carlo simulation

19 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 19 Self-insurance Costs (1)

20 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 20 Self-insurance Costs (2) Total Cost Comparison Max. Prep. vs. Min. Prep. Budgeted $ (1,440) Self-insurance $ 2,760 Annual Savings $ 1,320 Justifies extra spending on maximum preparation

21 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 21 Needed: Self-insurance Decision Framework Total Cost Comparison Max. Prep. vs. Min. Prep. Budgeted $ (1,440) Self-insurance $ 2,760 Annual Savings $ 1,320 Total Cost Comparison Max. Prep. vs. Min. Prep. Budgeted $ (1,440) Self-insurance INTEREST 10% $ 2,76 Annual Savings $ (1,164) Which is more credible? Which leads to better decisions? B. Self-borrowing A. Like other insurance

22 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 22 Summary of the Method Apply enterprise risk management methods Break InfoSec costs into three categories: –Budgeted –Self-insurance –Catastrophic Establish methods, targets, and decision processes for each category –Appropriate to the information and uncertainty involved –The nature of decisions that apply –Link the categories Use operational metrics plus inference to model costs in each category, as appropriate Focus energy on continuous organization learning

23 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 23 Next Steps Need more theoretical development and empirical testing –Esp. self-insurance concept, models, and decision rules. –Factoring in impact on revenue, market share, profitability (pricing power), and reputation Need to standardize Budgeted Costs and map to InfoSec assessments and frameworks Need proofs-of-concept using real companies and real data Make it work politically –Enterprise Risk Managers = your new best friends –TQM and 6 Sigma Specialists = your allies –CFOs = Status excelsior sponsors –Neutralize or convert opposition (legal department, auditors, etc.) –Lead industries = Financial Services? Supply Chain? other? –Political change role model = Indian Gaming?? Make it acceptable to the mainstream managers Q: is it sufficiently promising to continue pursuing?

24 Appendix Russell Cameron Thomas Principal, Meritology Mini-Metricon, February 5, 2007 San Francisco, CA

25 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 25 Why Measuring the Value of InfoSec is Hard (1) Information security (InfoSec) should be seen * as a component of enterprise risk management. –"Risk is a forward-looking estimate of uncertain loss over a time period (same as the timeframe for return on the assets). –Must cope with all forms of uncertainty and ignorance that apply to actors, assets, threats, vulnerabilities, and learning/adaptation over that timeframe. InfoSec is a repeating evolutionary game –Between threatening actors (incl. nature) and protecting actors (incl. nature) –Each with an evolving capability set, which may be emergent, nascent, and/or tacit. –The terrain for the security game is threats, vulnerabilities, assets, etc. –Thus, "security" is not a state of the system or the assets. It's how the protecting actors define success in the game over time. –Economics of repeating evolutionary games arent well understood yet. They dont fit existing static equilibrium investment models. They require emergent, dynamic models, e.g. agent-based simulation *From the viewpoint of business value

26 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 26 Why Measuring the Value of InfoSec is Hard (2) InfoSec* is inextricably part of the cyber trust fur ball, including –Privacy –Digital Rights –Intellectual Property, brands, reputation, trade secrets –Stakeholder disclosure … and physical security Historical loss data, even if copious and available, has limited use –The landscape changes too fast –Low frequency / high impact events matter –Unique events matter The business value of InfoSec isnt just loss prevention –Value comes from the ability to support profitable risk taking e.g. Brakes, condoms –Risk balancing is a reflexive process involving perceptions of risk and reward Varies dramatically by industry and sector –E.g. a bank vs. a rock quarry *From the viewpoint of business value

27 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 27 Blind Alleys and Dirt Roads Blind Alleys look good in concept, but wont work by themselves –Return on Investment (ROI), Net Present Value (NPV), Payback, etc. –Annualized Loss Expectancy (ALE) –Cyber insurance –Product liability and tort laws (actual damages) Dirt Roads work, but just barely –2x2 or 3x3 matrix categorization of incident types or risks by frequency vs. severity –Assessments using scoring and ranking systems –Balanced scorecards –Strategic scenario analysis and walkthroughs Are there any Autobahn approaches out there? –The null / realist hypothesis is no, assuming insurmountable problems –Total Cost of (In)security might be such an approach

28 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 28 Why ALE is Dumb A Simple Case of Three Loss Event Categories * –Firm Equity = $50 million; Annual Earnings = $5 million; ROE = 10% –Category A: Common flood 50% chance of $10,000 loss = $5,000 ALE –Category B: 100 year flood 1.0% chance of $500,000 loss [10% of earnings, 1% of equity] = $5,000 ALE 26% chance of happening at least once in 30 years –Category C: 10,000 year flood 0.01% chance of $50 million loss [100% of equity] = $5,000 ALE Reason 1: ALE math hides risk drivers –A+B+C = A+A+A = B+B+B = C+C+C = $15,000 ALE [1.5% of earnings] –Conflates simple random walks with random walks with avalanches Three independent common risks = three independent catastrophic risks Reason 2: Unreliable estimates of low probability events dominate –Lack of data + psychology means estimation errors for the tail are much higher 50% 55% chance for A $5,250 ALE 1.0% % chance for B $10,000 ALE (45% chance in 30 years!) 0.01% 0.05% chance for C $25,000 ALE $40,250 ALE (2.7 times larger!) * Pareto Distribution, k=1, min = 5,000

29 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 29 Finite Risk Programs Year 1 Fund established $$$ Operational losses Interest paid Balance carry-forward The insurance industry offers multi-year self-insurance plans that are commonly called finite risk insurance. The name arises from the fact that the risk transfer is very limited. Therefore, the insured will pay for most (or all) the losses From: Applying Insurance Modeling Techniques to Quantify OR Dr Marcelo Cruz, RiskMaths, presented at GARP OR Seminar October 2001 London time

30 © 2007 Meritology. All Rights Reserved Total Cost of Cyber (In)security Mini-Metricon, San Francisco - Feb 5, 2007 Page 30 Ruin Theory applied to Finite Risk Initial Finite Risk capital Percentage of gross income allocated against Finite Risk Losses following a certain stochastic process Finite Risk hedging needs ruin From: Applying Insurance Modeling Techniques to Quantify OR Dr Marcelo Cruz, RiskMaths, presented at GARP OR Seminar October 2001 London


Download ppt "Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron."

Similar presentations


Ads by Google