CS 589 Information Risk Management 23 January 2007

Today’s Discussion Start with risk Discuss types of information risk Start with systematic, modeling-based framework for assessing alternatives when risks are known Continue with the hard part – specification of risk when risks are unknown

Next Week Discuss specification of risks using probability distributions Discuss incorporation of this information into a decision tree Discuss ways to apply these techniques to Information Risk scenarios

After Next Week Discuss the Expected Utility decision criterion Discuss Multiple Objectives and Expected Value and Expected Utility Discuss Applications in Information Risk Analysis and Management

References for Today Clemen, R. L. and T. Reilly, Making Hard Decisions. Duxbury, Gaffney Jr., J. E., J. W. Ulvila, “Evaluation of Intrusion Detectors: A Decision Theory Approach”, Proceedings of the IEEE Symposium on Security and Privacy

Risk ??? Chance of something bad happening? Having something bad happen? Anything else?

Risk The probability of an event occurring combined with the consequences of that event Just about everything is risky How do we actually measure risk?

Risk vs Uncertainty Uncertainty –We don’t know what the key variables are –We don’t know how they relate to alternatives Risk –Specify probability distributions –Connect them with alternatives One goal: Uncertainty  Risk via Modeling

Thinking About Risk Probabilities and Outcomes Which is riskier? –Living near a large power generation station –International flight –Driving to Albuquerque We have to define factors, events, outcomes, and associated probabilities

Dealing with Risk Define Risk Assess Risk Define Alternatives for Handling the Risk Evaluate Alternatives Evaluate your Evaluation Model Sensitivity Analysis Implementation

Evaluation Choosing among Alternatives Should be Evaluated on the same dimension(s) –Expected Value –Expected Utility –Value at Risk (VAR) –Multiple criteria Measurement of Alternatives on criteria dimensions is key – and another modeling issue

Sensitivity Analysis Checking on the evaluation of each alternative by varying individual variables Find the variable(s) that have the largest impact(s) on the ordering of alternatives Goal: robust solutions

Visual Representation Influence Diagrams –Connect factors, events –Help us define risks –Decomposition Decision Trees –Ordering of decisions, risky events –Easy to see and present – and solve

Visual Representations Squares denote Decisions Circles denote Risks Influence Diagrams – arcs connect decision and risk (aka chance) nodes Decision Trees – decision and chance nodes are sequentially ordered from left to right

A Very Simple Example Coin Flip Game Decisions: Play/No Play Risks: Heads/Tails Outcomes Must be Specified

Coin Flip Game Decision Tree With $0 Outcomes

If All Outcomes are $0 We are Indifferent between Play and No Play based on the Expected Value criterion We Prefer Play to No Play if E(Play) > E(No Play) Which means that the sum of the outcomes (if we have a fair coin) must be positive Generally, Play if

What if we can play twice? Sequential decision – we see the result of the first coin flip, and decide to continue This leads to the notion of Strategies – we can make a plan contingent upon resolution of risks that are resolved between decision nodes Everything is still based on Expected Value

Suppose O(H) = $10, O(T) = -$7 p(H) = p(T) =.5 (Fair coin) We can easily see that we would choose to Play in the one-game case What about the 2-game case?

Strategy It’s pretty simple – keep playing Would you really do this? Do you believe this? Why or why not??

Simple Example Suppose we are assessing two alternative intrusion detection systems. What’s the problem? What are the key risks for this decision? What are the decisions? What are the outcomes? How would we measure the outcomes? What is the decision criterion?

Key Point The optimal choice will be the one that is associated with the best expected criterion value – such as expected total cost This will be determined by how we define the outcomes – in terms of total costs – and probabilities When we roll back a decision tree, we assume that the downstream decision is the best one

Expected Value Random Variable with possible discrete outcomes

What do we need to know? Probabilities –P(Detection|An Intrusion)  P(D|I) –Associated Info –P(I) –And, finally, P(I|D) Outcomes –Individually, these will not be stochastic – for now –They will still lead to an expectation for each decision node

Conditional Probability P(D|I) and P(D| Not I) P(Not D|I) and P(Not D|Not I) Where would we get this information? What about P(I)?

Bayes Rule – Simple Version

Interpretation Two types of Accuracy Two types of Error

Solving the Tree Establish the Outcomes Compute the Probabilities – the conditionals on the endpoints and others Find Expected Values and roll back the tree

Sensitivity Analysis What are the strategies given the numbers we used in the example? What are the key variables? How would we assess the base-case outcome of this example?

Different Conditional Information What if we don’t know P(D|I)? We can flip the tree according to what we do know Outcomes should remain the same And the decision should remain the same

Another Way – Info Dependent

Modeling Decisions, chance events Probability distributions for chance events –Lack of data  Bayesian methods –Expert(s) –Lots of data  Distribution model(s) Outcomes –Financial, if possible –Multiple measures/criteria/attributes

Decision Situation In the context of Firm or Organization Goals, Objectives, Strategies A complete understanding should lead to a 1-2 sentence Problem Definition –Could be risk-centered –Could be oriented toward larger info issues Problem Definition should drive the selection of Alternatives and, to some degree, how they are evaluated

Information Business Issues Integrity and reliability of information stored and used in systems Preserve privacy and confidentiality Enhance availability of other information systems

Risk Management Process of defining and measuring or assessing risk and developing strategies to mitigate or minimize the risk Defining and assessing –Data driven –Other sources Developing strategies –Done in context of objectives, goals