CS 589 Information Risk Management 30 January 2007

References for Today The Gaffney and Ulvila paper Gelman, Carlin, Stern, and Rubin, Bayesian Data Analysis. Chapman and Hall, Any decent statistics book that deals with Bayesian ideas

Today Continue with our introductory example Discuss the paper Specify Risks Using Probability Distributions –Event Risk(s) –Outcome Risk(s) Use our Risk Models in decision tree(s)

What do we need to know? Probabilities –P(Detection|An Intrusion)  P(D|I) –Associated Info –P(I) –And, finally, P(I|D) Outcomes –Individually, these will not be stochastic – for now –They will still lead to an expectation for each decision node

Conditional Probability P(D|I) and P(D| Not I) P(Not D|I) and P(Not D|Not I) Where would we get this information? What about P(I)?

Bayes Rule – Simple Version

Bayes’ Rule Prior Idea  Information  Revised Idea  Information  New Revision  etc. Specific form for computing new probabilities Where does the prior come from? Information?

Interpretation Two types of Accuracy Two types of Error

Solving the Tree Establish the Outcomes Compute the Probabilities – the conditionals on the endpoints and others Find Expected Values and roll back the tree

Outcomes These will often have to be estimated We can follow the paper (and some convention) and incorporate costs for false positives and false negatives Where would we get these numbers in an actual analysis? What did the authors (of the paper) do?

Sensitivity Analysis What are the strategies given the numbers we used in the example? What are the key variables? How would we assess the base-case outcome of this example?

Don’t Forget the Bottom Line Our decision criterion – Expected Cost We want to see what happens under different circumstances We can even show these situations – scenarios – graphically Relationships between cost ratio and P(D|I) and P(D not|I not)

Different Conditional Information What if we don’t know P(D|I)? We can flip the tree according to what we do know Outcomes should remain the same And the decision should remain the same

Another Way – Info Dependent

Modeling Decisions, chance events Probability distributions for chance events –Lack of data  Bayesian methods –Expert(s) –Lots of data  Distribution model(s) Outcomes –Financial, if possible –Multiple measures/criteria/attributes

Decision Situation In the context of Firm or Organization Goals, Objectives, Strategies A complete understanding should lead to a 1-2 sentence Problem Definition –Could be risk-centered –Could be oriented toward larger info issues Problem Definition should drive the selection of Alternatives and, to some degree, how they are evaluated

Information Business Issues Integrity and reliability of information stored and used in systems Preserve privacy and confidentiality Enhance availability of other information systems

Risk Management Process of defining and measuring or assessing risk and developing strategies to mitigate or minimize the risk Defining and assessing –Data driven –Other sources Developing strategies –Done in context of objectives, goals

What About the Paper? What is the problem/model structure? What are the key risks? How were they modeled? What do the authors say about the structure of the problem? What conclusions do you draw from the paper?

Probability Models For events –Discrete Events –Poisson Distribution –Binomial Distribution For outcomes Combine with Bayesian analysis

Poisson Distribution

Independent events happen along a continuum –Time – often the case in IA –Space –Distance Distribution parameter is the expected number of events per unit Parameter is also the distribution variance One possible model for discrete data

Using the Poisson in IRM Probability model of events per unit of measurement Expected number of events might impact technology choice Range of potential events might impact tech choice If we are risk averse, distributions help us explicitly consider event risks

What if we don’t have data? One way: –Select the model –Model parameters estimated using a prior distribution –Pick the right Prior Prior Expectation  Distribution Parameter Prior distribution should reflect degree of uncertainty

Priors Prior for a Poisson parameter is Gamma

Prior Distribution The prior should reflect our degree of certainty, or degree of belief, about the parameter we are estimating One way to deal with this is to consider distribution fractiles Use fractiles to help us develop the distribution that reflects the synthesis of what we know and what we believe

Prior + Information As we collect information, we can update our prior distribution and get a – we hope – more informative posterior distribution Recall what the distribution is for – in this case, a view of our parameter of interest The posterior mean is now the estimate for the Poisson lambda, and can be used in decision- making

Information For our Poisson parameter, information might consist of data similar to what we already collected in our example We update the Gamma, take the mean, and that’s our new estimate for the average occurrences of the event per unit of measurement.

Next Time You guys will present your responses to the homework assignment We will continue with Bayesian analysis and another example or two Segue into other topics given our framework that we have developed