1. Qualitative Risk Assessment Risk Analysis for Water Resources Planning and Management Institute for Water Resources 2008

2. Risk Assessment What can go wrong? How can it happen? How likely is it? What are the consequences?

3. Want to Improve Your Risk Analysis? Use simple narratives that answer these questions honestly Tell story of existing risk Tell story of residual or transformed risk

4. The Need Manage risk intentionally Do better than has been done Quantitative risk assessment not always possible or necessary Qualitative risk assessment can be a viable option

5. Qualitative Risk Assessment Is formal, organized, reproducible method based on science and sound evidence Flexible Easy to explain to others Supports risk management decision making

6. Three Sample Methods Enhanced Criteria Ranking Operational Risk Management (Risk Matrix) CARVER + Shock

7. Enhanced Criteria Based Ranking Criteria Ratings All Possible Combinations of Ratings Ranking Evaluate Reasonableness of Ranking Add Criteria New Combinations of Ratings New Ranking

8. Question? Which lock gates in division present the greatest potential risk to health and safety and therefore should be repaired first?

9. Step One: Criteria Assume criteria equally important (or not). Reflect most important aspects of evaluating risk. Define H, M, L scenarios for each criterion. Use three or four evidence-based criteria.

10. Step Two: Rating Use expert judgment to critically evaluate the available information Develop estimates for each “hazard” against criteria Use letters or numbers but numbers do not represent an absolute measurement of risk only a relative means for comparison

11. Step Three: All Possible Combinations Greatest RiskHHH HHM, HMH, MHH HHL, HLH, LHH, HMM, MMH, MHM HLM, MHL, HML, LMH, MLH, MMM, LHM HLL, LHL, LLH, MML, LMM, MLM MLL, LML, LLM Least RiskLLL This is for equally weighted criteria. Unequal weights yield different listings.

12. Step Four: Rank Subjectively Establish rank according to descending relative risk Identify subjective clusters.

13. Step Five: Add Criteria? Look at rankings, do they make sense? Have you thought properly about this issue? If they do not, perhaps you did not consider all the most relevant criteria A new criteria may be added to more accurately reflect the assessors rationale for ranking

14. Step Five: Add Criteria? (cont) Suppose the following was added to our example Criterion 4: Cost of emergency repair H = Major disruptions to navigation or power, much higher costs to repair M = Much higher costs to repair L = Same as scheduled repair

15. Step Six: New Ratings

16. Step Seven: New Ranking

17. Operational Risk Management (ORM)

18. ORM AKA the risk matrix Risk ranking tool Uses ranges of consequence and likelihood Combinations created enable assessors to qualitatively estimate a risk

19. Steps Determine purpose and use of matrix Identify the question to be answered Define consequences of interest Identify consequence ranges and definitions Identify likelihood ranges and definitions Identify levels of risk in the cells of the matrix

20. Your DE Has Seen This “Mishap Risk” DOD "Standard Practice For System Safety” MIL-STD-882D 10 February 2000

21. Consequence Severities

22. Probability Levels

23. Risk Assessment Values Each risk you assess is placed in a cell and managed accordingly

24. Risk Levels

25. Another Example Source: Assessing Environmental Risk, A Lecture to the Irish Environmental Law Association By: L. M. Ó Cléirigh 29 June 2004

26. Risk Matrix

27. Three Axioms Weak consistency Betweenness Consistent coloring 3x3 and 4x4 should look like this to minimize problems Source: What’s wrong with risk Matrices? By Louis Anthony Cox, Risk Analysis Vol. 28 No.2, 2008

28. The Risk Management Point of Matrix

29. CARVER + Shock Vulnerability assessment method developed for Department of Defense CARVER is an acronym Criticality - measure of public health and economic impacts of an attack Accessibility – ability to physically access and egress from target Recuperability – ability of system to recover from an attack Vulnerability – ease of accomplishing attack Effect – amount of direct loss from an attack as measured by loss in production Recognizability – ease of identifying target

30. SHOCK Technique modified to include seventh attribute that combines health, economic, and psychological impacts of an attack SHOCK attributes of target

31. Select a Process Identify a critical process or infrastructure and assess vulnerability across nation, or Assess vulnerability of components of a single process

32. Critical A target is critical when “loss” would have significant life, health or economic impacts

33. Accessibility A target is accessible when an “attacker” can reach it to conduct the attack and then escape the target undetected

34. Recuperability The time it will take for the specific facility to recover productivity is the target’s recuperability

35. Vulnerability Vulnerability measures the ease with which sufficient quantities of threat agents can be introduced to achieve the attacker’s purpose once the target has been accessed

36. Effect Effect is the percentage of system productivity damaged by an attack at a single facility

37. Recognizability Recognizability is the extent to which the target can be identified by an attacker without confusing it with other targets or components

38. Shock Shock combines the measure of the health, psychological, and collateral national economic impacts of a successful attack on the target system

39. Sandia Labs User friendly software has been developed for food defense by FDA and Sandia http://www.cfsan.fda.gov/~dms/carver.html Process diagrams Interviews Results

40. Take Away Points Not all risk assessment needs to be quantitative Develop a few consistent and well developed techniques for your usage