Qualitative Risk Assessment Risk Analysis for Water Resources Planning and Management Institute for Water Resources 2008

Risk Assessment What can go wrong? How can it happen? How likely is it? What are the consequences?

Want to Improve Your Risk Analysis? Use simple narratives that answer these questions honestly Tell story of existing risk Tell story of residual or transformed risk

The Need Manage risk intentionally Do better than has been done Quantitative risk assessment not always possible or necessary Qualitative risk assessment can be a viable option

Qualitative Risk Assessment Is formal, organized, reproducible method based on science and sound evidence Flexible Easy to explain to others Supports risk management decision making

Three Sample Methods Enhanced Criteria Ranking Operational Risk Management (Risk Matrix) CARVER + Shock

Enhanced Criteria Based Ranking Criteria Ratings All Possible Combinations of Ratings Ranking Evaluate Reasonableness of Ranking Add Criteria New Combinations of Ratings New Ranking

Question? Which lock gates in division present the greatest potential risk to health and safety and therefore should be repaired first?

Step One: Criteria Assume criteria equally important (or not). Reflect most important aspects of evaluating risk. Define H, M, L scenarios for each criterion. Use three or four evidence-based criteria.

Step Two: Rating Use expert judgment to critically evaluate the available information Develop estimates for each “hazard” against criteria Use letters or numbers but numbers do not represent an absolute measurement of risk only a relative means for comparison

Step Three: All Possible Combinations Greatest RiskHHH HHM, HMH, MHH HHL, HLH, LHH, HMM, MMH, MHM HLM, MHL, HML, LMH, MLH, MMM, LHM HLL, LHL, LLH, MML, LMM, MLM MLL, LML, LLM Least RiskLLL This is for equally weighted criteria. Unequal weights yield different listings.

Step Four: Rank Subjectively Establish rank according to descending relative risk Identify subjective clusters.

Step Five: Add Criteria? Look at rankings, do they make sense? Have you thought properly about this issue? If they do not, perhaps you did not consider all the most relevant criteria A new criteria may be added to more accurately reflect the assessors rationale for ranking

Step Five: Add Criteria? (cont) Suppose the following was added to our example Criterion 4: Cost of emergency repair H = Major disruptions to navigation or power, much higher costs to repair M = Much higher costs to repair L = Same as scheduled repair

Step Six: New Ratings

Step Seven: New Ranking

Operational Risk Management (ORM)

ORM AKA the risk matrix Risk ranking tool Uses ranges of consequence and likelihood Combinations created enable assessors to qualitatively estimate a risk

Steps Determine purpose and use of matrix Identify the question to be answered Define consequences of interest Identify consequence ranges and definitions Identify likelihood ranges and definitions Identify levels of risk in the cells of the matrix

Your DE Has Seen This “Mishap Risk” DOD "Standard Practice For System Safety” MIL-STD-882D 10 February 2000

Consequence Severities

Probability Levels

Risk Assessment Values Each risk you assess is placed in a cell and managed accordingly

Risk Levels

Another Example Source: Assessing Environmental Risk, A Lecture to the Irish Environmental Law Association By: L. M. Ó Cléirigh 29 June 2004

Risk Matrix

Three Axioms Weak consistency Betweenness Consistent coloring 3x3 and 4x4 should look like this to minimize problems Source: What’s wrong with risk Matrices? By Louis Anthony Cox, Risk Analysis Vol. 28 No.2, 2008

The Risk Management Point of Matrix

CARVER + Shock Vulnerability assessment method developed for Department of Defense CARVER is an acronym Criticality - measure of public health and economic impacts of an attack Accessibility – ability to physically access and egress from target Recuperability – ability of system to recover from an attack Vulnerability – ease of accomplishing attack Effect – amount of direct loss from an attack as measured by loss in production Recognizability – ease of identifying target

SHOCK Technique modified to include seventh attribute that combines health, economic, and psychological impacts of an attack SHOCK attributes of target

Select a Process Identify a critical process or infrastructure and assess vulnerability across nation, or Assess vulnerability of components of a single process

Critical A target is critical when “loss” would have significant life, health or economic impacts

Accessibility A target is accessible when an “attacker” can reach it to conduct the attack and then escape the target undetected

Recuperability The time it will take for the specific facility to recover productivity is the target’s recuperability

Vulnerability Vulnerability measures the ease with which sufficient quantities of threat agents can be introduced to achieve the attacker’s purpose once the target has been accessed

Effect Effect is the percentage of system productivity damaged by an attack at a single facility

Recognizability Recognizability is the extent to which the target can be identified by an attacker without confusing it with other targets or components

Shock Shock combines the measure of the health, psychological, and collateral national economic impacts of a successful attack on the target system

Sandia Labs User friendly software has been developed for food defense by FDA and Sandia Process diagrams Interviews Results

Take Away Points Not all risk assessment needs to be quantitative Develop a few consistent and well developed techniques for your usage