1. Qualitative Risk Assessment
Institute for Water Resources
2009
Qualitative Risk Assessment
Charles Yoe, PhD

2. The Need Manage risk intentionally Do better than has been done
Qualitative risk assessment can be a viable option
Partial assessments sometimes helpful (not all steps)

3. Use Qualitative Assessment
When consistency and transparency in handling risk are desired
When theory, data, time or expertise are limited
When dealing with broadly defined problems where quantitative risk assessment is impractical
E.g., national levee safety ranking

4. Qualitative Risk Assessment
The process of compiling, combining and presenting evidence to support a statement about risk
Descriptive or categorical treatment of information
Is formal, organized, reproducible method based on science, sound evidence and four steps
Flexible and consistent
Easy to explain to others
Supports risk management decision making

5. Qualitative Methods Toolbox
Increase or Decrease Risk
Risk Narratives
Evidence Mapping
Screening
Ratings
Rankings
Enhanced Criteria Ranking
Operational Risk Management (Risk Matrix)
Develop a Generic Process
Qualitative Assessment Models
Multi-Criteria Decision Analysis

6. Qualitative Assessment
May include all or just some of the risk assessment steps
Qualitative risk characterization is usually the endpoint
Not every qualitative assessment is a true risk assessment

7. For Any Method Necessary preparation! Use an assessment framework
Identify the problem
Identify the goals
Identify the questions to be answered
Use an assessment framework

8. Increase or Decrease Risk
For some problems it may be enough to know if things are getting better or worse
Identify the direction of change in a risk and the specific reasons for it
Storm damage has weakened structure
Funding uncertainty clouds future
Clarifies thinking and rationale
Not good for netting changes

9. Risk Narratives What can go wrong? How can it happen?
How likely is it?
What are the consequences?
Use simple narratives that answer these questions honestly
Tell story of existing risk
Tell story of mitigation effectiveness (risk reduction)
Tell story of residual, transferred or transformed risk

10. Evidence Frameworks A risk evaluation technique
Identify how experts evaluate current scientific evidence on chosen topics
What conclusions do they reach regarding risk potential
What evidence/arguments do they use to justify conclusions
What consensus/disagreement exists
What uncertainties remain
Source: Risk evaluation of the health effects of mobile phone communication (10/2005) by Peter Wiedermann, Holger Schütz, and Albena Spangenberg

11. Core Elements Evidence base or data Pro and con arguments
Includes supporting or attenuating arguments
Conclusions of claim about existence of a hazard with remaining uncertainties

13. Ordering Techniques
Screening, rating and ranking with increasing levels of detail/information
Used to identify hazards, commodities, commodity-pathogen pairs, pathways, mitigation measures, potential risk and the like that are of interest to decision makers

14. Screening
Process of separating elements into categories of interest and no interest through systematic elimination
Requires
Items to be screened
Carefully defined categories (yes/no)
Criteria for screening
Evidence for the criteria
An algorithm for using the criteria to separate the items into the desired categories

15. Screening Algorithms
Domination procedures (better/worse on all criteria)
Conjunctive procedures (meets all criteria thresholds)
Disjunctive procedures (meets a least one criterion threshold)
Elimination by aspects (set cut-off value for most important criterion and eliminate, then set cut-off value for next most important criterion, etc.)
Lexicographic rules (rank against all criteria then rank alternatives)

16. Rating
Systematic process of separating elements into multiple categories of varying degrees of interest
Individuals are rated
High, medium, low, no risk
Requires
Items to be rated
Carefully defined categories (non-ordinal is okay)
Criteria for rating
Evidence for the categories
An algorithm for using the criteria to separate the items into the desired categories

17. Ranking Systematic process used to put items in an ordinal sequence
Rated items can be ranked
May rely on ordinal ranked categories or an ordinal ranking of each individual item
Simple when objective measures  of a risk or other characteristic of interest are available
Requires
Items to be ranked (alternatives)
Carefully defined science-based criteria for ranking
Evidence of each item’s measurement or rating for each criterion
Differential weights for criteria when appropriate
A synthesis algorithm.

18. Enhanced Criteria Based Ranking
Ratings
All Possible Combinations of Ratings
Ranking
Evaluate Reasonableness of Ranking
Add Criteria
New Combinations of Ratings
New Ranking

19. Question?
Which lock gates in division present the greatest potential risk to health and safety and therefore should be repaired first?

20. Step One: Criteria Assume criteria equally important (or not).
Reflect most important aspects of evaluating risk.
Define H, M, L scenarios for each criterion.
Use three or four evidence-based criteria.

21. Step Two: Rating
Use expert judgment to critically evaluate the available information
Develop estimates for each “hazard” against criteria
Use letters or numbers but numbers do not represent an absolute measurement of risk only a relative means for comparison

22. Step Three: All Possible Combinations
Greatest Risk HHH HHM, HMH, MHH HHL, HLH, LHH, HMM, MMH, MHM HLM, MHL, HML, LMH, MLH, MMM, LHM HLL, LHL, LLH, MML, LMM, MLM MLL, LML, LLM Least Risk LLL
This is for equally weighted criteria. Unequal weights
yield different listings.

23. Step Four: Rank Subjectively
Establish rank according to descending relative risk
Identify subjective clusters.

24. Step Five: Add Criteria?
Look at rankings, do they make sense?
Have you thought properly about this issue?
If they do not, perhaps you did not consider all the most relevant criteria
A new criteria may be added to more accurately reflect the assessors rationale for ranking

25. Step Five: Add Criteria? (cont)
Suppose the following was added to our example
Criterion 4: Cost of emergency repair
H = Major disruptions to navigation or power, much higher costs to repair
M = Much higher costs to repair
L = Same as scheduled repair

26. Step Six: New Ratings

27. Step Seven: New Ranking

28. You are working a on a beneficial uses of dredged materials study that is using good
quality navigation maintenance dredging material to create wetlands in a wildlife refuge.
Over the last several decades 7000 acres of coastal wetlands have been lost.
You have limited budgets and no money for engineering studies. The risk of greatest concern is
that the wetlands will not be established. Your task is to decide where you will place the material
to establish wetlands. You will use hay bales to establish a containment structure adjacent to the
shoreline, then you will pump materials behind the bales, the emergent land will then be planted.
You will be ranking 20 different sites and you want to identify those sites with the greatest risk of
failure or, alternatively, with the greatest chance of success.
Using the methodology of the enhanced criteria hazard ranking method develop three criteria that
will help you to identify the most hazardous locations for these wetlands. Then define a high,
medium and low scenario for each of these criteria

29. Operational Risk Management (ORM)

30. Steps Determine purpose and use of matrix
Identify the question to be answered
Define consequences of interest
Identify consequence ranges and definitions
Identify likelihood ranges and definitions
Identify levels of risk in the cells of the matrix

31. Your DE Has Seen This “Mishap Risk”
DOD "Standard Practice For System Safety”
MIL-STD-882D 10 February 2000

32. Consequence Severities

33. Probability Levels

34. Risk Assessment Values
Each risk you assess is placed in a cell and managed accordingly

35. Risk Levels

36. Another Example
Source: Assessing Environmental Risk, A Lecture to the Irish
Environmental Law Association By: L. M. Ó Cléirigh 29
June 2004

37. Risk Matrix

38. Three Axioms Weak consistency Betweenness Consistent coloring
3x3 and 4x4 should look like this to minimize problems
Source: What’s wrong with risk Matrices? By Louis Anthony Cox, Risk Analysis Vol. 28 No.2, 2008

39. The Risk Management Point of Matrix

40. Recently, Congress passed the National Levee Safety Act, which for the first time directed the Corps
to inventory all private levees. In this case we have a programmatic level risk assessment and
management problem and we will never have detailed data and analysis for a full blown risk
assessment at every location.
Given that the levee assessment program is in its infancy, and given the reality of limited data over
time, use the ORM approach to devise probability and consequence categories that could be used
for an initial qualitative risk assessment of these levees once they are identified.

41. Generic Process Probability of Adverse Impact Consequence of
Risk = X
Age x Condition x Usage x Event
Economic Environmental Perceived
Damage Damage Damage
Potential Potential + +

42. Qualitative Assessment Models
Probability hazard exists-rank as H,M,L
Probability adverse consequence occurs if exposed-rank as H,M,L
Probability exposure occurs-rank as H,M,L
Overall risk estimate, integrate hazard, consequence, exposure- rank as H,M,L

43. MCDM/MCDA
Multi-Criteria Decision Making (MCDM) is the study of methods and procedures by which concerns about multiple conflicting criteria can be formally incorporated into the management planning process
Decision maker contemplates choice of action in an uncertain environment
MCDA helps people choose among a set of pre-specified alternatives
Decision making relies on information about these alternatives
Quality of information can be scientifically-derived hard data to subjective interpretations
Outcomes of decisions may be certain (deterministic information) or uncertain represented by probabilities and fuzzy numbers
MCDA can assist in information processing and may lead to better decisions

49. Take Away Points Not all risk assessment needs to be quantitative
Develop a few consistent techniques for your usage

50. Questions?
Charles Yoe, Ph.D.