1. Nature of Failures, Events, and terminology
Derek Brown and Peter Dunscombe
University of Calgary, Alberta

2. Disclosures Derek - Founding partner of TreatSafely, LLC.
Peter – Founding partner of TreatSafely, LLC.

3. An initial thought A couple of interesting quotes
I am very much afraid of definitions, and yet one is almost forced to make them. One must take care, too, not to be inhibited by them.
- Robert Delaunay (French artist, )
Let’s apply this philosophy to Quality and Safety improvement efforts in Radiation Oncology.
- Derek Brown (Canadian fisherman, 1976-…)

4. Learning Objectives
To review some commonly used definitions in the field
To review Rasmussen’s categories of human performance.
To discuss Reason and Hobbs’ failure categories
To look at how performance might be compromised, with clinical examples.
To map failure categories on to human performance categories.

5. A scenario to consider Patient is half-way through treatment
RO decides that the patient’s treatment be stopped until further notice
It’s a busy day on the unit and staff take the message, but neglect to do anything with the information
The patient shows up at the unit the next day and is treated as usual
This goes on for a week until someone notices

6. A scenario to consider Two questions… Is this an error, incident,
adverse event?

7. AAPM Consensus Recommendations for Incident Learning
Error - Failure to complete a planned action as intended
Incident - An unwanted or unexpected change from a normal system behavior which causes or has the potential to cause an adverse effect to persons or equipment
Adverse Event - An incident that occurs during the process of providing health care that results in suboptimal clinical outcome including unintended injury or complication leading to disability, death, or prolonged hospital stay for the patient

8. A scenario to consider Two questions… Is this an error, incident,
adverse event?
Does it matter?

9. What about quality?
The Institute of Medicine defines quality in health care as follows – “quality care is safe, effective, patient centered, timely, efficient, and equitable”
Let’s take the terms error, incident, and adverse event to mean any deviation from a process path that compromises, or has the potential to compromise, quality in health care, as defined above
Thoughts?

10. The Institute of Medicine defines quality as follows…
Quality care is safe, effective, patient centered, timely, efficient, and equitable
Quality care is the prevention of adverse events that impact the patient
Quality care is defined by the patient, not by the Institute of Medicine
Quality care is the provision of the best care possible given social and economic factors
Quality care is the provision of edible meals during a hospital stay 10

11. The Institute of Medicine defines quality as follows… - ANSWER
1. Quality care is safe, effective, patient centered, timely, efficient, and equitable

12. Performance Categories
Jens Rasmussen defined three categories of human performance:
Skill-based
Rule-based
Knowledge-based
Note: most activities encompass more than one level of performance

13. Skill-based performance
Applies to straightforward routine tasks which have been performed for some time. May or may not include checks along the way.
Documentation rarely needs to be referred to.
Skill can be increased through repetition.
Examples of predominantly skill based activities for the experienced practitioner:
Morning warm up on a machine.
Physics assistant monthly linac QC.

14. Rule-based performance
Applies to more complex or critical tasks which may be only occasionally performed.
Documentation (procedures, instruction manuals, protocols) need to be readily accessible.
Regulated practices require rules.
Examples of predominantly rule based activities:
Adjusting lasers.
Working up a patient for a clinical trial.
Radiation Safety.

15. Knowledge-based performance
Applies to dealing with unfamiliar and/or unprepared- for tasks. The “rules” have to be made up and the “skills” developed during performance of the task.
Prior specific documentation is not available.
Uses more education than training.
Examples of predominantly knowledge based activities:
Commissioning a new treatment technique for TBI.
Contouring on 4D-CT.

16. Performance categories
Complex
Knowledge
Rule
Skill
Relative Complexity
Straightforward
Rare
Frequent
Relative Frequency

17. Which of the following is one of Rasmussen’s performance categories?
Companion-based performance
Knowledge-based performance
Literature-based performance
Coordination-based performance
Aptitude-based performance 10

18. Which of the following is one of Rasmussen’s performance categories
Which of the following is one of Rasmussen’s performance categories? - ANSWER
2. Knowledge-based performance

19. Human Error
“The three performance levels form a basis for categorizing the varieties of human error.” – Reason and Hobbs (Managing Maintenance Error)
So what is human error?

20. Discussion Point Patient is overdosed by 15%
After an extensive, and costly, Root Cause Analysis, the conclusion is that human error is to blame
You’re the CEO of the organization. What are you going to do to ensure that this does not happen again?

21. Discussion Point Human error is NOT a useful root cause
“Careful analysis of these kinds of incidents usually gives the lie to such a story.
“… human error resulted from bad design”
- Norman (The Psychology of Everyday Things)

22. How might our performance be sub-optimal?
Definitions:
Error: Actions do not go as planned
Mistake: Actions go as planned but plan is flawed
Violations: Intentional deviation from approved path

23. Error – Expanded definition
An error is the failure of planned actions to achieve their desired goal, where this occurs without some unforeseeable or chance intervention.

24. Skill – Based Errors Recognition failures (1): Misidentification
Some examples are:
Laterality errors in patient treatment
Mistaking a mole for a tattoo
Setting the wrong scale on an electrometer

25. Skill – Based Errors Recognition failures (2): Non-detection
Some examples are:
Focusing on complex calculations and missing simple errors
Entering the time instead of the pressure into the output program
Failure to observe metastasis on a CT.

26. Skill – Based Errors Slips / (Memory) lapses:
A step is missed in a frequently performed routine activity.
Some examples are:
Not pressing the Last Person Out button
Omitting to set the electrometer zero between readings
Letting the patient leave the consult before signing the approval sheet.

27. Rule – Based Error Misapplying a good rule:
Using an inappropriate method or data
Some examples are:
Doing an SSD calculation for an SAD patient.
Using a hard wedge factor for a dynamic wedge.

28. Rule – Based Mistake Applying a bad rule:
Following a tradition (an unwritten rule)
Some examples are:
Completing the prescription sheet after the Oncologist has signed it.
Clearing computer warnings automatically.
Ignoring Mrs Smith’s medical complaints because she is always complaining.

29. Knowledge – Based Mistakes
Tackling unfamiliar problems:
Some examples are:
Treating the first IGRT patient
Commissioning a new TBI technique
Prescribing to a new (for you) tumor site.

30. Knowledge – Based Mistakes
Reason and Hobbs asked aircraft maintenance personnel the following question: At work in the last year or so, how often have you done an unfamiliar job, despite being uncertain whether you were doing it correctly? The answer was about 20% of the time!! Would our experience be any different?

31. Violations Routine violations:
Showing off, taking short cuts that are not in the written procedure, persistent carelessness.
Some examples are:
When setting up a phantom not checking both the ODI and the lasers
Not checking the patient’s ID properly before taking them in the room.
Not informing the unit of a cancelled fraction

32. Violations Thrill seeking violations: Taking a risk for the sake of it
Some examples are:
Exceeding the speed limit without good reason.
Skiing out of bounds.

33. Violations Situational violations:
A pragmatic approach to getting the job done.
Some examples are:
Signing purchase orders without reading them.
Not doing the full morning check so as not to delay patient treatments
Double booking patients on a machine

34. Mapping Performance Levels to Error Types
Skill
Errors
Rule
Mistakes
Knowledge
Violations

35. Memory lapses and slips relate to which type of human performance?
Aptitude-based performance
Knowledge-based performance
Rule-based performance
Skill-based performance
Slips can’t be related to a type of human performance 10

36. Memory lapses and slips relate to which type of human performance
Memory lapses and slips relate to which type of human performance? - ANSWER
4. Skill-based performance

37. Is There Another Way of Classifying Errors?
Sporadic:
An error that happens once is not likely to occur at the same place in the process again.
Systematic:
The same error will occur under the same set of circumstances

38. Mapping Error Categories to Error Types
Skill - recognition
Skill - misidentification
Slips
Sporadic
Lapses
Rule - oversight
Rule - misunderstanding
Systematic
Knowledge
(Violations)

39. Nature of Failures Summary
We’ve spent too much time discussing terminology
We have reviewed Rasmussen’s categories of human performance.
We have looked at how performance might be compromised, with clinical examples.
We have mapped error types on to human performance categories.