Presentation is loading. Please wait.

Presentation is loading. Please wait.

Integration of predictive and retrospective risk analysis in health care Tjerk van der Schaaf Leiden University Medical Center Eindhoven University.

Similar presentations

Presentation on theme: "Integration of predictive and retrospective risk analysis in health care Tjerk van der Schaaf Leiden University Medical Center Eindhoven University."— Presentation transcript:

1 Integration of predictive and retrospective risk analysis in health care Tjerk van der Schaaf Leiden University Medical Center Eindhoven University of Technology

2 overview retrospective method: PRISMA-medical
predictive method : HFMEA 3 examples of possible integration direct comparison of predicted vs “actual”causes (radiotherapy) components combined in a Healthcare Safety Management System (convergent approach) evaluating major interventions (impact of IT on medication safety)

3 retrospective risk analysis : PRISMA- medical
(voluntary) incident reporting and analysis learning from actual / reported process deviations

4 PRISMA-Medical Prevention and Recovery Information System for Monitoring and Analysis Three subsequent steps: Description by means of causal trees Classification according to the Eindhoven Classification Model (medical version) Determination of countermeasures by means of the Classification/Action Matrix

5 Causal tree example Wrong route Lines at same place Nurses not
informed Similar lines Connection possible Inadequate check Catheters not removed No coding O T H One of the incidents that has been analysed was about a patient who received medication via the wrong route. The Dutch Health Care Inspectorate investigated this incident and conclusions were drawn about the causes of this incident. In the incident database of the IGZ only one cause was recorded: the fact that the check that had been performed by the nurse was inadequate. This incident was thus registered as caused by an individual human error. On the basis of the information in the file, this incident was re-analysed by means of PRISMA-Medical. It appeared that four other factors also contributed to the occurrence of this incident. Firstly, both lines were situated at the same (left) side of the stomach; furthermore, the nurses had not been informed about this fact; also, the lines looked very similar; moreover, it was possible to interconnect the different systems. The PRISMA analysis thus yielded various direct causes and is therefore more broadly oriented than the current working method of the Dutch Health Care Inspectorate regarding the analysis of incidents. It is however very important to find out whether other causal factors underlie these direct causes. The lines were at the same place because the catheters had not been removed at the Intensive Care Unit because removal of the catheters was not required by protocol. The lines looked very similar because no colour coding was used to distinguish the lines. Because the file contained no further facts, no underlying causes could be identified for the other direct causes. The PRISMA analysis showed that both organisational, technical, and human factors contributed to the incident. This, in contrast with the cause that was registered by the Dutch Health Care Inspectorate, that only represents human failure. No protocol T O

6 Eindhoven Classification Model - (medical version)

7 Database Root causes for failure failure profile
Root causes for recovery recovery profile Context variables black-spot analysis The main goal of PRISMA is to build a quantitative database of incidents and their causes, from which conclusions may be drawn to suggest optimal countermeasures. First, an incident is described by means of a causal tree, separating the symptomatic, direct causes from the more fundamental root causes. Subsequently, the identified root causes are identified via a theoretical model of technical, organisational, and individual human causal factors, the medical version of the Eindhoven Classification Model. Finally, a PRISMA profile, consisting of the registered root causes of multiple incidents, is periodically assessed in terms of a so-called classification/action matrix. Because PRISMA-Medical probes deeply enough to identify the root causes of incidents and because it considers both the technical, the organisational and the human factors that contribute to incidents, PRISMA-Medical is a system-based method for the analysis of incidents.

8 PRISMA failure profile: hospital medication errors

9 Classification/Action Matrix
ECM code Design: Technology/work- place Procedures Information and Commu nication Training Motiva tion Escala Reflection T-EX × TD TC TM O-EX OK OP OM OC H-EX HK_ NO HR_ HS_

10 predictive risk analysis HFMEA / SAFER
series of group meetings to build a set of failure scenario’s for a (small) process of care : what may go wrong; why; what to do about it pro-active appeal

11 Healthcare Failure Mode and Effect Analysis (HFMEA)
A systematic approach to identify and prevent product and process problems before they occur Developed by the "VA National Center for Patient Safety" (

12 Relevance of predictive risk analysis
Retrospective (incident) analysis takes place after incidents did occur  hindsight bias Because of underreporting, biases can arise in incident databases  identification of "missing risks"

13 Definitions Failure Mode: Different ways that a process or subprocess can fail to provide the anticipated result (i.e. think of it as what could go wrong) Prescribing the wrong dose Failure Mode Cause: Different reasons as to why a process or subprocess would fail to provide the anticipated result (i.e. think of it as why it would go wrong) Miscalculation

14 HFMEA process Step 1: Define the topic Step 2: Assemble the team
Step 3: Graphically describe the process Step 4: Conduct a hazard analysis Step 5: Identify actions and outcome measures

15 examples of integration (1)
direct comparison of predicted (HFMEA) vs reported causes user problems with a new radiation therapy technology both types of failure causes expressed in the same PRISMA-medical classification (sub-)categories

16 PRISMA vs HFMEA : main categories
0% 10% 20% 30% 40% 50% 60% Percentage Tech Org Human other PRISMA main category PRISMA HFMEA : predicted causes Wat betekent het eigenlijk als beide methoden elkaar overlappen (dezelfde resultaten hebben)? Indien de resultaten van HFMEA en PRISMA met elkaar overeenkomen, geeft dit aan dat de mate van voorspelbaarheid van de risico’s groot is. HFMEA heeft dan veel toegevoegde waarde. En wat betekent het dan als de resultaten van elkaar verschillen? Wanneer ze verschillen wil dit niet direct zeggen dat dan met HFMEA de voorspelbaarheid van de risico’s niet groot is. Voorwaarde hiervoor is wel dat HFMEA moet zijn uitgevoerd zonder enige voorkennis van het proces. De uitvoering van de HFMEA en de PRISMA methode wordt beïnvloedt door verschillende factoren, waardoor de uitkomsten / resultaten mogelijk verschillend kunnen zijn. Hierbij moet men denken aan: Meldingsbereidheid / onderrapportage bij PRISMA Vaardigheid van de beoordelaars (interrater reliability) bij PRISMA en HFMEA Rol van de voorzitter bij HFMEA (sturen op b.v. technische faalwijzen) Het blijkt dat met het gebruik van weegfactoren in de tijd (vergelijking 9), de verdelingen aan de technische faalfactoren voor HFMEA en PRISMA precies gelijk zijn. Ook uit de andere vergelijkingen blijkt dat in dit onderzoek de technische risico’s goed te voorspellen zijn. De organisatorische risico’s zijn in dit onderzoek ook goed voorspelbaar. Zoals eerder gezegd blijken de menselijke risico’s minder goed voorspelbaar te zijn in deze situatie.

17 PRISMA vs HFMEA : subcategories
Frequency category HFMEA less than yearly yearly monthly weekly Weight-factor (= translation to 9 months) 0,1 0,89 9 36

18 examples of integration (2)
combining retrospective and predictive components in an overall Healthcare Safety Management System convergent approach of two imperfect risk identification methodologies mutual checks, comparisons, and inputs possible

19 examples of integration (2) continued
are repeatedly predicted problems (failure modes) ever being reported? can frequently reported problems help to select suitable processes for HFMEA and generate realistic failure modes? can frequently predicted causes steer the information gathering after an initial report? are proposed interventions for predicted vs “reported” causes similar? etc…

20 examples of integration (3)
developing a process-based evaluation methodology for major (patient safety) interventions predicting and monitoring the impact of IT on medication safety

21 Medication safety: definitions
[Van den Bemt et al., 2000]

22 Medication errors: causes (1)
Handwritten prescriptions and drug orders Look-alike drug names Sound-alike drugs and verbal orders Use of abbreviations Similar packaging and labelling Inadequate training and supervision Staff shortages Overwork and fatigue [Habraken, 2004]

23 Medication errors: causes (2)

24 IT: possibilities and problems

25 IT: possibilities and problems
56% 6% 4% 34% [Bates et al., 1995; Bates, 2000]

26 IT: possibilities and problems
IT application PROS CONS CPOE Legible prescriptions; no handwriting required Possibility of substitution errors Data entry only necessary once Failure to warn Exchange of data is easy Computerised decision support Drug information Risk of low vigilance and overtrust Patient-specific information and advice Bar coding Ensure five "rights": right drug, right patient, right dose, right route, right time Degraded coordination and communication Computerised medical record [Habraken and Van der Schaaf, 2006]

27 Barriers to the implementation of IT
Significant costs: technical, process redesign, and implementation and support Cultural obstacles: resistance to change Privacy and protection of (patient) data Lack of data standards Lack of (clinical) evaluation [Habraken, 2004]

28 Evaluation of effects and impact of IT: PRISMA and HFMEA
Not only outcomes of care but also the mechanisms underlying those outcomes Impact of IT on "error recovery " : Detection Diagnosis Correction of earlier errors / deviations

29 Evaluation of effects and impact of IT: PRISMA
PRISMA can be used to obtain an insight into the behavioural mechanisms underlying medication errors Classification/Action Matrix enables us to predict which types of human behaviour will be influenced by IT

30 Evaluation of effects and impact of IT: PRISMA
ECM code Design/ Technol Procedures Information and Communication Training Motivation Escalation Reflection T-EX × TD TC TM O-EX OK OP OM OC H-EX HK_ NO HR_ HS_

31 Evaluation of effects and impact of IT: PRISMA
IT applications would fall in two categories: "technology" and "information and communication" In case of improved technology  reduction of skill based human errors In case of information and communication support  reduction of knowledge based errors BUT: rule based human errors would not be influenced by IT

32 Evaluation of effects and impact of IT: PRISMA and HFMEA
Theoretical predictions could be reinforced by predictive risk analysis, such as HFMEA Empirical evaluation of actual impact of IT by means of intensified incident reporting Comparison of causal patterns of incidents that occur before, during, and after the IT intervention

33 Conclusion (1) IT often mentioned as prerequisite for reduction of medication errors Results regarding effects of IT vary greatly Effects of IT on behavioural mechanisms are not/hardly taken into account PRISMA and HFMEA offer a framework for in-depth analysis of impact of IT

34 Conclusion (2) Two types of predictions can be made of expected effects of IT on error and error recovery: Theoretical predictions by means of PRISMA HFMEA scenario-based predictions Intensified incident reporting and analysis would enable a fast comparison between predicted and actual effects On-line corrections of implementation process could prevent actual adverse events

35 Thank you for your attention

Download ppt "Integration of predictive and retrospective risk analysis in health care Tjerk van der Schaaf Leiden University Medical Center Eindhoven University."

Similar presentations

Ads by Google