Review of Reliable Design QUEST Frank Federico

What does this line graph tell you? Education Standardization

Improvement Improvement requires change Doing the same thing and expecting different results is insanity (Einstein) All improvement requires change; not all change is improvement.

Framework for Reliable Design Reliability occurs by design not by accident Process is the action point of all improvement methodologies Segmentation allows the perfection of the design

Starting Labels of Reliability Chaotic process: Failure in greater than 20% of opportunities 80 to 90 % (10 -1 ): 1 or 2 failures out of 10 opportunities 95% or better (10 -2 ): 5 failures or less out of 100 opportunities (These are IHI definitions and are not meant to be the true mathematical equivalent)

Non-Catastrophic Processes Definition: Failure of the process does not lead to death or severe injury within hours of the failure Less than 95% performance or worse is most commonly seen in these processes (hand-washing as an example) Poor outcomes do not occur with each defect due to either to biologic or system resilience

Reasons for the Reliability Gap in Healthcare Current improvement methods in healthcare are highly dependent on vigilance and hard work The focus on benchmarked outcomes tends to exaggerate the reliability within healthcare hence giving both clinicians and leadership a false sense of security Permissive clinical autonomy creates and allows wide performance margins The use of deliberate designs to achieve articulated reliability goals seldom occurs

Improvement Concepts Associated with < 95% Performance (Primarily can be described as intent, vigilance, and hard work) Common equipment, standard order sheets, multiple choice protocols, and written policies/procedures Personal check lists Feedback of information on compliance Suggestions of working harder next time Awareness and training

Focus on Benchmarks: Biology Protects Us All defects in process do not lead to bad outcomes Healthcare tends to look at outcomes and not the reliability of the process leading to outcomes (hand washing is an example) Benchmark to best practice not aggregate averages

Focus on Clinical Autonomy in Health Care Processes Desired - variation based on clinical criteria, no individual autonomy to change the process, process owned from start to finish, can learn from defects before harm occurs, constantly improved by collective wisdom - variation Current - Variable, lots of autonomy not owned, poor if any feedback for improvement, constantly altered by individual changes, performance stable at low levels Terry Borman, MD, Mayo Health System

Use of Deliberate Design The Reliability Design Strategy Prevent initial failure using intent and standardization Identify defects (using redundancy) and mitigate Measure and then communicate learning from defects back into the design process

The Three Step Design for Reliability Design TechniquesSteps 1-Identify the process to standardize 2-Segment the population to test the design for anomalies 3-Use both hard work and human factor concepts Prevent initial failure by standardizing the process to achieve 80% (step 1) 1-Utilise a robust human factors concept to make visible failures from step 1 after step 1 has achieved 80% reliability 2-Once the failure is identified, apply an action to mitigate the failure Identify failures in step 1 and apply an action to achieve 80% for these failures (step 2) 1-Identify common failures 2-Develop a method to measure and study failures 3-Utilise knowledge of common failures to redesign either step 1 or step 2 In either step 1 and/or step 2 detect the failures, and use the knowledge from analysis of the failures to redesign (step 3)

Using Segments Allows for the control of some variables Defines the boundaries around which sequential expectations for success can be found More likely to test the validity of the design rather than deal with barriers Fosters a deeper understanding of the design complexity required for the project Forces understanding of the differences between segments as design strategies Allows the formation of more predictable timelines

Examine Your Own Process What is the level of reliability of the process? Does the success of the process depend on a person? Is our main effort to improve the process training and education? What will you do to improve the level of reliability? Did you begin with a segment and the spread?

The “Set Up” for Reliability Select a topic whose outcome you want to improve Determine a high volume segment for initial design testing Build a high level flow chart for that segment Determine where the defects occur in the current system Determine where your design work will begin with by identifying where the commonest defects occur Your goal is always 95%

Topic: Transition Home Patient admitted to the unit Admission assessment completed Social worker notified to conduct discharge planning assessment SW assessment completed on day 2 or next business day Segment: Patient Population/ nursing unit Social worker not notified Our aim is to achieve a reliability of 95% with ensuring that the social worker is notified

The Reliability Design Strategy Prevent initial failure using intent and standardization Identify defects (using redundancy) and mitigate Measure and then communicate learning from defects back into the design process

Why Standardize? Contributes to building an infrastructure (who does what, when, where, how and with what) Support training and competency testing to sustain the process Achieve front line articulation of key processes by staff Allows the appropriate application of Evidence Based Medicine consistently Feedback about defects and application of learning to design is possible

Your Task What will you standardize? How will you engage front line staff? How will you know if the change is an improvement?

Next step Detection-Mitigation Develop a redundancy/contingency/back- up Must be independent to be effective Must be careful as it is a use of resources

Human Factor Concepts Decision aids and reminders built into the system Desired action the default (based on evidence) Redundant processes Use fixed current scheduling in design Take advantage of habits and patterns Standardization of process based on clear specification and articulation Human Factors and Reliability Science: (Designing sophisticated failure prevention, failure identification and mitigation)

Characteristics of “Redundancy Tools” Require careful consideration since they do represent a form of “waste” Needs to be connected to the process almost all the time (at least ) Requires a good standardization functions be in place before implementing a redundancy Need to be truly independent Need to be used or will no longer function as a good filter Must follow with a mitigation strategy

Measurement Measure and then communicate learning from defects back into the design process Measurement should take place at each step Measurement should be as close to real time as possible If the system is not performing as designed must make changes

As you begin your work Have we selected a segment? ─What advantage does this segment provide? Develop a high-level flow diagram Identify the most common defect Standardize: use small tests of change Measure the reliability of your process

Your Next Task What redundancy/back-up will you have in place? Remember that this process must also be standardized.

When to Re-design If you have a process and it is not reliable ask: ─Is it truly standardized? ─Why are staff not using the process? ─What is failing? ─Is it time to develop a totally new process?

Key Questions To Analyze Testing and Implementation Key QuestionYour Evaluation Is the connection between goals and process clear? Is the design strategy primarily vigilance and hard work? Has some degree of segmentation been used to test the design? Is standard work with testing been part of the design? Is a design methodology being used? Are small tests of change being used in a rapid cycle? Is data collection rapid enough?

Key Questions To Analyze Spread Key QuestionYour Evaluation Have you repeated the small test cycles as you spread from the initial site Is the process of spread dependent on one person Has some degree of segmentation been used to spread Has customization been allowed or encouraged Is the same team who developed the pilot now responsible for spread Have you shifted your focus from process reliability to outcomes too early