1. Observation Process May 2005 Revision The Key to Effective Review - The Performance Element of the Performance-Based Review Process! 10T10L

2. Upon successful completion of this topic, you should be able to: Describe how to prepare to observe an ORR/RA evolution. Explain the principle of observation vice participation (Heisenberg Principle). Describe requirements based observation techniques. List several types of ORR/RA evolutions. Explain the need for a facility to run a drill vice only respond to a drill. 10T10LMay 2005 Revision 2

3. 3 Evolution Purpose To perform, to the maximum extent possible, a representative sample of anticipated operations and evolutions, and responses to credible casualties and “off-normal events” in order to demonstrate the facility operational readiness...

4. 10T10L May 2005 Revision4 Types of Evolutions Some Samples Exercise Drills Operations Maintenance Surveillances Tours System startup Briefings Training session Critique Meetings Turnovers Walk downs Alarm response

5. 10T10L May 2005 Revision5 Steps for Conducting an Observation Preparation... Actual observation... Follow-up... Reporting...

6. 10T10L May 2005 Revision6 Step 1 - Preparation Single most important step in the process: – Serves as a foundation; –Team must select representative operations; –Surrogate material use... Steps for preparation: – Select the activity to be observed; – Determine appropriate standards of performance; – Become familiar with standards/knowledge required; –Integrate appropriate upset conditions.

7. 10T10L May 2005 Revision7 Selecting the Activity Important to plant, personnel, procedures, management system: –Plant-people-procedure-process relationship; –Surveillance versus routine preventive maintenance; –Critical process versus routine operation; –Minimize simulations! Known or suspected problem area: –ORPS review; –CORR or MSA results; Corrective actions to rectify previous problems...

8. 10T10L May 2005 Revision8 Selecting the Activity (continued) Existing facility condition: – Material condition of equipment; – Housekeeping practices; – Labeling; – Industrial safety conditions; –Surrogate material available and use...

9. 10T10L May 2005 Revision9 Selecting the Activity (continued) Targets of opportunity: – Drills / Dry-runs; – System operability checks; – Maintenance work in progress; Teamwork: – Keep other members informed of your observation; – Deploy Team to Achieve Complete Coverage!

10. 10T10L May 2005 Revision10 Selecting the Activity (continued) Review of off-normal condition preparedness: –process upset conditions; –emergency conditions; –Facility integrates upsets into selected evolutions... Practical considerations: – Time availability; – Accessibility of activity.

11. 10T10L May 2005 Revision11 Preparation Be a prepared and knowledgeable observer: –Obtain and study the procedure; –Understand what actions you should see and where; –Mark-up the procedure in advance to enhance notation; –Walk down the area, understand the systems; –Understand ancillary processes... log keeping, pre-job briefing processes, authorization/control mechanisms; –Have clear understanding as to who is to watch what; –Prepare for copious notes including time of happening.

12. 10T10L May 2005 Revision12 Rules for Observations Do not act as a Safety Observer! Do not violate rules! Do not interfere with operations! Do not impose conditions, simulations, or anomalies! Be transparent... Request Facility to communicate roles and rules!

13. 10T10L May 2005 Revision13 Step 2 – Actual Observation Interaction with observed personnel: –Establish rapport; –Explain purpose of observation; –Outline your role as an observer: Questioning techniques; Will not interfere with work; Be Transparent.

14. 10T10L May 2005 Revision14 Step 2 – Actual Observation (continued) Take thorough notes in real time: – Times actions occurred; – Procedure details- have procedure in hand; – Nomenclature and physical details; – Be quantitative, record pertinent data; – Include questions and follow-up items; –Follow procedure as conducted.

15. 10T10L May 2005 Revision15 Step 2 – Actual Observation (continued) Clearly understand what actually happened: – Don't assume... – Ask clarifying questions without interfering; – Seek additional details as needed; – Full understanding is imperative to accuracy of the observations; – Coordinate with other Team Members who observed.

16. 10T10L May 2005 Revision16 Observe beyond the Activity Start at the true beginning: –Observe Pre-job briefings; –Observe system alignment checks; –Observe daily checks; Note the peripheral activity: – People; – Environment; – Preparation/set-up activities. Become creatively inquisitive... Keep attention to important items...

17. 10T10L May 2005 Revision17 Heisenberg Uncertainty Principle Difficult to make an accurate measurement without affecting what is being measured... Difficult to measure people performance without influencing their performance... Do not ask prompting type questions that could change performance...

18. 10T10L May 2005 Revision18 Problems Often, if not always, symptomatic of problems elsewhere: –keep Team Leadership informed of problems that have broad impact (training, procedures, supervision, safety culture, etc.)! Isolated problems not likely... Often recurring: –those where management has taken corrective action...

19. 10T10L May 2005 Revision19 Step 3 – Follow-up Resolve open questions... Primary purpose of follow-up: –What you observed are indicators... –Follow-up will define real problem: need for improvement; underlying causes.

20. 10T10L May 2005 Revision20 Primary Follow-up Questions What is the problem? How big is the problem? Where else might this problem exist? Other Contributors ? –Program Problems? –Training Problems?

21. 10T10L May 2005 Revision21 Step 4 – Recording the Observation Perspective... –Clearly understood? –What does it really mean? Record immediately after observation: – (Observer notes / Form 1... ). Many indications/deficiencies lead to a finding – identified in Form 1 and summarized on a Form 2...

22. 10T10L May 2005 Revision22 Step 4 -- Recording the Observation Avoid speculation... Stick to specifics when describing observed strengths and weaknesses...

23. 10T10L May 2005 Revision23 Step 4 (continued) Avoid superlatives such as: – "is the worst I've ever seen”... – "is the ________ ever seen”... Stay focused on whether the objective is being met as defined by the criteria...

24. 10T10L May 2005 Revision24 Other Lessons Learned for Conducting Observations The ORR Team observes the drill process: – The Team does not run the drill; – DO NOT act as Safety Observers or Drill Initiators! Observe pre/post-job/setup activities... Have an “observation contingency plan” ready... Use more than one observer for most activities... “Off-normal events” are not “emergency drills”...

25. 10T10L May 2005 Revision25 Observations to Findings Observations provide the basis for: – Findings; – Observations; –Correlation with Interviews/Document Reviews. Combine several observations if appropriate to draw conclusions concerning program status: –Rarely should a single observations result in a finding; –Adequacy is the standard, perfection is the goal.