[Insert Exercise Name] Evaluator Briefing and Guidance

Welcome & Introductions

Presenter’s Name Date Meeting Agenda  Exercise Overview  Exercise Agenda  Evaluation Goals  Exercise Evaluation Guide (EEG) Overview  Evaluation Process  Assessment, Review, and Analysis of the Exercise  Conclusion

Presenter’s Name Date Purpose [Insert exercise purpose]

Presenter’s Name Date Scope [Insert exercise scope]

Presenter’s Name Date Target Capabilities –[Insert Target Capabilities]

Presenter’s Name Date Exercise Objectives  [Insert objectives. Use additional slides if necessary.]

Presenter’s Name Date Agenda [Insert exercise agenda]

Presenter’s Name Date 9 Evaluation Goals  Validate strengths and identify improvement opportunities  Validate plans, procedures, and protocols  Improve processes and plans  Determine level of capability in regard to the exercise’s target capabilities

Exercise Evaluation Guide (EEG) Overview

Presenter’s Name Date 11 EEG Concept  The Homeland Security Exercise and Evaluation Program (HSEEP) Volume III provides evaluation and improvement planning policy  Evaluations should be objective-based to remain in line with: –National Preparedness Goal –Target Capabilities List (TCL) –Jurisdiction and agency plans  HSEEP EEG templates provide streamlined process for standardized evaluation concept, not necessarily content

Presenter’s Name Date 12 EEG Concept (cont’d)  Capability-based approach relies on similar activities to achieve a capability  Activities are achieved through successful completion of tactical tasks  Task detail should come from actual plan being exercised Consistent across exercises from TCL Consistent across exercises from Universal Task List (UTL) Content should be based on UTL detail, but include actual text from applicable plan

Presenter’s Name Date 13 EEG Function  Allows evaluators to review validation of plan elements –Plans used will depend on participating agencies  Includes direct references to plan sections, tables, and charts –Allows evaluators to better prepare for exercise –Provides detail for what will be expected of players  Remains focused on exercise objectives as they relate to the selected target capability

Presenter’s Name Date 14 Use of EEGs  Evaluation team should use EEGs in conjunction with full plans to prepare  Use EEGs as guides for observation, but do not focus on completing them during the exercise  Use EEGs for data analysis after exercise and in preparation for the evaluator debriefing  Use measured results to make actionable improvement recommendations within the After Action Report (AAR)/ Improvement Plan (IP)

Presenter’s Name Date 15 Data Collection Process Facilitator Notes EEG Analysis Form AAR/IP Issue Writeup Evaluator Notes and Data Corrective Action Plan Hot Wash following exercises to resolve conflicts Policy, Plan, Procedure Revisions EEG AAR Data Input Form

Presenter’s Name Date 16 Evaluator Notes and Data  Evaluators should capture player discussions, decisions, actions, and activity during exercise play  The EEGs provided should serve as guides for documentation of exercise activity

Presenter’s Name Date 17 EEG Analysis Form  Each evaluator will be provided with EEGs that will give specific guidance on what data to collect during the exercise, how to record it, and how to analyze it  To be completed at the conclusion of the exercise, using data gathered during the exercise

Presenter’s Name Date 18 EEG AAR Data Input Form  Each EEG will include a second form to be completed during the Facilitator and Evaluator Debriefing  This form will serve as an expansion to the notes captured in the EEG Analysis Form, including an area to provide recommendations

Presenter’s Name Date 19 After Action Report (AAR)/Improvement Plan (IP)  Data from the EEGs and debriefing will be compiled into the AAR/IP  The AAR/IP will include assessments for each capability observed as well as issues and recommendations  AAR/IP data will be entered into the Corrective Action Program (CAP) System AAR Data Input Form Snapshot CAP System

Evaluation Process

Presenter’s Name Date 21 Evaluator Responsibilities  Understand the exercise concept and scenario  Collect information  Make observations on player deliberations, discussions, and decision processes  Analyze actions and develop a post-exercise AAR/IP

Presenter’s Name Date 22 Preexercise Responsibilities  Review appropriate plans, procedures, and protocols  Review EEGs  Review and be familiar with scenario, objectives, and target capabilities  Participate in Facilitator and Evaluator Briefing

Presenter’s Name Date 23 Exercise Responsibilities  Monitor and document player performance  Use EEGs to confirm objectives are being met  Focus on critical tasks, as specified in the EEGs and plans being exercised  Participate in the Hot Wash and Facilitator and Evaluator Debriefing

Presenter’s Name Date 24 Postexercise Responsibilities  Assist Lead Evaluator in compiling and reporting evaluation data  Use data to determine whether the exercise’s target capabilities and plans were effectively implemented or demonstrated  Use data to identify strengths and areas for improvement  Complete AAR/IP Input Form for each identified strength and area for improvement

Presenter’s Name Date 25 Evaluator Guidelines At the start of the exercise, evaluators should:  Position themselves within the appropriate group or area  Determine an ideal location to observe exercise play

Presenter’s Name Date 26 Evaluator Guidelines (cont’d) Evaluators should not:  Interfere with exercise discussion

Presenter’s Name Date 27 Observing the Exercise Evaluators must keep an accurate written record of important issues that take place during the exercise:  Who would perform the action or made the decision?  What was the discussion that occurred?  Why was the decision made?  How did they make the decision?

Presenter’s Name Date 28 Recording Important Events  Evaluators should record events such as: –Initiating scenario events –Discussions of players –Key decisions –Deviations from plans and procedures –Equipment discussed –Directives

Assessment, Review, and Analysis of the Exercise

Presenter’s Name Date 30 EEG Analysis Matrix Objective MetObjective Not Met Players followed plan procedures Plan works Document best practices and strengths Capture why plan didn’t work Document how to improve relevant plan elements Players did not follow plan procedures Capture how objective was met Recommend successful actions and procedures be included in plan rewrite Capture why plan was not used May require plan training and familiarization

Presenter’s Name Date 31 Player Hot Wash  Allows players to participate in a self-assessment of the exercise play  Allows evaluators to clarify additional points and collect missing information

Presenter’s Name Date 32 Facilitator and Evaluator (F/E) Debriefing  Work to complete EEGs for submission  Discuss degree to which each objective was completed  Provide strengths, best practices, lessons learned, and possible IP recommendations  Begin outlining issues to be included in AAR/IP

Presenter’s Name Date 33 Questions/Discussion