2 Welcome and Agenda Purpose and Value of After Action Reports How to develop an Improvement PlanEsther Corwin, MEPExercise Training Officer, Texas Division of Emergency ManagementContact Info is on handout
3 What is an AARA consolidation of information gathered during the testing and evaluation of a community’s emergency operations plan through an exerciseProvides feedback to executive leadership, participating entities and governing agencies in the achievement of the exercise objectives and overall capabilities of the communityInformation gleaned from this process, identifies and guides future improvement actions
4 Why Write an AARCreates a multi-agency process to review an exercise or incidentGeneral exercise identification information, but also includes evaluation informationFacilitates honest and critical appraisal of actions based on plans and policies
5 Where Do We Look OR Exercises to simulate a stressful realistic scene Exercises to evaluate a new or seldom used equipment or skillIncidents presenting unexpected challengesIncidents having “less than expected” resultsORIncidents employing innovative actionsIncidents with unexpected positive results
6 Exercise Evaluation is Based On Community policies, Emergency Operations Plan, agency SOPs and SOGs, MOUs, etc.Exercise Evaluation MethodologyS.MA.R.T. objectivesExercise Evaluation GuidesObservational DataHotwashParticipant Feedback
7 Incident Evaluation is Based On Community policies, Emergency Operations Plan, agency SOPs and SOGs, MOUs, etc.Incident Action PlansBriefings, De-BriefingsObservational DataParticipant Feedback
8 Types of Observational Data Descriptive reportingTypically yields reliable data.Inferential reportingIncludes terms like “adequate” or “timely”Yields inconsistent dataEvaluative reportingIncludes terms like “efficient” or “successful”Difficult to reliably to collect
10 Three Levels of Analysis Capability Being EvaluatedCritical Resource Logistics & DistributionOnsite Incident CommandLevel of EvaluationCapability-Level AnalysisResource Management – Identify, dispatch, mobilize, track, demobilize and pay for resourcesDirect/Control incident management activities.Activity-Level AnalysisIdentify the resources based on incident needDevelop all necessary components of an IAP and obtain approvalsTask-Level AnalysisInventory resources by type/ category that are available to support this incidentEstablish incident objectives and priorities and operational periods
11 Exercise Evaluation Guides Identify the activities, tasks, and performance measures to be observedDeveloped and customized before the exerciseEvaluators complete the EEG by:Logging times and actions accuratelyDecision making processes and participantsPlayer roles and responsibilitiesCoordination and cooperationHow actions were performed, resources involvedDocumenting issues and recommendations
12 Data AnalysisThe goal is to evaluate the ability of involved functions to perform target capabilities
13 Preliminary AnalysisEvaluators organize observations into key issues and a chronological narrativeAt a minimum, should include:Description of the assigned function or operation, analyzed by capability, activity, and taskDocumented record of significant observed actions
14 Review Objectives Review exercise objectives What was the intent of the objective?What would demonstrate the successful performance of the objective?If the objective was not met, what factors contributed to this result?Review incident operational objectivesReview exercise objectivesWhat was the intent of the objective?What would demonstrate the successful performance of the objective?If the objective was not met, what factors contributed to this result?Review incident operational objectives
15 4 Steps of Data Analysis Identify issues Determine root cause Compare performance data to standardsIdentify differencesDetermine consequencesDetermine root causeDiscuss conditions leading to observed behaviorThe “Why” staircaseFool’s Gold
16 Why Staircase Each step should explain the step above … Why did it happen?Each step should explain the step above …Why did that happen?Why was that?And why was that?And why was that?ROOT CAUSE… down to the underlying root cause.
17 4 Steps of Data Analysis Develop recommendations Sustain and improveShort and long termWhat needs to be changed and howGive referencesCapture lessons learnedInnovative practice or a piece of knowledge gained from experienceProvides guidance for approaching a similar problem in the futureSave time, conserve money, and accelerate preparedness
18 Draft AARProvides feedback to participating entities and governing agencies regarding the achievement of objectives and overall capabilitiesRecords recommendations for improvementEstablish consensus and buy-in on next steps
19 The AAR Suggested AAR format includes: Executive Summary Overview Exercise Goals and ObjectivesAnalysis of Capabilities DemonstratedConclusion
20 Observations and Recommendations ReferencesAnalysisFor improvements, list consequences of action or inactionRecommendationsObservationsReferencesAnalysisFor improvements, list consequences of action or inactionRecommendationsObservationsReferencesAnalysisFor improvements, list consequences of action or inactionRecommendationsObservationsReferencesAnalysisFor improvements, list consequences of action or inactionRecommendations
21 After Action Conference Planning Team, evaluators, and stakeholders meet to review and refine draft AAREvaluation leads to a disciplined process for implementing improvement actions and continually strengthening preparedness
22 After Action Conference Address identified issuesDevelop specific improvement actionsPrioritizes action itemsAssigns responsibility to track implementation
23 Developing Improvement Actions These questions aid development:What changes need to be made to plans and procedures to improve performance?What changes need to be made to organizational structures?What changes need to be made to leadership and management processes??
24 Developing Improvement Actions More questions to aid development:What training is needed?What changes to, or additional equipment are needed?What lessons can be learned that will direct how to approach a similar problem in the future?
25 Improvement Benchmarks Must be clearly defined and attainableExamples include:Number of personnel trained in a taskPercentage of equipment that is up-to-dateFinalization of an interagency agreement within a given amount of timeInclude concrete deadlines to track progress toward full implementation
26 Activity – Improvement Development Older elementary school in a rural area just before dismissal on a spring day, parents are arriving to take students homeNWS issues tornado warning estimating less than 10 minutes before impactPrincipal activates “bell” for sheltering, but students/parents confuse it with dismissalSome parents enter school to take child and “run” from the stormAs a large group, given a scenario and issue, discuss possible recommendations and develop a simple improvement action.
27 Activity – Develop Improvement Issue:Student SafetyBasic Recommendation:Develop something to assist the principle in communicating emergency informationChanges to plans and proceduresChanges organizational structuresChanges to leadership/ management processesWhat training is neededChanges or additional equipmentLessons learned for the futureJarrell Tornado May 27, 1997
28 Improvements make things better than before SummaryObjectives and Evaluation Methodology are the foundation of Improvement PlanningRoot Cause analysis helps find the key policy, procedure, training or equipment to which an improvement could be madeApplies to Exercises and actual IncidentsImprovements make things better than before
29 Any question can lead to a good idea. Synergy is the Energy of People. Questions ???Any question can lead to a good idea.Synergy is the Energy of People.Esther Corwin, MEPExercise Training Officer, TDEM