CERTS Workshop 3: "Research and Development Supporting the Cyber Training Environment and Tools" Robert (Bob) Kazimer, SES Deputy to the Commanding General, U.S. Army Cyber Center of Excellence January 18, 2018

Workshop #3 Co- Chair: Dr Workshop #3 Co- Chair: Dr. Norbou Buchler, Army Research Laboratory, Adelphi MD Subgroup 1: Best Practices in Observational Measurement and Automated Metrics of Performance. What measures are required? How to measure readiness? Assessment results Readiness Input Process INDIVIDUAL LEADER (capstone) COLLECTIVE (exercise) FORCE (cyber flag) Qualification Automated Certification Observational (Manual) Cyber Capability Gunnery model for Cyber largely focused on individual level (no consistent measure at collective echelons) Longitudinal studies of team maturation needed (Storming -> Norming -> Performing)

Workshop #3 Co-Chair: Dr. Kenneth Williams, Ph. D Workshop #3 Co-Chair: Dr. Kenneth Williams, Ph.D., Program Director Associate Professor, STEM, APU Subgroup 2: Cyber Scenario and Ontology Development to support Cyber training There is a need to capture what responses have been done to date and categorize them. Pattern analysis of real world missions to perform a front end analysis in order to determine the requirements definition process. Task, conditions, standards; based on a survey so all players understand their left and right limits, priorities, to help identify what will be the scenario Create a game-like environment, monitor, record, automate the collection process needed due to the speed in which cyber changes Need to understand the vectors of attack, record events in training environment, develop pattern analysis, to help determine skillsets needed to solve the problem.

Workshop #3 Co-Chair: COL Paul Stanton, USA, Commander, Cyber Protection Brigade Subgroup 3: Readiness of Cyber Teams and the Dynamics of Mission Preparation Adaptable, Dynamic, Agile Training Environment that supports skill level 1-4 training Need to incorporate individual, collective, and leader training Need battle staff and commanding control training for cyber Reporting, tracking, and training management: collection, analysis, solution, & feedback