Optimal CyberSecurity Analyst Staffing Plan Sponsors: Dr. Rajesh Ganesan Ankit Shah Thomas Lepp Jennifer Krajic Kendrick van Doorn

Agenda Background Problem Definition Model / Approach Results Validation Way Forward Background - Kendrick Problem Definition - Kendrick Model / Approach - Jennifer (Part 1 & 2) & Thomas (Part 3) Results – Thomas Validation - Thomas Way Forward - Kendrick Old Schedule Optimized Schedule

CSOC A CyberSecurity Operations Center (CSOC) protects against emerging and dynamic cybersecurity threats. It is critical that all alerts are analyzed in a timely manner to reduce risk to the organization. Alerts are unique notifications sent to analysts after correlation of network level events. Describe a CSOC and sensors all around the world. Lead into the IDS.

Background Intrusion Detection System (IDS) Device or software application that monitors a network or systems for malicious activity of policy violations[4] Main detection methods are signature-based, anomaly-based, and stateful protocol analysis [4] Some limitations are packet noise, false-alarms, legacy software, and lag time [4] Need for analysts to constantly review and provide disposition for all IDS identified threats Describe an IDS and lead into the model on the next page.

Background Intrusion Detection System (IDS)[1] Describe the model. From the book. That is referenced. That is so great.

Problem Definition & Scope The Optimal CyberSecurity Analyst Staffing Plan Team is tasked with delivering a 14 day staffing schedule model for a CSOC that minimizes payroll costs by scheduling an adequate number of analyst (Junior, Intermediate, and Senior) for the initial investigation of varying alert generation patterns. The staffing schedules will include variable overlapping shift patterns that satisfy staffing and schedule requirements. Deliverables: Staffing Model and Schedule Sensitivity Analysis of Results Model Validation Next Steps Assessment This is the problem definition and what we are going to deliver. Evaluation needs to be described for each deliverable.

Initial Analysis Time period: 14-day vs. 1-day Increments: 1 hour vs. 4 hour Scenario Period Increment Work hour Options 1 1 hour 4-12 hours 2 4,8,12 hours 3 4 hour Time Periods 14-Day 1-Day 1 hour 2336 224 4 hours 284 26 Speak to the 3 scenarios- equaled the same shift patterns and the same costs

Model / Approach The model contains three major parts: Input: Calculate average alert arrival rates and feasible shift patterns Python Programming Optimize: Minimize payroll costs Integer Programming Assign: Minimize staff and create staff schedules First Fit Decreasing Heuristic Input Optimize Assign

Model Part 1: Input Input Optimize Assign Demand: Alerts follow a Poisson Distribution with a varying alert arrival rate Alert arrival/ hour/ sensor: High (12), Moderate (9), and Low (6) Average alert arrival rate + 2 standard deviations for 10 sensors Assumption: Alerts are batched into 4-hour segments. All alerts will be analyzed by the end of 4-hours. New alerts are presented at the beginning of the next 4-hour segment. Time Period Frequency of Alert 06:00 AM to 10:00 AM High (12) 10:00 AM to 02:00 PM Moderate (9) 02:00 PM to 06:00 PM 06:00 PM to 10:00 PM 10:00 PM to 02:00 AM Low (6) 02:00 AM to 06:00 AM

Model Part 1: Input Feasible Shift Patterns: Staffing Constraints: Input Optimize Assign Feasible Shift Patterns: Staffing Constraints: Cannot work less than 4 hours or greater than 12 hours in a row. Minimum break of 8 hours in between shifts Payroll Calculations: Base Rate: Junior $38/hr, Intermediate $49/hr, Senior $61/hr Base Rate increase by 10% for 4 hour shifts Base Rate increase by 10% for working between 10PM and 6AM

Model Part 1: Input Output: Input Optimize Assign Output: Generation of all possible shift patterns for 1 day and payroll calculation for each analyst/ shift pattern 1 or 0 is assigned to each 4 hour period 1 = working 4 hour period 0 = break for 4 hours Example: Shift Pattern #14 06:00 AM- 10:00 AM 10:00 AM- 02:00 PM 02:00 PM- 06:00 PM 06:00 PM- 10:00 PM 10:00 PM- 02:00 AM 02:00 AM- 06:00 AM 1 - 4 (Base Rate + 10% Base Rate) 4(Base Rate + 10% Base Rate)

Model Part 2: Optimize Input Optimize Assign

Model Part 2: Optimize Input Optimize Assign Output: required shift patterns for optimal payroll cost. Required shift pattern by analyst type: Demand versus supply of alert analysis:

Model Part 2: Optimize Required shift pattern by analyst type: Input Optimize Assign Required shift pattern by analyst type:

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Families : 3, 1, 6, 4, 5, 2 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Decreasing Families : 6, 5, 4, 3, 2, 1 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Decreasing Families : 6, 5, 4, 3, 2, 1 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Decreasing Families : 6, 5, 4, 3, 2, 1 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Decreasing Families : 6, 5, 4, 3, 2, 1 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Input Optimize Assign Input – Required patterns from part 2 for each day Method - First Fit Decreasing (FFD) heuristic Worst case scenario of: 11 9 ∗𝑂𝑃𝑇+ 6 9 [2] Example[3] Decreasing Families : 6, 5, 4, 3, 2, 1 Bus #1 Bus #2 Bus #3

Model Part 3: Assign Intermediate Input Optimize Assign Assigned

Results Work Schedules Junior: 33 Intermediate: 15 Senior: 39

Two Week Payroll Cost: $183,356 Results Summary Two Week Payroll Cost: $183,356

Sensitivity Analysis – Weekend Constraint Senior Analyst Example – Each Analyst cannot work more than one weekend Two Week Work Schedule: Senior Workers with Weekend Constraint

Sensitivity Analysis – Weekend Constraint Senior Analyst Example – Each Analyst cannot work more than one weekend Two Week Work Schedule: Senior Workers with Weekend Constraint

Sensitivity Analysis – Weekend Constraint Senior Analyst Example – Each Analyst cannot work more than one weekend Two Week Work Schedule: Senior Workers with Weekend Constraint Two Week Work Schedule: Senior Workers without Weekend Constraint Total Reduction of Workforce: Junior 8 (21%), Intermediate 4 (27%), Senior 14(36%)

Each line represents a simulation against the scheduled supply Model Validation Each line represents a simulation against the scheduled supply

Way Forward and Additional Work The team has delivered the current model, sensitivity analysis, and final report to the project sponsors. Additional items discussed with the project sponsor for follow on work include: Manager friendly view and manipulation of data. Real world work schedules and habits. Inclusion of alert complexity. Further development of FFD heuristic or analysis of different heuristic. Analysis of Surge and backup employees. Analysis of backlog of alerts and completion timeline.

Acknowledgements Dr. Rajesh Ganesan Ankit Shah The team would like to thank the following individuals for their support throughout the project. Dr. Rajesh Ganesan Ankit Shah Dr. Karla Hoffman Dr. Kathryn Laskey

Questions

References [1] – Ganesan, R. Jajodia, S., Shah, A. and Cam, H. 2016b. Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning. ACM Trans. on Intelligent Systems and Technology, 8, 1, Article 4 (July 2016), 21 pages. DOI:http://dx.doi.org/10.1145/2882969 [2] – Chen, Bo, Mike Paterson, and Gouchuan Zhang. "ESCAPE'07 Proceedings of the First international conference on Combinatorics, Algorithms, Probabilistic and Experimental Methodologies." LNCS: Lecture Notes In Computer Science (n.d.): 1-11. Web. [3] - HEGARTYMATHS. "Packing algorithms - First-fit (decreasing) algorithms (Decision Maths 1)." YouTube. YouTube, 14 Feb. 2012. Web. 27 Apr. 2017. [4] "Intrusion detection system." Wikipedia. Wikimedia Foundation, 04 May 2017. Web. 06 May 2017.