2. The Coles Notes Approach to Effective Network Security Management Reporting Dave Millier

3. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Overview Brief Intro Some Terms of Reference A foundation for Reporting Who, Why, How? Typical types of Logs Gathering, Normalization, Archiving Key Performance Indicators Compliance Reporting Management Presentation

4. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Technical Definitions (From the Devil’s Security Dictionary 2.0) Active X A technology for making Web vulnerabilities more engaging and fun. Change Control A carefully defined and measured process of self-delusion.

5. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 E-mail A form of text communication similar to but far rarer than spam. Single Sign-on A process ensuring that one password gives hackers access to everything. Hash Table The place you roll a joint. Keystroke Loggers Men who type down trees for a living.

6. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Risk The unavoidable part of life that CEOs try to ignore, CFOs try to hide, CIOs try to understand and CSOs try to control.

7. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 From CSO Magazine 2005/2006 Surveys 46% of CISOs spend up to 1/3 of their day reading/analyzing reports generated from their security applications 35% of CIOs indicate network security improvements top to-do lists in 2005 65% of companies report they don’t have established ROI metrics for security risk management 56% of company boards surveyed rarely/never discuss policies, leaving IT Security Mgmt. to make compliance decisions and ensure adherence Some Interesting Statistics

8. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Considerations for Reporting to Management Align reports with corporate goals Communicate in their language Report residual risk, if it exists Highlight significant trends and events

9. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 How and Where Do We Get The Data To Report On?

10. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323

11. Collecting & Storing the Data

12. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Benefits of a good SIM/SEM Aggregate / Normalize data from unrelated devices into useful info Customized reporting to adhere to specific compliance requirements Analyze/correlate information from various devices to identify attacks as quickly as possible Provide the ability to conduct forensic investigations against all data gathered Increase value of existing security devices Improve effectiveness/responsiveness of existing personnel

13. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Overall Goals of Log Gathering Centralize log data –Fast forensic searching –Contextual reporting –Significant reduction in troubleshooting time –Regulatory/compliance drivers Normalize data –Make all data look the same regardless of source from a searching perspective –Ease of searching Archive data –Regulatory / compliance –Assist with historical investigations –Demonstrate due diligence

14. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Auditors want to see that there is a PROCESS in place Centralize Logs / Archive Logs Report on unusual activity Identify Action Items for follow-up Document ALL of these steps!!!!! Auditors care more about consistent process than the results themselves Process, Process, Process

15. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Using the Data for something meaningful: KPIs and Compliance Reporting

16. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Key Performance Indicators (KPIs) Specific measurement of an organization’s performance in some area of its business One purpose is to give business decision- makers quantifiable measurements of items it has determined important to its long-term success In order to be useful, they must be consistent, have a direct correlation to the area of the business being monitored, and not be susceptible to false readings

17. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Many Uses of KPIs Planning, Control, Evaluation Managing change Communication Measurement and improvement Resource Allocation Measurement & Motivation Long-term Focus

18. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Components of a good KPI Descriptive Title for KPI Purpose: Rationale underlying measure Relates to: Business objectives related to item being measured Target: Targets are necessary to evaluate level of performance Formula: The way performance is measured affects how people behave. The right formula ensures the right behaviour Frequency of measurement: Function of the importance of the measure and the volume of data available Frequency of review: Identify how often the gathered measurements should be reviewed Who measures: Identify the person(s) responsible for collecting/reporting the data Source of data: Consistent source of data is vital to performance tracking over time Who owns the measure: who is overall responsible for accuracy, response, etc.? Who acts on the data: person/role should be identified What do they do? Without defined response, measurement is pointless Any notes or comments about the KPI Source: Neely, 1997

19. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Example of Gathering KPI Information

20. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Typical Compliance Framework

21. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Compliance Reporting Focus on providing senior Mgmt. / auditors with current compliance status Unlike KPIs, usually require more detail More focus on identifying anomalies, rather than just reporting on a “number” Should provide access to detailed audit trail for any investigations of events (ticket or case management system)

22. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Compliance Reporting Goals Provide auditors with a centralized location to perform audit functions against process(es) related to regulatory/compliance requirements Provide attestation to senior C-level executives responsible for the integrity of financial systems, shareholder reporting, etc. Demonstrate “compliance” with established Policies and Enforcement strategies

23. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Sample Security Reports / KPIs User Identification/Authentication User Account Management User Privileges Configuration Management Security Device Specific Reports (firewall, IDS, spyware, A/V, etc.) Event Activity Monitoring / Logging

24. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Presenting Results Centralized Repository Display KPIs as both statistical numbers and graphs where possible Granular User-level Access Controls to every report / view Maintain historical copies of pre- generated reports Make sure reports can be saved / printed

25. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 1 2 3 4 EXECUTIVE SUMMARY INFORMATION High level summary information for executives and managers. Security Assessments indicate overall security posture, analysis from business and technical perspective, key metrics, case/ticket management. SUB SYSTEM SUMMARY INFORMATION Summary information for security and/or operational sub-systems, with more focused access to information. FORENSICS & REPORTING View correlated logs in near real time. Conduct forensic searches. Generate real-time and historical reports. RAW LOG SOURCES Key devices (firewalls, IDS, network devices, servers, etc.) located throughout the infrastructure. IDEAL REPORTING HIERARCHY

26. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Sample KPI Reporting

27. www.sentrymetrics.com - 1852 Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) 488-2323 Here’s Gord! Question/Answer At the End of Both Presentations