1. Sensitive Metric Collection and Reporting System Michael Aiello Hanning Gao Martin Goldberg Michael Sosonkin Jason Woloz

2. Agenda Introduction High level requirements Risk Analysis System Architecture Security Architecture Security Policies Security Compliance Matrix and Traceback Security Requirements Security Trade Study

3. Sensitive Metric Collection and Reporting System The aim of the system is to securely collect and report sensitive metrics. Currently, there are several ad-hoc processes that exist in the environment designed collect metrics of varying sensitivities. The oversight, organization, calculation, grouping and reporting on these metrics is currently accomplished through a tedious manual process. The goal of the collection and reporting system is to standardize, automate and allow privilege management for the different aspects of the collection, calculation and reporting processes.

4. High Level Requirements Repository  Handle metric storage and archival  Redundant off-site backup depository Metric-collection Subsystem  Automated metric collection  Manual metric collection Collection Job Configuration  Specified data point selection  Scheduled collection

5. High Level Requirements Statistical Application  Task and execution manager  Result Viewing  Automated monitoring and execution Reporting  Centrally managed administrative interface  Multi-level third-party reporting capabilities

6. Risk Analysis Assets Firm Reputation – The metrics information, if used, can damage the company’s reputation. Customer Information – The system can be used to discover vulnerabilities, if abused then other information stored in company’s information systems is also vulnerable. Availability of Metric repository Contents of the Metrics Database – The database contains information about the company’s vulnerabilities and information system setup. The information may be used to cause further damage. Knowledge of Firm vulnerabilities – This system provides a way of managing this, so if known then the company is exposed. Availability of Metric collection system

7. Risk Analysis Threats 1. Insider – People that control the system and use the system but do not have administrative control over it. a.Motivation: Manipulate metrics to look better to management. Get money for selling information to others. b.Capabilities: i.Intellectual: Medium ii.Computing Resources: Limited to what’s in the company the user. iii.Access to system: All system’s user interfaces, many areas of the database. iv.Budget: Low c.Probability of Attack: Low d.Probability of Success: High e.Risk: Low

8. Vulnerable and likelihood areas Automated Collection Component  Reception of manipulated information from in house developed systems- Medium  Reception of manipulated vendor feeds - Medium  Reception of manipulated emails with fraudulent metrics - High  Vulnerabilities in collecting software – Medium  Vulnerabilities on vendor interfaces- Low  Denial of Service attacks on collection system – Low

9. System Architecture

10. Security Architecture

11. Security Policies 1. System Level All communications must be secure between repository and its associated modules 2. Automated Collection Component Will only connect to authorized information gathering agents 3. Statistical Packages The statistical providers must not have write access to the database.

12. Security Policies 4. Reporting System Should only have read access to the repository 5. Configuration System Only administrator authorized modules can be imported into the collection system. 6. Metric Repository Metric database information should securely and redundantly in compliance with the mission critical information storage policy.

13. Security Requirements Statistical Package The providers may not be allowed to write to the database.  For any statistical computation, only reading of data is required.  A provider must be allowed to have some place where it can write temporary or swap data. Communication between the provider and the database must be confidential.  Either through an encrypted or a dedicated channel. The calculation must encapsulate the provider so that provider is locked with its environment.  Any function for computation must run in its own environment so that it does not interfere with the rest of the system  Any computation must not be allowed to profile and collect information about the other parts of the system. Secure Communication – in case there is distribution of processing then there should be secure communication between parallel processes.  Channels of communications between the computing processing must be either encrypted or dedicated to ensure no information leak. Temporary data security  Data written by the providers for temporary purposes must be in a secure location.

14. Security Matrix Security RequirementProcess/HardwareJustification Depository system will be distributed to provide fail over Database will be setup to run in a cluster environment To ensure metric data collection is not interrupted or backlogged. Encrypted communication between repository and subsystems Use industry standard encryption protocol such as SSL or VPN To protect metric data’s integrity and confidentiality The repository network is segmented from rest of corporate network Use firewall to restrict access to repository network To protect repository from unauthorized access and also to protect data confidentiality Communications between the offsite backup system and the primary system should be encrypted Use VPN to connect onsite and off site depository system To protect metric data’s integrity and confidentiality Direct access to the repository will be restricted to system administrators Enclose depository system in locked down physical area and issue access only to sys admins To protect repository from unauthorized access and also to protect data confidentiality

15. Trade Study A Security Metrics Collection and Storage system poses to be very useful in large enterprise networks. This is by no means a bleeding-edge concept, for organizations required to perform site-wide security audits. In such cases there is now a federally mandated need to collect, store and correlate security metrics. With a natural progression of business thinking this system will eventually be in a commercial solution. However, after doing a trade study, it was found that currently there is no commercial off the shelf software solution available. This is most likely due to the fact that most organizations use several vendor solutions, each of which have their own proprietary format. Up until now, home- grown solutions have been built to satisfy an organizations particular requirements. Until some kind of standard or vendor solution is developed that reads multiple proprietary formats, this will most likely remain the case.