DARPA SRS PI MTG 13 JULY 2005 SRS Program Red Teaming Sandia National Labs John Clem Program Manager IDART™ Cigital Mark Wilson Managing Security Consultant Raba Technologies John Reel Chief Scientist

DARPA SRS PI MTG 13 JULY 2005 Why red team? many reasons Confirms efficacy of security solutions Identifies design weaknesses Exposes system to malevolent adversary (emulated) –Identify high consequence assets –Investigate system security Shows where defenses should be increased Improves understanding of the system Demonstrates the unimagined Provides actionable information It’s cost effective Build better processes, systems Reveals adversary/defensive MOP

DARPA SRS PI MTG 13 JULY 2005 The Red Team is your friend! A part of the greater team responsible for providing system security Cooperates with designers to build stronger, more reliable systems Help designers identify best options for system hardening Red teaming usually results in the identification of previously unknown vulns. or opportunities for improvement

DARPA SRS PI MTG 13 JULY 2005 What to expect? Red Team is to be provided all designer knowledge / system documents –Design –Implementation –Configuration –Security –Administration –User Red Team is to be provided no PWs or keys Lifecycle attacks are OOB Blue Team will need to freeze system development

DARPA SRS PI MTG 13 JULY 2005 What to expect? (2) Red Team will work hard (within its constraints) to beat your solution Some perspectives –Multiple vectors –For the duration –Control of key services –System control –Less interesting: underlying infrastructure We’d prefer to focus on your technology But if an underlying service is crucial to your system’s success…harden it

DARPA SRS PI MTG 13 JULY 2005 Schedule TBD, but… Initial red team work must begin ASAP Active red team engagements require preparation –Plan to send all design documentation in early August –Blue team must freeze technology/system development by a mutually negotiated date –Weekly telecons Red Teams should commence in September/October

DARPA SRS PI MTG 13 JULY 2005 Sandia SRS projects (1) 1.CMU PASIS storage protocol (C.M.U.’s Increasing Intrusion Tolerance via Scalable Redundancy project) 2.Cortex (Honeywell’s Mission-Aware Closed- Loop Cyber Assessment and Response project 3.PMOP (M.I.T.’s Detecting & Preventing Misuse of Privilege project)

DARPA SRS PI MTG 13 JULY 2005 Sandia SRS projects (2) process 1.Perform high level, ultra quick look analysis on the three projects –Review problem/solution proposal Identify key system functionality Understand technical strategy –Review PI’s red team experiment proposal –Confirm state of readiness for red teaming –Reports to respective Pis, SRS Program Manager, and White Team Recommended red teaming activity Suggested areas of red teaming focus

DARPA SRS PI MTG 13 JULY 2005 Sandia SRS Projects (3) process 2.One of the three projects will undergo a whiteboard attack red team activity –Location TBD (at SNL, at developer site) –Will perform red team attack (on whiteboard) to produce graph of possible attack routes Done with the developers Graph will be refined based on adversary model Will focus on technology in development and system dependencies –Quick look report and results of White board activity to PI and SRS Program Manager

DARPA SRS PI MTG 13 JULY 2005 Sandia SRS Projects (4) process 3.Two projects will undergo lightweight* red teaming –Identify vulnerabilities and security-related performance information –Includes remote or on-site red team testing –Focus on key system technology and certain dependencies –Experiment design needs red team input –High level report of red team activity *Narrowly-defined/focused

DARPA SRS PI MTG 13 JULY 2005 Sandia Contact Info Sandia National Labs John Clem Program Manager IDART™

DARPA SRS PI MTG 13 JULY 2005 RABA Corporate Experience Founded in 1994 as Boutique Technology Company Related Past Performance: –RABA performed a Red Team review of Maryland's Direct Recording Electronic (DRE) voting system Only one of its kind in the nation Recommendations were adopted, in part, by Maryland, Ohio, and California –Trusted contractor to the Intelligence Community for over eight years designing and performing Red Team exercises for national systems Penetration testing in a classified environment both as government employees and private contractors –Numerous computer forensics investigations supporting clients involved in both criminal and civil cases Recreated timelines, pieced together communications sessions, recovered documents, and developed evidentiary support from computer media Our Team: –All TS SCI cleared individuals –All have extensive experience in US Gov, DoD, and Intel Community –All have extensive experience in Information Warfare and Information Operations

DARPA SRS PI MTG 13 JULY 2005 Teams RABA Will Evaluate MIT –Learning and Repair Techniques for Self-Healing Systems MIT –AWDRAT JHU –Scalable Byzantine Replication Cornell –QuickSilver

DARPA SRS PI MTG 13 JULY 2005 RABA: Basic Methodology Anticipate about one week on-site with prep and a one-day test –Anticipate this makes setting up the test env easier for performers? MIT (Rinarn-Ernst) –Attack data structures in various ways: input, dynamic corrupt, etc. MIT (AWDRAT) –Emphasis on trust mechanism of the infrastructure –Emphasis on making the adaptive behavior asymptotically unstable JHU (Scalable Byzantine Replication) –Emphasis to compromise (f+1) nodes –Probably via injection & propagation of false data –Attack the propagation or update mechanisms –Attack Risk Assessment protocol Cornell (QuickSilver) –Emphasis to use Quicksilver API’s to create disruptive clients –Attack the gossip protocol –Create communications disruptions

DARPA SRS PI MTG 13 JULY 2005 Cigital Approach to Red Team Testing Procedures

DARPA SRS PI MTG 13 JULY 2005 University of Virginia Carnegie Mellon University Genesis Evaluation will consist of applying the protection strategy to a software system with known vulnerabilities and verify that the vulnerabilities no longer exist Evaluation will also consist of how well Genesis protects against vulnerabilities whose presence is not already known (only to the Red Team) and verify that the vulnerabilities no longer exist

DARPA SRS PI MTG 13 JULY 2005 MIT (Williams, Robertson) Pervasive Self-Regeneration through Concurrent Model-Based Execution Evaluate the robustness for the system to be “Fault Aware”, adapting to failure Evaluate methods for transitioning to intended states: monitoring progress, diagnosing failure, and repairing or reconfiguring components and effectively enact recovery/repair actions Evaluate robustness of service components for irreparably failure Evaluate the robustness of continuous monitoring of system for optimal performance metrics Determine the robustness of the overall system to individual software components to effectively regenerate large legacy systems

DARPA SRS PI MTG 13 JULY 2005 Telecordia, Rutgers Mitigating the Insider Threat using High-dimensional Search and Modeling Determine if the system can detect anomalies in high dimensional state spaces Identify if the system can efficiently detect attacker goals and targets and pro- actively respond to protect critical services Evaluate the Sensor Network effectiveness Determine if the system will annotate an attack on the Network History Repository Determine if the system will rank the “Top K” list of annotated states that are similar to the attack through the High-Dimensional Search Engine Evaluate the graph-based Insider Threat Modeling and Analysis tool to determine potential insider attack points and threat scenarios Evaluate the Response Engine ability to perform impact analysis of the attack on critical services and minimize collateral damage within system parameters

DARPA SRS PI MTG 13 JULY 2005 Schedule Preparation –Work with White Team to determine Rules of Engagement –Background investigation of architectural design and system protocols: review documentation of system –Analysis, discussion and interaction with Blue Team on Rules of Engagement to reduce overall burden of tests –Determine classes of attack that are in scope with system attack vectors

DARPA SRS PI MTG 13 JULY 2005 Schedule Testing Schedule –Schedule and setup of testing environment at Blue team site –Evaluation of system, architecture and topology –Seed software with vulnerabilities known only to the Red Team and supply to the Blue team as appropriate –Determine exploitation of anomalous network traffic, as appropriate –Identify attack classes that should be detected by the system and conduct evaluation of systems ability to detect attack –Evaluate effectiveness of the system’s ability to detect, conduct analysis and response capability

DARPA SRS PI MTG 13 JULY 2005 Deliverables Evaluation of system and review of findings with Blue team –Evaluate if vulnerabilities were mitigated by the system transformation –Draft document developed –Comments from Blue team incorporated –Delivery of analysis report to DARPA