Grade Projections I’ve calculated two grades for everyone: – Realistic: assumes your performance in the course continues the same – Optimistic: assumes you get maximum scores for the rest of the course Some statistics: – Realistic: 1 A, 3 A-, 6 Bs, 4 Cs, 5 Ds, 1 F – Optimistic: 12 A, 7 A-, 1 B+, 1 B There is still a lot of room to improve

Dynamic Quarantine Worms spread very fast (minutes, seconds) – Need automatic mitigation If this is a new worm, no signature exists – Must apply behaviour-based anomaly detection – But this has a false-positive problem! We don’t want to drop legitimate connections! Dynamic quarantine – “Assume guilty until proven innocent” – Forbid access to suspicious hosts for a short time – This significantly slows down the worm spread C. C. Zou, W. Gong, and D. Towsley. "Worm Propagation Modeling and Analysis under Dynamic Quarantine Defense," ACM CCS Workshop on Rapid Malcode (WORM'03),

Dynamic Quarantine Behavior-based anomaly detection can point out suspicious hosts – Need a technique that slows down worm spread but doesn’t hurt legitimate traffic much – “Assume guilty until proven innocent” technique will briefly drop all outgoing connection attempts (for a specific service) from a suspicious host – After a while just assume that host is healthy, even if not proven so – This should slow down worms but cause only transient interruption of legitimate traffic

Dynamic Quarantine Assume we have some anomaly detection program that flags a host as suspicious – Quarantine this host – Release it after time T – The host may be quarantined multiple times if the anomaly detection raises an alarm – Since this doesn’t affect healthy hosts’ operation a lot we can have more sensitive anomaly detection technique

Dynamic Quarantine An infectious host is quarantined after time units A susceptible host is falsely quarantined after time units Quarantine time is T, after that we release the host A few new categories: – Quarantined infectious R(t) – Quarantined susceptible Q(t)

Slammer With DQ

DQ With Large T? T=10sec T=30sec

DQ And Patching?

Patch Only Quarantined Hosts Cleaning I(t) Cleaning R(t)

DOMINO The goal is to build an overlay network so that nodes cooperatively detect intrusion activity – Cooperation reduces the number of false positives Overlay can be used for worm detection Main feature are active-sink nodes that detect traffic to unused IP addresses The reaction is to build blacklists of infected nodes V. Yegneswaran, P. Barford, S. Jha, “Global Intrusion Detection in the DOMINO Overlay System,” NDSS 2004

DOMINO Architecture

Axis nodes collect, aggregate and share data – Nodes in large, trustworthy ISPs – Each node maintains a NIDS and an active sink over large portion of unused IP space Access points grant access to axis nodes after thorough administrative checks Satellite nodes form trees below an axis node, collect information, deliver it to axis nodes and pull relevant information Terrestrial nodes supply daily summaries of port scan data

Information Sharing Every axis node maintains a global and local view of intrusion activity Periodically a node receives summaries from peers which are used to update global view – List of worst offenders grouped per port – Lists of top scanned ports RSA is used to authenticate nodes and signed SHA digests are used to ensure message integrity and authenticity

How Many Nodes We Need? 40 for port summaries 20 for worst offender list

How Frequent Info Exchange? Staleness doesn’t matter much but more frequent lists are better to catch worst offenders

How Long Blacklists? About 1000 IPs are enough

How Close Monitoring Nodes? Blacklists in same /16 space are similar  satellites in /16 space should be grouped under the same axis node and sets of /16 spaces should be randomly distributed among different axis nodes

Automatic Worm Signatures Focus on TCP worms that propagate via scanning Idea: vulnerability exploit is not easily mutable so worm packets should have some common signature Step 1: Select suspicious TCP flows using heuristics Step 2: Generate signatures using content prevalence analysis Kim, H.-A. and Karp, B., Autograph: Toward Automated, Distributed Worm Signature Detection, in the Proceedings of the 13th Usenix Security Symposium (Security 2004), San Diego, CA, August, 2004.

Suspicious Flows Detect scanners as hosts that make many unsuccessful connection attempts (>2) Select their successful flows as suspicious Build suspicious flow pool – When there’s enough flows inside trigger signature generation step

Signature Generation Use most frequent byte sequences across flows as the signature Naïve techniques fail at byte insertion, deletion, reordering Content-based payload partitioning (COPP) – Partition if Rabin fingerprint of a sliding window matches breakmark = content blocks – Configurable parameters: window size, breakmark – Analyze which content blocks appear most frequently and what is the smallest set of those that covers most/all samples in suspicious flow pool

How Well Does it Work? Tested on traces of HTTP traffic interlaced with known worms For large block sizes and large coverage of suspicious flow pool (90-95%) Autograph performs very well – Small false positives and false negatives

Distributed Autograph Would detect more scanners Would produce more data for suspicious flow pool – Reduce false positives and false negatives

Automatic Signatures (approach 2) Detect content prevalence – Some content may vary but some portion of worm remains invariant Detect address dispersion – Same content will be sent from many hosts to many destinations Challenge: how to detect these efficiently (low cost = fast operation) S.Singh, C. Estan, G. Varghese and S. Savage “Automated Worm Fingerprinting,” OSDI 2004

Content Prevalence Detection Hash content + port + proto and use this as key to a table where counters are kept – Content hash is calculated over overlapping blocks of fixed size – Use Rabin fingerprint as hash function – Autograph calculates Rabin fingerprint over variable-length blocks that are non-overlapping – Rabin fingerprint is a hash function that is efficient to recalculate if a portion of the input changes

Address Dispersion Detection Remembering sources and destinations for each content would require too much memory Scaled bitmap: – Sample down input space, e.g., hash into values but only remember those values that hash into 0-31 – Set the bit for the output value (out of 32 bits) – Increase sampling-down factor each time bitmap is full = constant space, flexible counting

How Well Does This Work? Implemented and deployed at UCSD network

How Well Does This Work? Some false positives – Spam, common HTTP protocol headers.. (easily whitelisted) – Popular BitTorrent files (not easily whitelisted) No false negatives – Detected each worm outbreak reported in news – Cross-checked with Snort’s signature detection

Polymorphic Worm Signatures Insight: multiple invariant substrings must be present in all variants of the worm for the exploit to work – Protocol framing (force the vulnerable code down the path where the vulnerability exists) – Return address Substrings not enough = too short Signature: multiple disjoint byte strings – Conjunction of byte strings – Token subsequences (must appear in order) – Bayes-scored substrings (score + threshold) J. Newsome, B. Karp and D. Song, “Polygraph: Automatically Generating Signatures for Polymorphic Worms,” IEEE Security and Privacy Symposium, 2005

Worm Code Structure Invariant bytes: any change makes the worm fail Wildcard bytes: any change has no effect Code bytes: Can be changed using some polymorphic technique and worm will still work – E.g., encryption

Polygraph Architecture All traffic is seen, some is identified as part of suspicious flows and sent to suspicious traffic pool – May contain some good traffic – May contain multiple worms Rest of traffic is sent to good traffic pool Algorithm makes a single pass over pools and generates signatures

Signature Detection Extract tokens (variable length) that occur in at least K samples – Conjuction signature is this set of tokens – To find token-subsequence signatures samples in the pool are aligned in different ways (shifted left or right) so that the maximum-length subsequences are identified – Contiguous tokens are preferred – For Bayes signatures for each token a probability is computed that it is contained by a good or a suspicious flow – use this as a score – Set high value of threshold to avoid false positives

How Well Does This Work? Legitimate traffic traces: HTTP and DNS – Good traffic pool – Some of this traffic mixed with worm traffic to model imperfect separation Worm traffic: Ideally-polymorphic worms generated from 3 known exploits Various tests conducted

How Well Does This Work? When compared with single signature (longest substring) detection, all proposed signatures result in lower false positive rates – False negative rate is always zero if the suspicious pool has at least three samples If some good traffic ends up in suspicious pool – False negative rate is still low – False positive rate is low until noise gets too big If there are multiple worms in suspicious pool and noise – False positives and false negatives are still low

Borrowed from Brent ByungHoon Kang, GMU

A Network of Compromised Computers on the Internet IP locations of the Waledac botnet. Borrowed from Brent ByungHoon Kang, GMU

Botnets Networks of compromised machines under the control of hacker, “bot-master” Used for a variety of malicious purposes: Sending Spam/Phishing s Launching Denial of Service attacks Hosting Servers (e.g., Malware download site) Proxying Services (e.g., FastFlux network) Information Harvesting (credit card, bank credentials, passwords, sensitive data.) Borrowed from Brent ByungHoon Kang, GMU

Botnet with Central Control Server After resolving the IP address for the IRC server, bot-infected machines CONNECT to the server, JOIN a channel, then wait for commands. Borrowed from Brent ByungHoon Kang, GMU

Botnet with Central Control Server The botmaster sends a command to the channel. This will tell the bots to perform an action. Borrowed from Brent ByungHoon Kang, GMU

Botnet with Central Control Server The IRC server sends (broadcasts) the message to bots listening on the channel. Borrowed from Brent ByungHoon Kang, GMU

Botnet with Central Control Server The bots perform the command. In this example: attacking / scanning CNN.COM. Borrowed from Brent ByungHoon Kang, GMU

Botnet Sophistication Fueled by Underground Economy Unfortunately, the detection, analysis and mitigation of botnets has proven to be quite challenging Supported by a thriving underground economy – Professional quality sophistication in creating malware codes – Highly adaptive to existing mitigation efforts such as taking down of central control server. 41 Borrowed from Brent ByungHoon Kang, GMU

Emerging Decentralized Peer to Peer Multi-layered Botnets Traditional botnet communication – Central IRC server for Command & Control (C&C) – Single point of mitigation: C&C Server can be taken down or blacklisted Botnets with peer to peer C&C – No single point of failure. – E.g., Waldedac, Storm, and Nugache Multi-layered architecture to obfuscate and hide control servers in upper tiers. Borrowed from Brent ByungHoon Kang, GMU

Expected Use of DHT P2P Network Publish and Search  Botmaster publishes commands under the key.  Bots are searching for this key periodically  Bots download the commands =>Asynchronous C&C Borrowed from Brent ByungHoon Kang, GMU

Multi-Layered Command and Control Architecture Through P2P  Each Supernode (server) publishes its location (IP address) under the key 1 and key 2  Subcontrollers search for key 1  Subnodes (workers) search for key 2 to open connection to the Supernodes => Synchronous C&C  Borrowed from Brent ByungHoon Kang, GMU

Current Approaches to Botnet Virus Scanner at Local Host – Polymorphic binaries against signature scanning – Not installed even though it is almost free – Rootkit Network Intrusion Detection Systems – Keeping states for network flows – Deep packet inspection is expensive – Deployed at LAN, and not scalable to ISP-level – Requires Well-Trained Net-Security SysAdmin Borrowed from Brent ByungHoon Kang, GMU

46 Conficker infections are still increasing after one year!!! There are millions of computers on the Internet that do not have virus scanner nor IDS There are millions of computers on the Internet that do not have virus scanner nor IDS Borrowed from Brent ByungHoon Kang, GMU

Botnet Enumeration Approach Used for spam blocking, firewall configuration, DNS rewriting, and alerting sys-admins regarding local infections. Fundamentally differs from existing Intrusion Detection System (IDS) approaches – IDS protects local hosts within its perimeter (LAN) – An enumerator would identify both local as well as remote infections Identifying remote infections is crucial – There are numerous computers on the Internet that are not under the protection of IDS-based systems. 47 Borrowed from Brent ByungHoon Kang, GMU

How to Enumerate Botnet Need to know the method and protocols for how a bot communicates with its peers Using sand-box technique – Run bot binary in a controlled environment – Network behaviors are captured/analyzed Investigating the binary code itself – Reversing the binary into high level codes – C&C Protocol knowledge and operation details can be accurately obtained Borrowed from Brent ByungHoon Kang, GMU

Simple Crawler Approach Given network protocol knowledge, crawlers: 1.collect list of initial bootstrap peers into queue 2.choose a peer node from the queue 3.send to the node look-up or get-peer requests 4.add newly discovered peers to the queue 5.repeat 2-5 until no more peer to be contacted Can’t enumerate a node behind NAT/Firewall Would miss bot-infected hosts at home/office! Borrowed from Brent ByungHoon Kang, GMU

Passive P2P Monitor (PPM) Given P2P protocol knowledge that bot uses A collection of “routing-only” nodes that – Act as peer in the P2P network, but – Controlled by us, the defender PPM nodes can observe the traffic from the peer infected hosts PPM node can be contacted by the infected hosts behind NAT/Firewall Borrowed from Brent ByungHoon Kang, GMU

Crawler and Passive P2P Monitor (PPM) Crawler PPM Borrowed from Brent ByungHoon Kang, GMU

Crawler vs. PPM: # of IPs found Borrowed from Brent ByungHoon Kang, GMU

Botnet Enumeration Challenges DHCP NATs Non-uniform bot distribution Churn Most estimates put size of largest botnets at tens of millions of bots – Actual size may be much smaller if we account for all of the above

Fast Flux Botnets use a lot of newly-created domains for phishing and malware delivery Fast flux: changing name-to-IP mapping very quickly, using various IPs to thwart defense attempts to bring down botnet Single-flux: changing name-to-IP mapping for individual machines, e.g., a Web server Double-flux: changing name-to-IP mapping for DNS nameserver too Proxies on compromised nodes fetch content from backend servers

Fast Flux Advantages for the attacker: – Simplicity: only one back end server is needed to deliver content – Layers of protection through disposable proxy nodes – Very resilient to attempts for takedown

Fast Flux Detection Look for domain names where mapping to IP changes often – May be due to load balancing – May have other (non-botnet) cause, e.g., adult content delivery – Easy to fabricate domain names Look for DNS records with short-lived domain names, with lots of A records, lots of NS records and diverse IP addresses (wrt AS and network access type) Look for proxy nodes by poking them

Poking Botnets is Dangerous They have been known to fight back – DDoS IPs that poke them (even if low workers are scanned) They have been known to fabricate data for honeynets – Honeynet is a network of computers that sits in otherwise unused (dark) address space and is meant to be compromised by attackers

Privacy

What is Privacy? Privacy is about PII It is primarily a policy issue Privacy is an issue of user education Make sure users are aware of the potential use of the information they provide Give the user control Privacy is a security issue Security is needed to implement the policy

Security v. Privacy Sometimes conflicting Many security technologies depend on identification Many approaches to privacy depend on hiding one’s identity Sometimes supportive Privacy depends on protecting PII (personally identifiable information) Poor security makes it more difficult to protect such information

Debate on Attribution How much low level information should be kept to help track down cyber attacks Such information can be used to breach privacy assurances How long can such data be kept

Privacy is Not the Only Concern Business Concerns Disclosing Information we think of as privacy- related can divulge business plans Mergers Product plans Investigations Some “private” information is used for authentication SSN Credit card numbers

Aggregation of Data Consider whether it is safe to release information in aggregate Such information is presumably no longer personally identifiable But given partial information, it is sometimes possible to derive other information by combining it with the aggregated data.

Anonymization of Data Consider whether it is safe to release information that has been stripped of so called personal identifiers Such information is presumably no longer personally identifiable What is important is not just anonymity, but linkability If I can link multiple queries, I might be able to infer the identity of the person issuing the query through one query, at which point, all anonymity is lost

Traffic Analysis Even when specifics of communication are hidden, the mere knowledge of communication between parties provides useful information to an adversary E.g. pending mergers or acquisitions Relationships between entities Created visibility of the structure of an organizations Allows some inference about interests

Information for Traffic Analysis Lists of the web sites you visit logs Phone records Perhaps you expose the linkages through web sites like linked in Consider what information remains in the clear when you design security protocols

Network Trace Sharing Researchers need network data – To validate their solutions – To mine and understand trends Sharing network data creates necessary diversity – Enables generalization of results – Creates a lot of privacy concerns – Very few public traffic trace archives (CAIDA, WIDE, LBNL, ITA, PREDICT, CRAWDAD, MIT DARPA)