1. 1 Collaborative Online Passive Monitoring for Internet Quarantine Weidong Cui wdc@EECS.Berkeley.EDU SAHARA Winter Retreat, 2004

2. 2 Motivation Threats to Today’s Internet –Internet worms Code-Red, Nimda, MS-SQL (Slammer/Sapphire), Blaster –DDoS attacks –Email spams Disaster caused by these threats –Millions of PCs cannot work properly Automatic reboot Disconnected by network admins –Critical servers stopped working SQL servers DDoS attacked servers –Network outages Links congested Routers down

3. 3 Internet Quarantine Containing self-propagating malicious code is very important –Internet worms propagation caused huge problems –DDoS attacks rely on a large number of compromised zombies –Email spammers start exploiting compromised machines to forward spam emails To contain worms successfully, we need to [moore03internet] –Automatically detect and activate filtering mechanisms within minutes, –Generate signatures for content filtering –deploy content filtering in a large number of coordinated ISPs

4. 4 Can We Protect Our Own Network against Intruders? Yes, but limited… Network intrusion detection –Misuse detection (signature-based) Detect known malicious attacks very well Cannot detect new attacks without signatures –Anomaly detection Can detect new attacks high false alarm rates due to high variance of incoming traffic Firewalls –Not flexible, usually require human intervention –movable points (laptops) –Distributed firewall is still a research problem

5. 5 Our Idea Why is it hard to detect intruders? –So many of them… –Large variance of behaviors Can we monitor local hosts? –Limited number of them –Network behavior follows some pattern Basic idea –Monitor network behavior of local hosts –Prevent compromised local hosts from infecting others –Generate signatures based on traffic from those hosts

6. 6 Our Approach Detect compromised local hosts in an edge network –Online passively monitor all traffic into/from an edge network –Train a network behavior profile for each host inside the edge network and online update it –Alarm when an end host behaves anomalously –Assumption: the period of normal behavior of end hosts is long enough for this training purpose Generate signatures of malicious code –Redirect traffic from an anomalous host to a honeypot –Create signatures in the honeypot Distribute signatures to other networks –Can leverage on overlay multicast

7. 7 Design Choices Why support the proposed monitoring? –Compromised hosts may infect other hosts inside the edge network Why monitor at gateways of edge networks? –Single monitoring point for inbound and outbound traffic –Moderate traffic load –More information than end hosts –More reliable and harder to be compromised than end hosts

8. 8 Network Behavior Profile (I) Network behavior of an end host can be abstracted as a series of connections to/from that host –TCP connection; each UDP packet is a connection –Each connection can be represented by a vector of one-dimension variables: X=(X 1, X 2,… X n ) Duration, transport protocol, service, outgoing/incoming packet/data size, time since last connection, if the remote host is visited before, etc –Aggregated features of connections # connections/minute –Model of network behavior a multivariate distribution P(X) describes how likely a connection may happen

9. 9 Network Behavior Profile (II) A network behavior profile is an approximation of the multivariate distribution P(X) –Quantify the resolution of each variable Time-of-Day: day time/night; Day-of-Week: weekday/weekend –Select a subset of one-dimensional marginal and conditional distributions for approximating the multivariate distribution P(X)=P(X 1 )P(X 2 )P(X 3 |X 2 ) –Use a set of histograms to model one-dimensional distributions Histograms: nonparametric, each to update

10. 10 Proof-of-Concepts We do not have concrete results for anomaly detection. We need to find features which can be used to differentiate normal and anomalous network behavior. –Outgoing connections –New targets –Different services Data: 2 weeks (11/09/03-11/25/03) tcpdump traces of our group (40 active hosts) We will show network behavior of 4 end hosts which indicate some possible ways to do network anomaly detection.

11. 11 Network Behavior: TCP Connection Speed

12. 12 Network Behavior: New Targets

13. 13 Network Behavior: Services

14. 14 Discussion Is it possible to differentiate between normal and anomalous network behavior of end hosts? –Network behavior of most end hosts are relatively stable? –Client vs. Server –New service release –Planet lab hosts Coordination among edge networks –What information to share? –How to make decision based on shared information? Statistical learning theory for anomaly detection –Most data is normal behavior –Online update/detection Trace collection –Departmental/campus network –Commercial ISPs?

15. 15 Related Work Virus Throttle [williamson03implementing] –Limit/Watch the speed of connection made by an end host to detect if it’s compromised –Static: 1 connection/second –Only look at connection speed –Implemented at end hosts: maybe removed by malicious code Online Fraud Detection [lambert00detecting] –Online data mining of a stream of transactions for customer patterns –fraud detection applied to cell phones and credit cards Honeycomb [kreibich03honeycomb] –Honeypots: Decoy computing resources set up for monitoring and logging malicious activities –String-based pattern detection

16. 16 Summary Problem –Self-propagating malicious code is big threat to Today’s Internet Idea –Monitor network behavior of local hosts –Prevent compromised local hosts from infecting others –Generate signatures based on traffic from those hosts Approach –Collaborative online passive monitoring at edge networks –Redirect traffic to honeypots to create signatures Future work –Investigate anomaly detection algorithms on real world data –Study coordinated analysis algorithms –Efficient passive monitoring mechanism

17. 17 References [moore03internet] –Internet Quarantine: Requirements for Containing Self- Propagating Code –http://www.caida.org/outreach/papers/2003/quarantine /worm-infocom03.pdf [williamson03implementing] –Implementing and Testing a Virus Throttle –http://www.hpl.hp.com/techreports/2003/HPL-2003- 103.pdf [lambert00detecting] –Detecting Fraud in the Real World –http://cm.bell-labs.com/stat/doc/hmds.pdf [kreibich03honeycomb] –Honeycomb – Creating Intrusion Detection Signatures Using Honeypots –http://nms.lcs.mit.edu/HotNets- II/papers/honeycomb.pdf