Automatically Generating Models for Botnet Detection Presenter: 葉倚任 Authors: Peter Wurzinger, Leyla Bilge, Thorsten Holz, Jan Goebel, Christopher Kruegel, Engin Kirda European Symposium on Research in Computer Security (ESORICS'09)ESORICS'09

Outline  Introduction  System Overview  Model Generation Data  Generating Detection Models  Evaluation  Conclusion

Introduction  Two main kinds of network-based detection system Vertical correlation technique  Detection of individual bots  Checking traffic patterns, content of C&C traffic, and bot related activities.  Require prior knowledge of C&C channels and propagation vectors of bot Horizontal correlation technique  Detection of a group of bots  Based on network traffic  Require that at two bots in the monitor networks

Introduction (cont ’ d)  Characteristic behavior of a bot Receive commands from botmater Carry out some actions in response to these commands  This paper proposed a two-stage detection model to leverage these two characteristics  In the experiments, the authors generated 18 different bot families. 16 controlled via IRC, One via HTTP (Kraken) One via a peer-to-peer network (Storm Worm).

System Overview  Input of the system A collection of bot binaries  Launch a bot in a controlled environment and record its network activities (traces)  Identify the commands that this bot receives as well as its corresponding responses  Translate observations into detection models  Output of the system Detection models for different bot families

Detecting Procedure  Stateful model (two-stage detection) 1.Checking if a bot command is sent 2.If yes in stage 1, checking if the responses is above a threshold or not (e.g., the number of new connections opened by a host)  Use content-based specifications to model commands (comparable to intrusion detection signatures)  Use network-based specifications to model responses (comparable to anomaly detection)

Model Generation Data  Run each bot binary for a period of several days  Locating bot responses  Finding commands  Extracting model generation data

Locating bot responses  Assumption: bot responses that lead to a change in network behavior  Partition network traffic into consecutive time intervals of equal length  For each time interval, define 8 normalized features (called traffic profile):

Locating bot responses (cont ’ d)  Convert the traffic profiles (vectors) into time series data d(t) as follows: where ε is the sliding window size  Locate bot responses by using CUSUM algorithm  ε = 5 and an interval of 50 seconds delivered the best results in the tests

Finding bot commands  After locating bot responses, a small section of network traffic (snippet) is extracted for each response  Cluster those traffic snippets that lead to similar responses

Extracting model generation data  Extract two pieces of information the subsequent model generation step  A snippet Contains 90 seconds of traffic  Plus last 30 seconds of the previous one and first 10 seconds of the following one A snippet  Average of the traffic profile vectors This period is the time from the start of the current response to the next change in behavior

Generating Detection Models  Command model generation  Response model generation

Command model generation  The goal is to identify common elements in a particular behavior cluster  First, apply a second clustering refinement step that groups similar network packet payloads within each behavior cluster  The longest common subsequence algorithm is applied to each set of similar payloads  Generate one token sequence per set

Response model generation  Compute the element-wise average of the individual behavior profiles for a behavior cluster  Give minimal bounds for certain network features 1,000 for UDP packets 100 for HTTP packets 10 for SMTP packets 20 for different IPs  A detection model is not generated if a response profile exceeds none of these thresholds

Evaluation  Collected a set of 416 different (based on MD5 hash) bot samples From Anubis The collection period was more than 8 months Each bot produce a traffic trace with a length of five days  Divided into families of bots 16 different IRC bot families (with 356 traffic traces) One HTTP bot family (with 60 traffic traces) One p2p bot family (Storm Worm, with 30 traffic traces)

Detection Capability  Split our set of 446 network traces into training sets and test sets  Each training set contained 25% of one bot family's traces  This procedure was performed four times per family (four-fold cross validation)

Real-World Deployment  Deployed a sensor In front of the residential homes of RWTH Aachen University At a Greek university network  The total traffic is in the order of 94 billion network packets over a period of over three months at two different sites in Europe

Real-World Deployment  In the Greek network, most cases were false positives.  BotHunter w/o Blacklist means BotHunter without blacklists of known DNS names and IP addresses  The detection rate of BotHunter w/o Blacklist in the detection capacity experiment drops to 39%

Conclusion  This paper proposed a two-stage detection method which included a command model and a response model  Automatically derives signatures for the bot commands and network-level specifications for the bot responses  Can generate models for IRC bots, HTTP bots, and even P2P bots such as Storm