A Statistical Anomaly Detection Technique based on Three Different Network Features Yuji Waizumi Tohoku Univ.

Background The Internet has entered the business world Need to protect information and systems from hackers and attacks Network security has been becoming important issue Many intrusion/attack detection methods has been proposed

Intrusion Detection System Two major detection principles: Signature Detection  Attempts to flag behavior that is close to some previously defined pattern signature of a known intrusion Anomaly Detection  Attempts to quantify the usual or acceptable behavior and flags other irregular behavior as potentially intrusive.

Motivation Anomaly detection system Pro: can detect unknown attacks Con: many false positives Improve the performance of Anomaly detection system Analyze the characteristics of attacks Propose method to construct features as numerical values from network traffic Construct detection system using the features

Classification of Attacks DARPA Intrusion Detection Evaluation DoS: Denial of Service Probe: Surveillance of Targets Remote to Local(R2L), User to Root(U2R): Unauthorized Access to a Host or Super User

Re-classification of Attacks Classification by Traffic Characteristics DoS, Probe  Traffic Quantity  Access Range Probe  Structure of Communication Flows DoS, R2L, U2R  Contents of Communications To detect attacks with above characteristics, it is necessary to construct features corresponding those classes.

Network Traffic Feature Numerical values(vectors) expressing state of traffic We propose three different network feature sets Based of re-classification of attacks Analyzed independently

Time Slot Feature (34 dimension) Count various packets, flags, transmission and reception bytes, and port variety by a unit time Estimate scale and range of attacks Target Probe (Scan) DoS Each slot is expressed as a vector Ex) (TCP,icmp,SYN,FIN,RST,UDP,DNS, … )

Examples (Time Slot Feature) normal traffic only rst flag (port 21) rst flag (port 23) ftp scantelnet scan Vector element Element value Values are regularizes as mean=0, variance=1.0

Flow Counting Feature Flow is specified by (srcIP, dstIP, srcPort,dstPort,protocol) Count packets, flags, transmission and reception bytes in a flow Target Scan with illegal flags Ports used as backdoors TCP:19 dim., UDP:7 dim.

Examples (Flow Counting Feature) Normal traffic Port sweep(scan) Decrease of SYN packet Vector element Element value Specific packets of attacks are extremely high and low.

Flow Payload Feature Represent content of communication Histogram of character codes of a flow Count 8bit-unit(256 class) Transmission and reception are counted independently (total 512 class) Target Buffer overflow Malicious code

Examples (Flow Payload Feature) Specific character of attacks are extremely high and low. Normal traffic imap attack

Modeling Normal Behavior Each packet appears based on protocol Correlations between elements of the feature vectors Profile based on correlations can represent normal behavior of network traffic

Principal Component Analysis:PCA Extract correlation among samples as Principal Component Principal Component lay along sample distribution Principal Component Non-correlated data

Discriminant Function Projection Distance Principal Component Anomaly sample Projection Distance Long Distant Samples: Unordinary traffic Break Correlation Detection Criterion

Detection Algorithm Independent Detection The three features are used for PCA independently "Logical OR" operation for detection alerts by each feature Time Slot Flow Counting Flow Payload Features Network Traffic PCA Alert OR Alert

Performance Evaluation Two Examine Scenario Scenario1  Learn Week1 and 3  Test Week4 and 5 Scenario2  Learn Week 4 and 5  Test Week 4 and 5  More Practical Situation Real network traffic may include attack traffic Criterion for Evaluation Detection rate when number of miss-detection (false positive) per day is 10

Data Set 1999 DARPA off-line intrusion detection evaluation test set Contain 5 weeks data (from Monday to Friday) Week1,3: Normal traffic only Week2: Including attacks (for learning) Week4,5: Including attacks (for testing)

Scenario 1 Result # of detection # of target Detection rate Proposed Method % NETAD % Forensics % Expert % Expert % Dmine %

Scenario 2 Result # of detection # of target Detection rate Proposed Method % NETAD % NETAD Use IP address as white list Overfit learning data Proposed Method Independent of IP address Evaluate only anomaly of traffic

Detection Results every Features （ FP ）（ FC ） （ TS ） 3 （ TS ） & （ FC ） & （ FP ） 40 Flow Payload(FP ） 38 Flow Counting （ FC ） 2737 Time Slot Feature （ TS ） （ FP ） （ FC ）（ TS ） 5 （ TS ） & （ FC ） & （ FP ） 44 Flow Payload Feature(FP) 613 Flow Counting Feature(FC) 5922Time Slot Feature(TS) Scenario 1 Scenario 2 # of Detection by both TS & FP # of Detection by FP only # of Detection by all Three Features Low detection overlap Each feature detect different characteristic attacks

Conclusion For network security Classification attacks into three types Construct three features corresponding to three attack characteristics Detection method with PCA  Learning the three features independently Higher detection accuracy  With samples including attacks