1. Modeling IDS using hybrid intelligent systems
Peddabachigari et al. (2007)
Presenter: Andy Tang

2. Background
Intrusion defense system (IDS): 2nd line of defense, after authentication / encryption
Two main types:
Misuse intrusion – well-defined patterns of attack, encoded in advance
Anomaly intrusion – deviation from baseline behaviors, approximated by differences
Machine learning (ML) paradigm of IDS: model intrusion detection as a binary classification task with input features from various sensors
Neural nets (NN)
Genetic programming (GP)
Fuzzy logic (FL)
Decision trees (DT)
Support vector machines (SVM)

3. General IDS model

4. General IDS model Sequence of events E(1) – E(2) – … – E(n)
⇒ E(n)
General IDS model

5. Updated rules (statistical or rule-based) dependent on recent behaviors
General IDS model

6. Updates baseline behaviors
General IDS model

7. General IDS model Some Limitations:
A misuse detector paradigm, requires complimentary anomaly detection.
Rule-based (ES), limited representation power w.r.t. transition patterns.
Does not scale with complex (i.e., non-linear) sequences of behaviors.
Depends on quality and quantity of pre-defined rules.
General IDS model

8. Goal: represent characteristics of intrusion behavior by adaptive behavioral “models”.
ML style of IDS

9. Goal: represent characteristics of intrusion behavior by adaptive behavioral “models”.
End-to-end pipeline from input data streams to classification rules.
Direct feedback from empirical performance to adjust extracted patterns and model parameters.
ML style of IDS

10. ML style of IDS Neural Nets
Inputs: a sliding window of w previous records.
Output: classification rule
Learning Mechanism: backpropagation by vector-jacobian products.
“Hidden layers” are intermediary transformed feature spaces ≈ behavioral patterns.
ML style of IDS

11. ML style of IDS Fuzzy logic (FL)
Inputs: set of rules, operators and knowledge base(i.e., user-specified graph dependencies).
Output: classification rule
Learning Mechanism: linear combinations of pre-defined rules on transformed features.
ML style of IDS

12. ML style of IDS Support Vector Machines (SVM) Inputs: feature space
Output: classification rule
Learning Mechanism: linear combinations of support vectors based
Deals w/ nonlinearity in decision space by transforming the feature dimensions (Kernel Trick)
More on this later.
ML style of IDS

13. SVM Background

14. SVM Background

15. SVM Background e = vector of 1’s
Q is SPD, also called the “kernel” matrix.
SVM Background

16. Kernel Trick

17. SVM Cost Sample complexity:
Runtime of QP-solver for support vector selection:
w/ sparse feature space
w/ dense feature space
Key assumption in paper: Kernel selection (and support vector computation separate from “runtime” of SVM during training / testing)!
SVM Cost

18. ML style of IDS Decision Trees (DT)
Inputs: joint feature and output space
Output: a set of decision rules in the joint space
Learning Mechanism: weighted combination of decision rules
User redefines the max number of leaves, leading to fixed complexity
Already an ensemble system!
ML style of IDS

19. Decision tree example

20. Others:
Evolutionary computation (EC) and genetic programs (GP)
Hybrid system: DT-SVM
ML style of IDS

21. Hybrid DT-SVM mode.

22. Experiments
KDD cup 1999 data: 5 million connection records, 24 attack types categorized into 4 classes of intrusions:
Denial of service (DOS): sequence of behaviors leading to overload of resources for service.
Remote to user (R2L): static attack sending packages over network to gain access in vulnerable nodes.
User to Root (U2R): static attack with access to normal user account, gains root access by system vulnerabilities.
Probing: sends sequential information over network to identify new vulnerabilities.

23. SVM vs. DT vs. DT-SVM Experiment Design:
Input features: 41 attributes for each connection (e.g., content features, no. of failed logins)
Labels: multilabel (1 for each class of attack) for decision trees, multiclass (5 classes for SVM and DT-SVM)
“Ensemble” = DT-SVM + SVM + DT.
Train-test split over 11,982 records (5092 Tr, 6890 Te)
Kernel selection for SVM on training sets (Polynomial p=2) selected.
SVM vs. DT vs. DT-SVM

24. DT vs. SVM performance

25. Comparison w/ DT-SVM and ensemble
Key observations:
Performance gain almost exclusively from DT
No insight into statistical significance was investigated
1 train-test split for each task, failure to provide variance of performance
Remark: DT is already an ensemble approach. Lack of systematic approach for picking ensemble combinations.
Comparison w/ DT-SVM and ensemble

26. Moving Forward: Addition of Transfer Learning?

27. Discussion
1) It is interesting to consider how this ensemble combination (DT-SVM) would fair against other ensemble combinations: decision tree with neural nets (DT-NN), SVM-NN, RBF-SVM with Linear-SVM, ... etc. Do you think there is a general rule for choosing ensemble pairs (or feature / kernel spaces and loss functions) that best compliment each other?
2) What feasibility issues (e.g., cost, runtime issues) may occur when trying to implement this ensemble IDS in CAVs?
3) Why do you think the U2R attack predictions had the lowest performance across all the learning models? Can you think of other classes of attacks similar to U2R that may be difficult for the proposed method to identify correctly?

28. Q & A