1. An Enhanced Support Vector Machine Model for Intrusion Detection
J. T. Yao, S. L. Zhao, L. Fan
Department of Computer Science
University of Regina

2. Intrusion Detection Systems
Intrusion detection: the art of detecting inappropriate, incorrect, or anomalous activity.
Intrusion detection systems
A set of processes, procedures, tools, software, hardware and databases having intrusion detection technologies that fit together.
Misuse: attack originates from the internal network
Intrusion: attacks from the outside
9/16/2018
J T Yao: Detection

3. IDS Functional Components
Information Source
Provides a stream of event records.
Analysis Engine
Analyzes event records and detects intrusion.
Decision Maker
Decides the reactions for intrusions.
9/16/2018
J T Yao: Detection

4. Detection Methods Misuse detection Anomaly detection
Detecting intrusion by known intrusion signatures.
Anomaly detection
Mining normal event patterns from event records, then use these patterns to classify normal and intrusion events.
9/16/2018
J T Yao: Detection

5. Candidate AI Techniques
Expert Systems
Hidden Markov Model
Fuzzy logic
Classification
Support Vector Machines (SVM)
9/16/2018
J T Yao: Detection

6. Support Vector Machines
A machine learning method based on statistical learning theories.
Classifies data by a set of support vectors that represent data patterns.
Finds a discriminant function that classify new data.
9/16/2018
J T Yao: Detection

7. Benefits of Using SVM Good generalization ability
Capability of handling a large number of features
9/16/2018
J T Yao: Detection

8. Problem of SVM on IDS All features are treated equally
Noise features (some feature cause noise during classification)
Redundant features
High feature numbers affect performance
Training process
Detection process
9/16/2018
J T Yao: Detection

9. Thoughts of Solution
Reducing feature number while keep the useful information
Calculating the importance of features
Treating features differently based on their importance
9/16/2018
J T Yao: Detection

10. An Enhancing SVM Model Using Rough Set to calculate reducts
Calculate feature weights from reducts
Remove redundant features based on weights
Apply weights to SVM kernel
9/16/2018
J T Yao: Detection

11. Calculate Weights from Reducts
The principles of calculation are
If a feature is not in any reducts, its weight=0.
More times a feature appears in reducts, more important the feature is.
The fewer the number of features in a reduct, the more important these feature are.
9/16/2018
J T Yao: Detection

12. Apply Weights to Kernel Function
The training result of SVM is
where is the number of training records, is the Lagrange multipliers, is the label associated with the training data, is a constant, is the kernel function and is called a set of Support Vectors, is a bias term. Weight w is a diagonal matrix
9/16/2018
J T Yao: Detection

13. Weights Independent to Kernel Functions
Could apply weights to any known kernel functions
Restrict W>0 to make sure enhanced kernel function satisfies Mercer’s Condition
9/16/2018
J T Yao: Detection

14. Experiment Procedures
9/16/2018
J T Yao: Detection

15. Experiment Data Set
KDD (Knowledge Discovery in Databases) Cup 1999 data set.
Feature-value format.
41 features for each record.
Original data set contains 744 MB data with 4,940,000 connection records.
9/16/2018
J T Yao: Detection

16. Experiment Data Set 2 UNM (University of New Mexico) data set.
Sequence-based.
Generate a trace each time a user access a certain UNIX process.
9/16/2018
J T Yao: Detection

17. KDD Training results of conventional SVM with different value of gamma (table 1)
Exp1
Exp2
Exp3
Training record #
50,000
Feature # 41
Kernel type
RBF
Value of
Generated SV #
6,948
1,868
1,057
9/16/2018
J T Yao: Detection

18. KDD Test results of conventional SVM with different values of gamma (table 2)
Exp1
Exp2
Exp3
Test record #
10,000
Feature # 41
Value of
# of misclassified 44 63
211
Accuracy
99.56%
99.37%
97.89%
False Positive # 37 52
176
False Negative # 7 11 35
CPU seconds
49.53
11.34
8.32
9/16/2018
J T Yao: Detection

19. Comparisons of the experimental results on the KDD dataset (table 3)
Test Result
CPU
Test set 1
Conventional SVM
10,000 41
99.82
7.69
222.28
Enhanced SVM 16
99.86
6.39
77.63
Improvement
60.0%
0.4%
16.9%
66.0%
Test set 2
99.80
8.25
227.03
99.85
6.91
78.93
0.5%
16.2%
65.0%
Test set 3
99.88
7.45
230.27
99.91
5.49
77.85
60%
0.3%
26.3%
9/16/2018
J T Yao: Detection

20. Comparisons of the experimental results on the UNM lpr dataset (table 4)
Test Result
CPU
Test set 1
Conventional SVM
2,000
467
100
1.62
Enhanced SVM 9
0.28
Improvement
98%
83%
Test set 2
1.71
0.29
Test set 3
1.59
0.25
84%
9/16/2018
J T Yao: Detection

21. Experiment Results
Larger value of results a larger number of Support Vectors generated.
Larger number of SVs results in higher detection accuracy and higher computation costs.
Improvement of enhanced SVM is consistent for all the six test sets
9/16/2018
J T Yao: Detection

22. Experiment Results 2
Enhanced SVM outperforms the conventional SVM in precision, false negative rate and CPU time for KDD dataset.
Enhanced SVM is 80% faster for lpr dataset.
9/16/2018
J T Yao: Detection

23. Experiment Results
Although generated from a small training set, the decision boundary is consistent for whole data set
The test results show little difference between small and full size of training set, which prove the good generalization ability of SVM.
9/16/2018
J T Yao: Detection

24. Conclusion An enhanced SVM model is introduced.
Features are reduced and weighted.
It has good generalization ability.
It has better performance in two experiments.
9/16/2018
J T Yao: Detection

25. An Enhanced Support Vector Machine Model for Intrusion Detection
J. T. Yao, S. L. Zhao, L. Fan
Department of Computer Science
University of Regina