1. Multi-Core Packet Scattering to Disentangle Performance Bottlenecks Yehuda Afek Tel-Aviv University

2. Anat Bremler-Barr David HayYotam Harchol Yaron Koral Joint work with This work was supported by European Research Council (ERC) Starting Grant no. 259085

3. Deep Packet Inspection IPS/IDS/FW Heaviest processing part: Search for malicious patterns in the payload 1.Pipeline multi-core, not efficient. – Imbalance of pipeline stations, DPI much heavier 2.Parallel multi-core?

4. Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

5. Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

6. Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

7. Multi-Core Deep Packet Inspection (DPI) Option 1: Each core a subset of patterns Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 Pattern Set 1 Pattern Set 2 Pattern Set 3 Pattern Set 4

8. Multi-Core Deep Packet Inspection (DPI) Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 DPI

9. Multi-Core Deep Packet Inspection (DPI) Option 2: All cores are the same, Load-balance between cores Core 1 Core 2 Core 3 Core 4 DPI

10. Complexity DoS Attack Over NIDS Easy to craft – very hard to process packets 2 Steps attack: Attacker Internet 2. Steal CC. 1. Kill IPS/FW

11. Attack on Security Elements Combined Attack: DDoS on Security Element exposed the network – theft of customers’ information

12. Attack on Snort The most widely deployed IDS/IPS worldwide. Heavy packets rate

13. OUR GOAL: A multi-core system architecture, which is robust against complexity DDoS attacks

14. Airline Desk Example

15. A flight ticket

16. Airline Desk Example An isle seat near window!! Three carry handbags !!! Doesn’t like food!!! Can’t find passport!! Overweight!!!

17. Airline Desk Example

18. Domain Properties 1.Heavy & Light customers. 2.Easy detection of heavy customers. 3.Moving customers between queues is cheap. 4.Heavy customers have special more efficient processing method. Special training packets

19. Some packets are much “heavier” than others The Snort-attack experiment Property 1 in Snort Attack

20. DPI mechanism is a main bottleneck in Snort Allows single step for each input symbol Holds transition for each alphabet symbol Snort uses Aho-Corasick DFA Heavy Packet Fast & Huge Best for normal traffic Exposed to cache-miss attack Best for normal traffic Exposed to cache-miss attack

21. Crafting HEAVY packets Snort patterns DatabaseMalicious pkts Factory Chop last 2 bytes

22. Snort-Attack Experiment Cache Main Memory Normal TrafficAttack Scenario Does not require many packets!!!

23. The General Case: Complexity Attacks Trivial to Craft --- Hard to process packets Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

24. Property 2 in Snort Attack Detecting heavy packets is feasible

25. How Do We Detect? May be quickly classified Common states Claim: the general case in complexity attacks!!! threshold Percent non-common states

26. How Do We Detect? Common States Non Common States Heavy packet : # Not Common States # Common States ≤ α After at least 20 bytes

27. Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

28. System Architecture Processor Chip Core #8 NIC Core #1 Q Core #2 Q Q Q Q Detects heavy packets Core #9 Core #10 Routine Mode: Load balance between cores

29. System Architecture Processor Chip Core #8 Dedicated Core #9 NIC Core #1 Q Core #2 Q Q Q B Dedicated Core #10 B Q Detects heavy packets Alert Mode: Dedicated cores for heavy packets Others detect and move heavy to Dedicated. BB

30. Inter-Thread Communication Heavy packets queues are non-blocking – no locking mechanisms are used Writers write to the queue in cycles (incrementing every full round) – Cycle ID is stored before and after the passed message Readers read in the opposite direction: – If left ID != right ID then writer is now writing – wait and retry – If left ID > expected ID  overflow

31. Inter-Thread Communication Non-blocking IN-queues – Only one thread accesses Dedicated queues blocking (using test&set locks) – Non-dedicated threads “steal” packets from the HoL when sending a heavy packet Processor Chip Core #8 Dedicated Core #9 NIC Core #1 Q Core #2 Q Q Q B Dedicated Core #10 B Q BB

32. Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

33. Snort uses Aho-Corasick DFA

34. Full Matrix vs. Compressed Heavy packets rate

35. Domain Properties 1.Heavy & Light packets. 2.Easy detection of heavy packets 3.Moving packets between queues is cheap. 4.Heavy packets have special more efficient processing method.

36. Experimental Results

37. System Throughput Over Time Reaction time can be smaller

38. Different Algorithms Goodput

39. Additional Application for MCA 2 The Hybrid-FA-attack experiment

40. Hybrid-FA Space-efficient data structure for regular expression matching Faster than NFA Structure: – Head DFA – Border states – Tail DFAs More than one state can be active at the same time! s0s0 s7s7 s 12 s1s1 s2s2 s3s3 s5s5 s4s4 C C E D B ED s 14 s 13 s6s6 D s8s8 B s9s9 C s 10 A s 11 B A A.* [^\n]*

41. Hybrid-FA Attack Normal TrafficAttack Scenario Again: Does not require many packets!!! s0s0 s7s7 s 12 s1s1 s2s2 s3s3 s5s5 s4s4 C C E D B ED s 14 s 13 s6s6 D s8s8 B s9s9 C s 10 A s 11 B A A.* [^\n]* s0s0 s7s7 s8s8 s9s9 s 10 s 11 s 12 s2s2 s5s5 s 13 Input: CDBBCAB

42. Heavy Packet Detection threshold

43. MCA 2 With Hybrid-FA

44. Concluding Remarks A multi-core system architecture Robustness against complexity DDoS attacks In this talk we focused on specific NIDS and complexity attack – MCA 2 can handle more NIDS complexity attacks, like the Bro Lazy-FA We believe this approach can be generalized (outside the scope of NIDS)

45. Thank You!! Deep packet inspection