Detecting DDoS Attacks on ISP Networks Ashwin Bharambe Carnegie Mellon University Joint work with: Aditya Akella, Mike Reiter and Srinivasan Seshan.

Slides:



Advertisements
Similar presentations
New Directions in Traffic Measurement and Accounting Cristian Estan – UCSD George Varghese - UCSD Reviewed by Michela Becchi Discussion Leaders Andrew.
Advertisements

Data Streaming Algorithms for Accurate and Efficient Measurement of Traffic and Flow Matrices Qi Zhao*, Abhishek Kumar*, Jia Wang + and Jun (Jim) Xu* *College.
ABSTRACT We consider the problem of computing information theoretic functions such as entropy on a data stream, using sublinear space. Our first result.
A Fast and Compact Method for Unveiling Significant Patterns in High-Speed Networks Tian Bu 1, Jin Cao 1, Aiyou Chen 1, Patrick P. C. Lee 2 Bell Labs,
Estimating TCP Latency Approximately with Passive Measurements Sriharsha Gangam, Jaideep Chandrashekar, Ítalo Cunha, Jim Kurose.
Efficient Constraint Monitoring Using Adaptive Thresholds Srinivas Kashyap, IBM T. J. Watson Research Center Jeyashankar Ramamirtham, Netcore Solutions.
Detectability of Traffic Anomalies in Two Adjacent Networks Augustin Soule, Haakon Ringberg, Fernando Silveira, Jennifer Rexford, Christophe Diot.
Fast, Memory-Efficient Traffic Estimation by Coincidence Counting Fang Hao 1, Murali Kodialam 1, T. V. Lakshman 1, Hui Zhang 2, 1 Bell Labs, Lucent Technologies.
SKELETON BASED PERFORMANCE PREDICTION ON SHARED NETWORKS Sukhdeep Sodhi Microsoft Corp Jaspal Subhlok University of Houston.
Packet Caches on Routers: The Implications of Universal Redundant Traffic Elimination Ashok Anand, Archit Gupta, Aditya Akella University of Wisconsin,
Polymorphic blending attacks Prahlad Fogla et al USENIX 2006 Presented By Himanshu Pagey.
Packet Caches on Routers: The Implications of Universal Redundant Traffic Elimination Ashok Anand, Archit Gupta, Aditya Akella University of Wisconsin,
Edith C. H. Ngai1, Jiangchuan Liu2, and Michael R. Lyu1
Detecting Traffic Differentiation in Backbone ISPs with NetPolice Ying Zhang Zhuoqing Morley Mao Ming Zhang.
Streaming Algorithms for Robust, Real- Time Detection of DDoS Attacks S. Ganguly, M. Garofalakis, R. Rastogi, K. Sabnani Krishan Sabnani Bell Labs Research.
The Impact of False Sharing on Shared Congestion Management Aditya Akella and Srinivasan Seshan (Computer Science Department, Carnegie Mellon University)
An Effective Placement of Detection Systems for Distributed Attack Detection in Large Scale Networks Telecommunication and Security LAB. Dept. of Industrial.
1 Traffic Engineering for ISP Networks Jennifer Rexford IP Network Management and Performance AT&T Labs - Research; Florham Park, NJ
Communication-Efficient Distributed Monitoring of Thresholded Counts Ram Keralapura, UC-Davis Graham Cormode, Bell Labs Jai Ramamirtham, Bell Labs.
Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.
Time-Decaying Sketches for Sensor Data Aggregation Graham Cormode AT&T Labs, Research Srikanta Tirthapura Dept. of Electrical and Computer Engineering.
“On Scalable Attack Detection in the Network” Ramana Rao Kompella, Sumeet Singh, and George Varghese Presented by Nadine Sundquist.
Cumulative Violation For any window size  t  Communication-Efficient Tracking for Distributed Cumulative Triggers Ling Huang* Minos Garofalakis.
02/06/2006ecs236 winter Intrusion Detection ecs236 Winter 2006: Intrusion Detection #4: Anomaly Detection for Internet Routing Dr. S. Felix Wu Computer.
Measurement and Monitoring Nick Feamster Georgia Tech.
ANOMALY DETECTION AND CHARACTERIZATION: LEARNING AND EXPERIANCE YAN CHEN – MATT MODAFF – AARON BEACH.
EL 933 Final Project Presentation Combining Filtering and Statistical Methods for Anomaly Detection Augustin Soule Kav´e SalamatianNina Taft.
Tradeoffs in CDN Designs for Throughput Oriented Traffic Minlan Yu University of Southern California 1 Joint work with Wenjie Jiang, Haoyuan Li, and Ion.
Not All Microseconds are Equal: Fine-Grained Per-Flow Measurements with Reference Latency Interpolation Myungjin Lee †, Nick Duffield‡, Ramana Rao Kompella†
Intrusion and Anomaly Detection in Network Traffic Streams: Checking and Machine Learning Approaches ONR MURI area: High Confidence Real-Time Misuse and.
Tomo-gravity Yin ZhangMatthew Roughan Nick DuffieldAlbert Greenberg “A Northern NJ Research Lab” ACM.
Distributed Network Intrusion Detection An Immunological Approach Steven Hofmeyr Stephanie Forrest Patrik D’haeseleer Dept. of Computer Science University.
Computer Security: Principles and Practice First Edition by William Stallings and Lawrie Brown Lecture slides by Lawrie Brown Chapter 8 – Denial of Service.
Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA
Network Intrusion Detection Using Random Forests Jiong Zhang Mohammad Zulkernine School of Computing Queen's University Kingston, Ontario, Canada.
Intrusion Detection Dan Fleck CS 469: Security Engineering These slides are modified with permission from Bill Young (Univ of Texas) Coming up: Intrusion.
SIGCOMM 2002 New Directions in Traffic Measurement and Accounting Focusing on the Elephants, Ignoring the Mice Cristian Estan and George Varghese University.
Distributed Denial of Service CRyptography Applications Bistro Presented by Lingxuan Hu April 15, 2004.
Scalable and Efficient Data Streaming Algorithms for Detecting Common Content in Internet Traffic Minho Sung Networking & Telecommunications Group College.
Source-End Defense System against DDoS attacks Fu-Yuan Lee, Shiuhpyng Shieh, Jui-Ting Shieh and Sheng Hsuan Wang Distributed System and Network Security.
New Streaming Algorithms for Fast Detection of Superspreaders Shobha Venkataraman* Joint work with: Dawn Song*, Phillip Gibbons ¶,
DoWitcher: Effective Worm Detection and Containment in the Internet Core S. Ranjan et. al in INFOCOM 2007 Presented by: Sailesh Kumar.
Aditya Akella The Performance Benefits of Multihoming Aditya Akella CMU With Bruce Maggs, Srini Seshan, Anees Shaikh and Ramesh Sitaraman.
Efficient Elastic Burst Detection in Data Streams Yunyue Zhu and Dennis Shasha Department of Computer Science Courant Institute of Mathematical Sciences.
A Firewall for Routers: Protecting Against Routing Misbehavior1 June 26, A Firewall for Routers: Protecting Against Routing Misbehavior Jia Wang.
BGP topics to be discussed in the next few weeks: –Excessive route update –Routing instability –BGP policy issues –BGP route slow convergence problem –Interaction.
Resource/Accuracy Tradeoffs in Software-Defined Measurement Masoud Moshref, Minlan Yu, Ramesh Govindan HotSDN’13.
1 LD-Sketch: A Distributed Sketching Design for Accurate and Scalable Anomaly Detection in Network Data Streams Qun Huang and Patrick P. C. Lee The Chinese.
A Light-Weight Distributed Scheme for Detecting IP Prefix Hijacks in Real-Time Lusheng Ji†, Joint work with Changxi Zheng‡, Dan Pei†, Jia Wang†, Paul Francis‡
Detection of Routing Loops and Analysis of Its Causes Sue Moon Dept. of Computer Science KAIST Joint work with Urs Hengartner, Ashwin Sridharan, Richard.
Online Identification of Hierarchical Heavy Hitters Yin Zhang Joint work with Sumeet SinghSubhabrata Sen Nick DuffieldCarsten Lund.
April 4th, 2002George Wai Wong1 Deriving IP Traffic Demands for an ISP Backbone Network Prepared for EECE565 – Data Communications.
Packet-Marking Scheme for DDoS Attack Prevention
Architectural Impact of Stateful Networking Applications Javier Verdú, Jorge García Mario Nemirovsky, Mateo Valero The 1st Symposium on Architectures for.
Distributed Denial-of-Service Attack Detection (and Mitigation?) Mukesh Agarwal, Aditya Akella, Ashwin Bharambe.
Consensus Extraction from Heterogeneous Detectors to Improve Performance over Network Traffic Anomaly Detection Jing Gao 1, Wei Fan 2, Deepak Turaga 2,
Taming Internet Traffic Some notes on modeling the wild nature of OD flows Augustin Soule Kavé Salamatian Antonio Nucci Nina Taft Univ. Paris VI Sprintlabs.
Security System for KOREN/APII-Testbed
Network Anomography Yin Zhang Joint work with Zihui Ge, Albert Greenberg, Matthew Roughan Internet Measurement.
Approaches to Intrusion Detection statistical anomaly detection – threshold – profile based rule-based detection – anomaly – penetration identification.
Network Anomaly Detection Using Autonomous System Flow Aggregates Thienne Johnson 1,2 and Loukas Lazos 1 1 Department of Electrical and Computer Engineering.
SCREAM: Sketch Resource Allocation for Software-defined Measurement Masoud Moshref, Minlan Yu, Ramesh Govindan, Amin Vahdat (CoNEXT’15)
1 Netflow Collection and Aggregation in the AT&T Common Backbone Carsten Lund.
Continuous Monitoring of Distributed Data Streams over a Time-based Sliding Window MADALGO – Center for Massive Data Algorithmics, a Center of the Danish.
Jian Wu (University of Michigan)
Pyramid Sketch: a Sketch Framework
Optimal Elephant Flow Detection Presented by: Gil Einziger,
SCREAM: Sketch Resource Allocation for Software-defined Measurement
COS 461: Computer Networks
Lu Tang , Qun Huang, Patrick P. C. Lee
Presentation transcript:

Detecting DDoS Attacks on ISP Networks Ashwin Bharambe Carnegie Mellon University Joint work with: Aditya Akella, Mike Reiter and Srinivasan Seshan

ISP Perspective of DDoS Attack Victim Attacker (Incarnation II) My ISP ISP3 ISP2 Hot potato routing

Problem Statement How can an ISP find out if: How can an ISP find out if: Its Backbone is carrying “useless” attack traffic? Its Backbone is carrying “useless” attack traffic? Its Backbone is itself under attack? Its Backbone is itself under attack? Focus of this talk: Focus of this talk: Sketch a solution approach Sketch a solution approach Discuss the main challenges Discuss the main challenges

Approach Record “normal” traffic at routers; identify anomalies Exchange suspicions among routers to reinforce anomaly detection Traffic Profile Destination: #Flows: … #Bytes: … MB Traffic Profile Destination: #Flows: … #Bytes: … MB

Basic Approach 1. Record “normal” traffic at routers 2. Detect “abnormalities” in traffic Challenges a. What is normal and what is abnormal? b. Is it robust? c. How quickly can we identify deviations? d. Can it really be implemented on a backbone router? e. Response strategy?

Proposed Solution Maintain Traffic Profiles Each router constructs profiles of traffic Each router constructs profiles of traffic Longer time-windows  normal traffic Longer time-windows  normal traffic Smaller time-windows  current traffic Smaller time-windows  current traffic Become suspicious if current profile violates normal profile Become suspicious if current profile violates normal profile

Important Challenges 1. Day-of-week and Time-of-day effects Maintain per-day per-daytime statistics Maintain per-day per-daytime statistics 2. Flash crowds Example of “harmless” but infrequent event Example of “harmless” but infrequent event Attack-volume alone is not a sufficient indicator Attack-volume alone is not a sufficient indicator “Fingerprint” the destination-bound traffic “Fingerprint” the destination-bound traffic Number of sources, source-subnets, flows, distribution of flow lengths, etc. Number of sources, source-subnets, flows, distribution of flow lengths, etc.

Traffic Fingerprints Some examples Total traffic to destination Total traffic to destination Source subnet characterization Source subnet characterization Total number of “flows” to a destination Total number of “flows” to a destination How many /24 subnets are observed in the traffic to this destination How many /24 subnets are observed in the traffic to this destination Flow-length distribution Flow-length distribution E.g., are there a lot of small flows? E.g., are there a lot of small flows?

Stream Sampling Memory/computation constraints at routers Memory/computation constraints at routers Keep statistics about every destination? Keep statistics about every destination? Only for popular ones  traffic to whom exceeds a fraction  of link capacity Only for popular ones  traffic to whom exceeds a fraction  of link capacity Use sample-and-hold or multistage filters [Estan01] Use sample-and-hold or multistage filters [Estan01] Count unique subnets in a packet stream Count unique subnets in a packet stream Memory =  (size of stream)! Memory =  (size of stream)! Use F 0 computation algorithms [Alon96, Gibbons01] Use F 0 computation algorithms [Alon96, Gibbons01] Do it in much smaller (constant!!) space and time Do it in much smaller (constant!!) space and time

Proposed Solution Increasing Robustness Single router has only local view  can make mistakes Single router has only local view  can make mistakes Traffic perturbations due to traffic engineering Traffic perturbations due to traffic engineering False alarms! False alarms! Suppose attacker “mimics” normal traffic at a router Suppose attacker “mimics” normal traffic at a router Attack goes undetected! Attack goes undetected! Mimicking at more than a few routers within an ISP would be hard! Mimicking at more than a few routers within an ISP would be hard! Use router consensus for reinforcing suspicions across routers Use router consensus for reinforcing suspicions across routers

Preliminary Results Single Router Detection Accuracy Experimental Setup Abilene-II traffic trace (70 minutes) Abilene-II traffic trace (70 minutes) Samples taken across a window of about 1 minute Samples taken across a window of about 1 minute Synthetic attack traffic (trinoo, TFN, TFN2k, etc.) Synthetic attack traffic (trinoo, TFN, TFN2k, etc.) Attack Detection Accuracy False positive rates ≤ 6%, lower for “ unpopular ” destinations False positive rates ≤ 6%, lower for “ unpopular ” destinations False negative rates decrease rapidly as the “ rate ” of attack traffic increases False negative rates decrease rapidly as the “ rate ” of attack traffic increases

Conclusions and Future Work Conclusions Conclusions Fingerprinting traffic allows for detection of subtle attack patterns not apparent from volume alone Fingerprinting traffic allows for detection of subtle attack patterns not apparent from volume alone Distributed detection makes it harder for an attacker to mimic traffic at multiple routers Distributed detection makes it harder for an attacker to mimic traffic at multiple routers Directions for future work Directions for future work Identify various attack scenarios Identify various attack scenarios Optimize computation/space requirements Optimize computation/space requirements Consensus algorithm; convergence and effectiveness Consensus algorithm; convergence and effectiveness Validate over real attack datasets Validate over real attack datasets

Backup Slide Overheads Use  = 0.1  memory ~ 3600 bytes per destination Use  = 0.1  memory ~ 3600 bytes per destination Approximate number of popular destinations = 1/  Approximate number of popular destinations = 1/  where  is the fraction of link capacity where  is the fraction of link capacity 360 KB per statistic – if we use  = 1% 360 KB per statistic – if we use  = 1% Can a high-end router have a few MBs of SRAM? Can a high-end router have a few MBs of SRAM? AlgorithmsAMS96GT01 Accuracy 1+ ,  > 1 1 ± ,  > 0 Memory (bytes) 4 36/  2 Byte operations ~4 ~4~6 Counting unique items in a stream (zeroeth moment F 0 )