A Signal Analysis of Network Traffic Anomalies Paul Barford, Jeffrey Kline, David Plonka, and Amos Ron.

Slides:

Advertisements

Similar presentations

Characteristics of Network Traffic Flow Anomalies Paul Barford and David Plonka University of Wisconsin – Madison SIGCOMM IMW, 2001.

Advertisements

Sensor-Based Abnormal Human-Activity Detection Authors: Jie Yin, Qiang Yang, and Jeffrey Junfeng Pan Presenter: Raghu Rangan.

Regression Analysis Once a linear relationship is defined, the independent variable can be used to forecast the dependent variable. Y ^ = bo + bX bo is.

Chapter 6 - Part 1 Introduction to SPC.

Anomaly Detection Steven M. Bellovin Matsuzaki ‘maz’ Yoshinobu 1.

FLAME: A Flow-level Anomaly Modeling Engine

Nick Duffield, Patrick Haffner, Balachander Krishnamurthy, Haakon Ringberg Rule-Based Anomaly Detection on IP Flows.

Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring,

Cyber Threat Analysis  Intrusions are actions that attempt to bypass security mechanisms of computer systems  Intrusions are caused by:  Attackers accessing.

Short-term Load Forecasting Using Improved Similar Days Method Qingqing Mu, Yonggang Wu, Xiaoqiang Pan, Liangyi Huang, Xian Li Power and Energy Engineering.

Student : Wilson Hidalgo Ramirez Supervisor: Udaya Tupakula Filtering Techniques for Counteracting DDoS Attacks.

Probabilistic Aggregation in Distributed Networks Ling Huang, Ben Zhao, Anthony Joseph and John Kubiatowicz {hling, ravenben, adj,

originally developed by Walter A. Shewhart

Chapter 12 - Forecasting Forecasting is important in the business decision-making process in which a current choice or decision has future implications:

Hands-On Microsoft Windows Server 2003 Networking Chapter Four Subnetting.

Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.

Chapter 3 Forecasting McGraw-Hill/Irwin

Chapter 13 Forecasting.

ANOMALY DETECTION AND CHARACTERIZATION: LEARNING AND EXPERIANCE YAN CHEN – MATT MODAFF – AARON BEACH.

Intrusion Detection CS-480b Dick Steflik. Hacking Attempts IP Address Scans scan the range of addresses looking for hosts (ping scan) Port Scans scan.

EL 933 Final Project Presentation Combining Filtering and Statistical Methods for Anomaly Detection Augustin Soule Kav´e SalamatianNina Taft.

Forecasting McGraw-Hill/Irwin Copyright © 2012 by The McGraw-Hill Companies, Inc. All rights reserved.

Energy-efficient Self-adapting Online Linear Forecasting for Wireless Sensor Network Applications Jai-Jin Lim and Kang G. Shin Real-Time Computing Laboratory,

Business Forecasting Chapter 5 Forecasting with Smoothing Techniques.

Slides 13b: Time-Series Models; Measuring Forecast Error

INTRUSION DETECTION SYSTEMS Tristan Walters Rayce West.

A Signal Analysis of Network Traffic Anomalies Paul Barford with Jeffery Kline, David Plonka, Amos Ron University of Wisconsin – Madison Summer, 2002.

Inference for regression - Simple linear regression

LSS Black Belt Training Forecasting. Forecasting Models Forecasting Techniques Qualitative Models Delphi Method Jury of Executive Opinion Sales Force.

CHAPTER 3 FORECASTING.

A Statistical Anomaly Detection Technique based on Three Different Network Features Yuji Waizumi Tohoku Univ.

Traffic Classification through Simple Statistical Fingerprinting M. Crotti, M. Dusi, F. Gringoli, L. Salgarelli ACM SIGCOMM Computer Communication Review,

by B. Zadrozny and C. Elkan

IP Flow Measurement & Analysis with FlowScan IPAM Workshop, Los Angeles, March 21, 2002 Dave Plonka Division of Information Technology,

A Wavelet-based Anomaly Detector for Disease Outbreaks Thomas Lotze Galit Shmueli University of Maryland College Park Sean Murphy Howard Burkom Johns Hopkins.

Dividing the Pizza An Advanced Traffic Billing System An Advanced Traffic Billing System Christopher Lawrence Burke The University of Queensland.

1 The Research on Analyzing Time- Series Data and Anomaly Detection in Internet Flow Yoshiaki HARADA Graduate School of Information Science and Electrical.

1 Intrusion Detection Methods “Intrusion detection is the process of identifying and responding to malicious activity targeted at computing and networking.

CSCI 530 Lab Intrusion Detection Systems IDS. A collection of techniques and methodologies used to monitor suspicious activities both at the network and.

Mitsubishi Research Institute, Inc Analyses on Distribution of Malicious Packets and Threats over the Internet August 27-31, 2007 APAN Network Research.

Detection Unknown Worms Using Randomness Check Computer and Communication Security Lab. Dept. of Computer Science and Engineering KOREA University Hyundo.

Copyright © 2003 OPNET Technologies, Inc. Confidential, not for distribution to third parties. Session 1341: Case Studies of Security Studies of Intrusion.

Time Series Analysis and Forecasting

Network Anomography Yin Zhang – University of Texas at Austin Zihui Ge and Albert Greenberg – AT&T Labs Matthew Roughan – University of Adelaide IMC 2005.

Time-Series Forecasting Overview Moving Averages Exponential Smoothing Seasonality.

Intel Confidential – Internal Only Co-clustering of biological networks and gene expression data Hanisch et al. This paper appears in: bioinformatics 2002.

Is Sampled Data Sufficient for Anomaly Detection Ip Wing Chung Peter ( ) Ngan Sze Chung ( )

Open-Eye Georgios Androulidakis National Technical University of Athens.

Unconstrained Endpoint Profiling Googling the Internet Ionut Trestian, Supranamaya Ranjan, Alekandar Kuzmanovic, Antonio Nucci Reviewed by Lee Young Soo.

Boundary Detection in Tokenizing Network Application Payload for Anomaly Detection Rachna Vargiya and Philip Chan Department of Computer Sciences Florida.

Time Series Analysis and Forecasting. Introduction to Time Series Analysis A time-series is a set of observations on a quantitative variable collected.

ASTUTE: Detecting a Different Class of Traffic Anomalies Fernando Silveira 1,2, Christophe Diot 1, Nina Taft 3, Ramesh Govindan 4 1 Technicolor 2 UPMC.

Intrusion Detection Systems Paper written detailing importance of audit data in detecting misuse + user behavior 1984-SRI int’l develop method of.

Taming Internet Traffic Some notes on modeling the wild nature of OD flows Augustin Soule Kavé Salamatian Antonio Nucci Nina Taft Univ. Paris VI Sprintlabs.

EE515/IS523: Security 101: Think Like an Adversary Evading Anomarly Detection through Variance Injection Attacks on PCA Benjamin I.P. Rubinstein, Blaine.

1 Virtual Dark IP for Internet Threat Detection Akihiro Shimoda & Shigeki Goto Waseda University

I NTRUSION P REVENTION S YSTEM (IPS). O UTLINE Introduction Objectives IPS’s Detection methods Classifications IPS vs. IDS IPS vs. Firewall.

Sensitivity of PCA for Traffic Anomaly Detection Evaluating the robustness of current best practices Haakon Ringberg 1, Augustin Soule 2, Jennifer Rexford.

Time-Series Forecast Models  A time series is a sequence of evenly time-spaced data points, such as daily shipments, weekly sales, or quarterly earnings.

Network Anomaly Detection Using Autonomous System Flow Aggregates Thienne Johnson 1,2 and Loukas Lazos 1 1 Department of Electrical and Computer Engineering.

1 Creating Situational Awareness with Data Trending and Monitoring Zhenping Li, J.P. Douglas, and Ken. Mitchell Arctic Slope Technical Services.

Distributed Network Monitoring in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring,

Chapter 11 – With Woodruff Modications Demand Management and Forecasting Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved.McGraw-Hill/Irwin.

25/09/ Firewall, IDS & IPS basics. Summary Firewalls Intrusion detection system Intrusion prevention system.

Impact of Packet Sampling on Anomaly Detection Metrics

Roland Kwitt & Tobias Strohmeier

Baselining PMU Data to Find Patterns and Anomalies

Yining ZHAO Computer Network Information Center,

Unconstrained Endpoint Profiling (Googling the Internet)‏

Demand Management and Forecasting

Presentation transcript:

A Signal Analysis of Network Traffic Anomalies Paul Barford, Jeffrey Kline, David Plonka, and Amos Ron

Network Traffic Anomalies  Failures and attacks  Detection part of everyday work for administrators  Data derived mainly from two sources SNMP SNMP Queries to nodes; mostly counts of activityQueries to nodes; mostly counts of activity IP flows IP flows More specific than SNMPMore specific than SNMP

Related Work  Statistical detection of anomalies  Past work on malicious (DoS, port scan) behavior detection  Flash crowd studies

Data  Analysis based on SNMP and IP data  Taken from a border router at University of Wisconsin-Madison  Flows sampled 1 in 96 packets  Journal of known anomalies and events was kept Network Network Attack Attack Flash Flash Measurement Measurement

Current Practices  Network operators use ad hoc methods  Rely on operator’s personal experience  Handling SNMP data Graph network data Graph network data Alarms for certain events Alarms for certain events  Flow data handling less mature Popular tool converts into time-series data Popular tool converts into time-series data

Method  Wavelet analysis  Divides the data into strata  Low-frequency strata: slow-varying trends  High-frequency strata: spontaneous variations

Wavelet Processing  Analysis/Decomposition Break down the signal into the strata Break down the signal into the strata Run different filters for the different frequencies Run different filters for the different frequencies  Synthesis Inverse of decomposition Inverse of decomposition  Wavelet algorithms Recombine strata, but filtering out unwanted data Recombine strata, but filtering out unwanted data

Cont.  The technique used by the authors synthesizes 3 separate parts of the signal  Total amount within the parts will be longer than the actual signal  L – Captures long term patterns; ideal for weekly trends  M – Captures midrange patterns; ideal for daily trends  H – High frequency data capture

Anomaly Detection  Normalize H- and M- to a variance of 1 Compute local variability of data within a moving window (3 hours) Compute local variability of data within a moving window (3 hours)  Combine variability of H- and M-  Apply thresholding

IMAPIT  Development environment for anomaly detection  Used the H-, M-, and weights for both to determine deviation scores  Anomalies tend to have deviation over 2.0

Characteristics of Ambient Traffic  Need data free of anomalies as a calibration

Flash Crowds  Test data: New Linux release on ftp mirror

Short-lived Anomalies

Discriminator for Short-term Anomalies

Two DoS Events

Analysis of Network Outage

Deviation Score Evaluation  Used logged anomalies as baseline for evaluation Of 39 logged anomalies, detected 38 Of 39 logged anomalies, detected 38

Comparison to Holt-Winters  Holt-Winters is an exponential smoothing algorithm Uses baseline (intercept), linear trend (slope), and seasonal trend Uses baseline (intercept), linear trend (slope), and seasonal trend Aberrations are detected by detecting a certain amount of data outside the threshold range within a window Aberrations are detected by detecting a certain amount of data outside the threshold range within a window  Different from wavelet in that the different strata are processed separately whereas Holt-Winters is one prediction function  Compared to an alternative using Holt-Winters algorithm Holt-Winters detected 37 anomalies Holt-Winters detected 37 anomalies Both missed anomalies would have been detected with a larger window Both missed anomalies would have been detected with a larger window Holt-Winters more sensitive Holt-Winters more sensitive

Conclusion  Performs comparably to Holt-Winters  Deviation score detection can be effective  Learning methods potentially used in the future  Study ways of classification