Observed Structure of Addresses in IP Traffic CSCI 780, Fall 2005.

Slides:



Advertisements
Similar presentations
Finding Gold In The Forest …A Connection Between Fractal Trees, Topology, and The Golden Ratio.
Advertisements

Traffic Dynamics at a Commercial Backbone POP Nina Taft Sprint ATL Co-authors: Supratik Bhattacharyya, Jorjeta Jetcheva, Christophe Diot.
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Fast, Memory-Efficient Traffic Estimation by Coincidence Counting Fang Hao 1, Murali Kodialam 1, T. V. Lakshman 1, Hui Zhang 2, 1 Bell Labs, Lucent Technologies.
A Flexible Model for Resource Management in Virtual Private Networks Presenter: Huang, Rigao Kang, Yuefang.
Tries Standard Tries Compressed Tries Suffix Tries.
The Central Limit Theorem
CMPT 855Module Network Traffic Self-Similarity Carey Williamson Department of Computer Science University of Saskatchewan.
On the Self-Similar Nature of Ethernet Traffic - Leland, et. Al Presented by Sumitra Ganesh.
Topics: Inferential Statistics
Estimation from Samples Find a likely range of values for a population parameter (e.g. average, %) Find a likely range of values for a population parameter.
Homework discussion Read pages 388 – 391 Page 400: 49 – 52, 72.
Network Bandwidth Allocation (and Stability) In Three Acts.
Chapter 3 Normal Curve, Probability, and Population Versus Sample Part 2.
Probability Distributions
Network Traffic Measurement and Modeling CSCI 780, Fall 2005.
ISE 261 PROBABILISTIC SYSTEMS. Chapter One Descriptive Statistics.
A Nonstationary Poisson View of Internet Traffic T. Karagiannis, M. Molle, M. Faloutsos University of California, Riverside A. Broido University of California,
Multifractals in Real World
End-to-End Issues. Route Diversity  Load balancing o Per packet splitting o Per flow splitting  Spill over  Route change o Failure o policy  Route.
Connection Admission Control Schemes for Self-Similar Traffic Yanping Wang Carey Williamson University of Saskatchewan.
Assumption and Data Transformation. Assumption of Anova The error terms are randomly, independently, and normally distributed The error terms are randomly,
Scaling and Attitude Measurement in Travel and Hospitality Research Research Methodologies CHAPTER 11.
Chapter 9 Flashcards. measurement method that uses uniform procedures to collect, score, interpret, and report numerical results; usually has norms and.
Modelling and Simulation 2008 A brief introduction to self-similar fractals.
Applied Mathematics Complex Systems Fractals Fractal by Zhixuan Li.
Hydrologic Statistics
Self-Similarity of Network Traffic Presented by Wei Lu Supervised by Niclas Meier 05/
1 Chapters 9 Self-SimilarTraffic. Chapter 9 – Self-Similar Traffic 2 Introduction- Motivation Validity of the queuing models we have studied depends on.
Stat 1510: Introducing Probability. Agenda 2  The Idea of Probability  Probability Models  Probability Rules  Finite and Discrete Probability Models.
Census A survey to collect data on the entire population.   Data The facts and figures collected, analyzed, and summarized for presentation and.
Traffic Modeling.
Population All members of a set which have a given characteristic. Population Data Data associated with a certain population. Population Parameter A measure.
Lecture 14 Dustin Lueker. 2  Inferential statistical methods provide predictions about characteristics of a population, based on information in a sample.
● Final exam Wednesday, 6/10, 11:30-2:30. ● Bring your own blue books ● Closed book. Calculators and 2-page cheat sheet allowed. No cell phone/computer.
Sampling Distributions & Standard Error Lesson 7.
M11-Normal Distribution 1 1  Department of ISM, University of Alabama, Lesson Objective  Understand what the “Normal Distribution” tells you.
Vito Volterra, 1881: There exists a function, F(x), whose derivative, F '(x), exists and is bounded for all x, but the derivative, F '(x), cannot be integrated.
Wide-scale Botnet Detection and Characterization Anestis Karasaridis, Brian Rexroad, David Hoeflin In First Workshop on Hot Topics in Understanding Botnets,
Learning Objective Chapter 9 The Concept of Measurement and Attitude Scales Copyright © 2000 South-Western College Publishing Co. CHAPTER nine The Concept.
A Real-Time Packet Burst Metric TERENA Networking Conference 2004 Klaus Mochalski Computer Science Department University of Leipzig, Germany Jörg Micheel.
Multiscale Network Processes: Fractal and p-Adic analysis Vladimir Zaborovsky, Technical University, Robotics Institute, Saint-Petersburg, Russia .
Automating Analysis of Large-Scale Botnet Probing Events Zhichun Li, Anup Goyal, Yan Chen and Vern Paxson* Lab for Internet and Security Technology (LIST)
Alexey A. Didyk Kherson national technical university, Ukraine
Burst Metric In packet-based networks Initial Considerations for IPPM burst metric Tuesday, March 21, 2006.
BASIC STATISTICAL CONCEPTS Chapter Three. CHAPTER OBJECTIVES Scales of Measurement Measures of central tendency (mean, median, mode) Frequency distribution.
PCB 3043L - General Ecology Data Analysis.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall2(2)-1 Chapter 2: Displaying and Summarizing Data Part 2: Descriptive Statistics.
MODELING THE SELF-SIMILAR BEHAVIOR OF PACKETIZED MPEG-4 VIDEO USING WAVELET-BASED METHODS Dogu Arifler and Brian L. Evans The University of Texas at Austin.
Introduction to statistics I Sophia King Rm. P24 HWB
@ Carnegie Mellon Databases 1 Finding Frequent Items in Distributed Data Streams Amit Manjhi V. Shkapenyuk, K. Dhamdhere, C. Olston Carnegie Mellon University.
The Nature of Science The Methods of Science Scientific Measurements Graphing.
Network Anomaly Detection Using Autonomous System Flow Aggregates Thienne Johnson 1,2 and Loukas Lazos 1 1 Department of Electrical and Computer Engineering.
Notices of the AMS, September Internet traffic Standard Poisson models don’t capture long-range correlations. Poisson Measured “bursty” on all time.
Lesson 3 Measurement and Scaling. Case: “What is performance?” brandesign.co.za.
Political Science 30: Political Inquiry. The Magic of the Normal Curve Normal Curves (Essentials, pp ) The family of normal curves The rule of.
Statistics -Descriptive statistics 2013/09/30. Descriptive statistics Numerical measures of location, dispersion, shape, and association are also used.
 Normal Curves  The family of normal curves  The rule of  The Central Limit Theorem  Confidence Intervals  Around a Mean  Around a Proportion.
Class Six Turn In: Chapter 15: 30, 32, 38, 44, 48, 50 Chapter 17: 28, 38, 44 For Class Seven: Chapter 18: 32, 34, 36 Chapter 19: 26, 34, 44 Quiz 3 Read.
ISE 261 PROBABILISTIC SYSTEMS
Test for Goodness of Fit
Impact of Packet Sampling on Anomaly Detection Metrics
Analyzing Reliability and Validity in Outcomes Assessment Part 1
More Specific Announcements in BGP
MEGN 537 – Probabilistic Biomechanics Ch.3 – Quantifying Uncertainty
Matter and Measurement Vocabulary
EQ: How do we approximate a distribution of data?
Statistics: The Interpretation of Data
Unit 6 - Comparing Two Populations or Groups
Group 9 – Data Mining: Data
Presentation transcript:

Observed Structure of Addresses in IP Traffic CSCI 780, Fall 2005

More specific Structural characteristics of destination IP addresses seen on Internet links Traces are omni-directional Set of source addresses is roughly equal to set of destination addresses Destination address prefix based aggregation

Terminology Active address: an IP address visible in the trace as destination p-aggregate: a set of IP addresses that share the same p-bit address prefix Active p-aggregate: a p-aggregate containing at least one active address N: the number of active addresses in the trace

Problem statement How can we model the set of destination IP addresses visible on the access links? In particular, how can we model the addresses aggregate (address structure)?

Address structure The arrangement of active addresses in the address space

Trace collection

Importance of address structure A stable short-term fingerprint of a site Different sites have different address structure Address structure is the most important factor affecting aggregate packet count distribution

Packet count distribution No. of packets per flow; (heavy-tail) per destination address; (heavy-tail) per destination address aggregate (MORE heavy-tail)

Factors affecting aggregate packet counts Address packet counts No. of packet per destination address Address structure No. of active addresses per aggregate Correlation between these two

Semi-experiments

Address structure matters most

Tour of U1 ’ s address structure

Address structure looks self- similar Interesting structure all the way down self-similar Visually self-similar characteristics Validate the intuition: multi-fractal model for address structure An address structure viewed as a subset of the unit interval [0,1) Cantor dust with two parameters Dimension: active p-aggregates Mass: active addresses within each prefix aggregate

Canonical Cantor Dust

Dimension for address space

Fitness with prefix aggregates

Why multi-fractal Mono-fractal only captures global scaling behavior of aggregate counts Address structure has different local scaling behavior (active addresses) Besides (capacity) dimension, introduce another parameter: mass

Multi-fractal Model Start with a cantor dust with dimension D Repeatedly remove middle subinterval with proportion Unequally distribute a unit of mass between subintervals Unequal distribution of mass leads to different local scaling behaviors

The model fits well

Causes Cascade effects: A recursive subdivision plus a rule for distributing mass Procedure of address allocation ICANN allocates short prefixes to providers Providers allocates less shorter prefixes to customers Share the same rule: left-to-right allocation

Is the model useful?

How densely addresses are packed? Metrics: active p-aggregate counts for prefix p

Aggregation ratio

Characterize address separation Metrics: discriminating prefixes of an active address a The prefix length of the largest aggregate whose only active address is a : number of addresses that have discriminating prefix p

CDF of address discriminating Prefix counts

Aggregate population distribution Population of an aggregate is the number of active addresses contained in it Aggregates exhibit a wide range of population Aggregate population distributions are the most effect test to differentiate address structures

CDF of aggregate population distribution

Properties of Sampling effect How does the shape of the curve depend on N? Short-term stability Is stable over short time period?

Similar curves

Relatively stable

Worm changes the shape

Worm changes aggregate packet count