Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring, 2003

Talk Objectives Motivate and describe Wisconsin Advanced Internet Lab (WAIL) –Internal lab environment –External lab environment Provide some detail on three current projects –Anomaly detection and characterization –Distributed intrusion monitoring –Understanding packet loss

Motivation for New Tools Any area of scientific research is limited by the tools available for experimental study –“If your only tool is a hammer then everything looks like a nail” 2001 NRC report: “network research community is in danger of ossification due to strictures of experimental systems” –Challenge: “Capturing a day in the life of the Internet” New experimental tools can open up areas of research that have not previously been accessible

An Internet Instance Lab A hands-on test environment designed to recreate paths and conditions identical to those in the Internet from end-to-end-through-core –Requires large amount of routing and end host equipment Network and host equipment able to recreate (not emulate) a wide range of services, configurations and traffic conditions –Complete instrumentation of end-to-end paths –Deployment of disruptive prototypes

Key Challenges Design Configurations and management Traffic generation Propagation delay Validation

The Wisconsin Advanced Internet Lab Our realization of an IIL Developed over past 18 months by UW/Cisco team Supported by $3.5M equipment grant from Cisco and UW matching funds –Used to purchase over 75 pieces of networking equipment Phase 1 nearing completion => Abilene recreation Other partners: EMC, Spirent, Intel, Fujitsu, Sun Research initiatives in many areas…

External Environment Essential complement to internal environment Existing infrastructure –DOMINO systems (1 class A + 2 class B’s + Dshield) –Surveyor + WAWM systems (~70 nodes) New database and front end by summer ‘03 Partnerships and other available systems –Condor/Grid Infrastructures Passive flow measurements –FlowScan data from UW, Internet2, others…

Project 1: Detecting Anomalies in IP Flows Motivation: Anomaly detection remains difficult Objective: Improve understanding of traffic anomalies Approach: Multiresolution analysis of data set that includes IP flow, SNMP and an anomaly catalog Method: Integrated Measurement Analysis Platform for Internet Traffic (IMAPIT) Results: Identify anomaly characteristics using wavelets and develop new method for exposing short-lived events

Our Data Sets Consider anomalies in IP flow and SNMP data –Collected at UW border router (Juniper M10) –Archive of ~6 months worth of data (packets, bytes, flows) –Includes catalog of anomalies (after-the-fact analysis) Group observed anomalies into four categories –Network anomalies (41) Steep drop offs in service followed by quick return to normal behavior –Flash crowd anomalies (4) Steep increase in service followed by slow return to normal behavior –Attack anomalies (46) Steep increase in flows in one direction followed by quick return to normal behavior –Measurement anomalies (18) Short-lived anomalies which are not network anomalies or attacks

Multiresolution Analysis Wavelets provide a means for describing time series data that considers both frequency and time –Powerful means for characterizing data with sharp spikes and discontinuities –Using wavelets can be quite tricky We use tools developed at UW which together make up IMAPIT –FlowScan software –The IDR Framenet software

Ambient IP Flow Traffic

Flow Traffic During DoS Attacks

Deviation Score for Three Anomalies

Project 2: Coordinated Intrusion Detection Motivation: Intrusion detection is a moving target Objective: Coordinate intrusion monitoring between multiple sites around the Internet Approach: Share data from firewalls, NIDS and tarpits (on unused IP space) Method: Distributed Overlay for Monitoring Internet Outbreaks (DOMINO) Results: Blacklists can be rapidly generated, false positives can be substantially lowered, new outbreaks can be easily identified

DOMINO: A new approach to DNIDS Partnership with dshield.org –1600 firewall and NIDS logs Tarpits –Active monitor of unused IP space –1 class A (this week), 2 class B’s A protocol for node participation, data sharing and alert clustering –Chord-based overlay network –Extension of Intrusion Detection Message Exchange Format – Various clustering methods

Marginal Utility of Adding Nodes

SQL-Sapphire Analysis

Project 3: Understanding Packet Loss Motivation: Many of the most basic aspects of packet loss are not understood –Where, when, how long, how often? Focus: Developing a comprehensive understanding of packet loss in the Internet Approach: Combine understanding of protocols and queue behavior to create a probe train which can accurately measure delay and loss. Implications: End-to-end tools for pin-pointing loss, better transport protocols, better network management for congestion

Active versus Passive Loss Measures Hypothesis: Active measures of loss are correlated with passive measures of loss Assessment in Abilene –SNMP loss measures on all backbone routers –Active probes via Ping/Zing in Surveyor nodes at 10Hz, 20Hz and 100Hz –Tests in full mesh over one month period

Result: Active <> Passive

Summary Both internal lab building initiatives and external measurement initiatives in WAIL Internal facilities are intended to be open We are seeking partnerships in external measurement projects. –DOMINO in particular