Sven Ubik, CESNET TNC2004, Rhodos, 9 June 2004 Performance monitoring of high-speed networks from NREN perspective

NRENs Traffic on NREN links: backbone physical capacity 1-10 Gb/s typical long-term load 50 Mb/s – 1.5 Gb/s typical daily fluctuations 1:5 – flat throughput limited mostly by TCP congestion control reacting to router queue overflows

NREN performance monitoring Implications for performance monitoring: network is on the verge of over-provisioning and will probably remain so we need to monitor network behaviour closely in short-timescales monitoring must work reliably at high speeds

Monitoring methods Three primary ways of performance monitoring: Processing data from network components Active monitoring Passive monitoring

Data from network components router counters read by SNMP flow records Pros: per-link statistics (good space granularity) Cons: delayed update of MIB database (poor time granularity) flow records unreliable due to router overload and software bugs

SNMP counters 60-second averages: short term fluctuations

SNMP counters, cont. 1-second averages: unusable due to delayed MIB update

Active monitoring Pros: easy way to monitor one-way delay and jitter can provide definitive confirmation that high data rate can be passed through network Cons: Need to ping responsive points (not routers) Heavy-weighted throughput measurement stresses user traffic Light-weighted throughput “estimation” still not reliable

Passive monitoring Pros: observes properties experienced by real user traffic, rather than by artificially injected traffic non-intrusive Cons: difficult at high speeds

SCAMPI project SCAMPI – “SCAleable Monitoring Platform for the Internet” Concentrates on passive monitoring Should overcome network speed / PC CPU speed gap by hardware offloading Should allow easy writing of portable monitoring applications

Applications Packet capture Accounting Flow-based reporting Threshold alerting QoS monitoring Network intrusion detection system Flow-based IDS Denial of service attack detection

QoS monitoring Currently monitored characteristics: Short-timescale link load One-way delay Packet loss rate

Link utilization 10 ms averages

Packet loss rate - active Can we measure realistic packet loss rate actively? 100 testing packets per second => thousands of packets per second for continuous full mesh measurement seconds or 3 hours required to detect packet loss rate of 10E-6 (assuming fluid traffic model) If a testing packet is lost, can we imply packet loss rate? Comparison of active and SNMP loss monitoring: [Barford+Sommers, 2003]

Packet loss rate – other methods SNMP counters: unreliable (unlike byte counters) Passive: Capture packets on border connections Sampling possible, but reduces precision Can provide precise short-timescale information about low packet loss rates

Conclusion Short-timescale monitoring is needed to understand network behaviour Passive monitoring can provide more realistic results than active monitoring Passive monitoring at gigabit speeds requires hardware support with built-in monitoring functions, such as SCAMPI adapters

Sven Ubik Thank you for your attention

Backup slides

Types of network monitoring: operational performance security Network monitoring

Delay to routers difficult We should not fill router queues, because: No increase in throughput over using „wire pipe“ Filled-up queues are sensitive to losses caused by cross-traffic Can we determine bottleneck router by observing RTT increase? Interface before 9th link Interface after 9th link

SCAMPI adapter 1GE and 10GE Header filtering Sampling Payload searching Statistics

Header filter and payload string search: fd=mapi_create_flow("/dev/scampi/0"); mapi_apply_function(fd, BPF_FILTER, "src port 2001"); ctr_id1=mapi_apply_function(fd, PKT_COUNTER); mapi_apply_function(fd, STR_SEARCH, "malicious string", 0, 1500); ctr_id2=mapi_apply_function(fd, PKT_COUNTER); mapi_connect(fd); while(1) { sleep(1); mapi_read_results(fd, ctr_id1, &ctr_num1); mapi_read_results(fd, ctr_id2, &ctr_num1); /* … */ } Using MAPI