1. Standards Activities on Traffic Measurement

2. 2 Outline Applications requiring traffic measurement Packet capturing and flow measurement Existing protocols IETF IPFIX and PSAMP working groups

3. 3 Applications Requiring Traffic Measurement (1) Usage-based accounting –input to charging and billing –various business model time-based, volume-based, QoS class-based per application, per user, per user group Traffic engineering –optimizing network usage –traffic analysis on congested links origin of traffic type of traffic dynamic behavior (bursty, adaptive, …) Traffic profiling

4. 4 Applications Requiring Traffic Measurement (2) QoS monitoring –(passive) measurement of QoS properties –validating Service Level Agreements Attack detection and analysis –detecting (high volume) traffic patterns –investigation of origin of attacks Intrusion detection –detecting unexpected or illegal packets …

5. 5 Outline Applications requiring traffic measurement Packet capturing and flow measurement Existing protocols IETF IPFIX and PSAMP working groups

6. 6 The General Traffic Flow Measurement Process Classification & Flow Recording Store (TCPdump) Observation Point PAYLOAD HEAD Packet Capturing Filtering Display (Ethereal) Sampling Visualize (FlowScan) Analysis by applications (TE, attack detect., QoS monitoring, accounting, …) … other … packets Filtering Sampling flow records packets flow records

7. 7 The General Traffic Flow Measurement Process Packet capturing at observation point Packet sampling and filtering –both steps may be trivial (1:1 sampling, no filtering) –both steps may be applied repeatedly Packet classification, mapping to flow records, maintaining of flow records Flow record sampling and filtering –both steps may be trivial (1:1 sampling, no filtering) –both steps may be applied repeatedly Processing flow records in application

8. 8 Filtering Sampling packets Capturing packets Packet Capturing Protocols –Capture packets at observation point –Optionally: Sample and filter packets –Export packets or parts of packets (e.g., first 100 bytes) –Packet classification, flow recording and processing after transfer Proprietary: sFlow Standard (to be): PSAMP packet transfer Classification & Flow Recording Application flow records router or probe

9. 9 Filtering Sampling packets Capturing Flow Monitoring Protocols –Capture packets at observation point –Optionally: Sample and filter packets –Classify packets and update flow records –Export flow records –Flow record processing after transfer Proprietary: NetFlow, LFAP, CRANE Standard: Meter MIB, IPFIX flow record transfer Classification & Flow Recording Application flow records Filtering Sampling router or probe

10. 10 Comparison Packet Capturing Protocols osimple function on router or probe +low cost on router or probe –high data volume for packet transfer or unreliable recording because of sampling –packet classification required after data transfer Flow Monitoring Protocols omore complex functions on router or probe –high resource requirement on router or probe: fast memory for flow records +low data volume for flow record transfer +flow records available after data transfer

11. 11 Outline Applications requiring traffic measurement Packet capturing and flow measurement Existing protocols IETF IPFIX and PSAMP working groups

12. 12 Protocols Packet Capturing –sFlow (InMon Corporation, HP spin-off) –PSAMP (under standardization at IETF) Flow Monitoring –LFAP (Riverstone) –CRANE (XACCT) –NetFlow (Cisco) –IPFIX (under standardization at IETF) –RTFM Meter MIB (IETF standard) –RMONMIB (IETF standard)

13. 13 Data collector sFlow By InMon Corporation Includes packet capturing, sampling and packet transmission Statistical packet sampling Timestamping at data collector Configuration by sFlow MIB RFC 3176, www.sflow.org Applicable to high speed links when sampling is used Adopted by many vendors (HP, Hitachi, Alaxaia - by Hitachi and NEC, Extreme and more) sMon Meter Application Packets

14. 14 Data collector PSAMP Under standardization at IETF Packet Sampling WG Very similar to sFlow Time stamping by meter Configuration by PSAMP MIB Intention to use IPFIX protocol for packet transfer PSAMP Meter Application Packets

15. 15 LFAP Light-weight Flow Accounting Protocol Proprietary by Riverstone (Cabletron) Just data transfer protocol Meter at Connection Control Entity (CCE) communicates to Flow Accounting Server (FAS) Tight and reliable interaction between CCE and FAS Reliable data transport Flexible TLV coding of transferred data Larger overhead than NetFlow More cost-intensive at meter/CCE and at data collector/FAS CCE Application FAS Flow records

16. 16 CRANE Common Reliable Accounting for Network Element (CRANE) Protocol Proprietary by XACCT Just data transfer protocol Template-based data model Focus on reliability Not yet in extensive commercial use

17. 17 Data collector IPFIX Under standardization at IETF IP Flow Information eXport WG Very similar to NetFlow version 9 Will not use UDP, but use TCP or SCTP (Stream Control Transmission Protocol) Standardization close to completion Close collaboration with PSAMP WG IPFIX Meter Application Flow records

18. 18 Router NetFlow Proprietary by Cisco, but de-facto standard Fast and efficient, implemented for IOS Configurable measurement per 5-tuple Unreliable data transport (UDP) Hardware-supported on some models Not well documented –re-engineered by Juniper Versions 1, 3, 5, 7, 8 –fixed data model –no support of IPv6 flows Version 9 (starting point for IPFIX standard) –data model templates –can report IPv6 flows –optional reliable transport –not related to older versions! –RFC 3954 Meter Data collector Application Flow records

19. 19 Real-Time Flow Measurement (RTFM) Very flexible and powerful meter –programmable rule sets –can serve several readers –programmable overload behavior Reader polls meter Realization by SNMP Meter MIB Free software implementation NeTraMet No acceptance at manufacturers Complicated to use (too powerful) Specified by RFCs 2720 - 2724 Meter Manager Reader Application Flow records Config.

20. 20 Remote Network Monitoring MIB (RMON) Very flexible and powerful Serves more general goals (analysis on layers 2-4) Just a monitoring tool, no measurement architecture defined Suited for very specific analysis tasks High (hardware) performance requirements Too complicated and too expensive for massive usage in routers Specified by RFCs 2021(RMON2), 2613, 2819(RMON), 2895, 2896, 3144, 3287, 3273, 3395, 3434, 3577, 3729, 3737, 3919, 4149, 4150

21. 21 Outline Applications requiring traffic measurement Packet capturing and flow measurement Existing protocols IETF IPFIX and PSAMP working group

22. 22 IETF IPFIX Working Group IP Flow Information eXport (IPFIX) –BoF sessions 12/00 and 08/01 –active since 10/01 Successor of RTFM (Real-Time Flow Measurement) WG Target (official): standardizing current practice –Target (unofficial): standardizing (something like) Cisco NetFlow Chairs –Nevil Brownlee, CAIDA –David Plonka, University of Wisconsin

23. 23 IPFIX Scope and General Requirements Goal: Find or develop a basic common IP Traffic Flow measurement technology to be available on (almost) all future routers Fulfilling requirements of many applications Low hardware/software costs Simple and scalable Metering to be integrated in general purpose IP routers and other devices (probes, middleboxes) Data processing to be integrated into various applications Interoperability by openness or standardization

24. 24 Application IPFIX Architecture Flow Record Observation Point Flow Information Export PAYLOAD HEAD Metering Process Exporting Process Collecting Process

25. 25 IPFIX Devices O M E Probe O M E Simple Router OOOO M E Complex Router OOOO M OOOO M E Multiple Exporters OOOO M E OOO O M E Protocol Converter (Meter MIB) O M E O M E O M E ME Concen- trator CE Proxy C … M: Meter E: Exporter C: Collector

26. 26 IPFIX WG: Expected Output Planned documents –Requirements RFC (completed, RFC 3917) –Evaluation RFC (completed, RFC 3955) –Protocol specification (in progress) –Data Model (in progress) –Architecture RFC (in progress) –Information model RFC (in-progress) –Applicability RFC (in-progress) No new protocol development in working group Instead: protocol selection and refinement Selected protocol: NetFlow version 9 Configuration of measurements will not (yet?) be standardized

27. 27 IPFIX WG: Current Status Good support from IESG (Internet Engineering Steering Group) High interest from equipment manufacturers –Cisco designed NetFlow v9 compliant to IPFIX requirements and contributes to documents –Riverstone/Enterasys contributing actively –Juniper is closely monitoring progress Several accounting and billing system providers are monitoring and contributing –HP, XACCT, InMon,... More information at http://ipfix.doit.wisc.edu

28. 28 IETF PSAMP Working Group Packet SAMPling (PSAMP) –BoF session 03/02 –active since 07/02 Initiated by Nick Duffield, AT&T Target: standardizing new technology for sampling, filtering and exporting packets –can be interpreted as a component of the IPFIX measurement process –but different to IPFIX, there is no current practice Chairs –Juergen Quittek, NEC

29. 29 PSAMP Scope and General Requirements Goal: Develop effective but low-cost packet sampling technology Allowing measurements at high-speed links Fulfilling requirements of applications using per packet measurement –QoS analysis, traffic profiling Very low hardware/software costs Much simpler than IPFIX Will use subset of IPFIX protocol Metering to be integrated in general purpose IP routers and other devices (probes, middleboxes) Configuration of sampling included (different than IPFIX)

30. 30 Application PSAMP Architecture Packet Record Observation Point Packet Information Export PAYLOAD HEAD Sampling & Filtering Process Exporting Process Collecting Process

31. 31 PSAMP WG: Expected Output Planned documents –Architecture RFC (in progress) –Packet Sampling and Filtering Spec. RFC (in progress) –Report Format and Protocol specification (close to final document) –PSAMP MIB RFC (close to final document) –Applicability RFC (not started) Dependencies on IPFIX protocol development

32. 32 PSAMP WG: Current Status Good support from IESG (Internet Engineering Steering Group) Growing interest from equipment manufacturers –Main drivers are AT&T, Cisco and NEC –Avaya is actively contributing –Alcatel, Avici, InMon, Lucent are monitoring and joining discussions –Cisco shows strong interest in having PSAMP close to IPFIX in order to re-use their existing IPFIX software