Monitoring very high speed links Gianluca Iannaccone Sprint ATL joint work with: Christophe Diot – Sprint ATL Ian Graham – University of Waikato Nick McKeown – Stanford University

November 2nd, 2001IMW Packet-level trace collection: the DAG example Optical splitter to divert part of the signal toward the capture card. The capture card stores a record with a timestamp for each packet. The card uses the PCI bus to transfer the records to the host main memory. It is used by the Sprint IP Monitoring Project.

November 2nd, 2001IMW Challenges for next generation capture devices PCI bus throughput: –PCI 66Mhz/64bit is already challenged at OC-48 speed. Storage capacity and data management: –Several terabytes per trace collection. Memory access speed. Disk array speed: –OC-192 would require roughly 250Mbytes/sec (64 byte records, 300 byte average packet size).

November 2nd, 2001IMW Challenges for next generation capture devices (contd) Deployment of extensive monitoring facilities inside the routers: –Backplane speeds are in the order of hundreds of Gbps; –Stringent power and space constraints; –Switched backplanes reduce the advantage of being inside the router.

November 2nd, 2001IMW Our Goals Be capable of collecting packet at OC-192 speed with the current PCI technology. Develop a 4:1 compression technique. Minimize the loss of information due to compression.

November 2nd, 2001IMW System requirements Online processing required for compression. At OC-192 speed (10Gbps), a new packet arrives every 240ns (300-byte packets). –We can assume large average packet sizes because we can buffer incoming packets (once timestamped).

November 2nd, 2001IMW From packet traces to flow traces Store information at the flow-level: –a flow is defined by the usual 5-tuple. Flow information are shared among all the packets belonging to the same flow. Store few information per packet: –packet arrival time; –size; –sequence numbers –…

November 2nd, 2001IMW Flow and packet records TCP: 28 bytes/flow; 16 bytes/pkt; UDP: 20 bytes/flow; 12 bytes/pkt; Other: 16 bytes/flow; 12 bytes/pkt;

November 2nd, 2001IMW Flow termination Records of terminated flows are dumped to the host memory (and then to disk). Flow termination relies on a timeout: –A short timeout will require less memory; –A long timeout will give better compression rates; –Anyway, the timeout does not affect the accuracy of the system because we can post- process the trace;

November 2nd, 2001IMW Flow fragmentation Long-lived flows can be a problem: –Large records and high bursts of traffic on the PCI bus; We set a limit to memory occupancy for a flow record (with relative packet records): –If the memory limit is exceeded, the Record Number is increased and the record is stored. –When the flow terminates the entire record is stored to disk with the Last Record flag set.

November 2nd, 2001IMW Memory required

November 2nd, 2001IMW Preliminary performance study We have built a simple emulator to compress a packet-level trace. 5 different traces with different link utilizations and traffic patterns. Compression rates ranging between 3.20 and 4.02 (compared to DAG3 packet traces).

November 2nd, 2001IMW Hardware architecture Steps required in trace collection: –process framing information; –timestamp the packet; –classify the packet; –update flow/packet record; –store terminated flows to disk (background); All these steps can be pipelined: –Each step can be performed during the interval between two packet arrivals.

November 2nd, 2001IMW Packet classification May be very expensive in terms of computation time and resources: –One-to-one mapping between packets and flows. But a simple hash function may be enough: –Requires a large amount of fast memory; –Collisions can be solved using a second hash function or a lookup trie.

November 2nd, 2001IMW Conclusion New compression techniques are necessary to monitor OC-192/OC-768 links. We have proposed a novel scheme for flow- based trace collection. We have shown how to achieve compression ratio of 4:1 without losing much information.

November 2nd, 2001IMW Open issues and future work Detailed cost analysis (chipsets, memory). Packet classification performance. Second stage of record compression. Techniques to cope with traffic anomalies. Sampling techniques.