1 Address Lookup and Classification EE384Y May 23, 2006 Pankaj Gupta Principal Architect and Member of Technical Staff, Netlogic Microsystems

2 Generic Router Architecture (Review from EE384x) Lookup IP Address Update Header Header Processing DataHdrDataHdr ~1M prefixes Off-chip DRAM Address Table Address Table IP AddressNext Hop Queue Packet Buffer Memory Buffer Memory ~1M packets Off-chip DRAM

3 Lookups Must be Fast 1.Lookup mechanism must be simple and easy to implement 2.Memory access time is the bottleneck 200Mpps × 2 lookups/pkt = 400 Mlookups/sec → 2.5ns per lookup YearAggregate Line-rate Arriving rate of 40B POS packets (Million pkts/sec) Mb/s Gb/s Gb/s Gb/s Gb/s200

4 Memory Technology (2006) TechnologyMax single chip density $/chip ($/MByte) Access speed Watts/ chip Networking DRAM 64 MB$30-$50 ($0.50-$0.75) 40-80ns0.5-2W SRAM8 MB$50-$60 ($5-$8) 3-4ns2-3W TCAM2 MB$200-$250 ($100-$125) 4-8ns15-30W Note: Price, speed and power are manufacturer and market dependent.

5 Lookup Mechanism is Protocol Dependent Networking Protocol Lookup Mechanism Techniques we will study MPLS, ATM, Ethernet Exact match search –Direct lookup –Associative lookup –Hashing –Binary/Multi-way Search Trie/Tree IPv4, IPv6Longest-prefix match search -Radix trie and variants -Compressed trie -Binary search on prefix intervals

6 Outline I.Routing Lookups Overview Exact matching –Direct lookup –Associative lookup –Hashing –Trees and tries Longest prefix matching –Why LPM? –Tries and compressed tries –Binary search on prefix intervals References II.Packet Classification

7 Exact Matches in ATM/MPLS VCI/MPLS-label Address Memory Data (Outgoing Port, new VCI/label) VCI/Label space is 24 bits - Maximum 16M addresses. With 64b data, this is 1Gb of memory. VCI/Label space is private to one link Therefore, table size can be “negotiated” Alternately, use a level of indirection Direct Memory Lookup

8 Exact Matches in Ethernet Switches Layer-2 addresses are usually 48-bits long, The address is global, not just local to the link, The range/size of the address is not “negotiable” (like it is with ATM/MPLS) 2 48 > 10 12, therefore cannot hold all addresses in table and use direct lookup.

9 Exact Matches in Ethernet Switches (Associative Lookup) Associative memory (aka Content Addressable Memory, CAM) compares all entries in parallel against incoming data. Network address Data Associative Memory (“CAM”) Address 48bits Match Location Address “Normal” Memory Data Port

10 Exact Matches Hashing Use a pseudo-random hash function (relatively insensitive to actual function) Bucket linearly searched (or could be binary search, etc.) Leads to unpredictable number of memory references Hashing Function Memory Address Data Network Address 48 16, say Pointer Memory Address Data List/Bucket List of network addresses in this bucket

11 Exact Matches Using Hashing Number of memory references

12 Exact Matches in Ethernet Switches Perfect Hashing Hashing Function Memory Address Data Network Address 48 16, say Port There always exists a perfect hash function. Goal: With a perfect hash function, memory lookup always takes O(1) memory references. Problem: - Finding perfect hash functions (particularly a minimal perfect hash) is complex. - Updates make such a hash function yet more complex - Advanced techniques: multiple hash functions, bloom filters…

13 Exact Matches in Ethernet Switches Hashing Advantages: –Simple to implement –Expected lookup time is small –Updates are fast (except with perfect hash functions) Disadvantages –Relatively inefficient use of memory –Non-deterministic lookup time (in rare cases)  Attractive for software-based switches. However, hardware platforms are moving to other techniques (but they can do well with a more sophisticated form of hashing)

14 Exact Matches in Ethernet Switches Trees and Tries Binary Search Tree <> <><> log 2 N N entries Binary Search Trie Lookup time bounded and independent of table size, storage is O(NW) Lookup time dependent on table size, but independent of address length, storage is O(N)

15 Exact Matches in Ethernet Switches Multiway tries 16-ary Search Trie 0000, ptr1111, ptr 0000, 01111, ptr , , ptr Ptr=0 means no children Q: Why can’t we just make it a ary trie?

16 Exact Matches in Ethernet Switches Multiway tries Table produced from 2 15 randomly generated 48-bit addresses As degree increases, more and more pointers are “0”

17 Exact Matches in Ethernet Switches Trees and Tries Advantages: –Fixed lookup time –Simple to implement and update Disadvantages –Inefficient use of memory and/or requires large number of memory references More sophisticated algorithms compress ‘sparse’ nodes.

18 Outline I.Routing Lookups Overview Exact matching –Direct lookup –Associative lookup –Hashing –Trees and tries Longest prefix matching –Why LPM? –Tries and compressed tries –Binary search on prefix intervals References II.Packet Classification

19 Longest Prefix Matching: IPv4 Addresses 32-bit addresses Dotted quad notation: e.g Can be represented as integers on the IP number line [0, ]: a.b.c.d denotes the integer: (a*2 24 +b*2 16 +c*2 8 +d) IP Number Line

20 Class-based Addressing ABCD E ClassRangeMS bitsnetidhostid A – bits 1-7bits 8-31 B bits 2-15bits C bits 3-23bits D (multicast) E (reserved)

21 Lookups with Class-based Addresses Port 1 Port Class A Class B Class C Port 3 Exact match netidport#

22 Problems with Class-based Addressing Fixed netid-hostid boundaries too inflexible –Caused rapid depletion of address space Exponential growth in size of routing tables

23 Early Exponential Growth in Routing Table Sizes Number of BGP routes advertised

24 Classless Addressing (and CIDR) Eliminated class boundaries Introduced the notion of a variable length prefix between 0 and 32 bits long Prefixes represented by P/l: e.g., 122/8, /13, /22, /32 etc. An l-bit prefix represents an aggregation of 2 32-l IP addresses

25 CIDR:Hierarchical Route Aggregation Backbone Router R1 R2 R3 R4 ISP, PISP, Q / /22 Site, S /24 Site, T / / / / / /22, R2 Backbone routing table IP Number Line R2

26 Post-CIDR Routing Table sizes Source: Number of active BGP prefixes Optional Exercise: What would this graph look like without CIDR? (Pick any one random AS, and plot the two curves side-by-side)

27 Routing Lookups with CIDR /22, R /24, R / / / /22, R LPM: Find the most specific route, or the longest matching prefix among all the prefixes matching the destination address of an incoming packet Optional Exercise: Find the ‘nesting distribution’ for routes in your randomly-picked AS

28 Longest Prefix Match is Harder than Exact Match The destination address of an arriving packet does not carry with it the information to determine the length of the longest matching prefix Hence, one needs to search among the space of all prefix lengths; as well as the space of all prefixes of a given length

29 LPM in IPv4 Use 32 exact match algorithms for LPM! Exact match against prefixes of length 1 Exact match against prefixes of length 2 Exact match against prefixes of length 32 Network Address Port Priority Encode and pick

30 Metrics for Lookup Algorithms Speed (= number of memory accesses) Storage requirements (= amount of memory) Low update time (support >10K updates/s) Scalability –With length of prefix: IPv4 unicast (32b), Ethernet (48b), IPv4 multicast (64b), IPv6 unicast (128b) –With size of routing table: (sweetspot for today’s designs = 1 million) Flexibility in implementation Low preprocessing time

31 Radix Trie (Recap) P1111*H1 P210*H2 P31010*H3 P410101H4 P2 P3 P4 P1 A B C G D F H E Lookup Add P5=1110* I 0 P5 next-hop-ptr (if prefix) left-ptr right-ptr Trie node

32 Radix Trie W-bit prefixes: O(W) lookup, O(NW) storage and O(W) update complexity Advantages  Simplicity  Extensible to wider fields Disadvantages  Worst case lookup slow  Wastage of storage space in chains

33 Leaf-pushed Binary Trie A B C G D E left-ptr or next-hop Trie node right-ptr or next-hop P2 P4P3 P2 P1 111*H1 P210*H2 P31010*H3 P410101H4

34 PATRICIA 2 A B C E Patricia tree internal node 3 P3 P2 P4 P F G D 5 bit-position left-ptr right-ptr Lookup P1111*H1 P210*H2 P31010*H3 P410101H4 Bitpos 12345

35 W-bit prefixes: O(W 2 ) lookup, O(N) storage and O(W) update complexity Advantages  Decreased storage  Extensible to wider fields Disadvantages  Worst case lookup slow  Backtracking makes implementation complex PATRICIA

36 Path-compressed Tree 1, , 2 A B C ,P2,4 P4 P1 1 0 E D 1010,P3,5 bit-position left-ptrright-ptr variable-length bitstring next-hop (if prefix present) Path-compressed tree node structure Lookup P1111*H1 P210*H2 P31010*H3 P410101H4

37 W-bit prefixes: O(W) lookup, O(N) storage and O(W) update complexity Advantages  Decreased storage Disadvantages  Worst case lookup slow Path-compressed Tree

38 Multi-bit Tries Depth = W Degree = 2 Stride = 1 bit Binary trie W Depth = W/k Degree = 2 k Stride = k bits Multi-ary trie W/k

39 Prefix Expansion with Multi-bit Tries If stride = k bits, prefix lengths that are not a multiple of k need to be expanded PrefixExpanded prefixes 0*00*, 01* 11* E.g., k = 2: Maximum number of expanded prefixes corresponding to one non-expanded prefix = 2 k-1

40 Four-ary Trie (k=2) P2 P3P1 2 A B F 11 next-hop-ptr (if prefix) ptr00ptr01 A four-ary trie node P P4 2 H 11 P D C E G ptr10ptr11 Lookup P1111*H1 P210*H2 P31010*H3 P410101H4

41 Prefix Expansion Increases Storage Consumption Replication of next-hop ptr Greater number of unused (null) pointers in a node Time ~ W/k Storage ~ NW/k * 2 k-1 Optional Exercise: The increase in number of null pointers in LPM is a worse problem than in exact match. Why?

42 Generalization: Different Strides at Each Trie Level split split 24-8 split split Optional Exercise: Why does this not work well for IPv6?

43 Choice of Strides: Controlled Prefix Expansion [Sri98] Given a forwarding table and a desired number of memory accesses in the worst case (i.e., maximum tree depth, D) A dynamic programming algorithm to compute the optimal sequence of strides that minimizes the storage requirements: runs in O(W 2 D) time Advantages  Optimal storage under these constraints Disadvantages  Updates lead to sub- optimality anyway  Hardware implementation difficult

44 Binary Search on Prefix Intervals [Lampson98] P1 P4 P3 P5 P2 PrefixInterval P1/00000…1111 P200/20000…0011 P31/11000…1111 P41101/41101…1101 P5001/30010… I1I1 I3I3 I4I4 I5I5 I6I6 I2I2

45 I1I1 I3I3 I2I2 I4I4 I5I5 I6I  > Alphabetic Tree 1/21/4 1/8 1/16 1/32     > > > > P1 P4 P3 P5 P I1I1 I3I3 I4I4 I5I5 I6I6 I2I2

Another Alphabetic Tree I1I1 I2I2 I5I5 I3I3 I4I4 I6I /2 1/4 1/8 1/16 1/32

47 Advantages  Storage is linear  Can be ‘balanced’  Lookup time independent of W Disadvantages  But, lookup time is dependent on N  Incremental updates more complex than tries  Each node is big in size: requires higher memory bandwidth W-bit N prefixes: O(logN) lookup, O(N) storage Multiway Search on Intervals

48 Routing Lookups: References [lulea98] A. Brodnik, S. Carlsson, M. Degermark, S. Pink. “Small Forwarding Tables for Fast Routing Lookups”, Sigcomm 1997, pp [Example of techniques for decreasing storage consumption] [gupta98] P. Gupta, S. Lin, N.McKeown. “Routing lookups in hardware at memory access speeds”, Infocom 1998, pp , vol. 3. [Example of hardware-optimized trie with increased storage consumption] P. Gupta, B. Prabhakar, S. Boyd. “Near-optimal routing lookups with bounded worst case performance,” Proc. Infocom, March 2000 [Example of deliberately skewing alphabetic trees] P. Gupta, “Algorithms for routing lookups and packet classification”, PhD Thesis, Ch 1 and 2, Dec 2000, available at ~pankaj/phd.html [Background and introduction to LPM]

49 Routing lookups : References (contd) [lampson98] B. Lampson, V. Srinivasan, G. Varghese. “ IP lookups using multiway and multicolumn search”, Infocom 1998, pp , vol. 3. [Multi-way search] [PerfHash] Y. Lu, B. Prabhakar and F. Bonomi. “Perfect hashing for network applications”, ISIT, July 2006 [LC-trie] S. Nilsson, G. Karlsson. “Fast address lookup for Internet routers”, IFIP Intl Conf on Broadband Communications, Stuttgart, Germany, April 1-3, [sri98] V. Srinivasan, G.Varghese. “Fast IP lookups using controlled prefix expansion”, Sigmetrics, June [wald98] M. Waldvogel, G. Varghese, J. Turner, B. Plattner. “Scalable high speed IP routing lookups”, Sigcomm 1997, pp