1. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Label scalability of Carrier Ethernet Benchmarking Carrier Ethernet Technologies Workshop Session AI.2 - Scientific and Technical Results EuroNGI 2008, Krakow, Poland April 30, 2008 Wouter Tavernier, Koen Casier (Ghent University – IBBT) Luis Caro (University of Girona) Dimitri Papadimitriou (Alcatel-Lucent Bell NV)

2. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Agenda PROBLEM STATEMENT Labels in Carrier Ethernet Label scalability Label optimization Results Short word on label lookups Conclusion

3. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Problem statement Evolution of Ethernet being a LAN technology towards a Carrier Technology Ethernet = low cost Ethernet = ubiquitous Ethernet = plug & play LAN environments: traffic streams between tens of end-users Metro/Access environments: traffic streams between thousands of end-users PROBLEM STATEMENT: Is Carrier Ethernet able to cope with increasing number of traffic streams LAN Metro network L

4. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN No labels in bridged Ethernet A B C X Y Z ZCZC 1 2 3 5 1 2 3 4 DestOut Port X1 Z3 A4 DestOut Port B2 X5 Z5 C3 Forwarding in native Ethernet bridging : stateless (CL) 48 bit MAC-address based SAPayloadDA 766 SyncSD 146 - 1500 EthType 2 ZCZC ZCZC ZCZC ZCZC

5. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Labels in Carrier Ethernet A BC D E The concept of a connection (LSP) allows for (CO): Traffic Engineering Advanced recovery/protection techniques BW guarantees Forwarding is based on a label linked to the connection state Two connections (LSPs) from A to E Two connections (LSPs) from A to E: maintains state for the green and red connection and forwards based on a label Labell

6. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Domain-wide PBB-TE label A BC D E Distinct routes for dest E, by diff B-VID. Ingress A has 2 LSPs to E, for e.g.: Protection switching Load balancing B-VID 100 reused on same link Label remains constant along connection (no SWAP)! E,200

7. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Link local ELS label A BC D E Distinct routes for dest E, by diff S-VID. Ingress A has 2 LSPs to E, for e.g.: Protection switching Load balancing Label remains constant along connection (no SWAP)! 100 Label can be swapped in intermediate hops along connection! 200 B-VID 100 reused on same link 100 S-VID 100 reused on other link

8. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Label usage optimization Label balancing (online routing optimized ELS) Shortest path routing takes into account labels used on a link as cost Merging (ELS) 1 incoming label per interface + 1 outgoing label GAIN: x-1 labels on outgoing local interface Shared forwarding (PBB-TE) 1 label needed GAIN: x-1 global labels...... x incoming

9. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Label scalability Given the label: How do different label schemes react under changing conditions? What is the influence of the topology? What is the influence the traffic matrix? What is the influence of the traffic matrix on typical BW usage? ELS PBB-TE

10. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Study assumptions Study the impact on label usage of the following dimensions: Topology (CONNECTEDNESS) Traffic matrix (SIZE, UNIFORMITY) BW usage NORMALIZATION: Tests have run 100 times Only one dimension has been changed in a time Shortest Path Routing (Dijkstra hop count) is taken as base routing algorithm Base topology of 100 nodes (connectedness +-2) & 1000 demands A B C D E 1 2 3 4-9 10 i ii L i x a b L

11. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness of a network Connectedness of a topology: what is the average node degree of a node in a topology: ring vs. full mesh Example: Single (un-)connected 4-connected Full mesh

12. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness of the topology S

13. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness vs BW (Luis)  The topology generator considered for generating the topologies in this section is IGEN  For all the topologies the link capacity is set to 10Gb/s and bandwidth request of 100Mb, 200Mb and 300Mb are generated until the network is overloaded.  The implemented algorithm is the SPF  Overload network with links of 10G capacity A

14. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness vs. BW (Luis)

15. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness vs. BW (Luis)

16. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Topology size vs. BW (Luis)

17. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Topology size vs. BW (Luis)

18. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Uniformity of the traffic matrix UNIFORMITY: X demands directed to 1 vs. x destinations? Example: 4 demands 2 destinations vs 4 destinations 4 paths / 1 dest4 paths / 2 dest4 paths / 4 dest

19. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Uniformity of the traffic matrix S

20. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Size of the traffic matrix The size of a traffic matrix affects the number of demands that are routed over a network: 10 demands 100 demands 1000 demands 10000 demands …

21. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Scaling the order of the traffic S

22. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Lookups Must be Fast 1.Lookup mechanism must be simple and easy to implement 2.Memory access time is the bottleneck 200Mpps × 2 lookups/pkt = 400 Mlookups/sec → 2.5ns per lookup YearAggregate Line-rate Arriving rate of 40B POS packets (Million pkts/sec) 1997622 Mb/s1.56 19992.5 Gb/s6.25 200110 Gb/s25 200340 Gb/s100 200680 Gb/s200

23. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Direct lookup in ELS vs PBB-TE ELS: Label space is 12 bits With 64b data, this is 256 K of memory. Label space is private to one link Therefore, table size can be “negotiated” SVID/MPLS-label Address Memory Data (Outgoing Port, new SVID label) Direct Memory Lookup PBB-TE: Label space is 48+12 bits With 64b data, this is 64 EiB of memory (2^60). Label space is global 2^60 > 2^12, therefore cannot hold all addresses in table and use direct lookup, less efficient alternatives: Hashing Binary/Multi-way Search Trie/Tree

24. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Conclusion The specific traffic matrix & topology used, clearly affect label usage The uniformity of the traffic matrix affects PBB-TE-type domain-wide labelling: more uniform is better The topology connectedness affects ELS-type link-local labelling: more connected is better In typical metro-network with low node-degree, PBB-TE & ELS LL have similar performance PBB-TE with SF and ELS with merging consistently score better than alternatives Node local labelling techniques consistently score worse than alternatives. The label length affects the memory space needed and accordingly affects cost and lookup speed

25. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Appendices

26. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Carrier-Grade Ethernet challenges Flat address space  scalability of number of MAC addresses to be learned Beyond 100k learned addresses per node seems challenging Scalability in terms of number of VLANs 12-bit VLAN ID (VID): 4k VLANs possible network wide STP cannot converge faster than worst case 20s sec (root failure) Traffic Engineering (routing) constraint by tree based structure. Limited OA&M

27. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Reference network A B C D E 1 2 3 4-9 10 i ii L i x a b

28. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Assumptions (Luis)  Results are evaluated in terms of the maximum and average number of: For ELS (average calculated based on the number of links):  Labels per link  Labels per link with agg (meaning only paths are count)  Labels per link with agg and label merging For PBB-TE  Labels per destination  Labels per destination with agg (meaning only paths are count)  Labels per destination with VLAN-reut (meaning link disjoint paths can use same label)  Labels per destination with INV-trees (paths that intersect only on a common segment ending at the destination can also use the same label) B

29. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Tests on smaller networks not normalized

30. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Uniformity in small 3-connected network S

31. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Connectedness in 28n network S

32. Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Traffic scaling in small network S