1. http://imk.gmd.de Institute for Media Communication How To Do High Speed Multicast Right! Lothar.Zier@gmd.de Gundula.Doerries@gmd.de

2. 2 High-Speed Multicast –High-Speed Multicast –Distributing data to groups of receivers –Single multicast streams with 100+ Mbit/s –Outline –Challenges and risks –Practical technologies –Hot spots –Summary

3. 3 High-Speed Multicast –Challenges –Software-based multicast was restricted to a few Mbit/s –New hardware allows multicast with hundreds of Mbit/s –Risks –“Multicast disaster” –Server sends much less than expected –Production network is down –Client system does not even seem to receive much of the data –Implementation goals for multicast –End systems send + receive multicast with high data rates –Restricted distribution of multicast inside the network

4. 4 Practical High-Speed Multicast Technologies –Three technologies available for 100+ MBit/s multicast distribution –ATM –Native ATM (Winsock2, XTI APIs) –LAN Emulation (LANEv1; separate VLAN) –Gigabit Ethernet (GE) –IGMP snooping / separate VLAN –IP multicast routing

5. 5 Multicast Throughput - Theoretical Limits ProtocolPacket size/byteThroughput / Mbit/s XTI50282.6 1472536.7 8192541.5 XTI without Padding1432539.5 8152542.0 UDP/LANE50188.4 1472519.9 9190538.2 UDP/LANE without Padding1436523.6 9212539.5 UDP/Gigabit Ethernet50480.8 1472964.6 ATM 622 Mbit/s + Gigabit Ethernet

6. 6 Practical Environment / Measurements –Gigabit Testbed West in Germany –Networking Technologies: ATM (622 MBit/s, 2.4 Gbit/s), Gigabit Ethernet, IP routing (GE + 622 Mbit/s ATM) –End systems: UNIX workstations: Sun, SGI –Network equipment: CISCO, Fore (Marconi) –Multi-Generator (MGEN) Toolset –Tools for sending + receiving multicast packets –ftp://manimac.itds.nrl.navy.mil/Pub/MGEN/ftp://manimac.itds.nrl.navy.mil/Pub/MGEN/ –Extension: –Native ATM – XTI –Bigger buffers

7. 7 Test Environment

8. 8 Sender Performance –Measurements –Sending rate depending on network technology + packet size –XTI/native ATM, LANE, Gigabit Ethernet –Packet sizes: 50 – 1472 (- 8192, - 9000) bytes –Maximum sending rate –Native ATM: ~540 Mbit/s, >5000 bytes packet –LANE: ~510 Mbit/s, >6000 bytes packet –Gigabit Ethernet: ~340 Mbit/s, 1472 bytes packets –Sending performance is limited by bandwidth + number of packets (number of interrupts) –Comparison of technologies –Gigabit Ethernet better for a fixed packet size –LANE / XTI better overall throughput (because of bigger packet sizes)

9. 9 Native ATM (XTI) Send Rate

10. 10 Comparing the Technologies

11. 11 LAN Emulation and Multicast –Multicast with LAN Emulation (Version 1) –Multicast data is send to the BUS and distributed to all clients –The sender also receives its own multicast data – which is discarded by the ATM driver –Comparing multicast and unicast sending rate –Maximum throughput for 1472 byte packets is reduced ~20 % –190 Mbit/s with multicast traffic –240 Mbit/s with unicast traffic

12. 12 Comparing LANE Unicast and Multicast

13. 13 Receiver Performance –High data losses in overloaded receivers –Two techniques for improving –Interrupt Coalescing – network adapter aggregates packets before triggering an interrupt –Increasing the (UDP) receive buffer –Improvement –Reduces receiver losses –Smoothing traffic peaks –No cure-all for slow receivers

14. 14 Losses without/with Interrupt Coalescing

15. 15 IP Multicast Routing Protocols –Methods for constructing IP multicast distribution trees –Flood & prune –Flood all (!) multicast packets in regular intervals in the whole multicast network; prune the distribution tree –DVMRP, PIM dense-mode –Explicit receiver registration –Send only data to explicitly registered receivers –PIM sparse-mode –Flood & prune protocols may cause severe traffic bursts in the whole multicast network –PIM sparse-mode should be used

16. 16 PIM dense-mode Traffic Burst

17. 17 Congestion Avoidance –No feedback mechanism for multicast sender in bottleneck situations –Other traffic flows (especially TCP) are severely damaged –Intelligent queuing mechanisms reduce congestion –Weighted Fair Queuing (WFQ) / Per VC Queuing –Allocating equal bandwidths between traffic flows –Implemented in many modern ATM switches –Improvement –Multicast and TCP traffic get equal bandwidth resources

18. 18 Parallel TCP and Multicast Traffic with WFQ

19. 19 Summary –Modern hardware allows high-speed multicast with hundreds of megabits –Several technologies to realize high-speed multicast in practice –ATM (native, LAN Emulation), Gigabit Ethernet, IP Router –No clear winner –Traffic management for high-speed multicast distribution is absolutely necessary –Protect your (unicast) production traffic –Native ATM, VLANs, IGMP-snooping, PIM sparse-mode –Applications –Use large packets –Feedback mechanisms useful to avoid receiver/network overloading –What data rates are realistic? –200 – 500 Mbit/s for test programs –Your application may reduce this to 100 Mbit/s and below