1. London Internet Exchange Point Update Keith Mitchell, Executive Chairman NANOG15 Meeting Denver, Jan 1999

2. LINX Update LINX now has 63 members Second site now in use New Gigabit backbone in place Renumbered IXP LAN Some things we have learned! Statistics What’s coming in 1999

3. What is the LINX ? UK National IXP Not-for-profit co-operative of ISPs Main aim to keep UK domestic Internet traffic in UK Increasingly keeping EU traffic in EU

4. LINX Status Established Oct 94 by 5 member ISPs Now has 7 FTE dedicated staff Sub-contracts co-location to 2 neutral sites in London Docklands: Telehouse TeleCity Traffic doubling every 4-6 months !

5. LINX membership Now totals 63 +10 since Oct 98 Recent UK members: RedNet XTML Mistral ICLnet Dialnet Recent non-UK members: Carrier 1 GTE Above Net Telecom Eireann Level 3

6. LINX Members by Country

7. Second Site Existing Telehouse site full until 99Q3 extension ready TeleCity is new dedicated co-lo facility, 3 miles from Telehouse Awarded LINX contract by open tender (8 submissions) LINX has 16-rack suite Space for 800 racks

8. Second Site LINX has diverse dark fibre between sites (5km) Same switch configuration as Telehouse site Will have machines to act as hot backups for the servers in Telehouse Will have a K.root server behind a transit router soon

9. LINX Traffic Issues Bottleneck was inter-switch link between Catalyst 5000s Cisco FDDI could no longer cope 100baseT nearly full Needed to upgrade to Gigabit backbone within existing site 98Q3

10. Gigabit Switch Options Looked at 6 vendors: Cabletron/Digital, Cisco, Extreme, Foundry, Packet Engines, Plaintree Some highly cost-effective options available But needed non-blocking, modular, future-proof equipment, not workgroup boxes

11. Old LINX Infrastructure 5 Cisco Switches: 2 x Catalyst 5000, 3 x Catalyst 1200 2 Plaintree switches 2 x WaveSwitch 4800 FDDI backbone Switched FDDI ports 10baseT & 100baseT ports Media convertors for fibre ether (>100m)

12. Old LINX Topology

13. New Infrastructure Catalyst and Plaintree switches no longer in use Catalyst 5000s appeared to have broadcast scaling issues regardless of Supervisor Engine Plaintree switches had proven too unstable and unmanageable Catalyst 1200s at end of useful life

15. New Infrastructure Packet Engines PR-5200 Chassis based 16 slot switch Non-blocking 52Gbps backplane Used for our core, primary switches One in Telehouse, one in TeleCity Will need a second one in Telehouse within this quarter Supports 1000LX, 1000SX, FDDI and 10/100 ethernet

16. New Infrastructure Packet Engines PR-1000: Small version of PR-5200 1U switch; 2x SX and 20x 10/100 Same chipset as 5200 Extreme Summit 48: Used for second connections Gives vendor resiliency Excellent edge switch - low cost per port and 2x Gigabit, 48x 10/100 ethernet

17. New Infrastructure Topology changes: Aim to be able to have major failure in one switch without affecting member connectivity Aim to have major failures on inter-switch links with out affecting connectivity Ensure that inter-switch connections are not bottlenecks

18. New backbone All primary inter-switch links are now gigabit New kit on order to ensure that all inter-switch links are gigabit Inter-switch traffic minimised by keeping all primary and all backup traffic on their own switches

19. IXP Switch Futures Vendor claims of 1000baseProprietary 50km+ range are interesting Need abuse prevention tools: port filtering, RMON Need traffic control tools: member/member bandwidth limiting and measurement

20. Address Transition Old IXP LAN was 194.68.130/24 New allocation 195.66.224/19 New IXP LAN 195.66.224/23 “Striped” allocation on new LAN 2 addresses per member, same last octet About 100 routers involved

21. Address Migration Plan Configured new address(es) as secondaries Brought up peerings with their new addresses When all peers are peering on new addresses, stopped old peerings Swap over the secondary to the primary IP address

22. Address Migration Plan Collector dropped peerings with old 194.68.130.0/24 addresses Anyone not migrated at this stage lost direct peering with AS5459 Eventually, old addresses no longer in use

23. What we have learned... the hard way! Problems after renumbering Some routers still using /24 netmask Some members treating the /23 network as two /24s Big problem if proxy ARP is involved! Broadcast traffic bad for health We have seen >50 ARP requests per second at worst times.

24. ARP Scaling Issues Renumbering led to lots of ARP requests for unused IP addresses ARP no-reply retransmit timer fixed time-out Maintenance work led to groups of routers going down/up together  Synchronised “waves” of ARP requests

25. New MoU Prohibitions Proxy ARP ICMP redirects Directed broadcasts Spanning Tree IGP broadcasts All non-ARP MAC layer broadcasts

26. Statistics LINX total traffic 300 M/sec avg, 405 M/sec peak Routing table 9,200 out of 55,000 routes k.root-servers 2.2 Mbit/sec out, 640 Kbit/sec in nic.uk 150 Kbit/sec out, 60 Kbit/sec in

27. Statistics and looking glass at http://www2.linx.net/

28. Things planned for ‘99 Infrastructure spanning tree implementation Completion of Stratum-1 NTP server Work on an ARP server Implementation of route server Implementation of RIPE NCC test traffic box