1. Networks ∙ Services ∙ People www.geant.org Mian Usman TNC15, Porto GÉANT IP Layer 17 th June 2015 IP Network Architect, GÉANT

2. Networks ∙ Services ∙ People www.geant.org 2 Contents Service Architecture Challenges SDN Approach White Box Switches Services and IP Trunks over Infinera Packet TIM

3. Networks ∙ Services ∙ People www.geant.org 3 Service Architecture Dark Fibre Infrastructure IP/MPLS Network Network DWDM Network Services IP, L2VPN, L3VPN, MDVPN, Multicast, etc Operations

4. Networks ∙ Services ∙ People www.geant.org 4 Challenges Diverse Service Portfolio Support of Multicast and MDVPN type services Drive for early adoption of JUNOS version Demanding Big Science Users High performing and stable yet innovative network Deliver services at competitive cost Committed to long term contracts and relationship with supplier High switching cost (Dark fibre)

5. Networks ∙ Services ∙ People www.geant.org Synectics model of “Cycling Worlds” 5 Innovation in GÉANT Project Service Activity World Joint Research Activity World

6. Networks ∙ Services ∙ People www.geant.org SA1 will carry out two studies and deploy a pilot implementation based on the results: Layer 1 infrastructure study Multi-domain NREN Transit network Run technology trials to assess new technologies and new network solutions to enhance the GÉANT network and improve cost efficiency. Create a GÉANT network evolution plan 6 SA1 and JRA2 Work JRA2 will look at how the existing service and platform could be evolved and what new SDN based connectivity service be offered to NREN community: SDN based connectivity services (BoD) POC of SDX and SDN-IP using white label switches SDN at Optical Layer using Infinera Packet TIM INaaS (Infrastructure and Network as a Service) NetIC (Network in Campus)

7. Networks ∙ Services ∙ People www.geant.org GÉANT Network Topology

8. Networks ∙ Services ∙ People www.geant.org GÉANT Network Topology GÉANT leveraging on the existing NREN infrastructure to build GÉANT IP network Layer 1 study will look at which fibre routes could be potentially replaced with CBFs or Alien Waves Multi-domain transit network study will look at how NRENs can use the existing NREN and GÉANT optical infrastructure to peer with and offer transit to each other

9. Networks ∙ Services ∙ People www.geant.org Separate elements of control plane into individual controllers Reduce reliance on the vendor OS Reduce service interruptions due to OS issues Reduce the time it takes to test the OS version prior to upgrades Multicast service managed by a separate controller No Multicast on Juniper Routers 9 SDN Approach

10. Networks ∙ Services ∙ People www.geant.org GÉANT Network Topology

11. Networks ∙ Services ∙ People www.geant.org Juniper Support Contract up for renewal in Aug-16 Opportunity to replace Juniper MX routers in smaller PoPs with White Box Switches Reduce the maintenance and support cost Offer additional functionality Innovative solutions in GÉANT production network 11 White Box Switches

12. Networks ∙ Services ∙ People www.geant.org GÉANT running out of slots on Juniper MX in bigger PoPs e.g. Frankfurt, Amsterdam and Geneva 12 100G Aggregation Switches

13. Networks ∙ Services ∙ People www.geant.org GÉANT Network Topology

14. Networks ∙ Services ∙ People www.geant.org PXM GÉANT IP Trunk Over Packet Tim Use Case Current GÉANT Eastern Ring Topology IP Trunks follow the physical fibre path Started the trunk link optimization based on traffic flows Majority of the traffic flows from Western Ring to Eastern Ring Majority of the traffic goes through AT or HU router Majority of the traffic on CZ and SK router is a pass through traffic

15. Networks ∙ Services ∙ People www.geant.org PXM GÉANT IP Trunk Over Packet Tim Use Case Trunk Link Optimization using pass through links No through traffic via smaller PoPs e.g. GR, SI, SK, HR

16. Networks ∙ Services ∙ People www.geant.org PXM GÉANT IP Trunks Over Packet Tim

17. Networks ∙ Services ∙ People www.geant.org PXM GÉANT IP Trunk Over Packet Tim Use Case Reduce the number of 10GE interfaces Reduce the flow limitations based on 10GE interfaces Makes capacity planning easier Reduce the number of slots needed on the Juniper MX router Packet TIM can be used to upgrade Western Ring hence reducing the number of 100G required on Juniper

18. Networks ∙ Services ∙ People www.geant.org GÉANT Network Topology

19. Networks ∙ Services ∙ People www.geant.org PXM GÉANT Services Over Packet Tim The Packet TIM can also be to used deliver deterministic network services to the NRENs and end users Bandwidth on-demand and GÉANT Plus circuits can also be delivered over optical layer The big science users overlay networks like LHCONE and LHCOPN can also benefit from using packet TIM E-LAN service which connects NRENs at Optical Layer and enable them to peer directly with each other or using a route server

20. Networks ∙ Services ∙ People www.geant.org PXM OTS enabled Open Transport Switch (OTS) provides an Open Flow interface to the DTN-X OTS adds transport extensions to the Open Flow interface Initially REST based interface First availability expected in Q4 2015 however, early versions available now for evaluation. OTS would make a useful API to allow big science users to request EVP-LINE and EVP-LAN services via PXM

21. Networks ∙ Services ∙ People www.geant.org PXM Evaluation and trial Field trial in 2015. Evaluation and service development Field trial integration of PXM and OTS. Evaluate OTS and PXM together.

22. Networks ∙ Services ∙ People www.geant.org Cambridge Lab 3 x Juniper MX-480 1 x Juniper MX-960 1 x Juniper M120 2 x PICA8 Switches 2 x Dell Switches (Cumulus Linux) Servers for VMs Working with vendors to test new products (Infinera Packet TIM) 22 Cambridge Lab

23. Networks ∙ Services ∙ People www.geant.org 23 Summary Optimise the Layer 3 network by using direct pass through trunks and removing routers from some PoPs Move services and traffic down a layer to reduce the cost of the network Make use of SDN and advance path computation to build intelligence in the network

24. Networks ∙ Services ∙ People www.geant.org Thank you and any questions Networks ∙ Services ∙ People www.geant.org 24 mian..usman@geant.org

25. Networks ∙ Services ∙ People www.geant.org PXM Use case – LHCOPN/LHCONE LHCOPN: High capacity TDM, but not flexible LHCONE: flexibility of L3 VPN, but no reserved bandwidth Can we give users the best of both solutions?

26. Networks ∙ Services ∙ People www.geant.org NREN GÉANT 500G fiber cloud NREN Tier 1 site Wigner Data Center User service requests Tier 2 Campus CERN PXM EPL & EVPL Services over Layer 1 VPN MEF type EVP-LAN or EVP-Line pool of provisionable OTN B/W OTS REST/OF API to allow experimenter’s applications to manage connectivity OF/REST

27. Networks ∙ Services ∙ People www.geant.org LHCOPN NRENs are providing LHC with point-to-point Layer2 circuits LHC centers built a virtual routed network out of the circuits LHC centers are providing Network Services to each other: CERN is providing un-restricted transit Some centers are providing limited transit Some LHC centers are peering NRENs support individual link operations & management LHC Sites are responsible for network management (layer 3 configurations) including operations, monitoring, troubleshooting, capacity planning, security management, AUP enforcement, etc.

28. Networks ∙ Services ∙ People www.geant.org LHCONE – as it exists at the moment LHCONE is currently setup as an overlay VRF on existing NREN infrastructure which are interconnected via regional networks and Open Exchanges NRENs provide the network including core links and routers as a virtual overlay on their regular infrastructure NRENs have a peering or transit relationship with each other LHC centers are strictly users of the services Restrictions apply to the advertised IP Space LHCONE infrastructure includes dedicated access, Trans-Atlantic and some backbone links

29. Networks ∙ Services ∙ People www.geant.org LHC Workshop Recap Rather than maintaining distinct networks, the LHC community should aim to unify its network infrastructure Traffic aggregation on few links Concerns If the T1s and T2s upgrade to 100G, then the global infrastructure needs to follow LHCONE Evolution Currently LHCONE exists side-by-side with general R&E infrastructure Traffic is segregated but what's the real benefit?