1. Multi-Domain Virtual Private Network service
A seamless infrastructure for NRENs, GEANT and NORDUnet
GN3+ project SA3T3 team :
Xavier Jeannin (RENATER) - presenter, Tomasz Szewczyk (PSNC), Bojan Jakovljevic (AMRES), Thomas Schmid (DFN), Dave Wilson (HEAnet), Brian Bach Mortensen (NORDUnet)
TNC 15, Porto, Portugal
15-18 June 2015

2. What is MD-VPN? The service provides a seamless, scalable transport infrastructure
A joint service provided by the GÉANT network and NRENs
A seamless transport infrastructure that provides a connectivity service:
Layer3 or Layer2 VPNs spanning several domains
point-to-point or multipoint
Multi-domain networking
IPv4
IPv6
L3VPN
P2P L2VPN
MD - VPN
MP L2VPN

3. MD-VPN service highly scalable, seamless transport infrastructure
Configure only at the edge
NREN OPEX Reduced
VPN Provisioning as easy as in a single-domain
Easy to deploy
No CAPEX
VPN multiplexed
Lead-time reduced
An end-to-end extensible and flexible service
Configure only at the edge

4. A double benefit for NRENs with regional network

5. An innovative design with added value for end-users
An original connectivity network service
Multi-domain networking
Facilitate an foster distributed collaboration in Europe
Cover a wide scope of use cases
Reduce OPEX and CAPEX for use
Cost saving – VPN cheaper
Cost saving – No tender for research project
Safe infrastructure
Security opex saved on site
Reduce firewall usage

6. MD-VPN use cases A wide scope for MD-VPN use
All scientific projects based on international collaboration
LHCONE is an example of successful L3VPN multi-domain service
ITER, CONFINE
Distributed infrastructure
Cloud provider
Grid – HPC center
Scientific infrastructure: Telescope, sensor network

7. MD-VPN use cases A wide scope for MD-VPN use
Quick P2P connection
Conference demonstration
P2P data transport between to sites
_news
No firewall – smaller delay – better TCP throughput
Education
Remote lecture
E-learning

8. MD-VPN use cases A wide scope for MD-VPN use
MD-VPN transparent data transport layer for high level network services like SDN, … and in general by future internet project
No firewall – smaller delay – better TCP throughput

9. How does it work?
Underlying principle behind this Multi-Domain VPN technology
The LSP is extended from a PE up to the remote PE in another domain
Signaling is split in 2 parts
Signalling for multi-domain MPLS path between PE routers thanks to a BGP peering with labelled unicast SAFI (internal route)
Signalling for VPN labels and prefixes exchange between PE routers (external route) thanks to an external BGP VPNv4 family peering
GEANT and NORDUnet implement Carrier of Carriers (CoC) providing transparent transport of VPN traffic

10. MDVPN: BGP-signalling L2VPN, L3VPN
Multi-hop eBGP
VPNv4, VPNv6, L2VPN
iBGP
BGP-signaled L2VPN and L3VPN
label and prefix exchange
eBGP
labeled-unicast
iBGP
Multi-domain PE to PE
MPLS path

11. MDVPN: tLDP-signalling L2 circuit
Targeted LDP -signaled L2 circuit
label exchange
eBGP
labeled-unicast
iBGP
Multi-domain PE to PE MPLS path

12. MDVPN data plane label operations
incoming packet
push VPN label
push transport label
push LDP label
outgoing
packet
pop VPN label
swap transport label
pop transport
label
pop LDP label
swap transport label
swap transport label
swap transport label
push CoC label
pop CoC label
swap CoC label
MDVPN packets labels:
With the courtesy of Jani Myyry (Funet)
LDP
label
Transport
label
VPN
label
Data
CoC
label
Transport
VPN
Data

13. Global view of the service
Geographical extensibility
Service extensibility

14. CPE-NREN-B-VPN-ASTRO
GEANT
CPE-NREN-A-VPN-ASTRO
Peering BGP VPNv4
CPE-NREN-B-VPN-ASTRO
PE-RENATER
ASBR-1-GEANT
ASBR-NREN-A
ASBR-2-GEANT
ASBR-NREN-B
PE-NREN-B
RR-NREN-B
RR- NREN-A
NREN-A
NREN B
Peering Multi-hop E-BGP VPNv4 (No next-hop self)
Physical connections
Peering labeled-unicast
VRF ASTRO RT:22:30
VRF BIO - RT:22:32
VRF md-vpn1 - RT:33:10
VRF md-vpn2 - RT:13092:17
L2Circuit toward AMRES
L2Circuit PE-RENATER - PE-REMOTE-NREN
Standard Deployment
VPN-Route-Reflector
VRF CoC

15. CPE-NREN-B-VPN-ASTRO
GEANT
CPE-NREN-A-VPN-ASTRO
Peering BGP VPNv4
CPE-NREN-B-VPN-ASTRO
PE-RENATER
ASBR-1-GEANT
ASBR-NREN-A
ASBR-2-GEANT
ASBR-NREN-B
PE-NREN-B
RR- NREN-A
NREN-A
NREN B
Peering Multi-hop E-BGP VPNv4 (No next-hop self)
Physical connections
Peering labeled-unicast
VRF ASTRO RT:22:30
VRF BIO - RT:22:32
VRF CoC
VRF md-vpn1 - RT:33:10
VRF md-vpn2 - RT:13092:17
L2Circuit toward AMRES
L2Circuit PE-RENATER - PE-REMOTE-NREN
Limited deployment
VPN-Route-Reflector
MPLS is enabled only on the AS Border Router
VPN is propagated internally by any other internal means: VLAN, dedicated link, other solutions …

16. Where can you use MD-VPN?
MD-VPN service in the GÉANT portfolio
18 NRENs connected
(+ 1 NREN using MD-VPN Proxy + 1 NREN still working on)
Roughly 400 PoPs available that European scientist can already use MD-VPN

17. Reliability demonstrated since August 2014
Pilot phase :Service reliability checking during 3 months
Statistics available at m.jsp

18. A monitored service
atus_dashboard.jsp
Portal available at:

19. Security
MD-VPN provides the same level of security as VPN MPLS service,
There was no security concern related to users or even to MD-VPN users
But it is impossible to protect the access to VPNs if the core is compromised
The only threats that can occur
a NREN attacking another NREN
a NREN router compromised by a pirate
Very difficult and slow attack (never seen so far)
A “detector” of this type of attack was demonstrated and will be deployed at the end of this year
MPLS firewall under test with DELL company support

20. A scientist project using MD-VPN for production
16 sites connected in 12 countries
Using all types of connection:
Direct connection
Via VPN-Proxy
Private companies not connected to any NREN
A first scientist project FIWARE FIWARE is a project of the European Public-Private-Partnership on Future Internet (FI-PPP) programme

21. MD-VPN offers a new way of cooperating
MD-VPN enables a new way for GÉANT and NRENs to cooperate, which significantly increases network scalability from a service point of view
Operation Level Agreement
VPN provisioning, debugging, …
Acceptable User Policy
Manage service extension (regional, metropolitan network)

22. MD-VPN and GEANT Porfolio
MD-VPN service positioning regarding GEANT Plus service and L3VPN service
MD-VPN usage should be encouraged when it can be used

23. User Network Interface
Prospectives
MD-VPN service improvement
Scripting for VPN provisioning – VPN automation delivery
Improve lead time and NOC quality work
Optical transport
MD-VPN
Innovation
Users
User Network Interface
With MD-VPN, we create a seamless transmission infrastructure using MPLS as data plane and all services compatible with MPLS could use this infrastructure
NG-mVPN, EVPN
New experimental service can use this infrastructure SDN, CoCo project

24. Summary An innovative and highly scalable design
Seamless transport infrastructure
A bundle of services (IPv4, IPv6, P2P L2VPN, VPLS, L3VPN) with added value for our users that was added to GÉANT portfolio
An original and useful service unavailable in a commercial NSP portfolio
A FI-PPP project, FIWARE, use GÉANT’s MD-VPN to provide its network infrastructure
Broad European deployment
18 connected NRENs
Roughly 400 PoPs already available

25. The dream team
Tomasz Szewczyk (PSNC)
Thomas Schmid (DFN)
Magnus Bergroth (NORDUnet)
Daniel Lete (HEAnet)
Carlos Friacas (FCCN)
Jani Myyry (Funet)
Bojan Jakovljevic (AMRES)
Miguel Angel Sotos (RedIRIS)
Niall Donaghy (DANTE)
Xavier Jeannin (RENATER)
With the support of Brian Bach Mortensen (DiEC)
A small team, very small amount of manpower … but highly motivated and skilled

26. Xavier.jeannin “at” renater.fr
Any Questions?
Xavier.jeannin “at” renater.fr

27. How to connect “non MD-VPN site”? VPN-proxy

28. VPN-Proxy implementation
Implemented thanks to logical router available in Juniper
ASBR-GEANT
logical router
GEANT
NREN not MPLS-aware
BGP-LU peering
VPN-Route-Reflector
I-BGP VPNv4
Back-to-back connection, VRF BIO, VRF ASTRO, …
VPN-Proxy
Play the role of ASBR + PE + route exchange VRR

29. Scalability Scalability
MD-VPN is designed to provide thousands (and more) of service
Thanks to separation between data transport in the core network and services provided at the edge.
In the core network only labels and routes related to PE routers are maintained (1000 routes)
The services are maintained at the network edge, on PE routers. Each PE router maintains only the set of entries (labels or routes) related to services provided by this very PE router.
The number of VPNs that are active between NRENs has zero impact on the GEANT and NORDUnet infrastructure since they are completely transparent to the GEANT network.

30. Security
MD-VPN provides the same level of security as VPN MPLS service,
There was no security concern related to users or even to MD-VPN users
But it is impossible to protect the access to VPNs if the core is compromised
In case of MD-VPN, the core is multi-domain
The only threats that can occur
a NREN attacking another NREN
a NREN router compromised by a pirate

31. Data plan attack from another NREN
Label spoofing
we could imagine that a pirate, first take control of a NREN router and secondly forge packet and inject into the NREN with the purpose to compromise a user VPN that is not located in remote NREN in the MD-VPN infrastructure.
This case is very rare and our investigation demonstrates that such attack is very difficult to put in place and very long to lead.
Counter measure
label spoofing attack requires a specific step called “label scan” and this step is easily detectable thanks to NetFlow.
Sudden increase of number of VPN labels is easy to detect.
In this condition, the scan detection feature is capable to be our firewall

32. Label scan detection thanks to NetFlow
With the courtesy of
Julius Kriukas (LITnet)
2015/03/25 10:21:39 ALARM :29770 (#49), interface 104, label { }, threshold reached, 409 unique labels, 13 labels is allowed
2015/03/25 10:21:39 ALARM :2024 (#17), interface 104, label { }, threshold reached, 416 unique labels, 13 labels is allowed

33. NetFLow detection deployment