1. 1 Shigeo Urushidani SINET3: NII’s new Network National Institute of Informatics (NII) Fall 2007 Internet2 Member Meeting San Diego, October 8 th, 2007

2. 2 ‘87‘88‘89‘90‘91‘92‘93‘94‘95‘96‘97‘98‘99‘00‘01‘02‘03‘04‘05‘06 2007 ～ ▲2002.1  SINET3 is integrated successor network to two academic networks, SINET and Super-SINET, economically and flexibly providing rich variety of services.  SINET3 started its operations in April 2007 and completed its migration in May 2007. ▲2007.4 Packet Switching Network ▲1987.1 - Growing traffic and diversified user requirements - Limited abilities of existing IP routers - New trend of end-to-end circuit services Super-SINET SINET3 Evolution of Japanese Academic Networks ▲1992.4 Internet backbone for more than 700 universities and research institutions SINET Super-high-speed environment for cutting-edge research

3. 3 IP (L3)Ethernet (L2)Lambda/Dedicated (L1) IPv4IPv6 Multicast Application-based QoS L3VPN L2VPN VPLS L2VPN (QoS) VPLS (QoS) BW-specified L1VPN On-demand Lambda L1VPN Multi-homing Multicast (QoS) Best Effort High Priority QoS - guaranteed  SINET3 emphasizes four service aspects: transfer layer, virtual private network (VPN), quality-of-service (QoS), and bandwidth on demand. Service Categories in SINET3

4. 4 Multiple Layer Services Ethernet network (Layer 2) Dedicated line network (Layer 1) IP network (Layer 3) Past Networks  SINET3: integrated network providing all transfer layer services.  Users can freely choose best transfer layer for their applications.  It provides economical service provision and flexible network resource assignment for ever-changing and unpredictable service demands. IP Router Ethernet Switch Innovative Integration ★ Provides all transfer layer services ★ Integrated network User Equipment SINET3 Cutting-edge Device IP Router Ethernet Switch User Equipment Cutting-edge Device

5. 5 Multiple VPN Services  For collaborative research activity: closed user group environment (virtual private network: VPN) is essential for security reasons.  Users can choose from L3VPN (IP), L2VPN/VPLS (Ethernet), and L1VPN services. * Virtual Private Network (VPN); Virtual Private LAN Service (VPLS) IP-based VPN (L3VPN) VPLS L3VPN Super-SINET Expansion of Services & Sites IP Router Ethernet Switch IP Router L1VPN SINET3 Analysis device Secure Closed User Group

6. 6  National Institute for Fusion Science (NIFS) utilizes L3VPN services for its collaborative research with many universities and research institutions. L3VPN (IP-based VPN)

7. 7 L2VPN and VPLS (Ethernet-based VPNs)  SINET3 provides two types of Ethernet-based VPNs: Point-to-point-based VPN(L2VPN) Broadcast-based VPN(Virtual Private LAN Service (VPLS)). Point-to-point-based VPN (L2VPN) Broadcast-based VPN (VPLS) e.g. Grid computing researche.g. Earthquake research

8. 8 32 m 11 m  Virtual dedicated lines over shared platform form VPN among specified sites.  Users can obtain protocol-free and completely exclusive environment.  National Astronomical Observatory of Japan (NAOJ) utilizes L1VPN to transfer constantly-flowing ATM cells from remote telescopes through STM-16 interfaces.  On-demand capabilities will be available soon. 32 m 2.4 Gbps e.g. e-Very Long Baseline Interferometry (eVLBI) project L1VPN (Layer-1 VPN) * Asynchronous Transfer Mode (ATM)

9. 9  SINET3 provides BoD services as part of layer-1 services.  Users can specify the destinations, duration, bandwidth, and route option.  BoD server receives reservation requests, schedules accepted reservations, and triggers layer-1 path setup. 1 Gbps (13:00-14:00) 2 Gbps (17:00-18:00) 1 Gbps (15:00-16:00) Bandwidth on Demand (BoD) Services User Web-based Interface (Destination, Duration, Bandwidth, & Route option) On-demand layer-1 path Layer-1 path setup trigger SINET3 On-demand Server

10. 10 Service Parameters of L1 BoD Services  BoD server allows users to specify destinations, duration, bandwidth, & route option via Web-based interface. VPNExtranet Public Connection Style + Destinations : VPN-A : Non-VPN : VPN-B Pre-configured interfaces Duration - Start Time & - Finishing Time (by 15 minutes) Bandwidth GE STM-16 STM-64 GE 10GE VC-4-7v VC-4-17v VC-4-Av VC-4-Bv 1 ≤ A ≤ 7 1 ≤ B ≤ 64 STM-64 LambdaBandwidth-specified VC-4 Granularity (about 150 Mbps) Route Option - “Minimum Delay” or - “Unspecified”

11. 11 Middleware Layer 3 (IP) Layer 2 (Ethernet/MPLS) Layer 1 (TDM/Lambda) Network Control Platform Security Service Control Platform UNI, API, GUI … User side SINET3  Hybrid Optical and IP/MPLS Network Dynamic Control  User-oriented Service Control Platform  Adaptive Network Control Platform BoD IPv6, Multicast, VPN, QoS … - Dynamic resource control - Resilient network control - Performance monitoring - Bandwidth on demand - Enhanced network security - Middleware/application coordination - Multi-layer accommodation - Enriched VPN - Enhanced QoS - High availability - Flexible resource assignment - 40 Gbps (STM-256) lines High-level Network Architecture  High-level network architecture is composed of transport network, adaptive network control platform, and user-oriented service control platform.

12. 12  SINET3 has two-layer structure with edge and core nodes.  Edge nodes are edge layer-1 switches with layer-2 multiplexing, which are located in universities or research institutions and accommodate user equipment.  Core nodes are composed of high-end IP routers and core layer-1 switches located in public data centers. SINET/Super-SINET Edge L1 Switch With L2 Mux Core L1 Switch SINET3 IP Router Core Node Edge Node : L1 (Dedicated/On-demand): L3 (IP): L2 (Ethernet) : L3 (IP) Backbone Router Backbone SINET Router Super-SINET/SINET Router Network Structure of SINET3 10GE/GE/FE STM-16

13. 13  Has 63 edge nodes and 12 core nodes (75 layer-1 switches and 12 IP routers).  Deploys Japan’s first 40 Gbps lines between Tokyo, Nagoya, and Osaka.  Links form three loops in backbone to enable quick service recovery against link and node failures and for efficient use of network bandwidth. Network Topology of SINET3 : 40 Gbps : 10 to 20 Gbps : 1 to 20 Gbps : Core Node (L1 Switch + IP Router) : Edge Node (L1 Switch) 2.4 Gbps 10 Gbps 622 Mbps Japan’s first 40 Gbps (STM256) lines Los Angeles New York Hong Kong Singapore 40 Gbps package L1 Switch (NEC UN5000) IP Router (Juniper T640)

14. 14  L3 and L2 traffic are accommodated in shared bandwidth by L2 multiplexing and transferred to IP router, where each traffic is encapsulated with MPLS labels as needed.  L1 traffic is assigned dedicated bandwidth and separated from L2/3 traffic.  L2/3 (or IP/MPLS) traffic bandwidth can be hitlessly changed by LCAS to flexibly accommodate multi-layer services. Accommodation of Multi-layer Services L2 Mux IP Router Ethernet Switch Cutting-edge device IP/MPLS MPLS SINET3 Layer-1 traffic STM64/STM16 STM256/STM64 * Multi-protocol Label Switching (MPLS); Link Capacity Adjustment Scheme (LCAS) Hitless bandwidth change by LCAS FE/GE/10GE GE/10GE/ STM16 Edge L1 Switch Core L1 Switch IP Router dataIP dataEther dataIP dataEther VLAN dataIP dataEtherVLANMPLS L3 L2 L1 10GE Flow Control

15. 15 L2 MUX  L3VPN, L2VPN, and VPLS are logically separated by internal VLAN tags and logical routers. Each logical router exchanges different protocols for each VPN service.  L1VPN and on-demand services need GMPLS protocols to set up layer-1 paths and have separate control planes from that of IP routers. IP Router IPv4/IPv6 L3VPN L2VPN VPLS L1VPN Edge L1SW Core L1SW L2VPN (L2) L3VPN (L3) VPLS (L2) GMPLS Control Plane IP/MPLS traffic Shared Layer-2/3 traffic IPv4/IPv6 (L3) Accommodation of Multi-VPN Services Layer-1 traffic L1 VPN L1 VPN * Generalized MPLS (GMPLS) Aggregation dataIP dataEther dataIP dataEther VLAN dataIP dataEtherVLANMPLS dataIPMPLS or L3 L2 L1 : Virtual routing/forwarding table : Logical Router

16. 16 Architecture for BoD Services L2 MUX GMPLS control and management plane L1SW Layer-1 BoD Server GMPLS  BoD server receives reservation requests, schedules accepted requests, and triggers layer-1 path setup to source layer-1 switch via L1-OPS.  Source layer-1 switch sets up layer-1 path toward destination using GMPLS.  BoD server changes L2/L3 traffic bandwidth by LCAS via L1-OPS as needed. User L2 MUX IP Router Path setup trigger Hitless bandwidth change by LCAS Destinations, Duration, Bandwidth, & Route Option Path setup request Scheduling Path control Route calculation Resource management Front-end IP Ethernet On-demand L1-OPS

17. 17 Path Calculation in BoD server  BoD server calculates best path for route option using two metrics for each link: delay time and available bandwidth for layer-1 services. For Minimum delay, route is uniquely chosen. For Unspecified, route that has largest available bandwidth is chosen.  Available bandwidth for L1 changes depending on traffic volume of L2/L3. Hiroshima L1SW Kyoto L1SW Osaka L1SW Nagoya L1SW Tokyo １ L1SW Tsukuba L1SW Sendai L1SW Fukuoka L1SW Matsuyama L1SW Kanazawa L1SW Tokyo ２ L1SW Sapporo L1SW MonTueWedThuFriSatSun L2/L3 Traffic Pattern Link Bandwidth Available bandwidth for layer-1 services Route for Minimum Delay 1 Gbps (VC-4-7v) VCAT 1 Gbps (VC-4-7v) 0.6 Gbps (VC-4-4v) 0.45 Gbps (VC-4-3v) Route for Unspecified Route for Unspecified using VCAT

18. 18 Service Criteria IPv4/IPv6L3VPN, L2VPN, VPLSL1VPN, On-demand No. of usersVery largeSmall to mediumSmall Priority of availabilityHighestHighMedium HA function (normal)IP route recalculation MPLS protection & Fast Reroute None HA function (option)--GMPLS LSP Rerouting FukuokaHiroshimaKyotoKanazawaHokkaido SendaiTsukubaTokyo1NagoyaOsakaMatsuyama Tokyo2 IP route recalculation IP MPLS MPLS Protection & Fast Reroute (option) GMPLS LSP Rerouting TDM High-availability Networking Functions  Multiple loops easily enable multi-layer traffic to be detoured in different directions.  Layer-1 switches detect link failures very quickly and inform them to neighboring layer-1 switches and IP routers.

19. 19 Main Features of SINET3 (Summary) ItemsFeaturesExamples Services Multiple Layer L3 (IP), L2 (Ethernet), & L1 (dedicated/on-demand) Enriched VPN Virtual Private Network for layers 1 to 3 Enhanced QoS Support for real-time applications Layer-1 BoD Support for data-intensive applications Value-added Network performance monitoring Network Hybrid Network Architecture Hybrid network of layer-1 switches and IP routers 75 layer-1 switches nationwide 12 IP routers at backbone sites High Flexibility Flexible resource assignment to multiple layers High Availability Fast service recovery owing to multi-loop topology Large Capacity Introduction of Japan’s first STM-256(40 Gbps) lines Technologies NG SDH/SONET GFP, VCAT, & LCAS GMPLS RSVP-TE, OSPF-TE, GMPLS-UNI, & GMPLS LSP rerouting Logical Router Logical routers for IPv4/IPv6, L3VPN, L2VPN, & VPLS Advanced MPLS MPLS-based VPN for L3VPN, L2VPN, & VPLS Fast Detour Multi-layer detour triggered by layer-1 switches

20. 20 Schedule Year 200620072008 SINET3 Operation SINET3 Services Overlay construction In operation (01/04/2007) Migration Complete (31/05/2007) On-demand (GMPLS-based) L1VPN (static) IPv4/IPv6 dual stack L3VPN L2VPNVPLS Enhanced GMPLS Today  SINET3 started to provide L3VPN, L2VPN, & L1VPN (static) services, as well as IPv4/IPv6 dual stack services in April 2007.  Starting VPLS services soon and layer-1 BoD services in February 2008.

21. 21 Thank you very much!

22. 22 Backup Slides

23. 23 Multiple QoS Services  SINET3 provides QoS by identifying applications, VPNs, & physical/logical ports.  Layer-2/3-based QoS has four priority classes: expedited forwarding (EF), network control (NC), assured forwarding (AF), & best effort (BE).  Layer-1-based QoS has smallest packet delay, no delay variance, & no packet loss. SINET/Super-SINET SINET3 Expedited NW control Assured Best effort Node End-to-end on-demand path Best Effort Congestion  Network congestion affects all services HDTV QoS- aware  Application/VPN/port-based QoS control Uncompressed HDTV

24. 24 Prioritizing for IP Prioritizing for Ether User Priority based classifier IP Router Internal VLAN tag Identifier DSCP& EXP based classifier Edge L1SW Core L1SW Accommodation of Multi-QoS Services  Layer-3/2-based QoS User Priority bits of internal VLAN tags are marked at edge L2 MUX. User Priority bits are mapped into DSCP (IP) or EXP (MPLS) bits at IP router. There are four forwarding classes: EF, NC, AF, & BE.  Layer-1-based QoS Layer-1 switches assign end-to-end bandwidth on demand. L2 MUX IP VLAN BE IP MPLS AF NC EF BE AF NC EF Smallest packet delay No delay variance No packet loss IP (L3) Ethernet (L2) Dedicated (L1) Priority mapping for IP Priority mapping for Ether Marking User Priority bits by identifying IP/Ethernet header Mapping User Priority bits into IP DSCP or MPLS EXP bits Ether dataIP dataEther Shared Layer-2/3 traffic Layer-1 traffic

25. 25 Functions of BoD Server  BoD server software is composed of following function modules: Front-end functions Admission control and scheduling Path calculation Path control Resource management L1-OPS GUI (Web browser) IF Operator IF - User Authentication - Session management http(s) GUI (Web browser) IF User http(s) - Request Acceptance - Admission Control, Scheduling - Database Registration Admission Control, Scheduling Front-end - L1 Path Setup/Release - IP/MPLS Bandwidth Change Path Control - Route Selection - Link Selection Path Calculation - L1 Path Management - L1 Path Monitoring Resource Management Database User DBPath DBUsage DBRoute DBResource DBL1SW-DB Layer-1 BoD Server : Function Module