APII/APAN Measurement Framework: Advanced Network Observatory Masaki Hirabaru (NICT), Takatoshi Ikeda (KDDI Lab), Motohiro Ishii (QIC), and Yasuichi Kitamura (NICT) January 21, 2006 APII/APAN Network Engineering

3 Case Studies Acknowledgement to JaeHwa KOREN

Kwangju Busan Fukuoka Korea 2.5G SONET KOREN Taegu Daejon Seoul XP Genkai XP Fukuoka Japan 1,000km APII/JGNII Case 1: APII Seoul-Tokyo Redundant Connection Tokyo XP JGNII TransPAC2 Southern Route (10G) Northern Route (1G) 10G JGN II : Frequent Maintenance Outage but No Fault Detection and Redundancy! Establish Backup L2 VLAN Path (northern route) Switchover-Time Measurement on APII link was 37 secs improved by decreasing OSPF Hello Interval (40 to 1), 5 secs Next: Seoul-Tokyo E2E Path Needs a measurement point and NOC support in Korea Tokyo – Fukuoka Northern Route Tokyo – Fukuoka Southern Route switchover time

Case 2: APII Route Propagation Delays busan daejon seoul tokyo4 tokyo5 tokyo Generator at Koganei (A / W) 2-hour intervals monitor AS7660 AS /24 AS20965 AS2907 AS6509 AS2523 JGNII TransPAC2 AS9270 fukuoka AS10746 koganei AS2020 Data collected on January 19, 2006

Case 3: Common Measurement Machines and Traffic Monitoring for APII Fukuoka – Busan TCP Performance Measurement The statistics were last updated Monday, 26 September 2005 at 19:45 Max =>: Mbps Average =>: Mbps Current =>: Mbps Max <=: Mbps Average <=: Mbps Current <=: Mbps *Theoretical wire rate of TCP on GbE is 941 Mbps at 1500B MTU. No wire rate confident box has not been available in KOREN at that time. Busan BGP Instability Measurement Weathermap SmokePing with KOREN NOC from Tokyo XP

APAN/APII Measurement Framework for NOCs and Researchers : Advanced Network Observatory Two Targets  Reliable Operations  Research Support NOC Daily Operations Supported by Stable Measurement NOC B NOC C NOC A Research Activity of Advanced Measurement Provide data and environment Other Observatories Common data export

Observatory at Tokyo XP (1) NOC DATASET,TOOL and MACHINES Data setdescriptiontoolmachine Throughput1G throughputIperf & BWCTLnms1 10G throughputIperf & BWCTLnms5 One-way delayone-way delayOWAMPnms4 netflowFlow dataflow-toolsnms3 Usage statistics Interface usageucd-snmp & SNAPPnms2 Core router A (tpr4) Core router B (tpr5) nms1 nms2 nms3 nms4 L2 SW nms5 10G 1G NETWORK DIAGRAM perf4perf5perf6bwctl L2 SW NOC machines Research machines supported by NOC NOC machine

Observatory at Tokyo XP (2) SPECIFICATION of MACHINES machineOSCPUMemoryDiskNICcost nms1,2Linux web100 Dual Xeon 3.6GHz DDR266 1GB x 2 (nms1,2,4) U320 SCSI 72G (nms3) U320 SCSI 144G 2 x 1000base-TX (Onboard) (only nms1) 1 x1000base-SX ( Intel NIC) around $3,000 nms3,4FreeBSD nms5Linux Dual Opteron 2.2GHz PC3200 1GB x 2 IDE2 x 1000base-TX (Onboard) 1 x 10G-LR (Chelsio T210-LR) around $8,000 (PC $4k, NIC $4k) -10G NICs vendors: Intel (earliest), Neterion (s2io, PCI express, PCI-X version 2), Chelsio (w/ or w/o TOE), Mirinet (PCI express) - Remote console access PCI-X 64bit/133Mhz

Example Configuration for G-H 10G Link - Minimum Requirements for Operations - TARGET DATA –Throughput (10G) –One-way delay EQUIPMENTS (each node) –machineA (for throughput) same spec of nms5 –machineB (for One-way delay) same spec of nms1-4 –Router or Switch port 10G x 1 (for machineA) 10/100/1000 x 2 (for machineA,B) COST (each node) –Servers machineA $8,000 machineB $3,000 –Router or Switch port depend on vender –Rack space for research equipment free? Fukuoka Korea KOREN 10G Genkai XP Kitakyushu Fukuoka Japan APII Tokyo XP 10G Tokyo Japan

Current Genkai XP Configuration Tokyo XP summit400 summit5it Genkai XP (Fukuoka) APII Research Servers Juniper M10 JGN II summit1it TAP FI 4802 Genkai-perf1 Genkai-perf2 One-way delay servers (Zhang) Genka-bwctl Busan 10G Genkai-ia64 10G Research 10G measurement is ready Genkai XP (QIC’s) operational NOC support Ready for collaboration under Observatory framework A local 10G port for measurement is needed to isolate a problem from JGN II Fukuoka – Tokyo link.