1. Network Engineering (NOC) Workshop in APAN Challenges in Layer 3 – Operation Experiences Xing Li 2006-07-19 9:00-17:30

2. Outline Introduction The case study The problem statement The possible solution Remarks

3. Simple Case (where BGP can handle things easily) Global transit –To tier 1 or tier 2 commodity networks Care the aggregation Care the load balancing Don’t care the symmetry Peering (no transit, except for the down streams) –To domestic ISPs (bi-literal or via IX) Care the business model –To academic partners Care the performance Care the symmetry

4. Complicated Case (where BGP cannot handle things easily) Global transit –To tier 1 or tier 2 commodity networks Care the aggregation Care the load balancing Don’t care the symmetry Academic transit –To multiple transit backbones within academic scope Care the aggregation Care the load balancing Care the performance Care the symmetry Etc. Peering (no transit, except for the down streams) –To domestic ISPs (bi-literal or via IX) Care the business model –To academic partners Care the performance Care the symmetry

5. The Real World Routing

6. Possible Paths policy based routing and politics based routing

7. Outline Introduction The case study The problem statement The possible solution Remarks

8. April 2006 in KR It is not the CERNET – KR link It is not the CERNET –JP_KR link It is not the CERNET – TEIN2-KR link It is the CERNET – JP – Abilene – KR link Case 1

9. Case 2.1 (IPv4) C:\Documents and Settings\xing>tracert 202.112.0.56 Tracing route to ocean.net.edu.cn [202.112.0.56] over a maximum of 30 hops: 1 19 ms 1 ms 2 ms 180-rtr.mm.internet2.edu [206.196.180.1] 2 9 ms 2 ms 2 ms abilene-rtr.maxgigapop.net [206.196.177.2] 3 14 ms 6 ms 6 ms nycmng-washng.abilene.ucaid.edu [198.32.8.84] 4 34 ms 26 ms 39 ms chinng-nycmng.abilene.ucaid.edu [198.32.8.82] 5 28 ms 35 ms 36 ms 198.32.11.102 6 34 ms 27 ms 26 ms ae-0-157.br0.chi.us.rt.ascc.net [140.109.251.158] 7 243 ms 221 ms 597 ms 202.169.174.62 8 416 ms 409 ms 314 ms 202.169.174.45 9 545 ms * 507 ms 202.112.61.93 ! KOREN 10 * 355 ms * 202.112.53.17 11 408 ms * * 202.112.53.181 12 410 ms 407 ms 410 ms cd1.cernet.net [202.112.53.74] 13 491 ms * 500 ms 202.112.1.193 14 * 346 ms * ocean.net.edu.cn [202.112.0.56] 15 * 357 ms 409 ms ocean.net.edu.cn [202.112.0.56] Trace complete. C:\Documents and Settings\xing> traceroute 206.196.180.179 1 202.112.6.17 (202.112.6.17) 1.095 ms 0.183 ms 0.165 ms 2 * cd0.cernet.net (202.112.53.73) 1.348 ms 1.089 ms 3 202.112.53.190 (202.112.53.190) 0.277 ms 0.177 ms 0.163 ms 4 202.112.61.195 (202.112.61.195) 0.519 ms 0.559 ms 0.590 ms 5 202.112.61.30 (202.112.61.30) 0.809 ms 0.552 ms 0.599 ms 6 202.112.61.22 (202.112.61.22) 363.584 ms 164.103 ms 163.849 ms 7 * * 202.112.61.138 (202.112.61.138) 204.876 ms 8 * pass.bjnet.edu.cn (202.112.61.6) 212.191 ms * ! STARLIGHT 9 198.32.11.101 (198.32.11.101) 204.746 ms 214.373 ms 204.353 ms 10 nycmng-chinng.abilene.ucaid.edu (198.32.8.83) 225.701 ms 233.199 ms 225.343 ms 11 washng-nycmng.abilene.ucaid.edu (198.32.8.85) 229.185 ms 228.285 ms 228.708 ms 12 * dcne-abilene-oc48.maxgigapop.net (206.196.177.1) 230.307 ms 232.123 ms 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * *

10. Case 2.2 (IPv6) Microsoft Windows XP [ 版本 5.1.2600] (C) 版权所有 1985-2001 Microsoft Corp. C:\Documents and Settings\xing>tracert6 2001:250:C000:20::2 Tracing route to 2001:250:c000:20::2 from 2001:468:c00:7:9d28:d329:479b:b356 over a maximum of 30 hops: 1 2 ms 8 ms 8 ms 2001:468:c00:7:100::1 2 7 ms 8 ms 8 ms washng-max.abilene.ucaid.edu [2001:468:ff:184 c::1] 3 22 ms 12 ms 12 ms nycmng-washng.abilene.ucaid.edu [2001:468:ff:1518::1] 4 28 ms 32 ms 32 ms chinng-nycmng.abilene.ucaid.edu [2001:468:ff:f15::1] 5 36 ms 36 ms 36 ms iplsng-chinng.abilene.ucaid.edu [2001:468:ff:f12::2] 6 48 ms 43 ms 47 ms kscyng-iplsng.abilene.ucaid.edu [2001:468:ff:1213::2] 7 79 ms 56 ms 66 ms dnvrng-kscyng.abilene.ucaid.edu [2001:468:ff:1013::1] 8 81 ms 83 ms 81 ms snvang-dnvrng.abilene.ucaid.edu [2001:468:ff:1017::2] 9 354 ms 82 ms 90 ms 3ffe:80a::c 10 318 ms 257 ms 367 ms eth10-0-0.xr1.ams1.gblx.net [2001:7f8:1::a500:3549:1] 11 383 ms 407 ms 235 ms e0-0-0.6b2.AMS7.Alter.net [2001:7f8:1::a501:2702:1] 12 612 ms 636 ms 663 ms 2001:278:0:1000::11JP telecoms 13 483 ms 509 ms * 2001:250:c000:20::2 14 682 ms 503 ms 524 ms 2001:250:c000:20::2 Trace complete. C:\Documents and Settings\xing> bj-bgw-r0k#trace 2001:468:c00:7:9d28:d329:479b:b356 Type escape sequence to abort. Tracing the route to 2001:468:C00:7:9D28:D329:479B:B356 1 2001:250:C000:20::2 0 msec 0 msec 0 msec 2 2001:250:0:3::1 0 msec 0 msec 4 msec 3 2001:254:1:7::1 0 msec 0 msec 4 msec ! TEIN2 4 2001:254:1:3::2 40 msec 40 msec 40 msec 5 2001:254:1:4::2 76 msec 72 msec 72 msec 6 2001:254:8001:5::2 92 msec 92 msec 92 msec 7 2001:220:1000:282::2 92 msec 92 msec 92 msec! KOREN 8 2001:220:1000:42E::2 92 msec 92 msec 92 msec 9 2001:220:1000:400::1 96 msec 92 msec 92 msec 10 2001:220:400:200::1 96 msec 96 msec 96 msec 11 2001:220:1800:200::1 96 msec 96 msec 96 msec 12 apii-juniper-ge0-1-0-1.jp.apan.net (3FFE:8140:101:1A::162) 128 msec 128 msec 128 msec! APAN 13 3FFE:8140:101::4 148 msec 160 msec 148 msec 14 tpr5-ge0-1-0-0.jp.apan.net (3FFE:8140:101:1E::5) 128 msec 128 msec 128 msec 15 transpac-la-tpr5.jp.apan.net (3FFE:8140:101:1::1) 244 msec 244 msec 256 msec 16 2001:504:B:20::131 244 msec 244 msec 244 msec 17 hstnng-losang.abilene.ucaid.edu (2001:468:FF:1114::1) 280 msec 276 msec 276 msec ! ABILENE 18 atlang-hstnng.abilene.ucaid.edu (2001:468:FF:E11::1) 296 msec 316 msec 296 msec 19 washng-atlang.abilene.ucaid.edu (2001:468:FF:118::2) 468 msec 468 msec 468 msec 20 max-washng.abilene.ucaid.edu (2001:468:FF:184C::2) 468 msec 468 msec 468 msec 21 * * * 22 * * *

11. Outline Introduction The case study The problem statement The possible solution Remarks

12. Problems and the Descriptions There is no high throughput link between me and my partner –No hope If path is not the desired one –It is true in most of cases, because of Murphy’s law –Ask NOCs to help The performance is ok in one direction, but not in the other direction –Typical case for asymmetric routing phenomena, again Murphy’s law –Ask NOCs for help They told me the performance between ASx and ASy is fine, but the my applications performance is still bad –It is usually the local loop problem –Ask the campus network administrator for help Still have problem –Check your own machine and the application

13. The Questions Q1: can we get ride of L3? Q2: how could we control L3? Q3: where to do the L3 control? Q4: should we distinguish ordinary users from high-performance users? Q5: how do we help the user to solve the network problem? Q6: anything else should we do?

14. Outline Introduction The case study The problem statement The possible solution Remarks

15. Can we get ride of L3? L2 is easier to handle –Optical switch –TDM switch –Ethernet switch But –L2 has its own problems Availability (expensive) Scalability (point-to-point) –And we still need L3 mechanism for the L2 solution TCP/IP is the common platform

16. How could we control L3? 0. More specific win 1. If the path specifies a next hop that is inaccessible, drop the update. 2. Prefer the path with the largest weight. 3. If the weights are the same, prefer the path with the largest local preference. 4. If the local preferences are the same, prefer the path that was originated by BGP running on this router. 5. If no route was originated, prefer the route that has the shortest AS_path. 6. If all paths have the same AS_path length, prefer the path with the lowest origin type (where IGP is lower than EGP, and EGP is lower than Incomplete). 7. If the origin codes are the same, prefer the path with the lowest MED attribute. 8. If the paths have the same MED, prefer the external path over the internal path. 9. If the paths are still the same, prefer the path through the closest IGP neighbor. 10. Prefer the path with the lowest IP address, as specified by the BGP router ID.”

17. Where to do the L3 control? Backbone –a flexible and transparent routing policy Identification: additive community tag In/out control: as-path prepend NRNs –User controlled network path Identification: community tag/address/as-path Out control:local-pref, as-path prepend In control:med, as-path prepend, more specific

18. Should we distinguish ordinary users from high-performance users? Normal user –Throughput per single address peer < 5Mbps High performance user –Throughput per single address peer > 20Mbps –RTT (2nd requirement) –jitter/out of sequence/duplicates (Other requirements

19. How do we help the user to solve the network problem? Open NMS tools –Oregon's routerview server –APAN’s matrix tool –TEIN2 NOC as-search/ping/traceroute –etc

20. Anything else should we do? We need a better trouble ticket system. We need closer collaboration between people in network research areas and application areas. We need an integrated O&M system for both the network and the application. We may need a better network architecture.