Internet2 QBone: Building a Testbed for IP Differentiated Services INET99 June 23 rd, 1999 San Jose, California Ben Teitelbaum

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Internet2 Dogma: There is a circularity between advanced networks and advanced apps Networked Applications Network Engineering Enables Motivate

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Dogma Article1: Inverse Apps  Networking circularity has applied to QoS QoS-needy Applications Network QoS Inhibited Prevented

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Dogma Article2: Work with the neediest apps, build a testbed, and turn the arrows around! QoS-needy Applications Network QoS Enables Motivates

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Internet2 QBone Initiative n Build interdomain testbed infrastructure – Balance networking research with providing a service – Experiment and improve understanding of DiffServ – Iterate and improve testbed design n Support intradomain & interdomain deployment n Lead and follow IETF standards work – Some parts of DiffServ architecture cooked; others far from it – Our experience will inform standards process n Openness of R&E community gives us an edge – We can live with somewhat flaky infrastructure – We are open to sharing implementation experiences and measurement data

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Internet2 QoS Working Group – Osama Aboul-Magd (Nortel) – Andy Adamson (Michigan) – Grenville Armitage (Lucent) – Steve Blake (Torrent) – Scott Bradner (Harvard) – Scott Brim (Newbridge) – Larry Conrad (Florida State) – John Coulter (CA*net2) – Chuck Song (MCI/vBNS) – Fred Baker / Larry Dunn (Cisco) – Rüdiger Geib (Deutsche Telekom) – Terry Gray (U Washington) – Jim Grisham (NYSERNet) – Roch Guerin (Penn) – Susan Hares (Merit) – Joseph Lappa (CMU) – Jay Kistler (FORE) – Klara Nahrstedt (UIC) – Kathleen Nichols (IETF coordination) – Ken Pierce (3com) – John Sikora (ATT Labs) – Ben Teitelbaum (chair) – John Wroclawski (MIT) – a liaison from each MOU partner

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Internet2 Applications n Qualitative and quantitative improvements in how we conduct research, teaching, and learning n Require advanced networks n Examples: – Interactive research collaboration and instruction – Real-time access to remote scientific instruments – Large-scale, multi-site computation and database processing – Shared virtual reality

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Big Problem #1: Understanding Application Requirements n What services do tomorrow’s applications need? n Range of poorly-understood needs – Both intolerant apps (e.g. tele-immersion) and tolerant apps (e.g. large FTPs, desktop video conferencing) important – Many apps need absolute, per-flow QoS assurances – Adaptive apps may require a minimum level of QoS, but can exploit additional network resources if available – Some institutions/users want multiple classes of best-efforts service (CoS) with relative precedence levels Good Bad Intolerant Better Tolerant Adaptive n Need better understanding through experience Different App Needs

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Big Problem #2: Scalability n Convergence of flows on the core means: – Large numbers of flows through each router – High forwarding rate requirements n Need to support QoS end-to-end, but keep per-flow state & packet forwarding overhead out of the core Lots of flows here!

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Big Problem #3: Interoperability Campus Networks GigaPoPs Campus Networks … and between multiple implementations of network elements... Backbone Networks (vBNS, Abilene, …)... between separately administered and designed clouds... … is crucial if we are to provide end-to-end QoS.

ISOC INET99 Conference - San Jose, CA, June 23, 1999 DiffServ for Internet2 n July February 1998 – WG struggled to understand needs of advanced applications / realities of QoS engineering – Frustrations with RSVP give birth to IETF DiffServ n May 1998 – WG recommends EF/Premium DiffServ focus for I2 QoS – First Internet2 Joint Applications/Engineering Workshop, Santa Clara, CA (report on web site) n October 1998 – QBone initiative launched

ISOC INET99 Conference - San Jose, CA, June 23, 1999 DiffServ Overview n Exploits edge/core distinction for scalability n Applications contract for specific QoS profiles – Policing at network periphery – A few simple, differentiated per-hop forwarding behaviors (PHBs) n Indicated in packet header n Applied to PHB traffic aggregates – PHBs + policing rules = range of services n Clouds contract for aggregate QoS traffic profiles – Policing at cloud-cloud boundary – Supports simple, bilateral business agreements

ISOC INET99 Conference - San Jose, CA, June 23, 1999 DiffServ Architecture Leaf Router (police, mark flows) Ingress Edge Router (classify, police, mark aggregates) Egress Edge Router (shape aggregates) Core routers Core routers Bandwidth Brokers (perform admissions control, manage network resources, configure leaf and edge devices) BB Source Destination

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Example Service #1: Premium n Contract: leased line emulation at a specified peak rate n PHB = “forward me first” (EF) n Policing rule = drop out-of-profile packets n On egress, clouds must shape EF aggregates to mask induced burstiness

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Why Premium First? n Simplest absolute service to understand n Strongest flavor of DiffServ – Could support our most demanding applications – Less demanding applications should work fine on emerging high-performance BE infrastructure n Explore other PHBs (AF) later n To understand this, consider “typical” Internet2 performance: – I2 networks are largely uncongested – Jitter and loss still occur – Route flaps to the commodity Internet still occur Premium Assured Olympic (CoS)

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Typical 1999 Internet2 Performance East Coast University to West Coast DOE Lab Minimum Delay 50th Percentile Delay 90th Percentile Delay

ISOC INET99 Conference - San Jose, CA, June 23, Initial QIG * * 11 February 1999 (actual connectivity and participating networks may vary as deployment progresses) Other NGIXs ESNet NREN Abilene UMass UMN vBNS MAGPINYSERNetTexas GPPSCNCNI CA*Net2 APANSingAREN SURFNet Merit UMich UPenn CMU TAMUDukeNCSU UNC CTIT IU EVL iCAIR NWU UBC ARDNOC NTU NUS LBNL Ames ANL KDD Labs Korea RISQ CRCUNB MREN / STAR TAP Other DOE Labs Other NASA Labs

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone BB Group n Open group chartered to recommend BB solutions for the QBone n Lead by Sue Hares - Merit Networks n Six R&E proto-BBs: – Merit – BCIT – UCLA n Extensive participation from corporate partners n QBone BB requirements draft on web site n Prototype inter-BB signaling protocol due soon – Telia / Luleå University of Technology – ANL/Globus – LBNL Clipper

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Milestones n Sep 25th - Call for participation n Oct 30th - WG recommends initial QIG participants n Dec 1st - 1st QIG / QBone BB meeting (Evanston) n Jan 1st - WG makes major push on architecture draft n Jan 26th - 2nd QIG / QBone BB Meeting (RTP) n Mar 7th - Measurement sub-group drafts QMA n Mar 9th - 3rd QIG / QBone BB Meeting (Las Cruces) n May 21st - WG opens QIG n June 8th - Open QBone interop BOF (Pittsburgh)

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Architecture (10km view) n IETF “Diff” (EF PHB) + QBone “Serv” (QPS) n QBone Premium Service – Idea: converge on Jacobson’s VLL “Premium” service – Well-defined SLS: n Peak rate R & “Service MTU” M implying a token bucket meter n Near-zero loss n Low jitter – Delay variation due to queuing effects should be no greater than the packet transmission time of a service MTU sized packet – All bets are off if the reserved interdomain route flaps n Plus important value-adds: – Integrated measurement/dissemination infrastructure – Experimentation with pre-standards inter-domain bandwidth brokering and signaling

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Measurement Architecture 1 n Collection –  metrics, EF and BE... – Active metrics n One-way delay-variation n One-way loss n Traceroutes n e.g. IPPM Surveyors – Passive metrics n Load n Discards (suggested) n Link bandwidths (suggested) n EF reservation load n e.g. OCxMon, RTFM, MIBs Boundary Router Intra-Domain Premium Path Inter-Domain Premium Path Active Measurements MIB-based statistics Passive Measurements PM node AM node QBone Domain2 QBone Domain1 QBone Domain3

ISOC INET99 Conference - San Jose, CA, June 23, 1999 QBone Measurement Architecture 2 n Dissemination – HTTP, even for raw data – real-time + archived measurements – Canonical names for: n Metrics n Domains – Standard metric aggregations: n Mostly 5-minute aggregations – Standard URL name space for: n MRTG-style plots n Raw ASCII data n / / / / /..{html | gif | txt}

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Starting Simply n Intradomain – Van’s campus example – At least 10Mbps everywhere – “Count to ten” admissions control with no topological knowledge n Interdomain – Could we do something similar in the early QBone? – Problem: Worst case down-stream provisioning starts to look pretty bad even with small initial participant set. H H H H GigaPoP...

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Generic Internet2 Topology NGIXs CC GigaPoP CC CC CC Abilene CC GigaPoP CC CC CC vBNS L C ESNet, NREN, Int’l,... L C

ISOC INET99 Conference - San Jose, CA, June 23, 1999 ESNet, NREN, STARTAP,... Abilene GigaPoP vBNS Phase 0 Demand Assessment NGIXs CC CC CC CC CC CC CC CC Int’l L C L C

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Phase 0 Deployment Planning n Converge on a consensus reservation matrix n Reservations will be static for period of phase n Reservation = {S, D, R, M, TR} – S = source – D = dest – R = peak rate n S, D are on campus network demarks n All bets are off if routing between S and D changes n All SLSs still bi-lateral, but Internet2 engineering will facilitate convergence – M = service MTU – TR = inter-domain traceroute

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Phase 0 Demand Matrix Campus EF Ingress Load R D Campus Policer Config Implies Maximum EF load to be offered from here … to here

ISOC INET99 Conference - San Jose, CA, June 23, 1999 Coming Attractions... n Jun 99: QBone Architecture in last call n Jun 99: QBone BB Advisory Council will recommend a prototype inter-BB protocol n Jun/Jul 99: “Phase 0” rollout planning n Aug/Sep 99: Interdisciplinary QBone workshop n Fall 99: QBone Connect-a-thon (“QCon”) event n Fall 99: “Phase 0”

ISOC INET99 Conference - San Jose, CA, June 23, 1999 For more information... n QBone home page: n Internet2 QoS Working Group home page: