Internet2: A Tutorial Part 3 of 4 17 th Brazilian Symposium on Computer Networks Paul Love, Internet2 Chair, I2 Topology WG

Working Groups

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Working Groups IPv6 Measurement Multicast Network Management Network Storage Quality of Service Routing Security Topology

25-28 May 99 SBRC99 Salvador, Bahia, Brazil IPv6 Chair: Dale Finkelson, Univ Nebraska, Lincoln Focus: Explore the rôle that IPv6 will play in the Internet2 project Work with those interested in IPv6 to build IPv6 testbeds across the Internet2 structure, including vBNS and Abilene Must be coordinated across backbones, gigaPoPs, and campuses Must be interoperable among above and between vendors

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Measurement Chairs: David Wasley, Univ California and Matt Zekauskas, Internet2 staff Focus: Places to measure: At campuses, at gigaPoPs, within interconnect(s) Things to measure: Traffic utilization Performance: delay and packet loss Traffic characterization

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Multicast Chair: Kevin Almeroth, Univ California at Santa Barbara Focus: Make native IP multicast scalable and operationally effective Must be coordinated across backbones, gigaPoPs, and campuses Must be coordinated with unicast routing

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Network Management Chair: Mark Johnson, North Carolina Networking Initiative Focus: Common trouble ticket system How can all our interconnects and gigaPoPs and universities appear to be a seamless whole?

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Network Storage Chair: Micah Beck, Univ of Tennessee, Knoxville Focus: Develop and deploy a reliable, scalable, high performance network storage capability enabling broad access to stored video, very large data sets, etc.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Quality of Service Chair: Ben Teitelbaum, Internet2 staff Focus: Multi-network IP-based QoS Relevant to advanced applications Interoperability: carriers and kit Architecture Qbone: distributed testbed

25-28 May 99 SBRC99 Salvador, Bahia, Brazil AB The QoS Big Problems Understanding Application Requirements Scalability Interoperability

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Routing Chair: Steve Corbato, Univ Washington Focus: Internal & External routing Critical issues gigaPoP internal routing design Explicit routing requirement (the “fish problem”) gigaPoP external routing recommendations Subscribers (Internet2 campuses) National interconnects (vBNS, Abilene, and NGI networks)

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Nature of Explicit Routing Fish problem C1 routes via NSP1 and C2 routes via NSP2 C1 C2 GP NSP1 NSP2 D One potential solution - MPLS

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Security Chair: Peter Berger, Carniege Mellon Univ Focus: Authentication Application to QoS Application to Digital Libraries

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Topology Chair: Paul Love, Internet2 staff Focus: Topology of Internet2 Internal Internet2 connections Between I2 backbones Internet2 with other Advanced Research Networks NGI International R&E

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Working Group Summary Internet2’s WGs focused on project’s needs Complement IETF WGs Membership by invitation of the chair

IPv6

Internet2 & Abilene IPv6 Networking with thanks to Dale Finkelson, Univ of Nebraska, Lincoln

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Project Goals Deploying an IPv6 testbed Both in the vBNS and Abilene Understanding what IPv6 can contribute to the research agenda of the Internet 2 project.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Abilene IPv6 Description IP over Sonet backbone This effectively blocks deploying IPv6 in Native Mode within the backbone until Code becomes available for Cisco12000 It is stable It doesn’t block multicast & QoS IPv6 will be tunneled through Abilene

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Equipment and Protocols The initial deployment will be with routers donated by Bay Networks Routing will be done with BGP4+ Some gigapops will implement tunnel servers for local connectivity Gigapops with ATM connectivity will be open to native IPv6 connections, others will use tunnels Details still TBD

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Peering arrangements The IPv6 version of Abilene will peer with the vBNS at two or more points MREN (Chicago switch) NCNE (Pittsburgh gigapop) AbileneV6 will peer with other providers at the 6TAP (Chicago switch) ESnet CAnet3 European networks AbileneV6 will be available at both of the NGIX’s (3 rd still TBD)

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Schedule vBNS network was in place by the end of June 98 Backbone deployment of IPv6 routers in Abilene in the summer of 1999 By the end of summer Initial connectivity to gigapops Connectivity to other IPv6 networks

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Working Group Agenda Preparing “Good Practices” document for gigapop operators. Addressing options Configuration samples Working with the Abilene engineering staff to implement the IPv6 network Design an addressing plan for Abilene

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Gigapop Issues Obtaining Addresses Multi-homing Hosts This is specifically a problem for multihomed gigapops Providing DNS services for IPv6 Providing either Native IPv6 or tunnels to the backbones Providing IPv6 connectivity to their customers

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Addressing Questions Who gets PTLA’s. Abilene, vBNS, gigapops? How do campus address relate to the TLA’s? Can you do multiple addresses within a v6 host? For multiply attached gigapops Do you draw NLA’s from each provider? Do you do private addressing at the campus Some sort of translation at the edge?

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Possible Abilene IPv6 Backbone & Peering Points Seattle Kansas City Denver Cleveland New York Atlanta Houston Indianapolis Sacramento Los Angeles v6 Peering Point nbv6 will be in v4 tunnels inside Abilene STAR TAP & NGIX NGIX

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Pointers General Information sites Site for implementations All of the above sites have links to sites where implementation information can be found

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Pointers IETF Documentation has a link to IETF information draft-iab-nat-implications-04.txt draft-carpenter-transparency-01.txt The Case for IPv6 draft-ietf-iab-case-for-ipv6-04.txt

Network Storage

Internet2 Distributed Storage Infrastructure Update with thanks to Micah Beck; Univ. of Tennessee, Knoxville Bert Dempsey; Univ. of North Carolina, Chapel Hill

25-28 May 99 SBRC99 Salvador, Bahia, Brazil I2-DSI Participants UT Knoxville Micah Beck Terry Moore Martin Swany Judi Talley UNC Chapel Hill Bert Dempsey Paul Jones (MetaLab) Debra Weiss Zhiwei Xiao GigaPOP and Campus Site Managers UCAID/Internet2 Network Storage Working Group Ted Hanss Applications Director NC Networking Initiative Digital Library Federation

25-28 May 99 SBRC99 Salvador, Bahia, Brazil A Word From the Sponsors CiscoDNS redirection Ellemtel engineering effort IBM large storage & DCE servers Novellstorage & directory servers Starburst reliable multicast software StorageTeklarge storage servers Sun design collaboration

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Single Server Model High performance locally Unacceptable performance across commodity backbone

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Relying on Wide Area QoS High performance access with reserved bandwidth Essential for real-time communication Technically difficult, expensive, not generally available

25-28 May 99 SBRC99 Salvador, Bahia, Brazil I2-DSI Model: Replicated Services Clients access nearby server Everyone gets performance Local resources implement a global service

25-28 May 99 SBRC99 Salvador, Bahia, Brazil I2-DSI Service Architecture Replication Rsynch+, Omnicast, AFS/DFS Novell Replication Resolution Sonar DNS, Distributed Director Delegation Cache prefetch general users replicated core delegated server local users

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Internet Content Channels A channel is a collection of content which can be transparently delivered to end user communities at a chosen (price,performance) point through a flexible, policy-based application of resources

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Server Channel Examples Replicated Web Servers APIs: Standard HTML, Active Server Pages Channels: Web sites Streaming Media APIs: MPEG-2, proprietary file formats Channels: collections of multimedia presentations Executable content APIs: Java byte code, Tcl, Perl Channels: CGI programs

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Current Server Deployment

25-28 May 99 SBRC99 Salvador, Bahia, Brazil IBM Web Cache Manager RS/6000 AIX Server 1 GB RAM 72 GB Disk / 900 GB Tape ADSM Hierarchical Storage Mgt.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Infrastructure Expansion StorageTek 2 PC/Linux Servers 700GB disk, tape backup Novell 6 PC/NetWare Servers 100GB disk Smaller institutions or departments

25-28 May 99 SBRC99 Salvador, Bahia, Brazil I2-DSI Applications Workshop Chapel Hill, NC March 4 &5, technologies Minnesota: Scalable Video IBM Research: Multicast, Filter and Store Moscow Ctr. for New Info. Tech. in Med. Ed.: Semantic Text Analysis IBM Research: Narwhal Resolution Proxy Special issue of the Journal of Network and Computer Applications (Academic Press)

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Application Strategy Chose initial applications Available or easily ported services Low update demands Port to an I2-DSI server Our development effort is limited App developers can have access to the servers Distribute to homogeneous core Derive service abstractions

25-28 May 99 SBRC99 Salvador, Bahia, Brazil I2-DSI Applications Digital libraries Video Digitized originals Large data sets Medical imaging CERN instruments Satellite images & GIS Technical Archives Netlib/NHSR Scientific software Red Hat Linux Source code Viagenie Net. Eng. Documents

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Replication Performance and Scalability Issues Server placement Server resources Server description (metadata) Server Channel description (metadata) Object representation Characterization of replication mechanisms Channel-to-server mapping (subscription)

25-28 May 99 SBRC99 Salvador, Bahia, Brazil NetStore ‘99 Workshop Network Storage Technical Workshop Knoxville, TN, October Scope I2-DSI implementation I2-DSI applications Related networking projects Storage technology

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Conclusions A server platform is in place Infrastructure development Service abstractions (search, computation) Publication and replication protocols Portable representation and API Heterogeneous servers Six months to show results from initial application development efforts

Multicast

Multicast Update with thanks to Kevin Almeroth, Univ of California, Santa Barbara

25-28 May 99 SBRC99 Salvador, Bahia, Brazil 1999: A key year for multicast In the past, multicast has meant ‘MBone’ Core set of committed users and engineers ‘Legacy’ non-scalable approaches to routing Our hope for 1999: Needed, new protocols deployed Enable scalable use of high-speed multicast flows throughout the Internet2 structure

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Inter-Domain Guidelines All backbones will use MBGP/MSDP/PIM-SM MBGP: exchange multicast routing information. MSDP: connect SM clouds (source advertising). PIM-SM: shared tree routing protocol. Join/graft: only deliver traffic on links with active sources. Backbones actively discussing/deploying multicast peering: Abilene, vBNS, NREN, DREN, CA*Net2/3, ESnet, NORDUnet, and SURFnet.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Latest Status Abilene tested multicast code: stable code version found. NREN has successfully deployed PIM-SM. MSDP peering with vBNS and MIX at NASA- Ames. Recently switched to PIM-SM (for load reasons). vBNS has has also had success Switched to PIM-SM recently. MSDP peering with NREN, Merit (others soon). MBGP w/ 8 groups + translation w/ 20 others.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Moving in the Right Direction Doing native multicast is the right way to move forward. We are approaching the problem top down. Need to continue this effort into the Gigapops and to member institutions. Economies-of-scale, in terms of manual intervention, are significant. What does all this mean...

Requirements for Multicast Raise the bar for Internet2. No tunnels: fully deploy native multicast. Peering must be done with MBGP(& MSDP). Institutions who BGP peer should also MBGP peer. Caveats: If no BGP peering (default), then the same for multicast. If congruent unicast/multicast topology, MBGP translate service may be available. Not a complete prohibition of tunnels, but… Be careful about protecting low-capacity interfaces. Don’t create routing loops.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil The Challenges Where things break: Multicast in multi-homed environments when… Switch-over of unicast to I2 but multicast is still via some other network AND connection is via PIM. RPF failures. Things are better when: Multicast is a true I2 service and unicast/multicast topologies are congruent… or, Network uses MBGP.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil The Challenges, Part II Top two layers are key. Need vendor support for inter-domain multicast protocols. Vendor support is coming. Need network operators to be aggressive. Several have set the standard

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Solution: Two Action Items Communicate with upstream service provider about how multicast will be delivered. High confidence in backbones. Abilene NOC (and WG) is educating Gigapops (and members) about how to handle multicast. Members should be prepared to run MBGP/MSDP. Pressure vendors to deploy these protocols. Many vendors have time tables for releases. Can deploy/co-locate multicast in parallel to unicast.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Open Issues How does the I2 Multicast Working Group assist in deployment of multicast from the backbones all the way to member institutions? Use the I2 multicast mailing list (subscribe by mailing - place in the body subscribe wg-multicast Collect experience and create guidelines. Protecting low-capacity multicast environments from high-capacity groups. Replace dense-mode protocols with sparse mode. Set administrative boundaries.

25-28 May 99 SBRC99 Salvador, Bahia, Brazil Chicago vBNS Router CALREN 2 WAN OC48 vBNS WAN OC12 CALREN 2 UCB vBNS Router Client PC Host Client PC Host NREN GRC Router Client PC Host Client PC Host Client PC Host NREN Stanford Router UCSC LAN OC12 Stanford LAN OC12 ESnet ATM Switch GRC NREN ATM Switch Abilene Berkley Router Client PC Host Client PC Host NREN Navajo Router 100BaseTX Hub Uplink GRC Router SBS-5 Navajo Nation Downlink GRC Uplink GRC Uplink ATM Switch ATM downlink Switch Satellite Modem Satellite Modem UCSC Router w2w PC Server SGI Server NREN/NAS Fast Ethernet Switches DS3 OC3 ABILENE WAN OC12 NGIX ATM Switch Sprint / NREN WAN OC3 ARC NREN ATM Switch NREN NAS Router Virtual Clinic Network Diagram With thanks to Mark Foster, NASA Ames

25-28 May 99 SBRC99 Salvador, Bahia, Brazil The End