Virginia Tech Campus Network Overview Internet2 Campus Focused Workshop On Advanced Networks Clark Gaylord Virginia Tech 30 May 2001

Gaylord: Virginia Tech Campus Overview 2 Basic Services Basic Ethernet portal service (~25,000 ports) –Switched 10Base-T (10 Mbit/s) –Some switched 100Base-TX (100 Mbit/s) Dial-in modem pool with ~1,200 56k modems Wireless LAN (currently in pilot phase, beginning second round deployment) ATM portal service (25 or 155 Mbps), mostly for specialized applications (e.g. distance learning programs) Local network access point provides campus backbone access to local ISPs. Ethernet in apartment complexes Telephone (~13,000 phones) and cable television (~5,000 connections)

30 May 2001Gaylord: Virginia Tech Campus Overview 3 Some Network Management Tools Largely home-grown –ARP/Forwarding table/Cable plant databases (HC++ ™) –Device status reports –TACACS+ database query tool –nemisys - Network Engineering Management Information SYStem ™ –Various ad hoc tools –People Commercial –Smarts InCharge –Concord Network Health –InterMapper, What’s Up “inventory” tools (“Your switch is still alive.”) Freeware, etc –Usual suspects: MRTG, Big Brother, Netsaint

30 May 2001Gaylord: Virginia Tech Campus Overview 4 ARP/MAC mappings 1

30 May 2001Gaylord: Virginia Tech Campus Overview 5 ARP/MAC mappings 2

30 May 2001Gaylord: Virginia Tech Campus Overview 6 Roll your own

30 May 2001Gaylord: Virginia Tech Campus Overview 7 End-to-End Management Largely ad hoc, informal measurements ttcp, netperf, treno, Ganymede nlanr AMP project How to you obtain measurements from the far end? Usual SNMP tools cannot/do not address burstiness Configuration management/production quality

30 May 2001Gaylord: Virginia Tech Campus Overview 8 ISM

30 May 2001Gaylord: Virginia Tech Campus Overview 9 AMP

30 May 2001Gaylord: Virginia Tech Campus Overview 10 Switched 10 Mbps to the desktop. Switched 100 Mbps where desired. Switched 100 Mbps building backbone. N x 100 Mbps scalable building feed. Simple model ISL virtual LAN capability Standard Building Network Catalyst 1900/1924 Core n x 100Base-FX 100Base-FX downlinks 10Base-T to wall portal Cisco Catalyst 5000 Cisco Catalyst 6500 Catalyst 1900/1924

30 May 2001Gaylord: Virginia Tech Campus Overview 11 Cisco Catalyst /100 available to the desktop Scalable Gigabit Ethernet building backbone and building feed 802.1Q/ISL virtual LAN capability 802.1p packet prioritization capability b Wireless LAN Deployed in Torgersen Hall Advanced Building Network Cisco Catalyst 6500 Catalyst 3500 Core n x 1000Base-LX 1000Base-SX downlinks 1000Base-TX inter-switch links 10/100Base-TX to wall portal Wireless LAN Access point

30 May 2001Gaylord: Virginia Tech Campus Overview 12 Wireless LAN Deployment Plan ( ) Current/pilot: –Torgersen (ACITC) –Information Systems Summer 2001: –Library (partial) –Student Centers (partial) –Randolph (Engineering) –Pamplin (Business) : –Veterinary Medicine –Seitz (Agriculture) –McBryde (Math, CS)

30 May 2001Gaylord: Virginia Tech Campus Overview 13 Wireless LAN Deployment Two-three year deployment schedule –Instructional space –Student centers –Other space as needed Facilities to support portable computing Based on b –Economics of a dubious Keep it “standards” based Large perimeter –WEP/ESSID not supportable, negligible benefit 802.1X/EAP –a definite maybe Airspace internal policy Custom antenna design?

30 May 2001Gaylord: Virginia Tech Campus Overview 14 MSAP DSL T1/FR LMDS Modem Pool VT ATM Backbone N.W.V 1 x 1000Base-SX 1 x 1000Base-LX 2 x 1000Base-LX ATM OC3c 1 x 100Base-FX/TX SHA Catalyst 6500 OWE Catalyst 6500 HIL Catalyst 6500 ISB Cisco 7500 Internet Internet2 ISB Catalyst 6500 BUR Catalyst 6500 CAS Catalyst 6500 ISB Catalyst 6000 ISB Cisco 7500 CAS Cisco 7500 Future POS/DPT/ATM OC12c or 1000Base-ZX VT Gigabit Backbone

30 May 2001Gaylord: Virginia Tech Campus Overview 15 ATM Architecture ATM provides key benefits, not yet feasible or cost-effective using other technologies –Integrated, multi-service WAN access Voice transport (toll-bypass) Videoconferencing (distance learning) IP data transport –Layer 2 virtual private networks with explicit resource provisioning; vital to network layer research. Also provides last-resort backup for the gigabit backbone. –When all else fails, we haul it over ATM.

30 May 2001Gaylord: Virginia Tech Campus Overview 16 N.W.V VT Gigabit Backbone ATM OC3c multi-mode ATM OC3c single-mode 1000Base-SX Internet Internet2 SHA ASX-200WG OWE ASX-200WG BUR ASX-200BX CAS ASX-200BX ISB ASX-1000 HIL ASX-200BX CAS Cisco 7507 ISB Cisco 7507 ISB Cisco 7507 VT ATM Backbone

30 May 2001Gaylord: Virginia Tech Campus Overview 17 Net.Work.Virginia Provides ATM services throughout Virginia to K-12 schools, institutions of higher education, and state agencies Low cost access with level pricing statewide –OC3c (155 Mbit/s), ~ $134,000/annum –DS3 (45 Mbit/s), ~$53,000/annum –DS1 (1.5 Mbit/s), ~$12,000/annum Provides Internet access services, as well as supporting a wealth of ATM-based services (intranet, videoconferencing, PSTN toll bypass, etc) Provides access to research backbones (e.g. Abilene) Currently 595 sites

30 May 2001Gaylord: Virginia Tech Campus Overview 18 Net.Work.Virginia SprintLink (Internet) Abilene (Internet2) Sprint Public ATM ESnet Regional LEC ATM Regional LEC ATM OC-12c ATM OC-12c POS N x OC-3c/DS-3 ATM WTN ASX-1000 ROA ASX-1000 RIC ASX-1000 ROA GSR RIC GSR WTN GSR Regional LEC ATM Regional LEC ATM Regional LEC ATM Regional LEC ATM Regional LEC ATM Regional LEC ATM

30 May 2001Gaylord: Virginia Tech Campus Overview 19 Quality of Service Traffic Classification Congestion Avoidance –Policing –WRED Congestion Management –CBWFQ –Shaping/Conditioning

30 May 2001Gaylord: Virginia Tech Campus Overview 20 Traffic Classification DSCP/IP Precedence marking Classify different classes of “Best Effort” to protect each from each other Classes can be based on users or applications Use netflow to determine major applications Use business rules to determine major user bases

30 May 2001Gaylord: Virginia Tech Campus Overview 21 Traffic Classification Only bits 0-2 are used (DiffServLite?) –Control Plane (DSCP=56(?) set by routers) –IP Telephony (DSCP=40(?) set by end-station) –On-campus any-to-any (DSCP=24) [Following are destined off-campus] –Non-residential (aaanet) TCP (DSCP=16) –Residential (resnet) TCP (DSCP=8) –Other residential and non-residential (notTCP) (DSCP=0) –Should these be xxxx11? Except for IP Telephony and control plane, classification is done at entry to core (DSCP is only trusted in one edge VLAN, viz. telephony) No use of 802.1p (outside of telephony LAN)

30 May 2001Gaylord: Virginia Tech Campus Overview 22 Traffic Shaping Outgoing only (ATM OC-3 … nominally 100Mbps) –aaanet 40Mbps –resnet 40Mbps –notTCP 10Mbps –CBWFQ –No priority queuing (at this time) No shaping for on-campus traffic

30 May 2001Gaylord: Virginia Tech Campus Overview 23 Traffic Policing On-campus –No policing –All best-effort –Lots of excess capacity Incoming to campus –Same as on-campus Outbound –Police resnet to 60Mbps (nominal 60% of Internet pipe) at entry to border area –Formerly specifically policed “application of special interest”

30 May 2001Gaylord: Virginia Tech Campus Overview 24 Cisco Catalyst 5000 Path of a packet Cisco Catalyst 6500 Catalyst 1900/1924 Core n x 100Base-FX 100Base-FX downlinks DSCP marking happens here

30 May 2001Gaylord: Virginia Tech Campus Overview 25 Path of a packet MSAP DSL T1/FR LMDS Modem Pool N.W.V SHA Catalyst 6500 OWE Catalyst 6500 HIL Catalyst 6500 ISB Cisco 7500 Internet Internet2 ISB Catalyst 6500 BUR Catalyst 6500 CAS Catalyst 6500 ISB Catalyst 6000 CAS Cisco 7500 From building network DSCP marking happens here Policing happens here Conditioning happens here VT ATM Backbone ISB Cisco 7500

30 May 2001Gaylord: Virginia Tech Campus Overview 26 What we have learned Outbound congestion creates poor inbound TCP performance Conditioning alone does not work well when offered load is high Leave room for bursts (set sum of committed traffic to 90% of link?) Policing alone works for isolated traffic classes Doing anything is better than doing nothing - even for voice Divide and conquer

30 May 2001Gaylord: Virginia Tech Campus Overview 27 Futures Priority queuing for voice Traffic classes for distance learning (DLoIP ™), Research Computing, et al. Per-user classification and user-class policing Netflow data gathering and traffic classification Remote monitoring of end-to-end performance How do we measure bursts?

30 May 2001Gaylord: Virginia Tech Campus Overview 28 For More Information