Presentation on theme: "“Internet2 Presentation”"— Presentation transcript:
1 “Internet2 Presentation” Brian Stengel, Director of Operations, KinberPeter J. Heverin, Project Manager, KinberMike Carey, Network Engineer, KinberJon Paul Herron, Director of Engineering, GlobalNOCLuke Fowler, Senior Manager, Systems Engineering, GlobalNOCMarianne Chitwood, Director of Operations, GlobalNOCSeptember 20th, 2012
2 What is PennREN Capital Budget - $128,958,031 a high-speed, state-wide, research & education serving healthcare, K-20 and the public goodCapital Budget - $128,958,031Federal Stimulus Funds - $99,660,678Matching Funds - $29,297,353Outside Plant Infrastructure Constructed for PennREN48 Stands of NZD Fiber optic Cable1700± Route MilesOutsourced Fiber Maintenance13 Optical Regeneration Service Nodes56 Service Distribution Access NodesPeter talk about PennREN and KINBER, grant process, history
3 Project Route Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following;1,086 miles of new aerial construction.486 miles of aerial overlash.14 miles of new underground construction.27 miles of leased underground conduit.Network Backbone Engineering is 95% complete overall.Peter talk about outside plant3
5 PennREN Locations Service Nodes Amp Nodes Access Nodes Lehigh University401 N. BroadPenn State Hershey Medical CenterIndiana University of PennsylvaniaAllegheny Center MallSlippery Rock University of PennsylvaniaPenn State Erie (Behrend)Clarion University of PennsylvaniaPenn State DuboisPenn State – State CollegeWindstreamBucknell UniversityUniversity of ScrantonAmp NodesEast Stroudsburg University of PaKutztown University of PaWest Chester University of PaMillersville University of PaShippensburg University of PaPitt JohnstownCommunity College of Beaver CoAllegheny CollegePitt TitusvillePenn State HazeltonAccess Nodes46 Locations on PennREN fiberPeter talk about the locations
6 Project Teams Fiber OSP Building entrances Cabinets, Installation SplicingTestingLast mile solutionsHardware procurementDesignIntegrationProject ManagementProfessional ServicesPlan/DesignOptical engineeringField servicesProfessional servicesStagingShippingConfigurationTestingTurn-upField ServicesPlan/DesignConsultingEngineeringSupportTrainingService Desk, Tier 1, Tier 2Systems and Network ToolsNetwork EngineeringNet, Perf, Change ManagementOperations supportPeter talk about the RFP for OSP and for network equipmentEarthlink sourcing for OOB MPLS NetworkLast mile solutions
7 Node Installation Status Peter talk about the status of equipment installations and transition to MikeSegment in ProductionEquipment InstalledUnder Construction
8 Co-Location at Member Sites Construction DiagramsMike to start here talk about meeting with members to negotiate space, power, etc… for putting in equipment. Talk about the greenfield opportunity to design from the ground up….
9 Optical Core/DWDM System Talk about the selection of the core DWDM system – ADVA equipment
10 Packet Core – MPLSTalk about the selection of the MX480
11 Plan – Build – Integrate Talk about our approach of taking the Torrey Point design, ADVA/Juniper BOM and working with the various teams to take design to plan. Then moving from plan to build and then integration. Talk about using GRNOC systems as the unit of record and that our Google sites and collaboration tools free for us and not very structured… so we needed GRNOC structured systems for final unit or record.Talk about coordination with teams and how GRNOC provides support in PM, NE, and SD
12 Peering Points (Initial) Talk about how we will deliver external network connectivity and how GRNOC is helping us with thisInternetInternetR&ER&E
13 Network ManagementPS1Performance Measurement Servers at every Service NodePS2PS3In-band managementDeploying performance measurement servers at each service node3 servers per site:Active throughput measurement (1G and 10G)Active latency measurementlocal data collection / ad hoc performance measurementSpecific performance measurement tools include:BWCTLOWAMPMaDDashGlobalNOC tool-set updated to support all PennREN devices, including:SNAPP – High-resolution SNMP-based network utilization dataLLAMA – DWDM layer performance measurement dataCentral storage of other passive data like syslog, configuration, flow, etc. for regular and ad-hoc processing & analysisOut-of-band accessExternal NetworksDBExternal Networks
14 Operations - Support GRNOC provides to KINBER: Service Desk – 24x7x365 call center support, ticket management, technical support coordination, and workflow supportNetwork Engineering – Expert network engineers work with the Service Desk to ensure fast problem resolution, provisioning, and strategic engineering and planningSoftware and Systems – Provides support through a fully integrated system of network management, measurement, and visualization toolsThe GlobalNOC at Indiana University provides carrier-grade operations, tools, and network expertise while placing a singular focus on the unique requirements of the research and education (R&E) CommunityGRNOC supports 20+ R&E networks across the countryTalk about the GRNOC people as being their best asset and the tremendous job they are doing for us… allowing us to keep our costs low and exploiting the experience of GRNOC to bring this network up and into production.
15 Operations - Maintenance WarrantyHardware replacement (pre-ship)Software maintenanceFiber MaintenanceEmergency RestoralRoutine MaintenanceOSP RecordsWarrantyHardware replacement (pre-ship)Software maintenanceHost IT/Site teams provide remote eyes and hands support upon requestTalk about how GRNOC will help us manage maintenance on the network, change management, incidents, etc…KINBER Network Engineers
16 Foundation for Services Multi-DegreeROADMsMPLS PE Switches3rd Party ProvidersWest DWDM RingEast DWDM RingOur foundation for servicesASBR RouterExternal NetworksExternal Networks
17 On-Net: Delivery Access Nodes Access Node Access Node Access Node Talk about how our standards will apply to access nodes once we start installing equipmentAccess NodesInternetInternetR&ER&E
18 Off-Net: Delivery First/Last Mile NNI Commercial Co-Lo Member Off-Net CarrierCXFirst/Last MileCommercial Co-LoPrivateMember Off-NetNNITalk about how GRNOC will help usInternetInternetR&ER&E
19 EPC – Ethernet Port Connection Service PennRENR&E networksVPLS (pt – pt, multipoint)Customer RouterKMEXService Node: PE SwitchCommodity InternetCustomer Subscribes to 1/10GE Ethernet Port(s)
20 Services Services available to a customer with an EPC Member B Member AKINBER Member ExchangeR&E TransitVPLS – Pt-Pt, MultipointCommodity InternetMember CMember A
21 Services KINBER Member Exchange R&E Transit Commodity Internet Community-wide, distributed service for members to exchange traffic across a common network. Similar to an Internet ExchangeMember-to-Member peering, ad-hoc R&E activitiesBest-effort traffic exchange within the communityR&E TransitTransit service to major R&E networks such as Internet 2, ESNet, NLR…Provided by KINBER affiliatesVirtual private networks with committed bandwidth can be established using VPLS instances in the PennREN networkVPWS – Virtual Private Wire Service – Point to PointVPLS – Virtual Private LAN Service - MultipointVPLS – Virtual Private LAN Service (Point-to-Point Multi-Point)Commodity InternetAccess to commodity Internet service is available over the PennREN network
22 Optical – Wave/Lambda Services Optical Waves 10G can be provisioned across the networkMember AMember AYou may want to discuss plans for GMPLS and how we can use NM and SM in coordination with GRNOC.Member BMember AMember B
23 Performance Measurement PerfSONAR Measurement Archives allow exchange of data with other network operatorsRegularly scheduled testing across the backboneUser-initiated testing for applications such as problem diagnosisMultiple routing tables allow us to support both 1G and 10G testpoints on a single host
24 Performance Measurement Deploying performance measurement servers at each service node3 servers per site:Active throughput measurement (1G and 10G)Active latency measurementlocal data collection / ad hoc performance measurementSpecific performance measurement tools include:BWCTLOWAMPMaDDash24
25 Network MeasurementGlobalNOC tool-set updated to support all PennREN devices, including:SNAPP – High-resolution SNMP-based network utilization dataLLAMA – DWDM layer performance measurement dataCentral storage of other passive data like syslog, configuration, flow, etc. for regular and ad-hoc processing & analysis.
26 PennREN Service Desk GlobalNOC Specialized Support Technician Footprints PennREN project created – integrated to TickMon, Operations Calendars, Trouble TicketsTelephone number for PennREN customers integrated into shared GlobalNOC phone queueestablished to receive customer inquiries and or communications from vendors….this is monitored 24x7x365
27 PennREN Service Desk Network and Member impact guidelines defined Web form for customers/vendors to submit trouble tickets into FootprintsChange management process/form createdInternal documentation developed/published for staff training and reference
28 PennREN Service Desk Pro-active network monitoring Support for scheduled maintenances and changesVendor coordinationCustomer install processReportingSecurityTools and Communications
29 Weighing network needs Early Service Definition can guide the network plan in the right way.Projects are often looked at from the perspective of competing demands of cost, time, and quality. Networks have something similar, with Cost, Time, Capabilities, and Reach:Cost & Time: pretty obviousReach: How BIG is the network? What all needs to be connected?Capability: Speed, Service Level expectations, AND some idea of the kinds of services & features it will need to have day 1, and day 1826For PennREN, Cost and Time were mostly dictated by the award. Reach was also given high priority by the award. Capability and service were the area of most planning. Given the short time, the set cost, and the scope of the network, what services and capabilities were possible, and how do they maximize this as much as possible?
30 A new network is more than a construction project It’s a system:GoalsInfrastructurePeopleServicesOperationsBusinessMake decisions and plans early (also, there will be more decisions than you think, so watch for bottlenecks)The work on all of these areas starts right awayWe’ve found that new network builds often seem to focus so heavily on the build that all the other things get postponed until later. But really, a new network will be a larger system, with the infrastructure but also the services, operations, and business models that all need to be pointing the same direction, toward the network goals we mentioned before.In PennREN, everyone involved has been working hard to make sure that happens throughout the project.
31 Service DefinitionUnrealistic to imagine services will be completely defined from the beginning of the designBut, enough understanding is needed to guide the design and build planEarly “anchor” users help a lot!In PennREN’s case, it was enough to know that “off-net” connections for commodity and Internet2 access were needed, and that there was a need for multipoint, point-to-point L2 servicesThis helps to guide not just the network plan for what to buy, what protocols are needed, etc, but also the Operations plan. This includes what will need to be monitored, who the providers will be, who the users will be, how to define services in the database, and more.
32 Communications Communication needs: to be high-bandwidth Early is good Changing staff can be disruptive, do it carefully.Informal is goodMulti-channel is goodreliable technologies are goodFace to face is goodBuild-out like PennREN is big, and involves lots of organizations: fiber building group, administrative group, NOC, integrator, etcEveryone knows communication is important to do something like this. We found that this communication was most effective as the relationships grew, and the people understood the total context of the project and how best to work together. Getting to that point fast is important. Face to face helps.Changing people involved is disruptive during projects like this. New people have to come up to speed technically, but also have to assimilate themselves into how the group works.Meetings and formal tracking are important. But equally important are the informal communications at technical levels. Using chatrooms was immensely helpful for technical staff from all the organizations to get the real details done.
33 Documentation Documentation: Have a place for documents Keep the place for documents cleanKeep the purpose of each document clear/distinct
34 Operations Operations Preparedness: Start early, there are a LOT of things to think about, especially:Turn-up/acceptance processExpectations for facilitiesHow and where to keep network dataWe find that it makes things much easier to be thinking about operating the network throughout the build. For example:building a network produces a LOT of data. Think about how you might need to use that data for troubleshooting, maintenance, or upgrades, and put it somewhere where it’ll be easy and fast to find when you need it.the number of small, but long-lived decisions is more than you think. How are you identifying circuits? How are you defining individual services? What documentation of each site is needed? How will you keep track of asset changes due to maintenance, RMAs? What arrangements will you have with facilities for power, remote hands, etc?
35 Build-outs make for lots of information Need to be ready to put it somewhere where it’ll be usable laterData entry is cheapEntering/documenting is best when the information and its context are fresh
36 The lowly management network Sometimes doesn’t get enough attentionIt can get crazy complicated or crazy expensiveNOTE: in an SDN world, this becomes even MORE important!Maybe more on specific technical recommendations?