4How do we decide what locations to go to? Where is Netflix presently providing service?Our service is limited by the content licenses that we can buyFor example, we are in the Nordic countries, the UK, and Ireland but not France/GermanyCan we interconnect with networks that are relevant?Going to a peering point that’s all content providers does no goodCan we reduce the cost and improve traffic management for networks who need to deliver Netflix traffic to their end users?
5Netflix in the Nordics We provide our streaming service to DenmarkFinlandNorwaySwedenIceland and GreenlandCurrently co-located at Telecity Stockholm 1 & 2Stockholm 1 is a network extension from Stockholm 2 due to space/power constraintsLikely to expand to a second market in the Nordics this year
6IXes Netflix delivers their Nordic traffic from four different IXes NetNodNIXAMS-IXLINXThe latter two may not seem relevant here, but many networks have significant capacity built there
7Standard build - European A European Netflix Open Connect IX build has:20 Netflix Open Connect AppliancesTwo Juniper MX480 routersOpenGear console server and Juniper EX2200’s for OOB accessPrewired MTP to LC cabling from ElpeusA “network extension” Open Connect build has:Two Juniper EX4550 switchesTwo MRV 40 channel passive muxesOpenGear console server for OOB access
10Standard Build - US A US Netflix Open Connect IX build has: 40 Netflix Open Connect Appliances60 Netflix Open Connect Flash AppliancesTwo Juniper MX960 routersOpenGear console server and Juniper EX2200’s for OOB accessPrewired MTP to LC cabling from ElpeusA “network extension” Open Connect build has:Two Juniper EX4550 switchesTwo MRV 40 channel passive muxesOpenGear console server for OOB access
12Converting some POPs to Cisco ASR Due to concerns with Juniper code delivery and the need for higher density platforms, we decided in early 2013 to integrate some Cisco ASR 9K into our network to replace Juniper MXIn order to deliver our traffic goals (~1T per router), we need denser cards than the currently shipping Juniper 16 port MPC3’s
13High Density…Juniper MX960 w/ MPC3’s can deliver 176 ports per chassisA single fabric failure brings the chassis to 3:2 oversubscriptionJuniper MX960 w/ MPC4’s can deliver 336 ports per chassis (oversubscribed)A single fabric failure (out of 3) brings the chassis to 2:1 oversubscription – unacceptableCisco ASR9010 w/ Typhoon-based 36 port cards can deliver 288 ports per chassis
14High Density, continued… We conservatively need 130 host ports per routerThose 130 ports can deliver 1.13T of trafficAssuming we deliver 1 bit outbound for every bit inbound from a host, that puts us at 1.1:1 oversubscribed at peak3:2 and 2:1 just do not work for usJuniper MPC4’s also require brand new 12.3 codeWhat version of code are you running?
16The Netflix Open Connect Appliance (OCA) Developed in response to ISP requests to help scale Netflix traffic efficientlyReduces ISP cost by serving Netflix traffic from the local ISP datacenter, CO or headend, rather than upstream network interconnectsSpeeds up internet access for consumers to all third- party internet sites, because Netflix traffic is no longer a source of middle-mile or backbone congestionNetflix bears the capital and maintenance costs, not ISPISP provides space, power and a network portAn OCA is a component of the Netflix CDN (vs a cache)
17OCA Hardware Space optimized: 4U high-density storage Power optimized for low power/cooling requirements (≅500W)Redundant 10GE optical network interfacesRedundant power supplies (AC or DC)(see slide)
18Why Deploy a Netflix OCA? Netflix data is a significant percentage of ISP traffic in the markets we serveServe >80% of Netflix traffic from the local ISP datacenterRemaining traffic can be served by upstream OCA’s, peeringSaves on transit, transport and other upstream scaling costsProvided free of charge to participating ISPsISPs with > 3 Gbps of Netflix trafficISP provides rack space, power, 10 Gbps optical port(s)
19OCA Operation Used exclusively for Netflix content Completely integrated with the Netflix content delivery systemGreater effectiveness than transparent or proxy caches (>80% with Netflix OCA, versus 20-50% with other caches)Efficient content fill mechanismsOutside peak times (ISP selected time and BW per OCA)OCA offline during fill, staggered fill recommended.Fill can source from neighboring, peer or transit OCA’s~5TB fill per day (i.e ~4Gb/s for ~2.5 hrs)OCA must be reachable by end users (port 80)
20Directing Clients to OCAs Netflix Control ServersBroadband ISPUser routing is done by Netflix control servers, not dependent on client DNS configurationRequest is routed to the nearest available OCAWorking sets of popular content deliver up to >80% network offload2. Netflix control servers respond with address of local OCA1. Client contacts Netflix control serversNetflix OCA3. Client connects to local OCA4. Local OCA delivers video stream
21Network PositioningOCA’s are installed close to aggregations of end usersNetflix works with ISPs to determine proper location and user-to-OCA mappingISP Controls Traffic FlowEnd-user netblocks are associated with OCA via BGP advertisementNetflix honors AS path-length and MEDs allowing ISP to configure OCA hierarchy and failoverOCA’s only serve content to netblocks provided by ISPISPs choose peak throughput each OCA is allowed to generate
22The Netflix ISP Speed Index Published monthlyHelps consumers make an intelligent choice about their broadband providerWe want our users to have the best experience possibleWe also want to help networks deliver Netflix traffic as efficiently as possible