Presentation on theme: "Sofía Silva Berenguer lacnic.net"— Presentation transcript:
1 Sofía Silva Berenguer sofia @ lacnic.net Internet Exchange Points WorkshopParamaribo - Surinam
2 AGENDA How the Internet Works Intro to BGP IPv4 Exhaustion and IPv6 DeploymentInternet Exchange PointsHow to request Internet ResourcesAdvanced topicsRoute HijackingLeaksAttacks against the pathWell known incidentsSecuring the Routing System
5 The prefix 18.104.22.168/16 is propagated with BGP to the Internet Internet RoutingASN 6057 announces/16The prefix /16 is propagated with BGP to the InternetASN 8158 receives/16Atributos:/16 AS_PATH ASN1 ASN3 ASN6057
6 Transit and Peering Transit Peering Traffic and prefixes originating from one AS are carried across an intermediate AS to reach their destination ASUsually for a feePeeringPrivate interconnect between two ASNsUsually for no fee
8 Peering in an Internet Exchange Point (IXP) Common interconnect location where several ASNs exchange routing information and trafficASN 65538ASN 65536ASN 65537ASN 65539
9 IP address, where they come from? StandardsCentral RegistryDistribution*Regional Internet Registries (RIRs) distribute IPv4, IPv6 and Autonomous System NumbersDistributionEnd user*Sometimes the distribution is done through National Internet Registries (NIRs)Allocations and Assignments9
12 Border Gateway Protocol A Routing Protocol used to exchange routing information between different networksExterior gateway protocolDescribed in RFC4271RFC4276 gives an implementation report on BGPRFC4277 describes operational experiences using BGPWorks on TCP port 179
13 More about BGPLearns multiple paths via internal and external BGP speakers – Initial exchange of entire tableIncremental UpdatesPicks THE bestpath and installs it in the IP forwarding table – Policies applied by influencing the bestpath selectionKeepalive messages exchangedMany options for policy enforcementClassless Inter Domain Routing (CIDR)Widely used for Internet backbone
14 Neighbors BGP speakers Internal (iBGP) if they are in the same ASN External (eBGP) if they are in different ASNeBGPiBGPASN 65536ASN 65538
15 Where to use BGP: Stub Network ASN 65536,Transit ProviderOnly one exit for customerNot really need to add BGPASN 65538, Customer
16 Multihomed Network Transit Providers Peering in IXP ASN 65538 Different situations possibleMultiple links to same ISPSecondary for only backupLoad share between primary and secondarySelectively use different ISPsPeering at IXPASN 65536ASN 65537Peering in IXP
18 Why peer? Consider a region with one ISP It provides internet connectivity to it’s customersIt has one or two international connectionsInternet grows, another ISP sets up in competitionThey provide internet connectivity to their customersThey have one or two international connectionsHow does traffic from customer of one ISP get to customer of the other ISP?Via the international connections
20 Why peer? (Cont.) International bandwidth… Yes, international connections…If satellite, RTT is around 550ms per hopSo local traffic takes over 1s round tripInternational bandwidth…Is much more expensive than domestic bandwidthBecomes congested with local trafficWastes money, harms performance
21 Why peer? (Cont.) Solution: Result: Two competing ISPs peer with each otherResult:Both save moneyLocal traffic stays localBetter network performanceMore international bandwidth for international traffic
23 Why peer? (Cont.) A third ISP enters the equation Becomes a significant player in the regionLocal and international traffic goes over their international connectionsThey agree to peer with the two other ISPsTo save moneyTo keep local traffic localTo improve network performance
24 Why peer? (Cont.)Peering means that the three ISPs have to buy circuits between each otherWorks for three ISPs, but adding a fourth or a fifth means this does not scaleSolution:Internet Exchange Point
25 Why peer? – Non-financial Motivations Low latencyControl over routingRedundancyAggregation benefits w/peering and Transit at IXPISP relationships – be one of the cool kidsMarketing benefitsNetwork reliability
26 Internet Exchange Point Every participant has to buy just one whole circuitFrom their premises to the IXPRather than N-1 half circuits to connect to the N-1 other ISPs5 ISPs have to buy 4 half circuits = 2 whole circuits -> already twice the cost of the IXP connection
28 IXP DesignEach ISP participating in the IXP brings a router to the IXP locationRouter needs:One Ethernet port to connect to IXP switchOne WAN port to connect to the WAN media leading back to the ISP backboneTo be able to run BGP
29 IXP Design (Cont.)IXP switch located in one equipment rack dedicated to IXPAlso includes other IXP operational equipment (Management network, TLD DNS, Routing Registry, Looking Glass, etc.)Optional: Second switch for redundancyRouters from participant ISPs located in neighbouring/adjacent rack(s)Copper (UTP) connections made for 10Mbps, 100Mbps or 1Gbps connectionsFibre used for 10Gbs and higher speeds
30 Peering at an IXP Each participant need to run BGP They need their own AS numberPublic ASN, NOT private ASNEach participant configures external BGP with the other participants in the IXPPeering with all participantsOrPeering with a subset of participants
31 IXP - RoutingISP border routers at the IXP generally should NOT be configured with a default route or carry the full Internet routing tableCarrying default or full table means that this router and the ISP network is open to abuse by non-peering IXP membersISP border routers at the IXP should not be configured to carry the IXP LAN network within the IGP or iBGPSet BGP next-hop to local router (Cisco IOS next-hop-self)
32 IP Address Space Some IXPs use private addresses for the IXP LAN Public address space means the IXP network can be leaked to the Internet, which could be undesirableFiltering RFC1918 address space by ISPs is Best Practice; this avoids leakageSome IXPs use public addresses for the IXP LANAddress space is available from LACNICIXP terms of participation usually forbid carrying the IXP LAN addressing in the ISP backbone
33 Hardware The IXP Core is an Ethernet Switch (Mandatory) Therefore invest in the best and most expandable equipment that its financial circumstances allowHaving 2 switches is good for redundancy if the funds can allowRoute Server (Optional)Provides ease of configuration for new membersDirect peering between the IXP members can be implemented in the absence of the Route Server
34 Hardware (Cont.) Other optional equipment Web Server (website, monitoring, etc.)Mail Server ( , mailing list, etc.)Transit Router (to provide Internet access to the IXP website, and staff Internet access)Route Collector (Looking glass which assists IXP members with troubleshooting. It can also be used to collect routes for statistics measurements)
35 Hardware - Suggestions Try not to mix port speedsIf 10Mbps and 100Mbps connections available, terminate on different switchesInsist that IXP participants bring their own routerMoves buffering problem off the IXPEnsures integrity of the IXPSecurity is responsibility of the ISP, not the IXP
36 Location The location of the IXP is very important. The IXP location should be neutral and low cost.In considering the IXP location the following factors should be considered:SpaceEnvironment ControlSecurityPowerAccess to terrestrial InfrastructureCablingSupport
37 Recommendations and Best Practices Only announce your aggregates and your customer aggregates at IXPsOnly accept the aggregates which your peer is entitled to originateNever carry a default route on an IXP (or private) peering routerFailing to do so leads to route-hijacks and leaks
38 General Info about IXPs ..Source:https://prefix.pch.net/applications/ixpdir/summary/
39 General Info about IXPs .Source: https://prefix.pch.net/applications/ixpdir/?show_active_only=0&sort=traffic&order=desc
40 General Info about IXPs Source: https://prefix.pch.net/applications/ixpdir/summary/ipv6/
42 Go to https://solicitudes.lacnic.net/ Or fill out a form and send it in the body of a message toYou can find templates at:Once the online request or the form has been processed by the system, the requestor will receive a confirmation with a ticket number.After that the hostmasters will analyze the request.If the request is approved, it may be necessary to pay a fee and to sign the Registration Service Agreement.
43 Who can request resources? The person allowed to request resources for an organization is the Administrative POC.To request resources through the new Requests System you will have to log in using the Administrative POC handle.
44 Requesting an ASNIn order to qualify for an ASN allocation the organization should have:A unique routing policy, meaning a policy that differs from that applied by the upstream provider.Or, a network with more than one independent connection to the Internet. (Multi-homed site)From January 1, 2007 to December 31, 2010 Lacnic assigned ASN of 16 and 32bits upon request. However, since January 1, 2011 Lacnic stopped making distinctions between the assignment of 16- and 32-bit Autonomous Systems Numbers (ASNs) and will only assign ASNs from a general 32-bit pool. This change will be introduced to comply with the Global Policy "Internet Assigned Numbers Authority (IANA) Policy for Allocation of ASN Blocks to Regional Internet Registries" adopted in September 2010.
45 Micro-assignments to Critical Infrastructure Micro-assignment -> prefixes between /24 and /20.For projects and network infrastructure that are key or critical for the region, such as IXPs (Internet Exchange Points), NAPs (Network Access Points), RIRs, ccTLDs, among others.IXPs or NAPs must meet the following requirements:Duly document the following aspects:Prove by means of their bylaws their IXP or NAP capacity. The organization shall have at least three members and an open policy for the association of new members.Submit a diagram of the organization's network structure.Document the numbering plan to be implemented.Provide a utilization plan for the following three and six months.If the applicant does not already have an IPv6 block assigned by LACNIC, simultaneously request an IPv6 block in accordance with the corresponding applicable policy.The rest of the applications shall be studied based on the analysis of the documentation justifying the critical and/or key aspects of the project.Organizations receiving micro-assignments shall not sub-assign these IPv4 addresses.
46 Requesting an IPv4 block for ISPs To qualify for the allocation of a /22 block the org must:Prove usage or immediate necessity of a /24Submit a detailed one-year usage plan for a /23Agree to renumber from previously allocated space and return those IP addresses to their ISPs within 12 monthsIf the applicant does not already have an IPv6 block assigned by LACNIC, simultaneously request an IPv6 block in accordance with the corresponding applicable policy.For a larger block additional requirements apply
47 Requesting an IPv6 block for ISPs To qualify for an initial allocation of a /32 block the organization should:Be a LIR (Local Internet Registry), which means being an organization that assigns address spaces for its network services customersNot be an end site (end user)Document a detailed plan for the services and IPv6 connectivity to be offered to other organizations (clients)Announce the allocated block in the Internet inter-domain routing system, with the minimum possible level of disaggregation to the one that is publishing the IP blocks, within a period no longer than 12 months.Offer IPv6 services to clients physically located within the region covered by LACNIC within a period not longer than 24 months
48 More info Policy Manual Registration Services Registration Services
50 Route HijackingThis occurs when a participant in the Internet Routing announces a prefix for which it has no authorityMalicious or by operational errorsMore know cases:Pakistan Telecom vs. You Tube (2008)China Telecom (2010)Google in Eastern Europe (various AS, 2010)Latin American cases (beginning 2011)
51 Route Hijacking AS 6057 announces 200.40/16 ** Recordar que las rutas mas especificas son preferidasLuego del primer anuncio de rutas, el trafico normal fluye entre el AS 6057 y el AS 8158Luego del segundo anuncio, hay trafico dirigido desde y hacia un /24 que se ve desviado hacia el AS 15358AS announces /24ASN 8158 receives/16 y /24/16 AS_PATH ASN1 ASN3 ASN6057/24 AS_PATH ASN1 ASN15358ASN 8158 receives/16
52 Leaks There is not a standard definition of leaks But it happens when an ASN “leaks” non-customer or self-originated routes to other peers.The effects is to give transit to those networks for the peers of the ASN
53 Route Leaks How this should work without leaks ASN 64511 Transit 2001:db8::/2001:db8:100::/How this should work without leaksASN 64511ASN 65536ASN 655372001:db8:100:/402001:db8::/40Traffic to whole 2001:db8::/40 goes this wayTransit2001:db8::/402001:db8:1/48Peering2001:db8::/402001:db8:100:/40
54 Route Leaks Now a Route Leak ASN 64511 Transit ASN 65536 ASN 65537 2001:db8::/2001:db8:100::/2001:db8:1::/Now a Route LeakASN 64511ASN 65536ASN 655372001:db8:100:/402001:db8::/402001:db8:1::/482001:db8::/40TransitTraffic to 2001:db8:1::/48 goes this way2001:db8::/402001:db8:1/482001:db8::/40Peering2001:db8:100:/40
55 Attacks against the path AS Insertion:A router might insert one or more ASNs, other than its own ASN, into an update messageFalse (Route) Origination with valid ASN:A router might originate a route for a prefix using an ASN not authorized to originate routes for that prefix.
56 Attacks against the path AS 6057 announces /16** Recordar que las rutas mas especificas son preferidasLuego del primer anuncio de rutas, el trafico normal fluye entre el AS 6057 y el AS 8158Luego del segundo anuncio, hay trafico dirigido desde y hacia un /24 que se ve desviado hacia el AS 15358ASN 6057Fake AS 6057 announces /24ASN 8158 receives/16 y /24/16 AS_PATH ASN1 ASN3 ASN6057/24 AS_PATH ASN1 ASN15358ASN 8158 receives/16
58 Pakistan Telecom vs. Youtube On Sunday, 24 February 2008, Pakistan Telecom (AS17557) started an unauthorized announcement of the prefix /24 (Youtube)One of Pakistan Telecom's upstream providers, PCCW Global (AS3491) forwarded this announcement to the rest of the Internet, which resulted in the hijacking of YouTube traffic on a global scale.Reason: “Fat fingers”Video de RIPE NCC
59 Moratel vs Google Reported by Cloudflare on November 06, 2012 Google's services experienced a limited outage for about 27 minutes over some portions of the Internet.Moratel (23947) was “leaking” Google one route and packets were going through IndonesiaReason: “Fat fingers”
60 ASN and Prefix HijackOn 2011 one large European ISP complain that one of their prefixes was being announced by a Mexican ISPThe Mexican ISP review their network but could not found the problemLater it appears that a Brazilian ISP was using the Mexican ISP’s ASN to announce the European prefixReason: Poor BGP knowledge
63 Recall how Internet Resources are managed IANAARINISPEnd usersLACNICNIC.brNIC.mxISP mxISP #1APNICLIRs/ISPsRIPE NCCAfriNICEach RIR is an authoritative source of information about the relation “user” <-> “resource”Each RIR operates its registration data baseMembers and RIRs sign Service Agreements between them63
64 Who has the "right" to use resources? When an ISP obtains resources from its RIR (IPv6/IPv4/ASN):The ISP has to notify its upstream ASs which prefixes are going to be announced via BGPThis is usually done via , web forms or by updating an IRR (Internet Routing Registry)Upstreams verify (or at least they should) the right of use for the announced resourcesRIR WHOIS Text-based and not really suitable for automatic usageIRR WHOIS Non-signed information, little additional tools provided for verification of usage rights except for names, phone numbers and POCsThis verification process is sometimes not as thorough as it should be
65 What is RPKI?RPKI (Resource Public Key Infrastructure) allows the validation of an organization right to use of a certain resource (IPv4, IPv6, ASN)RPKI combines the hierarchy of the Internet resource assignment model through RIRs with the use of digital certificates based on standard X.509RPKI is standardized in the IETF through the SIDR WG. It has produced RFCs 6480 – 6492
66 RPKI All RPKI signed objects are listed in public repositories After verification, these objects can be used to configure filtering in routersValidation ProcessSigned objects have references to the certificate used to sign themEach certificate has a pointer to an upper level certificateThe resources listed in a certificate MUST be valid subsets of the resources listed in its parent's certificateIn this way a trust chain can be traced to a "trust anchor" both cryptographically as well as in CIDR terms
67 X.509 v3 certificates with RFC 3779 extensions Signature AlgorithmSerial NumberVersionIssuerSubjectSubject Public KeyExtensionsAddr:Asid: 65535Subject InformationAuthority (SIA)Authority InformationAccess (AIA)X.509 Digital CertificatesSubject, validity period, public key and other fieldsWith extensions:RFC 3779 defines extensions that allow the representation of Internet resources as certificate fieldsList of IPv4, IPv6 and ASNs assigned to an organizationImplemented in OpenSSL 1.0c onwards
68 ROAsUsing Certificates we can create objects describing the origin of a prefixROAs: Routing Origin AuthorizationROAs contain data on the allowed origin-as for a set of prefixesROAs are signed using the certificates generated by the RPKISigned ROAs are copied to the repository
69 ROAs (ii) A simplified ROA contains the following information: These ROAs states that:"The prefix /17 will be originated by ASN 6057 and could be de-aggregated up to /20" "This statement is valid starting on Jan 2, 2013 until Dec 31, 2013"Other ROA contentROAs contain cryptographic material that allows validation of the ROAs contentPrefixMaxLenOrigin ASValid SinceValid Until/17206057/222428000
70 ROAs (iii) - Validation ROAs validation process includes:Criptographic validation of End Entity certificates (EE) that are included in each ROACIDR validation of resources listed in the EE against the resources listed in the issuing certificateVerification that prefixes listed in the route origin attestations are included in the prefixes listed in the EE certificates of each ROA
73 Origin ValidationRouters build a database with the information they receive from the cachesThis table containsPrefix, Min length, Max length, Origin-ASBy applying a set of rules a validity status is assigned to each UPDATE prefixNetwork operators can use “validity” attribute to construct routing policies
74 Origin Validation VALID UPDATE /9 ORIGIN-AS 20IP prefix/[min_len – max_len]Origin AS/ [16-20]10/[8-21]20If the "UPDATE pfx" is not covered by any entry in the DB -> "not found"If the "UPDATE pfx" is covered by at least one entry in the DB, and the origin-AS matches the ASNs in the DB -> "valid"If the origin-AS does NOT match -> "invalid"
75 Origin Validation INVALID UPDATE /22 ORIGIN-AS 20IP prefix/[min_len – max_len]Origin AS/ [16-20]10/[8-21]20If the "UPDATE pfx" is not covered by any entry in the DB -> "not found"If the "UPDATE pfx" is covered by at least one entry in the DB, and the origin-AS matches the ASNs in the DB -> "valid"If the origin-AS does NOT match -> "invalid"
76 Origin Validation INVALID UPDATE /9 ORIGIN-AS 66IP prefix/[min_len – max_len]Origin AS/ [16-20]10/[8-21]20If the "UPDATE pfx" is not covered by any entry in the DB -> "not found"If the "UPDATE pfx" is covered by at least one entry in the DB, and the origin-AS matches the ASNs in the DB -> "valid"If the origin-AS does NOT match -> "invalid"
77 Origin Validation NOT FOUND UPDATE /9 ORIGIN-AS 66IP prefix/[min_len – max_len]Origin AS/ [16-20]10/[8-21]20If the "UPDATE pfx" is not covered by any entry in the DB -> "not found"If the "UPDATE pfx" is covered by at least one entry in the DB, and the origin-AS matches the ASNs in the DB -> "valid"If the origin-AS does NOT match -> "invalid"
78 Routing Policies with Origin-validation Using the BGP validation attribute network operators can construct routing policiesFor example:Assign higher preference to routes with status “valid” than routes with status “unknown”Drop routes with status of “invalid”Very important, RPKI is a source of data, operators are free to use it as it suits them better
79 RPKI Management System RPKI in actionCacheRPKI Management SystemRepositoryRouters validate updates from other BGP peersCaches feeds routers using RTR protocol with ROA informationCaches retrieve and cryptographically validates certificates and ROAs from repositories
80 RPKI Operation Modes “Hosted” mode “Delegated” mode LACNIC issues certificates and keeps public and private keys in its systemsCertificates are issued at the request of member organizationsLACNIC provides a web interface for management“Delegated” modeEach organization has it’s own certificate, signed by LACNIC’s CAThe organization sends signing requests to LACNIC, who returns them signed“Up-down” protocol
81 RPKI today5 CAs and repositories working (RIRs) in hosted and delegated mode (just APNIC)Validation, CA and origin validation working software and devicesSome other tools built around (bgpmon, LACNIC Labs, RIPE Labs)~ 11,700 routes signed (~ 8.9% 1,226 invalid or hijacks)Origin Validation available in Cisco, Juniper, Quagga, BIRD
82 LACNIC’s RPKI Structure Self-signed RTALACNIC RTALACNIC’s ResourcesLACNIC Production<<INHERITED>>ISP #2Resouces of ISP #2ROAEnd Entity cert.ISP #1Resources of ISP #1End User CA #1(Resources of EU#1)Signing chainExplicar la idea del offlineComentar sobre el single trust anchorLACNIC RTA es la raiz de la rPKI para la region LACEs un certificado **auto firmado**La clave privada de LACNIC RTA esta offlineLACNIC Produccion es un certificado intermedio, el que efectivamente firma los objetos con su clave privadaISP #n son CAs que pueden firmar otras CasCada uno de ellos lista los recursos del ISP correspondienteLos ROAs son objetos firmadosContienen informacion de enrutamiento (origin-as)Usan un certificado end-entity cuya clave privada se usa una vez y luego se descarta
83 LACNIC’s RPKI Structure (ii) CAsEntity that issues certificates (bit CA=1)ISPs can use the certificate to sign it’s clients’ certificatesCertificates RepositoryRepository of certificates, CRLs and manifestsAccesible through “rsync”Management InterfaceUser web interface for those who prefer “hosted” mode
84 RPKI CA ServicesChildren certificates issuance when the registration database is updated or on user demandChildren certificates revocation in a centralized manner or on user demandPeriodic issuance of CRL for CA certificateCA Certificate and children certificates publication in a public repository (rsync)
85 ConclusionsThe routing system is one of the core operations of the InternetStill is vulnerable to attacks and bad configsSome work has been done (RPKI, Origin Validation)We need to continue our workProtocol specificationDeployment (Filtering, RPKI, Origin validation)
86 Links / References LACNIC’s RPKI System LACNIC’s RPKI Repository LACNIC’s RPKI Repositoryrsync://repository.lacnic.net/rpki/To see the repositoryrsync --list-only rsync://repository.lacnic.net/rpki/lacnic/RPKI Statistics