Content Overlays (Nick Feamster) February 25, 2008.

Slides:



Advertisements
Similar presentations
Vincent Conitzer CPS Repeated games Vincent Conitzer
Advertisements

Basics on Game Theory Class 2 Microeconomics. Introduction Why, What, What for Why Any human activity has some competition Human activities involve actors,
Rarest First and Choke Algorithms Are Enough
Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.
The BitTorrent Protocol
The BitTorrent Protocol. What is BitTorrent?  Efficient content distribution system using file swarming. Does not perform all the functions of a typical.
Incentives Build Robustness in BitTorrent Author: Bram Cohen Presenter: Brian Liao.
Incentives Build Robustness in BitTorrent- Bram Cohen Presented by Venkatesh Samprati.
The BitTorrent protocol A peer-to-peer file sharing protocol.
Incentives Build Robustness in BitTorrent Bram Cohen.
Peer-to-Peer (P2P) Networks
Bit Torrent (Nick Feamster) February 25, BitTorrent Steps for publishing – Peer creates.torrent file and uploads to a web server: contains metadata.
Stochastic Analysis of File Swarming Systems The Chinese University of Hong Kong John C.S. Lui Collaborators: D.M. Chiu, M.H. Lin, B. Fan.
The BitTorrent content distribution system CS217 Advanced Topics in Internet Research Guest Lecture Nikitas Liogkas, 5/11/2006.
Clustering and Sharing Incentives in BitTorrent Systems Arnaud Legout 1, Nikitas Liogkas 2, Eddie Kohler 2, Lixia Zhang 2 1 INRIA, Projet Planète, Sophia.
Modelling and Performance Analysis of BitTorrent-Like Peer-to-Peer Networks.
Key Algorithms in a Content Delivery System Akamai Technologies and Carnegie Mellon University Bruce Maggs.
CompSci 356: Computer Network Architectures Lecture 21: Content Distribution Chapter 9.4 Xiaowei Yang
CMPT 401 Summer 2007 Dr. Alexandra Fedorova Lecture XV: Real P2P Systems.
Network Coding for Large Scale Content Distribution Christos Gkantsidis Georgia Institute of Technology Pablo Rodriguez Microsoft Research IEEE INFOCOM.
Peer-Assisted Content Distribution Networks: Techniques and Challenges Pei Cao Stanford University.
Modeling and analysis of BitTorrent-like P2P network Fan Bin Oct,1 st,2004.
Bounds on the Performance of P2P Networks Using Tit-for-Tat Strategies Dimitri DeFigueiredo Balaji Venkatachalam S. Felix Wu.
Modeling and Performance Analysis of Bitorrent-Like Peer-to-Peer Networks Dongyu Qiu and R. Srikant University of Illinois, 2004 Presented by : Ran Zivhon.
1 Incentives Build Robustness in BitTorrent? Analysis on Bit Torrent Client performance By Jimmy Wong.
A P2P file distribution system ——BitTorrent Fan Bin Sep,25,2004.
Courtesy: Philip Levis/David Mazieres, Stanford More application layer BitTorrent (Quick Overview) Tahir Azim.
Incentives Build Robustness in BitTorrent 1st Workshop on Economics of Peer-to-Peer Systems 2003 Bram Cohen
BitTorrent Background. Common Scenario Millions want to download the same popular huge files (for free) –ISO’s –Media (the real example!) Client-server.
The Bittorrent Protocol
1 BiToS: Enhancing BitTorrent for Supporting Streaming Applications Aggelos Vlavianos, Marios Iliofotou and Michalis Faloutsos Department of Computer Science.
Bit Torrent (Nick Feamster) February 25, BitTorrent Steps for publishing – Peer creates.torrent file and uploads to a web server: contains metadata.
BitTorrent Presentation by: NANO Surmi Chatterjee Nagakalyani Padakanti Sajitha Iqbal Reetu Sinha Fatemeh Marashi.
University of Bologna, Italy How to cheat BitTorrent and why nobody does Simon Patarin and David Hales University of Bologna ECCS 2006,
1 V1-Filename.ppt / yyyy-mm-dd / Initials P2P content distribution T Applications and Services in Internet, Fall 2008 Jukka K. Nurminen.
BitTorrent.
BitTorrent Internet Technologies and Applications.
BitTorrent Dr. Yingwu Zhu. Bittorrent A popular P2P application for file exchange!
A P2P file distribution system ——BitTorrent Pegasus Team CMPE 208.
1 BitHoc: BitTorrent for wireless ad hoc networks Jointly with: Chadi Barakat Jayeoung Choi Anwar Al Hamra Thierry Turletti EPI PLANETE 28/02/2008 MAESTRO/PLANETE.
2: Application Layer1 Chapter 2: Application layer r 2.1 Principles of network applications r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail  SMTP,
Do incentives build robustness in BitTorrent? Michael Piatek, Tomas Isdal, Thomas Anderson, Arvind Krishnamurthy, Arun Venkataramani.
How Akamai Handles Large Events Bruce Maggs Carnegie Mellon Duke Akamai Technologies.
Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007.
MULTI-TORRENT: A PERFORMANCE STUDY Yan Yang, Alix L.H. Chow, Leana Golubchik Internet Multimedia Lab University of Southern California.
Application Layer 2-1 Chapter 2 Application Layer Computer Networking: A Top Down Approach 6 th edition Jim Kurose, Keith Ross Addison-Wesley March 2012.
Peer-to-Peer File Sharing Jennifer Rexford COS 461: Computer Networks Lectures: MW 10-10:50am in Architecture N101
Content Overlays Nick Feamster CS 7260 March 14, 2007.
B IT T ORRENT T ECHNOLOGY Anthony Pervetich. H ISTORY Bram Cohen Designed the BitTorrent protocol in April 2001 Released July 2, 2001 Concept Late 90’s.
Impact of Incentives in BitTorrent By Jenny Liu and Seth Cooper.
Bit Torrent Nirav A. Vasa. Topics What is BitTorrent? Related Terms How BitTorrent works Steps involved in the working Advantages and Disadvantages.
PEAR TO PEAR PROTOCOL. Pure P2P architecture no always-on server arbitrary end systems directly communicate peers are intermittently connected and change.
© 2016 A. Haeberlen, Z. Ives CIS 455/555: Internet and Web Systems 1 University of Pennsylvania Decentralized systems February 15, 2016.
Md Tareq Adnan Centralized Approach : Server & Clients Slow content must traverse multiple backbones and long distances Unreliable.
Entry Deterrence Players Two firms, entrant and incumbent Order of play Entrant decides to enter or stay out. If entrant enters, incumbent decides to fight.
Algorithms used by CDNs Stable Marriage Algorithm Consistent Hashing.
Peer-to-Peer Networks 10 Fast Download Christian Schindelhauer Technical Faculty Computer-Networks and Telematics University of Freiburg.
November 19, 2016 Guide:- Mrs. Kale J. S. Presented By:- Hamand Amol Sambhaji. Hamand Amol Sambhaji. Pardeshi Dhananjay Rajendra. Pardeshi Dhananjay Rajendra.
Consistent Hashing. Hashing E.g., h(x) = (((a x + b) mod P) mod |B|), where P is prime, P > |U| a,b chosen uniformly at random from Z P x is a serial.
Lecture XV: Real P2P Systems
An example of peer-to-peer application
Introduction to BitTorrent
nTorrent: Peer-to-Peer File Sharing in Named Data Networking
Designing a new BitTorrent Client
Angelo Sapello University of Delaware
Small Is Not Always Beautiful
The BitTorrent Protocol
PEER-TO-PEER SYSTEMS.
Pure P2P architecture no always-on server
Presentation transcript:

Content Overlays (Nick Feamster) February 25, 2008

Hashing E.g., h(x) = (((a x + b) mod P) mod |B|), where P is prime, P > |U| a,b chosen uniformly at random from Z P x is a serial number Universe U of all possible objects, set B of buckets. object: set of web objects with same serial number bucket: web server Hash function h: U  B Assigns objects to buckets

f(d) = d + 1 mod 5 Difficulty changing number of buckets bucket object f(d) = d + 1 mod 4

Consistent Hashing Idea: Map both objects and buckets to unit circle. object bucket Assign object to next bucket on circle in clockwise order. new bucket

Complication – Different Views select servers within cluster Low-level DNS servers act independently and may have different ideas about how many and which servers are alive. a212.g.akamai.net Akamai Low-Level DNS Servers

Properties of Consistent Hashing Monotonicity: When a bucket is added/removed, the only objects affected are those that are/were mapped to the bucket. Balance: Objects are assigned to buckets “randomly”. -- can be improved by mapping each bucket to multiple places on unit circle Load: Objects are assigned to buckets evenly, even over a set of views. Spread: An object should be mapped to a small number of buckets over a set of views.

How we really do it a212: a213: a214: a215: random permutations of servers Why? To spread load for one serial number.

8 BitTorrent Steps for publishing –Peer creates.torrent file and uploads to a web server: contains metadata about tracker and about the pieces of the file (checksum of each piece of the file). –Peers that create the initial copy of the file are called seeders Steps for downloading –Peer downloads.torrent file –Peer contacts tracker –Peer downloads from seeder, eventually from other peers Uses basic ideas from game theory to largely eliminate the free-rider problem –Previous systems could not deal with this problem

9 Basic Idea Chop file into many pieces Replicate DIFFERENT pieces on different peers as soon as possible As soon as a peer has a complete piece, it can trade it with other peers Hopefully, we will be able to assemble the entire file at the end

10 Basic Components Seed –Peer that has the entire file –Typically fragmented into 256KB pieces Leecher –Peer that has an incomplete copy of the file Torrent file –Passive component –The torrent file lists SHA1 hashes of all the pieces to allow peers to verify integrity –Typically hosted on a web server Tracker –Allows peers to find each other –Returns a random list of peers

11 Pieces and Sub-Pieces A piece is broken into sub-pieces... Typically 16KB; pipeline multiple requests for sub-pieces Policy: Until a piece is assembled, only download sub-pieces for that piece This policy lets complete pieces assemble quickly

12 Prisoner’s Dilemma Nash Equilibrium (and the dominant strategy for both players) Pareto Efficient Outcome

13 Repeated Games Repeated game: play single-shot game repeatedly Subgame Perfect Equilibrium: Analog to NE for repeated games –The strategy is an NE for every subgame of the repeated game Problem: a repeated game has many SPEs Single Period Deviation Principle (SPDP) can be used to test SPEs

14 Repeated Prisoner’s Dilemma Example SPE: Tit-for-Tat (TFT) strategy –Start by cooperating –Each player mimics the strategy of the other player in the last round –Forgive and forget quickly Question: Use the SPDP to argue that TFT is an SPE.

15 Tit-for-Tat in BitTorrent: Choking Choking is a temporary refusal to upload; downloading occurs as normal –If a node is unable to download from a peer, it does not upload to it –Ensures that nodes cooperate and eliminates the free-rider problem –Cooperation involves uploaded sub-pieces that you have to your peer Connection is kept open

16 Choking Algorithm Goal is to have several bidirectional connections running continuously Upload to peers who have uploaded to you recently Unutilized connections are uploaded to on a trial basis to see if better transfer rates could be found using them

17 Choking Specifics A peer always unchokes a fixed number of its peers (default of 4) Decision to choke/unchoke done based on current download rates, which is evaluated on a rolling 20- second average Evaluation on who to choke/unchoke is performed every 10 seconds –This prevents wastage of resources by rapidly choking/unchoking peers –Supposedly enough for TCP to ramp up transfers to their full capacity Which peer is the optimistic unchoke is rotated every 30 seconds

18 Rarest Piece First Policy: Determine the pieces that are most rare among your peers and download those first This ensures that the most common pieces are left till the end to download Rarest first also ensures that a large variety of pieces are downloaded from the seed (Question: Why is this important?)

19 Piece Selection The order in which pieces are selected by different peers is critical for good performance If a bad algorithm is used, we could end up in a situation where every peer has all the pieces that are currently available and none of the missing ones If the original seed is taken down, the file cannot be completely downloaded!

20 Random First Piece Initially, a peer has nothing to trade Important to get a complete piece ASAP Rare pieces are typically available at fewer peers, so downloading a rare piece initially is not a good idea (why?) Policy: Select a random piece of the file and download it

21 Endgame Mode When all the sub-pieces that a peer doesn’t have are actively being requested, these are requested from every peer Redundant requests cancelled when piece arrives Ensures that a single peer with a slow transfer rate doesn’t prevent the download from completing