BitTorrent

Introduction to BitTorrent BT: BitTorrent BT is not itself a network it allows small Internet networks to be created to share files Does not perform all the functions of a typical p2p system, like searching Its virtual network is called data-oriented overlay Written by Bram Cohen in 2001 Written in Python and it uses GTK for its GUI It is an efficient content distribution system using file swarming Each file split into smaller pieces equal-sized blocks (typically 32- 256 KB) Nodes request desired pieces from neighbors Encourages contribution by all nodes The throughput increases with the number of downloaders via the efficient use of network bandwidth

About GTK GTK a cross-platform toolkit for creating graphical user interfaces GTK stands for GIMP ToolKit GIMP stands for GNU Image Manipulation Program It is an image retouching and editing tools It is open-source software It is developed by a self-organized group of volunteers under the banner of the GNOME Project It is licensed under the terms of the GNU LGPL GNU Lesser General Public License It allows developers and companies to use and integrate LGPL software into their own (even proprietary) software without being required to release the source code of their own software-parts GNU is a recursive acronym meaning "GNU's not Unix"

Terminology Leecher Initial seeder Leecher Seeder Seeder = a peer that provides the complete file Initial seeder = a peer that provides the file that is torrented Leecher Initial seeder One who is downloading Leecher Seeder

Terminology Peer: a client to the network dedicated to a torrent Seeding: serving a file for download Leeching: downloading without serving a complete file for download Leech: peer that’s downloading the file Fairer term might have been “downloader” Subpiece: Further subdivision of a piece The “unit for requests” is a subpiece But a peer uploads only after assembling complete piece

Leecher can become seeder As a leecher downloads pieces of the file, replicas of the pieces are created More downloads mean more replicas available As soon as a leecher has a complete piece, it can potentially share it with other downloaders Eventually each leecher becomes a seeder by obtaining all the pieces, assembles the file, and verifies the checksum

Swarm Swarm Set of peers all downloading the same file Organized as a random mesh BT is differentiated from the traditional file sharing as swarming File is divided into many small pieces for distribution Each node knows list of pieces downloaded by neighbors Node requests pieces it does not own from neighbors Clients request different pieces from the seeder or from other clients Clients become seeders for those pieces downloaded When all pieces are downloaded, clients can reconstruct the whole file There exists no single BT network, but thousands of temporary networks consisting of clients downloading the same file

Tracker The tracker is a central server keeping a list of all peers participating in the swarm A peer joins a swarm by asking the tracker for a peer list and connects to those peers The tracker gives the requester peers random selection of peers Get Request consists of: File ID Peer ID Peer IP Peer Port Tracker responses with: List of peers, containing ID, IP and Port of each peer Peers may rerequest on nonscheduled times, if they need more peers

How a node enters a swarm for file “popeye.mp4” The file distributor publishes details of the .torrent file on (well-known) web server File popeye.mp4.torrent hosted at the webserver The .torrent has address of tracker for file The tracker running on a webserver as well, keeps track of all peers downloading file Tracker supplies peers with addresses of other peers that share the wanted files

How a node enters a swarm for file “popeye.mp4” www.bittorrent.com URL of tracker .torrent file refers to the tracker which steers the download process This method makes it very clear who is responsible for the legitimacy of the content: the operator of the tracker server 1 Peer popeye.mp4.torrent

How a node enters a swarm for file “popeye.mp4” www.bittorrent.com File popeye.mp4.torrent hosted at a (well-known) webserver The .torrent has address of tracker for file The tracker, which runs on a webserver as well, keeps track of all peers downloading file 2 Peer Addresses of peers Tracker To download the file, peers access the tracker and join the torrent torrent is a group of peers connected to the same tracker The torrent is downloaded and the peer registers with the tracker, which provides a list of available peers and seeds

How a node enters a swarm for file “popeye.mp4” www.bittorrent.com Piece length – Usually 256 KB SHA-1 hashes of each piece in file For reliability Peer 3 Tracker Swarm

BT Client Software The BT client enables a host of features including multiple parallel downloads The BT client also intermediates peering between itself, source file servers (trackers) and other clients Thereby yielding great distribution efficiencies The BT client also enables users to create and share torrent files When a peer has finished downloading a file, it may become a seed by staying online for a while and sharing the file for free i.e., without bartering

Requirements from the Web Server and the Tracker The requirements from the Web hosting end are not too much To transmit a torrent you only need a standard HTTP Web server and a free program called a tracker The Web server should be configured to use MIME type application/x-bittorrent for any file with the ".torrent" extension The tracker's job is: to keep track of which clients can serve which files to other clients The tracker can be installed either on individual Web servers or operated centrally by the Web host Its traffic load is relatively light, and offering a tracker to your hosting customers can make using BT to distribute content a much simpler process for your customers

Overall Architecture Tracker Web Server .torrent C A [Seed] Peer Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker Web Server .torrent

Overall Architecture Tracker Web Server Get-announce C A [Seed] Peer Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker Get-announce Web Server

Overall Architecture Tracker Web Server Response-peer list C A [Seed] Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker Response-peer list Web Server

Overall Architecture Tracker Web Server Shake-hand C A [Seed] Peer Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker Shake-hand Web Server

Overall Architecture Tracker Web Server C A pieces [Seed] Peer [Leech] Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker pieces Web Server

Overall Architecture Tracker Web Server C A pieces [Seed] Peer [Leech] Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker pieces Web Server

Overall Architecture Tracker Web Server Get-announce Web page with link to .torrent A B C Peer [Leech] Downloader “US” [Seed] Tracker Get-announce Response-peer list pieces Web Server

Simple example {1,2,3,4,5,6,7,8,9,10} Seeder: A {} {1,2,3} {1,2,3,5} {1,2,3,4} {1,2,3,4,5} Downloader C Downloader B

Pipelining When transferring data over TCP, always have several requests pending at once (typically 5), to avoid a delay between pieces being sent Every time a piece or a sub-piece arrives, a new request is sent out

Piece Selection The order in which pieces are selected by different peers is critical for good performance If an inefficient policy is used, then peers may end up in a situation where each has all identical set of easily available pieces, and none of the missing ones. If the original seed is prematurely taken down, then the file cannot be completely downloaded

Piece Selection Small overlap is good Large overlap is bad -- wastes bandwidth

Piece selection Strict Priority Rarest First Random First Piece General rule Random First Piece Special case, at the beginning Endgame Mode Special case

Random First Piece Initially, a peer has nothing to trade Important to get a complete piece ASAP So as to assemble first complete piece quickly Then participate in uploads Select a random piece of the file and download it When first complete piece assembled, switch to rarest-first

Rarest Piece First Determine the pieces that are most rare among your peers, and download those first This ensures that the most commonly available pieces are left till the end to download Increases diversity in the pieces downloaded avoids case where a node and each of its peers have exactly the same pieces; increases throughput Increases likelihood all pieces still available even if original seed leaves before any one node has downloaded entire file

Endgame Mode Near the end, missing pieces are requested from every peer containing them This ensures that a download is not prevented from completion due to a single peer with a slow transfer rate This can potentially delay the finishing of a download When all the sub-pieces a peer lacks are requested this request is flooded to all peers This helps to get the last chunk of the file as quickly as possible To speed up completion of download Once a sub-piece arrives, it sends a cancel-message indicating that the peer has obtained it Some bandwidth is of course wasted by this flooding but not much because of the short period of the endgame mode

Difficulties with BitTorrent Works best with files that are widely copied on the network In practice, files appear and disappear There is no permanent archive, or incentive for users to keep old files Since blocks are not downloaded sequentially, a partial file is not useful

Reason of the success The success of Bittorent is unlikely to be due purely to the use of the tit-for-tat inspired protocol The real driving force behind the high cooperation might be the by-product of the lack of meta-data search within BitTorrent This results in the creation of a number of disconnected “tribes” at both the swarm and the tracker level The users are active in the tribal dynamics by selecting those tribes that best satisfy their needs hence tribes filled with free-riders will tend not to operate

An example The BitTorrent client BitTornado displays information about the peers and seeders engaged in sharing and distributing a torrent

Additional features BitTorrent uses 6881 as the default port if that port is unreachable BitTorrent tries to connect to a number of successive ports up to 6889 If the client cannot connect to port 6889, it gives up BitTorrent supports resuming, it resumes where it left off after checking the partial download How do the user know the download is not corrupted? BitTorrent does cryptographic hashing (SHA1) of all data When seeing "Download succeeded!" the user can be sure that BitTorrent has already verified the integrity of the data The integrity and authenticity of a BitTorrent download is as good as the original request to the tracker Checking the MD5/CRC32/other hash of a file downloaded via BitTorrent is redundant

An application of the BitTorrent Technology Cranberry Publishing uses Bittorrent as one of the means of distribution for its free Home Computer Magazine As the magazine is free to its readers, it is available simultaneously as a free download from the magazine website and as a torrent file for Bittorrent users However, the potential load on their server is enormous, so they wanted a way to ensure that it could be delivered Bittorrent offered the means to make sure the users can get the files they want distributed faster The best thing about it is that the more people that download it, the faster it gets for everyone, not slower Torrent users can just grab the torrent and download

Why is (studying) BitTorrent important? (From CacheLogic, 2004)

Legal Issues You should know about some inherent dangers to using BitTorrent to download movies and TV shows In the USA, organizations like the Recording Industry Association of America (RIAA) and the Motion Picture Association of America (MPAA) actively prosecute people and companies that are engaged in making copyrighted content available to others illegally Internationally, the laws are even more complex You are not anonymous when you use BitTorrent, because the process itself involves sharing of identifying information about your computer This lack of anonymity puts you at risk if you use BitTorrent or other file-sharing technologies to download or share music, movies, TV shows, and other content