Peer to Peer File Sharing Huseyin Ozgur TAN

What is Peer-to-Peer?  Every node is designed to(but may not by user choice) provide some service that helps other nodes in the network to get service  Each node potentially has the same responsibility  Sharing can be in different ways: CPU cycles: Storage space: Napster, Gnutella, Freenet…

P2P: Why so attractive?  Peer-to-peer applications fostered explosive growth in recent years. Low cost and high availability of large numbers of computing and storage resources, Increased network connectivity  As long as these issues keep their importance, peer-to-peer applications will continue to gain importance

Main Design Goals of P2P  Ability to operate in a dynamic environment  Performance and scalability  Reliability  Anonymity: Freenet, Freehaven, Publius  Accountability: Freehaven, Farsite

First generation P2P routing and location schemes  Napster, Gnutella, Freenet…  Intended for large scale sharing of data files  Reliable content location was not guaranteed  Self-organization and scalability: two issues to be addressed

Second generation P2P systems  Pastry, Tapestry, Chord, CAN…  They guarantee a definite answer to a query in a bounded number of network hops.  They form a self-organizing overlay network.  They provide a load balanced, fault- tolerant distributed hash table, in which items can be inserted and looked up in a bounded number of forwarding hops.

CAN  Content Addressable Network distributed infrastructure based on hash tables serves for Internet scale networks  Advantages Scalable Fault-tolerant Self orginazing

CAN  Current P2P systems at that time Napster Gnutella Both not scalable  Napster Central server Expensive (for server) Vulnerable (single point of failure)

CAN  Gnutella De-centralized file location as well File location is based on location Not scalable  Aim Building a scalable distributed infrastructure for p2p networks

CAN  Basic operations Insertion Lookup Deletion  Each node stores a zone of the entire hash table stores information about its neighbors

CAN - Design  Virtual d-dimensional Cartesian coordinate space on a d-torus  The entire space is partitioned among all nodes  The space is used to store key,value pairs K -> P -> zone  Each node holds info about each of its neigbors for efficient routing

CAN - Routing  Works by following the straight line path between source and destination  Local neighbor state is sufficient for routing CAN message includes dest. Coordinates A node routes it to the neighbor with closest to the destination coordinates  Complexity (d/4)(n 1/d ) avg path length 2d neighbors for a node Good for scalability  Many paths between source and destination In case of crashes the node routes the message to the next best available path

CAN - Construction  A new node that joins the system must be allocated its own zone Achieved by an existing node splitting its zone in half, retaining half and handing the other half to the new node  3 Steps New node must find a node already in CAN It must find a node whose zone will be split The neighbors of split zone must be notified

CAN - Construction  Bootstrap New node discovers an existing node CAN DNS name resolves to one or more nodes IP addresses which are believed to be in the system By querying the DNS new node gets a bootstrap node and from this node gets other nodes IP addresses  Finding a zone Chooses randomly a point P in space and sends JOIN request destined P Destination splits its zone and key,value pairs are transferred  Joining the routing The new node learns the IP addresses of the neighbors Both new and existing nodes update their neighbor set To inform others two nodes send an immediate update, followed by periodic refreshes  Complexity O(d)

CAN – Maintenance  Node departure Explicitly hand over its zone to one of its neighbors If node’s zones can be merged it is done If not, the responsibility is handed to the neighbor whose current zone is smallest  Takeover (a node becomes unreachable) One of its neighbors takes over the zone But the pairs are lost Nodes send periodic advertisements Absence of such adv. signals its failure  Each neighbor starts a timer  When it expires node sends TAKEOVER message to all neighbors of failed node

Chord  Distributed lookup protocol One operation : lookup = key -> node  Advantages Load balance Decentralization Scalability Availability Flexible naming  An infrastructure for p2p system

Chord - Overview  Specifies How to find locations of keys How new nodes join the system How to recover from failure of existing nodes  It uses consistent hashing  Improves the scalability of it by avoiding the requirement that every node know about every other node.  In N-node network each node maintains information about only O (log N) nodes a lookup requires O (log N) messages Updating info on join and leave requires O (log 2 N)

Chord – Consistent Hashing  Consistent hash function each node and key an m-bit identifier using a base hash function Node’s identifier = Hash of its IP Key’s identifier = Hash of key M must be big enough to make probability of collisions negligible  Assignment of keys to nodes Identifiers are ordered in an identifier circle Key k is assigned to the first node whose identifier is equal to or follows k in the identifier space This node is called successor (k)

Chord – Consistent Hashing  Scalable key location Very small amount of routing information suffices  Each node need only beware of its successor Routing can be done by following successors until the node is found  Not scalable: O(N) time lookup To accelerate Chord maintains additional routing information  Each node, n, maintains a routing table with at most m entries (finger table)  ith entry contains the identity of first node, s, that succeeds n by at least 2 i-1 on the identifier circle  i.e. s= successor(n+2 i-1 )

Chord

Chord – Node Joins  Main challenge is preserving the ability to locate every key in the network  2 invariants Each nodes successor is correctly maintained For every key k, node successor(k) is responsible for k  Also in order to be the lookup fast the finger tables must be correct

Chord – Node joins  To simplify the join and leave mechanism each node in Chord maintains a predecessor pointer  3 tasks Init predecessor and fingers of node n Update the finger and predecessors of existing nodes Notify higher layer so that it can transfer state associated with keys that node n is now responsible for

Chord – Node joins  Initializing fingers and predecessor Learns its predecessor and fingers by asking n’ to look them up  Updating fingers of existing nodes Node n will become the ith finger of node p iff  P precedes n by at least 2 i-1  The ith finger of p succeeds n The first node p that can meet these 2 conditions is the immediate predecessor of n- 2 i-1 The algorithm starts with the ith finger of node n and then continues to walk in the counter clockwise direction on identifier circle until it encounters a node whose ith finger precedes n  Transferring keys New node must contact the successor of it and transfer responsibility

Chord – Concurrent Operations  Stabilization The invariants are difficult to maintain in the face of concurrent joins in large networks Separate correctness and performance goals Stabilization protocol  Keep nodes’ successor pointers up to date, which is sufficient to correctness of lookups  Those successors are then used to verify and correct finger table entries

Stabilization

Chord