1. CMPT 765/408: P2P Systems Instructor: Dr. Mohamed Hefeeda
School of Computing Science
Simon Fraser University
CMPT 765/408: P2P Systems
Instructor: Dr. Mohamed Hefeeda

2. P2P Computing: Definitions
Peers cooperate to achieve desired functions
Peers:
End-systems (typically, user machines)
Interconnected through an overlay network
Peer ≡ Like the others (similar or behave in similar manner)
Cooperate:
Share resources, e.g., data, CPU cycles, storage, bandwidth
Participate in protocols, e.g., routing, replication, …
Functions:
File-sharing, distributed computing, communications, content distribution, …
Note: the P2P concept is much wider than file sharing

3. When Did P2P Start? Napster (Late 1990’s) Gnutella (2000)
Court shut Napster down in 2001
Gnutella (2000)
Then the killer FastTrack (Kazaa, ...)
BitTorrent, and many others
Accompanied by significant research interest
Claim
P2P is much older than Napster!
Proof
The original Internet!
Remember UUCP (unix-to-unix copy)?

4. What IS and IS NOT New in P2P?
What is not new
Concepts!
What is new
The term P2P (may be!)
New characteristics of
Nodes which constitute the
System that we build

5. What IS NOT New in P2P? Distributed architectures
Distributed resource sharing
Node management (join/leave/fail)
Group communications
Distributed state management ….

6. What IS New in P2P? Nodes (Peers) Quite heterogeneous Unreliable
Several order of magnitudes difference in resources
Compare the bandwidth of a dial-up peer versus a high-speed LAN peer
Unreliable
Failure is the norm!
Offer limited capacity
Load sharing and balancing are critical
Autonomous
Rational, i.e., maximize their own benefits!
Motivations should be provided to peers to cooperate in a way that optimizes the system performance

7. What IS New in P2P? (cont’d)
System
Scale
Numerous number of peers (millions)
Structure and topology
Ad-hoc: No control over peer joining/leaving
Highly dynamic
Membership/participation
Typically open 
More security concerns
Trust, privacy, data integrity, …
Cost of building and running
Small fraction of same-scale centralized systems
How much would it cost to build/run a super computer with processing power of that 3 Million PCs?

8. What IS New in P2P? (cont’d)
So what?
We need to design new lighter-weight algorithms and protocols to scale to millions (or billions!) of nodes given the new characteristics
Question: why now, not two decades ago?
We did not have such abundant (and underutilized) computing resources back then!
And, network connectivity was very limited

9. Why is it Important to Study P2P?
P2P traffic is a major portion of Internet traffic (50+%), current killer app
P2P traffic has exceeded web traffic (former killer app)!
Direct implications on the design, administration, and use of computer networks and network resources
Think of ISP designers or campus network administrators
Many potential distributed applications

10. Sample P2P Applications
File sharing
Gnutella, Kazaa, BitTorrent, …
Distributed cycle sharing …
File and storage systems
OceanStore, CFS, Freenet, Farsite, …
Media streaming and content distribution
PROMISE
SplitStream, CoopNet, PeerCast, Bullet, Zigzag, NICE, …

11. P2P vs. its Cousin (Grid Computing)
Common Goal:
Aggregate resources (e.g., storage, CPU cycles, and data) into a common pool and provide efficient access to them
Differences along five axes [Foster & Imanitchi 03]
Target communities and applications
Type of shared resources
Scalability of the system
Services provided
Software required

12. P2P vs Grid Computing (cont’d)
Issue
Grid
P2P
Communities and Applications
Established communities, e.g., scientific institutions
Computationally-intensive problems
Grass-root communities (anonymous)
Mostly, file-swapping
Resources Shared
Powerful and Reliable machines, clusters
High-speed connectivity
Specialized instruments
PCs with limited capacity and connectivity
Unreliable
Very diverse

13. P2P vs Grid Computing (cont’d)
Issue
Grid
P2P
System Scalability
Hundreds to thousands of nodes
Hundreds of thousands to Millions of nodes
Services Provided
Sophisticated services: authentication, resource discovery, scheduling, access control, and membership control
Members usually trust others
Limited services: resource discovery
limited trust among peers
Software required
Sophisticated suit: e.g., Globus, Condor
Simple: (screen saver), e.g., Kazza,

14. P2P vs Grid Computing: Discussion
The differences mentioned are based on the traditional view of each paradigm
It is conceived that both paradigms will converge and will complement each other [e.g., Butt et al. 03]
Target communities and applications
Grid: is going open
Type of shared resources
P2P: is to include various and more powerful resources
Scalability of the system
Grid: is to increase number of nodes
Services provided
P2P: is to provide authentication, data integrity, trust management, …

15. P2P Systems: Simple Model
P2P Substrate
Operating System
Hardware
Middleware
P2P Application
Software architecture model on a peer
System architecture: Peers form an overlay according to the P2P Substrate

16. Overlay Network An abstract layer built on top of physical network
Neighbors in overlay can be several hops away in physical network
Why do we need overlays?
Flexibility in
Choosing neighbors
Forming and customizing topology to fit application’s needs (e.g., short delay, reliability, high BW, …)
Designing communication protocols among nodes
Get around limitations in legacy networks
Enable new (and old!) network services

17. Overlay Network (cont’d)

18. Overlay Network (cont’d)
Some applications that use overlays
Application level multicast, e.g., ESM, Zigzag, NICE, …
Reliable inter-domain routing, e.g., RON
Content Distribution Networks (CDN)
Peer-to-peer file sharing
Overlay design issues
Select neighbors
Handle node arrivals, departures
Detect and handle failures (nodes, links)
Monitor and adapt to network dynamics
Match with the underlying physical network

19. Overlay Network (cont’d) Recall: IP Multicast
source

20. Overlay Network (cont’d) Application Level Multicast (ALM)
source

21. Peer Software Model A software client installed on each peer
Three components:
P2P Substrate
Middleware
P2P Application
P2P Substrate
Operating System
Hardware
Middleware
P2P Application
Software model on peer

22. Peer Software Model (cont’d)
P2P Substrate (key component)
Overlay management
Construction
Maintenance (peer join/leave/fail and network dynamics)
Resource management
Allocation (storage)
Discovery (routing and lookup)
Ex: Pastry, CAN, Chord, …
More on this later

23. Peer Software Model (cont’d)
Middleware
Provides auxiliary services to P2P applications:
Peer selection
Trust management
Data integrity validation
Authentication and authorization
Membership management
Accounting (Economics and rationality) …
Ex: CollectCast, EigenTrust, Micro payment

24. Peer Software Model (cont’d)
P2P Application
Potentially, there could be multiple applications running on top of a single P2P substrate
Applications include
File sharing
File and storage systems
Distributed cycle sharing
Content distribution
This layer provides some functions and bookkeeping relevant to target application
File assembly (file sharing)
Buffering and rate smoothing (streaming)
Ex: Promise, Bullet, CFS

25. P2P Substrate Key component, which P2P Substrates can be
Manages the Overlay
Allocates and discovers objects
P2P Substrates can be
Structured
Unstructured
Based on the flexibility of placing objects at peers

26. P2P Substrates: Classification
Structured (or tightly controlled, DHT)
Objects are rigidly assigned to specific peers
Looks like as a Distributed Hash Table (DHT)
Efficient search & guarantee of finding
Lack of partial name and keyword queries
Maintenance overhead
Ex: Chord, CAN, Pastry, Tapestry, Kademila (Overnet)
Unstructured (or loosely controlled)
Objects can be anywhere
Support partial name and keyword queries
Inefficient search & no guarantee of finding
Some heuristics exist to enhance performance
Ex: Gnutella, Kazaa (super node), GIA [Chawathe et al. 03]

27. Structured P2P Substrates
Objects are rigidly assigned to peers
Objects and peers have IDs (usually by hashing some attributes)
Objects are assigned to peers based on IDs
Peers in overlay form specific geometrical shape, e.g.,
tree, ring, hypercube, butterfly network
Shape (to some extent) determines
How neighbors are chosen, and
How messages are routed

28. Structured P2P Substrates (cont’d)
Substrate provides a Distributed Hash Table (DHT)-like interface
InsertObject (key, value), findObject (key), …
In the literature, many authors refer to structured P2P substrates as DHTs
It also provides peer management (join, leave, fail) operations
Most of these operations are done in O(log n) steps, n is number of peers

29. Structured P2P Substrates (cont’d)
DHTs: Efficient search & guarantee of finding
However,
Lack of partial name and keyword queries
Maintenance overhead, even O(log n) may be too much in very dynamic environments
Ex: Chord, CAN, Pastry, Tapestry, Kademila (Overnet)

30. Example: Content Addressable Network (CAN) [Ratnasamy 01]
Nodes form an overlay in d-dimensional space
Node IDs are chosen randomly from the d-space
Object IDs (keys) are chosen from the same d-space
Space is dynamically partitioned into zones
Each node owns a zone
Zones are split and merged as nodes join and leave
Each node stores
The portion of the hash table that belongs to its zone
Information about its immediate neighbors in the d-space

31. 2-d CAN: Dynamic Space Division7 n2 n1 n4 n3 n5 7

32. 2-d CAN: Key Assignment n1 n2 n3 n4 7 K1 K2 K3 K4 n5

33. 2-d CAN: Routing (Lookup)7 K1 K2
K4?
K4? K4 K3 n5

34. CAN: Routing
Nodes keep 2d = O(d) state information (neighbor coordinates, IPs)
Constant, does not depend on number of nodes n
Greedy routing
Route to the node that is closest to the destination
On average, is done in O(n1/d) = O(log n) when d = log n /2

35. CAN: Node Join New node finds a node already in the CAN
(bootstrap: one (or a few) dedicated nodes outside the CAN maintain a partial list of active nodes)
It finds a node whose zone will be split
Choose a random point P (will be its ID)
Forward a JOIN request to P through the existing node
The node that owns P splits its zone and sends half of its routing table to the new node
Neighbors of the split zone are notified

36. CAN: Node Leave, Fail Graceful departure Failure
The leaving node hands over its zone to one of its neighbors
Failure
Detected by the absence of heart beat messages sent periodically in regular operation
Neighbors initiate takeover timers, proportional to the volume of their zones
Neighbor with smallest timer takes over zone of dead node
notifies other neighbors so they cancel their timers (some negotiation between neighbors may occur)
Note: the (key, value) entries stored at the failed node are lost
Nodes that insert (key, value) pairs periodically refresh (or re-insert) them

37. CAN: Discussion Scalable Locality Maintenance cost
O(log n) steps for operations
State information is O(d) at each node
Locality
Nodes are neighbors in the overlay, not in the physical network
Suggestion (for better routing)
Each node measure RTT between itself and its neighbors
Forward the request to the neighbor with maximum ratio of progress to RTT
Maintenance cost
Logarithmic
But, may still be too much for very dynamic P2P systems

38. Unstructured P2P Substrates
Objects can be anywhere  Loosely-controlled overlays
The loose control
Makes overlay tolerate transient behavior of nodes
For example, when a peer leaves, nothing needs to be done because there is no structure to restore
Enables system to support flexible search queries
Queries are sent in plain text and every node runs a mini-database engine
But, we loose on searching
Usually using flooding, inefficient
Some heuristics exist to enhance performance
No guarantee on locating a requested object (e.g., rarely requested objects)
Ex: Gnutella, Kazaa (super node), GIA [Chawathe et al. 03]

39. Example: Gnutella Peers are called servents
All peers form an unstructured overlay
Peer join
Find an active peer already in Gnutella (e.g., contact known Gnutella hosts)
Send a Ping message through the active peer
Peers willing to accept new neighbors reply with Pong
Peer leave, fail
Just drop out of the network!
To search for a file
Send a Query message to all neighbors with a TTL (=7)
Upon receiving a Query message
Check local database and reply with a QueryHit to requester
Decrement TTL and forward to all neighbors if nonzero

40. Flooding in Gnutella
Scalability Problem

41. Heuristics for Searching [Yang and Garcia-Molina 02]
Iterative deepening
Multiple BFS with increasing TTLs
Reduce traffic but increase response time
Directed BFS
Send to “good” neighbors (subset of your neighbors that returned many results in the past)  need to keep history
Local Indices
Keep a small index over files stored on neighbors (within number of hops)
May answer queries on behalf of them
Save cost of sending queries over the network
Index currency?

42. Heuristics for Searching: Super Node
Used in Kazaa (signaling protocols are encrypted)
Studied in [Chawathe 03]
Relatively powerful nodes play special role
maintain indexes over other peers

43. Unstructured Substrates with Super Nodes
Super Node (SN)
Ordinary Node (ON)

44. Example: FastTrack Networks (Kazaa)
Most of info/plots in following slides are from Understanding Kazaa by Liang et al.
The most popular (~ 3 million active users in a typical day) sharing 5,000 Terabytes
Kazaa traffic exceeds Web traffic
Two-tier architecture (with Super Nodes and Ordinary Nodes)
SN maintains index on files stored at ONs attached to it
ON reports to SN the following metadata on each file:
File name, file size, ContentHash, file descriptors (artist name, album name, …)

45. FastTrack Networks (cont’d)
Mainly two types of traffic
Signaling
Handshaking, connection establishment, uploading metadata, …
Encrypted! (some reverse engineering efforts)
Over TCP connections between SN—SN and SN—ON
Analyzed in [Liang et al. 04]
Content traffic
Files exchanged, not encrypted
All through HTTP between ON—ON
Detailed Analysis in [Gummadi et al. 03]

46. Kazaa (cont’d) File search
ON sends a query to its SN
SN replies with a list of IPs of ONs that have the file
SN may forward the query to other SNs
Parallel downloads take place between supplying ONs and receiving ON

47. FastTrack Networks (cont’d)
Measurement study of Liang et al.
Hook three machines to Kazaa and wait till one of them is promoted to be SN
Connect the other two (ONs) to that SN
Study several properties
Topology structure and dynamics
Neighbor selection
Super node lifetime ….

48. Kazaa: Topology Structure [Liang et al. 04]
ON to SN: connections  Since there are ~3M nodes, we have ~30,000 SNs
SN to SN: 30 – 50 connections  Each SN connects to ~0.1 % of total number of SNs

49. Kazaa: Topology Dynamics [Liang et al. 04]
Average ON – SN connection duration
Is ~ 1 hour, after removing very short-lived connections (30 sec) used for shopping for SNs
Average SN – SN connection duration
23 min, which is short because of
Connection shuffling between SNs to allow ONs to reach a larger set of objects
SNs search for other SNs with smaller loads
SNs connect to each other from time to time to exchange SN lists (each SN stores 200 other SNs in its cache)

50. Kazaa: Neighbor Selection [Liang et al. 04]
When ON first joins, it gets a list of 200 SNs
ON considers locality and SN workload in selecting its future SN
Locality
40% of ON-SN connections have RTT < 5 msec
60% of ON-SN connections have RTT < 50 msec
RTT: E. US  Europe ~100 msec

51. Kazaa: Lifetime and Signaling Overhead [Liang et al. 04]
Super node average lifetime is ~2.5 hours
Overhead:
161 Kb/s upstream
191 Kb/s downstream
 Most of SNs are high-speed (campus network, or cable)

52. Kazaa vs. Firewalls, NAT [Liang et al. 04]
Default port WAS 1214
Easy for firewalls to filter out Kazaa traffic
Now, Kazaa uses dynamic ports
Each peer chooses its random port
ON reports its port to its SN
Ports of SNs are part of the SN refresh list exchanged among peers
Too bad for firewalls!
Network Address Translator (NAT)
A requesting peer can not establish a direct connection with a serving peer behind NAT
Solution: connection reversal
Send to SN of NATed peer, which already has a connection with it
SN tells NATed peer to establish a connection with requesting peer!
Transfer occurs happily through the NAT
Both peers behind NATs?

53. Kazaa: Lessons [Liang et al. 04]
Distributed design
Exploit heterogeneity
Load balancing
Locality in neighbor selection
Connection Shuffling
If a peer searches for a file and does not find it, it may try later and gets it!
Efficient gossiping algorithms
To learn about other SNs and perform shuffling
Kazaa uses a “freshness” field in SN refresh list  a peer ignores stale data
Consider peers behind NATs and Firewalls
They are everywhere!

54. Summary
P2P is an active research area with many potential applications in industry and academia
In P2P computing paradigm:
Peers cooperate to achieve desired functions
New characteristics
heterogeneity, unreliability, rationality, scale, ad hoc
 new and lighter-weight algorithms are needed
Simple model for P2P systems:
Peers form an abstract layer called overlay
A peer software client may have three components
P2P substrate, middleware, and P2P application
Borders between components may be blurred

55. Summary (cont’d) P2P substrate: A key component, which
Manages the Overlay
Allocates and discovers objects
P2P Substrates can be
Structured (DHT)
Example: CAN
Unstructured
Example 1: Gnutella,
Example 2: Kazza