1. Data Management in Large-scale P2P Systems
Patrick Valduriez, Esther Pacitti
Atlas group, INRIA and LINA
University of Nantes, France

2. Motivations P2P systems Distributed database systems
Decentralized control, large scale
Low-level, simple services
File sharing, computation sharing, com. sharing
Distributed database systems
High-level data management services
queries, transactions, consistency, security, etc.
Centralized control, limited scale
P2P + distributed database
Why? How?

3. Why high-level P2P data sharing?
Professional community example
Medical doctors in a hospital may want to share (some of) their patient data for an epidemiological study
They have their own, independent patient descriptions
They want to ask queries such as “age and weight of male patients diagnosed with disease X …” over their own descriptions
They don’t want to create a database and buy a server

4. Problem definition P2P system
No centralized control, very large scale
Very dynamic: peers can join and leave the network at any time
Peers can be autonomous and unreliable
Techniques designed for distributed data management no longer apply
Too static, need to be decentralized, dynamic and self-adaptive

5. Outline Data management in distributed systems P2P systems
Data management in P2P systems
Data management in APPA

6. Data management basic principle
Data independence
Hide implementation details
Provision for high-level services
Schema
Queries (SQL, XQuery)
Automatic optimization
Transactions
Consistency
Access control …
Application
Application
Logical view
(schema)
Storage
Storage

7. Distributed database system (DDBS)
Distribution transparency
Global schema
Common data descriptions
Distributed data placement
Centralized control through global catalog
Distributed functions
Schema mapping
Query processing
Transaction management
Access control
Etc.
Queries, Transactions
Site 1
Distributed
Database
System
Site 2
Site 3
DBMS1
DBMS2

8. Scaling up DDBS Distributed database systems Data integration systems
Enterprise information systems
Scale up to tens of databases
Data integration systems
strong heterogeneity and autonomy of data sources (files, databases, XML documents, ..)
Limited functionality (queries)
Scale up to hundreds of data sources
Parallel database systems
Focus on high-performance and high-availability
Strong homogeneity
Scale up to hundreds of data nodes

9. A generic P2P system
A user at a peer may access sharable data at remote peers
P2P software
private
sharable
P2P software
private
sharable
P2P software
private
sharable

10. Potential benefits of P2P systems
Scale up to very large numbers of peers
Dynamic self-organization
Load balancing
Parallel processing
High availability through massive replication

11. P2P vs DDBS P2P DDBS Joining the network Upon peer’s initiative
Controled by DBA
Queries
No schema,
key-word based
Global schema, static optimization
Query answers
Partial
Complete
Content location
Using neighbors
or DHT
Using directory

12. Requirements for P2P data management (1)
Autonomy of peers
Peers should be able to join/leave at any time, control their data wrt other (trusted) peers
Query expressiveness
Key-lookup, key-word search, SQL-like
Efficiency
Efficient use of bandwidth, computing power, storage

13. Requirements for P2P data management (2)
Quality of service (QoS)
User-perceived efficiency: completeness of results, response time, data consistency, …
Fault-tolerance
Efficiency and QoS despite failures
Security
Data access control in the context of very open systems

14. P2P network topologies Unstructured systems Structured (DHT) systems
e.g.
Structured (DHT) systems
e.g. CAN, CHORD
Super-peer (hybrid) systems
e.g. Napster

15. P2P unstructured network
data
p2p
data
p2p
data
peer 4
p2p
data
peer 1
peer 2
peer 3
High autonomy (peer needs to know neighbor to login)
Searching by flooding the network
general, inefficient
High-fault tolerance with replication

16. P2P structured network Efficient exact-match search
Distributed Hash Table (DHT)
h(k1)= p1
h(k2)= p2
h(k3)= p3
h(k4)= p4
p2p
d(k1)
p2p
d(k2)
p2p
d(k3)
p2p
d(k4)
peer 1
peer 2
peer 3
peer 4
Efficient exact-match search
O(log n) for put(key,value), get(key)
Limited autonomy since a peer is responsible for a range of keys

17. Super-peer network
sp2sp
sp2p
sp2sp
sp2p
p2sp
data
p2sp
data
p2sp
data
p2sp
data
peer 1
peer 2
peer 3
peer 4
Super-peers can perform complex functions (meta-data management, indexing, acces control, etc.)
Efficiency and QoS
Restricted autonomy
SP = single point of failure => use several

18. P2P systems comparison Requirements Unstructured DHT Super-peer
Autonomy
high
low
avg
Query exp.
Efficiency
QoS
Fault-tolerance
Security

19. Data management in P2P systems
Current research focuses on
Decentralized schema mappings
PeerDB: unstruct. network, keyword search only
Extending DHT for complex querying
PIER : exact-match and join queries
Query reformulation
Edutella: super-peer, RDF-based schemas
Piazza: graph of pair-wise schema mappings
Replication
generally limited to static read-only files
P-Grid addresses updates in structured networks

20. Data management in APPA (Atlas P2P Architecture)
Objectives
Scalability, availability and performance
Main features
Network-independent architecture
Layered, service-based architecture
Replication with semantics-based reconciliation
Decentralized schema management
Schema-based query support and optimization
Peer data caching
Prototype on JXTA
Network-independent P2P services

21. Network independent APPA
Advanced Services
Query Processing
Replication
Cache Management
Security
Basic Services
Group Membership Management
Consensus Management
P2P Data Management
Peer Management
Peer Communication
P2P Network
Key-based Storage and Retrieval
Peer ID Assignment
Peer Linking
Internet
...

22. Different APPA architectures
Peer
Advanced
services
Basic
P2P
network
DHT
data
local
Basic services
P2P network
Super-peer
Peer
P2P
data
local
Advanced
services

23. Schema management in APPA
Takes advantage of the collaborative nature of the applications
Peers that wish to cooperate agree on a Common Schema Description (CSD)
Given 2 CSD relation definitions, an example of peer mapping at peer p is:
p:r(A,B,D) csd:r1(A,B,C), csd:r2(C,D,E)
Peer mappings stored as P2P data

24. Replication in APPA
Small-world assumption: peers work in smaller groups with time locality
Lazy multi-master replication
n peers can update the same replica
Improves read performance and availability
Replica divergence solved by distributed log-based reconciliation
Exploit P2P data management service

25. Query processing in APPA
Given a SQL-like query on peer schema, performs
query reformulation
Maps the query on CSD schemas
query matching
Finds relevant peers
query optimization
Selects best peers, taking replication into account
query decomposition and execution
Exploits parallelism

26. Conclusion
Advanced P2P applications will need high-level data management services
Various P2P networks will improve
Network-independence crucial to exploit and combine them
Many technical issues
Important to characterize applications that can most benefit from P2P wrt other distributed architectures