Download presentation

Presentation is loading. Please wait.

Published byLayne Edds Modified over 2 years ago

1
Dynamic Data Partitioning for Distributed Graph Databases Xavier Martínez Palau David Domínguez Sal Josep Lluís Larriba Pey

2
Dynamic Data Partitioning Outline Introduction Contributions System Overview Experiments 2

3
Dynamic Data Partitioning Outline Introduction Contributions System Overview Experiments 3

4
Dynamic Data Partitioning Introduction: Databases Database Software to store large amounts of data High performance Several ways to store a graph Graph database Relational database RDF Key-value datastore … 4

5
Dynamic Data Partitioning Distributed Databases 5 Distributed databases store more data and improve throughput

6
Dynamic Data Partitioning Outline Introduction Contributions System Overview Experiments 6

7
Dynamic Data Partitioning Contributions System design in two levels Physical storage Memory management Data access pattern monitoring Specific data structure Load and network balancing Increased throughput 7

8
Dynamic Data Partitioning Outline Introduction Contributions System Overview Experiments 8

9
Dynamic Data Partitioning System Overview 9 Memory managment Storage

10
Dynamic Data Partitioning Partition Manager We propose a new data structure Monitors data access patterns Uses this information in a simple way to decide how to route queries 10 Matrix of data access sequencesNew compressed data structure

11
Dynamic Data Partitioning Outline Introduction Contributions System Overview Experiments 11

12
Dynamic Data Partitioning Experiments Scalability with cluster size Tested up to 32 machines Systems compared Static partitioning Dynamic partitioning (ours) R-MAT graph 37M vertices 1B edges Queries: BFS and k-hops 12

13
Dynamic Data Partitioning Experiments Throughput (more better) Imbalance (less better)

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google