CSCI5570 Large Scale Data Processing Systems

Slides:



Advertisements
Similar presentations
GraphChi: Large-Scale Graph Computation on Just a PC
Advertisements

Oracle Labs Graph Analytics Research Hassan Chafi Sr. Research Manager Oracle Labs Graph-TA 2/21/2014.
© Copyright 2012 STI INNSBRUCK Apache Lucene Ioan Toma based on slides from Aaron Bannert
Parallel Computing MapReduce Examples Parallel Efficiency Assignment
Distributed Graph Analytics Imranul Hoque CS525 Spring 2013.
GraphChi: Big Data – small machine
Disk Access Model. Using Secondary Storage Effectively In most studies of algorithms, one assumes the “RAM model”: –Data is in main memory, –Access to.
Design Patterns for Efficient Graph Algorithms in MapReduce Jimmy Lin and Michael Schatz University of Maryland Tuesday, June 29, 2010 This work is licensed.
Using Secondary Storage Effectively In most studies of algorithms, one assumes the "RAM model“: –The data is in main memory, –Access to any item of data.
SM3121 Software Technology Mark Green School of Creative Media.
A Lightweight Infrastructure for Graph Analytics Donald Nguyen Andrew Lenharth and Keshav Pingali The University of Texas at Austin.
Large-scale Recommender Systems on Just a PC LSRS 2013 keynote (RecSys ’13 Hong Kong) Aapo Kyrölä Ph.D. CMU
SYSTEMS SUPPORT FOR GRAPHICAL LEARNING Ken Birman 1 CS6410 Fall /18/2014.
HADOOP ADMIN: Session -2
BiGraph BiGraph: Bipartite-oriented Distributed Graph Partitioning for Big Learning Jiaxin Shi Rong Chen, Jiaxin Shi, Binyu Zang, Haibing Guan Institute.
Pregel: A System for Large-Scale Graph Processing
Research on cloud computing application in the peer-to-peer based video-on-demand systems Speaker : 吳靖緯 MA0G rd International Workshop.
Exercise problems for students taking the Programming Parallel Computers course. Janusz Kowalik Piotr Arlukowicz Tadeusz Puzniakowski Informatics Institute.
Carnegie Mellon University GraphLab Tutorial Yucheng Low.
SYSTEMS SUPPORT FOR GRAPHICAL LEARNING Ken Birman 1 CS6410 Fall /18/2014.
HBase A column-centered database 1. Overview An Apache project Influenced by Google’s BigTable Built on Hadoop ▫A distributed file system ▫Supports Map-Reduce.
Stochastic Algorithms Some of the fastest known algorithms for certain tasks rely on chance Stochastic/Randomized Algorithms Two common variations – Monte.
Ligra: A Lightweight Graph Processing Framework for Shared Memory
X-Stream: Edge-Centric Graph Processing using Streaming Partitions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. LogKV: Exploiting Key-Value.
CSE 486/586 CSE 486/586 Distributed Systems Graph Processing Steve Ko Computer Sciences and Engineering University at Buffalo.
Carnegie Mellon Yucheng Low Aapo Kyrola Danny Bickson A Framework for Machine Learning and Data Mining in the Cloud Joseph Gonzalez Carlos Guestrin Joe.
IT253: Computer Organization
Intermediate 2 Software Development Process. Software You should already know that any computer system is made up of hardware and software. The term hardware.
Carnegie Mellon Yucheng Low Aapo Kyrola Danny Bickson A Framework for Machine Learning and Data Mining in the Cloud Joseph Gonzalez Carlos Guestrin Joe.
Big data Usman Roshan CS 675. Big data Typically refers to datasets with very large number of instances (rows) as opposed to attributes (columns). Data.
Matrix Multiplication in Hadoop
BIG DATA/ Hadoop Interview Questions.
Hadoop Javad Azimi May What is Hadoop? Software platform that lets one easily write and run applications that process vast amounts of data. It includes:
MASS Java Documentation, Verification, and Testing
Recommendation in Scholarly Big Data
Big Data is a Big Deal!.
Memory Management.
University of British Columbia
Hadoop.
Sathya Ronak Alisha Zach Devin Josh
Lesson Objectives Aims You should be able to:
Distributed Programming in “Big Data” Systems Pramod Bhatotia wp
Pagerank and Betweenness centrality on Big Taxi Trajectory Graph
CSCI5570 Large Scale Data Processing Systems
CLUSTER COMPUTING Presented By, Navaneeth.C.Mouly 1AY05IS037
Advanced Science and Technology Letters Vol.30 (ICCA 2013), pp
Genomic Data Clustering on FPGAs for Compression
Real-Time Ray Tracing Stefan Popov.
Hadoop Clusters Tess Fulkerson.
CSCI5570 Large Scale Data Processing Systems
Selectivity Estimation of Big Spatial Data
April 30th – Scheduling / parallel
CSCI1600: Embedded and Real Time Software
湖南大学-信息科学与工程学院-计算机与科学系
COS 518: Advanced Computer Systems Lecture 12 Mike Freedman
Communication and Memory Efficient Parallel Decision Tree Construction
Apache Spark & Complex Network
On the effect of randomness on planted 3-coloring models
CSE451 Virtual Memory Paging Autumn 2002
Big Data, Bigger Data & Big R Data
Cloud Computing MapReduce in Heterogeneous Environments
LO2 – Understand Computer Software
Wednesday, 5/8/2002 Hash table indexes, physical operators
Minwise Hashing and Efficient Search
What is Computer Science About? Part 2: Algorithms
Computational Advertising and
CMPT 733, SPRING 2017 Jiannan Wang
CSCI1600: Embedded and Real Time Software
CSE 326: Data Structures Lecture #14
Presentation transcript:

CSCI5570 Large Scale Data Processing Systems Graph Processing Systems James Cheng CSE, CUHK Slide Ack.: modified based on the slides from Aapo Kyrola

GraphChi: Large-Scale Graph Computation on Just a PC Aapo Kyrola, Guy Blelloch, Carlos Guestrin OSDI 2012

GraphChi can compute on the full Twitter follow-graph with just a standard laptop. … that I can say that you can use a basic Mac laptop to compute on the actual, whole Twitter graph. This is data that is not normally available in the academic research, and I am really proud of this. It it hard to find results from the literature that would have done experiments on anything this big. And this is just on a laptop. Thanks Pankaj for the opportunity, and a especially appreciate Twitter’s commitment to open source, and I have been able to use and release the code I did while at Twitter without trouble. Ok – I said you can compute on the Twitter graph, but how fast? Roughly speaking, as fast as you can do on a huge Hadoop cluster. Goes without saying that energy wise or cost wise GraphChi is incredibly efficient. Response to Pankaj GraphChi can compute on the actual Twitter graph, on a Macbook Pro (Fall 2012) Cite Pankaj’s paper Roughly same performance as Twitter’s hadoop cluster Think about energy consumption, costs ~ as fast as a very large Hadoop cluster! (size of the graph Fall 2013, > 20B edges [Gupta et al 2013])

2 What is GraphChi Both in OSDI’12! So as a recap, GraphChi is a disk-based GraphLab. While GraphLab2 is incredibly powerful on big clusters, or in the cloud, you can use GraphChi to solve as big problems on just a Mac Mini. Of course, GraphLab can solve the problems way faster – but I believe GraphChi provides performance that is more then enough for many. Spin-off of GraphLab project Disk based GraphLab OSDI’12 Both in OSDI’12!

Parallel Sliding Windows Shard 1 Shard 2 Shard 3 1 src dst value 1 2 0.3 3 0.2 4 1.4 5 0.5 0.6 6 0.8 src dst value 1 3 0.4 2 0.3 4 0.8 5 0.2 6 1.9 src dst value 2 5 0.6 3 0.9 6 1.2 4 0.3 1.1 6 2 3 5 4

Parallel Sliding Windows Shard 1 Shard 2 Shard 3 1 src dst value 1 2 0.27 3 0.22 4 1.54 5 0.55 0.66 6 0.88 src dst value 1 3 0.36 2 0.27 4 0.8 5 0.2 6 1.9 src dst value 2 5 0.55 3 0.9 6 1.2 4 0.3 1.1 6 2 3 5 4

Parallel Sliding Windows Shard 1 Shard 2 Shard 3 1 src dst value 1 2 0.27 3 0.22 4 1.54 5 0.55 0.66 6 0.88 src dst value 1 3 0.36 2 0.27 4 0.8 5 0.2 6 1.9 src dst value 2 5 0.55 3 0.9 6 1.2 4 0.3 1.1 6 2 3 5 4

Parallel Sliding Windows Interval 1 Interval 2 Shard 1 Shard 2 Shard 3 Shard 4 Shard 1 Shard 2 Shard 3 Shard 4 Interval 3 Interval 4 Shard 1 Shard 2 Shard 3 Shard 4 Shard 1 Shard 2 Shard 3 Shard 4

Parallel Sliding Windows Details: Kyrola, Blelloch, Guestrin: “Large-scale graph computation on just a PC” (OSDI 2012) Parallel Sliding Windows or So how does GraphChi work? I don’t have time to go to details now. It is based on an algorithm we invented called Parallel Sliding Windows. In this model you split the graph in to P shards, and the graph is processed in P parts. For each part you load one shard completely in to memory, and load continuous chunks of data from the other shards. All in all, you need very small number of random accesses, which are the bottleneck of disk based computing. GraphChi is good on both SSD and hard drive! Only P large reads for each interval (sub-graph). P2 reads on one full pass.

GraphLab/Pregel style programming + with hacking + more Why GraphChi Performance Great scalability GraphLab/Pregel style programming + with hacking + more Applications Easy! I am now going into details of the features of GraphChi, which are the selling points fo GraphChi. So I am going to talk about the Performance, how it is – perhaps surprising – a really scalable system, then just to mention that you program GraphChi with the familiar vertex centric model of GraphLab or Pregel but GraphLab provides some extensions like support for dynamic graphs.

Performance Comparison See the paper for more comparisons. Performance Comparison PageRank WebGraph Belief Propagation (U Kang et al.) On a Mac Mini: GraphChi can solve as big problems as existing large-scale systems. Comparable performance. Matrix Factorization (Alt. Least Sqr.) Triangle Counting Unfortunately the literature is abundant with Pagerank experiments, but not much more. Pagerank is really not that interesting, and quite simple solutions work. Nevertheless, we get some idea. Pegasus is a hadoop-based graph mining system, and it has been used to implement a wide range of different algorithms. The best comparable result we got was for a machine learning algo “belief propagation”. Mac Mini can roughly match a 100node cluster of Pegasus. This also highlights the inefficiency of MapReduce. That said, the Hadoop ecosystem is pretty solid, and people choose it for the simplicity. Matrix factorization has been one of the core Graphlab applications, and here we show that our performance is pretty good compared to GraphLab running on a slightly older 8-core server. Last, triangle counting, which is a heavy-duty social network analysis algorithm. A paper in VLDB couple of years ago introduced a Hadoop algorithm for counting triangles. This comparison is a bit stunning. But, I remind that these are prior to PowerGraph: in OSDI, the map changed totally! However, we are confident in saying, that GraphChi is fast enough for many purposes. And indeed, it can solve as big problems as the other systems have been shown to execute. It is limited by the disk space. Notes: comparison results do not include time to transfer the data to cluster, preprocessing, or the time to load the graph from disk. GraphChi computes asynchronously, while all but GraphLab synchronously.

Scalability / Input Size [SSD] Throughput: number of edges processed / second. Conclusion: the throughput remains roughly constant when graph size is increased Performance  Here, in this plot, the x-axis is the size of the graph as the number of edges. All the experiment graphs are presented here. On the y-axis, we have the performance: how many edges processed by second. Now the dots present different experiments (averaged), and the read line is a least-squares fit. On SSD, the throughput remains very closely constant when the graph size increases. Note that the structure of the graph actually has an effect on performance, but only by a factor of two. The largest graph, yahoo-web, has a challenging structure, and thus its results are comparatively worse. No worries of running out of memory, or buying more machines when your data grows. Graph size 

GraphChi^2 Distributed Graph System Single-computer system (capable of big tasks) Task 7 Task 6 Task 5 Task 4 Task 3 Task 2 Task 1 Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 6 machines T11 T10 T9 T8 T7 T6 T5 T4 T3 T2 T1 (Significantly) less than 2x throughput with 2x machines Task 1 Exactly 2x throughput with 2x machines Task 2 The fact that GraphChi can scale to very big problems makes it surprisingly an interesting candidate for massive production systems. This is true in cases when you can sacrifice latency for throughput, and you have many problems that you run on the same graph, data. For example, Twitter computes recommendations for each user personally – and they are all on same graph. So let’s say you need to compute millions of new recommendations a day – but you don’t need to compute them in a few seconds. Then you could have a choice between a distributed efficient graph system – which needs many machines just to solve this one problem, and of using GraphChi to run one task a time. Note that one task can mean computing recommendations for thousands or even million of users a time – I will describe such a setting later today. This is a made-up example to illustrate a point. Here we have chosen T to be the time the single machine system, such as GraphChi, solves the one task. Let’s assume the cluster system needs 6 machines to solve the problem, and does it about 7 times faster than GraphChi. Then in Time T it solves 7 tasks while GraphChi solves 6 tasks with the same cluster. Now if we double the size of the cluster, to twelve machines: cluster systems never have linear speedup, so let’s assume the performance increases by say 50%. Of course this is just fake numbers, but similar behavior happens at some cut-off point anyway. Now GraphChi will solve exactly twice the number of tasks in time T. Task 3 Task 4 Task 5 Task 6 Task 10 Task 11 12 machines Task 12 Time T Time T

We are not only ones thinking this way…

Applications for GraphChi Graph Mining Connected components Approx. shortest paths Triangle counting Community Detection SpMV PageRank Generic Recommendations Random walks Collaborative Filtering (by Danny Bickson) ALS SGD Sparse-ALS SVD, SVD++ Item-CF + many more Probabilistic Graphical Models Belief Propagation One important factor to evaluate is that is this system any good? Can you use it for anything? GraphChi is an early project, but we already have a great variety of algorithms implement on it. I think it is safe to say, that the system can be used for many purposes. I don’t know of a better way to evaluate the usability of a system than listing what it has been used for. There are over a thousand of downloads of the source code + checkouts which we cannot track, and we know many people are already using the algorithms of GraphChi and also implementing their own. Most of these algos are now available only in the C++ edition, apart from the random walk system which is only in the Java version.

Easy to Get Started Java and C++ versions available No installation, just run Any machine, SSD or HD http://graphchi.org http://code.google.com/p/graphchi http://code.google.com/p/graphchi-java

What’s New

Extensions 1. Dynamic Edge and Vertex Values Block 1 Block 1 1. Dynamic Edge and Vertex Values Divide shards into small (4 mb) blocks that can be resized separately. Block 2 Block 2 Block 3 Block 3 Block N Shard(j) 2. Integration with Hadoop / Pig 3. Fast neighborhood queries over shards Sparse indices 4. DrunkardMob: Random Walks (next…)

Random Walk Simulations Personalized PageRank Problem: using the power method would require O(V2) of memory to compute for all vertices. Can be approximated by simulating random walks and computing the sample distribution. Other applications: Recommender systems: FolkRank (Hotho 2006), finding candidates Knowledge-base inference (Lao, Cohen 2009)

Random walk in an in-memory graph Compute one walk a time (multiple in parallel, of course): parfor walk in walks: for i=1 to numsteps: vertex = walk.atVertex() walk.takeStep(vertex.randomNeighbor()) So how would we do this if we could fit the graph in memory? Extremely slow in GraphChi / PSW ! Each hop might require loading of a new interval.

Random walks in GraphChi DrunkardMob –algorithm Reverse thinking parfor vertex in graph: mywalks = walkManager.getWalksAtVertex(vertex.id) foreach walk in mywalks: walkManager.addHop(walk, vertex.randomNeighbor()) Need to encode only current vertex and source vertex for each walk: 4-byte integer sufficient / walk With 144 GB RAM, could run 15 billion walks simultaneously (on Java) – recommendations for 15 million users CHUNKS! Load chunk of graph, chunk of walks and move them forward.

Keeping track of walks GraphChi Walk Distribution Tracker (DrunkardCompanion) Execution interval Source A top-N visits Source B top-N visits Vertex walks table (WalkManager)

Keeping track of walks GraphChi Walk Distribution Tracker (DrunkardCompanion) Execution interval Source A top-N visits Source B top-N visits Source A top-N visits Source B top-N visits Vertex walks table (WalkManager)

Application: Twitter’s Who-to-Follow Based on WWW’13 paper by Gupta et. al. Step 3: Compute SALSA and pick top scored users as recommendations. TODO: IMPROVE Step 1: Compute Circle of Trust (CoT) for each user Step 2: Bipartite graph with CoT + CoT’s followees. Neighborhood queries over shards. DrunkardMob

Conclusion GraphChi can run your favorite graph computation on extremely large graphs – on your laptop Unique features such as random walk simulations and dynamic graphs Most popular: Collaborative Filtering toolkit (by Danny Bickson)