Department of Computer Science, Johns Hopkins University Pregel: BSP and Message Passing for Graph Computations EN Randal Burns 14 November 2013

Tenets of Large-Scale Graphs Graph computations require scale-out computing architectures At some level of scalability this is true But, there is a big performance gap between what can be done on one machine and clusters (RB’s opinion) Graph computations require fault-tolerance All computations at scale require fault tolerance This aspect is quite different (even than MPI)

Why not Map/Reduce? Many iterations needed for graph algorithms? – Traversals, path finding M/R has fixed costs to set up each job Must materialize all inputs/outputs from each M/R stage Not the right model: – Not fine grained – Not asynchronous

Bulk- Synchronous Parallel

Pregel Bulk-synchronous parallel Strict message passing Computing in rounds (supersteps) Programming graphs from the perspective of the vertex – Can compute on local data – Can receive messages – Can send messages – Can modify local state – Can create new vertices (computational contexts) G. Malewicz et al. Pregel: A System for Large-Scale Graph Processing. SIGMOD, 2010.

Example: Maximum Value Express algorithms as operations on vertices

Compare with MPI Message passing, no shared memory [ ] Bulk-synchronous processing [ ] Supersteps=barriers [ ] SPMD [ ] Similar architectural properties and benefits. More restrictive in programming model (what each node can do) and communication protocols (when nodes can communicate These restrictions lead to some nice system properties

Model of Computation During a superstep – Receive messages from previous superstep (from the past) – Send messages to next superstep (to the future) – No interactive communication during rounds No deadlocks are possible. Why? All vertex-to-vertex operations are totally synchronous – Makes fault tolerance simple Vertices run their local compute function asynchronously

API Simple (if you can represent algorithm as round-by-round modification of vertices)

SSSP Is this Djikstra’s? How does it compare?

SSSP Combiner Concept from map reduce (aggregation of messages) – I’m not sure where combiners run in pregel to reduce message traffic

Fault Tolerance Checkpoint at the start of (every or some) superstep to shared storage Master election, worker pings Restart computation from checkpoint for nodes that do not respond Synchronous helpful – Single system view of state superstep) – No dependencies among nodes with superstep (message from previous step, to future step)

Scaleout Linear scaling properties (looks good) – But is the baselines slow? i.e. with message passing as a common denominator

GraphLab Asynchronous, dynamic, graph parallel computation – Asynchronous is different Graph lab is a “sequential, shared-memory abstraction where each vertex can read and write data to adjacent edges and vertices” Y. Low et al. Distributed GraphLab: A Framework for Machine Learning in the Cloud. VLDB 5(8), 2012.

Natural Graphs and Skewness In BSP programming, performance is limited by the slowest machine “Natural graphs” have a power-law distribution of edge degrees which leads to few highly-loaded nodes Graphlab concludes: need to compute asynchronously to avoid barrier waits

Asynchronous and Convergence Propogation algorithms converge more quickly in an asynchronous model – With some nodes converging immediately and few stragglers

GraphLab Abstraction Data graph: user-modifiable program state Update function: computation to modify the graph in small overlapping scopes Sync operation: extract and maintain global aggregates

PageRank in GraphLab

PageRank in Pregel

Differences Pregel lock-step on every node, iterate until some heuristic property is met (30?) – No ability to pull data, must wait for updates – GraphLab people say is weird GraphLab significant changes lead to “activations” – Just activate the portion of the network that needs to compute – More directed computation towards convergence

GraphLab Complexities Scheduling what can run: – Graph coloring and parallel execution of nodes of the same color (no adjacencies) – Distributed locking complex schemes What about synchronous algorithms – Many algorithms are. GraphLab can actually emulate Pregel semantics with sync. This is the topic of the next class.