# A Model of Computation for MapReduce

## Presentation on theme: "A Model of Computation for MapReduce"— Presentation transcript:

A Model of Computation for MapReduce
Karloff, Suri and Vassilvitskii (SODA’10) Presented by Ning Xie

Why MapReduce Tera- and petabytes data set (search engines, internet traffic, bioinformatics, etc) Need parallel computing Requirement: easy to program, reliable, distributed

What is MapReduce A new framework for parallel computing originally developed at Google (before ’04) Widely adopted and became standard for large scale data analysis Hadoop (open source version) is being used by Yahoo, Facebook, Adobe, IBM, Amazon, and many institutions in academia

What is MapReduce (cont.)
Three-stage operations: Map-stage: mapper operates on a single pair <key, value>, outputs any number of new pairs <key’, value’> Shuffle-stage: all values that are associated to an individual key are sent to a single machine (done by the system) Reduce-stage: reducer operates on the all the values and outputs a multiset of <key, value>

What is MapReduce (cont.)
Map operation is stateless (parallel) Shuffle stage is done automatically by the underlying system Reduce stage can only start when all Map operations are done (interleaving between sequential and parallel)

An example: kth frequency moment of a large data set
Input: x 2Σn , Σ is a finite set of symbols Let f(¾) be the frequency of symbol ¾ note: ¾f(¾)=n Want to compute ¾fk(¾)

An example (cont.) Input to each mapper: <i, xi>
M1(<i, xi>)= <xi , i> (i is the index) Input to each reducer: <xi,{i1, i2,…, im}> R1(<xi,{i1, i2,…, im}>)=<xi, mk> Each mapper: M2(<xi, v>)=<\$, v> A single reducer: R2(<\$,{v1,…,vl}>=<\$, ivi>

Formal Definitons A MapReduce program consists of a sequence <M1, R1, M2, R2,…, Ml, Rl> of mappers and reducers The input is U0, a multiset of <key,value>

Formal Definitons Execution of the program For r=1,2,…,l
1. feed each <k,v> in Ur-1 to mapper Mr Let the output be U’r 2. for each key k, construct the multiset Vk,r s.t. <k, vi> 2 Ur-1 3. for each k, feed k and some perm. of Vk,r to a separate instance of Rr. Let Ur be the multiset of <key, value> generated by Rr

The MapReduce Class (MRC)
On input {<key,value>} of size n Memory: each mapper/reducer uses O(n1-² ) space Machines: There are £(n1-²) machines available Time: each machine runs in time polynomial in n, not polynomial in the length of the input they receive Randomized algorithms for map and reduce Rounds: Shuffle is expensive MRCi : num. of rounds=O(login) DMRC: the deterministic variant

Comparing MRC with PRAM
Most relevant classical computation model is the PRAM (Parallel Random Access Machine) model The corresponding class is NC Easy relation: MRC µ P Lemma: If NC  P, then MRC * NC Open question: show that DMRC  P

Comparing with PRAM (cont.)
Simulation lemma: Any CREW (concurrent read exclusive write) PRAM algorithm using O(n2-2²) total memory and O(n2-2²) processors and runs in time t(n) can be simulated by an algorithm in DMRC which runs in O(t(n)) rounds

Example: Finding an MST
Problem: find the minimum spanning tree of a dense graph The algorithm Randomly partition the vertices into k parts For each pair of vertex sets, find the MST of the bipartite subgraph induce by these two sets Take the union of all the edges in the MST of each pair, call the graph H Compute an MST of H

Finding an MST (cont.) The algorithm is easy to parallelize
The MST of each subgraph can be computed in parallel Why it works? Theorem: the MST tree of H is an MST of G Proof: we did not discard any relevant edge when sparsify the input graph G

Finding an MST (cont.) Why the algorithm in MRC?
Let N=|V| and m=|E|=N1+c So input size n satisfies N=n1/1+c Pick k=Nc/2 Lemma: with high probability, the size of each bipartite subgraph has size N1+c/2 so the input to any reducer is n1-² The size of H is also n1-²

Functions Lemma A very useful building block for designing MapReduce algorithms Definition [MRC-parallelizable function]: Let S be a finite set. We say a function f on S is MRC-parallelizable if there are functions g and h so that the following hold: For any partition of S, S = T1 [ T2 [ …[ Tk f can be written as: f(S) =h(g(T1), g(T2),… g(Tk)). g and h each can be represented in O(logn) bits. g and h can be computed in time polynomial in |S| and all possible outputs of g can be expressed in O(logn) bits.

Functions Lemma (cont.)
Lemma (Functions Lemma) Let U be a universe of size n and let S = {S1,…, Sk} be a collection of subsets of U, where k· n2-3² and i=1k|Si|· n2-2². Let F = {f1, …, fk} be a collection of MRC-parallelizable functions. Then the output f1(S1), …, fk(Sk) can be computed using O(n1-²) reducers each with O(n1-²) space.

Functions Lemma (cont.)
The power of the lemma Algorithm designer may focus only on the structure of the subproblem and the input Distribution the input across reducer is handled by the lemma (existence theorem) Proof of the lemma is not easy Use universal hashing Use Chernoff bound, etc

Application of the Functions Lemma: s-t connectivity
Problem: given a graph G and two nodes, are they connected in G? Dense graphs: easy, powering adjacency matrix Sparse graphs?

A logn-round MapReduce algorithm for s-t connectivity
Initially every node is active For i=1,2,…, O(logn) do Each active node becomes a leader with probability ½ For each non-leader active node u, find a node v in the neighbor of u’s current connected component If the connected component of v is non-empty, then u become passive and re-label each node in u’s connected component with v’s label Output TRUE if s and t have the same label, FALSE otherwise

Conclusions A rigorous model for MapReduce
Very loose requirements for the hardware requirements Call for more research in this direction

THANK YOU!