1. A Model of Computation for MapReduce
Karloff, Suri and Vassilvitskii (SODA’10)
Presented by Ning Xie

2. Why MapReduce
Tera- and petabytes data set (search engines, internet traffic, bioinformatics, etc)
Need parallel computing
Requirement: easy to program, reliable, distributed

3. What is MapReduce
A new framework for parallel computing originally developed at Google (before ’04)
Widely adopted and became standard for large scale data analysis
Hadoop (open source version) is being used by Yahoo, Facebook, Adobe, IBM, Amazon, and many institutions in academia

4. What is MapReduce (cont.)
Three-stage operations:
Map-stage: mapper operates on a single pair <key, value>, outputs any number of new pairs <key’, value’>
Shuffle-stage: all values that are associated to an individual key are sent to a single machine (done by the system)
Reduce-stage: reducer operates on the all the values and outputs a multiset of <key, value>

5. What is MapReduce (cont.)
Map operation is stateless (parallel)
Shuffle stage is done automatically by the underlying system
Reduce stage can only start when all Map operations are done (interleaving between sequential and parallel)

6. An example: kth frequency moment of a large data set
Input: x 2Σn , Σ is a finite set of symbols
Let f(¾) be the frequency of symbol ¾
note: ¾f(¾)=n
Want to compute ¾fk(¾)

7. An example (cont.) Input to each mapper: <i, xi>
M1(<i, xi>)= <xi , i> (i is the index)
Input to each reducer: <xi,{i1, i2,…, im}>
R1(<xi,{i1, i2,…, im}>)=<xi, mk>
Each mapper: M2(<xi, v>)=<$, v>
A single reducer: R2(<$,{v1,…,vl}>=<$, ivi>

8. Formal Definitons
A MapReduce program consists of a sequence <M1, R1, M2, R2,…, Ml, Rl> of mappers and reducers
The input is U0, a multiset of <key,value>

9. Formal Definitons Execution of the program For r=1,2,…,l
1. feed each <k,v> in Ur-1 to mapper Mr Let the output be U’r
2. for each key k, construct the multiset Vk,r s.t. <k, vi> 2 Ur-1
3. for each k, feed k and some perm. of Vk,r to a separate instance of Rr. Let Ur be the multiset of <key, value> generated by Rr

10. The MapReduce Class (MRC)
On input {<key,value>} of size n
Memory: each mapper/reducer uses O(n1-² ) space
Machines: There are £(n1-²) machines available
Time: each machine runs in time polynomial in n, not polynomial in the length of the input they receive
Randomized algorithms for map and reduce
Rounds: Shuffle is expensive
MRCi : num. of rounds=O(login)
DMRC: the deterministic variant

11. Comparing MRC with PRAM
Most relevant classical computation model is the PRAM (Parallel Random Access Machine) model
The corresponding class is NC
Easy relation: MRC µ P
Lemma: If NC  P, then MRC * NC
Open question: show that DMRC  P

12. Comparing with PRAM (cont.)
Simulation lemma:
Any CREW (concurrent read exclusive write) PRAM algorithm using O(n2-2²) total memory and O(n2-2²) processors and runs in time t(n) can be simulated by an algorithm in DMRC which runs in O(t(n)) rounds

13. Example: Finding an MST
Problem: find the minimum spanning tree of a dense graph
The algorithm
Randomly partition the vertices into k parts
For each pair of vertex sets, find the MST of the bipartite subgraph induce by these two sets
Take the union of all the edges in the MST of each pair, call the graph H
Compute an MST of H

14. Finding an MST (cont.) The algorithm is easy to parallelize
The MST of each subgraph can be computed in parallel
Why it works?
Theorem: the MST tree of H is an MST of G
Proof: we did not discard any relevant edge when sparsify the input graph G

15. Finding an MST (cont.) Why the algorithm in MRC?
Let N=|V| and m=|E|=N1+c
So input size n satisfies N=n1/1+c
Pick k=Nc/2
Lemma: with high probability, the size of each bipartite subgraph has size N1+c/2
so the input to any reducer is n1-²
The size of H is also n1-²

16. Functions Lemma
A very useful building block for designing MapReduce algorithms
Definition [MRC-parallelizable function]: Let S be a finite set. We say a function f on S is MRC-parallelizable if there are functions g and h so that the following hold:
For any partition of S, S = T1 [ T2 [ …[ Tk
f can be written as: f(S) =h(g(T1), g(T2),… g(Tk)).
g and h each can be represented in O(logn) bits.
g and h can be computed in time polynomial in |S| and all possible outputs of g can be expressed in O(logn) bits.

17. Functions Lemma (cont.)
Lemma (Functions Lemma)
Let U be a universe of size n and let S = {S1,…, Sk} be a collection of subsets of U, where k· n2-3² and i=1k|Si|· n2-2². Let F = {f1, …, fk} be a collection of MRC-parallelizable functions. Then the output f1(S1), …, fk(Sk) can be computed using O(n1-²) reducers each with O(n1-²) space.

18. Functions Lemma (cont.)
The power of the lemma
Algorithm designer may focus only on the structure of the subproblem and the input
Distribution the input across reducer is handled by the lemma (existence theorem)
Proof of the lemma is not easy
Use universal hashing
Use Chernoff bound, etc

19. Application of the Functions Lemma: s-t connectivity
Problem: given a graph G and two nodes, are they connected in G?
Dense graphs: easy, powering adjacency matrix
Sparse graphs?

20. A logn-round MapReduce algorithm for s-t connectivity
Initially every node is active
For i=1,2,…, O(logn) do
Each active node becomes a leader with probability ½
For each non-leader active node u, find a node v in the neighbor of u’s current connected component
If the connected component of v is non-empty, then u become passive and re-label each node in u’s connected component with v’s label
Output TRUE if s and t have the same label, FALSE otherwise

21. Conclusions A rigorous model for MapReduce
Very loose requirements for the hardware requirements
Call for more research in this direction

22. THANK YOU!