Computations have to be distributed !

ExampleKey/value pair Distributed Grep map -> matched line reduce -> just pass Count of URL Access Frequency map -> reduce -> Reverse Web-Link Graph map -> reduce -> Term-Vector per Host map -> #term vector = a list of reduce -> Inverted Index map -> a sequence of reduce -> Distributed Sort map -> reduce -> just pass

Master Worker Master

SimilarityDifference Reduce Same code is used to implement both the combiner and the reduce functions. Output is written to the final output file. Combiner Output is written to an intermediate file that will be sent to a reduce task. MapperCombinerReducerMapperCombinerReducer

Worker Master

[ ]

The model is easy to use Large Variety of problems are easily expressible Developed that scales to large clusters of machines