Progress Report 2009/12/15

Add pipe in hadoop For now on hadoop can only do one thing, in one command like bin/hadoop fs –ls Pipes have the potential of reducing comunication and increasing the distributedness of computation Currently, we are still work on fully understanding hadoop’s local –cloud-node mechanisms

About MapReduce Because we want to add the pipe in hadoop, so we need to create local temp file in each client The challenge about that is how to tell JobTracker to create the file

WordCount example(1) We traced the wordCount example and tried to figure out how it works:

WordCount example(2) We used wireshark to find how the server told client to do the job

But we’re still working From the last slide, we see that the job is executed locally as a java code. But what exact java code is that? The answer is not as obvious as you would think. We hope to understand it soon.