1. Hola Hadoop

2. 0. Clean-Up The Hard-disks Delete tmp/ folder from workspace/mdp-lab3 Delete unneeded downloads

3. 0. Peligro! Please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please

4. … please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please please

5. Peligro! … please

6. … please be careful of what you are doing! Think twice before: rm mv cp kill emacs/vim/… configuration files

7. … please.

8. cluster.dcc.uchile.cl

9. 1. Download tools http://aidanhogan.com/teaching/cc5212- 1/tools/ http://aidanhogan.com/teaching/cc5212- 1/tools/ Unzip them somewhere you can find them

10. 2. Log-in PuTTy 1 2 3

11. 3. Open DFS Browser http://cluster.dcc.uchile.cl:50070/

12. 3. PuTTy: Upload data to HDFS hadoop fs -ls / hadoop fs -ls /uhadoop hadoop fs -mkdir /uhadoop/[username] – [username] = first letter first name, last name (e.g., “ahogan”) cd /data/hadoop/hadoop/data/ hadoop fs -copyFromLocal /data/hadoop/hadoop/data/es-abstracts.txt /uhadoop/[username]/es-abstracts.txt

13. Note on namespace If you need to disambiguate local/remote files HDFS file – hdfs://cm:9000/uhadoop/… Local file – file:///data/hadoop/...

14. 4. Let’s Build Our First MapReduce Job Hint: Use Monday’s slides for “inspiration” – http://aidanhogan.com/teaching/cc5212-1/ http://aidanhogan.com/teaching/cc5212-1/ 1.Implement map(.,.,.,.) method 2.Implement reduce(.,.,.,.) method 3.Implement main(.) method

15. 5. Eclipse: Build jar Right Click build.xml > dist (Might need to make a dist folder)

16. 6. WinSCP: Copy.jar to Master Server Don’t save password! 1 2 3 4

17. 6. WinSCP: Copy.jar to Master Server

18. Create dir: /data/2014/uhadoop/[username]/ Copy your mdp-lab4.jar into it

19. 7. Putty: Run Job hadoop jar /data/2014/uhadoop/[username]/mdp- lab4.jar WordCount /uhadoop/[username]/es- abstracts.txt /uhadoop/[username]/wc/ All one command!

20. 8. Look at output hadoop fs -ls /uhadoop/[username]/wc/ hadoop fs -cat /uhadoop/[username]/wc/part-00000 | more hadoop fs -cat /uhadoop/[username]/wc/part-00000 | grep - e "^de" | more All one command! Look for “de” … 4575144 occurrences in local run

21. 9. Look at output through browser http://cluster.dcc.uchile.cl:50070/