1. Parallel Computing Techniques

2. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

3. 1. Introduction SuperComputers  http://www.research.ibm.com/bluegene/  http://ungrid.unal.edu.co/ScDeadProjects. html http://ungrid.unal.edu.co/ScDeadProjects. html http://ungrid.unal.edu.co/ScDeadProjects. html

4. 1. Introduction  How to reduce processing time? 1. SuperComputers (10 1 - 10 4 µP, 10 1 - 10 2 m$) 2. Clusters (10 1 - 10 3 µP, 50.000 - 10 1 m$) 3. Grids (10 1 – 10 6..n µP, ~0$)  How to reduce processing time? 1. SuperComputers (10 1 - 10 4 µP, 10 1 - 10 2 m$) 2. Clusters (10 1 - 10 3 µP, 50.000 - 10 1 m$) 3. Grids (10 1 – 10 6..n µP, ~0$)

5. 1. Introduction  Multitasking != Multiprocessing Multitasking: 1 CPU  Multitasking != Multiprocessing Multitasking: 1 CPU t CPU ~10ms T1T1 T2T2 T3T3 T1T1 T2T2 T3T3 T1T1 T2T2 T3T3 T1T1 T2T2 T3T3...

6. 1. Introduction  Multitasking != Multiprocessing Multiprocessing : 2+ CPUs  Multitasking != Multiprocessing Multiprocessing : 2+ CPUs t CPU 1 T1T1 T3T3 T1T1 T3T3 T1T1 T3T3 T1T1 T3T3 T1T1 T3T3 T1T1 T3T3 T2T2 T4T4 T2T2 T4T4 T2T2 T4T4 T2T2 T4T4 T2T2 T4T4 T2T2 T4T4 CPU 2...

7. 1. Introduction  Parallel Computing: 2+ CPUs t CPU 1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 CPU 2......... T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 T1T1 CPU 3...

8. 1. Introduction  Distributed Computing: 2+ Machines NET (LAN, MAN, WAN) T1T1 T1T1 T2T2 T1T1 T2T2

9. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

10. 2. Parallel Machines  Flynn's Taxonomy A classification of computer architectures based on the number of streams of instructions and data: SISD / MISD / SIMD / MIMD

11. 2. Parallel Machines  SISD Single Instruction Stream/Single Data Stream. sequential computer: PC (CPU+RAM)  MISD Multiple Instruction Stream/Single Data Stream. unusual  SISD Single Instruction Stream/Single Data Stream. sequential computer: PC (CPU+RAM)  MISD Multiple Instruction Stream/Single Data Stream. unusual

12. 2. Parallel Machines  SIMD Single Instruction Stream/Multiple Data Stream. array processor or “vector processor” (Westinghouse, Fujitsy, NEC, Cray). all CPUs execute same instruction at same time. parallel done by different data  MIMD Multiple Instruction Stream/Multiple Data Stream. each cpu has own program and data. syncronization by communication  SIMD Single Instruction Stream/Multiple Data Stream. array processor or “vector processor” (Westinghouse, Fujitsy, NEC, Cray). all CPUs execute same instruction at same time. parallel done by different data  MIMD Multiple Instruction Stream/Multiple Data Stream. each cpu has own program and data. syncronization by communication

13. 2. Parallel Machines  Amdahl's Law (Gene Amdahl/IBM) “If it takes one man one minute to dig a post-hole then sixty men can dig it in one second”

14. 2. Parallel Machines S = 11 F + (1- F)/N S : speedup F : sequential faction N : number of processors

15. 2. Parallel Machines if N  infinite S = 1 / F e.g. if F = 20%  S max = 5 if N  infinite S = 1 / F e.g. if F = 20%  S max = 5

16. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

17. 3. Clusters adm LAN

18. 3. Clusters Distributed System:  Homogeneus platform (HW/SW)  Dedicated machines  Local area networks  Centralized administration Distributed System:  Homogeneus platform (HW/SW)  Dedicated machines  Local area networks  Centralized administration

19. 3. Clusters  Gigabit ethernet cards/switches (60-90 MB/s)  RAM ~10GB/s (~10 2 faster)  Latency (50-100 µs) (~5 µs w/special cards)  Gigabit ethernet cards/switches (60-90 MB/s)  RAM ~10GB/s (~10 2 faster)  Latency (50-100 µs) (~5 µs w/special cards)

20. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

21. adm LAN-MAN- WAN 4. Computational Grids

22. Distributed System:  Heterogeneus platform (HW/SW)  General purpose machines  LAN/MAN/WAN  Multiple admin domains (e.g. UN Bogotá campus/UN) Distributed System:  Heterogeneus platform (HW/SW)  General purpose machines  LAN/MAN/WAN  Multiple admin domains (e.g. UN Bogotá campus/UN)

23. 4. Computational Grids  Specific Purpose Grids (seti@HOME,...)  General Purpose Grids (unGrid,...)  Specific Purpose Grids (seti@HOME,...)  General Purpose Grids (unGrid,...)

24.  10-100 ethernet cards/switches (1-10 MB/s)  Latency (10-200 ms)  10-100 ethernet cards/switches (1-10 MB/s)  Latency (10-200 ms) 4. Computational Grids

25. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

26. 5. unGrid  National University of Colombia: comprises several campuses across the country  The Bogotá campus is interconnected by a FDDI based net, where several LAN and VLAN coexist  Around 5000 computers interconnected by this net  Free CPU cycles are in the 60%-90% range  National University of Colombia: comprises several campuses across the country  The Bogotá campus is interconnected by a FDDI based net, where several LAN and VLAN coexist  Around 5000 computers interconnected by this net  Free CPU cycles are in the 60%-90% range

27.  The unGrid Project attempts to construct a virtual supercomputer that takes profit from these CPU idle cycles knowledge grid information grid computation/data grid   5. unGrid

28. Factors to consider when participating in a grid:. Privacy. Integrity. Transparency. Configuration/Uninstallation. Advantages 5. unGrid

29. JavaSpac es

30. 5. unGrid Master-Worker Pattern

31. 1. Introduction 2. Parallel Machines 3. Clusters 4. Computational Grids 5. unGrid 6. Questions & Answers

32. 6. Questions & Answers ? ?

33. ungrid.unal.ed u.co  i n g r e s a r  i n s t a l e u n G r i d  u t i l i c e u n G r i d