1. Programming with CUDA WS 08/09 Lecture 13 Thu, 04 Dec, 2008

2. Today Last lecture of this half Last lecture of this half Analysis of parallel algorithms Analysis of parallel algorithms Final projects Final projects

3. Analyzing parallel algorithms Tightly coupled with the parallel architecture Tightly coupled with the parallel architecture Serial programs have a standard reference model, the von Neumann model Serial programs have a standard reference model, the von Neumann model

4. Analyzing parallel algorithms von Neumann model von Neumann model –Single CPU reads and writes to an attached memory

5. Analyzing parallel algorithms A parallel model supporting analysis has to be A parallel model supporting analysis has to be –Simple: understandability –Realistic: applicability

6. Analyzing parallel algorithms The multicomputer The multicomputer –A node is a von Neumann machine –Each node has its own program –Messages: read/write remote memory –Message cost depends on size

7. Analyzing parallel algorithms Distributed memory MIMD Distributed memory MIMD –Distributed memory –Similar to multicomputer, but message sending cost depends on node location and network traffic

8. Analyzing parallel algorithms Multiprocessor Multiprocessor –PRAM (Parallel Random Access Machine) model: all processors access memory with same cost –Special case: SIMD (CUDA)‏ Simplified hardware & software Simplified hardware & software Specialized problems Specialized problems

9. Analyzing parallel algorithms Network Network –Potentially high remote-access costs –New issues: reliability, security

10. Analyzing parallel algorithms Useful analysis tools Useful analysis tools –Brent's theorem (algorithm only)‏ –Amdahl's law –Iso-efficiency

11. Final Projects General guideline General guideline –Read up on the problem and possible solutions –Design your solution –Implement your solution –Optimize your solution –Analyze your solution A couple of lectures in the second half A couple of lectures in the second half

12. Final Exam When? When? –End of semester Submit project early Submit project early Get done with the course Get done with the course –First week of March More time to work on the project More time to work on the project Need to worry about exam during holidays Need to worry about exam during holidays –Might forget about projects in the meanwhile

13. See you next year? Happy holidays