Summary Background –Why do we need parallel processing? Applications Introduction in algorithms and applications –Methodology to develop efficient parallel (distributed-memory) algorithms –Understand various forms of overhead (communication, load imbalance, search overhead, synchronization) and various distributions (blockwise, cyclic) –Understand correctness problems (e.g. message ordering)

Summary Parallel machines and architectures –Processor organizations, topologies, criteria –Types of parallel machines arrays/vectors, shared-memory, distributed memory –Routing –Flynn’s taxonomy –What are cluster computers? –What networks do real machines (like the Blue Gene) use? –Speedup, efficiency (+ their implications), Amdahl’s law

Summary Programming methods, languages, and environments –Different forms of message passing naming, explicit/implicit receive, synchronous/asynchronous sending –Select statement –SR primitives (not syntax) –Master/worker model, search overhead (TSP) –MPI: message passing primitives, collective communication –Java parallel programming model and primitives –HPF: problems with automatic parallelization; division of work between programmer and HPF compiler; alignment/distribution primitives; performance implications

Summary Applications –N-body problems: load balancing and communication (locality) optimizations, costzones, performance comparison –Search algorithm (TDS): use asynchronous communication + clever (transposition-driven) scheduling –Bioinformatics (repeats detection): combine many forms of parallelism Grid computing –What are grids? Why do we need them? –Application-level tools for grid programming and execution –Why is parallel computing on grids feasible? –Ibis: goals, applications, design, programming vs. deployment, communication primitives, divide&conquer parallelism, performance behavior on a grid

Summary Multimedia content analysis on Grids –What is Multimedia Content Analysis (MMCA) ? Imaging applications, feature vectors, low-level patterns –Why parallel computing in MMCA - and how? Need for speed and transparency –Software Platform: Parallel-Horus. Task/data parallelism, Separable Recursive Filtering, Lazy Parallelization –Example – Parallel Image Processing on Clusters –Grids and their specific problems. Promise and problems of the grid –Towards a Software Platform for MMCA on Grids. Problems with wide- area servers –Large-scale MMCA applications on Grids (TRECVID, object recognition)