1. Multi-core Programming: Basic Concepts

2. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 2 Multi-core Programming: Basic Concepts Objectives After completion of this module you will be familiar with the basic concepts of: threads multithreaded programming Note: Threads will be “agents” doing work No code examples are used

3. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 3 Multi-core Programming: Basic Concepts Agenda Basics Design concepts Correctness concepts Performance concepts

4. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 4 Multi-core Programming: Basic Concepts Agenda Basics Processes and threads Parallelism and concurrency Design concepts Correctness concepts Performance concepts

5. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 5 Multi-core Programming: Basic Concepts Why Use Threads Benefits Increased performance Easy method to take advantage of multi-core Better resource utilization Reduce latency (even on single processor systems) Efficient data sharing Sharing data through memory more efficient than message-passing Risks Increases complexity of application Difficult to debug (data races, deadlocks, etc.) Processes & Threads

6. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 6 Multi-core Programming: Basic Concepts Processes and Threads Modern operating systems load programs as processes Resource holder Execution A process starts executing at its entry point as a thread Threads can create other threads within the process Each thread gets its own stack All threads within a process share code & data segments Processes & Threads Code segment Data segment thread main() … thread Stack

7. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 7 Multi-core Programming: Basic Concepts Concurrency vs. Parallelism Concurrency: two or more threads are in progress at the same time: Parallelism: two or more threads are executing at the same time Multiple cores needed Thread 1 Thread 2 Thread 1 Thread 2 Concurrency vs. Parallelism

8. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 8 Multi-core Programming: Basic Concepts Agenda Basics Design concepts Threading for functionality or performance? Threading for throughput or turnaround? Decomposing the work Correctness concepts Performance concepts

9. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 9 Multi-core Programming: Basic Concepts Threading for Functionality Assign threads to separate functions done by application Easiest method since overlap is unlikely Example: Building a house Bricklayer, carpenter, roofer, plumber,… Threading for Functionality or Performance?

10. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 10 Multi-core Programming: Basic Concepts Threading for Performance Increase the performance of computations Thread in order to improve turnaround or throughput Examples Automobile assembly line Each worker does an assigned function Searching for pieces of Skylab Divide up area to be searched US Postal Service Post office branches, mail sorters, delivery Threading for Functionality or Performance?

11. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 11 Multi-core Programming: Basic Concepts Turnaround Complete single task in the smallest amount of time Example: Setting a dinner table One to put down plates One to fold and place napkins One to place utensils Spoons, knives, forks One to place glasses Threading for Throughput or Turnaround?

12. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 12 Multi-core Programming: Basic Concepts Throughput Complete the most tasks in a fixed amount of time Example: Setting up banquet tables Multiple waiters each do separate tables Specialized waiters for plates, glasses, utensils, etc. Threading for Throughput or Turnaround?

13. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 13 Multi-core Programming: Basic Concepts Task Decomposition Divide computation based on natural set of independent tasks Assign data for each task as needed Example: Paint-by-Numbers Painting a single color is a single task Number of tasks = number of colors Two artists: one does even, other odd 1 1 1 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 55 5 5 5 5 5 5 5 5 5 5 5 5 3 6 6 7 9 8 3 8 3 3 8 8 9 1 10 7 6 11 Task Decomposition

14. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 14 Multi-core Programming: Basic Concepts Data Decomposition Large data sets whose elements can be computed independently Divide data and associated computation among threads Example: Grading test papers Multiple graders with same key What if different keys are needed? Data Decomposition

15. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 15 Multi-core Programming: Basic Concepts Agenda Basics Design concepts Correctness concepts Race Conditions and Synchronization Deadlock Performance concepts

16. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 16 Multi-core Programming: Basic Concepts Race Conditions Threads “race” against each other for resources Execution order is assumed but cannot be guaranteed Storage conflict is most common Concurrent access of same memory location by multiple threads At least one thread is writing Example: Musical Chairs Race Conditions and Synchronization

17. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 17 Multi-core Programming: Basic Concepts Mutual Exclusion Critical Region Portion of code that accesses (reads & writes) shared variables Mutual Exclusion Program logic to enforce single thread access to critical region Enables correct programming structures for avoiding race conditions Example: Safe Deposit box Attendants ensure mutual exclusion Race Conditions and Synchronization

18. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 18 Multi-core Programming: Basic Concepts Synchronization Synchronization objects used to enforce mutual exclusion Lock, semaphore, critical section, event, condition variable, atomic One thread “holds” sync. object; other threads must wait When done, holding thread releases object; some waiting thread given object Example: Library book One patron has book checked out Others must wait for book to return Race Conditions and Synchronization

19. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 19 Multi-core Programming: Basic Concepts 9 Barrier Synchronization Threads pause at execution point Threads waiting are idle; overhead When all threads arrive, all are released Example: Race starting line Race Conditions and Synchronization

20. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 20 Multi-core Programming: Basic Concepts Deadlock Threads wait for some event or condition that will never happen Example: Traffic jam at intersection Cars unable to turn or back up What is Livelock? Threads change state in response to each other Example: Robin Hood and Little John on log bridge Deadlock

21. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 21 Multi-core Programming: Basic Concepts Agenda Basics Design concepts Correctness concepts Performance concepts Speedup and Efficiency Granularity and load balance

22. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 22 Multi-core Programming: Basic Concepts Speedup (Simple) Measure of how much faster the computation executes versus the best serial code Serial time divided by parallel time Example: Painting a picket fence 30 minutes of preparation (serial) One minute to paint a single picket 30 minutes of cleanup (serial) Thus, 300 pickets takes 360 minutes (serial time) Speedup and Efficiency

23. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 23 Multi-core Programming: Basic Concepts Computing Speedup What if fence owner uses spray gun to paint 300 pickets in one hour? Better serial algorithm If no spray guns are available for multiple workers, what is maximum parallel speedup? Number of painters TimeSpeedup 130 + 300 + 30 = 3601.0X 230 + 150 + 30 = 2101.7X 1030 + 30 + 30 = 904.0X 10030 + 3 + 30 = 635.7X Infinite30 + 0 + 30 = 606.0X Illustrates Amdahl’s Law Potential speedup is restricted by serial portion Speedup and Efficiency

24. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 24 Multi-core Programming: Basic Concepts Efficiency Measure of how effectively computation resources (threads) are kept busy Speedup divided by number of threads Expressed as average percentage of non-idle time Number of painters TimeSpeedupEfficiency 13601.0X100% 230 + 150 + 30 = 2101.7X85% 1030 + 30 + 30 = 904.0X40% 10030 + 3 + 30 = 635.7X5.7% Infinite30 + 0 + 30 = 606.0Xvery low Speedup and Efficiency

25. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 25 Multi-core Programming: Basic Concepts Granularity Loosely defined as the ratio of computation to synchronization Be sure there is enough work to merit parallel computation Example: Two farmers divide a field. How many more farmers can be added? Granularity and Load Balance

26. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 26 Multi-core Programming: Basic Concepts Load Balance Most effective distribution is to have equal amounts of work per thread Threads that finish first sit idle Threads should finish close to same time Example: Busing banquet tables Better to assign same number of tables to each bus person Granularity and Load Balance

27. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 27 Multi-core Programming: Basic Concepts Multi-core Programming: Basic Concepts What’s Been Covered Processes and threads Parallelism and concurrency Design concepts Threading for throughput or turnaround? Threading for functionality or performance? Decomposing the work Correctness concepts Race Conditions and Synchronization Deadlock Performance concepts Speedup and Efficiency Granularity and load balance

28. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 28 Multi-core Programming: Basic Concepts

29. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 29 Multi-core Programming: Basic Concepts Alternative Slides

30. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 30 Multi-core Programming: Basic Concepts Turnaround Complete single task in the smallest amount of time Example: Decorating a cake Frosting One to frost top, one to frost sides Decoration One to do borders One to place flowers One to write “Happy Birthday” Threading for Throughput or Turnaround?

31. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 31 Multi-core Programming: Basic Concepts Throughput Complete the most tasks in a fixed amount of time Example: Decorating cakes Multiple bakers doing entire decoration Assembly line of frosters and decorators Threading for Throughput or Turnaround?

32. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 32 Multi-core Programming: Basic Concepts Turnaround Complete single task in the smallest amount of time Example: Parents to arrive in one hour Hugues cleans living room Otto cleans bathroom Thomas cleans kitchen Threading for Throughput or Turnaround?

33. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 33 Multi-core Programming: Basic Concepts Throughput Complete the most tasks in a fixed amount of time Example: Produce pins “One man draws out the wire, another straights it, a third cuts it, a fourth points it, a fifth grinds it at the top for receiving the head; …” (A. Smith 1776) Threading for Throughput or Turnaround?

34. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 34 Multi-core Programming: Basic Concepts Threading for Functionality Assign threads to separate functions done by application Easiest method since overlap is unlikely Example: Coaching a football team Head coach, position coaches, special teams coach Threading for Functionality or Performance?

35. Copyright © 2006, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners. 35 Multi-core Programming: Basic Concepts Deadlock Threads wait for some event or condition that will never happen Example: Traffic jam at intersection Cars unable to turn or back up What is Livelock? Threads change state in response to each other Example: Robin Hood and Little John on log bridge Deadlock