1. Parallel Thinking with Java Multithreading

2. Introduction Algorithms which make use of more than one processor Very hot topic today because all chip manufacturers are producing “multi- core” processors- (>1 processor per chip)

3. A little physics lesson To increase performance:::::: Smaller transistors  transistor density   faster processors  power consumption   increased heat  unreliable processors Solution = multicore computers

4. Can we parallelize following program? c = a+b d = a*b e = d+2

5. Answer Yes

6. Can we parallelize following program? c = a+b d = c*b e = d+2

7. Answer No. Such program’s are called inherently serial program

8. Why we need ever-increasing performance? Climate modeling century scale changes limited by minute scale timesteps Protein folding Mad Cow disease, allergies Drug discovery extensive computational analysis of genomes. Takes 10yrs and 2-3billion dollars for drug to market. Data analysis. terabytes of data : dynamic, unstructured cant store in relational db, faster analysis

9. Example Calculate sum of n numbers using p processors

10. Calculating the sum of an array of n integers Serial code: Serial Time = O(n)

11. How to do this in parallel? Algorithm1: Assume n = p P1 P2P3 P0 4 3 1 1 4+3=7 7+1=8 8+1=9 Parallel Time = T computation + T communication = O(n-1) + O(n-1) = O(n-1) = O(n) Not so great compared to serial algorithm. Still O(n) Can we do better? t1 t2 t3

12. Algorithm2: Assume n = p P1 P2P3 P0 4 3 1 1 4+3=7 1+1=2 7+2=9 Parallel Time = O(log n) Great time compared to serial algorithm. t1 t2

13. Algorithm1: Assume n > p (in practice n>>p) Assign n/p elements per process Each process calculate local sum of n/p elements Now each process has 1 element (local sum). So proceed as before to calculate global sum P1 P2P3 P0 4,3 3,5 1,1 1,2 7+8=15 15+2=17 17+3=20 t1 t2 t3 Local_sum= 7 8 2 3 Global_sum Parallel Time = T computation + T communication = O(n/p) + O(p) = O(n/p +p)

14. Algorithm2: Assume n > p (in practice n>>p) Assign n/p elements per process Each process calculate local sum of n/p elements Now each process has 1 element (local sum). So proceed as before to calculate global sum Local_sum= 7 8 2 3 Parallel Time = T computation + T communication = O(n/p) + O(logp) = O(n/p + logp) P1 P2P3 P0 4,3 3,5 1,1 1,2 7+8=15 2+3=5 15+5=20 Global_sum

15. Shared memory vs Distributed Memory Shared memoryDistributed memory

16. Java Multithreading Running Threads A thread is a program unit that is executed independently of other parts of the program. The jvm executes each thread for a short amount of time and then switches to another thread. This gives the illusion of executing the threads in parallel to each other. Actually, if a computer has multiple central processing units (CPUs), then some of the threads can run in parallel, one on each processor.

17. Running a Thread 1.Implement a class that implements the Runnable interface: public interface Runnable { void run(); } 2.Place the code for your task into the run method of your class: public class MyRunnable implements Runnable { public void run() { Task statements... }

18. 3.Create an object of your subclass: Runnable r = new MyRunnable(); 4.Construct a Thread object from the Runnable object: Thread t = new Thread(r); 5.Call the start method to start the thread: t.start();

19. A program to print a time stamp and “Hello World” once a second for ten seconds: Fri Dec 28 23:12:03 PST 2012 Hello, World! Fri Dec 28 23:12:04 PST 2012 Hello, World! Fri Dec 28 23:12:05 PST 2012 Hello, World! Fri Dec 28 23:12:06 PST 2012 Hello, World! Fri Dec 28 23:12:07 PST 2012 Hello, World! Fri Dec 28 23:12:08 PST 2012 Hello, World! Fri Dec 28 23:12:09 PST 2012 Hello, World! Fri Dec 28 23:12:10 PST 2012 Hello, World! Fri Dec 28 23:12:11 PST 2012 Hello, World! Fri Dec 28 23:12:12 PST 2012 Hello, World!

20. Program that generates previous... import java.util.Date; public class MyRunnable implements Runnable { public void run() { try { for (int i = 1; i <= 10; i++) { Date now = new Date(); System.out.println(now + " Hello, World!"); Thread.sleep(1000); //1000ms = 1 s //throws java.lang.InterruptedException } catch(Exception e) { } public static void main(String[] args) { Runnable r = new MyRunnable(); Thread t = new Thread(r); t.start(); }

21. Output

22. Run method Sleeping thread can generate an InterruptedException Catch the exception Terminate the thread public void run() { try { Task statements } catch (InterruptedException exception) { } Clean up, if necessary }

23. Programming Question Write a class GreetingRunnable, where you create two threads t1, t2 that run in parallel. Each thread prints a greeting message repeatedly for 10 times with a delay of 1 second. The greeting for t1 is “Hello” and for t2 is “Goodbye”. i.e for each thread: Loop 10 times through task actions: Print a time stamp Print the greeting Wait a second Find class template

24. Class Template import java.util.Date; /** A runnable that repeatedly prints a greeting. */ public class GreetingRunnable implements Runnable { private static final int REPETITIONS = 10; private static final int DELAY = 1000; private String greeting; /** Constructs the runnable object. @param aGreeting the greeting to display */ public GreetingRunnable(String aGreeting) { greeting = aGreeting; } public void run() { try { //TODO: print timestamp and greeting for REPETITIONS repetitions with a delay DELAY between repetitions } catch (InterruptedException exception) { } public static void main(String[] args) { //TODO: create two GreetingRunnable objects r1, r2 with greetings "Hello" and "GoodBye" respectively //TODO: Assign r1 to thread t1, and r2 to t2 respectively //TODO: start threads t1 and t2 }

25. Answer import java.util.Date; public class GreetingRunnable implements Runnable { private static final int REPETITIONS = 10; private static final int DELAY = 1000; private String greeting; /** Constructs the runnable object. @param aGreeting the greeting to display */ public GreetingRunnable(String aGreeting) { greeting = aGreeting; } public void run() { try { //print greeting for REPETITIONS repetitions with a delay DELAY for (int i = 1; i <= REPETITIONS; i++) { Date now = new Date(); System.out.println(now + " " + greeting); Thread.sleep(DELAY); } catch (InterruptedException exception) { } public static void main(String[] args) { //create two GreetingRunnable objects r1, r2 with greetings "Hello" and "GoodBye" respectively GreetingRunnable r1 = new GreetingRunnable("Hello"); GreetingRunnable r2 = new GreetingRunnable("Goodbye"); //assign r1, r2 to two thrads t1, t2 respectively Thread t1 = new Thread(r1); Thread t2 = new Thread(r2); //start threads t1 and t2 t1.start(); t2.start(); }

26. Output for 2 runs Are they different?

27. Thread Scheduler Thread scheduler: runs each thread for a short amount of time (a time slice) Then the scheduler activates another thread There will always be slight variations in running times – especially when calling operating system services (e.g. input and output) There is no guarantee about the order in which threads are executed

28. Terminating Threads A thread terminates when its run method terminates Notify a thread that it should terminate: t.interrupt(); interrupt does not cause the thread to terminate – it sets a boolean variable in the thread data structure The run method should check occasionally whether it has been interrupted: Use the interrupted method An interrupted thread should release resources, clean up, and exit: public void run() { for (int i = 1; i <= REPETITIONS && !Thread.interrupted(); i++) { Do work } Clean up }

29. Java does not force a thread to terminate when it is interrupted It is entirely up to the thread what it does when it is interrupted Interrupting is a general mechanism for getting the thread’s attention

30. Race Conditions When threads share a common object, they can conflict with each other Sample program: multiple threads manipulate a bank account Create two sets of threads: Each thread in the first set repeatedly deposits $100 Each thread in the second set repeatedly withdraws $100 Let us write a program to simulate this situation...

31. Programming Question Write class DepositRunnable makes periodic deposits to a bank account Write class WithdrawRunnable makes periodic withdrawals from a bank account Given: Class BankAccount Prints the deposit/withdraw amount and new balance when a transaction is made Class BankAccountThreadRunner Creates a BankAccount object and repatedly deposits and withdraws $100 from account Template given

32. BankAccount class public class BankAccount { private double balance; public BankAccount() { balance = 0; } public void deposit(double amount) { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } public void withdraw(double amount) { System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } public double getBalance() { return balance; } Copy and save in lec14 directory

33. BankAccountThreadRunner class public class BankAccountThreadRunner { public static void main(String[] args) { BankAccount account = new BankAccount(); final double AMOUNT = 100; final int REPETITIONS = 100; final int THREADS = 100; for (int i = 1; i <= THREADS; i++) { DepositRunnable d = new DepositRunnable(account, AMOUNT, REPETITIONS); WithdrawRunnable w = new WithdrawRunnable(account, AMOUNT, REPETITIONS); Thread dt = new Thread(d); Thread wt = new Thread(w); dt.start(); wt.start(); }

34. DepositRunnable class template public class DepositRunnable implements Runnable { private static final int DELAY = 1; private BankAccount account; private double amount; private int count; /** Constructs a deposit runnable. @param anAccount the account into which to deposit money @param anAmount the amount to deposit in each repetition @param aCount the number of repetitions */ public DepositRunnable(BankAccount anAccount, double anAmount,int aCount) { account = anAccount; amount = anAmount; count = aCount; } public void run() { try { //TODO: repeatedly deposit count number of times. //between deposits have a delay of 1 millisecond. } catch (InterruptedException exception) {} }

35. WithdrawRunnable class template public class WithdrawRunnable implements Runnable { private static final int DELAY = 1; private BankAccount account; private double amount; private int count; /** Constructs a withdraw runnable. @param anAccount the account from which to withdraw money @param anAmount the amount to withdraw in each repetition @param aCount the number of repetitions */ public WithdrawRunnable(BankAccount anAccount, double anAmount,int aCount) { account = anAccount; amount = anAmount; count = aCount; } public void run() { try { //TODO: repeatedly withdraw count number of times. //between withdrawals have a delay of 1 millisecond. } catch (InterruptedException exception) {} }

36. Answer - DepositRunnable public class DepositRunnable implements Runnable { private static final int DELAY = 1; private BankAccount account; private double amount; private int count; /** Constructs a deposit runnable. @param anAccount the account into which to deposit money @param anAmount the amount to deposit in each repetition @param aCount the number of repetitions */ public DepositRunnable(BankAccount anAccount, double anAmount, int aCount) { account = anAccount; amount = anAmount; count = aCount; } public void run() { try { for (int i = 1; i <= count; i++) { account.deposit(amount); Thread.sleep(DELAY); } catch (InterruptedException exception) {} }

37. Answer - WithdrawRunnable public class WithdrawRunnable implements Runnable { private static final int DELAY = 1; private BankAccount account; private double amount; private int count; /** Constructs a withdraw runnable. @param anAccount the account from which to withdraw money @param anAmount the amount to withdraw in each repetition @param aCount the number of repetitions */ public WithdrawRunnable(BankAccount anAccount, double anAmount, int aCount) { account = anAccount; amount = anAmount; count = aCount; } public void run() { try { for (int i = 1; i <= count; i++) { account.withdraw(amount); Thread.sleep(DELAY); } catch (InterruptedException exception) {} }

38. Expected output Normally, the program output looks somewhat like this: Depositing 100.0, new balance is 100.0 Withdrawing 100.0, new balance is 0.0 Depositing 100.0, new balance is 100.0 Depositing 100.0, new balance is 200.0 Withdrawing 100.0, new balance is 100.0... Withdrawing 100.0, new balance is 0.0 The end result should be zero, but sometimes the output is messed up, and sometimes end result is not zero: Depositing 100.0Withdrawing 100.0, new balance is 100.0, new balance is -100.0

39. What we get: What happened?

40. Scenario to explain problem: 1.A deposit thread executes the lines: System.out.print("Depositing " + amount); double newBalance = balance + amount; The balance variable is still 0, and the newBalance local variable is 100 Statement (balance = newBalance;) is not executed yet. 2.The deposit thread reaches the end of its time slice and a withdraw thread gains control 3.The withdraw thread calls the withdraw method which withdraws $100 from the balance variable; it is now -100 4.The withdraw thread goes to sleep

41. Scenario to explain problem (cont.): 5.The deposit thread regains control and picks up where it was interrupted. It now executes: System.out.println(", new balance is " + newBalance); balance = newBalance; The balance variable is now 100 instead of 0 because the deposit method used the old balance to compute the value of its local variable newBalance

42. Corrupting the Contents of the balance Variable

43. Race Condition  Occurs if the effect of multiple threads on shared data depends on the order in which they are scheduled  It is possible for a thread to reach the end of its time slice in the middle of a statement  It may evaluate the right-hand side of an equation but not be able to store the result until its next turn: public void deposit(double amount) { balance = balance + amount; System.out.print("Depositing " + amount + ", new balance is " + balance); }  Race condition can still occur: balance = the right-hand-side value

44. Synchronizing Object Access To solve problems such as the one just seen, use a lock object Lock object: used to control threads that manipulate shared resources In Java library: Lock interface and several classes that implement it ReentrantLock : most commonly used lock class Locks are a feature of Java version 5.0 Earlier versions of Java have a lower-level facility for thread synchronization

45. Typically, a Lock object is added to a class whose methods access shared resources, like this: public class BankAccount { private Lock balanceChangeLock;... public BankAccount() { balanceChangeLock = new ReentrantLock();... }... }

46. Code that manipulates shared resource is surrounded by calls to lock and unlock : balanceChangeLock.lock(); try { Manipulate the shared resource. } finally { balanceChangeLock.unlock(); } If code between calls to lock and unlock throws an exception, call to unlock never happens. To overcome this problem, place call to unlock into a finally clause

47. Code for deposit method: public void deposit(double amount) { balanceChangeLock.lock(); try { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is "+ newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); }

48. When a thread calls lock, it owns the lock until it calls unlock A thread that calls lock while another thread owns the lock is temporarily deactivated Thread scheduler periodically reactivates thread so it can try to acquire the lock Eventually, waiting thread can acquire the lock

49. Programming Question Synchronize thread access to BankAccount object using locks in our BankAccount class

50. Answer import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; public class BankAccount { private double balance; private Lock balanceChangeLock; public BankAccount() { balance = 0; balanceChangeLock = new ReentrantLock(); } public void deposit(double amount) { balanceChangeLock.lock(); try { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is "+ newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); } } public void withdraw(double amount) { balanceChangeLock.lock(); try { System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); } public double getBalance() { return balance; }

51. Output Correct final output. However, intermediate results show negative balances (WRONG!) We should rather wait till current balance builds up (with some deposits) if withdraw amount> current balance

52. Avoiding Deadlocks A deadlock occurs if no thread can proceed because each thread is waiting for another to do some work first BankAccount example: public void withdraw(double amount) { balanceChangeLock.lock(); try { while (balance < amount) Wait for the balance to grow System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); } }

53. How can we wait for the balance to grow? We can’t simply call sleep inside withdraw method; Why? thread will block all other threads that want to use balanceChangeLock In particular, no other thread can successfully execute deposit Other threads will call deposit, but will be blocked until withdraw exits But withdraw doesn’t exit until it has funds available DEADLOCK withdawThread depositThread

54. Condition Objects To overcome problem, use a condition object Condition objects allow a thread to temporarily release a lock, and to regain the lock at a later time Each condition object belongs to a specific lock object

55. You obtain a condition object with newCondition method of Lock interface: public class BankAccount { private Lock balanceChangeLock; private Condition sufficientFundsCondition;... public BankAccount() { balanceChangeLock = new ReentrantLock(); sufficientFundsCondition = balanceChangeLock.newCondition();... }

56. It is customary to give the condition object a name that describes condition to test; e.g. “sufficient funds” You need to implement an appropriate test

57. As long as test is not fulfilled, call await on the condition object: public void withdraw(double amount) { balanceChangeLock.lock(); try { while (balance < amount) { sufficientFundsCondition.await(); }... } finally { balanceChangeLock.unlock(); }

58. Calling await Makes current thread wait Allows another thread to acquire the lock object To unblock, another thread must execute signalAll on the same condition object : sufficientFundsCondition.signalAll(); signalAll unblocks all threads waiting on the condition

59. Answer import java.util.concurrent.locks.Condition; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; public class BankAccount{ private double balance; private Lock balanceChangeLock; private Condition sufficientFundsCondition; public BankAccount() { balance = 0; balanceChangeLock = new ReentrantLock(); sufficientFundsCondition = balanceChangeLock.newCondition(); } public void deposit(double amount) { balanceChangeLock.lock(); try { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is " + newBalance); balance = newBalance; sufficientFundsCondition.signalAll(); } finally { balanceChangeLock.unlock(); } public void withdraw(double amount) throws InterruptedException { balanceChangeLock.lock(); try { while (balance < amount) { sufficientFundsCondition.await(); } System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); } public double getBalance() { return balance; } }

60. Output (correct!)

61. Programming Question Write a program WordCount that counts the words in one or more files. Start a new thread for each file. For example, if you call java WordCount then the program might print address.txt: 1052 Homework.java: 445 report.txt: 2099 Assuming those files were in current directory. Find code templates for WrdCount.java and WordCountRunnable next slide

62. Code Template - WordCount /** Counts how many words in a file. */ import java.io.File; public class WordCount { public static void main(String argv[]) { File folder = new File("."); //current folder for (File file : folder.listFiles()) { // TODO: create and assign each thread to read a file }

63. Code Template - WordCountRunnable import java.util.Scanner; import java.io.IOException; import java.io.FileNotFoundException; import java.io.File; /** Counts how many words in a file. */ public class WordCountRunnable implements Runnable { private File file; /** Constructs a WordCountRunnable object with a file name. @param aFilename the file name that needs to count word */ public WordCountRunnable(File file) { this.file = file; } public void run() { //TODO: count and print the count of words with file name }

64. Answer /** Counts how many words in a file. */ import java.io.File; public class WordCount { public static void main(String argv[]) { File folder = new File("."); //current folder for (File file : folder.listFiles()) { WordCountRunnable wcr = new WordCountRunnable(file); Thread t = new Thread(wcr); t.start(); }

65. Answer import java.util.Scanner; import java.io.IOException; import java.io.FileNotFoundException; import java.io.File; /** Counts how many words in a file. */ public class WordCountRunnable implements Runnable { private File file; /** Constructs a WordCountRunnable object with a file name. @param aFilename the file name that needs to count word */ public WordCountRunnable(File file) { this.file = file; } public void run() { int count = 0; try { Scanner in = new Scanner(file); while (in.hasNext()) { in.next(); count++; } } catch (FileNotFoundException e) { System.out.println(file.getName() + " not found!"); } System.out.println(file.getName() + ": " + count); }

66. Programming Question Enhance the previous program so that the last active thread also prints a combined count. Use locks to protect the combined word count and a counter of active threads.

67. Answer import java.util.concurrent.locks.ReentrantLock; /** Counts how many words in a file. */ public class WordCount { public static void main(String argv[]) { WordCountRunnable.threadCount = 0; WordCountRunnable.threadCountLock = new ReentrantLock(); WordCountRunnable.combinedWordCount = 0; WordCountRunnable.combinedWordCountLock = new ReentrantLock(); for (int i = 0; i < argv.length; i++) { WordCountRunnable wcr = new WordCountRunnable(argv[i]); Thread t = new Thread(wcr); t.start(); }

68. Answer import java.io.IOException; import java.io.FileNotFoundException; import java.io.FileInputStream; import java.util.Scanner; import java.util.concurrent.locks.Lock; public class WordCountRunnable implements Runnable{ public static int threadCount; public static Lock threadCountLock; public static int combinedWordCount; public static Lock combinedWordCountLock; private String filename; public WordCountRunnable(String aFilename) { filename = aFilename; } public void run() { threadCountLock.lock(); threadCount++; threadCountLock.unlock(); int count = 0; try { Scanner in = new Scanner(new FileInputStream(filename)); while (in.hasNext()) { in.next(); count++; } } catch (FileNotFoundException e) { System.out.println(filename + " not found!"); } catch (IOException e) { System.out.println(e); } System.out.println(filename + ": " + count); combinedWordCountLock.lock(); combinedWordCount += count; threadCountLock.lock(); threadCount--; if (threadCount < 1) System.out.println("Combined Total: " + combinedWordCount); threadCountLock.unlock(); combinedWordCountLock.unlock(); }