1. Threads Written by Amir Kirsh, Dr. Yaron Kanza. Edited by Liron Blecher

2. Agenda Threads Overview Creating threads in Java Synchronization wait() and notify() Notes Thread Pools

3. 3 Threads Overview – Threads allow the program to run tasks in parallel – If two threads execute the same method, each will have its own copy of the local variables the methods uses – In many cases threads need to be synchronized, that is, be kept not to handle the same data in memory concurrently – There are cases in which a thread needs to wait for another thread before proceeding – Threads have access to the same memory, meaning, they can change the same objects

4. Agenda Threads Overview Creating threads in Java Synchronization wait() and notify() Notes Thread Pools

5. 5 Threads in Java The operation we want to be threaded: public class PrintNumbers { static public void printNumbers() { for(int i=0; i<1000; i++) { System.out.println( Thread.currentThread().getId() + ":" + i); }

6. 6 Threads in Java Option 1 – extending class Thread: public class Thread1 extends Thread { @Override public void run() { System.out.println("Thread1 ThreadId: " + Thread.currentThread().getId()); // do our thing PrintNumbers. printNumbers(); }

7. 7 Threads in Java Option 1 – extending class Thread (cont’): static public void main(String[] args) { System.out.println("Main ThreadId: " + Thread.currentThread().getId()); for(int i=0; i<3; i++) { new Thread1().start(); // don't call run! // (if you want a separate thread) } printNumbers(); }

8. 8 Threads in Java Option 2 – implementing Runnable: public class MyRunnable implements Runnable { @Override public void run() { System.out.println("Thread2 ThreadId: " + Thread.currentThread().getId()); // do our thing PrintNumbers. printNumbers(); }

9. 9 Threads in Java Option 2 – implementing Runnable (cont’): static public void main(String[] args) { System.out.println("Main ThreadId: " + Thread.currentThread().getId()); for(int i=0; i<3; i++) { new Thread(new MyRunnable()).start(); // again, don't call run! // (if you want a separate thread) } printNumbers(); }

10. 10 Threads in Java Option 3 – implementing Runnable as Anonymous: static public void main(String[] args) { System.out.println("Main ThreadId: " + Thread.currentThread().getId()); new Thread(new Runnable() { @Override public void run() { System.out.println("Thread3 ThreadId: " + Thread.currentThread().getId()); // do our thing printNumbers(); } }).start(); // don't call run!... printNumbers(); }

11. DEMO examples.threads.intro 11

12. Agenda Threads Overview Creating threads in Java Thread Scheduling Synchronization wait() and notify() Notes Thread Pools

13. 13 Thread Scheduling – Usually, threads run one at a time – Unless several processors are being used – Thread execution is merged in order to provide concurrency

14. I/O operation completes start() Currently executed thread Ready queue Waiting for I/O operation to be completed Waiting to be notified Sleeping Waiting to enter a synchronized section Newly created threads Blocked queue Thread Scheduling 14

15. Alive NewDead Running Runnable new PlusPrinter(); run() method returns for (…) { … } Blocked Object.wait() Thread.sleep() blocking IO call waiting on a monitor thread.start(); Don’t get confused with Runnable interface Thread.getState() Thread Scheduling 15

16. Thread scheduling is the mechanism used to determine how Runnable threads (in the ready queue) are allocated CPU time Java scheduling is preemptive, priority based Each thread has a priority - an integer number Use thread.getPriority()/setPriority() to control priorities In principle, the runtime system chooses the thread that has the highest priority Priority has range 1-10 What is the priority of the main thread? Thread Scheduling 16

17. If several threads have the same priority, an arbitrary one is chosen Scheduling may violate the priority-based policy to avoid starvation of low-priority threads Java's scheduling also uses time slicing, when it is supported by the operating system Mainly used for sharing CPU among equal highest- priority threads Thread Scheduling 17

18. A thread runs until one of the following occurs: The thread dies (e.g., run() completes) The thread becomes not Runnable (e.g., sleeps or waits for an IO operation to complete) A higher-priority thread becomes Runnable On systems that support time-slicing, its time allotment has expired The thread yields (discussed later) Thread Scheduling 18

19. Scheduling is OS Dependent! Java maps Java threads to OS threads In particular, Java relies on the operating system for Time slicing Priorities Thus, scheduling differs from one system to another Do not count on scheduling and priorities for algorithm correctness! 19

20. Relinquishing the CPU A running thread can explicitly relinquish the CPU for other threads to use The static method Thread.yield() temporarily pauses the currently executing thread and allows other threads to execute A thread can block for a while using the method Thread.sleep(milliseconds) 20

21. Daemon Threads There are two types of threads: daemon threads (like the garbage-collection thread) non-daemon threads (like the thread running main()) JVM will let all non-daemon threads complete (i.e., complete the execution of run()) When only daemon threads stay alive, they are killed and JVM exits Controlled by thread.isDaemon() and thread.setDaemon() 21

22. Agenda Threads Overview Creating threads in Java Thread Scheduling Synchronization wait() and notify() Notes Thread Pools

23. 23 Synchronization Synchronization of threads is needed for in order to control threads coordination, mainly in order to prevent simultaneous operations on data For simple synchronization Java provides the synchronized keyword For more sophisticated locking mechanisms, starting from Java 5, the package java.concurrent.locks provides additional locking options, see: http://docs.oracle.com/javase/6/docs/api/java/util/concurrent /locks/package-summary.html http://docs.oracle.com/javase/6/docs/api/java/util/concurrent /locks/package-summary.html

24. Monitors are key elements in Java's thread synchronization Every object has a monitor An object's monitor is used as a guardian that watches a block of code (called a critical section) and enables only one thread to enter that code To enter a critical section, a thread must first acquire an ownership over the corresponding monitor Synchronization 24

25. Only one thread can own a specific monitor If a thread A tries to enter a block under a monitor and a different thread B has already entered that block, A will wait until B releases the monitor and (hopefully) that monitor will be passed to A Hence, monitors are related to as locks When a thread leaves the critical section, the monitor is automatically released Threads awaiting a monitor are blocked and queued Synchronization 25

26. 26 public class SynchronizedCounter { private int c = 0; public synchronized void increment() { c++; } public synchronized void decrement() { c--; } public synchronized int value() { return c; } } Synchronization Example 1 – synchronizing methods: The synchronized keyword on a method means that if this is already locked anywhere (on this method or elsewhere) by another thread, we need to wait till this is unlocked before entering the method

27. 27 public void addName(String name) { synchronized(this) { lastName = name; nameCount++; } nameList.add(name); } Synchronization Example 2 – synchronizing blocks: When synchronizing a block, key for the locking should be supplied (usually would be this) The advantage of not synchronizing the entire method is efficiency

28. 28 public class TwoCounters { private long c1 = 0, c2 = 0; private Object lock1 = new Object(); private Object lock2 = new Object(); public void inc1() { synchronized(lock1) { c1++; } public void inc2() { synchronized(lock2) { c2++; } Synchronization Example 3 – synchronizing using different locks: You must be absolutely sure that there is no tie between c1 and c2

29. DEMO examples.threads.deadlock 29

30. 30 public class Screen { private static Screen theScreen; private Screen(){…} // private c’tor public static synchronized Screen getScreen() { if(theScreen == null) { theScreen = new Screen(); } return theScreen; } Synchronization Example 4 – synchronizing static methods: This is a Singleton example It is not the most efficient way to implement Singleton in Java

31. 31 Synchronization Example 4 – synchronizing static methods … Having a static method be synchronized means that ALL objects of this type are locked on the method and can get in one thread at a time. The lock is the Class object representing this class. The performance penalty might be sometimes too high – needs careful attention!

32. 32 public class Screen { private static Screen theScreen = new Screen(); private Screen(){…} // private c’tor public static getScreen() { return theScreen; } Synchronization Example 4’ – a better singleton: No synchronization

33. Agenda Threads Overview Creating threads in Java Thread Scheduling Synchronization wait() and notify() Notes Thread Pools

34. 34 wait(), notify(), notifyAll() wait() and notify() allows a thread to wait for an event A call to notifyAll() allows all threads that are on wait() with the same lock to be released A call to notify() allows one arbitrary thread that is on a wait() with the same lock to be released Read: (a) http://java.sun.com/docs/books/tutorial/ essential/concurrency/guardmeth.html (b) http://java.sun.com/javase/6/docs/api/ java/lang/Object.html#wait()http://java.sun.com/docs/books/tutorial/ essential/concurrency/guardmeth.htmlhttp://java.sun.com/javase/6/docs/api/ java/lang/Object.html#wait() Instead of “busy wait” or sleep loop!

35. wait(), notify(), notifyAll() Suppose that an object has some monitor, but conditions disable it from completing the critical section The wait/notify mechanism enables that object to release the monitor and wait until conditions are changed 35

36. wait() The method Object.wait() requires the current thread to own the monitor of the object When called, the current thread releases ownership on the object's monitor stops and waits until some other thread will wake it up and the monitor will be re-obtained 36

37. notify() Like wait, requires the object to own the monitor The method Object.notify() wakes up an arbitrary thread that waits on the monitor of the object Object.notifyAll() wakes all such threads When a thread is waken up, it regularly waits for the monitor to be available (since it called Object.wait()) The thread calling notify should release the monitor for the waiting thread to continue (e.g exit the synchronized scope) 37

38. 38 public class Drop { // Message sent from producer to consumer private String message; // A flag, True if consumer should wait for // producer to send message, False if producer // should wait for consumer to retrieve message private boolean empty = true;... wait(), notify(), notifyAll() Example (from http://java.sun.com/docs/books/tutorial/essential/concurrency/example/Drop.java ) : Flag must be used, never count only on the notify

39. 39 public class Drop {... public synchronized String take() { // Wait until message is available while (empty) { // we do nothing on InterruptedException // since the while condition is checked anyhow try { wait(); } catch (InterruptedException e) {} } // Toggle status and notify on the status change empty = true; notifyAll(); return message; }... } wait(), notify(), notifyAll() Example (cont’) Must be in synchronized context

40. 40 public class Drop {... public synchronized void put(String message) { // Wait until message has been retrieved while (!empty) { // we do nothing on InterruptedException // since the while condition is checked anyhow try { wait(); } catch (InterruptedException e) {} } // Toggle status, store message and notify consumer empty = false; this.message = message; notifyAll(); }... } wait(), notify(), notifyAll() Example (cont’) Must be in synchronized context

41. DEMO examples.threads.producerconsumer 41

42. Agenda Threads Overview Creating threads in Java Thread Scheduling Synchronization wait() and notify() Notes Thread Pools

43. Notes Threads inherit their priority and daemon properties from their creating threads The method thread.join() blocks and waits until the thread completes running A thread can have a name for identification Stopping a running thread was possible in old versions of Java, but it is now deprecated Instead, interruption mechanisms should be used ( thread.interrupt() ) 43

44. Notes isInterupted() – can be used by a thread to check if another thread has been interrupted isAlive() – can be used by a thread to check if another thread is alive join() – allows one thread to wait for the completion of another 44

45. DEMO examples.threads.poetthread 45

46. 46 Notes Some more reading material: http://java.sun.com/docs/books/tutorial/essential/threads/index.html http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Thread.html

47. Agenda Threads Overview Creating threads in Java Thread Scheduling Synchronization wait() and notify() Notes Thread Pools

48. What is a Thread Pool? A collection of threads that are created once (e.g. when a server starts) That is, no need to create a new thread for every client request Instead, the server uses an already prepared thread if there is a free one, or waits until there is a free thread 48

49. Why Using Thread Pools? Thread pools improve resource utilization The overhead of creating a new thread is significant Thread pools enable applications to control and bound their thread usage Creating too many threads in one JVM can cause the system to run out of memory and even crash There is a need to limit the usage of system resources such as connections to a database 49

50. Thread Pools in Servers Thread pools are especially important in client- server applications The processing of each individual task is short-lived and the number of requests is large Servers should not spend more time and consume more system resources creating and destroying threads than processing actual user requests When too many requests arrive, thread pools enable the server to force clients to wait until threads are available 50

51. The “Obvious” Implementation There is a pool of threads Each task asks for a thread when starting and returns the thread to the pool after finishing When there are no available threads in the pool the thread that initiates the task waits till the pool is not empty What is the problem here? “Synchronized” model - the client waits until the server takes care of its request… 51

52. The “Obvious” Implementation is Problematic When the pool is empty, the submitting thread has to wait for a thread to be available We usually want to avoid blocking that thread A server may want to perform some actions when too many requests arrive Technically, Java threads that finished running cannot run again 52

53. Task Queue Worker Threads wait () If Q is Empty All the worker threads wait for tasks A Possible Solution Every thread looks for tasks in the queue 53

54. Task Queue Worker Threads Task The number of worker threads is fixed. When a task is inserted to the queue, notify is called A Possible Solution “A-synchronized” model: “Launch and forget” 54

55. Task Queue Worker Threads Task The number of worker threads is fixed. When a task is inserted to the queue, notify is called A Possible Solution notify() 55

56. Task Queue Worker Threads A Possible Solution The task is executed by the thread 56

57. Task Queue Worker Threads A Possible Solution The task is executed by the thread The remaining tasks are executed by the other threads 57

58. Task Queue Worker Threads A Possible Solution When a task ends, the thread is released While the Q is not empty, take the task from the Q and run it (if the Q was empty, wait() would have been called) 58

59. Task Queue Worker Threads A Possible Solution A new task is executed by the released thread 59

60. Threads can leak A thread can endlessly wait for an I/O operation to complete For example, the client may stop the interaction with the socket without closing it properly What if task.run() throws a runtime exception (as opposed to other exceptions that a programmer of a client application has to catch in order to succeed compiling)? Solutions: Bound I/O operations by timeouts using wait(time) Catch possible runtime exceptions 60 Risks in Using Thread Pools

61. Pool Size What is better: to have a large pool or a small pool? Each thread consumes resources memory, management overhead, etc. A large pool can cause starvation Incoming tasks wait for a free thread A small pool can cause starvation Therefore, you have to tune the thread pool size according to the number and characterizations of expected tasks There should also be a limit on the size of the task queue (why?) 61

62. Executor The class Executors has 2 static methods to create thread pools ExecutorService newFixedThreadPool(int nThreads) Pool of a fixed size ExecutorService newCachedThreadPool() Creates new threads as needed New threads are added to the pool, and recycled ExecutorService has an execute method void execute(Runnable command) 62

63. 63 Thread Pools Prevent the thread-per-session pitfall! Class ThreadPoolExecutor: http://java.sun.com/javase/6/docs/api/java/util/concurrent/ ThreadPoolExecutor.html http://java.sun.com/javase/6/docs/api/java/util/concurrent/ ThreadPoolExecutor.html

64. DEMO examples.threads.threadexecutor 64