Semantics of Multithreaded Java Jeremy Manson and William Pugh Background Material Jack Newton University of Alberta

Semantics of Multithreaded Java 2 Overview  Multithreaded Programming in Java  Memory Consistency Models  Java Memory Model  Problems with the Java Memory Model

Semantics of Multithreaded Java 3 Multithreaded Programming in Java  Java is the first widely used programming language to provide support for concurrent programming at the language level  Java supports concurrency through the use of threads, and provides mechanisms for synchronization between threads

Semantics of Multithreaded Java 4 Java Threads: Example  This example will illustrate the basic principles of multithreaded programming in Java sum int a int b int c sumEditor Sum sum sumPrinter Sum sum

Semantics of Multithreaded Java 5 Java Threads: Example (Continued) public class Sum { int a, b, c; int a, b, c; public Sum() { public Sum() { } public String toString() { public String toString() { return this.a + " + " + this.b + " = " + this.c; return this.a + " + " + this.b + " = " + this.c; }}

Semantics of Multithreaded Java 6 Java Threads: Example (Continued) public class SumEditor extends Thread { Sum sum; Sum sum; public SumEditor(Sum _sum) { public SumEditor(Sum _sum) { this.sum = _sum; this.sum = _sum; } public void run() { public void run() { for (int i = 1; i < 10; i++) { for (int i = 1; i < 10; i++) { this.sum.a = i; this.sum.a = i; this.sum.b = i; this.sum.b = i; this.yield(); this.yield(); this.sum.c = this.sum.a + this.sum.b; this.sum.c = this.sum.a + this.sum.b; } }}

Semantics of Multithreaded Java 7 Java Threads: Example (Continued) public class SumPrinter extends Thread { Sum sum; Sum sum; public SumPrinter(Sum _sum) { public SumPrinter(Sum _sum) { this.sum = _sum; this.sum = _sum; } public void run() { public void run() { for (int i = 1; i < 10; i++) { for (int i = 1; i < 10; i++) { System.out.println(sum); System.out.println(sum); this.yield(); this.yield(); } }}

Semantics of Multithreaded Java 8 Java Threads: Example (Continued) public class ThreadingExperiment { public ThreadingExperiment() { public ThreadingExperiment() { } public static void main(String[] args) { public static void main(String[] args) { Sum sum = new Sum(); Sum sum = new Sum(); SumEditor sumEditor = new SumEditor(sum); SumEditor sumEditor = new SumEditor(sum); SumPrinter sumPrinter = new SumPrinter(sum); SumPrinter sumPrinter = new SumPrinter(sum); System.out.println("main: starting threads"); System.out.println("main: starting threads"); sumEditor.start(); sumEditor.start(); System.out.println("main: started sumEditor thread"); System.out.println("main: started sumEditor thread"); sumPrinter.start(); sumPrinter.start(); System.out.println("main: started sumPrinter thread"); System.out.println("main: started sumPrinter thread"); System.out.println("main: exiting"); System.out.println("main: exiting"); }}

Semantics of Multithreaded Java 9 Java Threads: Example (Continued)  Output from this example main: starting threads main: started sumEditor thread main: started sumPrinter thread main: exiting = = = = = = = = = 16

Semantics of Multithreaded Java 10 Java Threads: Example (Continued)  What’s going wrong? sumEditorsumsumPrinter for (int i = 1; i < 10; i++) { this.sum.a = i; this.sum.a = i; this.sum.b = i; this.sum.b = i; this.yield(); this.yield(); this.sum.c = this.sum.a this.sum.c = this.sum.a + this.sum.b; + this.sum.b;} a b c for (int i = 1; i < 10; i++) { System.out.println(sum); System.out.println(sum); this.yield(); this.yield();}         =

Semantics of Multithreaded Java 11 Java Threads: Synchronization  Java provides two methods to synchronize threads Synchronized methods Synchronized methods Synchronized statements Synchronized statements  Both of these methods use object locks to enforce mutual exclusion  Only one thread can have a lock on an object at a time  If a thread attempts to obtain a lock on an object that is already blocked, the thread will block

Semantics of Multithreaded Java 12 Java Threads: Synchronization (Continued)  Synchronized Methods Object to obtain lock on is implicit Object to obtain lock on is implicit Instance of class the synchronized method appears in for non-static methodsInstance of class the synchronized method appears in for non-static methods Class method the synchronized method appears in for static methodsClass method the synchronized method appears in for static methods public synchronized Object getValue()  Synchronized Statements Object to obtain lock on is explicit Object to obtain lock on is explicit synchronized (object) { …}

Semantics of Multithreaded Java 13 Java Threads: Synchronization (Continued) Thread 1 Thread 2 Thread 3 Thread 4 object Lock Blocked object Blocked Lock

Semantics of Multithreaded Java 14 Java Threads: Synchronized Example public class SumEditor extends Thread { Sum sum; Sum sum; public SumEditor(Sum _sum) { public SumEditor(Sum _sum) { this.sum = _sum; this.sum = _sum; } public void run() { public void run() { for (int i = 1; i < 10; i++) { for (int i = 1; i < 10; i++) { synchronized (sum) { synchronized (sum) { this.sum.a = i; this.sum.a = i; this.sum.b = i; this.sum.b = i; this.yield(); this.yield(); this.sum.c = this.sum.a + this.sum.b; this.sum.c = this.sum.a + this.sum.b; } } }}

Semantics of Multithreaded Java 15 Java Threads: Synchronized Example (Continued) public class SumPrinter extends Thread { Sum sum; Sum sum; public SumPrinter(Sum _sum) { public SumPrinter(Sum _sum) { this.sum = _sum; this.sum = _sum; } public void run() { public void run() { for (int i = 1; i < 10; i++) { for (int i = 1; i < 10; i++) { synchronized (sum) { synchronized (sum) { System.out.println(sum); System.out.println(sum); this.yield(); this.yield(); } } }}

Semantics of Multithreaded Java 16 Java Threads: Synchronized Example (Continued)  Impact of synchronization sumEditorsumsumPrinter for (int i = 1; i < 10; i++) { synchronized (sum) { synchronized (sum) { this.sum.a = i; this.sum.a = i; this.sum.b = i; this.sum.b = i; this.yield(); this.yield(); this.sum.c = this.sum.a this.sum.c = this.sum.a + this.sum.b; + this.sum.b; }} a b c         =   for (int i = 1; i < 10; i++) { synchronized (sum) { synchronized (sum) { System.out.println(sum); System.out.println(sum); this.yield(); this.yield(); }}

Semantics of Multithreaded Java 17 Memory Models  When we discuss memory models, we really mean memory consistency models  A memory consistency model should provide a formal specification of how the memory system behaves  This memory model essentially constrains the possible values that a read of a variable can return

Semantics of Multithreaded Java 18 Memory Models (Continued)  Memory models allow a programmer to reason about the potential outcomes of both synchronized and unsynchronized programs  Memory models take on special significance with Java, where data races could be used to compromise security

Semantics of Multithreaded Java 19 Memory Models: Sequential Consistency  Sequential consistency provides an intuitive model of memory consistency  Lamport defines a system to be sequentially consistent if: “the result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by the program”

Semantics of Multithreaded Java 20 Memory Models: Sequential Consistency (Continued) … P1P1 P2P2 P3P3 P4P4 PNPN Main Memory

Semantics of Multithreaded Java 21 Memory Models: Sequential Consistency (Continued) … P1P1 P2P2 P3P3 P4P4 PNPN Main Memory

Semantics of Multithreaded Java 22 Strong / Relaxed Memory Models  Strong memory models, such as sequential consistency, severely restrict the set of potential optimizations that can be applied For example, compiler optimizations such as code motion, register allocation, and common sub- expression elimination are disallowed under sequential consistency [Maessen 2000] For example, compiler optimizations such as code motion, register allocation, and common sub- expression elimination are disallowed under sequential consistency [Maessen 2000]  Relaxed (or weak) memory models relax memory ordering constraints to allow for certain optimizations

Semantics of Multithreaded Java 23 Memory Models: Coherence  Coherence is an example of a relaxed memory model  Coherence is similar to sequential consistency, with the following relaxation: For a given processor/thread, the program order is maintained on a per-location basis For a given processor/thread, the program order is maintained on a per-location basis

Semantics of Multithreaded Java 24 Memory Models: Coherence … P1P1 P2P2 Main Memory a b c d P2P2 PNPN R(a) R(b) W(a) W(b) R(a) R(b) R(c) W(a) R(b) W(d) R(a) W(d) R(a) W(c) W(b)

Semantics of Multithreaded Java 25 Java Memory Model  The Java Memory Model (JMM) is defined in Chapter 17 of the Java Language Specification  The JMM is hard to understand Ambiguous in places Ambiguous in places Complicated enough that even the authors of the Java Language Specification didn’t completely understand it Complicated enough that even the authors of the Java Language Specification didn’t completely understand it  All current JVM implementations violate the specification to some degree

Semantics of Multithreaded Java 26 Java Memory Model: Terminology  A variable refers to Static variable of a loaded class Static variable of a loaded class Field of an allocated object Field of an allocated object Element of an allocated array Element of an allocated array  All variables are stored in a main memory that is shared by all threads  Every thread has a working memory where it keeps working copies of variables

Semantics of Multithreaded Java 27 Java Memory Model Main Memory Thread 1 … Working Memory Execution Engine Buffer Thread 2 Working Memory Execution Engine Buffer Thread N Working Memory Execution Engine Buffer

Semantics of Multithreaded Java 28 Java Memory Model (Continued) Main Memory Thread 1 … Working Memory Execution Engine Buffer Thread 2 Working Memory Execution Engine Buffer Thread N Working Memory Execution Engine Buffer use read assign load write store

Semantics of Multithreaded Java 29 Java Memory Model (Continued) Main Memory Thread 1 … Working Memory Execution Engine Buffer Thread 2 Working Memory Execution Engine Buffer Thread N Working Memory Execution Engine Buffer lock/unlock

Semantics of Multithreaded Java 30 Java Memory Model (Continued) 3 1 Main Memory Thread 1 … Working Memory Execution Engine Buffer Thread 2 Working Memory Execution Engine Buffer Thread N Working Memory Execution Engine Buffer lock unlock

Semantics of Multithreaded Java 31 Java Memory Model: Example  When Thread 1 and Thread 2 have finished executing, can we have: a = 1, b = 1?  Answer: Yes! Thread 2Thread 1 x = y = 0 a = x y = 1 b = y x = 1 Threads Begin      Jan-Willem Maessen, Arvind, Xiaowei Shen. Improving the Java Memory Model using CRF. In Proceedings of the 15th Annual Conference on Object-Oriented Programming Systems, Languages and Applications, Portland, OR, Oct

Semantics of Multithreaded Java 32 Java Memory Model: Prescient Stores  The JMM allows prescient stores: storing the value that will be assigned to a variable by a future assign operation before the assign occurs  The prescient store of a variable V is allowed in thread T if the following conditions are satisfied: If the store on V occurs, the assign is bound to occur If the store on V occurs, the assign is bound to occur No lock operation occurs between the store and assign No lock operation occurs between the store and assign No load of V occurs between the store and assign No load of V occurs between the store and assign No other store of V occurs between the store and assign No other store of V occurs between the store and assign

Semantics of Multithreaded Java 33 Java Memory Model: Example (Continued) Thread 2 x = y = 0 a = x y = 1 b = y x = 1 Threads Begin Thread 1 load x use x assign y store y write y read y load y use y assign x store x write x Main Memory … Working Memory Execution Engine Buffer use read assign load write store Thread 1 Thread 2 read x

Semantics of Multithreaded Java 34 Java Memory Model: Example (Continued) load x use x assign y store y write y read y load y use y assign x store x write x Thread 1 Thread 2 read x write y read x load x use x assign y read y load y use y assign x store x write x Thread 1 Thread 2 store y

Semantics of Multithreaded Java 35 Java Memory Model: Volatile  Variables declared volatile are treated specially: Read from main memory on every use Read from main memory on every use Written to main memory on every assign Written to main memory on every assign Accesses to volatiles are sequentially consistent (i.e. prescient stores not allowed) Accesses to volatiles are sequentially consistent (i.e. prescient stores not allowed) Access to longs/doubles are atomic Access to longs/doubles are atomic  The memory semantics of accessing a volatile are similar to accessing an object exclusively through synchronized get() and set() methods

Semantics of Multithreaded Java 36 Java Memory Model: Final  Fields declared as final can only be assigned to once in the constructor for the class in which they are defined The Java Memory Model does not discuss final fields at all! The Java Memory Model does not discuss final fields at all! Several optimizations for final fields are obvious, but are not permitted under the current JMM Several optimizations for final fields are obvious, but are not permitted under the current JMM

Semantics of Multithreaded Java 37 Java Memory Model: Summary  The JMM is difficult to understand – research papers that discuss the JMM interpret it differently  The JMM is at least as strong as Coherence  William Pugh has shown that the JMM is actually stronger than Coherence

Semantics of Multithreaded Java 38 Problems with the Java Memory Model  Three main problems exist with the current Java Memory Model: It prohibits many important compiler optimizations It prohibits many important compiler optimizations It is incomplete: the semantics for final fields are not defined It is incomplete: the semantics for final fields are not defined It is hard to interpret It is hard to interpret

Semantics of Multithreaded Java 39 Problems with the JMM: Compiler Optimizations  The JMM requires Coherence, which prohibits many important compiler optimizations  A compiler would not be permitted to perform fetch elimination w = q.f x = p.f v = p.f p.f = 2 Thread 1Thread 2 If p == q at runtime (p and q are aliased) w = p.f x = p.f v = p.f p.f = 2 Thread 1Thread 2 x = v

Semantics of Multithreaded Java 40 Problems with the JMM: Semantics of Final  The current JMM does not provide any discussion of final fields  Final fields are another potential source of optimizations that cannot be performed under the current JMM  Current semantics may result in immutable objects appearing mutable

Semantics of Multithreaded Java 41 Problems with the JMM: Complexity  The three stages presented in the JMM make it difficult to reason about the order of memory operations Main Memory Working Memory Execution Engine Buffer use read assign load write store Thread

Semantics of Multithreaded Java 42  The current JMM is complex and restrictive enough that all current JVM implementations violate it to some degree  Pugh suggests that the JMM, in its current form, will never be clear. New proposals for semantics, such as those in Semantics of Multithreaded Java, should offer a completely new description Problems with the JMM: Complexity (Continued)

Semantics of Multithreaded Java 43 References  Sarita V. Adve and Kourosh Gharacholoo. Shared Memory Consistency Models: A Tutorial. Rice University ECE Technical Report  James Gosling, Bill Joy, Guy Steele, Gilad Bracha. The Java Language Specification, Second Edition.  David Holmes. The Java Memory Model: Defining Concurrency Correctness on Multiprocessors. holmes.ppt.  Jan-Willem Maessen, Arvind, Xiaowei Shen. Improving the Java Memory Model using CRF. In Proceedings of the 15th Annual Conference on Object- Oriented Programming Systems, Languages and Applications, Portland, OR, Oct David Mosberger. Memory Consistency Models. University of Arizona Technical Report 93/11.  William Pugh. Fixing the Java Memory Model. Java Grande Conference, June 12-14,  William Pugh. The Java Memory Model is Fatally Flawed. Concurrency: Practice and Experience.