1. java.sun.com/javaone/sf | Concurrency Utilities in Practice 1 Concurrency Utilities in Practice Tim Peierls Prior Artisans, LLC Using java.util.concurrent

2. | Concurrency Utilities in Practice 2 Goals Review concurrency utilities Present several applications Measure and evaluate performance

3. | Concurrency Utilities in Practice 3 Speakers Tim Peierls is a member of the JSR 166 Expert Group

4. | Concurrency Utilities in Practice 4 Examples PseudoRandom Memoize SwingWorker BoundedBuffer WebCrawler

5. | Concurrency Utilities in Practice 5 Concurrency Utilities Executors Executor ExecutorService ScheduledExecutorService Callable Future ScheduledFuture Delayed CompletionService ThreadPoolExecutor ScheduledThreadPoolExecutor AbstractExecutorService Executors FutureTask ExecutorCompletionService Queues BlockingQueue ConcurrentLinkedQueue LinkedBlockingQueue ArrayBlockingQueue SynchronousQueue PriorityBlockingQueue DelayQueue Concurrent Collections ConcurrentMap ConcurrentHashMap CopyOnWriteArray{List,Set} Synchronizers CountDownLatch Semaphore Exchanger CyclicBarrier Timing TimeUnit Locks Lock Condition ReadWriteLock AbstractQueuedSynchronizer LockSupport ReentrantLock ReentrantReadWriteLock Atomics Atomic[Type] Atomic[Type]Array Atomic[Type]FieldUpdater Atomic{Markable,Stampable}Reference

6. | Concurrency Utilities in Practice 6 Caveats Access modifiers omitted unless relevant InterruptedException abbreviated as IE Most interrupt handling omitted for brevity, but very important in real life! Irrelevant methods, fields, and arguments elided with “…”

7. | Concurrency Utilities in Practice 7 PseudoRandom Example PseudoRandom: similar to java.util.Random A simple but broken implementation Reimplement PseudoRandom in different ways Exercise the implementations Compare their performance Overview

8. | Concurrency Utilities in Practice 8 PseudoRandom Example interface PseudoRandom { /** * Return pseudorandom int in range [0, n) */ int nextInt(int n); } PseudoRandom: similar to java.util.Random

9. | Concurrency Utilities in Practice 9 PseudoRandom Example class NaivePseudoRandom implements PseudoRandom { public int nextInt(int n) { int s = seed; seed = computeNext(seed); return s % n; } int computeNext(int s) { return …; } int seed = …; … } A simple but broken implementation

10. | Concurrency Utilities in Practice 10 PseudoRandom Example Fixed using synchronized class PseudoRandomUsingSynch implements PseudoRandom { public synchronized int nextInt(int n) { int s = seed; seed = computeNext(seed); return s % n; } int computeNext(int s) { return …; } int seed = …; }

11. | Concurrency Utilities in Practice 11 PseudoRandom Example Better fix using Lock class PseudoRandomUsingLock implements PseudoRandom { public int nextInt(int n) { lock.lock(); try { int s = seed; seed = computeNext(seed); return s % n; } finally { lock.unlock(); } int computeNext(int s) { return …; } int seed = …; final Lock lock = new ReentrantLock(true); }

12. | Concurrency Utilities in Practice 12 Review: Lock interface Lock { void lock(); void lockInterruptibly() throws IE; boolean tryLock(); boolean tryLock(long timeout, TimeUnit unit) throws IE; void unlock(); … } class ReentrantLock implements Lock { ReentrantLock(boolean fair) {…} } Ensure consistency via exclusion

13. | Concurrency Utilities in Practice 13 Even better fix using AtomicInteger PseudoRandom Example class PseudoRandomUsingAtomic implements PseudoRandom { public int nextInt(int n) { for (;;) { int s = seed.get(); int nexts = computeNext(s); if (seed.compareAndSet(s, nexts)) return s % n; } int computeNext(int s) { return …; } final AtomicInteger seed = new AtomicInteger(…); }

14. | Concurrency Utilities in Practice 14 Review: AtomicInteger  class AtomicInteger extends Number implements Serializable { AtomicInteger(int initialCount) {…} boolean compareAndSet(int expect, int update) {…} boolean weakCompareAndSet(int expect, int update){…} int get(); int getAndSet(int newValue) {…} int getAndAdd(int delta) {…} int addAndGet(int delta) {…} int getAndIncrement() {…} int getAndDecrement() {…} int incrementAndGet() {…} int decrementAndGet() {…} … } Ensure consistency via atomic CAS

15. | Concurrency Utilities in Practice 15 PseudoRandom Example DieRollTask repeatedly rolls a die Coordinate tasks with CountDownLatch Use Executor to run several DieRollTasks Time multiple trials of each implementation Test methodology disclaimer Results Exercise the implementations

16. | Concurrency Utilities in Practice 16 PseudoRandom Example final int NTASKS; // #instances of DieRollTask final int NROLLS; // #die rolls per task instance final PseudoRandom rnd; final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch doneSignal = new CountDownLatch(NTASKS); class DieRollTask implements Runnable { public void run() { startSignal.await(); for (int i = 0; i < NROLLS; ++i) rnd.nextInt(6); doneSignal.countDown(); } /* IE handling omitted */ }; Coordinate tasks with CountDownLatch

17. | Concurrency Utilities in Practice 17 Review: CountDownLatch Waiting threads are released when count becomes 0 class CountDownLatch { CountDownLatch(int initialCount) {…} void countDown(); void await() throws IE; void await(long timeout, TimeUnit unit) throws IE; long getCount(); } Coordinate thread activity

18. | Concurrency Utilities in Practice 18 PseudoRandom Example final Executor threadPool = Executors.newFixedThreadPool(NTASKS); long trial(final int NROLLS, final PseudoRandom r) { … for (int j = 0; j < NTASKS; ++j) threadPool.execute(new DieRollTask()); …give tasks time to wait for startSignal… long start = System.nanoTime(); startSignal.countDown(); // release tasks doneSignal.await(); // wait ‘til done return System.nanoTime() – start; } Use Executor to run several DieRollTasks

19. | Concurrency Utilities in Practice 19 Review: Executor Executor executes submitted tasks interface Executor { void execute(Runnable task); } Executors has static factory methods class Executors { static ExecutorService // extends Executor newFixedThreadPool(int nThreads) {…} …many more… } Decouple task submission from execution

20. | Concurrency Utilities in Practice 20 PseudoRandom Example interface PseudoRandomImpl { PseudoRandom newInstance(); } Collection impls; final int NTRIALS = …; // #trials/implementation final int NROLLS = …; // #die rolls/trial for (PseudoRandomImpl impl : impls) { long nanos = 0; PseudoRandom rnd; for (int k = 0; k < NTRIALS; ++k) { rnd = impl.newInstance(); nanos += trial(NROLLS, rnd); } long avg = nanos / (NTRIALS * NROLLS); if (rnd != null) report(rnd.getClass(), avg); } Time multiple trials of each implementation

21. | Concurrency Utilities in Practice 21 PseudoRandom Example One machine, one JVM, one platform Other things that could affect results: ─Packing of fields on objects ─Cache locality of generated code ─Thresholds for native compilation ─Background user and system processes Test methodology disclaimer

22. | Concurrency Utilities in Practice 22 PseudoRandom Example Results on 3.2 GHz Pentium 4, -client (Windows)

23. | Concurrency Utilities in Practice 23 Memo Function: A function that memorizes its previous results ─Optimization for recursive functions, etc. ─Invented by Prof. Donald Michie, Univ. of Edinburgh Goal: Implement memoizer ─Function wrapper ─Provide concurrent access ─Compute each result at most once Materials: ConcurrentHashMap, FutureTask Implement "Memoizer" Memoize Example

24. | Concurrency Utilities in Practice 24 Memoize Example interface Computable { V compute(A arg) throws Exception; } class Function implements Computable { public BigInteger compute(String arg) { // after deep thought... return new BigInteger("2"); } Function f = new Function(); // f = Memoize(f); f.compute("1 + 1"); >>> 2 Generic computation and example of use

25. | Concurrency Utilities in Practice 25 class Memoize1 implements Computable { final Map cache = new HashMap (); final Computable c; Memoize1(Computable c) { this.c = c; } public synchronized V compute(A arg) throws... { if (!cache.containsKey(arg)) cache.put(arg, c.compute(arg)); return cache.get(arg); } Computable f; f = new Function(); f = new Memoize1 (f); f.compute("1 + 1"); >>> 2 Memoize Example Caching the computed result

26. | Concurrency Utilities in Practice 26 Memoize Example class Memoize2 implements Computable { final Map cache = new ConcurrentHashMap (); final Computable c; Memoize2(Computable c) { this.c = c; } public V compute(A arg) throws Exception { if (!cache.containsKey(arg)) cache.put(arg, c.compute(arg)); return cache.get(arg); } No synchronization, but computes may overlap

27. | Concurrency Utilities in Practice 27 Memoize Example class Memoize3 implements Computable { final Map > cache = new ConcurrentHashMap >(); … public V compute(final A arg) throws Exception { if (!cache.containsKey(arg)) { Callable eval = new Callable () { public V call() throws Exception { return c.compute(arg); } }; FutureTask ft = new FutureTask (eval); cache.put(arg, ft); ft.run(); } return cache.get(arg).get(); } Overlapping computes less likely

28. | Concurrency Utilities in Practice 28 Review: Callables and Futures Callable is invoked and returns a value interface Callable { V call() throws Exception; } Future holds result of asynchronous computation  interface Future { V get() throws InterruptedException, ExecutionException; V get(long timeout, TimeUnit unit) throws IE, …; void cancel(boolean mayInterrupt); boolean isCancelled(); boolean isDone(); } FutureTask is a Runnable Future class FutureTask implements Future, Runnable { FutureTask(Callable c) {…} … } Representing asynchronous tasks

29. | Concurrency Utilities in Practice 29 Memoize Example class Memoize implements Computable { final ConcurrentMap > cache = new ConcurrentHashMap >(); final Computable c; Memoize(Computable c) { this.c = c; } public V compute(final A arg) throws Exception { Future f = cache.get(arg); if (f == null) { Callable eval = new Callable () { public V call() throws Exception { return c.compute(arg); } }; FutureTask ft = new FutureTask (eval); f = cache.putIfAbsent(arg, ft); if (f == null) { f = ft; ft.run(); } } return f.get(); } Exactly one compute() for each argument value

30. | Concurrency Utilities in Practice 30 Inside Memoize Future f = cache.get(arg); if (f == null) { Callable eval = new Callable () { public V call() throws Exception { return c.compute(arg); } }; FutureTask ft = new FutureTask (eval); f = cache.putIfAbsent(arg, ft); if (f == null) { f = ft; ft.run(); } return f.get(); Exactly one compute() for each argument value

31. | Concurrency Utilities in Practice 31 SwingWorker performs GUI-related work in a new thread ─Created by Hans Muller Goal: Reimplement using java.util.concurrent ─Provide more features with less code Materials: FutureTask, Executor Reimplementation of SwingWorker SwingWorker Example

32. | Concurrency Utilities in Practice 32 SwingWorker Revisited abstract class SwingWorker { protected abstract V construct(); protected void finished() { } public void start(); public V get(); } SwingWorker sw = new SwingWorker () { protected String construct() { Thread.sleep(5000); return "Done"; } protected void finished() { label.setText(get()); } }; label.setText("Working..."); sw.start(); Perform GUI-related work in a new thread

33. | Concurrency Utilities in Practice 33 SwingWorker Revisited Working.. Done

34. | Concurrency Utilities in Practice 34 SwingWorker Reimplemented abstract class SwingWorker { FutureTask task = new FutureTask (new Callable () { public V call() throws Exception { return construct(); } }) { protected void done() { EventQueue.invokeLater(new Runnable() { public void run() { finished(); } }); }; protected abstract V construct() throws Exception; protected void finished { } public void start() { new Thread(task).start(); } public V get() throws IE, ExecutionException { return task.get(); }

35. | Concurrency Utilities in Practice 35 Inside SwingWorker FutureTask task = new FutureTask (new Callable () { public V call() throws Exception { return construct(); } }) { protected void done() { EventQueue.invokeLater(new Runnable() { public void run() { finished(); } }); }; Calls construct, then invokes finished

36. | Concurrency Utilities in Practice 36 SwingWorker Revisited abstract class SwingWorker implements Future, Runnable { … public void run() { task.run(); } public V get() throws IE, ExecutionException { return task.get(); } public V get(long timeout, TimeUnit unit) throws... { return task.get(timeout, unit); } public boolean cancel(boolean mayInterruptIfRunning)... public boolean isCancelled()... public boolean isDone()... } Implements Future and Runnable

37. | Concurrency Utilities in Practice 37 SwingWorker Revisited abstract class SwingWorker... { static final Executor EXECUTOR = new Executor() { public void execute(Runnable command) { new Thread(command).start(); } }; private Executor executor = EXECUTOR; public void setExecutor(Executor e)... public Executor getExecutor()... public void start() { executor.execute(this); } } Pluggable Executor

38. | Concurrency Utilities in Practice 38 SwingWorker Revisited Parallel execution in cached thread pool SwingWorker sw; Executor e; e = Executors.newCachedThreadPool(); e.execute(sw); Sequential execution on a single thread e = Executors.newSingleThreadExecutor(); e.execute(sw); Executor choices

39. | Concurrency Utilities in Practice 39 Concurrency utilities enable simple, powerful implementation ─cancellation ─get with timeout ─pluggable executor SwingWorker = Future + Callable + Runnable + Executor Note: Also see http://foxtrot.sourceforge.net Summary SwingWorker Example

40. | Concurrency Utilities in Practice 40 BoundedBuffer Example BoundedBuffer interface Implement BoundedBuffer with Condition Exercise the implementation Compare with other implementations Alternative approach for single producer / single consumer settings using Exchanger Overview

41. | Concurrency Utilities in Practice 41 BoundedBuffer Example interface BoundedBuffer { /** * Adds element to buffer, blocking until * buffer not full */ void put(E element) throws InterruptedException; /** * Removes element from buffer, blocks until * buffer not empty */ E take() throws InterruptedException; } BoundedBuffer interface

42. | Concurrency Utilities in Practice 42 BoundedBuffer Example class BoundedBufferUsingCondition implements BoundedBuffer { final Lock lock = new ReentrantLock(); final Condition notFull = lock.newCondition(); final Condition notEmpty = lock.newCondition(); E[] items; int putIndex, takeIndex, size; public void put(E element) throws IE { lock.lock(); try { while (size == items.length) notFull.await(); items[putIndex++] = element; ++size; if (putIndex == items.length) putIndex = 0; notEmpty.signal(); } finally { /* IE handling omitted */ lock.unlock(); } } // to be continued… Implement BoundedBuffer with Condition

43. | Concurrency Utilities in Practice 43 Review: Condition Condition analogous to Object.wait/notify/notifyAll  interface Condition { void await() throws IE; void await(long timeout, TimeUnit unit) throws IE; void awaitUninterruptibly(); void signal(); void signalAll(); … } Create Conditions with a Lock  interface Lock { … Condition newCondition(); } Multiple wait sets

44. | Concurrency Utilities in Practice 44 BoundedBuffer Example // …continued: public E take() throws InterruptedException { lock.lock(); try { while (size == 0) notEmpty.await(); E element = items[takeIndex++]; --size; if (takeIndex == items.length) takeIndex = 0; notFull.signal(); } finally { /* IE handling omitted */ lock.unlock(); } Implement BoundedBuffer with Condition, cont’d

45. | Concurrency Utilities in Practice 45 WebCrawler Example Application of producer/consumer pattern Implemented using thread pool Problem: Implementation blocks on fetch Solution: Pool runs each fetch as task Wrinkle: Need concurrent collections Wrinkle: Denial of service Resolution: Schedule periodic fetches Refinements Overview

46. | Concurrency Utilities in Practice 46 WebCrawler Example WebCrawler webCrawler = …; BlockingQueue rq = …; // result queue Future crawl = webCrawler.crawl(startUrl, rq); try { while (!foundEnough()) { URL found = rq.poll(timeout, unit); if (found == null) break; process(found); } } finally { crawl.cancel(true); } Application of producer/consumer pattern

47. | Concurrency Utilities in Practice 47 Review: BlockingQueue interface BlockingQueue extends Queue { boolean add(E e); boolean offer(E e); boolean offer(E e, long timeout, TimeUnit unit) throws IE; E poll(long timeout, TimeUnit unit) throws IE; void put(E e) throws IE; E take() throws IE; int remainingCapacity(); int drainTo(Collection elements); int drainTo(Collection elements, int maxElements); } Blocking version of java.util.Queue

48. | Concurrency Utilities in Practice 48 WebCrawler Example class WebCrawler { final ExecutorService pool = Executors.newCachedThreadPool(); Future crawl(URL startUrl, BlockingQueue rq) { return pool.submit(new CrawlTask(startUrl, rq)); } class CrawlTask implements Runnable { /** * Gets URL contents and parses list of URL links * from them; may block indefinitely. */ List fetchLinks(URL url) throws IOException {…} … } Implemented using cached thread pool

49. | Concurrency Utilities in Practice 49 WebCrawler Example class CrawlTask implements Runnable { public void run() { for (seen.add(url); !done(); url = poll()) try { for (URL link : fetchLinks(url)) add(link); if (!rq.offer(url, timeout, unit)) return; } catch (IOException e) { // skip over url, couldn’t get its contents } void add(URL u) { if (!seen.contains(u)) { seen.add(u); pq.offer(u); } } boolean done() { return url == null; } URL poll() { return pq.poll(); } URL url = startUrl; final Set seen = new HashSet (); final Queue pq = new PriorityQueue (…, searchOrder); } Problem: Implementation blocks on fetch

50. | Concurrency Utilities in Practice 50 WebCrawler Example public void run() { for (seen.put(url, true); !done(); url = poll()) pool.execute(new Runnable() { public void run() { try { for (URL link : fetchLinks(url)) add(link); if (!rq.offer(url, timeout, unit)) { done = true; return; } } catch (IOException e) { /* skip */ } } }); } boolean done() { return done || url == null; } volatile boolean done = false; …continued on next slide… Solution: Pool runs each fetch as task

51. | Concurrency Utilities in Practice 51 WebCrawler Example final BlockingQueue pq = new PriorityBlockingQueue (CAP, searchOrder); final ConcurrentMap seen = new ConcurrentHashMap (); void add(URL url) { if (seen.putIfAbsent(url, true) == null) pq.put(url); } URL poll() { url = pq.poll(timeout, unit); } Wrinkle: Need concurrent collections

52. | Concurrency Utilities in Practice 52 Review: PriorityBlockingQueue class PriorityBlockingQueue implements BlockingQueue { PriorityBlockingQueue(int initialCapacity, Comparator comp); … } BlockingQueue with least element available first

53. | Concurrency Utilities in Practice 53 Review: ConcurrentMap interface ConcurrentMap extends Map { V putIfAbsent(K key, V value); V replace(K key, V value); boolean replace(K key, V oldValue, V newValue); boolean remove(K key, V value); } Atomic get-and-maybe-set methods for Map

54. | Concurrency Utilities in Practice 54 WebCrawler Example class PeriodicHostTask implements Runnable { final String host; final Queue hq = new ConcurrentLinkedQueue (); public void run() { // called periodically URL url = hq.poll(); if (url != null) pq.put(url); } void add(URL url) { hq.offer(url); } Wrinkle: Denial of service

55. | Concurrency Utilities in Practice 55 WebCrawler Example final ScheduledExecutorService sched = Executors.newScheduledThreadPool(1); final ConcurrentMap hosts = new ConcurrentHashMap (); void add(URL url) { if (seen.putIfAbsent(url, true) != null) return; PeriodicHostTask hostTask = new PeriodicHostTask(); PeriodicHostTask existing = hosts.putIfAbsent(url.getHost(), hostTask); if (existing == null) { existing = hostTask; sched.scheduleWithFixedDelay( hostTask, 0, 1, TimeUnit.SECONDS); } existing.add(url); } Resolution: Schedule periodic fetches

56. | Concurrency Utilities in Practice 56 Review: ScheduledExecutorService interface ScheduledExecutorService extends ExecutorService { ScheduledFuture schedule(Callable c, long delay, TimeUnit unit); ScheduledFuture schedule(Runnable r, long delay, TimeUnit unit); ScheduledFuture scheduleAtFixedRate(Runnable r, long initDelay, long period, TimeUnit unit); ScheduledFuture scheduleWithFixedDelay(Runnable r, long initDelay, long delay, TimeUnit unit); } Periodic and one-shot delayed tasks

57. | Concurrency Utilities in Practice 57 Summary Don’t use low level constructs when more appropriate high level ones exist, e.g., ─Use Queues, BlockingQueues, or Exchangers in producer/consumer designs. ─Use Executors instead of new Thread(runnable).start() Measuring performance of concurrent programs is hard, but must be done ─Try simple examples first, but remember that they won’t necessarily scale ─Be skeptical; instrument as much as possible to confirm that your program is really working

58. | Concurrency Utilities in Practice 58 For More Information API docs for java.util.concurrent ─In Tiger download or on Sun website Doug Lea's concurrency-interest mailing list ─http://gee.cs.oswego.edu/dl/concurrent/dist/docs/index.html Concurrent Programming in Java The Last Word in Swing Threads ─http://java.sun.com/products/jfc/tsc/articles/threads/threads3.html ─More articles at http://java.sun.com/products/jfc/tsc/articles/

59. | Concurrency Utilities in Practice 59 Q&A 59

60. | Concurrency Utilities in Practice 60 Supplemental Material

61. | Concurrency Utilities in Practice 61 BoundedBuffer Example Comparing three implementations

62. | Concurrency Utilities in Practice 62 BoundedBuffer Example Compare hand-rolled to library implementation

63. | Concurrency Utilities in Practice 63 BoundedBuffer Example May not be significant ─Crude testing methodology (e.g., no warmup) ─Different results obtained using –server If it is significant, a possible explanation: ─HotSpot not caching final fields across code blocks ─Library implementations have been tuned Moral: Don’t reinvent the wheel! ArrayBlockingQueue faster than hand-rolled!

64. | Concurrency Utilities in Practice 64 BoundedBuffer Example final DataBuffer ebuf = …, fbuf = …; final Exchanger exch = new Exchanger(); exec.execute(new Runnable() { // producer DataBuffer buf = ebuf; public void run() { while (buf != null) { putIntoBuffer(buf); if (buf.isFull()) buf = exch.exchange(buf); } } /* IE handling omitted */ }); exec.execute(new Runnable() { // consumer DataBuffer buf = fbuf; public void run() { while (buf != null) { takeFromBuffer(buf); if (buf.isEmpty()) buf = exch.exchange(buf); } } /* IE handling omitted */ }); Alternative for 1 producer / 1 consumer settings

65. | Concurrency Utilities in Practice 65 Review: Exchanger Atomic exchange between threads class Exchanger { T exchange(T x) throws IE; T exchange(T x, long timeout, TimeUnit unit) throws IE; }

66. | Concurrency Utilities in Practice 66 WebCrawler Example Tune thread pools Remove HostTasks after poll failure ─Need ScheduledFuture to cancel tasks Better approach using asynchronous I/O: ─Reduces context switching overhead ─Could use java.nio.channels.Selector wrapped as a CompletionService > ─Fetch requests return immediately ─Separate task polls completion service for next ready list of links Refinements

67. java.sun.com/javaone/sf | Concurrency Utilities in Practice 67 Concurrency Utilities in Practice Tim Peierls Prior Artisans, LLC Using java.util.concurrent