# Refactoring to Concurrency (Java Edition)

## Presentation on theme: "Refactoring to Concurrency (Java Edition)"— Presentation transcript:

Refactoring to Concurrency (Java Edition)
Prepared for: Boston Java Meetup Group February 2011 Presented at Coverity; Copyright 2010 Concurrency Magic

Overview Step 1: Establish Performance Criteria Step 2: Annotate Existing Class Thread Safety Step 3: Increase Thread Safety (as needed) Step 4: Use Tasks instead of loops/steps Step 5: Use Thread Pool Step 6: Increase Concurrency Copyright 2010, Concurrency Magic

Common Concurrency Problems
Data Race Condition Deadlock Liveliness Memory Visibility And, “Heisenbugs” POINT: Much can be statically checked ** CDROM MAGIC TRICK ** Race Condition: Atomic operations Lock (synchronized) Spin Locks Versioning Lock-free (LoadLinked/StoreConditional or CompareAndSwap) Dirty read, phantom Read Lost locks (no release) Deadlock: Recovery Time-out and retry Lock ordering (multi-tier) Open Calls Liveliness: Starvation, Priority Inversion Heisenbugs as opposed to bohrbug Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

Solving Sudoku Puzzles
Example Problem Solving Sudoku Puzzles

3 There are 12 “Houses” 4 Rows 4 Columns 4 Boxes Each Value (1 - 4) must appear exactly once in each House Algorithm: Look for “Singles” (Cells where only one value is valid) 2 1 1 3 4 2 2 1 3 4 4 2 1 3

1 2 X * 1 X X X * X X Thread Algorithm: 4 Threads. Each considers one value Mark which cells cannot have that value Look for Singles for that value 2 X X X 1 X X 2 X X X 1 X 2 X X 3 4 3 * X X X * 4 X X X 4 X X X 3 X X X X X 4 X X X X X

Alternative Algorithm #1: Thread per cell
2 1 2 1 2 1 2 Thread Algorithm: 16 threads each consider one cell and its 4 “candidates“ Look at neighbor cells in 3 “houses” Remove invalid candidates 3 4 3 4 3 4 3 4 1 2 1 2 1 3 3 4 3 4 1 2 1 2 1 2 4 3 4 3 4 3 4 1 2 1 2 1 2 3 3 4 3 4 3 4

Alternative Alogorithm #2: Thread per “House”
3 4 1 4 2 1 3 4 2 2 3 1 4 3 3 4 1 2 1 4 2 1 4 2 4 2 3 3 4 1 2 1 3 1 4 4 2 1 2 2 1 3 4 2 1

Alternative Algorithm #3: Thread per “Candidate”
1 2 1 2 1 2 1 2 Thread Algorithm: 64 Threads. Each considers one candidate Look at neighbor cells in 3 “houses” Remove if invalid 3 4 3 4 3 4 3 4 1 2 1 2 1 3 3 4 3 4 1 2 1 2 1 2 4 3 4 3 4 3 4 1 2 1 2 1 2 3 3 4 3 4 3 4

Package Architecture Copyright 2010, Concurrency Magic

Step 1: Establish Criteria
“If you don’t know where you’re going, how will you know when you gotten there?”

Success Criteria Examples
Good Examples: Response time < 1ms for typical queries Process >100 queries per second Deterministic Results Bad Examples: Keep all processors busy Keep all threads busy Avoid waiting for locks Copyright 2010, Concurrency Magic

Performance Test Copyright 2010, Concurrency Magic

Timing Java Apps (1st attempt)
Start timer Run algorithm A Stop timer Report elapsed time Run algorithm B . . . Run algorithm Z Copyright 2010, Concurrency Magic

Timing Java Apps (2nd attempt)
Repeat 1000 times Start timer Repeat 100 times (with 100 different puzzles) Run algorithm A Stop timer Report min/ave/max times (divided by 100,000) Run algorithm B . . . Copyright 2010, Concurrency Magic

Timing Java Apps (3rd attempt)
Repeat 1000 times Start timer A Repeat 100 times: Run algorithm A Stop timer A Start timer B Repeat 100 times: Run algorithm B Stop timer B . . . Start timer Z Repeat 100 times: Run algorithm Z Stop timer Z Report min/ave/max (divided by 100,000) Copyright 2010, Concurrency Magic

Timing Java Apps (4th attempt)
Start JVM Repeat 1000 times Start timer Repeat 100 times (with 100 different puzzles) Run algorithm A Stop timer Report min/ave/max times (divided by 100,000) Stop JVM Repeat for each algorithm Copyright 2010, Concurrency Magic

Statistics Gathering Best Time Ave Time Nickname Algorithm ms ms STP Pencil ms ms STS Scanner Performance results solving 100 puzzles 1000 times. Sample Goal: cut time in half using 4 cores Copyright 2010, Concurrency Magic

Step 2: Annotate Type Safety
Assign Type Safety Annotation to every class that might be used concurrently

Concurrency Categories for Classes

New Package: Annotations

Value @Immutable public enum Value { UNKNOWN(0, '.'), ONE(1, '1'), TWO2, '2'), FIFTEEN (15, 'F'), SIXTEEN (16, 'G'); final private int number; final private char name; final static private Map<Character,Value> charMap = new HashMap<Character, Value>(); final static private List<Value> intList = new ArrayList<Value>(16); . . . IMMUTABLE Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

Location @ThreadNeutral public class Location { private int row; private int col; private int offset; private int boxSize; . . . ThreadNeutral Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

ValueSet @ThreadNeutral public class ValueSet { private Value value = Value.UNKNOWN; private EnumSet<Value> includedValues; . . . ThreadNeutral Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

Puzzle @ThreadHostile public class Puzzle implements Cloneable { public static final List<Integer> SUPPORTED_SIZES = Arrays.asList(4, 9, 16); static int size = 0; static int boxSize = 0; static int gridSize = 0; private static List<List<Location>> rows; private static List<List<Location>> cols; private static List<List<Location>> boxes; private static List<List<Location>> houses; private static List<Location> allLocations; private List<Value> values; . . . ThreadHostile Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

Step 3: Increase Thread Safety (as needed)
Modify classes or create new classes with higher levels of atomicity Copyright 2010, Concurrency Magic

Location final public class Location { final private int row; final private int col; final private int offset; final private int boxSize; // Getters but not Setters Now IMMUTABLE Copyright 2010, Concurrency Magic Presented at Coverity; Copyright 2010 Concurrency Magic

Puzzle Hierarchy Copyright 2010, Concurrency Magic

Puzzle (Base Class) public abstract class Puzzle implements Cloneable { /** List of Puzzle sizes supported (4x4, 9x9 and 16x16). */ public static final List<Integer> SUPPORTED_SIZES = Arrays.asList(4, 9, 16); /** Puzzle table used when this Puzzle was created. */ protected final PuzzleTables tables; /** Size of this Puzzle (4, 9 or 16). */ protected final int size; Copyright 2010, Concurrency Magic

@ThreadNeutral public final class BasicPuzzle extends Puzzle { /** Table containing current Values for each cell in grid. */ private final List<Value> values; Copyright 2010, Concurrency Magic

BasicPuzzle get/set @Override public Value getValue (Location loc) { return values.get(loc.getOffset()); } public void setValue (Location loc, Value v) { values.set(loc.getOffset(), v); Copyright 2010, Concurrency Magic

SynchronizedPuzzle @ThreadSafe public class SynchronizedPuzzle extends Puzzle private final List<Value> public synchronized Value getValue (Location loc) { return values.get(loc.getOffset()); } public synchronized void setValue (Location loc, Value v) { values.set(loc.getOffset(), v); Copyright 2010, Concurrency Magic

LockingPuzzle.LockableValue
@ThreadSafe public final class LockingPuzzle extends Puzzle { /** Value that can be locked */ private class LockableValue { private Value value; public LockableValue() { value = Value.UNKNOWN;} public synchronized Value get() { return value; } public synchronized void set(Value v) { value = v; } } Copyright 2010, Concurrency Magic

LockingPuzzle get/set
@GuardedBy("itself") private final List<LockableValue> public Value getValue (Location loc) { return values.get(loc.getOffset()).get(); } public void setValue (Location loc, Value v) { values.get(loc.getOffset()).set(v); Copyright 2010, Concurrency Magic

AtomicPuzzle @ThreadSafe public final class AtomicPuzzle extends Puzzle { AtomicReferenceArray<Value> public Value getValue (Location loc) { return values.get(loc.getOffset()); } public void setValue (Location loc, Value v) { values.set(loc.getOffset(), v); . . . Copyright 2010, Concurrency Magic

ImmutablePuzzle @Immutable public final class ImmutablePuzzle extends Puzzle { private final List<Value> public Value getValue (Location loc) { return values.get(loc.getOffset()); } public void setValue (Location loc, Value v) { throw new IllegalArgumentException( "setValue is not implemented for ImmutablePuzzle"); Copyright 2010, Concurrency Magic

New class: GenericScanSolver
public class ScanSolver extends Solver { /** Puzzle currently being solved. */ final BasicPuzzle puzzle; ... public class GenericScanSolver extends Solver { final Puzzle puzzle; Copyright 2010, Concurrency Magic

Use SynchronizedPuzzle
ST_SCANNER_WTH_SYNCHRONIZED_PUZZLE("STS.S") { public String getName() { return "ST Scanner with SynchronizedPuzzle"; } public Solver newSolver(Puzzle input, Verbosity verbosity) { Puzzle p = SynchronizedPuzzle.makeCopy(input); Solver s = GenericScanSolver.newSolver(p, verbosity); return s; } }, Copyright 2010, Concurrency Magic

Use LockingPuzzle ST_SCANNER_WTH_LOCKING_PUZZLE("STS.L") { public String getName() { return "ST Scanner with LockingPuzzle"; } public Solver newSolver(Puzzle input, Verbosity verbosity) { Puzzle p = LockingPuzzle.makeCopy(input); Solver s = GenericScanSolver.newSolver(p, verbosity); return s; } }, Copyright 2010, Concurrency Magic

Creating Executor private boolean runAllTests() { // Create pool of threads to execute tasks ExecutorService exec = Executors.newSingleThreadExecutor(); try { for (PuzzleResults r : results) { if (runOneTest(exec, r) != true) { // Report error return false; } } finally { exec.shutdown(); return true; Copyright 2010, Concurrency Magic

Pass (unused) Executor
public static Solver newSolver(Executor exec, Puzzle input, Verbosity verbosity) { return new ScanSolver(input, verbosity); } Copyright 2010, Concurrency Magic

protected void seekHiddenSingles() { final CountDownLatch latch = new CountDownLatch(unsolvedValues.count()); for (final Value v : unsolvedValues.getAllValues()) { Runnable task = new Runnable() { public void run() { boolean foundAll = seekOneHiddenSingle(v); if (foundAll) { unsolvedValues.removeValue(v); } latch.countDown(); }; exec.execute(task); try { latch.await(); ... Error handling Copyright 2010, Concurrency Magic

Creating Thread Pool private boolean runAllTests() { // Create pool of threads to execute tasks ExecutorService exec = Executors.newFixedThreadPool(numberOfThreads); try { for (PuzzleResults r : results) { if (runOneTest(exec, r) != true) { // Report error return false; } } finally { exec.shutdown(); return true; Copyright 2010, Concurrency Magic

Step 6: Increase Concurrency
In this case, rather than solve one Puzzle in parallel, solve different Puzzles in parallel Copyright 2010, Concurrency Magic

Concurrent Sudoku #5: Thread per puzzle
3 4 3 1 3 4 2 4 1 2 3 4 1 2 1 4 1 2 1 2 3 4 4 3 2 2 4 1 2 1 3 2 1 3 2 3 4 4 3 3 4 2 1 4 1

class PuzzleSolver implements Callable<Solvability> { private final SolverType solverType; private final PuzzleInfo info; public PuzzleSolver (SolverType s, PuzzleInfo info) { … } public Solvability call () { … } } Copyright 2010, Concurrency Magic

protected boolean solveOneSetOfPuzzles(SolverType s) { ExecutorService exec = Executors.newSingleThreadExecutor(); for (PuzzleInfo info : puzzles) { Callable<Solvability> task = new PuzzleSolver(s, info); exec.submit(task); } exec.shutdown(); // Error handling Copyright 2010, Concurrency Magic