1. Concurrent Hashing and Natural Parallelism Chapter 13 in The Art of Multiprocessor Programming Instructor: Erez Petrank Presented by Tomer Hermelin

2. Hash-table Recap Int hash_function(T item) Void add(T item) Void remove(T item) Bool contains(T item) resize and resize policy. Closed address vs open address

3. Agenda ● Closed address:  3 gradually improvements  Lock free model ● Open address  2 gradually improvements

4. Some notations: ● N – Current capacity ● k – Hash-code of the given item ● IC – Initial capacity (constructor argument)

5. Concurrent Closed address

6. Base class - abstract ● Constructor(int capacity): init –  int setSize  array of lists in size capacity ● Contains(T x):  acquire (x)  Checks for x  release (x) ● Add/remove(T x):  acquires (x)  Adds and inc size if not already in list  release (x)  Check policy and resize if needed

7. Name of the game: ● acquire(T x): acquires the locks necessary to manipulate item x. ● release(T x) releases the relevant locks. ● policy() decides whether to resize the set. ● resize() doubles the capacity of the table.

8. Coarse-Grained Hash Set

9. Coarse-Grained The naïve solution: Add one main lock, to lock for each method. The only thing to do: When resize, after locking, make sure no one has already resized. Why shouldn’t we do that for add, remove and contains? Easy to understand and implement. But every thread stops all the other threads…

10. Striped Hash set

11. acquire: given item with hash-code k, we’ll lock the lock in index k (mod IC) Lets say we create a Hash Table with capacity 8, and it was double in size once. Then: Can modify Buckets 0 and 5 in parallel Can’t modify Buckets 0 with two threads in parallel Can’t modify Buckets 0 and 8 in parallel 

12. Locks Table Resizing Save table size Validate table size

13. No Deadlock contains, add, or remove cannot deadlock (also with resize), because they require only one lock to operate. A resize call cannot deadlock with another resize call because both calls start without holding any locks, and acquire the locks in the same order

14. Draw back After multiple resizing there would be large groups of cells that cannot be modified in parallel.

15. Reasons not to grow the locks array? 1. Associating a lock with every table entry could consume too much space, especially when tables are large and contention is low. 2. While resizing the table is straightforward, resizing the lock array (while in use) is more complex.

16. Striped Hash set - summary ● Striped locking permits some concurrency. ● add(), contains(), and remove() methods take constant expected time. ● After multiple resizing, not ideal locks-buckets ratio.

17. Refinable Hash set

18. Propose: Refine the resolution of locking when resizing The main step – Making sure the lock array is not in use, while resizing.

19. Atomic Markable Reference Add AtomicMarkableReference owner We use the owner as a mutual exclusion flag between the resize() call and all other calls (including other resizes)

20. acquire() Locks Table Owner + Validate

21. Resize() Locks Table Owner R R + - C R’ Locks Table

22. Resize() Locks Table Owner R R1R1 R

23. Striped Hash set - summary ● Control over the locks array and table size ratio. ● Resize is ‘stop the world’ method.

24. Lock free Models We want to not “stop-the world” in order to resize, while still doing contains, add, and remove in constant time Atomic operations work only on a single memory location. Resizing is really really not the case. We’ll take care of resizing incrementally, during add, remove and contains.

25. Recursive Split-Ordering

26. A list structure ● All the bucket are part of one long list. ● Add(), remove() and contains() through pointers in table. ● To make our life easy, we make special nodes. ● Initialize when first accessed.

27. The order of the items We want items not to move in resizing! Every item is inserted according to the reverse order of its hash-code bit representation.

28. The order of the items 0010 Size of the table = 2^n 0110 0000 6 42 0 n=1 0 00 n=2 01110001 0 0 8 14 0100 0 0 0 0 111

29. Triggered only by a small action – change bucketSize. The table is in fixed size, and each cell points to the correct ‘logical bucket’ in the list (a pointer is initialized when first accessed). Resize()

30. Adding Example

31. Resizing Example When the capacity is 2, to add item with hash-code = 3, we would be directed by the table with index no. 1. after changing the capacity from 2 to 4, we’ll access for the same item with index no. 3

32. So how do we implement? The list is almost the same as LockFreeList: ● The items are sorted in recursive-split order ● While the LockFreeList class uses only two sentinels, we place a sentinel at the start of each new bucket.

33. So how do we implement? 0 Table 1 2 3 4 5 6 7 AtomicInteger bucketSize AtomicInteger setSize

34. An item inserted before the table was resized must be accessible afterwards from both its previous and current buckets. With our ordering, we ensure that these two groups of items are positioned one after the other in the list. This organization keeps each item in the second group accessible from bucket b. Correctness while Resizing

35. Open-Addressed Hash Set

36. Cuckoo Hashing Some Cuckoos are nest parasites: they lay their eggs in other birds’ nests. Cuckoo chicks hatch early, and quickly push the other eggs out of the nest.

37. Sequential Cuckoo ● Two tables, each with own hash function. ● Remove and contains: simply check in both tables. ● Add method is done by ‘kicking out’ the item in the way and letting him find a new cell. If no free cell can be found, we resize.

38. ● add() ● remove() ● contains() ● relocate() The main problem in making the sequential Cuckoo concurrent is the add method Concurrent Base Class - abstract

39. probe sets: a constant-sized set of items with the same hash code. we use a two-dimensional table of probe sets. 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 2 3 1 0 X 2

40. Concurrent Cuckoo Hashing remove() and contains(): k 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 0 1 1 0 Table 1 Table 0 check

41. Add() 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 0 1 1 0 Table 0 Table 1 threshold k k k k k Resize!! Relocate!! acquire(k)

42. relocation 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 0 1 1 0 Table 0 Table 1 threshold n k s acquire(s) a c b r acquire(a) And we start all over again!

43. Name of the game: ● acquire(T x): acquires the locks necessary to manipulate item x. ● release(T x) releases the relevant locks. ● resize() doubles the capacity of the table. ● policy() decides whether to resize the set.

44. Striped Concurrent Cuckoo Hashing

45. Striped Concurrent Cuckoo Adding a fixed 2-by-L array of reentrant locks As before, lock[i][j] protects table[i][k], where k (mod L) = j 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 0 1 1 0 Table 1 Table 0 Locks 0 Locks 1

46. Still no deadlock The acquire() method locks lock[0][h0(x)] and only then lock[1][h1(x)], to avoid deadlock. When resizing we only acquire the locks in lock[0].

47. Refinable Concurrent Cuckoo Hashing

48. Refinable Concurrent Cuckoo Owner 10 0 1 2 3 4 5 6 7 9 8 11 12 13 14 15 0 1 1 0 Table 1 Table 0 Locks 0 Locks 1

49. The end Questions?