1. (Nested) Open Memory Transactions in Haskell
Marino Miculan Marco Peressotti Andrea Toneguzzo
University of Udine
CINA Final Meeting
Civitanova Marche, January 2015

2. Software Transactional Memory
Software transactional memory is a coordination mechanism aiming to solve issues plaguing lock-based coordination mechanisms (semaphores, monitors, barriers…)
Typical form:
atomically { <code> }
guarantees that <code> is executed atomically with respect to every other atomic block (as in ACID database transactions)
Usually implemented by optimistic execution
No deadlocks (no locks), no priority inversions, etc.
STM is implemented in many languages (Java, Haskell, C, C++ …)
close to modern hardware multi-core architectures (e.g., Haswell)
Marino Miculan - UniUD
Nested Open Memory Transactions

3. STM Haskell [Harris et al. PPOPP 2006]
data STM a
-- Sequencing, do notation (>>=) :: STM a -> (a -> STM b) -> STM b return :: a -> STM a
-- Exceptions throw :: Exception e => e -> STM a catch :: Exception e => STM a->(e-> STM a)-> STM a
-- Running isolated and atomic computations atomically :: STM a -> IO a
-- Transactional variables
data TVar a
newTVar :: a -> STM (TVar a) readTVar :: TVar a -> STM a writeTVar :: TVar a -> a -> STM ()
Marino Miculan - UniUD
Nested Open Memory Transactions

4. STM Haskell [Harris et al. PPOPP 2006]
Example: an integer semaphore
check :: Bool -> STM () check p = if p then return () else retry
type Semaphore = TVar Int
up :: Semaphore -> STM () up s = do v <- readTVar s writeTVar s (v + 1)
down :: Semaphore -> STM () down s = do v <- readTVar s check v > 0 writeTVar s (v - 1)
Operations are transactional (STM) and composable
Typing prevents I/O actions inside transactions
Marino Miculan - UniUD
Nested Open Memory Transactions

5. Synchronizations and transactions
Synchronization a la CCS can be implemented with a semaphore pair, and interleaving execution
e.g.: a.P|ā.Q
Typical interaction paradigm of RPC, RMI, XML/RPC…
a.P = down a1;
up a2; P
ā.Q = up a1;
down a2; Q
Marino Miculan - UniUD
Nested Open Memory Transactions

6. Synchronizations and transactions
But what if synchronizations are part of transactions?
atomically do{
down a1;
up a2;
P }
atomically do{
up a1;
down a2;
Q }
Both processes are stuck for none can commit
Marino Miculan - UniUD
Nested Open Memory Transactions

7. Atomicity and isolation in block execution
Atomicity: “all or nothing” semantics; executions may overlap, with uncontrolled access to shared memory
Isolation: “as if it were the only one” semantics; mutual exclusion, but no rollback on errors/exceptions
Isolation No
Yes
Atomicity
Normal blocks IO
Lock (patterns)
2PL, synchronized ?
Transactional Memory
STM Haskell
Open Transactions
OTM Haskell
Marino Miculan - UniUD
Nested Open Memory Transactions

8. Sharing transactional variablesx t2
start k1
start k2
write x
read x
abort
Marino Miculan - UniUD
Nested Open Memory Transactions

9. Sharing transactional variablesx t2
start k1
start k2
write x
k2 depends on data generated by k1
read x
abort
k2 cannot commit until k1 does
Marino Miculan - UniUD
Nested Open Memory Transactions

10. Sharing transactional variablesx t2
start k1
start k2
write x
k2 depends on data generated by k1
read x
abort
k2 cannot commit until k1 does
Communication introduces dependencies
Locking schemes avoid dependencies, but prevent communication
Dependent transactions commit/abort as one
Marino Miculan - UniUD
Nested Open Memory Transactions

11. Nested Open Memory Transactions
Transaction merges t1 x t2
start k1
start k2
write x
read x
Dependent transactions c./a. as one… merge them!
Transactions “claim” locations
Interaction is limited to “claimed” locations
Transaction merge to share locations
Marino Miculan - UniUD
Nested Open Memory Transactions

12. Nested Open Memory Transactions
Transaction merges
k1 claims x,
before writing t1 x t2
start k1
start k2
write x
read x
Dependent transactions c./a. as one… merge them!
Transactions “claim” locations
Interaction is limited to “claimed” locations
Transaction merge to share locations
Marino Miculan - UniUD
Nested Open Memory Transactions

13. Transaction merges Dependent transactions c./a. as one… merge them!
k1 claims x,
before writing t1 x t2
start k1
start k2
k2 merges with k1 in order to access x
write x
k1+k2
read x
Dependent transactions c./a. as one… merge them!
Transactions “claim” locations
Interaction is limited to “claimed” locations
Transaction merge to share locations
Marino Miculan - UniUD
Nested Open Memory Transactions

14. Transaction merges Dependent transactions c./a. as one… merge them!
k1 claims x,
before writing t1 x t2
start k1
start k2
k2 merges with k1 in order to access x
write x
k1+k2
read x
All participants (threads) agree on
commit/abort/retry
Dependent transactions c./a. as one… merge them!
Transactions “claim” locations
Interaction is limited to “claimed” locations
Transaction merge to share locations
Marino Miculan - UniUD
Nested Open Memory Transactions

15. Nested Open Memory Transactions
Formal model
Without nesting: M. Miculan, M. Peressotti, A. Toneguzzo: Open Transactions on Shared Memory. COORDINATION 2015.
With nesting: work in progress
Formal operational semantics
Marino Miculan - UniUD
Nested Open Memory Transactions

16. Nested Open Memory Transactions
Quite few rules… see the paper(s)
Marino Miculan - UniUD
Nested Open Memory Transactions

17. Nested Open Memory Transactions
Formal model
Without nesting: M. Miculan, M. Peressotti, A. Toneguzzo: Open Transactions on Shared Memory. COORDINATION 2015.
With nesting: work in progress
Formal operational semantics
Complete w.r.t. STM Haskell semantics [Harris et al ]
Complete w.r.t. TCCS [Koutavas et al. 2014]
Complete w.r.t. message-passing communicating transactions [Donnelly et al. 2008]
Marino Miculan - UniUD
Nested Open Memory Transactions

18. Nested Open Memory Transactions
Opacity [GK07]
Definition: A TM implementation is opaque if transactions always observe consistent states
Subsumes serializability
Important property for transactional memories
Opacity corresponds to absence of cycles in the dependency graph
Prop: OTM guarantees opacity
Merges collapse cycles
The dependency graph is always acyclic
Works as it is for directed dependency graphs
Marino Miculan - UniUD
Nested Open Memory Transactions

19. Nested Open Memory Transactions
Nested transactions t1 x t2 k1 k2 k3
Marino Miculan - UniUD
Nested Open Memory Transactions

20. Nested open transactionsx t2 k1 k2 k3
Marino Miculan - UniUD
Nested Open Memory Transactions

21. Nesting open transactionsx k1 k2 k3
k3 claims x
Marino Miculan - UniUD
Nested Open Memory Transactions

22. Nesting open transactionsx k1 k2 k3
k3 claims x
k3 commits its effects to k2
x is handed over to k2
Marino Miculan - UniUD
Nested Open Memory Transactions

23. Nesting open transactionsx k1 k2 k3
k3 claims x
k3 commits its effects to k2
x is handed over to k2
k2 aborts, it effects are discarded
Marino Miculan - UniUD
Nested Open Memory Transactions

24. Nesting open transactionsx t2 k1 h1
Create h3 to match the nesting of k3 and merge them.
Merging h1 and k3 brakes nesting k2 k3 h3 h2
k3+h3 commits its effects to k2 and h2
Hence merge k2 and h2
k2+h2 aborts
k1 and h1 do not see any effect
Marino Miculan - UniUD
Nested Open Memory Transactions

25. Atomically, split: OTM/ITM
STM
Isolation
isolated
OTM
atomically
Atomicity
atomic
Consistency, Durability IO IO
OTM: a monad for atomic transactional actions
ITM: a monad for atomic and isolated transactional actions (i.e. a drop-in replacement for STM)
Marino Miculan - UniUD
Nested Open Memory Transactions

26. Nested Open Memory Transactions
OTM and ITM
data ITM a :: * -> * data OTM a :: * -> * -- Sequencing, do notation (>>=) :: t a -> (a -> t b) -> t b return :: a -> t a -- Exceptions throw :: Exception e => e -> t a catch :: Exception e => t a -> (e -> t a) -> t a -- Running isolated and atomic computations atomic :: OTM a -> IO a isolated :: ITM a -> OTM a atomically = atomic . isolated
Marino Miculan - UniUD
Nested Open Memory Transactions

27. Nested Open Memory Transactions
OTM and ITM
-- Blocking retry :: t a orElse :: t a -> t a -> t a -- Threading fork :: OTM () -> OTM ThreadId -- Transactional variables data OTVar a :: * -> * newOTVar :: a -> ITM (OTVar a) readOTVar :: OTVar a -> ITM a writeOTVar :: OTVar a -> a -> ITM ()
Marino Miculan - UniUD
Nested Open Memory Transactions

28. Choosing informative types
Consider the functions g, h:
g = isolated (foldM f)
h = foldM (isolated f)
f :: a -> (OTVar b) -> ITM a
foldM :: Monad m => (a->b->m a)-> a-> [b]-> m a
Same type: a -> [b] -> OTM a
h allows communication inside foldM (hence OTM)
g happens in isolation (should be ITM)
(isolated, atomic are not monad morphisms)
Marino Miculan - UniUD
Nested Open Memory Transactions

29. Nested Open Memory Transactions
Example: semaphores
type Semaphore = OTVar Int up :: Semaphore -> ITM () up s = do x <- readOTVar s writeOTVar s (x+1) down :: Semaphore -> ITM () down s = do x <- readOTVar s check (x > 0) writeOTVar s (x-1)
Marino Miculan - UniUD
Nested Open Memory Transactions

30. Example: synchronizations and transactions
Two-party rendezvous inside open transactions:
atomic $ do
...
isolated (down a1)
isolated (up a2) P
atomic $ do
...
isolated (up a1)
isolated (down a2) Q
Transactions merge to interact via a1
More than two participants but, they sync in pairs
Implements TCCS
Marino Miculan - UniUD
Nested Open Memory Transactions

31. Example: Petri nets Multiple transitions can fire at the same time
type Place = Semaphore put = up take = down transition :: [Place] -> [Place] -> IO () transition ins outs = fork (forever fire) where fire = atomic $ do mapM_ (isolated . take) ins mapM_ (isolated . put) outs
Firing is atomic
Single updates are isolated
Marino Miculan - UniUD
Nested Open Memory Transactions

32. Nested Open Memory Transactions
Example: Petri nets
Marino Miculan - UniUD
Nested Open Memory Transactions

33. Example: Dining Philosophers
Marino Miculan - UniUD
Nested Open Memory Transactions

34. Conclusion and further work
Open memory transactions in Haskell
transactions may interact at runtime
interaction are implicit by access to shared memory
greater space/time decoupling
atomicity and isolation have their own monads
formal semantics (COORDINATION 2015)
implementation in Haskell (no dep. on GHC)
drop-in replacement for STM
Future work
Better integration with the Haskell runtime
Transaction invariants (smarter scheduling)
Axiomatization (effect algebras)
Marino Miculan - UniUD
Nested Open Memory Transactions

35. Thanks for your attention.