1. Programming Paradigms for Concurrency Lecture 10 Part III – Message Passing Concurrency

2. Message Passing Paradigms Two important categories of MP paradigms: 1.Actor or agent-based paradigms  unique receivers: messages are sent directly from one thread to another 2.Channel-based paradigms – multiple receivers: messages are sent to channels that are shared between threads

3. Channel-based Message Passing threads perform local computations and synchronize via MP over channels channels can be shared between threads (no unique receiver) communication is typically synchronous (blocking send) threads are anonymous threads and channels are created dynamically A A A A B B

4. A Brief History of Channel-Based Paradigms Hoare 1978, 1985: Communicating sequential processes (CSP) Milner 1980: Calculus of communicating systems (CCS) May et al. 1985: Occam language Reppy 1991: Concurrent ML Milner, Parrow, Walker 1992: ¼ calculus Gropp, Lusk, Skjellum 1994: Message passing interface (MPI) Griesemer, Pike, Thompson 2009: Go language

5. Concurrent ML higher-order concurrent language extension of Standard ML part of the SML of New Jersey language distribution: http://www.smlnj.org/http://www.smlnj.org/ first-class events: provides special event type and event combinators  enables building protocol abstractions threads and channels are garbage collected  supports speculative communication

6. Simple Example: Reference Cells

7. signature CELL = sig type 'a cell val cell : 'a -> 'a cell val read : 'a cell -> 'a val write : 'a cell -> 'a -> unit end Simple Example: Reference Cells cell x : creates new cell holding value x cell x : creates new cell holding value x read c : retrieves value stored in cell c read c : retrieves value stored in cell c write c x : updates cell c to hold value x write c x : updates cell c to hold value x

8. Cell implementation each cell has an associated thread holding the stored value each cell encapsulates two channels to its associated thread for updating/retrieving the stored value

9. Cell Implementation datatype 'a request = READ | WRITE of 'a datatype 'a cell = CELL of { reqCh : 'a request chan, replyCh : 'a chan } C 1 (x) C 2 (x) reqCh?WRITE(y) reqCh?READ replyCh!x

10. Implementing read and write fun read (CELL {reqCh, replyCh}) = (send (reqCh, READ); recv replyCh) fun write (CELL {reqCh,...}) x = send(reqCh, WRITE x) C 1 (x) C 2 (x) reqCh?WRITE(y) reqCh?READ replyCh!x send : ('a chan * 'a) -> unit send (ch, x) : send value x over channel ch send : ('a chan * 'a) -> unit send (ch, x) : send value x over channel ch recv : 'a chan -> 'a recv ch : receives a value from channel ch recv : 'a chan -> 'a recv ch : receives a value from channel ch

11. Implementing cell fun cell x = let val reqCh = channel () val replyCh = channel () fun loop x = case recv reqCh of READ => (send (replyCh, x); loop x) | WRITE y => loop y in spawn (fn () => loop x); CELL{reqCh = reqCh, replyCh = replyCh} end C 1 (x) C 2 (x) reqCh?WRITE(y) reqCh?READ replyCh!x channel : unit -> 'a chan channel () : creates a new channel channel : unit -> 'a chan channel () : creates a new channel spawn : (unit -> unit) -> unit spawn f : creates a new thread executing f spawn : (unit -> unit) -> unit spawn f : creates a new thread executing f

12. Selective Communication Allow a process to block on a nondeterministic choice of several blocking communications – the first communication that becomes enabled is chosen – if two or more communications are available simultaneously, then one is chosen nondeterministically.

13. Simplified Cell Implementation datatype 'a cell = CELL of { readCh : 'a chan, writeCh : 'a chan } fun read (CELL{readCh,...}) = recv readCh fun write (CELL{writeCh,...}) = send (writeCh, x) C(x) readCh?y writeCh!x

14. Simplified Cell Implementation C(x) readCh?y writeCh!x fun cell x = let val readCh = channel () val writeCh = channel () fun loop x = select [ wrap (sendEvt(readCh, x), fn () => loop x), wrap (recvEvt writeCh, loop) ] in spawn (fn () => loop x); CELL{readCh = readCh, writeCh = writeCh} end

15. First-Class Synchronous Events

16. First-Class Events Design principle for MP programming use communication primitives to implement complex protocols use abstraction to hide actual communication channels and thus details of thread interactions  implement abstract protocols Problem: standard abstraction mechanisms provided by sequential languages hide too much.

17. Example: RPC Server Server provides service via a remote procedure call (RPC) protocol RPC protocol: – a client must send a request before trying to read a reply – following a request, a client must read exactly one reply before issuing another request

18. Example: RPC Server fun doit request =... (* the actual service *) fun serverLoop () = if serviceAvailable () then let val request = recv reqCh in send (replyCh, doit request); serverLoop () end else doSomethingElse () Problem: if some client does not respect the RPC protocol, then the server may get out of sync.

19. Example: RPC Server fun serverLoop () = if serviceAvailable () then let val request = recv reqCh in send (replyCh, doit request); serverLoop () end else doSomethingElse () Solution: use procedural abstraction to hide protocol details on client side fun clientCall x = (send (reqCh, x); recv replyCh) Problem: client blocks if service is not available  one would like to use selective communication in client, but procedural abstraction hides too much Problem: client blocks if service is not available  one would like to use selective communication in client, but procedural abstraction hides too much

20. First-Class Synchronous Events Concurrent ML provides special event type type ‘a event Events decouple the description of synchronous operations from the actual act of synchronizing on the operation val sync : ‘a event -> ‘a Event combinators enable construction of complex events that implement abstract protocols

21. Primitive Events val recvEvt : ‘a chan -> ‘a event val sendEvt : ‘a chan * ‘a -> unit event recv and send are defined using these primitive events: fun recv ch = sync (recvEvt ch) fun send (ch, x) = sync (sendEvt (ch, x))

22. Post-Synchronization Actions: wrap val wrap : ‘a event * (‘a -> ‘b) -> ‘b event wrap (e, f) : returns an event that when synchronized, synchronizes e and applies f to the result

23. Selective Synchronization: choose val choose : ‘a event list-> ‘a event choose es : returns an event that when synchronized, nondeterministically synchronizes one enabled event in es. fun select es = sync (choose es)

24. Simplified Cell Implementation (revisited) C(x) readCh?y writeCh!x fun cell x = let val readCh = channel () val writeCh = channel () fun loop x = select [ wrap (sendEvt(readCh, x), fn () => loop x), wrap (recvEvt writeCh, loop) ] in spawn (fn () => loop x); CELL{readCh = readCh, writeCh = writeCh} end

25. Example: RPC Client (revisited) fun clientCall x : ‘a -> ‘b option event = wrap (sendEvt (reqCh, x), fn () => SOME (recv replyCh)) fun timeOut () : ‘b option event = let val to = Time.now + Time.fromSeconds 60 in wrap (atTimeEvt to, fn () => NONE) end fun client request : ‘a -> ‘b option = select [clientCall request, timeOut ()]

26. Example: Swap Channels signature SWAP_CHANNEL = sig type ‘a swap_chan val swapChannel : unit -> ‘a swap_chan val swap : (‘a swap_chan * ‘a) -> ‘a event end Swap channels enable symmetric communication between threads where values are exchanged in both directions.

27. Swap Channel Implementation (1 st try) datatype ‘a swap_chan = SC of ‘a chan fun swapChannel () = SC (channel ()) fun swap (SC ch, msgOut) = choose [ wrap (recvEvt ch, fn msgIn => (send(ch, msgOut); msgIn)), wrap (sendEvt (ch, msgOut), fn () => recv inCh) ]

28. Swap mismatch P1P1 P2P2 Q1Q1 Q2Q2 swap(ch, 1) swap(ch, 2) swap(ch, 3) swap(ch, 4) send(ch, 1) recv ch send(ch, 3)recv ch send(ch, 2) recv ch send(ch, 4)recv ch Race on shared channel!

29. Pre-Synchronization Actions: guard val guard : (unit -> ‘a event) -> ‘a event guard f : returns an event that when synchronized, first evaluates f and then synchronizes the resulting event

30. Swap Channel Implementation datatype ‘a swap_chan = SC of (‘a * ‘a chan) chan fun swapChannel () = SC (channel ()) fun swap (SC ch, msgOut) = guard (fn () => let val inCh = channel () in choose [ wrap (recvEvt ch, fn (msgIn, outCh) => (send (outCh, msgOut); msgIn)), wrap (sendEvt (ch, (msgOut, inCh)), fn () => recv inCh) ] end) fresh private channel for each swap event

31. Example: RPC Server (revisited) fun serverLoop () = let val request = recv reqCh in send (replyCh, doit request); serverLoop () end fun clientCall x = guard (fn () => send (reqCh, x); recvEvt replyCh) fun client request = select [clientCall request, timeoutEvent ()]

32. Client Timeout TC1C1 SC2C2 send(reqCh,1) recv reqCh send(replyCh,42) doit 42 atTimeEvt Server S and other clients might deadlock if client C 1 times out send(reqCh,2) deadlock

33. Cancellation: withNack val withNack : (unit event -> ‘a event) -> ‘a event When withNack f is used in a select, the select evaluates f with a fresh abort event nack. The resulting event f nack is then used with the other events in the select. If the select does not choose f nack when it is synchronized, then nack is synchronized, effectively informing f nack that it is not chosen.

34. Example: RPC Server (revisited) fun serverLoop () = let val (request, nack) = recv reqCh in select [sendEvt (replyCh, doit request), nack]; serverLoop () end fun clientCall x = withNack (fn nack => send (reqCh, (x, nack)); recvEvt replyCh) fun client request = select [clientCall request, timeoutEvent ()]

35. Client Timeout TC1C1 SC2C2 send(reqCh,(1,na)) recv reqCh doit 42 atTimeEvt send(reqCh,(2,na)) na recv reqCh

36. More Features of Concurrent ML more event combinators buffered channels multicast channels sync variables built-in RPC support...

37. Implementation of First-Class Events

38. Event Represenation Canonical form of an event is a binary tree with: wrapped base-event values as leaves choice operators as internal nodes Canonical form is obtained using the following equivalences: wrap(wrap(ev,g),f) = wrap(ev, f ± g) wrap(choose(ev 1,ev 2 ),f) = choose(wrap(ev 1,f), wrap(ev 2,f))

39. Event Represenation choose wrap recv wrap choose wrap recv send choose wrap recv wrap recv send choose wrap

40. Base Event Representation pollFn doFn blockFn pollFn doFn blockFn Each base event is a record of three functions: test if base event is enabled synchronize on the enabled base event block calling thread on the base event

41. Protocol for sync Each call to sync from a PCL thread executes the following protocol: 1.Polling Phase: poll the base events using pollFn to see if any of them are enabled 2.Commit Phase: if one or more base events are enabled, pick one and synchronize on it using doFn 3.Blocking Phase: if no base events are enabled: a.enqueue a continuation for the calling thread on each of the base events using its blockFn b.switch to some other thread c.eventually, some other thread will complete the synchronization.

42. First-Class Continuations type ‘a cont val callcc : (‘a cont -> ‘a) -> ‘a val throw : ‘a cont -> ‘a -> ‘b

43. Event Representation datatype event_status = INIT | WAITING | SYNCHED type event_state = event_status ref datatype ‘a evt = BEVT of { pollFn : unit -> bool doFn : ‘a cont -> unit blockFn : (event_state * ‘a cont) -> unit } | CHOOSE of ‘a evt * ‘a evt

44. Implementing sync fun sync ev = callcc (fn resumeK => let fun poll (ev, enabled) =... fun doEvt enabled =... and blockThd () =... in doEvt (poll (ev, [])) end)

45. Implementing sync fun sync ev = callcc (fn resumeK => let fun poll (BEVT{pollFn, doFn,...}, enabled) = if pollFn () then doFn::enabled else enabled | poll (CHOOSE(ev1, ev2)), enabled) = poll(ev2, poll(ev1, enabled)) fun doEvt enabled =... and blockThd () =... in doEvt (poll (ev, [])) end)

46. Implementing sync fun sync ev = callcc (fn resumeK => let fun poll (ev, enabled) =... fun doEvt [] = blockThd () | doEvt (doFn::enabled) = doFn resumeK; (* if doFn terminates then sync attempt failed, so continue *) doEvt enabled and blockThd () =... in doEvt (poll (ev, [])) end)

47. Implementing sync fun sync ev = callcc (fn resumeK => let fun poll (ev, enabled) =... fun doEvt enabled =... and blockThd () = let val flg = ref INIT fun block (BEVT {blockFn,...}) = blockFn (flg, resumeK) block (CHOOSE (ev1, ev2)) = (block ev1; block ev2) in block ev; flg := WAITING; dispatch() end in doEvt (poll (ev, [])) end)

48. Implementing wrap fun wrap (BEVT {pollFn, doFn, blockFn}, f) = BEVT { pollFn = pollFn, doFn = fn k => callcc (fn retK => throw k (f (callcc (fn k’ => doFn k’; throw retK ()))))), blockFn = fn (flg, k) => callcc (fn retK => throw k (f (callcc (fn k’ => blockFn(flg, k’); throw retK ()))))) | wrap (CHOOSE(ev1, ev2), f) = CHOOSE(wrap(ev1, f), wrap(ev2, f))

49. Event Representation datatype ‘a chan = CH of { lock : bool ref, sendq : (‘a * unit cont) queue recvq : (event_state * ‘a cont) queue }

50. Implementing recvEvt fun recvEvt (CH {lock, sendq, recvq}) = let fun pollFn () =... fun doFn k =... fun blockFn (flg : event_state, k) =... in BEVT {pollFn = pollFn, doFn = doFn, blockFn = blockFn} end

51. Implementing recvEvt fun recvEvt (CH {lock, sendq, recvq}) = let fun pollFn () = not(isEmptyQ(sendq)) fun doFn k =... fun blockFn (flg : event_state, k) =... in BEVT {pollFn = pollFn, doFn = doFn, blockFn = blockFn} end

52. Implementing recvEvt fun recvEvt (CH {lock, sendq, recvq}) = let fun pollFn () =... fun doFn k = let val _ = spinLock lock val item = dequeue sendq in spinUnlock lock; case item of NONE => () | SOME(msg, sendK) => (enqueueRdy sendK; throw k msg) end fun blockFn (flg : event_state, k) =... in BEVT {...} end

53. Implementing recvEvt fun recvEvt (CH {lock, sendq, recvq}) = let fun pollFn () =... fun doFn k =... fun blockFn (flg : event_state, k) = spinLock lock; case dequeue sendq of SOME (msg, sendK) => ( spinUnlock lock; flg := SYNCHED; enqueueRdy sendK; throw k msg ) | NONE => ( enqueue (recvq, (flg, k)); spinUnlock lock ) end in BEVT {...} end

54. Further Reading For more details about CML see CML: A higher-order concurrent language. J. Reppy. In PLDI 1991. Concurrent Programming in ML. J. Reppy, Cambridge University Press, 1999. Toward a parallel implementation of Concurrent ML. J. Reppy and Y. Xiao. In DAMP 2008. Parallel Concurrent ML. J. Reppy, C. Russo, and Y. Xiao. In ICFP 2009.

55. Project Proposals 1.Performance/Conciseness Evaluation of Concurrent Programming Paradigms Compare idiomatic implementations of the same algorithm in Scala using different programming paradigms: – a sequential implementation – a concurrent implementation based on actors – a concurrent implementation based on shared memory

56. Project Proposals 2.First-Class Synchronous Events in Scala – Take inspiration from the Scala Actor Library and the CML library – Implement a Scala library supporting first-class synchronous events.

57. Project Proposals 3.Verification of Scala Actor Programs –Implement a non-trivial Scala Actor program or take an existing one –Manually compute finite-state abstractions of all actors –Verify properties of the abstracted program using a formal verification tool for MP programs Please choose your project until 12/22/2010!