1. Reducing Interprocess Communication Overhead in Concurrent Programs Erik Stenman Kostis Sagonas

2. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 2 Motivation Concurrency as an abstraction is important.  Systems that need to interact with the outside world are hard to model without concurrency. Unfortunately concurrency costs.  There are two types of runtime overheads:  ”Direct overhead” of concurrency primitives.  ”Indirect overhead” from hiding the data-flow from the optimizing compiler.

3. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 3 Goal of this work Reduce the overhead of concurrency in concurrent programs. Idea Optimize the code that implements process communication. We call this interprocess optimization, and we will present three techniques: 1. Rescheduling send. 2. Direct dispatch send 3. Interprocess inlining.

4. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 4 Rescheduling Send Typically, when a process sends a message, it is because it wants the receiver to act on that message. It is therefore in the interest of the sender to yield to the receiver and allow it to promptly act on the sent message. We call this type of send, a rescheduling send.

5. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 5 Implementation: The send operation suspends the currently active (sending) process. Benefits:  lower message passing latency.  better cache behavior (the receiver has access to the message while it is still hot in the cache). Rescheduling Send

6. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 6 Direct Dispatch Send The sender contributes the remaining part of its time-slice to the receiver (hoping this would lead to a faster response). The receiver then is woken up directly (ignoring the ready queue).  Overhead of the scheduler is eliminated. If the receiver also performs a direct dispatch send back to the sender, interprocess communication becomes as fast as a function call.

7. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 7 Interprocess Inlining Merge code from the sender with code from the receiver.

8. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 8 Process  The receiver Process  The sender A message

9. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 9

10. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 10

11. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 11 Known communication protocol. Can be optimized. Process  only needs to be ready to receive the communication.

12. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 12 The state of process  has changed. Without really participating in the actual communication.

13. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 13 Interprocess Inlining Merge code from the sender with code from the receiver. In the merged code, the overhead of the communication protocol can be optimized away. We suggest using profiling to find situations where this is feasible. This requires that the optimized code is guarded by a runtime test.

14. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 14 An example in Erlang… ref_server(V) -> receive {‘++’,From}-> From ! {self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From ! {self(),NewV}, ref_server(NewV); {‘=’,From}-> From ! {self(),V}, ref_server(V) end inc(Beta) -> Beta ! {‘++’,self()}, receive {Beta,Answer} -> Answer; end.

15. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 15 …could be rewritten as ref_server(V) -> receive {‘++’,From}-> From ! {self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From ! {self(),NewV}, ref_server(NewV); {‘=’,From}-> From ! {self(),V}, ref_server(V) end inc´(Beta) -> if ready(Beta)-> B_V=get(‘V’,Beta)+1, save(‘V’,B_V,Beta), B_V; true -> inc(Beta) end.

16. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 16 Code merging: the general case  g  - Head Create message Send  -Tail g - Head Receive g – tail ’’  - Head Create message Send  -Tail Extracted g – tail Ready Copy Restore β-State Save β-State Copy of  -Tail Yes

17. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 17 Code merging Can only be done if:  Code explosion does not occur.  The code does not suspend.  The control flow does not escape the included code.  The extracted code terminates (o/w, the sending process might hang after code merging.)

18. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 18 Code merging These instructions are not extracted:  A call or a return.  Instructions that lead to the suspension of the process.  Throwing an exception.  Some BIFs are large and uncommon and not worth the effort to adapt for external execution. Non-terminating code is unacceptable.  Hence, we do not accept any loops in the control flow graph.

19. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 19 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). Create message tail sen d f(Beta) -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end.

20. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 20 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). Test f-tail f(Beta) -> Mess = {hi,self()}, if ready(Beta)-> NEWCODE ; true -> Beta ! Mess, receive {Beta,fine} -> right; {Beta,_Answer} -> wrong end end.

21. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 21 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}], case  -Mbox of [{hi,  -From}]->  -From !  -{  -self(),fine}; _ ->  -ignore  -end, save_state(Beta), COPY-OF-f-TAIL

22. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 22 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}], case [{hi,self()}]of [{hi,  -From}]->  -From !  -{  -self(),fine}; _ ->  -ignore  -end, save_state(Beta), COPY-OF-f-TAIL

23. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 23 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}],  -From = self(),  -From !  -{  -self(),fine},  save_state(Beta), COPY-OF-f-TAIL

24. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 24 Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}],  -From = self(), self()!  -{  -self(),fine},  save_state(Beta), COPY-OF-f-TAIL

25. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 25 Return messages An important special case Handled by applying the same type of rewriting. The ready test is replaced by a test that checks that both mailboxes are empty.

26. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 26 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}],  -From = self(), self()!  -{  -self(),fine},  save_state(Beta), COPY-OF-f-TAIL

27. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 27 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = [{hi,self()}],  -From = self(),  -Mbox = {  -self(),fine}, save_state(Beta), COPY-OF-f-TAIL

28. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 28 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = {  -self(),fine}, save_state(Beta), COPY-OF-f-TAIL

29. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 29 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = {  -self(),fine}, save_state(Beta), case  -Mbox of {Beta,fine} -> right; {Beta,_Other} -> wrong end

30. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 30 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = {  -self(),fine}, save_state(Beta), case {  -self(),fine} of {Beta,fine} -> right; {Beta,_Other} -> wrong end

31. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 31 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE :  -Mbox = {  -self(),fine}, save_state(Beta), right

32. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 32 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE : save_state(Beta), right

33. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 33 More Code merging g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f´(Beta) -> if ready(Beta)-> save_state(Beta), right; true -> f(Beta) end.

34. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 34 This optimization gives Access to variables in another process with almost no overhead. (Two reads, one test, and two writes.) The overhead of the communication protocol can be reduced. ( Creating a tuple and switching on it. )

35. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 35 Profiling We have profiled parts of Eddie and the AXD code and found:  That each send goes to one particular receive.  The receiving process is typically suspended with an empty mailbox. The same profiling tool could be used in a dynamic compiler to find pairs of senders/receivers whose inter-process communication can be optimized.

36. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 36 Conclusions Presented several different methods for cross-process optimization that reduce the overhead of interprocess communication. No modifications to existing code are required. Open issue is their integration into the Erlang development environment

37. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 37 Praeterea censeo "0xCA" scribere Erlang posse. Happi That’s all folks!

38. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 38 Problem How does one find senders and receivers that communicate?  Static analysis  Problematic in a language with dynamic typing, dynamic process creation, communication through mailboxes, and asynchronous communication.  Profiling & dynamic recompilation

39. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 39 Interprocess inlining Do:  Find two communicating processes.  Merge code from the sender with code from the receiver.  Optimize the merged code. Get:  Reduced message passing.  Short-circuited code for switching on messages.

40. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 40 An example in Erlang… g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f(Beta) -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end.

41. Informationsteknologi ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 41 …could be rewritten as g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f´(Beta) -> if ready(Beta)-> save_state(Beta), right; true -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end end.