1. SUMMARY ID1218 Lecture 122009-12-07 Christian Schulte cschulte@kth.se Software and Computer Systems School of Information and Communication Technology KTH – Royal Institute of Technology Stockholm, Sweden

2. Overview  Only functional programming summary  C++ summary and example questions, see Lecture 10 L12 2009-12-07ID1218, Christian Schulte 2

3. Functional Programming L12 2009-12-07ID1218, Christian Schulte 3  Evaluate functions returning results executed by MiniErlang machine  Techniques recursion with last-call-optimization pattern matching list processing higher-order programming accumulators

4. MiniErlang Machine L12 2009-12-07 4 ID1218, Christian Schulte

5. MiniErlang: Values, Expressions, Instructions  MiniErlang values V := int | [] | [ V 1 | V 2 ]  MiniErlang expressions E := int | [] | [ E 1 | E 2 ] | X | F(E 1,…, E n ) where X stands for a variable  MiniErlang instructions CONS, CALL L12 2009-12-07ID1218, Christian Schulte 5

6. MiniErlang Machine  MiniErlang machine Es ; Vs → Es’ ; Vs’ transforms a pair (separated by ;) of expression stack Es and value stack Vs into a new pair of expression stack Es’ and value stack Vs’ according to topmost element of Es  Initial configuration: expression we want to evaluate on expression stack  Final configuration: single value as result on value stack L12 2009-12-07ID1218, Christian Schulte 6

7. MiniErlang: Expressions  Evaluate values V  Er ; Vs → Er ; V  Vs provided V is a value  Evaluate list expression [ E 1 | E 2 ]  Er ; Vs → E 1  E 2  CONS  Er ; Vs  Evaluate function call F(E 1, …, E n )  Er ; Vs → E 1  …  E n  CALL( F/n )  Er ; Vs L12 2009-12-07ID1218, Christian Schulte 7

8. MiniErlang: Instructions  CONS instruction CONS  Er ; V 1  V 2  Vs → Er ; [ V 2 | V 1 ]  Vs  CALL instruction CALL( F/n )  Er ; V 1  …  V n  Vs → s(E )  Er ; Vs F(P 1, …, P n ) -> E first clause of F/n such that [ P 1, …, P n ] matches [ V n, …, V 1 ] with substitution s L12 2009-12-07ID1218, Christian Schulte 8

9. MiniErlang Pattern Matching  Patterns P :=int | [] | [ P 1 | P 2 ] | X  match(P,V) s:=try(P,V) if error  s or X  V 1, X  V 2  s withV 1 ≠V 2 then no else s where try( i, i ) =  try( [], [] ) =  try( [ P 1 | P 2 ], [ V 1 | V 2 ] )= try(P 1,V 1 )  try(P 2,V 2 ) try(X,V) = {X  V} otherwise= {error} L12 2009-12-07ID1218, Christian Schulte 9

10. Substitution  Defined over structure of expressions s( i ) = i s( [] ) = [] s( [ E 1 | E 2 ] ) = [ s(E 1 ) | s(E 2 ) ] s(F(E 1, …, E n )) = F(s(E 1 ), …, s(E n )) s(X)= if X  V  s then V else X L12 2009-12-07ID1218, Christian Schulte 10

11. Iterative Computations L12 2009-12-07ID1218, Christian Schulte 11

12. A Better Length? L12 2009-12-07ID1218, Christian Schulte l([]) -> 0; l([_|Xr]) -> 1+l(Xr). l([],N) -> N; l([_|Xr],N) -> l(Xr,N+1).  Two different functions: l/1 and l/2  Which one is better? 12

13. Running l/1 l([1,2,3]) ;  → [1,2,3]  CALL(l/1) ;  → CALL(l/1) ; [1,2,3] → 1+l([2,3]) ;  → 1  l([2,3])  ADD ;  → l([2,3])  ADD ; 1 → [2,3]  CALL(l/1)  ADD ; 1 → CALL(l/1)  ADD ; [2,3]  1 → 1+l([3])  ADD ; 1 → 1  l([3])  ADD  ADD ; 1 → l([3])  ADD  ADD ; 1  1 → …  requires stack space in the length of list L12 2009-12-07ID1218, Christian Schulte 13

14. Running l/2 l([1,2,3],0) ;  → [1,2,3]  0  CALL(l/2) ;  → 0  CALL(l/2) ; [1,2,3] → CALL(l/2) ; 0  [1,2,3] → l([2,3],0+1) ;  → [2,3]  0+1  CALL(l/2) ;  → 0+1  CALL(l/2) ; [2,3] → 0  1  ADD  CALL(l/2) ; [2,3] → 1  ADD  CALL(l/2) ; 0  [2,3] → ADD  CALL(l/2) ; 1  0  [2,3] → CALL(l/2) ; 1  [2,3] → l([3],1+1) ;  → …  requires constant stack space! L12 2009-12-07ID1218, Christian Schulte 14

15. L12 2009-12-07ID1218, Christian Schulte Iterative Computations  Iterative computations run with constant stack space  Make use of last optimization call correspond to loops essentially  Tail recursive functions computed by iterative computations 15

16. Accumulators Making Computations Iterative L12 2009-12-07 16 ID1218, Christian Schulte

17. L12 2009-12-07ID1218, Christian Schulte Recursive Function pow(_,0) -> 1; pow(X,N) -> X*pow(X,N-1).  Is not tail-recursive not an iterative function uses stack space in order of N 17

18. L12 2009-12-07ID1218, Christian Schulte Using Accumulators  Accumulator stores intermediate result  Finding an accumulator amounts to finding an invariant recursion maintains invariant invariant must hold initially result must be obtainable from invariant 18

19. L12 2009-12-07ID1218, Christian Schulte State Invariant for pow/2  The invariant is X N = pow(X,N- i ) * X i where i is the current iteration  Capture invariant with accumulator for X i initially (i=0)1= X 0 finally (i= N ) X N 19

20. L12 2009-12-07ID1218, Christian Schulte The pow/3 Function pow(_,0,A) -> A; pow(X,N,A) -> pow(X,N-1,X*A). pow(X,N) -> pow(X,N,1). 20

21. L12 2009-12-07ID1218, Christian Schulte Naïve Reverse Function rev([]) -> []; rev([X|Xr]) -> app(rev(Xr),[X]).  Some abbreviations r(Xs) for rev(Xs) Xs ++ Ys for app(Xs,Ys) 21

22. L12 2009-12-07ID1218, Christian Schulte Invariant for rev  Now it is easy to see that rev([ x 1, …, x n ]) = rev([ x i+1, …, x n ]) ++ [ x i, …, x 1 ] as we go along 22

23. L12 2009-12-07ID1218, Christian Schulte rev Final rev([],Ys) -> Ys; rev([X|Xr],Ys) -> rev(Xr,[X|Ys]).  Is tail recursive now  Cost: n+1 calls for list with n elements 23

24. Higher-Order Programming L12 2009-12-07 24 ID1218, Christian Schulte

25. L12 2009-12-07ID1218, Christian Schulte Generic Procedures  Sorting a list in increasing or decreasing order by number or phone book order (maybe even a Swedish phone book!) sorting algorithm + order  Mapping a list of numbers to list of square numbers to list of inverted numbers to list of square roots … 25

26. L12 2009-12-07ID1218, Christian Schulte Map map(_,[]) -> []; map(F,[X|Xr]) -> [F(X)|map(F,Xr)]. msq(Xs) -> map(fun (X) -> X*X end, Xs).  Use map/2 for any mapping function!  Syntax of fun like case 26

27. L12 2009-12-07ID1218, Christian Schulte Using Map  Mapping to square numbers map(fun (X) -> X*X end,[1,2,3])  Mapping to negative numbers map(fun (X) -> -X end,[1,2,3]) 27

28. L12 2009-12-07ID1218, Christian Schulte Other Examples  filter(F,Xs) returns all elements of Xs for which F returns true  any(F,Xs) tests whether Xs has an element for which F returns true  all(F,Xs) tests whether F returns true for all elements of Xs 28

29. L12 2009-12-07ID1218, Christian Schulte Folding Lists  Consider computing the sum of list elements …or the product …or all elements appended to a list …or the maximum …  What do they have in common?  Consider example: sl 29

30. L12 2009-12-07ID1218, Christian Schulte Left-Folding  Two values define “folding”  initial value 0 for sl  binary function + for sl  Left-folding foldl(F,[ x 1, …, x n ],S) F(… F(F(S, x 1 ), x 2 ) …, x n ) or (…((S  F x 1 )  F x 2 ) …  F x n ) 30

31. L12 2009-12-07ID1218, Christian Schulte foldl foldl(_,[],S) -> S; foldl(F,[X|Xr],S) -> foldl(F,Xr,F(S,X)). 31

32. L12 2009-12-07ID1218, Christian Schulte 32 Right-Folding  Two values define “folding”  initial value  binary function  Right-folding foldr(F,[ x 1 … x n ],S) F( x 1,F( x 2, … F( x n,S)…)) or x 1  F ( x 2  F ( … ( x n  F S) … ))

33. L12 2009-12-07ID1218, Christian Schulte 33 foldr foldr(_,[],S) -> S; foldr(F,[X|Xr],S) -> F(X,foldr(F,Xr,S)).

34. Concurrency L12 2009-12-07 34 ID1218, Christian Schulte

35. Erlang Concurrency Primitives  Creating processes spawn(F) for function value F spawn(M,F,As) for function F in module M with argument list As  Sending messages PID ! message  Receiving messages receive … end with clauses  Who am I? self() returns the PID of the current process L12 2009-12-07ID1218, Christian Schulte 35

36. Processes  Each process has a mailbox incoming messages are stored in order of arrival sending puts message in mailbox  Processes are executed fairly if a process can receive a message or compute… …eventually, it will It will pretty soon… simple priorities available (low) L12 2009-12-07ID1218, Christian Schulte 36

37. Message Sending  Message sending P ! M is asynchronous the sender does not wait until message has been processed continues execution immediately evaluates to M  When a process sends messages M1 and M2 to same PID, they arrive in order in mailbox FIFO ordering  When a process sends messages M1 and M2 to different processes, order of arrival is undefined L12 2009-12-07ID1218, Christian Schulte 37

38. Message Receipt  Only receive inspects mailbox all messages are put into the mailbox  Messages are processed in order of arrival that is, receive processes mailbox in order  If the receive statement has a matching clause for the first message remove message and execute clause always choose the first matching clause  Otherwise, continue with next message  Unmatched messages are kept in original order L12 2009-12-07ID1218, Christian Schulte 38

39. Communication Patterns L12 2009-12-07 39 ID1218, Christian Schulte

40. Broadcast foreach(_,[]) -> void; foreach(F,[X|Xr]) -> F(X),foreach(Xr). broadcast(PIDs,M) -> foreach(fun(PID) -> PID ! M end,PIDs). L12 2009-12-07ID1218, Christian Schulte 40

41. Broadcast With Ordered Reply collect(PIDs,M) -> map(fun (P) -> P ! {self(),M}, receive {P,Reply} -> Reply end end, PIDs).  Collect a reply from all processes in order L12 2009-12-07ID1218, Christian Schulte 41

42. Low-latency Collect collect(PIDs,M) -> foreach(fun (P) -> P ! M end, PIDs), map(fun (P) -> receive {P,R} -> R end end, PIDs).  No order L12 2009-12-07ID1218, Christian Schulte 42

43. Coordination L12 2009-12-07 43 ID1218, Christian Schulte

44. L12 2009-12-07ID1218, Christian Schulte 44 Protocols  Protocol: rules for sending and receiving messages programming with processes  Examples broadcasting messages to group of processes choosing a process  Important properties of protocols safety liveness

45. L12 2009-12-07ID1218, Christian Schulte 45 Choosing a Process  Example: choosing the best lift, connection, …  More general: seeking agreement coordinate execution  General idea: Master send message to all slaves containing reply PID select one slave: accept all other slaves: reject Slaves answer by replying to master PID wait for decision from master

46. Master: Blueprint decide(SPs) -> Rs=collect(SPs,propose), {SP,SPr}=choose(SPs,Rs), SP ! {self(),accept}, broadcast(SPr,reject}.  Generic: collecting and broadcasting  Specific: choose single process from processes based on replies Rs L12 2009-12-07ID1218, Christian Schulte 46

47. Slave: Blueprint slave() -> receive {M,propose} -> R=…, M ! {self(),R}, receive {M,accept} -> …; {M,reject} -> … end L12 2009-12-07ID1218, Christian Schulte 47

48. L12 2009-12-07ID1218, Christian Schulte 48 Avoiding Deadlock  Master can only proceed, after all slaves answered will not process any more messages until then receiving messages in collect  Slave can only proceed, after master answered will not process any more messages until then  What happens if multiple masters for same slaves?

49. L12 2009-12-07ID1218, Christian Schulte 49 Avoiding Deadlock  Force all masters to send in order: First A, then B, then C, … Guarantee: If A available, all others will be available difficult: what if dynamic addition and removal of lists does not work with low-latency collect low-latency: messages can arrive in any order high-latency: receipt imposes strict order  Use an adaptor access to slaves through one single master slaves send message to single master problem: potential bottleneck

50. Liveness Properties  Important property of concurrent programs liveness  An event/activity might fail to be live other activities consume all CPU power message box is flooded (denial of services) activities have deadlocked …  Difficult: all possible interactions with other processes must guarantee liveness reasoning of all possible interactions L12 2009-12-07ID1218, Christian Schulte 50

51. Process State Reconsidered L12 2009-12-07 51 ID1218, Christian Schulte

52. L12 2009-12-07ID1218, Christian Schulte 52 State and Concurrency  Difficult to guarantee that state is maintained consistently in a concurrent setting  Typically needed: atomic execution of several statements together  Processes guarantee atomic execution

53. Locking Processes  Idea that is used most often in languages which rely on state  Before state can be manipulated: lock must be acquired for example, one lock per object  If process has acquired lock: can perform operations  If process is done, lock is released L12 2009-12-07ID1218, Christian Schulte 53

54. A Lockable Process outside(F,InS) -> receive {acquire,PID} -> PID ! {ok,self()}, inside(F,InS,PID) end. inside(F,InS,PID) -> receive {release,PID} -> outside(F,InS) {PID,Msg} -> inside(F,F(Msg,InS)); end. L12 2009-12-07ID1218, Christian Schulte 54

55. Safety Properties  Maintain state of processes in consistent fashion do not violate invariants to not compute incorrect results  Guarantee safety by atomic execution exclusion of other processes ("mutual exclusion") L12 2009-12-07ID1218, Christian Schulte 55

56. Runtime Efficiency L12 2009-12-07 56 ID1218, Christian Schulte

57. L12 2009-12-07ID1218, Christian Schulte 57 Approach  Take MiniErlang program  Take execution time for each expression  Give equations for runtime of functions  Solve equations  Determine asymptotic complexity

58. L12 2009-12-07ID1218, Christian Schulte 58 Execution Times for Expressions  Give inductive definition based on function definition and structure of expressions  Function definition pattern matching and guards  Simple expression values and list construction  More involved expression function call recursive function call leads to recursive equation often called: recurrence equation

59. L12 2009-12-07ID1218, Christian Schulte 59 Execution Time: T(E)  Value T(V) = c value  List construction T( [ E 1 | E 2 ] ) = c cons + T(E 1 ) + T(E 2 )  Time T(E) needed for executing expression E how MiniErlang machine executes E

60. L12 2009-12-07ID1218, Christian Schulte 60 Function Call  For a function F define a function T F (n)for its runtime and determine size of input for call to F input for a function

61. L12 2009-12-07ID1218, Christian Schulte 61 Function Call T(F ( E 1,..., E k ) ) = c call + T(E 1 ) +... + T(E k ) + T F (size(I F ({1,..., k}))) input argumentsI F ({1,..., k}) input arguments for F sizesize(I F ({1,..., k})) size of input for F

62. Function Definition  Assume function F defined by clauses H 1 -> B 1 ; … ; H k -> B k. T F (n) = c select + max { T(B 1 ), …, T(B k ) } L12 2009-12-07ID1218, Christian Schulte 62

63. L12 2009-12-07ID1218, Christian Schulte 63 Example: app/2 app([],Ys) -> Ys; app([X|Xr],Ys) -> [X|app(Xr,Ys)].  What do we want to compute T app (n)  Knowledge needed input argumentfirst argument size functionlength of list

64. L12 2009-12-07ID1218, Christian Schulte 64 Append: Recurrence Equation  Analysis yields T app (0)= c 1 T app (n)= c 2 + T app (n-1)  Solution to recurrence is T app (n) = c 1 + c 2  n  Asymptotic complexity T app (n) is of O(n)“linear complexity”

65. L12 2009-12-07ID1218, Christian Schulte 65 Recurrence Equations  Analysis in general yields a system T(n)defined in terms of T(m 1 ), …, T(m k ) for m 1, …, m k < n T(0), T(1), …values for certain n  Possibilities solve recurrence equation (difficult in general) lookup asymptotic complexity for common case

66. L12 2009-12-07ID1218, Christian Schulte66 Common Recurrence Equations T(n)Asymptotic Complexity c + T(n–1)O(n) c 1 + c 2  n + T(n–1) O(n 2 ) c + T(n/2)O(log n) c 1 + c 2  n + T(n/2) O(n) c + 2T(n/2)O(n) c + 2T(n-1)O(2 n ) c 1 + c 2  n + T(n/2) O(n log n)

67. That’s It! L12 2009-12-07 67 ID1218, Christian Schulte