1. Message Passing Distributed Systems with examples from Socket based Client/Server Systems

2. Overview 1.Distributed Computing Topologies 2.Critical Points in C/S Topologies 3.The Message Passing Model 4.Delivery Guarantees 5.Request Ordering and Reliable Broadcast 6.Programming C/S Systems with Sockets Starting with the common C/S model, we will develop core properties for distributed protocols.

3. Distributed Computing Topologies - Client/Server - Hierarchical - Totally Distributed - Bus Topologies

4. Client/Server Systems client server request response Request processing Clients initiate communication and (typically) block while waiting for the server to process the request. Still the most common DS topology.

5. Hierarchical Systems Every node can be both client and server but some play a special role, e.g. are Domain Name System (DNS) server. A reduction of communication overhead and central control options are some of the attractive features of this topology. Client and Server

6. Totally distributed Every node IS both client and server. Watch out: peer to peer systems need not be totally distributed! Client and Server

7. Bus Systems Every node listens for data and posts data in response. This achieves a high degree of separation and indepencence. Event-driven systems follow this topology. Client and Server

8. Critical Points in C/S Topologies - Load Management, Delays and Bottlenecks - Security - Queuing Theory Aspects - Sync. vs. Async. Processing Patterns

9. Critical Points in C/S Systems client server request response Processing: clustering?, single/multithr.? Session state? Authentication? authorization? Privacy? Requester: locate server? Authenticate? Synchronous vs asynchr. call? Scalability, Availability and Security all depend on the answers to those questions. Bandwidth/latency?

10. Queuing Theory for Multi-tier Process Networks A modern application servers performance largely relies on the proper configuration of several queues from network listening to threadpools etc. Queuing theory lets us determine the proper configurations (see resources). In general, architectures like above are very sensitive for saturated queues. Good architectures create a funnel from left to right and limit resources like max. threads. Caching an batching are directly derived from queuing theory. Picture from: http://publib.boulder.ibm.com/infocenter/wasinfo/v5r1//index.jsp?topic=/com.ibm.websph ere.base.doc/info/aes/ae/rprf_queue.html

11. Synchronous I/O (blocking calls) Many threads are required to stay responsive. Many context switches occur and each thread needs extra memory. see: Aruna Kalaqanan et.al. http://www- 128.ibm.com/developerworks/java/library/j-javaio Thread Input Channel FileSys Output Channel Wait for client cmd. Process client cmd, e.g. get file Send response to client Wait for client cmd. Switch

12. Non-Blocking: Reactor Pattern From: Aruna Kalaqanan et.al. http://www-128.ibm.com/developerworks/java/library/j- javaio. The downside: all processing needs to be non-blocking and the threads need to maintain the state of the processing between handler calls (explicit state management vs. implicit in normal multi-threaded designs). “Server applications in a distributed system must handle multiple clients that send them service requests. Before invoking a specific service, however, the server application must demultiplex and dispatch each incoming request to its corresponding service provider. The Reactor pattern serves precisely this function. It allows event-driven applications to demultiplex and dispatch service requests, which are then delivered concurrently to an application from one or more clients.” The Reactor pattern is closely related to the Observer pattern in this aspect: all dependents are informed when a single subject changes. The Observer pattern is associated with a single source of events, however, whereas the Reactor pattern is associated with multiple sources of events.”

13. Reactor Pattern From Doug Lea (see resources). Note that this is the single threaded version. Symbian OS uses a similiar concept called Active Objects. Handlers are not allowed to block. In many cases a single thread may be enough.

14. The Message Passing Model - Modelling and Automata - Async. vs. Sync Systems - Protocol Properties: Correctness, Liveness, Fairness,... - Complexity - Failure Types

15. Modeling of Distributed Systems Process I State variables Inbuf Outbuf (After: J.Aspnes): A processing function takes the Inbuf Data from other processes, the internal state variables and computes a new internal state and new Outbuf data Communication ist point-to-point and deterministic. A configuration is the state vector for all processes. Events change configurations into new ones. An execution is a sequence of Configurations and Events: C0 e0, C1 e1, C2 e2... Send -> < - Receive Ptp primitives:

16. Synchronous vs. Asynchronous Systems Synchronous (lockstep): e== event, t== time e0t0 ---> delivery at t0+1, e1t1 ---> deliv. T1+1, ….. Asynchronous (delayed): e0t0 ---> delivery at ?, e1t1 ---> deliv. T1+?, ….. Reqs: infinitely many comp. Steps possible, events will be eventually delivered. Synchronous systems have simpler distributed algorithms, but are harder to build. The reality is async. Systems with additonal help from failure detectors, randomization etc.

17. Protocol Properties - Correctness: invariant properties are shown to hold throughout executions - Liveness/Termination: the protocol is shown to make progress in the context of certain failures and in a bounded number of rounds - Fairness: no starvation for anybody - Agreement: e.g. all processes agree output the same decision - Validity: for the same input x, all processes output according to x (Or: there is a possible execution for every possible output value) (after: Aspnes).

18. Complexity - Time complexity: the time of the last event before all processes finish (Aspens) - Message complexity: the number of messages sent Message size and the number of rounds needed for termination are important for the scalability of protocols

19. Failure Types - Crash failure: a process stops working and stays down - Connectivity failures: network failures e.g. causing split brain situations with two separate networks or node isolation. Typically the time for message propagation is affected. - Message loss: single messages are lost, nodes are up. - Byzantine Failures: „Evil“ nodes violating protocol assumptions and promises. E.g. breaking a promise due to disk failure, configuration failer etc. All protocols are validated with respect to certain failure scenarios!!

20. The Role of Delivery Guarantees - Problem Scenario: Shop order - TCP Communication properties - Message complexity: the number of messages sent

21. Crash During Shop Order Shop user shop Order Order confirmation Order processing What happens to the order when certain failure types apply? What kind of guarantees do you have? Does TCP Help? What outcomes to you expect? Browser crashed Message lost Server crashed Lost in transmit

22. TCP communication properties lost messages retransmitted Re-sequencing of out of order messages Sender choke back (flow control) No message boundary protection These features form a “reliable communication channel”. This does not include proper behavior in case of connection failures! (timeout problem). (Ken Birman, building secure and reliable network applications, chapter 1)

23. Communication Failure: timeout Failure Cases: a) network problem, server did not receive request b) Server problem: OS did receive request but server crashed during work c) OS/Network problem: Server finished request but response got lost or OS crashed during send. Case: Client sends request and receives timeout Client options: drop request (ok in c), resend request (ok in a and b), send request to different server (ok in a and b). Other client actions lead either to lost or duplicated requests.

24. Delivery Guarantees Best effort (doesn’t guarantee anything) At least once (same request several times received) At most once (not more than once but possibly not at all) Once and only once (simply the best) In case of channel break-down TCP does NOT make ANY delivery guarantees. This becomes your job then (or better: a job for your middleware)

25. „At most once“ implementation for non- idempotent requests Request # server request Response # (ack) A response is stored until client confirms By adding a request number to each request the server can detect duplicate requests and throw them away. The server itself needs to store a response until the client acknowledges that it was received. This creates state on the server! client

26. Idempotent Requests? Get bank account balance Push elevator button Get /index.html …. Book flight Cancel flight …. What kind of delivery guarantee do you need for idempotent service requests?

27. Idempotency Is not a medical condition (Pat Helland) The first request needs to be idempotent The last request can only be best effort Messages may be reordered. Your partner may experience amnesia as a result of failures, poorly managed durable state, or load-balancing switchover to its evil twin. …. Pat Helland, http://queue.acm.org/detail.cfm?id=2187821http://queue.acm.org/detail.cfm?id=2187821

28. Request Ordering with Multiple Nodes - Reliable Broadcast - FifoCast - Causal Cast - Absolute Ordered Casts Taken from: C. Karamanoulis and K.Birman

29. Fault-tolerant Broadcast Model Process I Watch out: messages can be delivered without respect to some order. Or they can be sorted, kept back at the middleware layer and only delivered when a certain order can be guaranteed. Notice the self-delivery of messages by the sending process. Send (m,#) Ptp primitives: Process II Middle-Ware Communication Layer Receive (m,#) Broadcast primitives: Bcast (m,#)Deliver (m,#) Del. (m,#)

30. Reliable Broadcast with no Order Taken from: C. Karamanoulis, Reliable Broadcasts client server order rebate cancel ???? A cancel request without previous order

31. Reliable Broadcast with FiFo-Order Taken from: C. Karamanoulis, client server M2M1 M3 Recv. Rel.delivery FIFO del. Delayed!!! M3 M1 Recv. Rel.delivery FiFo delivery M3 gets FiFO delvd. Here:

32. Causal Violation with FiFO Order Taken from: C. Karamanoulis, Local Order: If a process delivers a message m before broadcasting a message m’, then no correct process delivers m’ unless it has previously delivered m. Stud1 Stud2 M2M1 M1 FiFo delivered! M3 Stud3 Lecture cancelled! Let's go somewhere! But we have a lecture???

33. Solutions for Causal Ordered Broadcasts - Piggyback every message sent with privious messages: Processes which missed a message can learn about it with The next incoming message and then deliver correctly - Send event history with every message (e.g. using vector Clocks. Delay delivery until order is correct. Taken from: C. Karamanoulis and K.Birman. What are the advantages/disadvantages of both solutions?

34. Causal Re-Ordering Taken from: C. Karamanoulis, P3 has delivered M2 to itself, before delivering M1. Is this a problem? Think about causal dependency and what causes it! p1 p2 M2 M1 M3 delayed, until M1 delivered! M3 p3 M2 delivered!

35. Replication Anomalies with Causal Order Taken from: C. Karamanoulis, Total Order: If correct processes p and q both deliver messages m and m’, then p delivers m before m’ if and only if q delivers m before m’. Replica 1 Add 100 Multiply by 10 State:100 State:200 State:1000 State:2000 State:1100 Replica 2

36. Solutions for Atomic Broadcasts (Total Order) - All nodes send messages to every other node. - All nodes receive messages, but wait with delivery - One node has been selected to organize total order. - This node orders all messages into a total order - This node sends the total order to all nodes - All nodes receive the total order and deliver their messages According to this order. Taken from: K.Birman. What are the advantages/disadvantages of this solution?

37. Programming Client/Server Systems with Sockets and different I/O Models

38. Overview Socket primitives Process Model with sockets Example of server side socket use Transparency and socket programming? Security, Performance, Availability, Flexibility etc. of socket based C/S. Typical C/S infrastructure (Proxies, Firewalls, LDAP)

39. Protocol Stack for Sockets Physical Data Link Network Transport/Session Socket: host A, port 3500, tcp-conn Socket: host B, port 80, tcp-conn Tcp connectionUdp connection Reliable comm. channel

40. Socket Properties Using either tcp or udp connections Serving as a programming interface A specification of “Host”, “Port”, “Connection type” A unique address of a channel endpoint.

41. Berkeley Sockets (1) Socket primitives for TCP/IP. PrimitiveMeaning SocketCreate a new communication endpoint BindAttach a local address to a socket ListenAnnounce willingness to accept connections AcceptBlock caller until a connection request arrives ConnectActively attempt to establish a connection SendSend some data over the connection ReceiveReceive some data over the connection CloseRelease the connection From: van Steen, Tanenbaum, Distributed Systems

42. Berkeley Sockets (2) Connection-oriented communication pattern using sockets. From: van Steen, Tanenbaum, Distributed Systems

43. Server Side Processing using Processes Server (process) Server Dispatcher Process Client Listening on port X Accept and spawn process on Port Y Connection established between client on port C and server on port Y Connecting on arbitrary port C After spawning a new process the dispatcher goes back to listening for new connection requests. This model scales to some degree (process creation is expensive and only few processes are possible). Example: traditional CGI processing in web-server

44. Server Side Processing using Threads Server (thread) Server Dispatcher Process Client Listening on port X Accept and spawn thread on Port Y Connection established between client on port C and server on port Y Connecting on arbitrary port C After spawning a new thread the dispatcher goes back to listening for new connection requests. This model scales well (thread creation is expensive but they can be pooled) and a larger number of threads are possible). Example: servlet request processing in servlet engine (aka “web-container”)

45. Server Side Concurrency addMoney(account, value) Thread Threaded server In the case of the threaded server the function needs to be re-entrant. No unprotected global variables. Keep state per thread on stack. addMoney(account, value) Process per request

46. Designing a socket based service a)Design the message formats to be exchanged (e.g. “http1.0 200 OK …). Try to avoid data representation problems on different hardware. b)Design the protocol between clients and server: - Will client wait for answer? (asynchronous vs. synchr. Comm.) - Can server call back? (== client has server functionality) - Will connection be permanent or closed after request? - Will server hold client related state (aka session)? - Will server allow concurrent requests?

47. Stateless or Stateful Service? Stateless: Scales extremely well Makes denial of service attacks harder Forces new authentication and authorization per request Stateful Allows transactions and delivery guarantees Can lead to resource exhaustion (e.g. out of sockets) on a server Needs somehow reliable hardware and networks to succeed.

48. Server Dangers: Keeping State and expecting clients to behave -TCP SYN flooding clientserver Client info stored SYN SYN,ACK(SYN) request client server SYN Client info stored SYN Client info stored Client never sends request, only SYN, Server buffer gets filled and other clients cannot connect

49. A Client using sockets 1.Define hostname and port number of server host 2.Allocate a socket with host and port parameters 3.Get the input channel from the socket (messages from server) 4.Get output channel from socket (this is where the messages to the server will go) 5.Create a message for the server, e.g. “GET /somefile.html HTTP/1.0” 6.Write message into output channel (message is sent to server) 7.Read response from input channel and display it. A multithreaded client would use one thread to read e.g. from the console and write to the output channel while the other thread reads from the input channel and displays the server messages on the console (or writes to a file)

50. A server using sockets 1.Define port number of service (e.g. 80 for http server) 2.Allocate a server socket with port parameter. Server socket does “bind” and “listen” for new connections. 3.“Accept” an incoming connection, get a new socket for the client connection 4.Get the input channel from the socket and parse client message 5.Get output channel from socket (this is where the messages to the client will go) 6.Do request processing (or create a new thread to do it) 7.Create a response message e.g. “HTTP/1.0 2000 \n…” 8.Write message into output channel (message is sent to client) 9.Read new message from client channel or close the connection A bare bone server. Could be extended through e.g. a command pattern to match requests with processing dynamically. New commands could get loaded dynamically as well. (“Application Server”)

51. Distribution Transparency with Sockets? Invocation: The server side function cannot be called on the client side. Instead, socket operations must be used and messages defined. Location/Relocation/Migration: If service moves, client breaks. Replication/Concurrency: No support yet Failure: No support yet Persistence: No support yet To be fair: socket based services need to deal with all that but they are still fairly simple to write!

52. Infrastructure of C/S Systems Client (initiate) Server (process) Directory: help locate server Proxy: check client authorization, route via firewall firewall: allow outgoing calls only Firewalls Proxy Load Balancer Reverse Proxy Authent. server Directory Reverse Proxy: cache results, end SSL session, authenticate client Authentication server: store client data, authorize client Load Balancer: distribute requests across servers

53. I/O Models Thread per Connection (classic, stack- based, latency-hiding with threads) Non- Blocking (select() types), NIO, stackless Async (AIO, NIO2, node.js), stackless –Callback Style –Future/Promise Style

54. Non-Blocking Single-selector thread doing all work (no CPU intensive stuff, no blocking anywhere, scale through more instances Single selector thread with delegation of work to worker thread-pool. Worker threads can block. Except for key (event) change semantics this model is quite easy to understand. Originator: Berkeley Unix See: Rox NIO Tutorial http://rox- xmlrpc.sourceforge.net/niotut/

55. Asynchronous I/O Single thread doing all work (no CPU intensive stuff, no blocking anywhere, scale through more instances Potentially blocking calls need to be handled by hidden threads Callbacks called asynchronously. Ugly source Sequential illusion with futures/promises which represent unfinished call. AIO kernel interfaces used This model requires full support of async. Interfaces by the kernel/vm. Examples is node.js. See: An NIO.2 primer, Part 1: The asynchronous channelAPIs, Catherine Hope, Oliver Deakin, http://www.ibm.com/developerworks/library/j-nio2-1/

56. Things to consider with I/O Models Service Requests/sec Memory and context switching costs of threads Number of cores Do services perform CPU intensive tasks? Do services need to block?

57. Exercises Build a generic client Build an echo server Build a http client and server Build a proxy/firewall Using code pieces from the Java examples book we will: We will discuss general design issues as well! (patterns etc)

58. Resources David Flanagan, Java Examples in a Nutshell, O’Reilly, chapter 5. Code: www.davidflanagan.com/javaexamples3 Ted Neward, Server Based Java Programming chapter 10, Code:www.manning.com/neward3 Doug Lea, Concurrent Programming in Java Pitt, Fundamental Java Networking (Springer). Good theory and sources (secure sockets, server queuing theory etc.) Queuing Theory Portal: http://www2.uwindsor.ca/%7Ehlynka/queue.html Performance Analysis of networks: http://www2.sis.pitt.edu/~jkabara/syllabus2120.htm (with simulation tools etc.) Meet the experts: Stacy Joines and Gary Hunt on WebSphere performance (performance tools, queue theory etc.) http://www- 128.ibm.com/developerworks/websphere/library/techarticles/0507_joines/0507_joines.html Doug Lea, Java NIO http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf Learn how to handle thousands of requests per second in Java with a smaller number of threads. Event driven programming, Design patterns like reactor, proactor etc. Abhijit Belapurkar, CSP for Java programmers part 1-3. Explains the concept of communicating sequential processes used in JCSP library. Learn how to avoid shared state multithreading and its associated dangers. Core tips to Java NIO: http://www.javaperformancetuning.com/tips/nio.shtml Schmidt et.al. POSA2 book on design patterns for concurrent systems. Nuno Santos, High Peformance servers with Java NIO: http://www.onjava.com/pub/a/onjava/2004/09/01/nio.html?page=3. Explains design alternatives for NIO. Gives numbers of requests per second possible. James Aspnes, Notes on Theory of Distributed Systems, Spring 2014, www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf‎