1. 1 Communication Inter-process Communication is the heart of a Distributed System Basic mechanisms are: RPC RMI MOM Streams

2. 2 How to Communicate? Problem: a process in machine A wants to communicate (an item) to a process in machine B How is this done? Build a message and send it through the network (cable) to the “other” side. MANY things have to be agreed upon! 1.Voltage (0/1)? For bit representation.. 2.Correctness of messages 3.Representation of numbers, strings, and data items(in different machines) 4.Etc.

3. 3 Approach: OSI Layered Protocols OSI Open System Interconnection Reference: Layers, interfaces, and protocols in the OSI model. 2-1

4. 4 Layered Protocols: as you dip into the layers you “add” more info A typical message as it appears on the network. 2-2

5. 5 OSI Model Design for “Open System”: allows the communication of one system with another – possibly different one. Open Systems are not exactly “Open” ; They are governed by a set of rules/regulations… Protocols: formalization of these rules Connection-oriented protocols Connectionless protocols.

6. 6 Physical Level Mostly concerned with the transmission of 0/1s.. Voltage level Rate of transmission Network connectors (plugs) Example: RS232c (standard for serial communication).

7. 7 Data Link Layer: frames & checksums Discussion between a receiver and a sender in the data link layer. 2-3

8. 8 Network Layer Does the “routing” of packets.. Problem: shortest path is not always available!! Even if it is available “routing” may opt for a different path? Why? IP Internet Protocol ( main representative of Network Layer ) An IP packet ( frame ) can be sent without any setup. Virtual Path is gaining popularity ( in ATM networks )

9. 9 Transport Layer Data will have to delivered to the other side with no loss.. The job of the TL is exactly this! Examples.. TCP (Transmission Control protocol) UDP (Universal Datagram Protocol): almost similar to IP. RTP (Real-Time Protocol)

10. 10 Example: Client-Server TCP a)Normal operation of TCP. b)Transactional TCP (TCP for Transactions Suite) 2-4

11. 11 High(-er) Level Protocols.. Session and Presentation Layer –Provides dialog facilities, keeps track of who is talking at this time, and provides synchronization if needed. –All these are useful into long transfers as precautions can be taken for the transmission of data. Examples: –FTP (file transfer protocol) –HTTP (HyperText Transfer Protocol) –TELNET, SSH

12. 12 Example of Higher Level Protocols : Middleware Protocols An adapted reference model for networked communication that may enforce the notion of “atomicity” 2-5

13. 13 Communication of Processes-RPCs [Birell & Nelson TOCS94] When a process A in machine a calls a routine on machine B, the calling process A suspends and the execution of the called program in B takes place transparently.. Information is transported from caller to callee in the parameters and the result returns No message passing is visible to the user. Remote Procedure Call (RPC). Idea: make a remote procedure call appear as a local one!

14. 14 Conventional Procedure Call a)Parameter passing in a local procedure call: the stack before the call to read (from main() ) b)The stack while the called procedure is active

15. 15 Client and Server Stubs The client stub works in a similar way to a usual system call (for instance read()). The difference is that a different version (the client stub) is loaded into the frame stack. This stub does not ask the OS to get any data on behalf of the process. The stub packs the parameters into a single message and requests the message to be sent to the server. When the msg arrives into the server, the OS of the server surrenders the msg to the server stub ( server side equivalent ).

16. 16 Client and Server Stubs Principle of RPC between a client and server program.

17. 17 Steps of a Remote Procedure Call 1.Client procedure calls client stub in normal way 2.Client stub builds message, calls local OS 3.Client's OS sends message to remote OS 4.Remote OS gives message to server stub 5.Server stub unpacks parameters, calls server 6.Server does work, returns result to the stub 7.Server stub packs it in message, calls local OS 8.Server's OS sends message to client's OS 9.Client's OS gives message to client stub 10.Stub unpacks result, returns to client

18. 18 Possible Issues Passing parameters is always a problem.. Pass by value Pass by reference Call by copy/restore Marshaling of parameters has to initially take place.

19. 19 Passing Value Parameters Steps involved in doing remote computation through RPC 2-8

20. 20 Possible Problems with Passing Value Parameters a)Original message on the Pentium (little Endian- number their bytes from right to left ) b)The message after receipt on the SPARC (big Endian- numbers the bytes from left to right ) c)The message after being inverted is not a general solution. d)The little numbers in boxes indicate the address of each byte (need some type of deli-meter in order to understand the various representations involved) e)integers and strings have to be handled differently!

21. 21 Passing Reference Parameters How are pointers being passed?? –Generally difficult problem… –One solution is forbid passing of pointers all together (rather restrictive) –Some things can be done with arrays: Copy the array into the message and send it over.. Call-by-reference is replaced by copy/restore.. If the stubs know whether the buffer is an Input parameter to the server or an output parameter to the client optimizations can be done.

22. 22 Parameter Specification and Stub Generation En example.. A procedure The corresponding message. In order to hide the details of RPC some conventions (protocol) between theClient and the Servers have to be established.. Representations (formats) of int, float, double, char, etc have to be Be agreed upon. RPC protocols can be established with IDL specs (that are compiled Into client/server stubs…)

23. 23 Extensions of RPCs:Doors[Hamilton & Kougiouris 94] What if caller/callee are in the same machine?? (classic IPC problem) Door is a generic name for a procedure in the address space of a server process & can be called by processes colocated with the server. Requires support from the local OS. Plus: allows the use of procedure calls mechanism for a distributed system. Minus: developers still need to be aware whether a call is done locally or remotely

24. 24 Asynchronous RPC a)The interconnection between client and server in a traditional RPC b)The interaction using asynchronous RPC 2-12

25. 25 Asynchronous RPC A client and server interacting through two asynchronous RPCs One-way RPCs 2-13

26. 26 Distributed Computing Environment(DCE) Idea: take a group of machines (running Unix, OS2, Windows etc) add a layer of software and then be able to run distributed applications without disturbing the running of existing applications! Some basic services offered such as: 1.Distributed File Service (the worldwide file system) 2.Directory Service 3.Security Service 4.Distributed Time Service (why?)

27. 27 Writing a Client and a Server The steps in writing a client and a server in DCE RPC. 2-14

28. 28 Binding a Client to a Server A client can call a server as long as the server is registered and ready to accept calls! Registration makes possible that the client can locate the server and bind to it. Server location is down into two steps: Locate the server’s machine Locate the server (ie, the correct process on that machine!).

29. 29 Binding of Clients and Servers In order to achieve the step (2) the port for the service needs to be known.. (or endpoint) Ports are used by servers as different entry points to different procedure calls. In DCE there is a daemon (DCE Daemon) that makes this look up between (server, endpoint) The server also registers with a directory machine(name of machine and IP number)

30. 30 Binding a Client to a Server Client-to-server binding in DCE. 2-15 Example: a client wants to bind to a video server that is locally known as /local/multimedia/video/movies DCE semantics Default: at-most-once operation Otherwise: idempotent (done with the help of IDL) a remote Procedure call can be called a number of times.

31. 31 Remote Object Invocation Object-Oriented Technology (CORBA, DCOM) is another way to develop distributed applications. RPCs could be applied to those frameworks as well. Object:: –Encapsulates data –Methods (operations on data) –Methods are available via interfaces –The differentiation between object & interfaces is critical as far as Distributed Systems concerns. –Interface can be on one machine – Object on another.

32. 32 Distributed Objects Common organization of a remote object with client-side proxy. Proxy = client stub ; Skeleton = Server Stub The state of such distributed objects is NOT distributed! A real-distributed object should be one that is physically distributed across multiple machines. 2-16

33. 33

34. 34

35. 35 Implicit/Explicit Binding of a Client to an Object a)An example with implicit binding using only global references b)An example with explicit binding using global and local references (the client first calls a a special function that binds the object before the client can invoke the object’s methods) Distr_object* obj_ref;//Declare a systemwide object reference obj_ref = …;// Initialize the reference to a distributed object obj_ref-> do_something();// Implicitly bind and invoke a method (a) Distr_object objPref;//Declare a systemwide object reference Local_object* obj_ptr;//Declare a pointer to local objects obj_ref = …;//Initialize the reference to a distributed object obj_ptr = bind(obj_ref);//Explicitly bind and obtain a pointer to the local proxy obj_ptr -> do_something();//Invoke a method on the local proxy (b)

36. 36 Types of Remote Method Invocation (RMI) Static Invocation –Interfaces of an object are know when client application is being developed. Dynamic Invocation –Be able to compose a methods invocation at run time! – invoke(object, method, in_params, out_params) –Example: append an integer int to a file object fobject for which the object provides the method append Static: fobejct.append(int) Dynamic: invoke(fobject, id(append), int) where id returns an identifier for the method append. Dynamic Invocation: when is it useful?

37. 37 Parameter Passing in RMI Passing an object by reference or by value. There are subtleties with RMI invocation (more so than with RPCs) References to remote/local objects are handled differently When invoking a method with an object reference as parameter Reference is copied and passed as a value parameter only when it refers to a remote object When the reference is made on a local (client) object, the object is copies as a whole and passed along with the invocation

38. 38 Message Oriented Communication RPCs & RMIs hide communication element of the interaction In general both RPCs & RMIs are blocking An alternative to those is communication based on messages General organization for message-oriented communication is shown below

39. 39 Example: Mail-Server Every host is connected to one mail-server (can think of it as being the “comm” server of the picture in the previous slide). A client interface allows users to get access to the messages (located on the mail servers). When a user submits an message, the host forwards the message its corresponding mail server. Mail servers forward/delete messages. Such a system is an example of persistent communication

40. 40 Mail-servers follow the Pony Express Model Persistence has to do with the fact that the mail is stored as long as it takes to deliver it.. Transient Communication: if a message cannot be delivered to the next server/destination is discarded (typical of the transport layer – corresponds to store-and-forward router).

41. 41 Possible Combinations(sync/async-persi/transi) a)Persistent asynchronous communication b)Persistent synchronous communication(msg actually delivered to the receiving site) 2-22.1

42. 42 Combos c)Transient asynchronous communication (examples are UDP, asynchronous RPCs) d)Receipt-based transient synchronous communication 2-22.2

43. 43 Combos e)Delivery-based transient synchronous communication at message delivery f)Response-based transient synchronous communication (RPCs & RMIs mostly adhere to this type of communication).

44. 44 Berkeley Sockets Socket primitives for TCP/IP. PrimitiveMeaning SocketCreate a new communication endpoint BindAttach a local address to a socket ListenAnnounce willingness to accept connections AcceptBlock caller until a connection request arrives ConnectActively attempt to establish a connection SendSend some data over the connection ReceiveReceive some data over the connection CloseRelease the connection A socket is a communication endpoint to which the application from which can read data from or write data to.

45. 45 How Berkeley SocketsWork Connection-oriented communication pattern using sockets.

46. 46 The MPI Interface Some of the most intuitive message-passing primitives of MPI. PrimitiveMeaning MPI_bsendAppend outgoing message to a local send buffer MPI_sendSend a message and wait until copied to local or remote buffer MPI_ssendSend a message and wait until receipt starts MPI_sendrecvSend a message and wait for reply MPI_isendPass reference to outgoing message, and continue MPI_issendPass reference to outgoing message, and wait until receipt starts MPI_recvReceive a message; block if there are none MPI_irecvCheck if there is an incoming message, but do not block MPI designed for parallel application programs (transient communication) Communication happens among a group of known processes.

47. 47 Message-Queuing Systems Four combinations for loosely-coupled communications using queues. Offer intermediate storage capacity for msgs Good for transferring bulky message (in seconds) than simple messages (usually in milliseconds). Important: sender is provided with the guarantee that her message will ultimately be inserted in the recipient’s queue. Servers/receivers can execute completely independent of each other

48. 48 Message-Queuing Model: calls Basic interface to a queue in a message-queuing system. PrimitiveMeaning PutAppend a message to a specified queue Get Block until the specified queue is nonempty, and remove the first message Poll Check a specified queue for messages, and remove the first. Never block. Notify Install a handler to be called when a message is put into the specified queue.

49. 49 General Architecture of a Message-Queuing System The relationship between queue-level addressing and network-level addressing. source queues, destination queues database of queue names queuing layer

50. 50 General Architecture of a Message-Queuing System The general organization of a message-queuing system with routers. 2-29 Relays can be good (in a messaging system) Usually the queue-to-location mapping is managed statically Otherwise, routers have to be used (in this case, relays can be used to build scalable messaging systems).

51. 51 Message Brokers The general organization of a message broker in a message- queuing system. Brokers are used to convert to various message types. For example convert message from X.400 to Internet messages The database can handle many such conversion rules.

52. 52 Stream-Oriented Communication Thus far, communication as long as it happens correctly is all we mind. Timing of this communication was not the issue. Timing is important in certain types of communication … –CD quality audio transmission ( 16bit samples at 44,1kHz) –Video streams

53. 53 Streams - Definitions Information Representation –Text with ASCII/Unicode –Images with GIF/JPEG –Audio Streams with 16bit samples using PCM Continuous (Representation) Media: temporal relationship among the different data items are fundamental to interpret what the data means. –Motion requires 30-40 msec per image (if represented as a sequence of images). Discrete(Representation) Media: temporal relations among data items are not of essence –Representations of text, still-images, code, files, etc.

54. 54 Data Streams Data stream: a sequence of data units Timing is essential into continuous data streams. Asynchronous Transmission Mode –Data are xmited one after the other with no “timing” strings attached. Synchronous Transmission Mode –There is a maximum end-to-end delay defined for every unit in the data stream. Isochronous Transmission Mode –Data have to be transferred on time –There are minimum and maximum end-to-end delay (jitter)

55. 55 Data Streams Setting up a stream between two processes across a network. Simple stream: single sequence of data Complex stream: consists of several related simple streams (known as substreams) A stream is often considered a (virtual) connection between a source and a sink.

56. 56 Data Streams with Direct Connection between Source and Sink Setting up a stream directly between two devices.

57. 57 What if the sink is a multiparty? Receivers may have different requirements Filters have to be attached… The data stream is multicast to several (receivers) sinks.

58. 58 Specifying QoS A flow specification. Characteristics of the InputService Required maximum data unit size (bytes) Token bucket rate (bytes/sec) Toke bucket size (bytes) Maximum transmission rate (bytes/sec) Loss sensitivity (bytes) Loss interval (  sec) Burst loss sensitivity (data units) Minimum delay noticed (  sec) Maximum delay variation (  sec) Quality of guarantee Time-dependent requirements are collectively known as Quality of Service (QoS) requirements. How one could define such a QoS?.. Via a flow spec

59. 59 Partridge’s Model expressed with a Token Bucket Algorithm The principle of a token bucket algorithm. This model specifies how the stream will shape its network traffic Idea: tokens appear in the network at a constant rate. When the bucket fills, tokens are dropped. Each time the application wants to pass N bytes removes N/k tokens from the bucket (each token represents k bytes).

60. 60 Setting Up a Stream- RSVP Protocol The basic organization of RSVP for resource reservation in a distributed system.

61. 61 Synchronization Mechanisms The principle of explicit synchronization on the level data units. Complex Streams have to be synchronized.. How is this done?

62. 62 Synchronization Mechanisms The principle of synchronization as supported by high-level interfaces. Multimedia Middleware offers a collection of interfaces interfaces control audio/video streams control devices such as mics, cameras, monitors each device/stream has its own interface (used to write synchronization handlers).