1. Lecture V: Inter-Process Communication
CMPT
Dr. Alexandra Fedorova

2. Course Progress: the Big Picture
Architectural models of distributed systems
determined by placement and roles of components
covered
Organized interaction of all components
future
OS Support for implementation of DS of components
covered
Communication among two individual components
today
Clipart courtesy of and MS Office

3. Today: Inter-Process Communication
RPC – Remote Procedure Call
RMI – Remote Method Invocation
IPC abstractions
Protocol: rules of message exchange on the wire

4. Application Level Protocols
We will discuss application-level protocols
Built on top of transport protocols, such as TCP, UDP
Why do we need more protocols?
We’ll see next…

5. Inter-Process Communication
Here is what we want to happen
Client
Server
request
reply
Time

6. The Grim Reality of IPC CANNOT TELL BETWEEN THESE FAILURES
Here is what actually happens
Client
Server
Client
Server
Client
Server
request
request
request
waiting…
reply
server crashed
has the server crashed?
message
lost
the server is being slow
was there message loss?
Before or after servicing the request?
CANNOT TELL BETWEEN THESE FAILURES
reply

7. Motivation for App-level Protocols
Transport protocols, TCP or UDP take care of message delivery from host A to host B
TCP will take care of communication failures (message loss)
But there are other failures too: server crashes, for example
To recover from those failures you need an application-level protocol

8. Handling Failures on the Client
What should the client do?
Wait forever for the server?
Retransmit the message?
Contact another server?
Report error to the user?
The client can decide best if it knows what happened
If the server is slow – just wait
If the message was lost – retransmit it
If the server has crashed after processing the message – don’t do anything – the message has already been processed
What if the server crashed before processing the message???
What if the server crashed while processing the message?
It also depends on how the server handles failures

9. Handling Failures on the Server
Statefull server
The server remembers its state
It remembers message exchange and processing done for the client before the crash
After reboot from the crash, the server can continue with the operation it was doing before the crash
Stateless server
The server completely forgets what happened before the crash
It is up to the client how to handle recovery

10. Stateful Server Crash after request processing
Crash before request processing
Crash during request processing
Client
Server
Client
Server
Client
Server
request
request
request
record request
process request
process request
crash
crash
crash
reboot
process request
reboot
reboot
finish request
reply
reply
reply

11. Stateless Server Crash after request processing
Crash before request processing
Crash during request processing
Client
Server
Client
Server
Client
Server
request
request
request
record request
process request
process request
crash
crash
crash
reboot
reboot
reboot
REMEMBER NOTHING
REMEMBER NOTHING
REMEMBER NOTHING

12. The Role of a Protocol Protocols are designed to:
Help determine the cause of failure
Help recover from failures
Help the parties to determine the best way to recover from a failure
Protocol choice depends on:
Statefulness of the server
What the client does to handle failures
Types of failures
Today we cover protocols where servers are stateless
We deal with stateful servers later in the course

13. Messaging in a Stateless Protocol
Client
Server
request
reply
request
timeout
reply
retransmission

14. Stateless Protocol Semantics
Exactly-once message delivery
Each message is guaranteed to be delivered and processed exactly once
Cannot be achieved if there are failures (Why?)
At-least-once message delivery
Each message is delivered and processed at least once
At-most-once message delivery
Each message is delivered and processed at most once

15. At-Least-Once Protocols
Deliver messages exactly once in the absence of failures
May deliver more than once when failures occur
This works when requests are idempotent
Idempotent request:
Carrying it out once has the same effect as carrying it out multiple times
Can you think of a distributed system with idempotent requests?
WWW?
Yes. You can request the static file content multiple times without harm. (HTTP protocol is stateless)
A distributed file system?
Yes. Most file system operations are idempotent (Some FS are at-least-once)
An electronic store?
Hmm, probably not. Processing the same order twice will get you angry customers.

16. At-Least-Once Protocol: Sun’s NFS
Used to access your home directories on Unix systems in CSIL
NFS server receives requests from the client and executes them on the local file system (FS)
If NFS server crashes
Forgets about all outstanding file requests from client
The client times out and retransmits the requests
If the server crashes while modifying the file system, local OS cleans up (just like after local FS crash)
This works, because most file operations are idempotent:
Read the block
Write the block (there are no append operations)
Create/delete file (local FS will not perform same create/delete more than once)
A bit of cheating: NFS server does not keep state, but local FS keeps the state and cleans up after failures

17. At-Most-Once Protocols
The same request cannot be delivered and processed more than once
When do we want that?
A popular use: shopping cart
Payment for the order is done at most once
At-most-once protocols use sessions

18. Achieving At-Most-Once Semantics with Sessions
Communication is based on sessions
A session is used to keep track of the interactions between the client and the server
A session is identified by a unique identifier
If a session is broken, the server will not process requests from the old session
The credit card will not be charged twice
If the communication is continued after a broken session, a new session must be established

19. Creating a Session
Communicating parties must agree on the session identifier (so they can distinguish one session from another)
Sessions have unique names
How to choose a unique name?
Include a local timestamp in a session name
Include a random number in a session name

20. Case Study: HTTP Sessions
A series of related browser requests that
come from the same client
during a certain time period.
ties together a series of logically related browser requests, such as a shopping cart application.
HTTP protocol is stateless: the server has no way to associate multiple requests to the same client of browser
Nevertheless, many client-server Web applications have learned to support HTTP sessions via HTTP cookies

21. HTTP Cookies Cookie: a parcel of text Used for
sent by a server to a web browser
sent back unchanged by the browser
each time it accesses that server
Used for
Authentication (webmail)
Tracking user preferences (my Google page)
Shopping carts
Client
Server
request
reply

22. Example Text file in C:\Windows\cookies Contents:
UserID ABCD
Exercise: how would you use cookies in a good way? (I.e. for shopping cart or another service?)

23. Digression: Dark Criminal Past of the HTTP Cookie
Cookies can be used to track user requests across a single site or across multiple sites
This raises concerns of privacy
Can sell information about user preferences, use it for marketing purposes
White House Drug Policy office used cookies to track users viewing its online anti-drug advertising
In 2002 it was found that CIA had been leaving persistent cookies on user computers for ten years
Use of cookies is regulated by law in the US and the EU

24. Setting HTTP Cookies GET /index.html HTTP/1.1 Host: www.w3.org
Browser
Server
HTTP/ OK Content-type: text/html Set-Cookie: name=value   (content of page)
Browser
Server
GET /spec.html HTTP/1.1 Host: Cookie: name=value Accept: */*
Browser
Server

25. HTTP Sessions Based on Cookies
A new session is created by setting a new cookie
Each related request from the browser is accompanied by a cookie
Helps create at-most-once semantics:
If a user attempts to pay for the order the second time, error is reported
If the server crashes, the session is terminated, the client needs to start over
There are alternatives to cookies, such as URL rewriting, hidden form fields, etc.

26. Sessions and Retransmission
A client sends a request
No response from the server
The client retransmits
Problem: How does a server now if this is a retransmission or a new request?
Recall – we have to maintain at-most-once semantics!
Client
Server
request
That’s taking too long!
process request
I am going to retransmit
request (r) ?

27. Solution: Message Numbering
Ah, I’ve already seen that message. Still working on it
Client
Server
request0
process request
Those clients are so impatient!
request0 (r)

28. Protocols and Applications
A protocol is usually tailored to the type of application that uses it
Different protocols exist for:
Remote operations
Bulk data transfer
One-to-many communication
Continuous media

29. Protocols for Remote Operations
the most basic form of communication
client sends a message to a server, asking it to do some work
server does the work and responds to the client
Learn how to build remote operation protocols with the desired properties
Investigate what aspects contribute to high performance

30. Useful Properties of Protocols
At-most-once semantics
Positive feedback when no failures occur
The server notifies the client that the request has been completed
The server notifies the client that it is still working on the request (avoid useless retransmissions)
Notification of errors when there must have been failure
Tell the client if the network is broken
This does not tell the client what really happened with its request
May help performance – eliminates some uncertainty
Low end-to-end latency
Support for very large request and reply messages

31. A Protocol With Feedback
Each message is acknowledged
Keep-alive messages are used to detect server crashes (is this guaranteed to work?)
Problem: this is too slow
Usually a server sends response quickly enough, obviating the need for ACK
A reply doubles as a server ACK
A client’s next request doubles as a client ACK
Client
Server
request
ack
alive?
ack
reply
ack

32. A Fast Protocol A protocol optimized for “fast operations”
No ACKs of keep-alive messages
Works fine when there are no failures
What happens if there are failures?
Ideally: need a protocol that
Works as Fast Protocol when there are no failures
Works as the Protocol with Feedback when there are failures
Client
Server
request
reply
request
reply
request
reply

33. Fast Protocol With Feedback
ID of next message in sequence
Client
Server
Common case: no errors, no extra messages
seq = 0
request0
timer= s
reply0
retransmission timer
seq = 1
And when there are errors…
request1 s
timer expires
request1 (r)
BUT WHAT ABOUT DETECTING SERVER CRASHES?
client retransmits

34. Letting the Client Know the Server is Alive
seq = 0
server sends ACK when it sees request for the second time (allow for fast common case)
request0
timer= s s
gotack = F
request0 (r)
keeps track of server ACKs
ack0
gotack = T
timer expired, but gotack=T retransmit only the header (like a keepalive)
timer= s+l
set gotack flag
s+l
request1 (r-hdr)
extend the timer
(the server must be slow…)
ack1

35. Recall Desired Properties of Protocols
At-most-once semantics
Positive feedback when no failures occur
Notification of errors when there must have been failure
Low end-to-end latency
Support for very large request and reply messages

36. Protocol With Large Message Support
So far we have assumed that each request is sent in one packet
What if we need more packets?
We represent a request as a packet blast
A request is represented as n blasts (B1, B2, …, Bn)
Our protocol is easily extended – the client does to a blast what it did to each packet
With one modification: a server ACKs each blast
We want to avoid retransmissions by the client – this is costly with large requests

37. Blast Protocol For i = 1, to n, the client: Sends Bi
Starts a retransmission timer
Waits for timer expiration or an ACK
If timer expires, goes to Step 1.
If ACK is received, the client cancels the timer and iterates

38. Protocol for Large Requests
Client
Server
seq = 0
blast0
server sends ACK for every blast
(avoid costly retransmissions)
timer= s
gotack = F
ack0
seq = 1
timer= s
blast1
gotack = T s
blast1 (r)

39. Case Study: HTTP Protocol
A simple protocol with retransmission and feedback
Stateless
Sessions can be used on top of HTTP, but are not part of HTTP
Typical operation:
A client sends a request, starts a retransmission timer
A server sends a response
If there is no response and the timer expired, the client retransmits the request

40. HTTP Request Message Format
© Pearson Education 2001
Method: a type of an operation to be performed on a resource
URL: unique name of a resource
HTTP version: protocol version
Headers: additional information about the resource
Message body: the data itself

41. HTTP Response Message Format
© Pearson Education 2001
Status code: indicates success or failure Reason: human-readable explanation of the code

42. HTTP Methods
HEAD – ask for information about the resource, such as the modification time
GET – request the data representing the resource
POST – submit data to be processed (i.e., an HTML form)
PUT – upload data representing the resource
DELETE – delete the specified resource
TRACE – Echoes back the received request, so that a client can see what intermediate servers are adding or changing in the request.
OPTIONS – Returns the HTTP methods that the server supports. This can be used to check the functionality of a web server.
CONNECT – Converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS).

43. HTTP Status Codes 1xx Informational 2xx Success 3xx Redirection
100 Continue
2xx Success
200 OK Example: HTTP/ OK
3xx Redirection
301 Moved Permanently
4xx Client Error
404 Not Found Example: HTTP/ Not Found
5xx Server Error
500 Internal Server Error

44. Examples of HTTP Headers
Allowed: *Method
Lists the set of requests which the requesting user is allowed to issue for this URL.
Example of use: Allow: GET HEAD PUT
Content-Length: int
Specifies length for the binary content
Example of use: Content-Length: 1354
Content-Type: type
Type of data being transferred
Example of use: Content-Type: text/html

45. Let’s Look At An Example Exchange
Client request:
GET /index.html HTTP/1.1
Host:
\r\n at the end of line
blank line (\r\n)
Server response:
HTTP/ OK
Date: Mon, 23 May :38:34 GMT
Server: Apache/ (Unix) (Red-Hat/Linux)
Last-Modified: Wed, 08 Jan :11:55 GMT
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
(page content goes here)

46. Protocols Summary We have talked about application level protocols
need them for end-to-end failure management
transport protocols do not handle client and server crashes, only message loss
Protocol semantics:
Exactly once – cannot be provided in presence of failures
At least once – used by stateless services, not enough for many applications
At most once – must rely on sessions
Stateless and stateful servers
NFS is a stateless file system protocol

47. Protocols Summary (cont)
Desirable properties of protocols:
At-most-once
Feedback on success or failure
Low latency
Support for large messages
At-most-once property is achieved using sessions
HTTP sessions via use of cookies

48. Protocols Summary (cont)
Low latency is achieved by reducing unnecessary ACKs and retransmissions
The server sends an ACK only in case of retransmission
If the client suspects the server is alive but slow, retransmit only the header
For very large messages the server acknowledges every blast to avoid unnecessary and costly retransmissions by the client
We looked at message format in HTTP protocol

49. Today: Inter-Process Communication
RPC – Remote Procedure Call
RMI – Remote Method Invocation
IPC abstractions
DISCUSSED THAT
Protocol: rules of message exchange on the wire

50. RPC A remote procedure that looks just like a local procedure
Transparent call semantics: an RPC call looks just like a local call

51. What RPC Specifies
Abstract Syntax specifies a procedure interface for remote services that is integrated with client programming language.
Transfer Syntax specifies rules for encoding and decoding arguments and results.

52. Overview of the RPC System
Server exports
a collection of procedures
data-type definitions for the arguments
How this looks to client
like a standard programming-language library linked into the application and executed locally
RPC library
handles the encoding, transfer, and decoding of arguments sent from client to server and of results send back from server to client.
RPC protocol
Built on top of a reliable request-response client-server IPC protocol (discussed earlier in the lecture). .

53. Approaches to Implementing RPC
First-class RPC Integrated with programming language.
The first RPC system was from Xerox PARC's Cedar system; it used first-class RPC incorporated with Cedar's programming language, called Mesa,
Possible drawback is that it limits openness, because it restricts what languages clients can use.
Second-class RPC An extension to the language.
Interface language describes the types and procedures exported by a server module.
Interface-language "compiler" generates client and server parts for many different programming languages from the same interface definition file.
Example is SunRPC, which uses an interface definition language called XDR.

54. Case Study: Sun RPC Client application calls the client stub procedure
© Pearson Education 2001
Client application calls the client stub procedure
Client stub procedure has the same interface as the procedure the client wants to invoke on the server
Client stub procedure asks the server to execute that procedure remotely

55. Sun RPC (continued)
© Pearson Education 2001
Client stub copies parameters from the stack into a request message
Calls on communication module (a protocol implementation) to ship the request to the server
The server’s communication module receives the message

56. Sun RPC (continued) The server stub procedure decodes the message
© Pearson Education 2001
The server stub procedure decodes the message
Takes call parameters out of the message
Executes the service procedure, the actual remote subroutine

57. Sun RPC (continued)
© Pearson Education 2001
The server puts the result parameters in a reply message
Invokes communication module to ship them back to client
The client receives and decodes the results

58. How To Program With RPC?
The programmer does not need to implement client and server stubs
The programmer just writes the definition of the RPC procedure and arguments in a special language called XDR
The rpcgen compiler creates client and server stubs from the XDR file
RPC library provides the implementation of the protocol

59. Automatic Stub Generation
Example: an RPC that returns “date” from a remote server
File date.x contains interface and data representation of the RPC
date.x
rpcgen
Client stubs:
date.h
date_clnt.c
Server stubs:
date.h
date_server.c

60. The Entire IPC Program CLIENT SERVER Generated by rpcgen: date.h
includes
includes
includes
includes
Written by programmer:
rpc_date_client.c
Written by programmer:
rpc_date_server.c
calls
calls
Generated by rpcgen:
date_client.c
Generated by rpcgen:
date_server.c
client stub
server stub

61. Let’s Look at the Code: date.x
/* * date.x - Specification of remote date and time service. */ * Define 2 procedures: * bin_date_1() returns the binary time and date (no arguments). * str_date_1() takes a binary time and returns a string. program DATE_PROG { version DATE_VERS { long BIN_DATE(void) = 1; /* procedure number = 1 */ string STR_DATE(long) = 2; /* procedure number = 2 */ } = 1; /* version number = 1 */ } = 0x ; /* program number = 0x */

62. Server Stub: date_server.c
/* * Please do not edit this file. * It was generated using rpcgen. */ ... #include "date.h" static void date_prog_1(); main() { register SVCXPRT *transp; //register the server functions svc_run(); }
A thread that listens for incoming RPC requests

63. Server Stub date_server.c (continued)
static void date_prog_1 (struct svc_req * rqstp, SVCXPRT * transp) { ... switch (rqstp->rq_proc) { case BIN_DATE: xdr_argument = (xdrproc_t)xdr_void; xdr_result = (xdrproc_t)xdr_long; local = (char *(*)()) bin_date_1; break; case STR_DATE: xdr_argument = (xdrproc_t)xdr_long; xdr_result = (xdrproc_t)xdr_wrapstring; local = (char *(*)()) str_date_1;
Unmarshall the arguments, find the address of the procedure

64. Server Stub date_server.c (continued)
... result = (*local)(&argument, rqstp); if (result != NULL && !svc_sendreply(transp, xdr_result, result)) { svcerr_systemerr(transp); }
Execute the procedure, send the response back
ALL THIS CODE IS GENERATED AUTOMATICALLY!!!
THE PROGRAMMER ONLY WRITES THE ACTUAL PROCEDURE

65. RPC Service Procedures: rpc_date_server.c
/* * rpc_date_server.c - remote procedures; called by server stub. */ #include <rpc/rpc.h> /* standard RPC include file */ #include "date.h" /* this file is generated by rpcgen */ * Return the binary date and time. long * bin_date_1() { static long timeval; /* must be static */ long time(); /* Unix function */ timeval = time((long *) 0); return(&timeval); }
THIS IS THE PART WRITTEN BY THE PROGRAMMER

66. The Rest of the Code We will skip the client code: The idea is similar
rpc_date_client.c (written by programmer)
date_client.c (generated by rpcgen)
The idea is similar
If you’d like to see full examples, visit

67. Is RPC Completely Transparent?
Not easy to get transparency
Reasons for obstacles:
Client and server are distributed
Client and server run in different address spaces
Integrating RPC with a programming language creates problems

68. Obstacles Due to Distribution: Naming
Clients must name the server they are calling as part of the remote-procedure call.
This often means that the remote version of a function call has an extra argument (i.e, the server communication handle).
server says: foo(i)
client says: foo(server,i)

69. Obstacles Due to Distribution: Name Binding
The name of a procedure is "bound" to its implementation
Local procedure calls are bound to their implementation at compile time by the linker (e.g., ld)
Remote procedure calls are bound to their implementation at runtime
This is necessary to allow the same code to contact different servers when it runs
New error semantics: a program may fail after at runtime because of failed binding

70. Obstacles Due to Distribution: Failures
During a local procedure, the program fails as a whole
Client and server can fail independently.
What to do?
Fail the client program anytime a called server fails?
But this reduces the reliability of clients.
A conventional solution:
Expose a possible error return to clients
Introduce a new kind of error code, which would not occur for local calls
Example: an RPC procedure date() may return input/output error!

71. Obstacles Due to Distribution: Performance
Remote calls are much slower than local calls.
Remote-call performance is typically much less predictable than local-call performance
A client might want to deal with performance:
Use multithreading for RPC calls
Make sure not to use RPC in performance critical section
If you hide “remoteness” from client, the client cannot deal with performance issues
So it is better to expose remoteness to the client
It’s a bad idea to completely maintain transparency

72. Obstacles due to Different Address Spaces of Caller and Callee (I)
Global Variables
Server and client can't access each other's global variables.
For example, a client and server can't use a shared lock to synchronize in the say way that a local procedures can
Solution: Must rely on distributed locks
Context-Sensitive Variables
Some variables have meaning only from within a particular address space
Examples: Unix sockets and file descriptors
Solution: do not use file descriptors, pass the file path to the server

73. Obstacles due to Different Address Spaces of Caller and Callee (II)
Pointers
A client can't pass a pointer to a server because the server can't read the clients memory (virtual address spaces are different)
Solution: pointer pickling
Client "pickles" the data structure by following all of the pointers and copying the data that's pointed to a byte stream
The server then gets the "transitive closure" of objects the pointer references

74. Pointer Picking (continued)
Solution: pointer pickling
In the bytestream, pointers are replaced with a reference to the object in the bytestream
“Unpickling” on the server:
Reading the byte stream from a client
Creating versions of each object in the stream
Changing the pointers to point to virtual addresses of the newly created objects on the server
Pickling requires type information in order to find the pointers in an object
Pickling is difficult in weakly-typed languages (e.g., C)

75. Obstacles Due to Client Programming Language Issues
IN/OUT Parameters
IN parameters are those that are only read
OUT parameters are those that are written
Function signature does not tell us if a parameter is IN or OUT
It is up to the implementation of the function to write the parameter
If the RPC does not know that the parameter was written by the server procedure, it will not transmit the correct return value to the client

76. Example: Argument Aliasing
void square (int* out, int* in) {
*out = *in * *in; }
Argument aliasing: the same actual arg passed to multiple formal args:
int x = 2;
square(&x,&x);
What is the value of x after calling square on local host?
x equals 4
What will it be if we execute it remotely?

77. square Executed Remotely
On client: put_int(*out) put_int(*in) send(server)
On server: in = get_int() // in = 2 out = get_int() out = in * in; // Oops, the server did not modify the “in” parameter!!! add_int_to_resp(out) // out = 4 add_int_to_resp (in) // in = 2
Back on client: *out = get_int() *in = get_int()
Result: x equals 2! The RPC does not know that x should have been modified

78. RPC Summary Remote procedure call
The idea: provide local call semantics
Transparency cannot be provided:
Different naming
Different name binding
Different failure semantics – client and server can fail independently
Slower performance – do not want to hide this from programmer
Cannot use global variables (have to use distributed locks)
Cannot use pointers (point pickling)
Transparency cannot be provided due to IN/OUT parameters