CS 582 / CMPE 481 Distributed Systems Communications (cont.)

Remote Procedure Call (RPC)

Motivation Construction of distributed applications is a difficult task –Heterogeneity –Separation –Partial failure Leverage a local procedure call mechanism [Birrell & Nelson] –procedure calls are a well-known and well-understood mechanism for transfer of control and data within a program –Well engineered processes operate in isolation (black box) –the same principle can be extended to the client-server interaction clean and simple semantics Efficiency Generality –Communication between caller & callee can be hidden by using procedure call mechanism

Local Procedure Call before after E.g. read(int fd, char* buf, int bytes) 1.push parameter values of the procedure on a stack 2.call procedure 3.use stack for local variables 4.pop results (in parameters) “communication” with local procedure – copy data to/from stack

RPC Principle

LPC vs. RPC Client-server fails independently –extra exception handling –return value needs to be overloaded with failure codes from the server Global variables and pointers cannot be used across the interface

Design Consideration Abstraction of RPC semantics Heterogeneity Parameter passing Binding Failure Transparency Support for concurrency Orphan handling Performance

Level of Abstraction of RPC Semantics Integrate into the language –extend language to provide constructs for RPC semantics –examples: Cedar, Argus, Mercury Describe in a special purpose interface definition language –interface definition language supports language independence limited representation of parameter types –examples: Sun XDR, DCE IDL, Xerox Courier

Example: Argus Provides programming language constructs for remote operation invocation –Guardian a special kind of an object which implements a number of procedures that are run in response to remote requests interface is defined as a set of handlers –Action supports atomic transactions Ref –B. Liskov, “Distributed programming in Argus,” CACM 31(3), pp , March 1988.

Examples: Sun RPC Provides XDR language for interface definition –each procedure is numbered with a version number –the type of a result and an input parameter is either simple or complex (e.g. struct) –only one parameter is allowed (when -N option is not used) parameter is always input multiple parameters -> a single structure

Heterogeneity Marshalling policy –conversion to the common data type agreed by both a sender and a receiver: Sun RPC –a receiver makes it right (tagging): NCS RPC –sender negotiates with a receiver RPC mechanisms –define a veneer layer to encapsulate multiple RPC implementations [HRPC] control, data representation, and transport components Accommodate multiple transport protocols

Parameter Passing Considerations for parameter passing in RPC –direction of parameters input: client -> server output: server -> client input & output: client server –call-by-reference use call-by-value-result instead –pointer parameters no meaning to pass a pointer to memory location linearize before passing it –procedure parameter procedure should be accessible across the network

Binding Naming –mapping from a name to a particular service static linking: most RPCs dynamic linking: Apollo NCS RPC procedure variable: Argus, Mercury –exportation and importation server exports its service interface client imports exported service –interface consists of name: uniquely identifiable –version info and location info can be included signature: parameter name & type

Binding How to determine where server is? Which procedure to call? –“Resource discovery” problem Broadcast requests –broadcast request & process incoming replies Name service: advertises servers and services. –Example: Birrel et al. uses Grapevine. Early versus late binding. –Early: server address and procedure name hard-coded in client. –Late: go to name service.

Binding (cont.) Locating a server –via a binder (like a directory service) well-know address run-time notification broadcasting (recruiting)

Failure Transparency Ordered delivery –request and reply delivered in order (e.g. Argus) RPC Execution semantics –How many times will a remote procedure be executed by the callee per call? –Semantics at-least-once at-most-once exactly-once

Execution Semantics At-least-once –when a caller receives the expected result, it implies that the requested call has been executed one or more (at least once) at a callee –the callee does not remember multiple executions due to call duplication or retry after failure –acceptable if operations are idempotent –example: Sun RPC - NFS

Execution Semantics (cont.) At-most-once –when a caller receives the expected result, it implies that the requested call has been executed exactly once. If not completed, executed zero or one time –the callee can detect duplicated requests at the absence of failure but persist through the abnormal termination (failure amnesia) –example: NCS RPC

Execution Semantics (cont.) Exactly-once –when a caller receives the expected result, it implies that the requested call has been executed exactly once. If not completed, executed zero times –the callee is failure-resilient: logging and two-phase commit –example: Argus

Support for Concurrency RPC behaves differently from LPC –the client does not need to be blocked while the server executes the requested call –the client may not need the result Approaches –Synchronous RPC a process/thread per call (at a client or a server) –Asynchronous RPC two types –no result –delayed result: delayed synchronous Examples: Argus, Mercury(Promises), X11 RPC

Synchronous & Asynchronous RPC Synchronous Asynchronous Client Server

Orphan Handling A server becomes an orphan when its results are no longer needed due to –duplicate call messages –client crashes or aborts in the middle of a call Orphans are bad because –resource waste –Interference –Inconsistency Orphan termination –server detects a duplicate message and aborts an orphan if exists –client keeps a list of servers and issues abort requests to them as part of the crash recovery procedure

Performance Performance consideration points –marshalling and unmarshalling –message transmission –request execution semantics –execution time Approaches –Leverage highly tuned transport protocols –Allow an application to make a right choice based on its needs request-n-reply: high throughput -> RPC large data transfer: low latency -> byte streams

RPC Implementation work CallerCallee Call packet Result User Client stub RPC runtime RPC runtime Server stub Server call pck args xmit rcv unpk call return pck result xmit rcv unpk result return

RPC Implementation RPC runtime mechanism responsible for retransmissions, acknowledgments. Stubs responsible for data packaging and un- packaging; –AKA marshalling and un-marshalling: putting data in form suitable for transmission. Example: Sun’s XDR.

Case Study : Sun RPC Defines format for messages, arguments, and results. –Uses UDP or TCP. over UDP message length ≤ 64 kbytes ~ 8-9 kbytes –broadcast RPC is an option –Uses XDR (eXternal Data Representation) to represent procedure arguments and header data. –Compiler system to automatically generate distributed programs. –Remote execution environment: remote program mutually exclusive execution of procedure in remote program

Identifying Remote Programs and Procedures Conceptually, each procedure on a computer is identified by pair : –(prog, proc) prog: 32-bit integer identifying remote program proc: integer identifying procedure –Set of program numbers partitioned into 8 sets. 0x x1fffffff assigned by SUN 0x x3fffffff assigned by local system manager 0x x5fffffff temporary 0x xffffffff reserved –Multiple remote program versions can be identified: (prog, version, proc)

Example RPC Program Numbers nameassigned nodescription portmap100000port mapper rstatd100001rstat, rup, perfmeter rusersd100002remote users nfs100003file system ypserv100004yp (NIS) mountd100005mount, showmount dbxd100006DBXprog (debug) ypbind100007NIS binder walld100008rwall, shutdown yppasswdd100009yppasswd

Communication Semantics TCP or UDP ? Sun RPC semantics defined as function of underlying transport protocol. RPC on UDP: calls can be lost or duplicated. at-least-once semantics if caller receives reply. zero-or-more semantics if caller does not receive reply. Programming with zero-or-more semantics: idempotent procedure calls. Sun RPC retransmission mechanism: –non-adaptive timeouts –fixed number of retransmissions

Remote Programs & protocols Ports