Presentation on theme: "Three types of remote process invocation"— Presentation transcript:
1 Three types of remote process invocation Single (jobtype=single)grpc_function_handle_init(…);Multiple (jobtype=multiple)grpc_function_handle_array_init_np(…);gatekeeperClientServergatekeeperClientServer
2 Three types of remote process invocation (cont’d) MPI (jobtype=MPI)grpc_function_handle_init(…);“Backend=MPI” is specified in the IDLmpirunprungatekeeperClientServer
3 Ninf-G fnctions for fault detection Detects errors appropriately and an error code is returnedserver processes diednetwork disconnectionOther functions for fault detectiontimeoutfor initialization (e.g. stacked in the queue, etc.)for rpc (something unexpected would happen)for heatbeat (heatbeat messages have been continuously missed)If these faults are detected, API returns an error and the error code indicates the timeout error.
4 Remarkable Features of Ninf-G4 (summary) Enabling RPC via any middlewarePre-WS GRAM, WS GRAM, UNICORECondor, SSH, NAREGI SSAny othersClient configuration file that allows detailed description of execution environmentsServer, Client, Function attributes, etc.Capability for fault detectionExplicit faultsServer process died, network disconnection, etc.Implicit faultsTimeout of Invocation, Execution, HeatbeatHigh quality as the software
6 How to use Ninf-G Build remote libraries on server machines Write IDL filesCompile the IDL filesBuild and install remote executablesDevelop a client programProgramming using GridRPC APICompileRunCreate a client configuration fileGenerate a proxy certificate
7 Rough steps for RPC (client-side programming) InitializationCreate a function handleabstraction of a connection to a remote executableCall a remote librarygrpc_initialize(config_file);grpc_function_handle_t handle;grpc_function_handle_init(&handle, host, port, “lib_name”);grpc_call(&handle, args…); orgrpc_call_async(&handle, args…);grpc_wait( );
8 Terminloogy (cont’d) Ninf-G Executable Session GridRPC API Ninf-G executable is an executable file that will be invoked by Ninf-G systems. It is obtained by linking a user-written function with the stub code, Ninf-G and the Globus Toolkit libraries.SessionA session corresponds to an individual RPC and it is identified by a non-negative integer called Session ID.GridRPC APIApplication Programming Interface for GridRPC. The GridRPC API is going to be standardized at the GGF GridRPC WG.
9 Terminology Ninf-G Client Ninf-G IDL Ninf-G Stub This is a program written by a user for the purpose of controlling the execution of computation.Ninf-G IDLNinf-G IDL (Interface Description Language) is a language for describing interfaces for functions and objects those are expected to be called by Ninf-G client.Ninf-G StubNinf-G stub is a wrapper function of a remote function/object. It is generated by the stub generator according to the interface description for user-defined functions and methods.
10 How to build Remote Libraries - server side operations - Ninf-GHow to build Remote Libraries- server side operations -
11 Ninf-G remote libraries Ninf-G remote libraries are implemented as executable programs (Ninf-G executables) whichcontain stub routine and the main routinewill be spawned off by GRAM or other middlewareThe stub routine handlescommunication with clients and Ninf-G system itselfargument marshallingUnderlying executable (main routine) can be written in C, C++, Fortran, etc.
12 Ninf-G remote libraries (cont’d) Ninf-G provides two kinds of Ninf-G remote executables:FunctionStatelessDefined in standard GridRPC APINinf-G objectstatefulenables to avoid redundant data transfersmultiple methods can be definedinitializationcomputation
13 How to build Ninf-G remote libraries (1/3) Write an interface information using Ninf-G Interface Description Language (Ninf-G IDL). Example: Module mmul; Define dmmul (IN int n, IN double A[n][n], IN double B[n][n], OUT double C[n][n]) Require “libmmul.o” Calls “C” dmmul(n, A, B, C);Compile the Ninf-G IDL with Ninf-G IDL compiler % ng_gen <IDL_FILE> ns_gen generates stub source files and a makefile (<module_name>.mak)
14 How to build Ninf-G remote libraries (2/3) Compile stub source files and generate Ninf-G executables and LDIF files (used to register Ninf-G remote libs information to GRIS). % make –f <module_name>.makPublish the Ninf-G remote libraries (optional) % make –f <module_name>.mak install
15 How to build Ninf-G remote libraries (3/3) Ninf-G IDL file <module>.idlClientng_gen_stub_foo.cMDS_stub_bar.c_stub_goo.c<module>.mak_stub_foomake –f <module>.mak_stub_bar_stub_gooLibrary programlibfoo.a<module>::foo.ldif<module>::bar.ldif<module>::goo.ldif
16 How to call Remote Libraries - client side APIs and operations - Ninf-GHow to call Remote Libraries- client side APIs and operations -
17 (Client) User’s Scenario Write client programs in C/C++/Java using APIs provided by Ninf-GCompile and link with the supplied Ninf-G client compile driver (ng_cc)Write a client configuration file in which runtime environments can be describedRun grid-proxy-init command (if you use GRAM)Run the program
18 GridRPC API / Ninf-G API APIs for programming client applications
19 The GridRPC API and Ninf-G API Standard C API defined by the GGF GridRPC WG.Provides portable and simple programming interface.Enable interoperability between implementations suchas Ninf-G and NetSolve.Ninf-G APINon-standard API (Ninf-G specific)complement to the GridRPC APIprovided for high performance, usability, etc.ended by _npeg: grpc_function_handle_array_init_np(…)
20 Rough steps for RPC Initialization Create a function handle abstraction of a connection to a remote executableCall a remote librarysynchronous or asynchronous callgrpc_initialize(config_file);grpc_function_handle_t handle;grpc_function_handle_init(&handle, host, port, “func_name”);grpc_call(&handle, args…); orgrpc_call_async(&handle, args…);grpc_wait( );