Presentation is loading. Please wait.

Presentation is loading. Please wait.

The HDF Group www.hdfgroup.org Milestone 5.1: Initial POSIX Function Shipping Demonstration Jerome Soumagne, Quincey Koziol 09/24/2013 © 2013 The HDF Group.

Published byCornelius Bradford Modified over 8 years ago

Similar presentations

Presentation on theme: "The HDF Group www.hdfgroup.org Milestone 5.1: Initial POSIX Function Shipping Demonstration Jerome Soumagne, Quincey Koziol 09/24/2013 © 2013 The HDF Group."— Presentation transcript:

1 The HDF Group www.hdfgroup.org Milestone 5.1: Initial POSIX Function Shipping Demonstration Jerome Soumagne, Quincey Koziol 09/24/2013 © 2013 The HDF Group 1

2 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Overview – Mercury © 2013 The HDF Group 2 Mercury “Function Shipper”: RPC layer that supports Non-blocking transfers Large data arguments (w/RMA) Native transport protocols of HPC systems Mercury serves as a basis for higher-level frameworks that need to operate on/store/access data remotely HDF5 IOD virtual object plugin IOFSL I/O forwarding scalability layer Storage systems Analysis frameworks

3 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Overview – Mercury © 2013 The HDF Group 3 Already largely presented in previous milestones No major modification of Mercury for this deliverable in order to support POSIX calls But Mercury is still being improved: Performance tuning on Infiniband cluster Support for additional network transports is being added (TCP / ibverbs / SSM) Paper submitted at end of Q4 now accepted and being presented at IEEE Cluster 2013: J. Soumagne, D. Kimpe, J. Zounmevo, M. Chaarawi, Q.Koziol, A. Afsahi, and R. Ross, “Mercury: Enabling Remote Procedure Call for High-Performance Computing”, IEEE International Conference on Cluster Computing, Sep 2013

4 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Fast Forward Stack – Function Shipping © 2013 The HDF Group 4 HDF5 API VOL Mercury (Client) Mercury (Client) Mercury (Server) Mercury (Server) Native (H5) IOD VOL Network IOD VOL VFL … …

5 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration POSIX Function Shipping (Example) © 2013 The HDF Group 5 HDF5 API VOL VFL File System Mercury (Client) Mercury (Client) Mercury (Server) Mercury (Server) Native (H5) IOD VOL sec2 Network POSIX I/O

6 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX © 2013 The HDF Group 6 Support POSIX I/O routines through Mercury Completely separate package built on top of Mercury called: Mercury POSIX (lightweight library + server) Design keys: Support 32/64 bit platforms and large files No modification of original source code that uses POSIX I/O (e.g., HDF5 sec2 driver) Redirects I/O to Mercury server with dynamic linking Can make use of all the transports available through Mercury (although MPI dynamic connection is not really flexible and always available) Code for supporting POSIX routine is automatically generated inside Mercury POSIX by using BOOST preprocessor macros

7 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 7 Most routines are generated with one line macro Built on top of existing Mercury/Boost macros However supporting variable arguments routines requires some extra lines to create encoding / decoding routines that check argument flags etc

8 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 8 Two main macros: /* Non-bulk routines */ MERCURY_POSIX_GEN_STUB(func_name, ret_type, in_types, out_types) /* Bulk routines */ MERCURY_POSIX_GEN_BULK_STUB(func_name, ret_type, in_types, out_types, bulk_read)/* 1/0 if reading/writing bulk data */

9 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 9 Example, showing results of the following macro: /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), )

10 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 10 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate input structure typedef struct { hg_int32_t in_param_0; hg_off_t in_param_1; hg_int32_t in_param_2; } lseek_in_t;

11 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 11 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate proc routine for input structure static __inline__ int hg_proc_lseek_in_t(hg_proc_t proc, void *data) { lseek_in_t *struct_data = (lseek_in_t *) data; hg_proc_hg_int32_t(proc, &struct_data->in_param_0); hg_proc_hg_off_t(proc, &struct_data->in_param_1); hg_proc_hg_int32_t(proc, &struct_data->in_param_2); return ret; }

12 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 12 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate output structure typedef struct { hg_off_t ret; } lseek_out_t;

13 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 13 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate proc routine for output structure static __inline__ int hg_proc_lseek_out_t(hg_proc_t proc, void *data) { lseek_out_t *struct_data = (lseek_out_t *) data; hg_proc_hg_int64_t(proc, &struct_data->ret); return ret; }

14 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 14 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate client stub (simplified version) hg_off_t lseek(hg_int32_t in_param_0, hg_off_t in_param_1, hg_int32_t in_param_2) { lseek_in_t in_struct; lseek_out_t out_struct; hg_off_t ret; /* Initialization */...

15 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 15 /* Register function if not registered */ MERCURY_REGISTER("lseek", lseek_in_t, lseek_out_t); /* Fill input structure */ in_struct.in_param_0 = in_param_0; in_struct.in_param_1 = in_param_1; in_struct.in_param_2 = in_param_2; /* Forward call to remote addr and get a new request */ HG_Forward(addr, id, &in_struct, &out_struct, &request); /* Wait for call to be executed */ HG_Wait(request, HG_MAX_IDLE_TIME, &status); /* Get output parameters */ ret = out_struct.ret; return ret; }

16 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 16 /* off_t lseek(int fildes, off_t offset, int whence) */ MERCURY_POSIX_GEN_STUB(lseek, hg_off_t, (hg_int32_t)(hg_off_t)(hg_int32_t), ) Generate server stub (simplified version) static int lseek_cb(hg_handle_t handle) { lseek_in_t in_struct; lseek_out_t out_struct; hg_int32_t in_param_0; hg_off_t in_param_1; hg_int32_t in_param_2; hg_off_t ret;

17 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX – Stub Generation © 2013 The HDF Group 17 /* Get input buffer */ HG_Handler_get_input(handle, &in_struct); /* Get parameters */ in_param_0 = in_struct.in_param_0; in_param_1 = in_struct.in_param_1; in_param_2 = in_struct.in_param_2; /* Call function */ ret = lseek (in_param_0, in_param_1, in_param_2); /* Fill output structure */ out_struct.ret = ret; /* Free handle and send response back */ HG_Handler_start_output(handle, &out_struct); }

18 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX © 2013 The HDF Group 18 Routines currently supported: accessfdatasyncmkdirtruncate chdirfpathconfmkfifoumask chmodfstatmknodunlink chownfsyncopenwrite creatftruncatepathconf closegetcwdread+Large file versions: duplchownreadlinkcreat64 dup2linkrmdirftruncate64 fchdirlockfstatlseek64 fchmodlseeksymlinkopen64 fchownlstatsyncetc.

19 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX © 2013 The HDF Group 19 Routines not yet supported: closedir pipereaddir fcntlpreadrewinddir+Large file versions: opendirpwriteutime?

20 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX - Configuration © 2013 The HDF Group 20 Environment variables required: MERCURY_NA_PLUGIN : Underlying network transport method used to forward calls to remote servers. e.g., "bmi” MERCURY_PORT_NAME : Port name information (IP/port) specific to the network transport chosen – used to establish a connection with a remote server. e.g., "tcp://72.36.68.242:22222” LD_PRELOAD: Path to Mercury POSIX shared library. e.g., “/usr/local/lib/libmercury_posix.so” Setting LD_PRELOAD redirects all POSIX calls to the Mercury server (can be an issue with local scripts, etc. that make use of POSIX I/O)

21 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Mercury POSIX - Testing © 2013 The HDF Group 21 Integrated regression tests (limited POSIX test suite) HDF5 sec2 driver (demo) Lustre POSIX test suite However: framework issues, needs to be modified, possibly need to support fdopen and FILE* routines?

22 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Demo – Mercury POSIX and HDF5 tools © 2013 The HDF Group 22 $ pwd ~jsoumagne/demo $ ls *.h5 ls: *.h5: No such file or directory $ export MERCURY_NA_PLUGIN=“bmi” $ export MERCURY_PORT_NAME=“tcp://127.0.0.1:22 222” $ export LD_PRELOAD=/path/to/libmercury_posix. so $ pwd ~jsoumagne/demo_server $ ls coord.h5 $ mercury_posix_server bmi Waiting for client...

23 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Demo – Mercury POSIX and HDF5 tools © 2013 The HDF Group 23 $ h5dump -H coord.h5 HDF5 "coord.h5" { GROUP "/" { DATASET "multiple_ends_dset" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 4, 5, 3, 4, 2, 3, 6, 2 ) / ( 4, 5, 3, 4, 2, 3, 6, 2 ) } } DATASET "multiple_ends_dset_chunked" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 4, 5, 3, 4, 2, 3, 6, 2 ) / ( 4, 5, 3, 4, 2, 3, 6, 2 ) } } DATASET "single_end_dset" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 2, 3, 6, 2 ) / ( 2, 3, 6, 2 ) }... (skip) $ mercury_posix_server bmi Waiting for client... Thu, 19 Sep 13 17:31:00 CDT: Executing open64 Thu, 19 Sep 13 17:31:00 CDT: Executing __fxstat64 Thu, 19 Sep 13 17:31:00 CDT: Executing lseek64 Thu, 19 Sep 13 17:31:00 CDT: Executing hg_posix_read Thu, 19 Sep 13 17:31:00 CDT: Executing lseek64 Thu, 19 Sep 13 17:31:00 CDT: Executing hg_posix_read Thu, 19 Sep 13 17:31:00 CDT: Executing getcwd... (skip)

24 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Demo – Mercury POSIX and HDF5 tools © 2013 The HDF Group 24 $ h5copy -i coord.h5 -s single_end_dset -o coord_simple.h5 -d simple Thu, 19 Sep 13 17:33:51 CDT: Executing open64... (skip) Thu, 19 Sep 13 17:33:51 CDT: Executing __fxstat64 Thu, 19 Sep 13 17:33:51 CDT: Executing lseek64 Thu, 19 Sep 13 17:33:51 CDT: Executing hg_posix_read Thu, 19 Sep 13 17:33:51 CDT: Executing lseek64 Thu, 19 Sep 13 17:33:51 CDT: Executing hg_posix_read... (skip) Thu, 19 Sep 13 17:33:51 CDT: Executing hg_posix_write Thu, 19 Sep 13 17:33:51 CDT: Executing close

25 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Demo – Mercury POSIX and HDF5 tools © 2013 The HDF Group 25 $ h5dump coord_simple.h5 HDF5 "coord_simple.h5" { GROUP "/" { DATASET "simple" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 2, 3, 6, 2 ) / ( 2, 3, 6, 2 ) } DATA { (0,0,0,0): 0, 1, (0,0,1,0): 1, 2,... (skip) (1,2,2,0): 122, 123, (1,2,3,0): 123, 124, (1,2,4,0): 124, 125, (1,2,5,0): 125, 126 } Thu, 19 Sep 13 17:36:57 CDT: Executing open64 Thu, 19 Sep 13 17:36:57 CDT: Executing __fxstat64 Thu, 19 Sep 13 17:36:57 CDT: Executing lseek64 Thu, 19 Sep 13 17:36:57 CDT: Executing hg_posix_read Thu, 19 Sep 13 17:36:57 CDT: Executing lseek64 Thu, 19 Sep 13 17:36:57 CDT: Executing hg_posix_read... (skip) Thu, 19 Sep 13 17:36:57 CDT: Executing lseek64 Thu, 19 Sep 13 17:36:57 CDT: Executing hg_posix_read Thu, 19 Sep 13 17:36:57 CDT: Executing close

26 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Conclusion – Future Work © 2013 The HDF Group 26 Very easy to forward POSIX I/O calls and does not require modification of existing tools / code Mercury POSIX can be easily extended to support additional system / library calls Can directly take advantage of updates to Mercury (network transports, etc.) Next Quarter: Support remaining POSIX routines Test with MPI I/O (ROMIO driver) Test with Lustre POSIX test suite If framework issues are solved

27 www.hdfgroup.org 09/24/2013Initial POSIX Function Shipping Demonstration Questions © 2013 The HDF Group 27

Download ppt "The HDF Group www.hdfgroup.org Milestone 5.1: Initial POSIX Function Shipping Demonstration Jerome Soumagne, Quincey Koziol 09/24/2013 © 2013 The HDF Group."

Similar presentations

Message Passing Vs Distributed Objects

Message Passing Vs Distributed Objects
A PLFS Plugin for HDF5 for Improved I/O Performance and Analysis Kshitij Mehta 1, John Bent 2, Aaron Torres 3, Gary Grider 3, Edgar Gabriel 1 1 University.

A PLFS Plugin for HDF5 for Improved I/O Performance and Analysis Kshitij Mehta 1, John Bent 2, Aaron Torres 3, Gary Grider 3, Edgar Gabriel 1 1 University.
RPC Robert Grimm New York University Remote Procedure Calls.

RPC Robert Grimm New York University Remote Procedure Calls.
Remote Procedure Call (RPC)

Remote Procedure Call (RPC)
Remote Procedure Call Design issues Implementation RPC programming

Remote Procedure Call Design issues Implementation RPC programming
1/1/ / faculty of Electrical Engineering eindhoven university of technology Introduction Part 3: Input/output and co-processors dr.ir. A.C. Verschueren.

1/1/ / faculty of Electrical Engineering eindhoven university of technology Introduction Part 3: Input/output and co-processors dr.ir. A.C. Verschueren.
Tam Vu (tvu@mail.eecis.udel.edu) Remote Procedure Call CISC 879 – Spring 03 Tam Vu (tvu@mail.eecis.udel.edu) March 06, 03.

Tam Vu Remote Procedure Call CISC 879 – Spring 03 Tam Vu March 06, 03.
GridRPC Sources / Credits: IRISA/IFSIC IRISA/INRIA Thierry Priol et. al papers.

GridRPC Sources / Credits: IRISA/IFSIC IRISA/INRIA Thierry Priol et. al papers.
Implementing Remote Procedure Calls Andrew Birrell and Bruce Nelson Presented by Kai Cong.

Implementing Remote Procedure Calls Andrew Birrell and Bruce Nelson Presented by Kai Cong.
T-FLEX DOCs PLM, Document and Workflow Management.

T-FLEX DOCs PLM, Document and Workflow Management.
 Introduction Originally developed by Open Software Foundation (OSF), which is now called The Open Group (www.opengroup.org) Provides a set of tools and.

 Introduction Originally developed by Open Software Foundation (OSF), which is now called The Open Group ( Provides a set of tools and.
Ameoba Designed by: Prof Andrew S. Tanenbaum at Vrija University since 1981.

Ameoba Designed by: Prof Andrew S. Tanenbaum at Vrija University since 1981.
Copyright 2004 Monash University IMS5401 Web-based Systems Development Topic 2: Elements of the Web (g) Interactivity.

Copyright 2004 Monash University IMS5401 Web-based Systems Development Topic 2: Elements of the Web (g) Interactivity.
DIRECT MEMORY ACCESS CS 147 Thursday July 5,2001 SEEMA RAI.

DIRECT MEMORY ACCESS CS 147 Thursday July 5,2001 SEEMA RAI.
File System Implementation

File System Implementation
CS490T Advanced Tablet Platform Applications Network Programming Evolution.

CS490T Advanced Tablet Platform Applications Network Programming Evolution.
I/O Channels I/O devices getting more sophisticated e.g. 3D graphics cards CPU instructs I/O controller to do transfer I/O controller does entire transfer.

I/O Channels I/O devices getting more sophisticated e.g. 3D graphics cards CPU instructs I/O controller to do transfer I/O controller does entire transfer.
Master’s course Bioinformatics Data Analysis and Tools Lecture 6: Internet Basics Centre for Integrative Bioinformatics.

Master’s course Bioinformatics Data Analysis and Tools Lecture 6: Internet Basics Centre for Integrative Bioinformatics.
Design and Implementation of the Log-based File I/O Library for Sandboxing CSE 542 Operating Systems Fall 2005 Qi Liao and Xuwen Yu.

Design and Implementation of the Log-based File I/O Library for Sandboxing CSE 542 Operating Systems Fall 2005 Qi Liao and Xuwen Yu.
Systems Architecture, Fourth Edition1 Internet and Distributed Application Services Chapter 13.

Systems Architecture, Fourth Edition1 Internet and Distributed Application Services Chapter 13.

Similar presentations

Ads by Google