Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hidden Gems: UNIX Shared Libraries Way better than PROBUILD.

Similar presentations


Presentation on theme: "Hidden Gems: UNIX Shared Libraries Way better than PROBUILD."— Presentation transcript:

1 Hidden Gems: UNIX Shared Libraries Way better than PROBUILD.

2 A Few Words About The Speaker Tom Bascom, Roaming DBA & Progress User since 1987 President, DBAppraise, LLC – Remote Database Management Service. – Simplifying the job of Managing and Monitoring The World’s Best Business Applications. – VP, White Star Software, LLC – Expert Consulting Services related to all aspects of Progress and OpenEdge. –

3 What is a UNIX Shared Library? A collection of executable code and data which is loaded into memory separately from the launch of the main program (_progres). Progress 4GL has supported Windows DLLs since (at least) version 8. UNIX DLL support arrived with Progress version 9. I’ll be using Linux as an example but the principles apply to all UNIX platforms. More Info:

4 Why Would You Use One? To share code from 3 rd party, non-Progress, sources. To leverage proprietary internal libraries. To gain performance improvements from routines written in faster languages. To access low-level OS features that are not otherwise available.

5 /lib (and /lib64) /lib> ls *.so* ld-2.4.so libcrypt-2.4.so libnss_dns-2.4.so libresmgr.so.1 ld-linux.so.2 libcrypt.so.1 libnss_dns.so.2 libresmgr.so ld-lsb.so.2 libc.so.6 libnss_files-2.4.so libresolv-2.4.so ld-lsb.so.3 libdevmapper.so libnss_files.so.2 libresolv.so.2 libacl.so.1 libdevmapper.so.1.02 libnss_hesiod-2.4.so librt-2.4.so libacl.so libdl-2.4.so libnss_hesiod.so.2 librt.so.1 libanl-2.4.so libdl.so.2 libnss_nis-2.4.so libSegFault.so libanl.so.1 libe2p.so.2 libnss_nisplus-2.4.so libss.so.2 libattr.so.1 libe2p.so.2.3 libnss_nisplus.so.2 libss.so.2.0 libattr.so libext2fs.so.2 libnss_nis.so.2 libsysfs.so.1 libaudit.so.0 libext2fs.so.2.4 libnss_winbind.so.2 libsysfs.so libaudit.so libgcc_s.so.1 libnss_wins.so.2 libthread_db-1.0.so libblkid.so.1 libhistory.so.5 libpamc.so.0 libthread_db.so.1 libblkid.so.1.0 libhistory.so.5.1 libpamc.so libutil-2.4.so libBrokenLocale-2.4.so libm-2.4.so libpam_misc.so.0 libutil.so.1 libBrokenLocale.so.1 libmemusage.so libpam_misc.so libuuid.so.1 libbz2.so.1 libm.so.6 libpam.so.0 libuuid.so.1.2 libbz2.so libncurses.so.5 libpam.so libwrap.so.0 libc-2.4.so libncurses.so.5.5 libpcprofile.so libwrap.so libcap.so.1 libnscd.so.1 libpthread-2.4.so libxcrypt.so.1 libcap.so.1.92 libnscd.so libpthread.so.0 libxcrypt.so libcidn-2.4.so libnsl-2.4.so libreadline.so.5 libz.so.1 libcidn.so.1 libnsl.so.1 libreadline.so.5.1 libz.so libcom_err.so.2 libnss_compat-2.4.so libresmgr.so libcom_err.so.2.1 libnss_compat.so.2 libresmgr.so.0.9.8

6 Can You Narrow That Down? For backwards compatibility there are several versions of many libraries. Many libraries are “niche” libraries and not generally interesting. The most interesting (to me) are: – libc, the standard “C” library – libm, the math library – libz, the compression library – libcrypt, the encryption library

7 What Functions are Inside? $ nm -DC /lib/libz.so | grep -v " _" | grep " T " b0 T adler a70 T adler32_combine c10 T compress b40 T compress b20 T compressBound c70 T crc T crc32_combine T deflate b0 T deflateBound f70 T deflateCopy …

8 A Few Words of Warning The slides that you are about to see contain very little in the way of: – Error checking. – Exception handling. – Garbage collection. – And other niceties. While this may be customary for sample code it is very dangerous when working with real C code. You have been warned.

9 Calling Functions in libc

10 Problem: No OS-PUTENV() NAME putenv - change or add an environment variable SYNOPSIS #include int putenv(char *string); DESCRIPTION The putenv() function adds or changes the value of environment variables. The argument string is of the form name=value. If name does not already exist in the environment, then string is added to the environment. If name does exist, then the value of name in the environment is changed to value. The string pointed to by string becomes part of the environment, so altering the string changes the environment. RETURN VALUE The putenv() function returns zero on success, or non-zero if an error occurs.

11 Calling putenv() define variable x as integer no-undo. procedure putenv external "/lib64/libc.so.6": define input parameter env as character. define return parameter x as long. end. display os-getenv( "XYZZY" ). pause. run putenv( "XYZZY=pflugh", output x ). display os-getenv( "XYZZY" ). os-command value( 'echo "$XYZZY"' ). return.

12 Calling putenv() ┌────────────────────┐ │? │ └────────────────────┘ Press space bar to continue. ┌────────────────────┐ │? │ └────────────────────┘ Press space bar to continue. ┌────────────────────┐ │? pflugh │ └────────────────────┘ Press space bar to continue. ┌────────────────────┐ │? pflugh │ └────────────────────┘ Press space bar to continue. pflugh Press space bar to continue. pflugh Press space bar to continue.

13 Problem: No Scientific Notation NAME sprintf - formatted string output SYNOPSIS int sprintf( char *str, const char *format,... ); THE CONVERSION SPECIFIER A character that specifies the type of conversion to be applied. The conversion specifiers and their meanings are: … e,E The double argument is rounded and converted in the style [-]d.ddde±dd where there is one digit before the decimal-point character and the number of digits after it is equal to the precision; if the precision is missing, it is taken as 6; if the precision is zero, no decimal-point character appears. An E conversion uses the letter E (rather than e) to introduce the exponent. The exponent always contains at least two digits; if the value is zero, the exponent is 00.

14 Calling sprintf() define variable x as integer no-undo. define variable c as memptr no-undo. define variable m as memptr no-undo. procedure sprintf external "/lib64/libc.so.6": define input-output parameter fStr as memptr. define input parameter mask as memptr. define input parameter arg as double. define return parameter x as long. end. set-size( c ) = set-size( m ) = put-string( m, 1 ) = "%1.4e". run sprintf( input-output c, m, , output x ). display get-string( c, 1 ) format “x(20)”. return.

15 Calling sprintf() ┌────────────────────┐ │1.2300e-02 │ └────────────────────┘ Procedure complete. Press space bar to continue. ┌────────────────────┐ │1.2300e-02 │ └────────────────────┘ Procedure complete. Press space bar to continue.

16 Data Type Mapping “C” Data TypeProcedure Parameter Definition charBYTE shortSHORT unsigned shortUNSIGNED SHORT longLONG intLONG floatFLOAT doubleDOUBLE char* (read-only)CHARACTER or MEMPTR char* (writeable)MEMPTR pointer to …HANDLE TO …

17 Creating Your Own Shared Library

18 Overview You’ll need some “C” source code. Compile to object code. Run the Linker and create a Shared Library. Test with Progress.

19 Problem: Sub-second PAUSE define variable i as integer no-undo. define variable s as integer no-undo. s = time. do i = 1 to 1000: display i. etime( yes ). do while etime < 500: end. end. display string( time - s, “hh:mm:ss" ). return.

20 What’s Wrong With That? 8 minutes and 20 seconds of CPU time! $ ps -ef | grep [_]progres tom :16 pts/0 00:02:30 /usr/pro102a/bin/_progres -1 $ pro –p shortsleep.p ┌──────────────┐ │ i │ │────── ───── │ │ 1,000 08:20 │ └──────────────┘ $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1 $ ps -ef | grep [_]progres tom :16 pts/0 00:02:30 /usr/pro102a/bin/_progres -1 $ pro –p shortsleep.p ┌──────────────┐ │ i │ │────── ───── │ │ 1,000 08:20 │ └──────────────┘ $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1

21 Source Code #include long shortsleep( unsigned long milliseconds ) { struct timespec tmReq; tmReq.tv_sec = (time_t)(milliseconds / 1000) ; tmReq.tv_nsec = (time_t)(milliseconds % 1000) * 1000 * 1000 ; nanosleep( &tmReq, NULL ) ; }

22 Compile to Object Code $ cc –c shortsleep.c -fPIC $ ls shortsleep.c shortleep.o $ file shortleep.o ELF 64-bit LSB relocatable, AMD x86-64, version 1 (SYSV), not stripped

23 Link to Create a Shared Lib $ ld -shared shortsleep.o -o shortsleep.so $ ls shortsleep* shortsleep.c shortsleep.o shortsleep.so $ file shortsleep.so shortsleep.so: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped $ nm -DC shortsleep.so shortsleep.so: A __bss_start shortsleep.so: A _edata shortsleep.so: A _end shortsleep.so: U nanosleep shortsleep.so: T shortsleep

24 Progress 4GL Test Harness define variable i as integer no-undo. define variable x as integer no-undo. define variable s as integer no-undo. procedure shortsleep external "./shortsleep.so": define input parameter ms as long. define return parameter x as long. end. s = time. do i = 1 to 1000: display i. run shortsleep( 500, output x ). end. display string( time - s, “hh:mm:ss" ). return.

25 Test With Progress $ pro –p shortsleep.p Could not open Dynamic Library:./shortsleep.so (8013) DLL Error :./shortsleep.so: wrong ELF class: ELFCLASS32 (8014) Press space bar to continue. $ file $DLC/bin/_progres _progres: setuid ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), for GNU/Linux 2.4.0, not stripped $ pro –p shortsleep.p Could not open Dynamic Library:./shortsleep.so (8013) DLL Error :./shortsleep.so: wrong ELF class: ELFCLASS32 (8014) Press space bar to continue. $ file $DLC/bin/_progres _progres: setuid ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), for GNU/Linux 2.4.0, not stripped

26 Test Again! $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1 $ pro –p shortsleep.p ┌──────────────┐ │ i │ │────── ───── │ │ 1,000 08:20 │ └──────────────┘ $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1 $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1 $ pro –p shortsleep.p ┌──────────────┐ │ i │ │────── ───── │ │ 1,000 08:20 │ └──────────────┘ $ ps -ef | grep [_]progres tom :16 pts/0 00:10:50 /usr/pro102a/bin/_progres -1 No measurable CPU time!

27 Problem: Regular Expressions NAME regcomp, regexec, regerror, regfree - POSIX regex functions SYNOPSIS #include int regcomp( regex_t *preg, const char *regex, int cflags ); int regexec( const regex_t *preg, const char *string, size_t nmatch, regmatch_t pmatch[], int eflags ); size_t regerror( int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size ); void regfree( regex_t *preg );

28 Source Code #include long regex_match( char *pattern, char *string ) { regex_t xpattern; size_t matchDepth = 2; regmatch_t pmatch[2]; int offset; if ( regcomp(&xpattern, pattern, REG_EXTENDED|REG_ICASE ) != 0 ) { offset = -2; } else { if ( regexec(&xpattern, string, matchDepth, pmatch, 0 ) != 0 ) { offset = -1; } else { offset = pmatch[1].rm_so; } } regfree( &xpattern ); return offset; }

29 Compile and Link… $ cc –c rx2.c -fPIC $ ld –shared rx2.o –o rx2.so $ file rx2.so rx2.so: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped $ nm -DC rx2.so e8 A __bss_start e8 A _edata e8 A _end U regcomp U regexec T regex_match U regfree

30 Progress 4GL Test Harness define variable pattern as character no-undo format "x(60)". define variable string as character no-undo view-as editor inner-lines 10 inner-chars 60. define variable x as integer no-undo. procedure regex_match external "./rx2.so": define input parameter pattern as character. define input parameter string as character. define return parameter x as long. end. update pattern skip string skip with side-labels. run regex_match( pattern, string, output x ). display x. return.

31 Test! $ pro –p rx.p ┌──────────────────────────────────────────────────────────┐ │pattern:.*(Fox).* │ │string: The quick brown fox jumped over the lazy dog! │ │ │x: 16 │ └──────────────────────────────────────────────────────────┘ $ pro –p rx.p ┌──────────────────────────────────────────────────────────┐ │pattern:.*(Fox).* │ │string: The quick brown fox jumped over the lazy dog! │ │ │x: 16 │ └──────────────────────────────────────────────────────────┘

32 Shared Library Challenges

33 Which Library to Use? Use a variable? NOT!  Use a pre-processor. Use symbolic links. Wait for 10.2B. procedure putenv external "/lib64/libc.so.6": define input parameter env as character. define return parameter x as long. end. procedure putenv external "/lib64/libc.so.6": define input parameter env as character. define return parameter x as long. end.

34 Variable Argument Lists Isolate different cases in multiple.p files. Use wrapper libraries. Wait for 10.2B. int sprintf( char *str, const char *format,... );

35 Call-Backs? Not supported . void qsort( void *base, size_t nmemb, size_t size, int(*compar)(const void *, const void *) ); void qsort( void *base, size_t nmemb, size_t size, int(*compar)(const void *, const void *) );

36 Persistence About 25% faster in crude testing. RELEASE EXTERNAL “/lib64/libc.so.6”. procedure putenv external "/lib64/libc.so.6" PERSISTENT: define input parameter env as character. define return parameter x as long. end. procedure putenv external "/lib64/libc.so.6" PERSISTENT: define input parameter env as character. define return parameter x as long. end.

37 MEMPTRs define variable x as integer no-undo. define variable c as memptr no-undo. define variable m as memptr no-undo. procedure sprintf external "/lib64/libc.so.6": define input-output parameter fStr as memptr. define input parameter mask as memptr. define input parameter arg as double. define return parameter x as long. end. set-size( c ) = set-size( m ) = put-string( m, 1 ) = "%1.4e". run sprintf( input-output c, m, , output x ). display get-string( c, 1 ) format “x(20)”. set-size( c ) = 0. set-size( m ) = 0. return.

38 Dynamic Shared Library Call (Special Bonus Slide!)

39 Dynamic Shared Lib Call define variable LIBC as character no-undo initial "/lib64/libc.so.6". function os-putenv returns integer ( input env as character ): define variable hCall as handle no-undo. define variable rVal as integer no-undo. create call hCall. assign hCall:call-name = "putenv" hCall:library = LIBC hCall:call-type = DLL-CALL-TYPE hCall:num-parameters = 1 hCall:return-value-dll-type = "LONG". hCall:set-parameter( 1, "CHARACTER", "INPUT", env ). hCall:invoke(). rVal = hCall:return-value. delete object hCall. return rVal. end.

40 Conclusion Share code from 3 rd party, non-Progress, sources. Gain performance improvements from routines written in faster languages. Access low-level OS features that are not otherwise available. It beats the heck out of using PROBUILD.

41 Questions?

42 Thank-you!

43

44 dlopen() and Friends NAME dladdr, dlclose, dlerror, dlopen, dlsym, dlvsym - programming interface to dynamic linking loader SYNOPSIS #include void *dlopen( const char *filename, int flag ); char *dlerror( void ); void *dlsym( void *handle, const char *symbol ); int dlclose( void *handle ); DESCRIPTION The four functions dlopen(), dlsym(), dlclose(), dlerror() implement the interface to the dynamic linking loader.


Download ppt "Hidden Gems: UNIX Shared Libraries Way better than PROBUILD."

Similar presentations


Ads by Google