1. Brad McCusker Transition Engineering and Consulting HP
UNIX Portability
Brad McCusker
Transition Engineering and Consulting HP

2. Agenda What is UNIX Portability (UP)? UP features in V8.3
Proposed future UP features
UP success story
September 18, 2018September 18, 2018

3. What is UNIX Portability?

4. What is UNIX Portability?
An initiative to ease the porting of applications from UNIX and Linux to OpenVMS
New UNIX-like APIs and utilities
Integrated operating system support for UNIX features
Heavily utilized by HP and partners to develop layered products for OpenVMS
Used by HP to improve Java performance
September 18, 2018September 18, 2018

5. Goals of UNIX Portability
Provide a full set of UNIX interfaces and tools within OpenVMS
In native, integrated fashion
No layered emulator (e.g. old “POSIX for OpenVMS” product)
No performance issues
No interoperability issues
Make the cost of porting to OpenVMS equal or comparable to porting from one “flavor” of UNIX to another (HP-UX, Solaris, AIX, Linux)
Retain native OpenVMS behavior as the default
September 18, 2018September 18, 2018

6. Various terms – all mean the same…
UNIX Portability is the original term used to describe this effort
The effort quickly evolved to include:
Linux Interoperability
Linux Portability
Open Source Interoperability
Open Source Portability
All these are part of UNIX Portability
September 18, 2018September 18, 2018

7. Components of UNIX Portability
C Run-Time Library (C RTL)
GNV
Other components
September 18, 2018September 18, 2018

8. C Run-Time Library (C RTL)
Library of APIs
Implementations of routines typically found on UNIX
Examples: printf(), open()
Open Group Base Specification system interfaces
Curses screen management functions
Socket routines (not documented in the C RTL Reference Manual)
Delivered in shareable image DECC$SHR.EXE (and DECC$SHR_EV56.EXE)
September 18, 2018September 18, 2018

9. GNV GNV = “GNU’s Not (Open)VMS” UNIX-like shell (“bash”)
Utilities and tools
Examples: cp, ls, vi
Open Group Base Specification shells and utilities
OpenSource software
September 18, 2018September 18, 2018

10. GNV – Current Status GNV 2.1 Shipped with VMS 8.3
Reportedly has problem with some configure scripts
Reportedly has installation problems
Reprotedly has removal problems
GNV 1.6 – found at
September 18, 2018September 18, 2018

11. Other components Java Perl POSIX threads (pthreads) Ant Python
Pkg-config
Base operating system and RMS changes
September 18, 2018September 18, 2018

12. UP features in V8.3
September 18, 2018September 18, 2018

13. UP features in V8.3 C RTL features GNV utilities Other features
September 18, 2018September 18, 2018

14. V8.3 C RTL Features Symbolic links and POSIX pathname processing
Byte-range locking
Encryption routines
Other changes
September 18, 2018September 18, 2018

15. What is a symbolic link?
A symbolic link is a directory entry that associates a name with a text string
The text string is interpreted as a POSIX pathname when accessed by certain services
It is implemented on OpenVMS as a file of organization SPECIAL and type SYMBOLIC_LINK
Symbolic links are also known as “Symlinks”
September 18, 2018September 18, 2018

16. Example of symbolic link creation
New DCL qualifier /SYMLINK for CREATE
Example:
$ sho def
SYS$SYSDEVICE:[MCCUSKER.WORK]
$ create/symlink="a/b.txt" link_to_b.txt
$ dir/date link_to_b.txt
Directory SYS$SYSDEVICE:[MCCUSKER.WORK]
LINK_TO_B.TXT;1 -> a/b.txt
6-APR :13:29.83
Total of 1 file. $
September 18, 2018September 18, 2018

17. Example of symbolic link access
Assume file being referenced does not exist:
$ type LINK_TO_B.TXT
%TYPE-W-OPENIN, error opening SYS$SYSDEVICE:[MCCUSKER.WORK]LINK_TO_B.TXT;1 as input
-RMS-E-ACC, ACP file access failed
-SYSTEM-F-FILNOTACC, file not accessed on channel $
$ create/dir [.A]
$ TYPE LINK_TO_B.TXT
September 18, 2018September 18, 2018

18. Example of symbolic link access (continued)
Create the missing file through the link:
$type LINK_TO_B.TXT
%TYPE-W-OPENIN, error opening SYS$SYSDEVICE:[MCCUSKER.WORK]LINK_TO_B.TXT;1 as input
-RMS-E-ACC, ACP file access failed
-SYSTEM-F-FILNOTACC, file not accessed on channel $
$ create link_to_b.txt
This text is in file [.a]b.txt
Exit
September 18, 2018September 18, 2018

19. Example of symbolic link access (continued)
Now we can type the file through the link:
In this example, RMS noticed the input file was a symbolic link, read its contents and interpreted those contents as a POSIX pathname
$ TYPE LINK_TO_B.TXT
This text is in file [.a]b.txt
$ DIR [.a]
Directory SYS$SYSDEVICE:[MCCUSKER.WORK.A]
b.txt;1
Total of 1 file.
$ type [.a]b.txt $
September 18, 2018September 18, 2018

20. RMS support for symbolic links
sys$open() operates on the target file pointed to by the symbolic link
sys$create() creates the file pointed to by the symbolic link
sys$search() returns the DVI and FID of the target file; DID is zero; resultant name is that of the symbolic link and not the target file
Setting flag NAML$V_OPEN_SPECIAL cause sys$open() and sys$search() to not follow the symbolic link
September 18, 2018September 18, 2018

21. C RTL support for symbolic links
Six newly documented APIs:
symlink() -- create a symbolic link
readlink() -- read the contents of a symbolic link
unlink() -- delete a symbolic link
realpath() -- return a direct pathname from the root
lchown() -- change the owner of a symbolic link
lstat() – return attributes of a symbolic link
Other APIs that accept pathnames recognize symbolic links
September 18, 2018September 18, 2018

22. POSIX pathname processing
To provide a consistent programming environment, developers must be able to use POSIX pathnames through OpenVMS interfaces such as the C RTL and system services
Other standard POSIX features (system-wide root, mount points, current working directory, version limits) must also be provided
Rules must be devised to deal with the differences between POSIX pathnames and OpenVMS file names
September 18, 2018September 18, 2018

23. POSIX pathnames for RMS and DCL
Issue: The POSIX name-separator ‘/’ character has a different meaning to DCL (as a qualifier indicator)
Quoting a pathname allows us to pass the pathname through DCL (since quoted strings are already allowed in DCL for DECnet)
Adding a prefix to the pathname allows RMS to recognize the string as a POSIX pathname
Format: “^UP^pathname”
Example: a/b.txt becomes “^UP^a/b.txt”
September 18, 2018September 18, 2018

24. DCL POSIX pathname example
$ dir [.a]
Directory SYS$SYSDEVICE:[MCCUSKER.WORK.A]
B.C;2
Total of 1 file.
$ cc "^UP^a/b.c"/obj="^UP^a/b.obj“ $
$ link/exe="^UP^a/b.exe" "^UP^a/b.obj“
B.C; b.exe; b.obj;1
Total of 3 files.
September 18, 2018September 18, 2018

25. System-wide root Available for use with POSIX pathnames
New ROOT keyword for SET
Example:
$ set root sys$sysdevice:[000000]
%SET-I-PSXROOSET, system POSIX root set to SYS$SYSDEVICE:[000000]
$ sho root
DISK$ALPHASYS:[000000] $
$ ty "^UP^/mccusker/work/link_to_b.txt"
This text is in file [.a]b.txt
$ ty sys$sysdevice:[mccusker.work]link_to_b.txt
September 18, 2018September 18, 2018

26. Mount points Allows the crossing of volumes from the root
New mnt and umnt utilities
Example:
$ dir dkb100:[newtest]
NEWDIR.DIR;1
Total of 1 file.
$ mnt dkb100:[newtest] /a/mnt
$ dir DKB0:[TEST.A.MNT]
Total of 1 file $
September 18, 2018September 18, 2018

27. Current working directory
Similar to OpenVMS default directory
May not be a search list
Must exist
Example:
$ SET DEFAULT “^UP^/a/mnt”
$ SHOW DEFAULT
DKB100:[NEWDIR] $
September 18, 2018September 18, 2018

28. C RTL and GNV
The C RTL and GNV will accept pure POSIX pathnames (no need for the ^UP^ quoted format)
This is an alternative to the standard C RTL UNIX pathname features
Post V8.3, file versions will be treated as on UNIX
only one version of a file will be deleted on a delete operation
no new version of a file will be created on a create operation
September 18, 2018September 18, 2018

29. C RTL and GNV (continued)
DECC$POSIX_COMPLIANT_PATHNAMES controls how input pathnames are interpreted
1 = UNIX-only
2 = leans UNIX (unless pathname contains brackets or ends with a colon and passes sys$filescan())
3 = leans OpenVMS (unless pathname contains a slash)
4 = OpenVMS only
Modes 1 and 4 are not recommended due to interactions with other shareable libraries and utilities
September 18, 2018September 18, 2018

30. File naming POSIX allows filenames “a” and “a.” in the same directory
A file created through GNV and the C RTL that does not have a “.” or that ends in a “.” will have an additional “.” appended to its name to ensure uniqueness
To allow POSIX filename “a.DIR” and directory “a” to co-exist in the same directory, GNV and the C RTL will append a “.” to a filename ending in “.DIR”
September 18, 2018September 18, 2018

31. Byte-range locking Implemented in the C RTL
fcntl() API
F_GETLK, F_SETLK, F_SETLKW options
Can be used against any file type
Accessed via a file descriptor
Functional across processes and across a cluster
Locks are advisory; to be effective, processes must agree to cooperate
4 GB limit
New privileged image: DECC$SHRP
September 18, 2018September 18, 2018

32. Encryption routines crypt() setkey() encrypt()
Encrypts an input string
Note: algorithm has nothing to do with OpenVMS password encryption
setkey()
Sets an encoding key to be used with encrypt()
encrypt()
Encrypts an array in-place using key generated by setkey()
September 18, 2018September 18, 2018

33. Other C RTL changes fchmod() confstr()
New function that changes the mode of a file that is specified with a file descriptor
confstr()
Existing function that returns system configuration information
New symbols supported include:
_CS_MACHINE_IDENT
_CS_PARTITION_IDENT
_CS_MACHINE_SERIAL
September 18, 2018September 18, 2018

34. GNV utilities Better handling of pipes and subprocesses in bash
Behavior of cc is now more UNIX-like, particularly for certain config scripts
Improved script processing
make files run much better
September 18, 2018September 18, 2018

35. Other features NFS improvements
Perl has been updated and made POSIX-aware (Note: this kit is not in general release yet)
terminate() can now be called from any thread in a C++ program (previously, terminate() was only callable from the default thread)
September 18, 2018September 18, 2018

36. Proposed Future UP Features
September 18, 2018September 18, 2018

37. Shared-stream I/O Shared-write access to stream files
Standard UNIX I/O default
Applicable across the nodes of a cluster
Can be used on any stream file type
Accessible via the C RTL
Set per file by an option on file open APIs
Set per process by C RTL feature switch
September 18, 2018September 18, 2018

38. Shared memory Implemented via the C RTL POSIX shared memory
shm_open(), shm_unlink()
System V shared memory
shmat(), shmctl(), shmdt(), shmget()
Possibly ftok()
Not applicable across nodes of a cluster
September 18, 2018September 18, 2018

39. ioctl() enhancements Support more than socket operations
For terminal I/O, allow read termination when a given number of characters is entered (TCGETA, TCSETA)
For UDP sockets, I_SETSIG (signal on data in)
For ethernet, SIOCGIFMTU (check MTU size)
September 18, 2018September 18, 2018

40. UNIX Domain Sockets Sockets within a node Uses socket APIs
Doesn’t use the IP stack
Alternative to pipes
Bi-directional
September 18, 2018September 18, 2018

41. Non-socket poll()/select()
poll/select currently only work for TCP/IP sockets
Extend to work with pipes, devices, etc
September 18, 2018September 18, 2018

42. Child process survival
Ability of a child process to survive termination of its parent
Console shareable between a parent and its detached child
Job logical name table and mounted volume list shareable and would persist after parent process termination
Used by nohup in bash, fork(), cron
September 18, 2018September 18, 2018

43. Semaphores Implemented via the C RTL POSIX semaphores
sem_open(), sem_post(), sem_wait(), etc.
System V semaphores
semop(), semctl(), semget()
Thread-aware
September 18, 2018September 18, 2018

44. Upgrade BASH Current BASH in GNV is based on v1.14.8
Upgrade to current version of BASH
V3.* is current
Gets latest functionality
Easier to keep up with bug fixes
September 18, 2018September 18, 2018

45. UNIX Portability Roadmap
2006
2007
2008
2009
2010
OpenVMS V8.2 Symbolic Links SDK
CRTL APIs
DCL support
NFS support (in TCP/IP ECO)
OpenVMS V8.x
Ongoing enhancements
OpenVMS V8.3
Symbolic Links
Byte range locking
POSIX Pathname SDK
Support for OpenVMS and 8.3
GNV update
OpenVMS V8.4
POSIX style pathname support
Semaphores
Shared write for stream files
ioctl()
GNV update
Shared memory APIs
September 18, 2018September 18, 2018

46. References OpenVMS web site: UNIX Portability CRTL Reference Manual
UNIX Portability
CRTL Reference Manual
GNV Page – OpenVMS
GNV Sourceforge
September 18, 2018September 18, 2018

47. Contact info
Paul Cerqua (UNIX Portability Leader)
Gaitan D’Antoni (OpenVMS Technical Director)
Leo Demers (UP Program Manager)
September 18, 2018September 18, 2018

48. Presentation Title