1. Unix System Interface Programming Part 6.1 – System Interface Overview Prepared by Xu Zhenya( xzy@buaa.edu.cn )xzy@buaa.edu.cn Draft – Xu Zhenya( 2002/10/01 ) Rev1.0 – Xu Zhenya( 2002/10/10 )

2. Unix Programming Environment Dept. of CSE, BUAA Agenda  1. System Interface Programming Standards  2. how a “C” program is started?  3. System call and library functions  4. String functions  5. Error handling  6. Memory management  7. man & api  8. Debugging programs  9. Summary  Appendix:  Understanding O.S. Kernel  Recommanded Books

3. Unix Programming Environment Dept. of CSE, BUAA Standards (1)

4. Unix Programming Environment Dept. of CSE, BUAA Standards (2)  X/Open Common Applications Environment (CAE) Portability Guide Issue 3 (XPG3) and Issue 4 (XPG4)  SUS( Single UNIX Specification ), SUSv2  XNS4: Networking Services Issue4  ILP32 and LP64 programming environments  Notes:  1. The developers of SVID3( UNIX Systems Laboratories) are no longer in business, and this specification defers to POSIX and X/Open CAE.  2. Utilities: conflictions with historical Solaris utility  Included /usr/xpg4/bin into PATH, and before other utilities.

5. Unix Programming Environment Dept. of CSE, BUAA Standards (3)

6. Unix Programming Environment Dept. of CSE, BUAA Standards (4)  POSIX inlcues the following standards:  1003.1 : System Interface( POSIX.1 )  POSIX.1 has adopted virtually all ANSI C library calls.  However, POSIX.1 has not adopted include operations on wide characters and multi-byte characters( as used for Chinese ).  1003.1b : Real-time extensions  1003.1c: User-level threads( pthreads, POSIX threads library )  1003.1g: Networking standards  1003.2: Shells and utilities

7. Unix Programming Environment Dept. of CSE, BUAA Standards (5)  Feature test macros  __EXTENSIONS__: the application with access to all interfaces and headers not in conflict with the specified standard. SpecificationCompiler/FlagsFeature Test Macros ANSI/ISO Cc89none SVID3cc -Xtnone POSIX.1-1990c89_POSIX_SOURCE POSIX.1-1990 and POSIX.2-1992 C-Language Bindings Option c89 _POSIX_SOURCE and POSIX_C_SOURCE=2 POSIX.1b-1993c89_POSIX_C_SOURCE=199309L POSIX.1c-1996c89_POSIX_C_SOURCE=199506L CAE XPG4 c89 _XOPEN_SOURCE and _XOPEN_VERSION=4 SUSv2(includes XNS5)c89_XOPEN_SOURCE=500

8. Unix Programming Environment Dept. of CSE, BUAA Standards (6) #if ( __STDC__ == 0 && !defined(_POSIX_C_SOURCE) && \ !defined(_XOPEN_SOURCE)) || \ (defined(_XOPEN_SOURCE) && _XOPEN_VERSION - 0 >= 4) || \ defined(__EXTENSIONS__)  Case Study  Solaris: /usr/include/sys/feature_test.h  Linux: /usr/include/features.h  Compiler options  - see C/C++ Manual: for Gnu C/C++, see the links page.

9. Unix Programming Environment Dept. of CSE, BUAA How a C program is started? (1)

10. Unix Programming Environment Dept. of CSE, BUAA How a C program is started? (2) AddressKbytesResidentSharedPrivatePermissionsMapped File 0804600088-8read/write/exec[ stack ] 0805000044-4read/execa.out 0806000044-4read/write/execa.out 0806100088-8read/write/exec[ heap ] DFB0000044-4read/write/exec[ anon ] DFB1000052852046456read/execlibc.so.1 DFBA400024 - read/write/execlibc.so.1 DFBAA00088-8read/write/execlibc.so.1 DFBB000044-4read/execlibdl.so.1 DFBC0000116 1124read/execld.so.1 DFBED00088-8read/write/execld.so.1 DFBEF00044-4read/write/execld.so.1

11. Unix Programming Environment Dept. of CSE, BUAA How a C program is started? (3) LINK EDITOR MEMORY MAP (ELF): ld –m –o who who.o Output sectionVirtual addresssize.interp80500d411.hash80500e890.dynsym8050178110.dynstr8050288bb.SUNW_ve805034420.rel.bss805036410.rel.plt805037420.plt805039450.text80503e8d5.rodata180504c043.got80605041c.dynamic8060520a8.bss80605c83c8

12. Unix Programming Environment Dept. of CSE, BUAA How a C program is started? (4)   Command line arguments & environment variables   => Limits on the length of arguments and environment   sysconf( _SC_ARG_MAX )   exec(): copy the arguments list and environmental variables after setting the user’s stack ( crt_init() )   getenv, setenv() & putenv()   Environmental table   extern char ** environ;   int main( int argc, char *argv[], char *env[] );

13. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 1 )

14. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 2 )  1. malloc & sbrk  2. time & date: time()  3. We can conclude that:  A few of syscalls (256 or less)  Apps. -> syscall / libc(->syscall)  Making a syscall  When we make a system call parameter that is a pointer to a data object, we must allocate space for the object and pass its address in the call. Example: time( time_t * p_time )

15. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 3 )  Make a library call  For a return value or parameter declared as a pointer type, there are three possibilities:  Don’t Allocate space:  The same as the system calls  Allocate space statically:  Copy the object pointed by the address: ctime(3C)  Allocate space dynamically:  MUST remember to free the space: strdup()  Read the man pages carefully.

16. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 4 )   12 time( tptr );   0x08050820: main : pushl %ebp   0x08050821: main+0x0001: movl %esp,%ebp   0x08050823: main+0x0003: subl $8,%esp /* time_t *tptr */   0x08050826: main+0x0006: movl -8(%ebp),%eax   0x08050829: main+0x0009: pushl %eax   0x0805082a: main+0x000a: call time [PLT]   0x0805082f: main+0x000f: addl $4,%esp   13 printf( "Machine time in sec = %d\n", *tptr );   0x08050832: main+0x0012: movl -8(%ebp),%eax   0x08050835: main+0x0015: movl (%eax),%eax   0x08050837: main+0x0017: pushl %eax   0x08050838: main+0x0018: pushl $0x8050900   0x0805083d: main+0x001d: call printf [PLT]   0x08050842: main+0x0022: addl $8,%esp

17. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 5 ) 1. 1. static char cbuf[26]; 2. 2. char * 3. 3. ctime( t ) 4. 4. const time_t*t; 5. 5. { 6. 6. return ( asctime( localtime( t ) ) ); 7. 7. } 8. 8. char * 9. 9. asctime( t ) 10. 10. const struct tm *t; 11. 11. { 12. 12. register char *cp; 13. 13. 14. 14. /* …… */15. 16. 16. return( cbuf ); 17. 17. }   Ctime.c

18. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 6 ) struct sysent { char sy_narg; /* total number of arguments */ #ifdef _LP64 unsigned short sy_flags; /* various flags as defined below */ #else unsigned char sy_flags; /* various flags as defined below */ #endif int (*sy_call)(); /* argp, rvalp-style handler */ krwlock_t *sy_lock; /* lock for loadable system calls */ int64_t (*sy_callc)(); /* C-style call hander or wrapper */ }; #define NSYSCALL 256 /* number of system calls */ extern struct sysent sysent[]; #ifdef _SYSCALL32_IMPL extern struct sysent sysent32[]; #endif /* See /usr/include/sys/systm.h & /etc/name_to_sysnum */

19. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 7 )  fork –  __asm__(" movl $0x2, %eax // SYS_fork, see /etc/name_to_synum lcall $0x27, $0x0 ");  Solaris Kernel  1. trap into the kernel  2. enter the common trap handler  Save context : the pointer to CPU structure, the return address  Save sycall arguments into LWP_CB  if ( t_pre_sys ) the do the pre-action /* truss, micro-state */  Call sysent[ 2 ].sy_callc() – sys_fork(); return;  Check if here is a signal?  If ( t_post_sys) then post_action  Set the returned value or errno  Return;  Fast System call – gethrtime(), gethrvtime() & gettimeofday()  Performance: using the registers

20. Unix Programming Environment Dept. of CSE, BUAA System calls & library calls ( 8 ) System Call Library Call Section 2 of the man pages Section 3 of the man pages Never allocates space for parameters 3 possibilities Executes in the system mode ( the kernel mode ) Executes in the user mode When a failure occurs:  Return -1;  Set errno( perror() ) When a failure occures:  Often returns NULL ( See man)  Can set errno ( See man ) Signal can interrupt a system call. ( restartable? ) Async-signal safe issues Misc: Tools used to trace system calls made by a process: 1.It executes the specified command and produces a trace of the system calls performed by theat command, the signals the command receives, and the machie faults the command incurs. 2.Solaris: truss 3.Linux: strace

21. Unix Programming Environment Dept. of CSE, BUAA String Functions  1. textbook, p129 – Table6-2  2. Parsing an input string:  strpbrk, strstr, strspn, strcspn, strtok, index, rindex  3. Memory functions:  memcpy, memmove, memccpy, memchr, memcmp, memset  Used to copy structures. But some compilers support structures assignement.  Don’t use the memcmp() to compare data structures: padding  4. String Conversion Functions:  strtol, strtoll, atol, atoll, atoi  5. Byte string functions:  bzero, bcopy, etc  6. Notes:  Buffer overflow => security ( size_t count )  For programs in C++, we should use its native string manipulation class & methods.

22. Unix Programming Environment Dept. of CSE, BUAA Dynamic Memory Allocation  1. Memory Allocation functions:  malloc, free, realloc, calloc, alloca  Heap & stack( automatic storage class )  => Memory leaks: CASE Tools  => Alignment: malloc() & compiler supporting  2. Resource management  Resource includes many things in our code: memory, file descriptors, locks  Resource management: allocate & free, lock & unlock, state management, etc  Open issues:  who should allocate and free them? Specially who free them?  => answer: do allocate and free resources at the same level  In multithreaded programs, very difficult!!!! => so be carefully.  But here C++-class can be used to the basic unit to manage resources.  In some situations, we have to implement our own memory pool for the central management: performance, debugging, etc.

23. Unix Programming Environment Dept. of CSE, BUAA Error Handling ( 1 )  1. Error handling functions:  perror, strerror  Notes:  The return value & “errno”  System calls & Library functions  When system calls fail they always (1) return -1; (2) set errno.  Library calls might set error when an error occurs. Check the man pages.  2. terminating a process:  exit(), abort()  Signals => terminate abnormally.

24. Unix Programming Environment Dept. of CSE, BUAA Error Handling ( 2 )  3. Discussion about error handling  1. we should handle errors at high-level, and check errors in the low- level => but how to return the error information?  In Unix, return -1, and set errno  If there are more information than one integer value?  In C++, pair in STL is one choice.  In multithreaded programs, TLS is one choice.  2. carefully using exception in C++:  Performance issues:  Control sequence, and our code are filled with try.  Memory may not be freed. ( in java, GC can do it. Maybe it’s why Java uses exceptions everywhere? )

25. Unix Programming Environment Dept. of CSE, BUAA Man Pages & API (1)  1. Xman, Answerbook2, etc  2. Discussion:  UML  Syntax:  Signature: internal & external representation  Type check  Semantics  Parameters:value/reference  Resource?

26. Unix Programming Environment Dept. of CSE, BUAA Man Pages & API (2) Corba-IDL: the famous example –ATM interface BankServer { ::xaction HandleTransaction( inout ::xaction Transaction ); long BankID(); }; interface Customers { exception CustomerException{ string s; }; // Get the data packet for a single customer any GetCustomers( in boolean metadata ); // Apply a delta packet to the customer table any ApplyCustomerUpdates( in any Delta, out long ErrorCount ); };

27. Unix Programming Environment Dept. of CSE, BUAA How to document our own code?(1)  documenting our assumptions, our approach, and our reasons for choosing the approach we did.  Donald Knuth once observed that we should be able to read a well- written program just as we read a well-written book.  "Self-Documenting Code," Chapter 19 by Steve McConnell  We also need to keep our comments coordinated with the code.  Each function or method needs a sentence or two that clarifies the following information:  What the routine does  What assumptions the routine makes  What each input parameter is expected to contain  What each output parameter is expected to contain on success and failure  Each possible return value

28. Unix Programming Environment Dept. of CSE, BUAA How to document our own code?(2)  Each part of the function that isn't completely obvious from the code needs a sentence or two that explains what it's doing.  Any interesting algorithm deserves a complete description.  Any nontrivial bugs we've fixed in the code need to be commented with the bug number and a description of what we fixed.  Well-placed trace statements, assertions, and good naming conventions can also serve as good comments and provide excellent context to the code.  Comment as if we were going to be the one maintaining the code in five years.

29. Unix Programming Environment Dept. of CSE, BUAA How to document our own code?(3)  Case Study int32_tint32_t terminal_mngr::configure ( sal_himkey * p_conf ) int32_t terminal_mngr::configure ( sal_himkey * p_conf ) sal_himkey int32_tsal_himkey Read the configuration information from "p_conf", create terminals & configure them, and add them into the terminal table. One example of the format of HiM configuration is defined in conf/term1234_conf.orig. terminal  the caller of this member method is "login"  executed in the main thread Parameters: p_conf : sal_himkey [in] the pointer to the configuration inforamtion. Returns: if successfully loading, then return 0, or a non-zero errval. Error Codes  ERRID_TERMINAL_MNGR_CONF_NR_TERM  invalid nr_terms in the configuration paratmers  ERRID_TERMINAL_MNGR_CONF_NOSUBKEY  no subkey for one terminal terminal  ERRID_TERMINAL_MNGR_CONF_INVALIDTYPE  invalid type for one terminal terminal  ERRID_TERMINAL_MNGR_CONF_CREATE_INSTANCE  fail to create the instance for a terminal terminal

30. Unix Programming Environment Dept. of CSE, BUAA Debugging programs (1)  1. Debuggers on Unix ( Linux here )  Backend: ( gdb, read the page “links” to get details )  Frontend: insight, DDD, kdbg  Notes:  Commercial development environments: workshop for Solaris  Other CASE Tools: runtime memory check, etc  2. Suggestions – Controlling the process  the debugger can answer all our debugging questions as long as we ask it the right questions.  having one or more hypothesis in mind—something you want to prove or disprove—before leveraging the debugger.

31. Unix Programming Environment Dept. of CSE, BUAA Debugging programs (2) 3. Debugging Process

32. Unix Programming Environment Dept. of CSE, BUAA Summary  In our programming practice, the following issues are the most important:  Interface  Error handling  Resource management: ( memory )  Document code, add many “assert” into our code  Control our debugging process  refractoring

33. Unix Programming Environment Dept. of CSE, BUAA Exercise: Part6-1  Read the homework page to get details.  Makefile, C/C++ Compilers, and the debugger(gdb)  Sample Output: [upe@linux exercise1]$./shell [upe@linux exercise1]$./shell myshell -> who am i myshell -> who am i [0] : who [0] : who [1] : am [1] : am [2] : i [2] : i myshell -> I am UPE myshell -> I am UPE [0] : I [0] : I [1] : am [1] : am [2] : UPE [2] : UPE myshell -> exit myshell -> exit [upe@linux exercise1]$ [upe@linux exercise1]$

34. Unix Programming Environment Dept. of CSE, BUAA  Appendix - A  Understanding O.S. Kernel – Process and its environment

35. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  1. a process is a VM implemented by the kernel to run a executable program.  2. define the VM – thinking about a computer  Instruction set: the user-level instructions & system calls  Memory: address space  I/O subsystem: file system  Interrupt: signal, asynchronous event  Process-to-process communication: IPC  Misc: protection mechanisms, debugging, etc  3. executable programs – produced by the linker, map file  Text, data segment( bbs, ro, etc ), stack, heap  Image after loaded into the memory by exec()

36. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  4. the first step: Process control block ( 1 )  Hardware context: registers, flags, etc ( user & kernel stack pointer !!! )  Address space: text, data, stack, heap  File system: hierarchical name management (current directory, umask, etc), opened file descriptors( object references )  Signal management: signal handlers, blocked & ignored, pending  Current terminal: input and output text  Identification: process ID  5. O.S. is a hardware resource manager for many VM. (2)  Scheduling class, priority management  State management/control: the famous state machine  6. relationship management ( 3 )  relationships among VMs: family( father and child ), process groups  VM and the owner( users )/credentials: user’s ID, user’s group ID

37. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  7. it’s very important to understand the runtime scenario:  How to enter the kernel? Interrupt, trap and exception  How to leave from the kernel? signal processing, scheduling  Synchronization:  among processes: lock( mutex ), sleep/wakeup  the interrupt handlers and the remainder: the interrupt level  7. now we can further to a multiprocessors (SMP) (4)  Threads & process: now two context (control block)  PCB should include threads management:  threads list:  CPU context, scheduling parameters and stack pointer (SP) should move to the thread’s context

38. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  8. Solaris’ thread and process model:  Process, the user-level thread, LWP and kernel thread  Why choose this model:  Flexibility and high-performance  Book, p7-10  Concurrency and parallel  ULT: not require kernel resource, and fast context switch, user-level scheduling  LWP: real parallelism  Questions:  Too complicated for tuning and programming  mapping the user-level thread(Pthread)’s scheduling attributes?  Scheduling activation & SIGWAITING  Need more consideration here!!!!  user-level scheduling  Computation-bound thread

39. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  Modeling the concepts: process, LWP and kthread  Kthreads own the scheduling parameters specific TS, RT, IA, SYS  Process control block  Program: vnode, arg & env  Lock: fine-grain lock for threads  Cred:  Address space: including “segments”- segdriver – page table  Pid, pgid:  P_child: family relationship  Singal: siganl-q, single hander vector, mask of ignored blocked  u-area: file descriptors  See: /usr/include/sys/proc.h, /usr/include/sys/user.h

40. Unix Programming Environment Dept. of CSE, BUAA Understanding “process” – its environment  LWP Control Block: /usr/include/sys/kwp.h  An LWP identifier  A signal mask that tells the kernel which signals will be accepted  Saved values of user-level registers (when the LWP is not running)  system call arguments, results, and error  Resource usage and profiling data  Pointer to the corresponding kernel thread  Kthread control block: /usr/include/sys/thread.h  Reference to its own scheduling parameters – specific to the class  A kernel stack  some pointer to CPU structure: affinity, bind,  Flags: t_schedflag, t_state, t_preempt  Linked pointer to other kthreads

41. Unix Programming Environment Dept. of CSE, BUAA  Appendix - B  Recommended books

42. Unix Programming Environment Dept. of CSE, BUAA Appendix - B  System Interfaces Programming  The practice of programming, Brian Kernighan  Advanced Programming in the UNIX Environment, Richard Stevens, Addison-Wesley, 1993  UNIX Networking Programming Volume 2: Inter-Process Communication, Richard Stevens, Addison-Wesley  UNIX System Programming, Keith Haviland, Dina Gray and Ben Salama, 2nd ed., Addison-Wesley, 1999.  Interprocess Communications in UNIX, John Gray, Prentice Hall, 1997.  Programming with UNIX Threads, Charles J. Northrup, John Wiley & Sons, Inc., 1997.  Practical UNIX Programming: A Guide to Concurrency, Communication, and Multithreading, Kay Robbins and Steven Robbins, Prentice Hall, 1996.  A Practical Guide to the UNIX System, Mark Sobell, 3rd ed., Benjamin Cummings, 1995.  UNIX for Programmers and Users, Graham Glass & King Ables, 2nd ed., Prentice Hall, 1999.  The following three books on the Unix Kernel are absolutely classic.  The Design of the UNIX operating System, Maurice Bach, Prentice Hall, 1986.  The Design and Implementation of the 4.4 BSD Operating System, Marshall McKusick, Keith Bostic, Michael Karels and John Quarterman, Addison-Wesley, 1996.  UNIX Internals: The New Frontiers, Uresh Vahalia, Prentice Hall, 1996.  UNIX Internals: The New Frontiers, Uresh Vahalia, Prentice Hall, 1996.