1. University of Maryland Profile-Driven Selective Program Loading Tugrul Ince tugrul@cs.umd.edu Jeff Hollingsworth Department of Computer Science University of Maryland, College Park, MD 20742

2. University of Maryland 2 Motivation Programs are getting larger! –Many frameworks and libraries Many supercomputers lack demand-paging –Example: Cray XT and BlueGene series –Available memory is scarce Observation: Most programs do not use every available function! –Frameworks and libraries are too general –Code that handles errors or special cases Why not remove functions that are not used in the common case?

3. University of Maryland 3 Aim Reduce memory footprint by selectively loading parts of shared libraries

4. University of Maryland Target Platforms and Applications Unix/Linux systems that support ELF –Modifies ELF program headers Applications with many libraries –Most current reasonable applications Parallel programs running on multiple nodes –MPI etc. Platforms without demand-paging –Cray XT and BlueGene series 4

5. University of Maryland Architecture Overview 5 Application is profiled. It is rewritten with –Modified Shared Libraries –A Signal Handler Application is executed as usual.

6. University of Maryland Profiler Need a list of never-called functions in each shared library –Profile the application several times –May not be perfect DynInst-based profiler –Write small program (~ 70 LOC) –Rewrite shared libraries –Profile as many times as necessary 6

7. University of Maryland Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x000000 0x00000000 0x00000000 0x090000 0x090000 R E 0x1000 LOAD 0x112000 0x00112000 0x00112000 0x012584 0x012584 R E 0x1000 Rewriting Do not load unused functions –Modify ELF program headers –Example: libpetsc.so 7 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x000000 0x00000000 0x00000000 0x124584 0x124584 R E 0x1000.text LOAD 0x124584 0x00125584 0x00125584 0x013f8 0x0a434 RW 0x1000 DYNAMIC 0x12459c 0x0012559c 0x0012559c 0x00130 0x00130 RW 0x4 GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4 First Loadable Section:.text,.init,.fini,.plt Second Loadable Section:.dynamic,.got,.got.plt,.data,.bss

8. University of Maryland Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x000000 0x00000000 0x00000000 0x090000 0x090000 R E 0x1000 LOAD 0x112000 0x00112000 0x00112000 0x012584 0x012584 R E 0x1000 Rewriting Do not load unused functions –Modify ELF program headers –Example: libpetsc.so 8.text LOAD 0x124584 0x00125584 0x00125584 0x013f8 0x0a434 RW 0x1000 DYNAMIC 0x12459c 0x0012559c 0x0012559c 0x00130 0x00130 RW 0x4 GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4 First Loadable Section:.text,.init,.fini,.plt Second Loadable Section:.dynamic,.got,.got.plt,.data,.bss

9. University of Maryland Rewriting Rewriter based on DynInst Profile data is used to create lists of Used and Unused functions Access / Modify symbols Defragment functions to maximize space savings –Requires moving functions inside shared libraries 9

10. University of Maryland Function Defragmentation 10 Used Unused

11. University of Maryland Challenges: Relative Calls Common way of calling functions in PIC. If either callee or caller is moved, their relative positioning changes. Offsets in such relative call instructions need to be updated 11 call d foo d call d’ foo d'

12. University of Maryland Challenges: Symbols Runtime linker uses symbols to resolve cross-library calls. –Uses procedure linkage tables (plt) If a function is moved, its associated symbol has to be updated. 12 call foo@plt foo@plt foo: 0xdeadbeef foo call foo@plt foo@plt foo: 0xbeefdead foo

13. University of Maryland Challenges: Jump Tables Used to represent n-way branches at machine level Targets are read from jump table –Entries are offsets of targets from the GOT address Becomes invalid if the function referenced in a jump table is moved DynInst reads jump tables to generate CFGs We update entries so that they can be used to point to new location of targets 13

14. University of Maryland Unexpectedly Called Function Execution is not always predictable –Unexpected function calls Rewrite original executable with a Signal Handler Load the function upon an unexpected call –Signal Handler picks up page faults (SIGSEGV) –Loads requested page on-demand –Execution resumes User-level: No OS modifications 14

15. University of Maryland 15 Experiments Tested on –PETSc ex5 in snes package –PETSc ex2 in ksp package –GS2 Compiled with debug flag and no optimization Used Open MPI Tested on 64-node cluster at UMD –Dual-core x86 processors –Unmodified Linux kernel Space savings of about 82% on average

16. University of Maryland PETSc – snes (ex5) 16 Library Name Text Pages (Original) Text Pages (Modified) Reduction % petsc2606873.85 petscdm1611988.2 petscksp3353988.36 petscmat7724094.82 petscvec2045274.51 petscsnes20 0 mpi_cxx10550 mpi1423773.94 open-pal623445.16 open-rte553438.18 m28389.29 Library Name Text Pages (Original) Text Pages (Modified) Reductio n % X11146795.21 lapack866299.77 blas80396.25 stdc++1331290.98 gcc_s12283.33 Xau220 Xdcm330 gfortran123496.75 dl220 nsl14285.71 util220 OVERALL202134882.78

17. University of Maryland PETSc – snes (ex5) 17

18. University of Maryland PETSc – ksp (ex2) 18 Library Name Text Pages (Original) Text Pages (Modified)Reduction % petsc2607272.31 petscdm161398.14 petscksp3354985.37 petscmat7724993.65 petscvec2045473.53 mpi_cxx10550 mpi1424766.9 open-pal623740.32 open-rte553634.55 OVERALL200135282.41

19. University of Maryland GS2 19 Library NameText Pages (Original) Text Pages (Modified)Reduction % MdsLib210100 MdsShr210100 TdiShr220398.64 TreeShr380100 fftw702564.29 rfftw58886.21 mpi_f7713284.62 mpi1424071.83 open-pal623641.94 open-rte553634.55 OVERALL70015078.57

20. University of Maryland Running Times GS2 takes 5 seconds less on average –(36m 38s vs. 36m 33s) Overhead on PETSc examples –ex2 runs for 2.7 secs, ex5 runs for 1.05 secs. 20

21. University of Maryland Running Times Results suggest no overhead for reasonably-long running programs –Initial cost for signal handler registration –Better instruction cache and TLB performance 21

22. University of Maryland 22 Summary Our tool reduces memory footprint of shared libraries Rewrite shared libraries with holes –Defragment functions to maximize space savings On-demand page loading if a not-yet- loaded function is called About 82% memory space savings for shared libraries Might improve instruction cache and TLB performance