1. The Makefile utility

2. Motivation  Small programs single file  “Not so small” programs : –Many lines of code –Multiple components –More than one programmer

3. Mozilla

4. Mozilla Directory Structure One very large directory tree 26,000 files 2,500 subdirectories 240 file types One very large directory tree 26,000 files 2,500 subdirectories 240 file types

5. Mozilla Directory References 1,521 directory references Avg. fan-in/fan-out 12 Max fan-in=100 Max fan-out=42 Median fan-in=5 Median fan-out=12 1,521 directory references Avg. fan-in/fan-out 12 Max fan-in=100 Max fan-out=42 Median fan-in=5 Median fan-out=12

6. Build-level  File (atomic entity) –Source –Documentation  Directory tree (container) = source tree  Build process –To build/install software –Driven by make/ANT/…  Configuration process –To control build process –Driven by configure/configuration files/…  Build-level interfaces –Build interface –Configuration interface

7. Mozilla Build Level 1,350 Makefiles 40,000 LOC build instructions 16,000 LOC configuration Cyclic dependencies Two-phase build process Centralized build/configuration knowledge Code duplication Component implementations scattered 1,350 Makefiles 40,000 LOC build instructions 16,000 LOC configuration Cyclic dependencies Two-phase build process Centralized build/configuration knowledge Code duplication Component implementations scattered

8. Motivation – continued  Problems: –Long files are harder to manage (for both programmers and machines) –Every change requires long compilation –Many programmers can not modify the same file simultaneously –Division to components is desired

9. Motivation – continued  Solution : divide project to multiple files  Targets: –Good division to components –Minimum compilation when something is changed –Easy maintenance of project structure, dependencies and creation

10. Project maintenance  Done in Unix by the Makefile mechanism  A makefile is a file (script) containing : –Project structure (files, dependencies) –Instructions for files creation  The make command reads a makefile, understands the project structure and makes up the executable  Note that the Makefile mechanism is not limited to C programs

11. Project structure  Project structure and dependencies can be represented as a DAG (= Directed Acyclic Graph)  Example : –Program contains 3 files –main.c., sum.c, sum.h –sum.h included in both.c files –Executable should be the file sum

12. sum (exe) sum.o main.o sum.csum.h main.c

13. Make: Header Dependencies  What if … –Sensor.cc changes? –Sensor.h changes? –Robot.h changes?  Pattern rule ignores header dependencies!  Requires unnecessary “make clean; make”  Let gcc figure out the header dependencies for us!  The –MM option produces make rules in a.d file which we can then include in the Makefile Sensor.hSensor.cc Robot.hRobot.cc robotest.cc

14. makefile sum: main.o sum.o gcc –o sum main.o sum.o main.o: main.c sum.h gcc –c main.c sum.o: sum.c sum.h gcc –c sum.c

15. Rule syntax main.o: main.c sum.h gcc –c main.c tab dependency action Rule

16. Equivalent makefiles .o depends (by default) on corresponding.c file. Therefore, equivalent makefile is : sum: main.o sum.o gcc –o sum main.o sum.o main.o: sum.h gcc –c main.c sum.o: sum.h gcc –c sum.c

17. Equivalent makefiles - continued  We can compress identical dependencies and use built-in macros to get another (shorter) equivalent makefile : sum: main.o sum.o gcc –o $@ main.o sum.o main.o sum.o: sum.h gcc –c $*.c

18. make operation  Project dependencies tree is constructed  Target of first rule should be created  We go down the tree to see if there is a target that should be recreated. This is the case when the target file is older than one of its dependencies  In this case we recreate the target file according to the action specified, on our way up the tree. Consequently, more files may need to be recreated  If something is changed, linking is usually necessary

19. make operation - continued  make operation ensures minimum compilation, when the project structure is written properly  Do not write something like: prog: main.c sum1.c sum2.c gcc –o prog main.c sum1.c sum2.c which requires compilation of all project when something is changed

20. Make operation - example File Last Modified sum 10:03 main.o09:56 sum.o 09:35 main.c 10:45 sum.c 09:14 sum.h 08:39

21. Make operation - example  Operations performed: gcc –c main.c gcc –o sum main.o sum.o  main.o should be recompiled (main.c is newer).  Consequently, main.o is newer than sum and therefore sum should be recreated (by re- linking).

22. Another makefile example # Makefile to compare sorting routines BASE = /home/blufox/base CC = gcc CFLAGS = -O –Wall EFILE = $(BASE)/bin/compare_sorts INCLS = -I$(LOC)/include LIBS = $(LOC)/lib/g_lib.a \ $(LOC)/lib/h_lib.a LOC = /usr/local OBJS = main.o another_qsort.o chk_order.o \ compare.o quicksort.o $(EFILE): $(OBJS) @echo “linking …” @$(CC) $(CFLAGS) –o $@ $(OBJS) $(LIBS) $(OBJS): compare_sorts.h $(CC) $(CFLAGS) $(INCLS) –c $*.c # Clean intermediate files clean: rm *~ $(OBJS)

23. Example - continued  We can define multiple targets in a makefile  Target clean – has an empty set of dependencies. Used to clean intermediate files.  make –Will create the compare_sorts executable  make clean –Will remove intermediate files

24. Passing parameters to makefile  We can pass parameters to a makefile by specifying them along with their values in the command line.  For example: make PAR1=1 PAR2=soft1 will call the makefile with 2 parameters: PAR1 is assigned the value “1” and PAR2 is assigned the value “soft1”. The same names should be used within the makefile to access these variables (using the usual “$(VAR_NAME)” syntax)

25. Passing parameters - continued  Note that assigning a value to a variable within the makefile overrides any value passed from the command line.  For example: command line : make PAR=1 in the makefile: PAR = 2  PAR value within the makefile will be 2, overriding the value sent from the command line

26. Conditional statements  Simple conditional statements can be included in a makefile.  Usual syntax is: ifeq (value1, value2) body of if else body of else endif

27. Conditional statements - example sum: main.o sum.o gcc –o sum main.o sum.o main.o: main.c sum.h gcc –c main.c #deciding which file to compile to create sum.o ifeq ($(USE_SUM), 1) sum.o: sum1.c sum.h gcc –c sum1.c –o $@ else sum.o: sum2.c sum.h gcc –c sum2.c –o $@ endif

28. Make: Advanced Options  Text manipulation functions –$(patsubst pattern,replacement,text) –$(patsubst %.o,%.cc, )  Pattern rules –Uses a pattern in the target with % as wildcard –Matched % can be used in dependencies as well –Simple Example: %.o : %.cc command …  Pattern rules with automatic variables –$@ full target name –$< first dependency –$* string which matched % wildcard –Advance Example: %.o : %.cc $(CC) $(CCFLAGS) –c $< $(INCPATHS)

29. Make: A Simple Example CC=g++ # Compiler to use FLAGS=-g # Compile flags MASLAB_ROOT=maslab-software # Maslab software root directory LIB_DIR=$(MASLAB_ROOT)/liborc # orc-related library directory INC_DIR=$(MASLAB_ROOT)/liborc # orc-related include directory LIBS=-lm –lpthread -lorc # Library files all : helloworld helloworld.o : helloworld.cc $(CC) $(FLAGS) –c $*.cc –o $@ helloworld: helloworld.o $(CC) -o helloworld helloworld.o $(LIBS) clean: rm -f *.o helloworld

30. Make: Example Makefile MASLABROOT = /mnt/maslab/software/maslab-software-current #This is the list of places to look for include files INCPATHS = -I$(MASLABROOT)/libs/liborc \ -I$(MASLABROOT)/libs/libim \ -I$(MASLABROOT)/libs/libbotserver \ -I/opt/intel/ipp/include #This is the list of places to look for libraries LIBPATHS = -L$(MASLABROOT)/libs/liborc \ -L$(MASLABROOT)/libs/libim \ -L$(MASLABROOT)/libs/libbotserver \ -L/opt/intel/ipp/sharedlib #This is the names of the libraries LIBS = -lippipx -lippcvpx -lim -lorc -lbotserver -lm -lpthread -ljpeg -lpng #This is the compiler to use CC = g++ # This is your c++ file extension (usually cc or cpp) CCEXT = cc

31. Make: Example Makefile (cont) CCFLAGS = -g -Wall # This rule builds everything. You should put the names of all your # programs in this list. all : robotest # This rule says how to turn any.cpp file into a.o file. %.o : %.$(CCEXT) $(CC) $(CCFLAGS) -c $< $(INCPATHS) #---------------------------------------------------------------------- # robotest PROGRAM_NAME = robotest PROGRAM_OBJS = robotest.o BumpSensor.o Robot.o GLOBAL_OBJS += $(PROGRAM_OBJS) JUNK += $(PROGRAM_NAME) $(PROGRAM_NAME): $(PROGRAM_OBJS) $(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS) chmod a+x $(PROGRAM_NAME)

32. Extending Example Makefile #---------------------------------------------------------------------- # Sensor Incremental Test PROGRAM_NAME = sensor-test PROGRAM_OBJS = sensor.t.o BumpSensor.o GLOBAL_OBJS += $(PROGRAM_OBJS) JUNK += $(PROGRAM_NAME) $(PROGRAM_NAME): $(PROGRAM_OBJS) $(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS) chmod a+x $(PROGRAM_NAME) So to add a new executable program, simply cut and paste the robotest section and enter your own PROGRAM_NAME and PROGRAM_OBJS

33. Extending example Makefile DEPENDS += $(patsubst %.o, %.d, $(GLOBAL_OBJS)) JUNK += $(DEPENDS) include $(DEPENDS) depend : $(DEPENDS) depend.$(CCEXT) = set -e; $(CC) $(CCFLAGS) -MM $(DEFS) $(INCPATHS) depend.filt = sed 's/\($*\)\.o[ :]*/\1.o $@ : /g' > $@; [ -s $@ ] || rm -f $@ %.d: %.$(CCEXT) $(depend.cc) $< | $(depend.filt) #====================================================================== # Miscellaneous Rules #---------------------------------------------------------------------- # This rule cleans your directory. You could also add commands here # to remove program files and any other files that should be cleaned clean: rm -f $(JUNK) *~ *.o core.[0-9]* core

34. CMAKE: a cross platform Make

35. Build-System Generator  Provides single-sourcing for build systems  Knowledge of many platforms and tools  Users configure builds through a GUI CMakeLists.txt CMake Native Build System Native Build Tools Executables Libraries

36. Source and Build Trees  The Source Tree contains: –CMake input files (CMakeLists.txt) –Program source files (hello.cxx)  The Binary Tree (build tree) contains: –Native build system files (hello.dsp) –Program libraries and executables (hello.exe)  Source and Binary trees may be: –In the same directory (in-source build) –In different directories (out-of-source build)

37. The CMake Cache  Represents build configuration  Populated by CMake code  Stored in CMakeCache.txt at top of build  Entries have a type to help the GUI  Holds global information for CMake code  Updated by CMake configuration phase

38. Command Line Usage  Can be used from scripts  Set current-working-directory to binary tree  Pass path to source tree as first argument  Use -G to select build system generator  Use -D to set cache variables $ cd Foo-msvc-6 $ cmake../Foo –G“Visual Studio 6” –DBAR:BOOL=1

39. GUI UsageCMakeSetupccmake  Edit cache entries to configure the build  Use configure button after a change  Use OK (generate) button when finished

40. Source Tree Structure  Every directory has a CMakeLists.txt file  Subdirectories specified by SUBDIRS  Directories depend only on parents  A subset of commands are inherited CMakeLists.txt SUBDIRS(Dir1 Dir2) Dir1/CMakeLists.txt Dir2/CMakeLists.txt

41. Writing CMakeLists.txt Files  CMake language evolved while in use  Scripting language with simple syntax –Comments –Commands –Lists –Variables –Control structures  Processed during CMake configure phase # Comment ends at a newline COMMAND(arg1 arg2...) A;B;C # Semicolon-separated IF(CONDITION) ${VAR}

42. Commands  Simple syntax:  Each command must start on its own line  Variable references are replaced by values  Lists in unquoted arguments are expanded  Argument meanings defined by command  Both positional and keyword arguments used COMMAND(ARG “ARG WITH SPACES” ${A_LIST} “${A_STRING}”) TARGET_LINK_LIBRARIES(myTarget lib1 lib2) FIND_LIBRARY(MY_LIB NAMES my1 my2 PATHS /foo /bar)

43. Variables  Named by C-style identifier  Value is always a string  No associated type  Initialized by CMake cache entries  Assigned through commands like SET  Referenced by ${VAR} (only one level) SET(A_LIST ${A_LIST} foo) SET(A_STRING “${A_STRING} bar”)

44. Control Structures  IF  FOREACH  MACRO IF(CONDITION) MESSAGE(“Yes”) ELSE(CONDITION) MESSAGE(“No”) ENDIF(CONDITION) FOREACH(c A B C) MESSAGE(“${c}: ${${c}}”) ENDFOREACH(c) MACRO(MY_MACRO arg1 arg2) SET(${arg1} “${${arg2}}”) ENDMACRO(MY_MACRO) MY_MACRO(A B)

45. A Typical Project PROJECT(FOO) SUBDIRS(Foo Bar Executable)CMakeLists.txt ADD_LIBRARY(foo foo1.cxx foo2.cxx)Foo/CMakeLists.txt ADD_EXECUTABLE(zot zot1.cxx zot2.cxx) TARGET_LINK_LIBRARIES(zot bar)Executable/CMakeLists.txt ADD_LIBRARY(bar bar1.cxx bar2.cxx) TARGET_LINK_LIBRARIES(bar foo)Bar/CMakeLists.txt

46. Developer Documentation  Command-line documentation: –Run “ cmake --help ” for summary –Run “ cmake --help COMMAND ” for detailed help with a specific listfile command –Try “ cmake --help IF ”  Online documentation: –http://www.cmake.org/HTML/Documentation.html  Mastering CMake –Published by Kitware, Inc. –ISBN 1-930934-09-2

47. Editing CMake Code  EMACS mode for CMake –cmake-mode.el located in CMake/Docs directory –Provides highlighting and indentation –Use this code in your.emacs file:  VIM mode is also available (setq load-path (cons “/path/to/cmake-mode” load-path)) (require 'cmake-mode) (setq auto-mode-alist (append '(("CMakeLists.txt". cmake-mode) ("\\.cmake$". cmake-mode)) auto-mode-alist))

48. ANT: Another Nice Tool

49. What is Ant?  Java-based Build tool from Apache  De facto standard for building, packaging, and installing Java applications  Accomplishes same objectives that make does on Unix based systems  Files are written in XML

50. Why Ant?  Many claim.. That –Unlike makefiles, Ant files work cross platform - No need for multiple, complex makefiles depending on the operating system. - Tasks declared in platform independent way; Ant engine translates to OS specific commands. But still need to know (potentially) complex path/install data. So not as general as they claim  Easy to create own Ant “tasks”, in addition to core tasks

51. Installing Ant  Download Ant binary distribution from: http://ant.apache.org/bindownload.cgi  Set ANT_HOME to where you installed Ant  Include $ANT_HOME/bin in PATH  Make sure JAVA_HOME is set to point to JDK

52. Running Ant  Type “ant” at the command line  Automatically looks for build.xml file in current directory to run  Type “ant –buildfile buildfile.xml” to specify another build file to run.

53. Ant Output

54. Sample build.xml

55. Ant Overview: Project  Each build file contains exactly one project and at least one target  Project tags specify the basic project attributes and have 3 properties: - name - default target - basedir  Example:

56. Ant Overview: Targets  Target is a build module in Ant  Each target contains task(s) for Ant to do  One must be a project default  Overall structure of targets:

57. Ant Overview: Tasks  Each target comprises one or more tasks  Task is a piece of executable Java code (e.g. javac, jar, etc)  Tasks do the actual “build” work in Ant  Ant has core (built in) tasks and the ability to create own tasks

58. Ant Overview: Tasks Example:

59. Ant Overview: Core Tasks  javac – Runs the Java Compiler  java – Runs the Java Virtual Machine  jar (and war) – Create JAR files  mkdir – Makes a directory  copy – Copies files to specified location  delete – Deletes specified files  cvs – Invokes CVS commands from Ant

60. Ant Overview: Writing Own Task  Create a Java class that extends org.apache.tools.ant.Task  For each attribute, write a setter method that is public void and takes a single argument  Write a public void execute() method, with no arguments, that throws a BuildException -- this method implements the task itself

61. Ant Overview: Properties  Special task for setting up build file properties:  Example:  Can use ${src} anywhere in build file to denote /home/src  Ant provides access to all system properties as if defined by the task

62. Ant Overview: Path Structures  Ant provides means to set various environment variables like PATH and CLASSPATH.  Example of setting CLASSPATH:

63. Command Line Arguments  -buildfile buildfile – specify build file to use  targetname – specify target to run (instead of running default)  -verbose, -quiet, -debug – Allows control over the logging information Ant outputs  - logger classname – Allows user to specify their own classes for logging Ant events

64. IDE Integration  Eclipse, NetBeans, JBuilder, VisualAge, and almost any other Java IDE has Ant integration built-in to the system  Refer to each IDE’s documentation for how to use Ant with that IDE

65. Web Development with Ant  Tomcat comes with special Ant tasks to ease Web application development and deployment  Copy $TOMCAT_HOME/server/lib/catalina-ant.jar to $ANT_HOME/lib  Ant tasks for Tomcat: - install - reload - deploy - remove

66. Documentation/References  Good tutorial for makefiles http://www.gnu.org/manual/make-3.80/make.html  Good tutorial for cmakef http://www.cmake.org/HTML/Documentation.html  ANT User Manual: http://ant.apache.org/manual/index.html http://ant.apache.org/manual/index.html  Sun’s Web development tutorial (Ant and JSPs): http://java.sun.com/webservices/docs/1.2/tutorial/doc/GettingStarted3.html

67. Standardized Build Interface  Different build systems exist, e.g., –make, ANT, shell scripts, IDE  Different software systems require different build actions, e.g., –make –make bootstrap, make, make install make clean make all make check make (un)install make dist make clean make all make check make (un)install make dist

68. The real Problem  How do we handle platform specific issues? –Providing a different Makefile for each architecture –Using Autoconf, Automake and Libtool  The installer needs only –Bourne shell –C compilers –Make program

69. Standardized Configuration Interface  Different mechanisms exist to control software construction, e.g., –configuration files, configuration tools, Makefile editing  Standardized configuration interface to enable uniform compile-time configuration: configure --help configure --prefix=/usr configure --with-aterm=/usr/lib configure --with-optimization=true configure --with-debug=false configure --help configure --prefix=/usr configure --with-aterm=/usr/lib configure --with-optimization=true configure --with-debug=false

70. Explicit Context Dependencies  Dependencies on build-level components are declared in configuration interfaces > configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR … > configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …

71. Independent Deployment  Build-level components are deployed as packages  A package is a versioned release of a build-level component, e.g., firefox-1.0.tar.gz  Packages are published on web/ftp sites

72. Third-party Composition  A configuration interface enables late-binding of dependencies  Compositions are thus not predefined  and can be defined by a third party > configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…= > configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=

73. Some advantages when using GNU autotools  The installation of a program is straightforward:./configure; make; make install  This procedure checks for system parameters, libraries, location of programs, availability of functions and writes a Makefile ./configure supports many options to overwrite defaults settings

74. GNU autoconf autoscan configure.scan configure.ac (configure.in) autoconf configure Source Code aclocal

75. GNU automake Makefile.am automake Makefile.in configure Makefile

76. configure.ac  dnl Comment … …  AC_INIT(main.c)  AM_INIT_AUTOMAKE(project_name, 1.2.8)  AM_PROG_LIBTOOLit supports libtool and shared libraries  AC_PROG_CC (or AC_PROG_CXX)it locates the C (C++) compiler  AC_HEADER_STDCit checks for standard headers  AC_CHECK_HEADERS(sys/time.h /header.h)it checks for headers availability  AC_CHECK_LIB(crypto SSLeay_version)it checks for libraries availability  AC_CHECK_FUNCS(ctime)it checks for functions availability  AC_PROG_INSTALLit checks for BSD compatible install utility  AC_OUTPUT([Makefile])

77. Makefile.am  bin_PROGRAMS = foo /configure --prefix=… (default /usr/local)  foo_PROGRAMS=foo.c foo.h  noist_PROGRAMS=test ( make compiles, make install does nothing)  EXTRA_DIST=disclaimer.txt

78. Example  foo.c : #include main() { printf(“Cum grano salis\n"); }  Makefile.am : bin_PROGRAMS = foo foo_SOURCES = foo.c  configure.ac : AC_INIT(foo.c) AM_INIT_AUTOMAKE(latin_words, 0.9) AC_PROG_CC AC_HEADER_STDC AC_PROG_INSTALL AC_OUTPUT([Makefile])

79. Summary  Source Code, configure.ac, Makefile.am  autoscan; aclocal; autoconf  Create NEWS README AUTHORS ChangeLog  automake –add-missing ./configure; make; make dist  Result: project_name-2.10.tar.gz aclocal.m4 autom4te-2.53.cache ChangeLog config.status configure.in COPYING install-sh Makefile.am missing NEWS README AUTHORS autoscan.log config.log configure configure.scan INSTALL Makefile.in mkinstalldirs code.c

80. References  GNU Autoconf, Automake, and Libtool http://sources.redhat.com/autobook/autobook/autobook_toc.html http://sources.redhat.com/autobook/autobook/autobook_toc.html  GNU Autoconf Manual http://www.gnu.org/manual/autoconf http://www.gnu.org/manual/autoconf  GNU Automake Manual http://www.gnu.org/manual/automake http://www.gnu.org/manual/automake  GNU Libtool Manual http://www.gnu.org/manual/libtool http://www.gnu.org/manual/libtool  Learning the GNU development tools http://autotoolset.sourceforge.net/tutorial.html http://autotoolset.sourceforge.net/tutorial.html  The GNU configure and build system http://www.airs.com/ian/configure/configure_toc.html http://www.airs.com/ian/configure/configure_toc.html  GNU macro processor (GNU m4) http://www.gnu.org/manual/m4-1.4/m4.html http://www.gnu.org/manual/m4-1.4/m4.html