1. Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Automated Builds

2. Introduction A typical working day

3. Automated BuildsIntroduction A typical working day Re-draw Figure 8

4. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data

5. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Recompile stats program

6. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Recompile stats program Update Java

7. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java

8. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java

9. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

10. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

11. Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space...shave the yak...

12. Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

13. Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task

14. Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task dependencies

15. Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task dependencies Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

16. Automated BuildsIntroduction This pattern arises frequently

17. Automated BuildsIntroduction This pattern arises frequently New data collected? Recalculate statistics

18. Automated BuildsIntroduction This pattern arises frequently Source files changed? Recompile program New data collected? Recalculate statistics

19. Automated BuildsIntroduction This pattern arises frequently Source files changed? Recompile program New data collected? Recalculate statistics New content written? Update web site

20. Automated BuildsIntroduction Hard or impossible to keep track of:

21. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what

22. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't

23. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating."

24. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process

25. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file

26. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file Along with commands used to update things

27. Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file Along with commands used to update things Build manager does the rest

28. Automated BuildsIntroduction Most widely used build manager is Make

29. Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular"

30. Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975

31. Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language

32. Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language A very cryptic little language, without a debugger...

33. Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language A very cryptic little language, without a debugger......that requires an understanding of the Unix shell

34. Automated BuildsIntroduction GNU Make is fast, free, and well-documented

35. Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it

36. Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features

37. Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features Companion lecture explores SCons

38. Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features Companion lecture explores SCons Java users should look at Ant

39. August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

40. Automated BuildsIntroduction Basics Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Automated Builds

41. Introduction Manage tasks and dependencies

42. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

43. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp

44. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp sgr -N -r summary-1.dat \ > figure-1.svg

45. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat

46. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py summary-1.dat data-1-*.dat sgr -N -r summary-1.dat \ > figure-1.svg

47. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat

48. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py summary-1.dat \ data-1-1.dat data-1-2.dat \ data-1-3.dat stats.py sgr -N -r summary-1.dat \ > figure-1.svg

49. Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

50. Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

51. Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ Data file is newer than SVG image

52. Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

53. Automated BuildsIntroduction Comment 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

54. Automated BuildsIntroduction Rule 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

55. Automated BuildsIntroduction Target 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

56. Automated BuildsIntroduction Prerequisite 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

57. Automated BuildsIntroduction Action 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

58. Automated BuildsIntroduction Must indent with a single tab character 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

59. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk

60. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk -f filename

61. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $

62. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ Prerequisite is newer than target

63. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ Prerequisite is newer than target So Make executes the action

64. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $

65. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $ Target is newer than prerequisite

66. Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $ Target is newer than prerequisite So action not executed

67. Automated BuildsIntroduction Usually have multiple rules per file

68. Automated BuildsIntroduction Usually have multiple rules per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

69. Automated BuildsIntroduction Usually have multiple rules per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $

70. Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ Update timestamps on files

71. Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ gmake -f double.mk sgr -N -r summary-1.dat > figure-1.svg $

72. Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ gmake -f double.mk sgr -N -r summary-1.dat > figure-1.svg $ Why isn't figure-2.svg rebuilt?

73. Automated BuildsIntroduction First rule in Makefile is default rule

74. Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise

75. Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly

76. Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly $ gmake -f double.mk figure-2.svg sgr -N -r summary-2.dat > figure-2.svg $

77. Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly $ gmake -f double.mk figure-2.svg sgr -N -r summary-2.dat > figure-2.svg $ Better than typing commands one by one, but only slightly

78. Automated BuildsIntroduction Introduce a phony target

79. Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file

80. Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date

81. Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it

82. Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it # phony.mk all : figure-1.svg figure-2.svg figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

83. Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it # phony.mk all : figure-1.svg figure-2.svg figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

84. Automated BuildsIntroduction "make all" rebuilds everything

85. Automated BuildsIntroduction "make all" rebuilds everything $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $

86. Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed "make all" rebuilds everything

87. Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first "make all" rebuilds everything

88. Automated BuildsIntroduction "make all" rebuilds everything $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first Or re-create both in parallel

89. Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first Or re-create both in parallel Return to that idea later "make all" rebuilds everything

90. Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat

91. Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph

92. Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic

93. Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic Y X Z

94. Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic Y X Z

95. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat

96. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

97. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

98. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat How to generalize to any number of files?

99. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat How to generalize to any number of files? And get rid of repeated filenames

100. Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

101. Automated BuildsIntroduction Patterns Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Automated Builds

102. Introduction Manage tasks and dependencies

103. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

104. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat

105. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

106. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

107. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

108. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

109. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat

110. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat

111. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable

112. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable "the target of this rule"

113. Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable "the target of this rule" No, there isn't a more readable form

114. Automated BuildsIntroduction # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat Still a lot of redundancy summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

115. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

116. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

117. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ All prerequisites of this rule summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

118. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ All prerequisites of this rule $< is "the first prerequisite" summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

119. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ All prerequisites of this rule $< is "the first prerequisite" $? is "all out-of-date prerequisites" summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

120. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

121. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 Expect more data files

122. Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Want to do this

123. Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat

124. Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Just like shell wildcard

125. Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $@ $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Just like shell wildcard Must use $^ in action, since filenames not fixed in advance

126. Automated BuildsIntroduction figure-1.svg summary-1.dat figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat The whole tree one more time

127. Automated BuildsIntroduction The makefile so far paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^

128. Automated BuildsIntroduction Still some redundancy paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ The makefile so far

129. Automated BuildsIntroduction Still some redundancy Fix in next episode paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ The makefile so far

130. Automated BuildsIntroduction Still some redundancy Fix in next episode paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ Doesn't handle summaries' dependency on stats.py The makefile so far

131. Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : stats.py data-1-*.dat stats.py $@ $^ summary-2.dat : stats.py data-2-*.dat stats.py $@ $^ Option 1: add to existing rules

132. Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $@ $^ summary-2.dat : stats.py data-2-*.dat stats.py $@ $^ ⋮ Option 1: add to existing rules $^ is now stats.py data-1-1.dat data-1-1.dat...

133. Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $@ $^ summary-2.dat : stats.py data-2-*.dat stats.py $@ $^ ⋮ $^ is now stats.py data-1-1.dat data-1-1.dat... So the invocation of stats.py is wrong Option 1: add to existing rules

134. Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $@ $^ summary-2.dat : stats.py data-2-*.dat stats.py $@ $^ ⋮ $^ is now stats.py data-1-1.dat data-1-1.dat... So the invocation of stats.py is wrong Having it ignore one argument is an ugly hack Option 1: add to existing rules

135. Automated BuildsIntroduction figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ data-1-1.dat : stats.py touch $@ data-1-2.dat : stats.py touch $@ Option 2: make data files depend on stats.py

136. Automated BuildsIntroduction ⋮ data-1-1.dat : stats.py touch $@ data-1-2.dat : stats.py touch $@ ⋮ Option 2: make data files depend on stats.py A false dependency

137. Automated BuildsIntroduction Option 2: make data files depend on stats.py ⋮ data-1-1.dat : stats.py touch $@ data-1-2.dat : stats.py touch $@ ⋮ A false dependency Updating raw data files triggers update of summary

138. Automated BuildsIntroduction Option 2: make data files depend on stats.py ⋮ data-1-1.dat : stats.py touch $@ data-1-2.dat : stats.py touch $@ ⋮ A false dependency Updating raw data files triggers update of summary Back to listing all raw data files explicitly…

139. Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Option 3: add additional dependencies

140. Automated BuildsIntroduction ⋮ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py ⋮ Option 3: add additional dependencies Full set of dependencies is union of lists

141. Automated BuildsIntroduction ⋮ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py ⋮ Option 3: add additional dependencies Full set of dependencies is union of lists But $^ in the action is still just data-1-*.dat

142. Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

143. Automated BuildsIntroduction Rules Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Automated Builds

144. Introduction Manage tasks and dependencies

145. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py

146. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat

147. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

148. Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Makefile so far

149. Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $@ $^ figure-2.svg : summary-2.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Makefile so far Eliminate this redundancy

150. Automated BuildsIntroduction Use a pattern rule to capture the common idea

151. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Use a pattern rule to capture the common idea

152. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Use a pattern rule to capture the common idea % is a wildcard

153. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Use a pattern rule to capture the common idea

154. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Undefined in the action Use a pattern rule to capture the common idea

155. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Undefined in the action Have to use $@, $^, etc. Use a pattern rule to capture the common idea

156. Automated BuildsIntroduction Try running it

157. Automated BuildsIntroduction $ make -f pattern-rule.mk stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat $ Try running it

158. Automated BuildsIntroduction $ make -f pattern-rule.mk stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat $ Try running it Why didn't other commands run?

159. Automated BuildsIntroduction Pattern rule doesn't create dependencies itself

160. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py If Make wants to create figure-1.svg, it can use this rule Pattern rule doesn't create dependencies itself

161. Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py If Make wants to create figure-1.svg, it can use this rule Still have to tell Make what it wants to do Pattern rule doesn't create dependencies itself

162. Automated BuildsIntroduction Put the rule for paper.pdf back in the file

163. Automated BuildsIntroduction # use-pattern.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Put the rule for paper.pdf back in the file

164. Automated BuildsIntroduction # use-pattern.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Make now knows that it needs to create the figures, so it finds and uses the rule Put the rule for paper.pdf back in the file

165. Automated BuildsIntroduction # doesnt-work.mk paper.pdf : paper.wdp figure-*.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py This doesn't work!

166. Automated BuildsIntroduction # doesnt-work.mk paper.pdf : paper.wdp figure-*.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Figures don’t exist when Make starts to run, so this is empty This doesn't work!

167. Automated BuildsIntroduction # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-1.dat : data-1-*.dat stats.py $@ $^ summary-2.dat : data-2-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Get rid of more redundancy

168. Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py

169. Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Make wildcard

170. Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-1.dat : stats.py summary-2.dat : stats.py Shell wildcard

171. Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-%.dat : stats.py

172. Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-%.dat : stats.py summary-*.dat still only depends on data-*-*.dat

173. Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ summary-%.dat : stats.py summary-*.dat still only depends on data-*-*.dat Make only uses the first pattern rule it finds

174. Automated BuildsIntroduction Back to using false dependencies... # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@

175. Automated BuildsIntroduction Back to using false dependencies... # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ It's a less-than-perfect tool...

176. Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

177. Automated BuildsIntroduction Macros Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Automated Builds

178. Introduction Manage tasks and dependencies

179. Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

180. Automated BuildsIntroduction "must conform to university style"

181. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp

182. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers home

183. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers/lib/styles/ homelab

184. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp data-1-1.datdata-1-2.dateuphoric.fig

185. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers home data-1-1.datdata-1-2.dateuphoric.fig

186. Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers /lib/styles/ home lab data-1-1.datdata-1-2.dateuphoric.fig

187. Automated BuildsIntroduction # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Makefile so far

188. Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Add directories for working at home

189. Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Add directories for working at home

190. Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Usually don't list "system" files explicitly Add directories for working at home

191. Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Usually don't list "system" files explicitly But what about the lab? Add directories for working at home

192. Automated BuildsIntroduction 1. Write two Makefiles

193. Automated BuildsIntroduction 1. Write two Makefiles Write and maintain

194. Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain

195. Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain Consistently every time

196. Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain Consistently every time Will create lots of noise in version control

197. Automated BuildsIntroduction 1.Write two Makefiles 2.Comment and uncomment lines 3.Refactor Write and maintain Consistently every time Will create lots of noise in version control

198. Automated BuildsIntroduction Use a macro

199. Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Use a macro

200. Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Use a macro

201. Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Use a macro

202. Automated BuildsIntroduction Only have one thing to change

203. Automated BuildsIntroduction Only have one thing to change ✓ Consistency

204. Automated BuildsIntroduction Only have one thing to change ✓ Consistency ✗ But still have noise

205. Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO ✓ Consistency ✗ But still have noise

206. Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO ✓ Consistency ✗ But still have noise

207. Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" ✓ Consistency ✗ But still have noise

208. Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" Which is probably not what you want ✓ Consistency ✗ But still have noise

209. Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" Which is probably not what you want yet another legacy wart ✓ Consistency ✗ But still have noise

210. Automated BuildsIntroduction Commonly define macros for control flags

211. Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Commonly define macros for control flags

212. Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Commonly define macros for control flags

213. Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ Commonly define macros for control flags

214. Automated BuildsIntroduction # config.mk STYLE_DIR=c:/papers/ Now put the first macro in a separate file

215. Automated BuildsIntroduction # with-include.mk include config.mk WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ And include it from the main file

216. Automated BuildsIntroduction # with-include.mk include config.mk WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $@ $^ summary-%.dat : data-%-*.dat stats.py $@ $^ data-*-*.dat : stats.py touch $@ And include it from the main file

217. Automated BuildsIntroduction Actually create two configuration files

218. Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files

219. Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles

220. Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles These two files stay in version control

221. Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles These two files stay in version control Copy to create config.mk per machine

222. Automated BuildsIntroduction paper/ config-lab.mk config-home.mk Makefile Home

223. Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home

224. Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config-lab.mk config-home.mk Makefile Lab

225. Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config.mk config-lab.mk config-home.mk Makefile Lab

226. Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config.mk config-lab.mk config-home.mk Makefile Lab

227. Automated BuildsIntroduction Can also define macro value on command line

228. Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line

229. Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea

230. Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time

231. Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time correctly

232. Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time correctly And there's no record of the flag

233. Automated BuildsIntroduction Many other approaches

234. Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent)

235. Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines

236. Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug

237. Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug A build file is a program

238. Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug A build file is a program Requires the same degree of respect

239. Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.