Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Slides:

Advertisements

Similar presentations

Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Advertisements

The make Utility Programming Tools and Environments Winter 2006.

Separate compilation Large programs are generally separated into multiple files, e.g. tuples.h, ray.h, ray.c, tuples.c main.c With several files, we can.

CS201 – Makefile Tutorial. A Trivial Makefile # Trivial Makefile for puzzle1.c # Ray S. Babcock, CS201, MSU-Bozeman # 1/5/05 # puzzle1: puzzle1.c gcc.

Compilation & linkage.h read.h.c read.c.c main.c.c list.c.h list.h prog1 Linkage: g++ read.o main.o list.o –o prog1.o main.o.o list.o.o read.o Compilation:

Copyright © 2008 Pearson Addison-Wesley. All rights reserved. Chapter 12 Separate Compilation Namespaces Simple Make Files (Ignore all class references.

1 CS 201 Makefile Debzani Deb. 2 Remember this? 3 What is a Makefile? A Makefile is a collection of instructions that is used to compile your program.

Guide To UNIX Using Linux Third Edition

CS465 - Unix C Programming (cc/make and configuration control)

1 Building. 2 Goals of this Lecture Help you learn about: The build process for multi-file programs Partial builds of multi-file programs make, a popular.

Basic Unix Dr Tim Cutts Team Leader Systems Support Group Infrastructure Management Team.

Android 4: Creating Contents Kirk Scott 1. Outline 4.1 Planning Contents 4.2 GIMP and Free Sound Recorder 4.3 Using FlashCardMaker to Create an XML File.

Lecture 8  make. Overview: Development process  Creation of source files (.c,.h,.cpp)  Compilation (e.g. *.c  *.o) and linking  Running and testing.

1 Introduction to Tool chains. 2 Tool chain for the Sitara Family (but it is true for other ARM based devices as well) A tool chain is a collection of.

Mechanics Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Version Control with Subversion. What is Version Control Good For? Maintaining project/file history - so you don’t have to worry about it Managing collaboration.

Lesson 7-Creating and Changing Directories. Overview Using directories to create order. Managing files in directories. Using pathnames to manage files.

1 Chapter 2 & Chapter 4 §Browsers. 2 Terms §Software §Program §Application.

July 29, 2003Serguei Mokhov, 1 Makefile Brief Reference COMP 229, 346, 444, 5201 Revision 1.2 Date: July 18, 2004.

1 Lecture 19 Configuration Management Software Engineering.

Makefiles CISC/QCSE 810. BeamApp and Tests in C++ 5 source code files After any modification, changed source needs to be recompiled all object files need.

Introduction Use of makefiles to manage the build process Declarative, imperative and relational rules Environment variables, phony targets, automatic.

Scons Writing Solid Code Overview What is scons? scons Basics Other cools scons stuff Resources.

Makefile M.A Doman. Compiling multiple objects Card.cpp -> Card.o Deck.cpp -> Deck.o main.cpp -> main.o main.o Deck.o Card.o -> Dealer.exe.

Macros Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Test Automation For Web-Based Applications Portnov Computer School Presenter: Ellie Skobel.

Files and Directories Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

GNU Make Computer Organization II 1 © McQuain What is make ? make is a system utility for managing the build process (compilation/linking/etc).

Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Setting up Cygwin Computer Organization I 1 May 2010 ©2010 McQuain Cygwin: getting the setup tool Free, almost complete UNIX environment emulation.

Makefiles. Multiple Source Files (1) u Obviously, large programs are not going to be contained within single files. u C provides several techniques to.

Linux+ Guide to Linux Certification, Third Edition

Linux+ Guide to Linux Certification, Third Edition

1 Day 18 Bash and the.files. 2 The.files ls shows you the files in your directory –Or at least most of them. –Some files are hidden. Try: ls –a –This.

Finding Things Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

C code organization CSE 2451 Rong Shi. Topics C code organization Linking Header files Makefiles.

Exceptions Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Chapter Linux Basics. Acknowledgements This presentation was prepared by – Banyat Settapanich – Bahran Madaen This presentation will be updated later.

1 CS3695 – Network Vulnerability Assessment & Risk Mitigation – Introduction to Unix & Linux.

Build Tools 1. Building a program for a large project is usually managed by a build tool that controls the various steps involved. These steps may include:

CSc 352 An Introduction to make Saumya Debray Dept. of Computer Science The University of Arizona, Tucson

1 Week 8 Creating Simple Shell Scripts. 2 Chapter Objectives  In this chapter, you will :  Learn how to create Shell Scripts  Commenting / Making Portable.

UNIX Development: g++ and make CS 2204 Class meeting 8 Created by Doug Bowman, 2001 Modified by Mir Farooq Ali, 2002.

Patterns Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

L071 Introduction to C Topics Compilation Using the gcc Compiler The Anatomy of a C Program Reading Sections

Makefiles Manolis Koubarakis Data Structures and Programming Techniques 1.

Week Seven Agenda Link of the week Review week six lab assignment This week’s expected outcomes Next lab assignment Break-out problems Upcoming deadlines.

The make utility (original presentation courtesy of Alark Joshi)

CSE 303 Lecture 17 Makefiles reading: Programming in C Ch. 15

Prof: Dr. Shu-Ching Chen TA: Hsin-Yu Ha Fall 2015

C – Multi-file development and make

Compilation and Debugging

Compilation and Debugging

SCMP Special Topic: Software Development Spring 2017 James Skon

Introduction to C Topics Compilation Using the gcc Compiler

Makefile Tutorial CIS5027 Prof: Dr. Shu-Ching Chen

What is make? make is a system utility for managing the build process (compilation/linking/etc). There are various versions of make; these notes discuss.

Prof: Dr. Shu-Ching Chen TA: Yimin Yang

Prof: Dr. Shu-Ching Chen TA: Samira Pouyanfar Hector Cen Fall 2017

Prof: Dr. Shu-Ching Chen TA: Hsin-Yu Ha

Makefiles and the make utility

Data Structures and Programming Techniques

SCMP Software Development Spring 2018 James Skon

Build Tools (make) CSE 333 Autumn 2018

Week 1 – Lesson 2: Creating Shell Scripts, Linux Commands

Cygwin: getting the setup tool

Makefiles and the make utility

SCMP Software Development Spring 2018 James Skon

What is make? make is a system utility for managing the build process (compilation/linking/etc). There are various versions of make; these notes discuss.

The make utility (original presentation courtesy of Alark Joshi)

Presentation transcript:

Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information. Automated Builds

Introduction A typical working day

Automated BuildsIntroduction A typical working day Re-draw Figure 8

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Recompile stats program

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Recompile stats program Update Java

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

Automated BuildsIntroduction A typical working day Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space...shave the yak...

Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task

Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task dependencies

Automated BuildsIntroduction Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space task dependencies Re-draw Figure 8 Recalculate data Re-install graph tool Recompile stats program Update Java Free up disk space

Automated BuildsIntroduction This pattern arises frequently

Automated BuildsIntroduction This pattern arises frequently New data collected? Recalculate statistics

Automated BuildsIntroduction This pattern arises frequently Source files changed? Recompile program New data collected? Recalculate statistics

Automated BuildsIntroduction This pattern arises frequently Source files changed? Recompile program New data collected? Recalculate statistics New content written? Update web site

Automated BuildsIntroduction Hard or impossible to keep track of:

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating."

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file Along with commands used to update things

Automated BuildsIntroduction Hard or impossible to keep track of: – what depends on what – what's up-to-date and what isn't "Anything worth repeating is worth automating." So use a build manager to automate the process Describe dependencies in a build file Along with commands used to update things Build manager does the rest

Automated BuildsIntroduction Most widely used build manager is Make

Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular"

Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975

Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language

Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language A very cryptic little language, without a debugger...

Automated BuildsIntroduction Most widely used build manager is Make Note: "most widely used", not "most popular" Invented by a student intern at Bell Labs in 1975 Has grown into a little programming language A very cryptic little language, without a debugger......that requires an understanding of the Unix shell

Automated BuildsIntroduction GNU Make is fast, free, and well-documented

Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it

Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features

Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features Companion lecture explores SCons

Automated BuildsIntroduction GNU Make is fast, free, and well-documented And many other tools know how to work with it Look at basics and a few advanced features Companion lecture explores SCons Java users should look at Ant

August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information.

Automated BuildsIntroduction Basics Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information. Automated Builds

Introduction Manage tasks and dependencies

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp sgr -N -r summary-1.dat \ > figure-1.svg

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py summary-1.dat data-1-*.dat sgr -N -r summary-1.dat \ > figure-1.svg

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py summary-1.dat \ data-1-1.dat data-1-2.dat \ data-1-3.dat stats.py sgr -N -r summary-1.dat \ > figure-1.svg

Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ Data file is newer than SVG image

Automated BuildsIntroduction 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction Comment 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $

Automated BuildsIntroduction Rule 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction Target 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction Prerequisite 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction Action 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction Must indent with a single tab character 1. What is newer than what? $ ls -t *.dat *.svg summary-1.dat figure-1.svg $ 2. Put this in a Makefile called hello.mk # hello.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk -f filename

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ Prerequisite is newer than target

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ Prerequisite is newer than target So Make executes the action

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $ Target is newer than prerequisite

Automated BuildsIntroduction 3. Run GNU Make from the shell $ gmake -f hello.mk sgr -N -r summary-1.dat > figure-1.svg $ 4. Run Make again $ gmake -f hello.mk $ Target is newer than prerequisite So action not executed

Automated BuildsIntroduction Usually have multiple rules per file

Automated BuildsIntroduction Usually have multiple rules per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

Automated BuildsIntroduction Usually have multiple rules per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $

Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ Update timestamps on files

Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ gmake -f double.mk sgr -N -r summary-1.dat > figure-1.svg $

Automated BuildsIntroduction Usually have multiple targets per file # double.mk figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg Force it to run $ touch *.dat $ gmake -f double.mk sgr -N -r summary-1.dat > figure-1.svg $ Why isn't figure-2.svg rebuilt?

Automated BuildsIntroduction First rule in Makefile is default rule

Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise

Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly

Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly $ gmake -f double.mk figure-2.svg sgr -N -r summary-2.dat > figure-2.svg $

Automated BuildsIntroduction First rule in Makefile is default rule Make only does this unless told otherwise Force it to rebuild figure-2.svg explicitly $ gmake -f double.mk figure-2.svg sgr -N -r summary-2.dat > figure-2.svg $ Better than typing commands one by one, but only slightly

Automated BuildsIntroduction Introduce a phony target

Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file

Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date

Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it

Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it # phony.mk all : figure-1.svg figure-2.svg figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

Automated BuildsIntroduction Introduce a phony target Doesn't correspond to a file So never up to date But things can depend on it # phony.mk all : figure-1.svg figure-2.svg figure-1.svg : summary-1.dat sgr -N -r summary-1.dat > figure-1.svg figure-2.svg : summary-2.dat sgr -N -r summary-2.dat > figure-2.svg

Automated BuildsIntroduction "make all" rebuilds everything

Automated BuildsIntroduction "make all" rebuilds everything $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $

Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed "make all" rebuilds everything

Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first "make all" rebuilds everything

Automated BuildsIntroduction "make all" rebuilds everything $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first Or re-create both in parallel

Automated BuildsIntroduction $ touch *.dat $ gmake -f phony.mk sgr -N -r summary-1.dat > figure-1.svg sgr -N -r summary-2.dat > figure-2.svg $ Order is not guaranteed Could re-create figure-2.svg first Or re-create both in parallel Return to that idea later "make all" rebuilds everything

Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat

Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph

Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic

Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic Y X Z

Automated BuildsIntroduction all Things can be both targets and prerequisites figure-1.svg summary-1.datsummary-2.dat A directed graph Must be acyclic Y X Z

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat How to generalize to any number of files?

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat How to generalize to any number of files? And get rid of repeated filenames

Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information.

Automated BuildsIntroduction Patterns Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information. Automated Builds

Introduction Manage tasks and dependencies

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # multiple.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable "the target of this rule"

Automated BuildsIntroduction summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat Automatic variable "the target of this rule" No, there isn't a more readable form

Automated BuildsIntroduction # target-variable.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat Still a lot of redundancy summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ All prerequisites of this rule summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ All prerequisites of this rule $< is "the first prerequisite" summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ All prerequisites of this rule $< is "the first prerequisite" $? is "all out-of-date prerequisites" summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-2.dat 1 3 Expect more data files

Automated BuildsIntroduction # variables.mk summary-1.dat : data-1-1.dat data-1-2.dat data-1-3.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Want to do this

Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Just like shell wildcard

Automated BuildsIntroduction # wildcard.mk summary-1.dat : data-1-*.dat stats.py $^ summary-1.dat data-1-1.datdata-1-3.datdata-1-2.dat stats.py summary-1.dat data-1-*.dat Just like shell wildcard Must use $^ in action, since filenames not fixed in advance

Automated BuildsIntroduction figure-1.svg summary-1.dat figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp wdp2pdf paper.wdp stats.py sgr -N -r summary-1.dat \ > figure-1.svg stats.py summary-1.dat data-1-*.dat The whole tree one more time

Automated BuildsIntroduction The makefile so far paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^

Automated BuildsIntroduction Still some redundancy paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ The makefile so far

Automated BuildsIntroduction Still some redundancy Fix in next episode paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ The makefile so far

Automated BuildsIntroduction Still some redundancy Fix in next episode paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ Doesn't handle summaries' dependency on stats.py The makefile so far

Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : stats.py data-1-*.dat stats.py $^ summary-2.dat : stats.py data-2-*.dat stats.py $^ Option 1: add to existing rules

Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $^ summary-2.dat : stats.py data-2-*.dat stats.py $^ ⋮ Option 1: add to existing rules $^ is now stats.py data-1-1.dat data-1-1.dat...

Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $^ summary-2.dat : stats.py data-2-*.dat stats.py $^ ⋮ $^ is now stats.py data-1-1.dat data-1-1.dat... So the invocation of stats.py is wrong Option 1: add to existing rules

Automated BuildsIntroduction ⋮ summary-1.dat : stats.py data-1-*.dat stats.py $^ summary-2.dat : stats.py data-2-*.dat stats.py $^ ⋮ $^ is now stats.py data-1-1.dat data-1-1.dat... So the invocation of stats.py is wrong Having it ignore one argument is an ugly hack Option 1: add to existing rules

Automated BuildsIntroduction figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ data-1-1.dat : stats.py touch data-1-2.dat : stats.py touch Option 2: make data files depend on stats.py

Automated BuildsIntroduction ⋮ data-1-1.dat : stats.py touch data-1-2.dat : stats.py touch ⋮ Option 2: make data files depend on stats.py A false dependency

Automated BuildsIntroduction Option 2: make data files depend on stats.py ⋮ data-1-1.dat : stats.py touch data-1-2.dat : stats.py touch ⋮ A false dependency Updating raw data files triggers update of summary

Automated BuildsIntroduction Option 2: make data files depend on stats.py ⋮ data-1-1.dat : stats.py touch data-1-2.dat : stats.py touch ⋮ A false dependency Updating raw data files triggers update of summary Back to listing all raw data files explicitly…

Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Option 3: add additional dependencies

Automated BuildsIntroduction ⋮ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py ⋮ Option 3: add additional dependencies Full set of dependencies is union of lists

Automated BuildsIntroduction ⋮ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py ⋮ Option 3: add additional dependencies Full set of dependencies is union of lists But $^ in the action is still just data-1-*.dat

Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information.

Automated BuildsIntroduction Rules Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information. Automated Builds

Introduction Manage tasks and dependencies

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp stats.py stats.py summary-1.dat data-1-*.dat ?

Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Makefile so far

Automated BuildsIntroduction paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-1.svg : summary-1.dat sgr -N -r $^ figure-2.svg : summary-2.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Makefile so far Eliminate this redundancy

Automated BuildsIntroduction Use a pattern rule to capture the common idea

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Use a pattern rule to capture the common idea

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Use a pattern rule to capture the common idea % is a wildcard

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Use a pattern rule to capture the common idea

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Undefined in the action Use a pattern rule to capture the common idea

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py % is a wildcard Has the same value on both sides Undefined in the action Have to use $^, etc. Use a pattern rule to capture the common idea

Automated BuildsIntroduction Try running it

Automated BuildsIntroduction $ make -f pattern-rule.mk stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat $ Try running it

Automated BuildsIntroduction $ make -f pattern-rule.mk stats.py summary-1.dat data-1-1.dat data-1-2.dat data-1-3.dat $ Try running it Why didn't other commands run?

Automated BuildsIntroduction Pattern rule doesn't create dependencies itself

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py If Make wants to create figure-1.svg, it can use this rule Pattern rule doesn't create dependencies itself

Automated BuildsIntroduction # pattern-rule.mk figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py If Make wants to create figure-1.svg, it can use this rule Still have to tell Make what it wants to do Pattern rule doesn't create dependencies itself

Automated BuildsIntroduction Put the rule for paper.pdf back in the file

Automated BuildsIntroduction # use-pattern.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Put the rule for paper.pdf back in the file

Automated BuildsIntroduction # use-pattern.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Make now knows that it needs to create the figures, so it finds and uses the rule Put the rule for paper.pdf back in the file

Automated BuildsIntroduction # doesnt-work.mk paper.pdf : paper.wdp figure-*.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py This doesn't work!

Automated BuildsIntroduction # doesnt-work.mk paper.pdf : paper.wdp figure-*.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Figures don’t exist when Make starts to run, so this is empty This doesn't work!

Automated BuildsIntroduction # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-1.dat : data-1-*.dat stats.py $^ summary-2.dat : data-2-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Get rid of more redundancy

Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py

Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Make wildcard

Automated BuildsIntroduction Get rid of more redundancy # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-1.dat : stats.py summary-2.dat : stats.py Shell wildcard

Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-%.dat : stats.py

Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-%.dat : stats.py summary-*.dat still only depends on data-*-*.dat

Automated BuildsIntroduction But this doesn't work! # all-patterns.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ summary-%.dat : stats.py summary-*.dat still only depends on data-*-*.dat Make only uses the first pattern rule it finds

Automated BuildsIntroduction Back to using false dependencies... # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch

Automated BuildsIntroduction Back to using false dependencies... # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch It's a less-than-perfect tool...

Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information.

Automated BuildsIntroduction Macros Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information. Automated Builds

Introduction Manage tasks and dependencies

Automated BuildsIntroduction figure-1.svg summary-1.dat Manage tasks and dependencies figure-2.svg data-1-1.datdata-1-3.datdata-1-2.dat paper.pdf paper.wdp

Automated BuildsIntroduction "must conform to university style"

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers home

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers/lib/styles/ homelab

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp data-1-1.datdata-1-2.dateuphoric.fig

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers home data-1-1.datdata-1-2.dateuphoric.fig

Automated BuildsIntroduction "must conform to university style" figure-1.svgfigure-2.svg paper.pdf euphoric.wpspaper.wdp C:\papers /lib/styles/ home lab data-1-1.datdata-1-2.dateuphoric.fig

Automated BuildsIntroduction # false-dependencies.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf $< figure-%.svg : summary-%.dat sgr -N -r $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Makefile so far

Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Add directories for working at home

Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Add directories for working at home

Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Usually don't list "system" files explicitly Add directories for working at home

Automated BuildsIntroduction # with-directories-at-home.mk paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style c:/papers/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s c:/papers/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Usually don't list "system" files explicitly But what about the lab? Add directories for working at home

Automated BuildsIntroduction 1. Write two Makefiles

Automated BuildsIntroduction 1. Write two Makefiles Write and maintain

Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain

Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain Consistently every time

Automated BuildsIntroduction 1.Write two Makefiles 2. Comment and uncomment lines Write and maintain Consistently every time Will create lots of noise in version control

Automated BuildsIntroduction 1.Write two Makefiles 2.Comment and uncomment lines 3.Refactor Write and maintain Consistently every time Will create lots of noise in version control

Automated BuildsIntroduction Use a macro

Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Use a macro

Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Use a macro

Automated BuildsIntroduction # with-macro.mk STYLE_DIR=c:/papers/ paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf --style ${STYLE_DIR}/euphoric.wps $< figure-%.svg : summary-%.dat sgr -N -r -s ${STYLE_DIR}/euphoric.fig $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Use a macro

Automated BuildsIntroduction Only have one thing to change

Automated BuildsIntroduction Only have one thing to change ✓ Consistency

Automated BuildsIntroduction Only have one thing to change ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" Which is probably not what you want ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Only have one thing to change Must use ${MACRO} or $(MACRO), not $MACRO Make reads $MACRO is ($M)ACRO Which is probably just "ACRO" Which is probably not what you want yet another legacy wart ✓ Consistency ✗ But still have noise

Automated BuildsIntroduction Commonly define macros for control flags

Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Commonly define macros for control flags

Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Commonly define macros for control flags

Automated BuildsIntroduction # with-lots-of-macros.mk STYLE_DIR=c:/papers/ WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch Commonly define macros for control flags

Automated BuildsIntroduction # config.mk STYLE_DIR=c:/papers/ Now put the first macro in a separate file

Automated BuildsIntroduction # with-include.mk include config.mk WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch And include it from the main file

Automated BuildsIntroduction # with-include.mk include config.mk WDP2PDF_FLAGS=--style ${STYLE_DIR}/euphoric.wps SGR_FLAGS=-N -r -s ${STYLE_DIR}/euphoric.fig paper.pdf : paper.wdp figure-1.svg figure-2.svg wdp2pdf ${WDP2PDF_FLAGS} $< figure-%.svg : summary-%.dat sgr ${SGR_FLAGS} $^ summary-%.dat : data-%-*.dat stats.py $^ data-*-*.dat : stats.py touch And include it from the main file

Automated BuildsIntroduction Actually create two configuration files

Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files

Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles

Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles These two files stay in version control

Automated BuildsIntroduction # config-home.mk STYLE_DIR=c:/papers/ Actually create two configuration files # config-lab.mk STYLE_DIR=/lib/styles These two files stay in version control Copy to create config.mk per machine

Automated BuildsIntroduction paper/ config-lab.mk config-home.mk Makefile Home

Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home

Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config-lab.mk config-home.mk Makefile Lab

Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config.mk config-lab.mk config-home.mk Makefile Lab

Automated BuildsIntroduction paper/ config.mk config-lab.mk config-home.mk Makefile Home paper/ config.mk config-lab.mk config-home.mk Makefile Lab

Automated BuildsIntroduction Can also define macro value on command line

Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line

Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea

Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time

Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time correctly

Automated BuildsIntroduction $ make -DSTYLE_DIR=/lib/styles -f Makefile Can also define macro value on command line Generally a bad idea Have to remember to type definition each time correctly And there's no record of the flag

Automated BuildsIntroduction Many other approaches

Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent)

Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines

Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug

Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug A build file is a program

Automated BuildsIntroduction Many other approaches CMake and Autoconf/Automake: compile higher-level specification into a Makefile (or equivalent) ✓ Automatically discover/manage differences between machines ✗ But even harder to debug A build file is a program Requires the same degree of respect

Automated BuildsIntroduction August 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See for more information.