Safeness of Make-Based Incremental Recompilation Niels Jørgensen Roskilde University, Denmark

Plan of talk What is make ? The safeness problem The result ● Criteria for rules: completeness, fairness, and soundness ● Editing constraints ● Applications The formal model & related work Questions for further work

What is make ? pgm library.a codegen.oparser.o parser.c parser.ydefinitions.hcodegen.c The purpose of make is to support incremental recompilation ● only dependent-on files are recompiled ● "cascading" recompilation (along chains of dependency) ● crude approach, change of comment spurs recompilation

What is make ? make = the make program ● reads and executes makefiles make can be defined by ● the syntax of makefiles ● the semantics of their execution make is widely used in large software projects ● syntax and semantics is as in late 70s

What is make : syntax of makefiles Makefile = ● makefile rules ● plus more which is omitted from model (macros, directives..) Syntax of a makefile rule: G : D 1 D 2.. D n C list of dependency targets command derived target

What is make : Semantics of makefile execution G : D 1 D 2.. D n C If file G is older than one or more of the files D 1 D 2.. D n (i.e. rule is unsatisfied) then execute command C (i.e. rule "fires") Bottom-up order ● a rule is evaluated after the rules defining its dependencies

The basic problem: when is make 's optimization safe ? "safeness of make -based incremental recompilation" = Assume ● there is an initial brute-force compilation Then, after editing, ● make -based incremental recompilation is "as good as" a (second) brute-force compilation. Given the semantics of make, what criteria apply to ● formulation of makefile rules (cf. Mozilla) ? ● editing of source files, and makefile ● generically, independent of C Not addressed previously

The basic problem: Software developers do not trust make For example, the Mozilla browser is built twice - incrementally and brute-force: ".. to make sure our dependencies are right, and out of sheer paranoia.. " ( May )

The result: rules of thumb for the makefile programmer Build rules must be: Complete: ● The rule's dependency list D 1 D 2.. D n must contain all the files that the rule's target G depends on Fair: ● Executing the rule's command C may not create or update any target other than the rule's own Sound: ● Upon executing C, the rule must be satisfied, and its parent rules must be unsatisfied G : D 1 D 2.. D n C

The result: criteria for editing Incremental recompilation is desirable not only after editing of source files but also when the makefile is modified (although rare) Deletion of a makefile rule is safe ● always Addition of a makefile rule is safe ● if the new rule's target does not exist Modifying a rule if safe ● cf. deletion + addition ● or if change is restricted to dependency list Any combination is permitted

The result: example pgm library.a codegen.o parser.o parser.c parser.ydefinitions.hcodegen.c functions.o functions.c Modification: ● part of codegen.c moved to new file functions.c ● add rule defining functions.o + modify rule defining pgm

The result: precise criteria for makefiles For example, completeness: Rule R is complete in state S iff for any state S' equivalent to S on (D 1 D 2.. D n ): (exec C S' G) is equivalent to (exec C S G) where (roughly) ● a state is the current state of file system ● equivalence is defined in terms of file contents, not time stamps Completeness is with respect to a given state ● verification that a makefile for e.g. a C program is correct requires knowledge of the source files ● i.e., all recursively included header files G : D 1 D 2.. D n C R =

Formal model & related work: level of abstraction Formal make -model abstracts away: ● any knowledge of the effect of execution of rule commands ● details of satisfiability (e.g., t < t' or t =< t') Related work on incremental recompilation requires knowledge of language under compilation ● smart recompilation (Pascal) (Tichy, 86) ● variant of s.r. in ML Kit with Regions compiler (Elsman, 86) ● syntax directed editors (Deters + Reps + Teitelbaum, 81) Such language-specific approach is more fine-grained ● e.g., modification of comments do not lead to recompilation

Formal model & related work: semantics of make Semantics of make resembles (propositional) logic programming ● query driven ● no global variables Denotational definition of make resembles definition given for (constraint) logic programs (Marriott + Søndergaard, 88)

Formal model: anatomy of proofs Goal is to prove equivalence of incremental and brute-force compilation, i.e. of various ● sequences of command executions ● interleaved with editing Under appropriate conditions, ● commands are idempotent: (exec C) equiv. (exec C; C) ● commands commute: (exec C C') equiv. (exec C' C) ● this is combined to show equivalence of complex sequences Editing is required to preserve "partial safeness" ● for any rule, the files are unedited or else the rule must fire

Conclusion and future work Criteria for safeness have been established, provide basis for ● guidelines for makefile programmers ● verification of makefile tools Practical benefit: ● slightly improved use, of widely used tool Future work on the make model ● can (part of) the formal model be seen as an instance of an existing formalism ? ● is it possible to weaken the criteria for safeness, e.g. the definition of rule completeness ? ● error handling: upon abort of make due to compilation failure, is incremental recompilation safe ?