Joint work with Mike Barnett, Robert DeLine, Manuel Fahndrich, and Wolfram Schulte Verifying invariants in object-oriented programs K. Rustan M. Leino Microsoft Research, Redmond, WA, USA Computer Science Colloquium ETH Zurich 24 Nov 2003

Vision Record design decisions +Utilize automatic checking =Detect errors and improve maintainability

A# and Boogie: Programmer's view Boogie C# compiler A# compiler A# C# MSIL Warnings Other.NET compilers

Boogie: under the hood Theorem prover weakest-precondition generator translator MSIL BoogiePL verification condition Warnings Inference engine

Boogie technology and research Programming methodology – model for writing code and specifications Inference – abstract domains over heap structures Verification-condition generation – formulas that are efficient for theorem prover

Invariants: straw man class T { // field declarations... invariant J ; T(...) requires P modifies v ensures Q { … } method m(...) requires R modifies w ensures T { … } … class T { // field declarations... T(...) requires P modifies v ensures Q J { … } method m(...) requires R J modifies w ensures T J { … } … Object invariants hold on public method boundaries and are shorthands for pre/post-conditions

Invariants, example class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } …

Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … callers are expected to establish property about internal data structure

Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … possible solution (?): callers dont need to be checked for this precondition it holds automatically!

Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … invariant does not hold here

Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; p(...) ; y = 4 * y ; } … invariant does not hold here, so what if p calls m?!

The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies hasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies foodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } these lists are long, and they mention private state

The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies hasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies foodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } possible solution (?): dont need to declare modifications of private state it cant be observed anyhow

The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies { d.Clean() ; hasGarbage = false ; s.Wipe() ; assert hasGarbage ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } SpiffUp treats Wipe as if Wipe modified nothing, so what if Wipe calls a method in Kitchen that sets hasGarbage to true?!

Soundness of verification Soundness = verification finds all errors Soundness follows from: – pre- and postconditions are the same for caller and callee Note: In addition to soundness, we want something usable

Methodology object invariant declaration – class T { int x, y ; invariant x < y ; special variable st: {Invalid, Valid} Idea: program invariant (o o.st = Invalid Inv(o)) st is changed by commands pack and unpack holds at every program point! for any o: T, we write Inv(o) o.x < o.y

pack and unpack pack o assert o.st = Invalid ; assert Inv(o) ; o.st := Valid unpack o assert o.st = Valid ; o.st := Invalid

Example class T { int x, y ; invariant 0 x < y ; method init(t) requires t.st = Invalid modifies t.st, t.x, t.y ensures t.st = Valid { t.x := 0 ; t.y := 1 ; pack t } method m(t) requires t.st = Valid modifies t.x, t.y { unpack t ; t.x := t.x + 3 ; t.y := 4 * t.y ; pack t } receiver parameter (this, self, current)

Program invariant (o o.st = Invalid Inv(o)) x := new(T)... ; assume x.st = Invalid pack o... ; assert Inv(o) ; o.st := Valid unpack o... ; o.st := Invalid o.f := E assert o.st = Invalid ;... Inv(o) can mention only the fields of o

Methodology, summary invariant... st: {Invalid, Valid} pack, unpack modifications of o.f require o.st=Invalid Inv(o) can mention only the fields of o (o o.st = Invalid Inv(o))

Methodology, extended component declarations – class Kitchen { component Stove s ; st: {Invalid, Valid, Committed} Idea: program invariant (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed))) pack o and unpack o change st for o and o's components for any k: Kitchen, we have k.s Comp(k)

pack and unpack, extended pack o assert o.st = Invalid Inv(o) ; assert (p Comp(o) p.st=Valid) ; o.st := Valid ; foreach p Comp(o) do p.st :=Committed ; unpack o assert o.st = Valid ; foreach p Comp(o) do p.st=Valid ; o.st := Invalid ;

Example class Kitchen { method SpiffUp(k) requires k.st=Valid … { unpack k ; k.d.Clean() ; k.s.Wipe() ; pack k } class Stove { method Wipe(s) requires s.st=Valid … s d Kitchen k Stove Dishes Valid Committed

Valid Invalid Example class Kitchen { method SpiffUp(k) requires k.st=Valid … { unpack k ; k.d.Clean() ; k.s.Wipe() ; pack k } class Stove { method Wipe(s) requires s.st=Valid … s d Kitchen k Stove Dishes Committed Valid

Program invariant (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed))) x := new(T)... ; assume x.st=Invalid pack o, unpack o o.f := E assert o.st=Invalid ;... Inv(o) can mention only the fields of o and of o.p for any component field p

Extended methodology, summary invariant... component... st: {Invalid, Valid, Committed} pack, unpack modifications of o.f require o.st=Invalid Inv(o) can mention only the fields of o and of o.p for any component field p (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed)))

Verification system We let st be used in method specifications (requires, modifies, ensures) We must address the Problem of Specifying Modifications

A heap model Heap is a two-dimensional array class T { f: U; g: V;... } x := o.f= x := Heap[o, f] o.f := E= Heap[o, f] := E

Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f] (o,f) w 0 ) viewed as set of object/field-name pairs

Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f](o,f) w 0Heap 0 [o,alloc] )

Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f](o,f) w 0Heap 0 [o,alloc]Heap 0 [o,st]=Committed )

Example class Kitchen { method SpiffUp(k) requires k.st=Valid modifies k.hasGarbage { unpack k ; k.d.Clean() ; k.hasGarbage := false ; k.s.Wipe() ; assert k.hasGarbage ; pack k } class Stove { method Wipe(s) requires s.st=Valid modifies s.foodPieces

Another example method m(p) requires p.st=Committed { var y, z in y := p.x ; z := sqrt(49) ; assert y = p.x end }

Soundness Pre- and postconditions are the same for callers and callees, so verification system is sound!

Related work rep types in CLU dynamically checked invariants in Eiffel valid idiom in ESC/Modula-3 universe types and invariants in Muller's thesis invariant declarations in ESC/Java and JML (implicit) pack/unpack operations in Vault and Fugue capability calculus ownership types locking and monitor disciplines in concurrent programming

Conclusions Invariants different from pre/post-conditions Resulting program invariants hold at every program point Uses pack/unpack commands, but good defaults can be constructed for these No linear type system Components are not unique references objects (pointers) can freely be copied Fields can freely be read No additional features of abstraction needed to support the specification of modifications Sound

Further research challenges experience, understanding of limits extensions to support more good programs (joint work with Peter Muller)