1. Rigorous Software Development CSCI-GA 3033-011 Instructor: Thomas Wies Spring 2012 Lecture 9

2. Class Invariants Class invariants are properties that must hold at the entry and exit point of every method They often express properties about the consistency of the internal representation of an object. They are typically transparent to clients of an object. They are sometimes also called object invariants or instance invariants.

3. The Problem with Class Invariants There are some problems with class invariants: Ownership: invariants can depend on fields of other objects. – For example, the invariant of List accesses Node fields. Callback: invariants can be temporarily violated. – While the invariant is violated, we call a different method that calls back to the same object. Atomicity: invariants can be temporarily violated. – While the invariant is violated, another thread accesses object.

4. Invariants May Depend on Other Objects class Node { Node prev, next; } class List { private Node first; /*@ private ghost JMLObjectSet nodes; @*/ /*@ invariant (\forall Node n; nodes.has(n); n.prev.next == n && n.next.prev == n); @*/ public void add() { Node newnode = new Node(); newnode.prev = first.prev; newnode.next = first; first.prev.next = newnode; first.prev = newnode; //@ set nodes = nodes.insert(newnode); }

5. Invariants May Depend on Other Objects List Node first Node next prev Node next prev Node next prev next List Node first Node next prev n.next = val Invariants must be protected from unsolicitous updates of dependent fields.

6. The Owner-As-Modifier Property JML supports a type system for checking the owner-as-modifier property, when invoked as jmlc --universes. The underlying type system is called Universes: The class Object has a ghost field owner. Fields can be declared as rep, peer, readonly. – rep Object x adds an implicit invariant (or requires) x.owner = this. – peer Object x adds an implicit invariant (or requires) x.owner = this.owner. – readonly Object x does not restrict owner, but does not allow modifications of x. The new operation supports rep and peer : – new /*@rep@*/Node() sets owner field of new node to this. – new /*@peer@*/Node() sets owner field of new node to this.owner.

7. List with Universes Type System class Node { /*@ peer @*/ Node prev, next; } class List { private /*@ rep @*/ Node first; /*@ private ghost JMLObjectSet nodes; @*/ /*@ invariant (\forall Node n; nodes.has(n); n.prev.next == n && n.next.prev == n && n.owner == this); @*/ public void add() { Node newnode = new /*@ rep @*/ Node(); newnode.prev = first.prev; newnode.next = first; first.prev.next = newnode; first.prev = newnode; //@ set nodes = nodes.insert(newnode); }

8. Modification only by Owner All write accesses to a field of an object obj are in a method of the owner of obj or in a method of an object having the same owner as the object that was invoked (directly or indirectly) by the owner of obj. Invariants that only depend on fields of owned objects can only be invalidated by the owner or methods that the owner invokes.

9. Modification only by Owner List Node first Node next prev Node next prev Node next prev next List Node first Node next prev n.next = val peer rep peer rep Universes type system ensures that all write accesses of dependent fields go through the owner.

10. Limitations of Universes Type System The Universes type system can solve many ownership related problems. but It’s granularity is often too coarse. What happens if invariants are temporarily violated?

11. Temporarily Violating Invariants public class Container { int[] content; int size; /*@ invariant 0 <= size && size <= content.length; @*/ public void add(int v) { /* 1 */ size++; /* 2 */ if (size > content.length) { newContent = new int[2*size+1];... content = newContent; }... /* 3 */ } When do Invariants Hold? Before a public method is called. /* 1 */ After a public method returns. /* 3 */ However, it may be violated in between. /* 2 */

12. Calls to Private Methods public class Container { int[] content; int size; /*@ invariant 0 <= size && size <= content.length; @*/ private /*@ helper @*/ void growContent() {... content = newContent; } public void add(int v) { /* invariant should hold */ size++; /* invariant may be violated */ if (size > content.length) growContent();... /* invariant should hold, again */ } Sometimes an invariant may not hold before a private method call. JML provides the annotation /*@ helper @*/ for this.

13. Calls to Methods of Other Classes public class Container { int[] content; int size; /*@ invariant 0 <= size && size <= content.length; @*/ private /*@helper*/ void growContent() { /* invariant may be violated */ newContent = new int[2*size+1]; System.arraycopy(content, 0, newContent, 0, content.length); content = newContent; }... } The invariant still needs not to hold, when other methods are called, because there is the callback problem.

14. The Callback Problem public class Log { public void log(String p) { logfile.write("Log: “ + p + " list is “ + Global.theList); } public class Container { int[] content; int size; /*@ invariant 0 <= size && size <= content.length; @*/ public void add(int v) { /* invariant should hold */ size++; /* invariant may be violated */ if (size > content.length) { Logger.log("growing array.");... } public String toString() { /* invariant should hold */... } implicit call to method toString

15. The Callback Problem A method of a different class can be called while an invariant is violated. This method may call a method of the first class. Who has to ensure that the invariant holds? – jmlrac complains that the invariant does not hold. – ESC/Java checks that most invariants hold at every method call. – But not in every case; this may lead to unsoundness.

16. Adding Ghost Fields to Guard Invariants Idea of David A. Naumann and Mike Barnett: Make the places where an invariant does not hold explicit. Add a ghost field packed that indicates wether the invariant should hold. Before modifying an object set this field to false. When modification is complete, set it back to true. The following invariant should always hold: packed ==> invariants of object The caller has to ensure that all objects he uses are packed.

17. The pack/unpack Mechanism //@ public ghost boolean packed; //@ private invariant packed ==> (size >= 0 && size <= content.length); /*@ requires packed; @ ensures packed; @*/ public void add(int v) { unpack this; size++;... pack this; } The pre- and post-conditions explicitly state that invariant holds. unpack this is an abbreviation for: assert this.packed; set this.packed = false; pack this is an abbrevation for: assert !this.packed; assert /*invariant of this holds*/; set this.packed = true;

18. The pack/unpack Mechanism An object must be unpacked before its fields may be modified. The invariant has to hold only while object is packed. The invariant may only depend on fields of the object. object unpack pack packed == true packed == false invariants hold invariants may be broken object.f = val

19. Static Checking with pack/unpack Static Checking with packed ghost field: Fields may only be modified if packed is false. For each pack operation check that invariants hold again. Thus packed ==> invariants holds for all states.

20. Example: Tree Data Structure class TreeNode { int key, value; TreeNode left, right; //@ invariant left != null ==> left.key <= key &&...; //@ invariant right != null ==> right.key >= key &&...; public void add(int k, int v) { if (k < key) { if (left == null) left = new TreeNode(k, v); else left.add(n); } else... }

21. Adding the packed Ghost Field class TreeNode { int key, value; TreeNode left, right; //@ public ghost boolean packed = false; //@ invariant packed ==> left != null ==> left.key <= key &&...; //@ invariant packed ==> right != null ==> right.key >= key &&...; //@ requires packed; //@ ensures packed; public void add(int k, int v) { // unpack this if (k < key) {... } else... // pack this }

22. Runing ESC/Java > escj -q TreeNode.java TreeNode.java:40: Warning: Precondition possibly not established (Pre) left.add(n); ^ Associated declaration is "TreeNode.java", line 20, col 8: //@ requires packed; The nodes left and right must also be packed !

23. Adding the Additional Invariants class TreeNode { int key, value; TreeNode left, right; //@ public ghost boolean packed = false; /*@ invariant packed ==> left != null ==> @ left.packed && left.key <= key &&...; @*/ /*@ invariant packed ==> (right != null ==> @ right.packed && right.key >= key &&...; @*/ //@ requires packed; //@ ensures packed; public void add(int v, int k) {... }

24. Adding Ownership There are still problems: The invariants also depend on fields of left and right. In particular the fields left.key and left.packed. – Can unpack this violate the invariant of another TreeNode ? – Can insertion of a new node break the sortedness of the overall tree? Solution: Use the ownership principle.

25. Ownership and pack/unpack The owner must be unpacked before an owned object can be unpacked. The invariant of owner may depend on owned objects. owner unpack owner owner.packed == true left.pacled == true owner.f = val left.f = val left right left right owner left right pack owner owner.packed == false left.pacled == true owner.packed == false left.pacled == false left.f = val unpack left pack left

26. Ownership and pack/unpack How does pack/unpack work with ownership? To modify an object, you must unpack it first. To unpack an object, you must first unpack the owner. To pack the owner again, its invariants must hold. unpack obj is an abbreviation for: assert obj.packed; assert obj.owner == null || !obj.owner.packed; set obj.packed = false; pack obj ensures that its owned classes are packed. assert !obj.packed; assert left != null ==> (left.owner == this && left.packed); assert right != null ==> (right.owner == this && right.packed); assert /* other invariants of obj hold*/; set obj.packed = true;

27. Adding Ownership class TreeNode { int key, value; TreeNode left, right; //@ public ghost boolean packed = false; /*@ invariant packed ==> (left != null ==> @ left.owner == this && left.packed && left.key <= key); @*/ /*@ invariant packed ==> (right != null ==> @ right.owner == this && right.packed && right.key >= key); @*/ //@ requires packed; //@ requires owner == null || !owner.packed; //@ ensures packed; public void add(int k, int v) {... } }

28. Limitations of Ownership Discipline The ownership discipline has a few restrictions. An object invariant can only depend on fields of (transitively) owned objects. An object can have at most one owner. A field may only be changed by the owner, or if the owner is unpacked. Sometimes this is too restrictive!

29. Friendship Friendship is an orthogonal discipline for dealing with invariants that depend on other objects: Objects can be friends and granters. An invariant of a friend can depend on the fields of a granter. The friend must define update guards for the fields of the granter it depends on. The granter has a list of friends that depend on its fields. A field of a granter may be changed only if the update guards of all friends holds.

30. Friendship Friendship is not symmetric. The allies are: Granter G that gives the right to depend on one of its fields. class G { int f; //@ friend C reads f; } Friend C whose invariant depends on a field. class C { //@ invariant packed ==>... g.deps.has(this) && @ g.f ==... @ guard g.f := val by...; @*/ } Every class that changes a field f of G has to check the guards of all of G ’s friends that depend on f.

31. Update Guards and Invariants class FriendClass { //@ invariant packed ==> friendInvariant(granter.field) //@ guard granter.field := val by @ updateGuardForField(granter, val); @*/ } The update guard must guarantee that the invariant is not invalidated: friends.packed && friendInvariant(granter.field) && updateGuardForField(granter, val) ==> friendInvariant(val)

32. Updating Granted Fields A friend's invariant can depend on granted fields. Updates of granted fields are checked against update guards. A granter can have many friends. Update guards of all current friends must be checked. The friend objects can be packed or unpacked. Update guard is not checked for unpacked friends. friend1granterfriend2 deps granter.f = val guard

33. Friendship Example: HashMap class Object { /*@ spec_public @*/ int hashCode; //@ friend Map reads hashCode; //@ ghost JMLObjectSet deps; } class Map { JMLObjectSet buckets[]; /*@ invariant (\forall int i ; 0 <= i && i < buckets.length; @ (\forall Object o; buckets[i].has(o); o.deps.has(this) && @ Math.abs(o.hashCode % buckets.length) == i)); @*/ /*@ guard obj.hashCode := val by @ val % buckets.length == obj.hashCode % buckets.length; @*/ }

34. static class Node { Node next, prev; Object value; //@ friend Node reads next, prev, deps //@ guard obj.next = val by obj != prev; //@ guard obj.prev = val by obj != next; /*@ invariant packed ==> next != null && prev != null && deps.equals(new JMLObjectSet().insert(next).insert(prev)) && next.deps.has(this) && next.prev == this && prev.deps.has(this) && prev.next == this); */ } Friendship Example: Doubly-Linked Lists

35. static class Node {... //@ requires n.prev == n.next == n; public void addBefore(/*@non_null*/ Node n) { //@ unpack n; //@ unpack this; //@ unpack this.prev; n.prev = this.prev; n.next = this; this.prev.next = n; this.prev = n; /*@ set n.deps = @ new JMLObjectSet().insert(this).insert(this.prev); @*/ //@ set this.deps = this.deps.remove(prev).add(n); //@ set prev.deps = prev.deps.remove(this).add(n); //@ pack this.prev; //@ pack this; //@ pack n; }

36. What May Appear in an Invariant? Only the following field accesses are allowed in an invariant: A field of the current class: this.field for all fields. A field of a (transitively) owned class: x.field if x.owner...owner == this can be proved. A field of a granter class: x.field if x.deps.has(this) can be proved.

37. Why Is this Discipline Sound? We need to show that the following invariant holds at all times for each instance this : this.packed ==> this.invariant A field update obj.f = val can change the truth of the invariant if: obj == this is the current instance: Then this is unpacked, formula holds trivially. obj.owner...owner == this ( obj is an owned instance of this ): Then obj is unpacked, hence this must also be unpacked. The formula holds trivially. obj.deps.has(this) ( f is a field of a granter of this ): Then the update guard this.guard(f, val) is true. If this.packed is true, the invariant held before. Hence it must hold afterwards.