A Universe-Type-Based Verification Technique for Mutable Static Fields and Methods Alexander J Summers Sophia Drossopoulou Imperial College London Peter Müller Microsoft Research Redmond

What is it all about? ►A new verification technique ►Extend Visibility Technique – handle static fields, methods and invariants ►Visible state semantics –safely handle/restrict call-backs ►Multiple-object invariants ►Global data structures –mutable static fields and static methods ►Expressive invariants: quantification over instances ►Minimal code annotations using Universe Types

Heap topology

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper ►All object references must respect ownership

Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper ►All object references must respect ownership

Framework Paper (ECOOP 2008) ►Identifies 7 parameters to describe a technique X invariants expected at visible states of a method V invariants vulnerable to execution of a method D invariants which may depend on a particular field B invariants which must be proven before a method call E invariants which must be proven at end of a method call U permitted receivers for field updates (who updates fields) C permitted receivers for method calls (who calls who) ►Identifies 5 sufficient conditions, in terms of these –e.g., before a permitted method call, all invariants expected by the new method which are not currently known to hold, must be proven

Visible State Semantics ►Invariants need only hold at ‘visible states’ –beginning of a method call –end of a method call –may be temporarily broken in between ►Flexible, but must handle call-backs with care –avoid control returning to an object temporarily broken ►This problem can be avoided by: –avoiding ‘loops’ in sequences of calls, or, –requiring invariants to be proven before such calls ►Ensure expected invariants hold for a new receiver

Visibility Technique (Müller et al.)

►Calls are restricted

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist –extra proof obligations

Static fields and methods ►Static fields can control/refine instantiation of class –Restricting/counting number of instances (Singleton) –Maintaining invariants across all instances Instances of Thread are assigned unique identifiers –Sharing data across all instances String can maintain a ‘pool’ of shared instances for use ►Static fields are internal ‘representation’ of the class –Motivates static rep fields –Objects owned by classes (or objects) –Classes do not have owners themselves ►Static methods are methods of the class –Treat classes as potential receivers, like objects

Heap topology

►Classes in the topology

Heap topology ►Classes in the topology

Heap topology ►Classes in the topology –Classes may be owners

Heap topology ►Classes in the topology –Classes may be owners Multiple trees

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods?

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods? ?

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods?

Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods? ?

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs ►Idea to avoid call-backs: –A static method can only be called if the class is not a prior receiver –depends on call-stack –approximate with effects

Effect annotations ►Ensure that a class cannot receive a call-back ►Annotate methods with a set effects(c,m) –Our effects sets are sets of class-names –Which classes might have static methods called on them, as a result of executing method m of class c? –Predict this set of classes (conservatively) ►Static method of c is legal only if c is not in effects ►Effects sets can be computed iteratively ►Further restriction necessary: –An overridden method may not have any extra effects –Ensures effects conservatively predict runtime calls

Soundness (outline) ►If the invariants of an object (or class) do not hold, then either it or one of its peers must be a receiver somewhere on the call-stack. ►If an object o is a receiver on the call-stack, the most recently-preceding class receiver to o is the ‘root’ of the tree in which o resides. ►Effects are conservative: if a static method of c is called, c was in effects of all methods on the stack ►Call-backs to classes are restricted to self-calls. ►Call-backs to objects are restricted to peer calls. ►Proof obligations imposed are sufficient

Problematic example Class MyClass extends Object { boolean equals(Object o) { System.out.println(new String(“equals”)); return (o == this); }

Problematic example Class MyClass extends Object { boolean equals(Object o) { System.out.println(new String(“equals”)); return (o == this); } } ►String must be in the effects of MyClass::equals() ►String must be in the effects of Object::equals() ►Annotation overhead problem ►Information-hiding problem ►Practicality problem

Levels

►Divide forest into (totally ordered) levels

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes.

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs –Effects only computed for one level at a time

Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs –Effects only computed for one level at a time ►MyClass is now legal

Soundness (outline) ►If c ≥ c’ then level (c) ≥ level (c’) ►If an object o is transitively owned by c, and c’ is the dynamic class of o, then level (c)≥ level (c’) ►If a sequence of legal calls can be made starting from receiver r and ending with receiver r’, then level (r) ≥ level (r’) ►If a sequence of legal calls starts and ends with receiver r, then for all intermediate receivers r’, level (r’) = level (r) ►Effects sets for one level are enough to guarantee no call-backs to that level

Finally.. ►We refine static invariants to allow quantification over instances –e.g., all Thread instances have distinct identifiers ►Use ECOOP paper to calculate necessary changes –Satisfy the 5 soundness conditions presented there –These imply sufficient changes to the 7 parameters ►Note: we allow static invariants to quantify over instances, whereas JML allows instance invariants to mention static fields –Similar expressiveness in logical terms –Different visible state semantics (future work)

Related work ►JML supports some statics and Universe Types –Limited support for both in combination –No static rep or peer fields: only readonly ►Leino and Müller extend Boogie technique to statics –supports static rep fields –refine work with a class ordering –restrictions on static initialisation ►Jacobs et. al. present modular verification for multithreaded programs in the context of Spec# –partially order locks, to avoid deadlock –similar to our levels, but without flexibility of effects

Possible future work ►Practicality: examples, case studies ►Static initialisation –Trees ‘appear’ in the topology –Rules for verifying static initialisers ►Formal definitions and proofs ►Extend framework (ECOOP) to cover our technique –formal proofs ‘for free’ ►Cover static factory methods –Ownership Transfer ►Considering ‘levels’ for other problems ►Path types to allow further flexibility

Any questions? Thank you for listening!