Stateful Manifest Contracts Taro Sekiyama IBM Research – Tokyo Atsushi Igarashi Kyoto University 11/11/2018 POPL'17

Contract Executable (run-time checkable) specification Expressed by Boolean predicates written in the same language as program E.g., positive numbers: Pos = { x:int | x > 0 } Extended to higher-order functional language E.g., functions over positives: Pos → Pos 11/11/2018 POPL'17

Example: functional Set of positives Pos ≡ { x:int | x > 0 } type set val create : unit → set val mem : set → Pos → Bool val add : set → Pos → set let s = create () let x = 1 if mem s x then add s (x-1) x is positive? As a more practical example, let me show a functional set module of positive numbers. This ML-like module provides three functions: creation of a set, membership check, and addition to a set. To represent members in a set are positive, I use Pos in this example. The program below firstly creates a empty set and checks x is a member of set s, and, if so, x-1 is added to s From the interface of set module, there are two contract checks. The first is that x is positive and the second is x-1 is positive. To check these contracts, three approaches have been studied so far. x-1 is positive? 11/11/2018 POPL'17

Static verification let s = create () let x = 1 if mem s x then Pos ≡ { x:int | x > 0 } type set val create : unit → set val mem : set → Pos → Bool val add : set → Pos → set let s = create () let x = 1 if mem s x then add s (x-1) Statically checked The first is static verification. With this, both contracts are checked statically. Statically checked 11/11/2018 POPL'17

Dynamic verification let s = create () let x = 1 if mem s x then Pos ≡ { x:int | x > 0 } type set val create : unit → set val mem : set → Pos → Bool val add : set → Pos → set let s = create () let x = 1 if mem s x then add s (x-1) Dynamically checked The second is dynamic verification. In this approach, they are checked at run time. Dynamically checked 11/11/2018 POPL'17

Hybrid verification let s = create () let x = 1 if mem s x then Pos ≡ { x:int | x > 0 } type set val create : unit → set val mem : set → Pos → Bool val add : set → Pos → set let s = create () let x = 1 if mem s x then add s (x-1) Statically checked The third approach is hybrid verification, which combines static and dynamic verification. For example, suppose that we can check the first contract but cannot check the second statically. Then, hybrid verification allows us to check the second at run time Dynamically checked 11/11/2018 POPL'17

Our work Goal: Hybrid verification for stateful programs with imperative features Goal: Hybrid verification for stateful programs Example: imperative Set type set … val create : unit → set val remove : set → int → unit Spec: returns the empty set Spec: Called in a state that given set contains given integer Ensures given integer is removed 11/11/2018 POPL'17

Lambda calculus where contracts are embedded into static types Theoretical foundation of hybrid checking Approach Manifest contract calculus [F, POPL’06, G+, ESOP’10] with mutable states Design a sound calculus by: Extending type language with Hoare types [N+, ICFP’06] for state-depend. contracts Type system tracks what state-depend. contracts may be invalidated Distinguishing state-dependent/state-independent contracts with effect system 11/11/2018 POPL'17

Outline Introduction Background: manifest contract calculus Challenge to extend with mutable states Our work 11/11/2018 POPL'17

Outline Introduction Background: manifest contract calculus Challenge to extend with mutable states Our work 11/11/2018 POPL'17

Manifest contract calculus Lambda calculus + types expressing contracts + dynamic check Refinement types Given to terms satisfying contract e Ex: ✔ 2 : {x:int|x>0} ✘ -1 : {x:int|x>0} Term e ::= λx.e | e1 e2 | ... | ref e | !e | e1 := e2 Type T ::= Bool | { x:T | e } | x:T1→T2 | Ref T 11/11/2018 POPL'17

Manifest contract calculus Lambda calculus + types expressing contracts + dynamic check Dependent function type T2 can refer to arguments by x Ex: succ : x:int → { y:int | y = x+1 } Term e ::= λx.e | e1 e2 | ... | ref e | !e | e1 := e2 Type T ::= Bool | { x:T | e } | x:T1→T2 | Ref T 11/11/2018 POPL'17

A naïve extension with mutable states Lambda calculus + types expressing contracts + dynamic check + mutable references Problem: this extension is UNSOUND Term e ::= λx.e | e1 e2 | ... | ref e | !e | e1 := e2 Type T ::= Bool | { x:T | e } | x:T1→T2 | Ref T 11/11/2018 POPL'17

Problematic example: imperative Set type set val is_empty : set → Bool val create : unit → { s:set | is_empty s } val add : set → int → unit let s : { s:set | is_empty s } = create () let () = add s 1 s s : { s:set | is_empty s } is_empty s false ⟹ UNSOUND 11/11/2018 POPL'17

Explanation of the unsoundness State-dependent contracts may be invalidated by assignment afterward E.g., contract { s:set | is_empty s } is invalidated when a value is added to the set Solution: Hoare type to detect what state-depend. contracts may be invalidated 11/11/2018 POPL'17

Outline Introduction Background: manifest contract calculus Challenge to extend with mutable states Our work 11/11/2018 POPL'17

Denote resulting values in e2 Hoare type Denote resulting values in e2 Type of result values Precondition {e1} x:T {e2} Postcondition Types for state-dependent contracts Inspired by Hoare type theory [N+, ICFP’06] Given to terms which: Expect e1 holds before executing them Produce values of type T Guarantee the final state and resulting value satisfy e2 11/11/2018 POPL'17

Example: imperative Set with Hoare type Nothing is assumed Returns the empty set type set val is_empty : set → Bool val create : unit → { true } s:set { is_empty s } val mem : set → int → Bool val remove : s:set → i:int → { mem s i } x:unit { not (mem s i) } Let me a program example using Hoare types. In this program, create function is given a Hoare type, where the precondition means create function assumes nothing and the postcondition means it returns the empty set Nest example is remove function. The Hoare type given to remove means that it assumes given set contains given integeer and after the call it ensures given integer is removed from the given set. Given integer should be contained Given integer is removed 11/11/2018 POPL'17

Issues in Hoare types How to detect contracts may be invalidated How to check contracts in Hoare types dynamically How to restrict contracts Contracts in refinement types are pure Contracts in Hoare types are read-only 11/11/2018 POPL'17

Issues in Hoare types How to detect contracts may be invalidated How to check contracts in Hoare types dynamically How to restrict contracts Contracts in refinement types are pure Contracts in Hoare types are read-only 11/11/2018 POPL'17

Key typing rule: assignment Detect contract e may be invalidated e1 : ref T e2 : T e1 := e2 : { e } x:unit { true } After assignment, nothing is guaranteed The contract true can be strengthened by assert(…) (explained later) Nothing is guaranteed after call to add Example: type set = int list Ref let add (s:set) (i:int) = let l = !s in s := (l :: i) This rule supposes the assignment may invalidate precondition. : set → int → { … } x:unit { true } 11/11/2018 POPL'17

Key typing rule: sequence e1 : {e1’}y:T’{e2’} y:T’ ⊢ e2 : {e2’}x:T{e3’} e1; e2 : {e1’}x:T{e3’} Postcondition of the last expression is that of program Example: type set val is_empty : set → Bool val create : unit → {true} s:set {is_empty s} val add : set → int → {…} x:unit {true} let s = create () let () = add s 1 s Same as the postcondition of add The second important rule is for sequence This rule means that postcondition of the last expression matches with that of the whole program Let me see how this rule solves the problematic example. Now, create and add functions are given these Hoare types. Using this rule, the type of the program results in this, where the postcondition is the same as the postcondition of add function, and it doesn’t say that the returned set is empty. So, the problem is solved. : { true } s:set { true } 11/11/2018 POPL'17

Issues in Hoare type How to detect contracts may be invalidated How to check contracts in Hoare types dynamically How to restrict contracts Contracts in refinement types are pure Contracts in Hoare types are read-only 11/11/2018 POPL'17

Dynamic check of state-dependent contracts assert(e) Check state-dependent contract e at run time If e true, the rest program is executed If e false, an exception is raised Typing rule strengthens postcondition The paper discusses static verification of assertions e : Bool Move the note to the last assert(e) : {e’}x:T{e’ & e} 11/11/2018 POPL'17

Issues in Hoare type How to detect contracts may be invalidated How to check contracts in Hoare types dynamically How to restrict contracts Contracts in refinement types are pure Contracts in Hoare types are read-only 11/11/2018 POPL'17

Restriction for soundness Contracts in refinement types are restricted to be observationally pure Contracts in Hoare types are restricted to be observationally read-only ✘ { x:int ref | !x = 1 } ✔ { x:int | let y = ref 1; y := 2; x = !y } 説明を練る Refine the title ✘ { true } x:int ref { x := 1; true } ✔ { true } x:int ref { let y = ref 1; y := 2; !x = !y } 11/11/2018 POPL'17

Solution: region-based effect system [T&T, POPL’94] Memory region R: readable region set W: writeable region set e ::= … | let r in e T ::= … | {e1}x:T{e2}<R,W> Give restriction by adjusting R and W Contracts in refinement types are disallowed to manipulate references other than locally allocated ones Contracts in Hoare types are disallowed to write into references other than locally allocated ones { x:T | e } wf x:T ⊢ e : {true}x:T{true}<∅, ∅> Can’t manipulate program references Can’t write into but can read from program references { e1 } x:T { e2 }<R,W> wf e1 : {true}x:T{true}<R, ∅> 11/11/2018 POPL'17

Type Soundness For any term e and program region r, if e : {true}x:T{e2}<r, r>, then evaluation of e: causes dynamic checking error; diverges; or results in some state μ and value v such that e2[v/x] evaluates to true under μ If T = {y:T’|e’}, then e’[v/y] evaluates to true 11/11/2018 POPL'17

In the paper… Full definition of our calculus Syntax in a monadic style Semantics for dynamic checking Cast semantics is extended to reference type and Hoare type Type-and-effect system based on regions Static verification of state-dependent contracts Sufficient conditions to eliminate assertions Region-based local reasoning 11/11/2018 POPL'17

Related work Hoare type theory (HTT) [Nanevski et al., ICFP’06] proposes Hoare types to verify stateful programs Purpose of study HTT: static verification Our work: hybrid verification Specification language HTT: first-order logic with heap variables Our work: programming language 11/11/2018 POPL'17

Conclusion Manifest contract calculus with mutable states Hoare type for state-dependent contracts Type system tracks what state-depend. contracts may be invalidated Assertion Check state-dependent contracts dynamically Effect system Control effects in contracts 11/11/2018 POPL'17