1 An Inheritance-Based Technique for Building Simulation Proofs Incrementally Idit Keidar, Roger Khazan, Nancy Lynch, Alex Shvartsman MIT Lab for Computer Science Theory of Distributed Systems Group

2 State of the Art Software Engineering Managing complexity of software systems –Modularity: interacting system components –Incremental techniques: OO, inheritance Formal modeling and verification –Modularity: compositional theorems –Incremental techniques: lag behind  Limited scalability  Not sufficiently cost-effective

3 Our Work: Scalable Formal Methods Provide incremental techniques for –Specifying systems –Modeling systems –Reusing formal proofs about systems Evolved as part of our experience modeling complex group communication service –Middleware with intricate semantics –Implemented in C++ –[Keidar, Khazan, ICDCS 2000]

4 Talk Outline Motivation: need formal framework for incremental proofs The approach, in a nutshell Background: specifications, simulation proofs The challenge: reuse simulation proofs The solution –Modification constructs –Proof reuse theorem Experience using the technique Summary

5 Our Approach, in a Nutshell

6 The Idea OO SWE Techniques Formal Modeling and Verification Techniques Incremental Specification, Modeling, and Proof Reuse Techniques

7 Inheritance Incremental modification of components Many different kinds. We consider: 1Specialization (sub-typing, substitution) –Child constraints parent behavior –E.g., Parent is unordered messaging protocol; Child specializes Parent to ordered messaging 2Interface extension –Together with specialization allows new behavior, but does not override parent behavior (“sub-classing for extension”)

8 Inheritance in Spec and Proofs Specification S System A Implements Specification S’ parent System A’ parent ?! Prove that A’ implements S’ by relying on proof that A implements S, but without repeating reasoning of that proof.

9 Background: Specifications and Proofs

10 Specifying and Modeling System Abstract state machines [Lampson; Schneider ’93] –States (named variables) –Actions –Specify in which states each action is enabled and how it modifies state –Some actions are externally observable –State is not externally observable Execution: “state, action, state, action, …” Defined behavior: all possible sequences of externally observable actions  Traces

11 Example -- Monotonic Sequence Specification UpSeq State: Integer last, init any Actions: print(x) pre: x  last eff: last:= x Sample Traces: –1, 3, 4, … –-10, -5, 1, 2, 3, 5, 8, 13, …

12 What “Implements” Means? System A implements specification S if every trace of A is a trace of S “Implements”  “trace inclusion” A is indistinguishable from S by looking only at A’s traces

13 Example -- Fib Sequence System FibSeq State: Integer n=0,m=1 Actions: print(x) pre: x == n + m eff: n := m m := x Trace: 1, 2, 3, 5, 8, 13, … –is also trace of monotonic sequence UpSeq

14 Proving that A Implements S Simulation mapping / abstraction function F : {the states of A}  {the states of S}  F maps initial states of A to initial states of S  For every action of A and for every state t, if (t, , t’) is a step of A then there is a sequence of actions of S that starts in F(t), ends in F(t’), and has the same trace as . Simulation Mapping  Trace Inclusion

15 F(t)F(t’) Action  FF t t’ Action  Spec S: System A:  pre(S.  ) holds whenever pre(A.  ) holds  State of A after eff(A.  ) is executed maps into state of S after eff(S.  ), assuming pre-states map  of A Simulates  of S:

16 FibSeq Implements UpSeq UpSeq.print(x) pre: x  last eff: last:=x FibSeq.print(x) pre: x==n+m eff: n:=m m:=x last:=m F For simulation proof, need to show:  pre(FibSeq.print(x))  pre(UpSeq.print(x)) (x=n+m)  (x  last=m) ; relies on n  0  last still equals m in post-state

17 Simulation Proofs -- Benefits Complete [e.g., Abadi and Lamport ‘93] –Any finite trace inclusion can be shown Tractable –Inductive reasoning –Reason about single steps, not about executions Verifiable by humans and/or machines

18 The Challenge: Reuse of Simulation Proofs

19 Our Proof Reuse Goal Specification S Specification S’ System A System A’ Simulates Parent ?! A formal framework –for proving that A’ simulates S’ –without repeating parent’s proof.

20 Why Not Immediate Traces of S Traces of A Traces of A’ Traces of S’

21 The Solution: Formal Framework for Incremental Specifications and Simulation Proofs

22 What We Did Specialization and interface extension for specifying and modeling systems “Proof Reuse” Theorem –Defines simulation between children –Reuses and extends simulation between parents –Requires proving conditions about extension –Involves reasoning only about modifications

23 Specialization Construct In precondition-effect notation, child can –Introduce new state variables –Restrict parent actions with new preconditions –Add new effects that modify new variables only A’ = specialize(A)(NewVars, ActionRestriction) Child may only restrict parent behavior

24 Recall: Monotonic Sequence Specification UpSeq State: Integer last, init any Actions: print(x) pre: x  last eff: last:= x Sample Traces: –1, 3, 4, … –-10, -5, 1, 2, 3, 5, 8, 13, …

25 Specialization of UpSeq : Accelerating Sequence, “Guiness” Specification GnSeq specializes UpSeq New State: Integer diff, init any Action Restriction: print(x) new pre: x-last  diff new eff: diff:= x-last Sample Trace: 1, 2, 3, 5, 8, 13, … –also trace of UpSeq

26 Using “Proof Reuse” Theorem In order to show simulation from A’ to S’: Extend simulation mapping F from A to S with mapping F’ from states of A’ to new variables of S’ Reason only about how S’ restricts S: 1.new preconditions are enabled; and 2.mapping F’ is preserved after new effects occur  Allows to reuse F and simulation proof of F

27 Example UpSeq GnSeq FibSeq Simulates Specializes F FibSeq ?! F’

28 FibSeq Implements GnSeq UpSeq.print(x) pre: x  last eff: last:=x FibSeq.print(x) pre: x==n+m eff: n:=m m:=x GnSeq.print(x) pre: x-last  diff eff: diff:=x-last last:=m diff:=m-n For simulation proof, need to show :  pre(FibSeq.print(x))  newpre(GnSeq.print(x)) (x=n+m)  (x-last  diff)  diff still equals m-n in post-state

29 Experience: Using the Technique

30 Group Communication Powerful building blocks for fault tolerant distributed systems Reliable multicast to groups Group membership “who is in the group” Virtual Synchrony semantics synchronize messages and membership changes –Processes see events in same order Formal specification, modeling, and verification: a challenge

31 Keidar and Khazan, ICDCS 2000 Modeling a full-fledged group communication service –Specification –Algorithm description, matching C++ implementation (9,000 lines) –Environment specification matching services we use (developed by other teams): membership server (20,000 lines) reliable communication service (4,000 lines) –Simulation proof from algorithm to spec

32 Incremental Specification, Algorithm and Proof FIFO communication in views –Algorithm: half page –Proof 1: 5 pp., 7 major invariants, history vars Virtual Synchrony + Transitional Set –Algorithm modification: full page –Proof 2: 2.5 pp., 1 invariant –Proof 3: 2.5 pp., 3 invariants, prophecy vars Self Delivery –Proof 4: 2.5 pp., 3 invariants

33 A Modeling Methodology Child cannot write to parent data structures Sometimes need hooks at parent for potential children to specify policy –Non-determinism at parent –Parent specifies coherence, safety –Like “abstract” or “virtual” methods Example: forwarding of messages to others –At parent arbitrary forwarding, but forwarded messages go into “right” place –Child specifies forwarding policy

34 Benefits of Reuse Present complex algorithm step by step –Easy to see which part of algorithm corresponds to which part of spec –Efficient: re-use of data structures (as opposed to compositional layered approach) Manageable proof –Focus attention on specific property –No need to re-prove that previous proof preserves (contrast with layers)

35 Summary of Contributions Formal framework for incremental development of specs & simulation proofs Child simulation proof may reuse parent proof – involves reasoning only about modified parts Successfully used to model and validate complex communication system [ICDCS 00] Formalism extends IOA programming and modeling language [Garland&Lynch] Suitable for other state-machine models

36 The End

37 Non-triviality illustration Traces of S Traces of A Traces of A’

Traces of S’

39 Example UpSeqPrint GrSeqPrint FibSeqPrint Simulates Specializes ?!

40 Why not immediate A’ specializes A, which in turn implements S, thus A’ implements S S’ specializes S, thus S’ also implements S But does A’ implements S’ ?! Traces(A’)  Traces(A)  Traces(S) Traces(S’)  Traces(S) Traces(A’)  Traces(S’) ?

41 Note: new preconditions and effects are in addition to those in UpSeq specification.

42 Formalism and Notation Need formalism to express specifications, systems, incremental construct, proofs Used I/O Automaton model –Widely used for formal specs and algorithms –Compositional –There is a programming and modeling language Notation: precondition-effect –State variables: typed, with initial values –Actions: specified by preconditions and effects