1. Luca de Alfaro Thomas A. Henzinger Ranjit Jhala UC Berkeley Compositional Methods for Probabilistic Systems

2. Compositional Methods for Probababilistic Systems2 Introduction Compositional Model : –Construct large systems from models of components Shallow Compositionality: Syntactic –Given P, Q can construct PkQ Deep Compositionality: Semantic –|[ P k Q ]| a function of |[P]|, |[Q]|

3. Compositional Methods for Probababilistic Systems3 Deep Compositionality : Example Transition systems with Trace Semantics Variable-based version: –System made of variables X –X-State: A valuation of the variables in X –X-Trace: A sequence of X-States, corresponding to a run –|[P]| : Set of X-Traces corresponding to all possible runs –Private variables projected away Given components P, Q: –Read variables written by each other –|[P k Q]| = |[P]| Å |[Q]|

4. Compositional Methods for Probababilistic Systems4 Deep Compositionality Composition of properties –Allows decomposition of large verification tasks Simple Refinement Decomposition: –To check: P 1 k P 2 ¹ Q 1 k Q 2 –Suffices that: P 1 ¹ Q 1 and P 2 ¹ Q 2 Assume-Guarantee Decomposition: –To check: P 1 k P 2 ¹ Q 1 k Q 2 –Suffices that: P 1 k Q 2 ¹ Q 1 and Q 1 k P 2 ¹ Q 2 Crucial for non-deterministic systems –Even more beneficial in the probabilistic setting

5. Compositional Methods for Probababilistic Systems5 Our Contribution First Deeply compositional model for systems with both Probabilistic and Non-deterministic choice Generalise semantic properties of trace-based models to the probabilistic setting First Assume-Guarantee rule for decomposing refinement checks for such systems

6. Compositional Methods for Probababilistic Systems6 Previous Work A large body of work on the modelling and verification of probabilistic systems –Vardi 85, Courcoubetis & Yannakakis 89 –Basic Model : Markov Decision Processes –Defining the behaviour using schedulers “Branching-time” models based on Process Algebras: Jonson & Larsen 91 Probabilistic Process Algebras –Performance properties Models based on I/O Automata by Segala 95 –Semantics described as Trace Distributions –Refinement as trace distribution inclusion

7. Compositional Methods for Probababilistic Systems7 Plan Systems with Probabilistic and Non-determinisitic choice Why is deep compositionality tricky ? –Atoms, the solution to the scheduler problem Concrete Model : Probabilistic Modules Bundle Algebra Theorems Conclusions etc.

8. Compositional Methods for Probababilistic Systems8 Probabilistic Systems We wish to model transition systems that can make both Probabilistic and Non-deterministic choice ¼ ¾ ½ ½ At a state, the system does the following: 1.Picks one of several available distributions (or moves) over next state non-deterministically 2.Picks a next state randomly out of the chosen distribution

9. Compositional Methods for Probababilistic Systems9 Prob. Systems: Example ¼ ¾ ½ ½ There are 2 possible behaviors arising from the non-deterministic choice at ¼, ¾ ½, ½

10. Compositional Methods for Probababilistic Systems10 Semantics: dealing with choices Non-deterministic, Probabilistic choice are “orthogonal” Factor out non-determinism using schedulers [Derman70, Vardi 1985, Courcoubetis & Yannakakis 1989] Given a scheduler, the execution is fully probabilistic –Outcome: A sequence of bundles of length i, 8 i > 0 –Semantics: Sum of the outcomes for all the different schedulers

11. Compositional Methods for Probababilistic Systems11 Schedulers: Example 1/2 4 Possible Schedulers, one outcome (bundle) for each ½ :, ½ : Outcomes (Bundles)Schedulers

12. Compositional Methods for Probababilistic Systems12 Non-Det. Choice Vs Prob. Choice 1/2 AB Non-deterministic choice is more flexible than probabilistic choice We want A ¹ B, but … Bundle of A ½, ½ Bundles of B 11 1

13. Compositional Methods for Probababilistic Systems13 , 1-  Non-Det. Choice Vs Prob. Choice 1/2 AB Solution: Let the scheduler be randomized The scheduler of B can flip a coin to select nondeterministic choice The move of B is then the convex combination of its simple moves Bundles of B: For every  2 [0,1] In particular  = ½ matches A’s bundle

14. Compositional Methods for Probababilistic Systems14 Semantics of Probabilistic Systems X-State: A valuation of the variables in X 1/3 2/3 X-Move: A probability distribution over X-States Given a set of variables X: X-Trace: A sequence of X-States X-Bundle: A probability distribution over X-Traces 1/2 1/3 1/6 X-Probabilistic Language: A set of X-Bundles

15. Compositional Methods for Probababilistic Systems15 Semantics of Probabilistic Systems Refinement corresponds to bundle inclusion: –P ¹ Q if |[ P ]| µ |[ Q ]| Given a Probabilistic system P with variables X, semantics |[ P ]| is an X-Probabilistic language X-Probabilistic Language: A set of X-Bundles

16. Compositional Methods for Probababilistic Systems16 Plan Systems with Probabilistic and Non-determinisitic choice Why is deep compositionality tricky ? –Atoms, the solution to the scheduler problem Concrete Model : Probabilistic Modules Bundle Algebra Theorems Conclusions etc.

17. Compositional Methods for Probababilistic Systems17 Why is it tricky ? (1) P X0X0 Y0Y0 P0P0 X0X0 Y0Y0 P1P1 X0X0 Y0Y0 P0P0 X0X0 Y0Y0 P1P1 X1X1 Y1Y1 P Priv: P Ctr : X Extl: Y Q X0X0 Y0Y0 Q0Q0 X0X0 Y0Y0 Q1Q1 X0X0 Y0Y0 Q0Q0 Y0Y0 X0X0 Q1Q1 Y1Y1 X1X1 Q Priv: Q Ctr : Y Extl: X PkQ Priv: P, Q Ctr : X, Y Q X0X0 Y0Y0 P Q1Q1 X0X0 Y0Y0 P1P1 Q0Q0 X0X0 Y0Y0 P1P1 Q1Q1 X0X0 Y0Y0 P0P0 Q0Q0 X0X0 Y0Y0 P0P0 Q0Q0 X0X0 Y0Y0 P0P0 Q1Q1 X0X0 Y1Y1 P0P0 Q0Q0 X1X1 Y0Y0 P1P1 Q1Q1 X1X1 Y1Y1 P1P1 1/2 1/4 This is the ONLY bundle of P k Q ) |[P]| Å |[Q]| ¾ |[ P k Q ]| !! A bundle in |[P]| and |[Q]|

18. Compositional Methods for Probababilistic Systems18 Why is it tricky ? (1) |[P]| Å |[Q]| ¾ |[ P k Q ]| !! P X0X0 Y0Y0 P0P0 X0X0 Y0Y0 P1P1 X0X0 Y0Y0 P0P0 X0X0 Y0Y0 P1P1 X1X1 Y1Y1 P Priv: P Ctr : X Extl: Y Q X0X0 Y0Y0 Q0Q0 X0X0 Y0Y0 Q1Q1 X0X0 Y0Y0 P0P0 Y0Y0 X0X0 Q1Q1 Y1Y1 X1X1 Q Priv: Q Ctr : Y Extl: X PkQ Priv: P, Q Ctr : X, Y Q X0X0 Y0Y0 P Q1Q1 X0X0 Y0Y0 P1P1 Q0Q0 X0X0 Y0Y0 P1P1 Q1Q1 X0X0 Y0Y0 P0P0 Q0Q0 X0X0 Y0Y0 P0P0 Q0Q0 X0X0 Y0Y0 P0P0 Q1Q1 X0X0 Y0Y0 P0P0 Q0Q0 X0X0 Y0Y0 P1P1 Q1Q1 X0X0 Y0Y0 P1P1 External variable was scheduled looking at private variable … … this breaks compositionality ) must have two schedulers 1.CONTROLLED-VAR scheduler: can look at private variables 2.EXTERNAL-VAR scheduler: cannot look at private variables

19. Compositional Methods for Probababilistic Systems19 Why is it tricky ? (2) P Ctr : X, non-det Extl: Y Q Ctr : Y, non-det Extl: X PkQ Ctr : X, Y X,Y are non-det. set With a single scheduler we get : No matching bundle in |[P]| or |[Q]| |[P]| Å |[Q]| ½ |[ P k Q ]| !! ) A composed system must be made up of schedulers for individual components X:=1 X:=0  1-  Y:=1 Y:=0  1-  X0X0 Y0Y0 X1X1 X0X0 X1X1 Y0Y0 Y1Y1 X1X1 (1-  )(1-  )(1-  )  (1-  )    1/2 X0X0 Y0Y0 X1X1 X0X0 X1X1 Y0Y0 Y1Y1 X1X1 00

20. Compositional Methods for Probababilistic Systems20 Ex 2: After composition, joint scheduling breaks compos. Ex 1: Environment must not see private variables ! Schedulers and Compositionality Q: Why are previous models not deeply compositional ? A: Monolithic Schedulers are bad !! Module P Interface x Private p External y Module Q Interface y Private q External x Module P k Q Interface x Private p Interface y Private q Compose

21. Compositional Methods for Probababilistic Systems21 Atoms : The Solution to the Scheduler Problem A single scheduler associated with each atom - Module Scheduler is the “composition” of atomic schedulers Atomic (scheduling) structure preserved after parallel composition Module P Reads x,p,y… Writes x,p… External y,… Writes y Reads Obs Reads x,… Writes … Module Q Reads y,p,x… Writes x,p… External x,… Writes x Reads Obs Reads x,… Writes … Compose Module P k Q Reads x,p,y… Writes x,p… Reads x,… Writes … Reads y,p,x… Writes x,p… Reads x,… Writes … External … Writes … Reads Obs Atoms : Units of Scheduling Variables written by the atom Variables read : on whose history non-det. is resolved

22. Compositional Methods for Probababilistic Systems22 The Importance of Atoms Module A Atom Axy controls x,y Init [] true-> x,y:=0,0 [] true-> x,y:=0,1 [] true-> x,y:=1,0 [] true-> x,y:=1,1 Module B Atom Bx controls x Init [] true-> x:=0 [] true-> x:=1 Update []... Atom By controls y Init [] true-> y:=0 [] true-> y:=1 Update []... |[A]|  |[B]| because: A has a bundle where x,y have correlated values { ½: 0,0 ½: 1,1} In B’s bundle it is not possible to get correlation, despite complete non-det in each atom, as the schedulers are independent

23. Compositional Methods for Probababilistic Systems23 Plan Systems with Probabilistic and Non-determinisitic choice Why is deep compositionality tricky ? –Atoms, the solution to the scheduler problem Concrete Model : Probabilistic Modules Bundle Algebra Theorems Conclusions etc.

24. Compositional Methods for Probababilistic Systems24 Probabilistic Modules Module A Interface x,w Private y External z Atom A XY control x,y read x,y,z Init [] true-> ½ x,y:=0,0 ½ x,y:=1,1 Update [] true-> x’,y’:= x,x [] y ->’¼ x’y:= : z,z ¾ x’y’= z, : z Atom A w control w read y,z Init [] true-> w:=0 [] true-> w:=1 Update [] true-> w’:= z Update : To each state, associate a set of distributions (moves), for next state Z1Z1 X1X1 Y1Y1 X1X1 1 Move 1 Y1Y1 X0X0 1/4 Y1Y1 X1X1 Y0Y0 3/4 Move 2 The atom scheduler Chooses between moves

25. Compositional Methods for Probababilistic Systems25 Operations : Parallel Composition Module P Reads x,p,y… Writes x,p… External y,… Writes y Reads Obs Reads x,… Writes … Module Q Reads y,p,x… Writes x,p… External x,… Writes x Reads Obs Reads x,… Writes …

26. Compositional Methods for Probababilistic Systems26 Operations : Parallel Composition Module P k Q Reads x,p,y… Writes x,p… External y,… Writes y Reads Obs Reads x,… Writes … Reads y,p,x… Writes x,p… Reads x,… Writes …

27. Compositional Methods for Probababilistic Systems27 Module A Interface x,w Private y External z Atom A XY control x,y read x,y,z Init [] true-> ½ x,y:=0,0 ½ x,y:=1,1 Update [] true-> x’,y’:= x,x [] y ->¼ x’y’:= : z,z ¾ x’y’= z, : z Atom A w control w read y,z Init [] true-> w:=0 [] true-> w:=1 Update [] true-> w’:= z Module Semantics

28. Compositional Methods for Probababilistic Systems28 Module Semantics Module A Reads x,y,z Writes x,y External z Writes z Reads x,w Reads y,z… Writes w… 11 22  env Schedulers for every atom Each Scheduler takes a trace, returns a move  : 1/3 2/3 Every triple (  1,  2,  env ) generates a bundle 1/2 1/3 1/6 |[A]| = Union over all triples (  1,  2,  env ) :

29. Compositional Methods for Probababilistic Systems29 Composing Atomic Schedulers XPXP Ctr P XQXQ Ctr Q XPXP Ctr P XQXQ Ctr Q Project PP Ctr P Move QQ Ctr Q Move  P £  Q =  P||Q £ Ctr P [ Ctr Q = X P k Q Move

30. Compositional Methods for Probababilistic Systems30 Semantics: Atomic Schedulers Composing Atom Schedulers: For schedulers  1 from X 1 to Y 1,  2 from X 2 to Y 2, s.t. Y 1 Å Y 2 = ?, (  1 £  2 ) : from X 1 [ X 2 to Y 1 [ Y 2 s.t. (  1 £  2 )(t) =  1 (t[X 1 ]) £  2 (t[X 2 ]) For sets of schedulers  1 from X 1 to Y 1,  2 from X 2 to Y 2,  1 £  2 = {  1 £  2 |  1 2  1,  2 2  2 }

31. Compositional Methods for Probababilistic Systems31 Module Semantics Schedulers of P extl  (P) = set of all schedulers from extlX(P) [ intfX(P) to extlX(P) mod  (P) = extl  (P) £  A 2 Atoms(P) atom  (A) Language of P L(P) = [  2 mod  (P) Outcome(  ) Trace Semantics of P |[ P ]| = L(P)[obsX(P)] – the language projected to the observables

32. Compositional Methods for Probababilistic Systems32 Plan Systems with Probabilistic and Non-determinisitic choice Why is deep compositionality tricky ? –Atoms, the solution to the scheduler problem Concrete Model : Probabilistic Modules Bundle Algebra Theorems Conclusions etc.

33. Compositional Methods for Probababilistic Systems33 Semantics of Probabilistic Systems X-State: A valuation of the variables in X 1/3 2/3 X-Move: A probability distribution over X-States Given a set of variables X: X-Trace: A sequence of X-States X-Bundle: A probability distribution over X-Traces 1/2 1/3 1/6 X-Probabilistic Language: A set of X-Bundles

34. Compositional Methods for Probababilistic Systems34 Bundle Algebra For reasoning about parallel composition Decomposing : Projection Given sets of variables X, X’ s.t. X’ µ X –X-Bundle  X’-Bundle Composing : Product Given sets of variables X, Y – X-Bundle £ Y-Bundle  (X [ Y) – Bundle

35. Compositional Methods for Probababilistic Systems35 Projection : States X’ X X State X’State

36. Compositional Methods for Probababilistic Systems36 Projection : Moves 1/9 1/6 1/9 1/3 X X Move X’ Move X’

37. Compositional Methods for Probababilistic Systems37 Projection : Bundles 1/81/12 1/241/6 1/9 1/3 X Bundle X’ Bundle

38. Compositional Methods for Probababilistic Systems38 Product : States X X YX Z X [ Y StateX [ Z State X [ Y [ Z State Y X Z

39. Compositional Methods for Probababilistic Systems39 Product : Moves, Bundles £ X [ Y Move X [ Z Move X [ Y [ Z Move Y X = X Z 1 2 1.25.5 1 2 3 1 2.166.25 11 12 13 23 22 21 11 12.25 x.166 /.5.5 x.25 /.5

40. Compositional Methods for Probababilistic Systems40 Operations : Product Product : Given 2 sets of variables X 1, X 2 : –Given an X 1 -State s 1, a X 2 -State s 2 : s 1, s 2 can be multiplied if s 1 [X 1 Å X 2 ] = s 2 [X 1 Å X 2 ] –Same condition for for Traces and Bundles –Given an X 1 -Bundle b 1, X 2 -Bundle b 2 : (b 1 £ b 2 ): X 1 [ X 2 – Bundle s.t. (b 1 £ b 2 )(t) = b 1 (t[X 1 ]) £ b 2 (t[X 2 ]) / b 1 (t[X 1 Å X 2 ]) –Given an X 1 -Language L 1, X 2 -Language L 2 : L 1 £ L 2 = { b 1 £ b 2 | b 1 2 L 1 and b 2 2 L 2 can be multiplied }

41. Compositional Methods for Probababilistic Systems41 Plan Systems with Probabilistic and Non-determinisitic choice Why is deep compositionality tricky ? –Atoms, the solution to the scheduler problem Concrete Model : Probabilistic Modules Bundle Algebra Theorems Conclusions etc.

42. Compositional Methods for Probababilistic Systems42 Compositional Semantics Theorem: |[ P 1 k P 2 ]| = |[ P 1 ]| Å |[ P 2 ]| This is because L(P 1 k P 2 ) = L(P 1 ) £ L(P 2 ) For every b 1 2 L(P 1 ), b 2 2 L(P 2 ), s.t. b 1 [X(P 1 ) Å X(P 2 )] = b 2 [X(P 1 ) Å X(P 2 )] … are multipliable b 1 £ b 2 2 L(P 1 k P 2 ) For every b 2 L(P 1 k P 2 ) b[X(P 1 )] 2 L(P 1 ) and b[X(P 2 )] 2 L(P 2 )

43. Compositional Methods for Probababilistic Systems43 Recall : Probabilistic Refinement Refinement corresponds to bundle inclusion: –P ¹ Q if |[ P ]| µ |[ Q ]| Given a Probabilistic system P with variables X, semantics |[ P ]| is an X-Probabilistic language X-Probabilistic Language: A set of X-Bundles

44. Compositional Methods for Probababilistic Systems44 Refinement Is Compositional Module Refinement: P ¹ Q iff |[ P ]| µ |[ Q ]| Theorem: Refinement is Compositional P k Q ¹ P If P ¹ Q, then P k R ¹ Q k R –Follows from deep compositionality Theorem: Assume-Guarantee If P 1 k Q 2 ¹ Q 1 and Q 1 k P 2 ¹ Q 2, then P 1 k P 2 ¹ Q 1 k Q 2 –Deep compositionality –Induction

45. Compositional Methods for Probababilistic Systems45 Conclusions Deeply compositional semantics for systems with Non-deterministic and Probabilistic choice Assume-Guarantee rule Only possible by restricting the visibility and influence of schedulers Checking Bundle Inclusion –Simulation based approach Adding combinational (0-delay) dependencies Logics for Specification: –Correctness and performance properties –Compositional reasoning