Probabilistic CEGAR* Björn Wachter Joint work with Holger Hermanns, Lijun Zhang TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A AAA A AVACS Supported by Uni Saar *To appear in CAV

2 Introducing Probabilistic Model Checking CEGAR (counterexample-guided abstraction refinement) PASS does CEGAR for probabilistic models 1

3 PRISM & PASS PRISM  Very popular probabilistic model checker  Finite-state PASS  Supports PRISM models  handles infinite-state as well  Under the Hood: Predicate abstraction SMT Interpolation

4 Comparison to PRISM Network protocols  Wireless LAN, CSMA  Bounded Retransmission  Sliding Window Model (#)State reduction Speed-up WLAN (3) WLAN (1) 16x-152x ? 1,3x-7x TO->311s CSMA (4)41x-248x1x-2x BRP (3)1x1/2x - 1/3x PRISM vs PASS

5 Basics  Paths, Markov Chains, MDPs  Counterexamples  Probabilistic Programs  Predicate Abstraction Abstraction Refinement  Abstract Counterexamples  Path Analysis  Strongest Evidence  CEGAR algorithm Experimental Results Conclusion Program e Probabilistic Reachability Problem Overview

6 Paths, MCs, MDPs Weighted Path Markov Chain non-determinism … 2/3 1/3 2/31/3

7 Paths, MCs, MDPs 2/3 1/3 2/3 1/3 1 1/2 1/3 2/31/3 Weighted Path Markov Chain Markov Decision Process

8 Adversary Adversary resolves transition non-determinism 2/3 1/3 1 1/2

9 Probabilistic Reachability Probability to get from green to red Weighted Path Markov Chain Markov Decision Process 2/3 1/3 2/3 1/3 1 1/2 1/3 2/31/3

10 Guarded command language à la PRISM  Variables: integer, real, bool  Non-determinism: interleaving Example: Program = (variables, commands, initial condition) Probabilistic Programs x=1 0.2: (x‘:=x+1) x=2 Update #1 0.8: (x‘:=x+2) x=3 Update #2 Guard: x>0 guard Labels for CEX Analysis

11 Predicates: partition the state space  are boolean expressions x>0, x<y, x + y = 3 (variables x,y)  Abstract MDP  Probabilistic may-transitions Similar to Blast, SLAM, Magic …  See our [Qest’07] paper Abstraction guarantees upper bound Predicate Abstraction actual 1 0 Probability: Abstract MDP

12 May Transitions Hier ist‘s noch nicht verständlich genug! Besseres Beispiel wo #abs. trans < #conc. trans abstract concrete

13 CEGAR Loop p actual upper abstract check refine Probability CEX ? Real CEX Low enough

14 Counterexamples (CEX) Resolution of non-determinism  initial state  adversary induces a Markov chain Counterexample:  Resolution of non-det such that probability threshold exceeded Example: CEX for Witness of Reachability probability in MDP 2/3 1/3 1 1/2

15 Path 1Path 2Path 3Path 4… Counterexample Analysis: Idea Idea:  Enumerate paths of Markov chain  Sort paths by probability [Han\Katoen2007]: visit paths with highest measure first  Realizable Spurious Path 1Path 2Path 3Path 4… Probability of Abstract CEX / Markov Chain How much MEASURE is REALIZABLE? More than p?

16 Path Analysis Abstract path: Two cases  Realizable if there‘s a corresponding concrete path  Spurious: no corresponding path Splitter predicate exists iff path spurious Interpolation: predicate from unsatisfiable path formula uu´ u´´ uu´ u´´ uu´ u´´ Reachable with prefix Can do postfix Path formula SAT UNSAT Logic (SMT)

17 Path Analysis Abstract path: Two cases  Realizable if there‘s a corresponding concrete path  Spurious: no corresponding path Splitter predicate (interpolant): uu´ u´´ uu´ u´´ 0 1 x´:=x x´:=x+1 Reachable with prefix Can do postfix Path formula SAT UNSAT Logic (SMT) x=0 x=1 X 10 x>1

18 Example 1.0 concrete abstract Probability: Upper: ?

19 Example(cont): after refinement 0.4 Concrete abstract Probability: Upper: lower

20 Example concrete abstract lower 0.8 Upper 1.0 Multiple Initial states

21 Example concrete abstract Maximum Find Maximal Combination by MAX-SMT (  paper) Probability: lower 0.8 Upper 1.0

22 CEX Analysis: Semi decision procedure Problem in general: undecidable Too many spurious paths  abort counterexample analysis  Output: collection of predicates Enough realizable probability Path 1Path 2Path 3Path 4…Path 1Path 2Path 3Path 4… > C Limit # of spurious paths to enforce termination Path 1Path 2Path 3Path 4…Path 1Path 2Path 3Path 4… Can take many paths To obtain enough realizable probability 0 lower = real

23 Related Work Probabilistic Counterexamples:  … however not in the context of abstraction Hermanns/Aljazzar (FORMATS’05), Han/Katoen (TACAS’07) Abstraction Refinement for Prob. Finite-state Models  CEGAR for stochastic games, Chatterjee et al (UAI’05)  Not based on counterexamples D‘Argenio (Papm-Probmiv02), Fecher & al (SPIN’06): simulation Magnifying-lens, de Alfaro et al (CAV’07): probability values

24 Conclusion & Future Work Abstraction refinement …  Counterexamples ~ Markov Chains Markov Chains have cycles Model Checking Infinite-state Probabilistic Models Speed-up for huge finite-state models Future Work  Better Lower bounds

25 References Tool website Literature  Our work Hermanns, Wachter, Zhang: Probabilistic CEGAR (CAV’08) Wachter, Zhang, Hermanns: MC Modulo Theories (Qest’07)  Counterexamples Hermanns, Aljazar: CEX for timed prob reachability, FORMATS‘05 Han, Katoen: CEX in probabilistic model checking, TACAS‘07  Probabilistic Abstraction Refinement De Alfaro, Magnifying-lens abstraction for MDPs, CAV‘07 Chatterjee, Henzinger, Majumdar: CEX-guided planning, UAI’05

26 Questions?

27 Is Counterexample analysis problem undecidable? Semi-decision algorithm  heuristics If we only need finiteley many paths  decidable if logic is If we need infinitely many  undecidable