1. SS 2017 Software Verification Probabilistic modelling – DTMC / MDP
Prof. Dr. Holger Schlingloff 1,2
Dr. Esteban Pavese 1
(1) Institut für Informatik der Humboldt Universität
(2) Fraunhofer Institut für offene Kommunikationssysteme FOKUS

3. Model checking so far...

4. Model checking so far...

5. Model checking so far...

6. Model checking so far... Consider the following situation
N processes run and communicate in a ring topology; i.e. each one sees two neighbours (left/right)
The processes need to agree on a leader (a distinct process)
Can only communicate left or right, and processes are anonymous (i.e., cannot send a message with pid)

7. Leader election
Could you model this in SPIN or NuSMV or UPPAAL, or ...?
We can’t. (At least not a deterministic version)
Prove it!

8. Leader election We need to break the symmetry
Idea – each process has a ”coin”
Toss the coin, send result to process on the right, and receive from the left
If tossed heads and received tails, keep participating but already decide not to be leader
Receive a counter from left, add 1 to counter if already decided not to be leader, send counter to right
If received a counter of N-1, become leader

9. Model checking so far...
The techniques we’ve seen so far are somewhat lacking
...especially when we attempt to perform more realistic modelling
... or when the process we are modelling relies on randomizing its decisions
... or if we want to verify non-functional properties of the system

10. Probabilistic information
The leader election is a nice example
Many verification questions we can ask
does it actually work?
does it terminate?
if it terminates, does it pick a unique leader?
what is the chance it does not terminate?
if 0, how long does it take to terminate?
what is the chance it terminates in T time?

11. Probabilistic automata
Of course, neither Finite Automata or Timed Automata suffice to answer these questions
We need a new formalism (again!) – that of Probabilistic Automata
“Probabilistic Automata” is a very general blanket term
In this lesson we will focus on Discrete Time Markov Chains

12. Discrete Time Markov Chain (DTMC)
State-based definition of a DTMC
Outside of sw verification a random variable sequence definition seems more common
A DTMC is a tuple with

13. No actions? DTMC transitions are purely probabilistic.
Having different actions would imply some level of non-determinism
So we can safely drop actions from the model

14. Example – Knuth/Yao dice
or, how to play dice with coins

15. Knuth/Yao dice Is it correct?
What is the chance of ending up on 2 pips?
and 6 pips?

16. Probability basics

17. Probability basics Given an experiment
the set Ω denotes the set of possible outcomes
For 1 coin toss, Ω = {Heads,Tails}
For 2 tosses, Ω = {HH,HT,TH,TT}
An event E is any subset of Ω
For 2 tosses, “at least 1 heads” is event = {HH,HT,TH}
infinite tosses an event could be “4 heads in a row”
A probability measure is a function μ: E → [0,1] such that
μ(∅)= 0
μ(Ω)=1
For any pair of disjoint events e1 and e2, μ(e1) + μ(e2) = μ(e1∪e2)

18. Probability basics

19. Knuth/Yao dice Is it correct?
What is the chance of ending up on 2 pips?
and 6 pips?

20. Knuth/Yao dice Is it correct?
do all have 1/6 chance?
What is the chance of ending up on 2 pips?
1/6 (?)
and 6 pips?
1/6...but how to measure?

21. Measuring over DTMCs
Ω = (possibly infinite) set of (possibly infinite) paths
Events: (possibly infinite) sets of (possibly infinite) paths
Things get strange quite fast
How is the probability of a path calculated?
What about infinite paths?
What about infinite sets of infinite paths?

22. Cylinder sets
The cylinder (also called cone) of a finite path on a DTMC is the set of infinite paths that stems from it
The measure of the cylinder is then easily calculated from the probability of the finite path

23. Knuth/Yao dice What is the chance of 2 pips? Infinite cones!
(0,1,4,2p) + (0,1,3,1,4,2p) (0,1,(3,1)*,4,2p) + ...
Infinite cones!
How to solve this?

24. Knuth/Yao dice 2 pips state = 1, all other pip states 0
Each intermediate state measures the chance to get to 2p
Define a (recurrent) equation system

25. Knuth/Yao dice

26. Another example...

27. Another example...
It is customary to define the DTMC by its stochastic adjacency matrix

28. Some properties to check
What is the probability of failing the first time?
Measure the cones of the paths where the first send results in failure

29. Some properties to check
What is the chance of eventually succeeding, without any failure in between?

30. Some notes for next class
A large set of properties over DTMCs can be expressed as reachability properties
In particular, both “safety” and “liveness” properties can be expressed this way
Moreover, the probability of reaching a given state can be measured from the set of disjoint cones that characterises this reachability
The set may be infinite
but it is countably infinite

31. Some notes for next class
Next class: the logic PCTL
Its relationship with CTL and LTL
Model checking procedure for PCTL
Maybe you can already figure it out?
Implementation details to increase feasibility of the model checking procedure
DTMC extensions