1. Spring 2018. CS 599. Instructor: Jyo Deshmukh
Autonomous Cyber-Physical Systems: Fairness, Monitors and Branching-Time Logics
Spring CS 599.
Instructor: Jyo Deshmukh
Acknowledgment: Some of the material in these slides is based on the lecture slides for CIS 540: Principles of Embedded Computation taught by Rajeev Alur at the University of Pennsylvania.

2. Overview Last lecture Introduction to LTL Various LTL equivalences
𝐅𝜑=true 𝐔 𝜑
𝐆𝜑=¬𝐅¬𝜑
This lecture
Requirements and Fairness using LTL
Computation Tree Logic
Probabilistic Computation Tree Logic

3. LTL is a language for expressing system requirements
So far we have seen how we can express behaviors of individual system traces using LTL
A system 𝑀 starting from some initial state 𝑞 0 satisfies a LTL requirement 𝜑 if all system behaviors starting in 𝑞 0 satisfy the requirement 𝜑
Denoted as 𝑀, 𝑞 0 ⊨𝜑
E.g. a system is safe w.r.t. a safety requirement 𝜑 if all behaviors satisfy 𝜑
Does (Blinker, (x↦0,y↦0)) ⊨ 𝐆(x≥0)?
nat x := 0; bool y:= 0
A: x := x + 1
B: even(x) →
y: = 1-y
Blinker

4. Processes & Fairness A: x := x + 1 B: even(x) → y: = 1-y Blinker
Liveness property: 𝐅 (x ≥ 10)
Is this property guaranteed to hold?
No, task A may be executed less than 10 times.
Liveness Property: 𝐅 y (eventually y is 1)
No, task B may never be selected for execution!
But, this seems like a very unrealistic or broken scheduler!
For infinite executions involving multiple tasks, it is important for the execution to be fair to each task
nat x := 0; bool y:= 0
A: x := x + 1
B: even(x) →
y: = 1-y
Blinker

5. Weak vs. Strong fairness
A fairness assumption is a property that encodes the meaning of what it means for an infinite execution to be fair with respect to a task.
Weak fairness: If a task is persistently enabled, then it is repeatedly executed.
I.e. if after some point the task guard is always true, then the task is infinitely often executed.
Strong fairness: If a task is repeatedly enabled, then it is repeatedly executed.
I.e. if the task guard is infinitely often true, then the task is infinitely often executed.
nat x := 0; bool y:= 0
A: x := x + 1
B: even(x) →
y: = 1-y
Blinker

6. Expressing fairness assumptions in LTL: I
Fairness assumptions can be expressed in LTL!
Add a new variable taken that takes value ‘A’, ‘B’
Weak fairness: 𝐅𝐆 𝑔𝑢𝑎𝑟𝑑_𝑖 ⇒ (𝐆𝐅(taken = 𝑇 𝑖 ))
Task A: 𝑔𝑢𝑎𝑟𝑑_𝐴 is 𝑡𝑟𝑢𝑒, so this simplifies to: wf(A) ≔ 𝐆𝐅(taken=A)
Task B: wf(B) ≔ 𝐅𝐆 (even(x)) ⇒ 𝐆𝐅 (taken=B)
Does (wf(A)∧ wf(B)) ⇒ 𝐅 (x ≥ 10)?
Yes!
Does (wf(A)∧ wf(B)) ⇒ 𝐅 y?
No!
nat x := 0; bool y := 0
A: x := x + 1; taken≔ A
B: even(x) →
y: = 1-y; taken ≔ B
{A,B,∅} taken ≔ ∅
Blinker

7. Expressing fairness assumptions in LTL: II
Strong fairness: 𝐆𝐅 𝑔𝑢𝑎𝑟𝑑_𝑖 ⇒ (𝐆𝐅(taken = 𝑇 𝑖 ))
Task A: 𝑔𝑢𝑎𝑟𝑑_𝐴 is 𝑡𝑟𝑢𝑒, so this simplifies to: sf(A) ≔ 𝐆𝐅(taken=A)
Task B: sf(B) ≔𝐆𝐅 (even(x)) ⇒ 𝐆𝐅 (taken=B)
Does (sf(A)∧ sf(B)) ⇒ 𝐅 (x ≥ 0)?
Yes!
Does (sf(A)∧ sf(B)) ⇒ 𝐅 y?
nat x := 0; bool y := 0
A: x := x + 1; taken≔ A
B: even(x) →
y: = 1-y; taken ≔ B
{A,B,∅} taken ≔ ∅
Blinker
If a process satisfies a liveness requirement under strong fairness, it satisfies it under weak fairness: strong fairness is a stronger formula than weak fairness

8. Monitors
A safety monitor classifies system behaviors into good and bad
Safety verification can be done using inductive invariants or analyzing reachable state space of the system
A bug is an execution that drives the monitor into an error state
Can we use a monitor to classify infinite behaviors into good or bad?
Yes, using theoretical model of Büchi automata proposed by J. Richard Büchi in 1960

9. Büchi automaton Example 1
Extension of finite state automata to accept infinite strings
𝑞 0
𝑞 𝑓
𝐴 1
𝑥= 0
𝑥= 1
States 𝑄: { 𝑞 0 , 𝑞 𝑓 }
Input variable 𝑥 with domain Σ: 0,1
Final state: { 𝑞 𝑓 }
Transitions: (as shown)
Given trace 𝜌 (infinite sequence of symbols from Σ), 𝜌 is accepted by 𝐴 1 , if 𝑞 𝑓 appears inf. often
𝑥= 1
𝑥= 0
What is the language of 𝐴 1 ?
LTL formula 𝐆𝐅(𝑥=1)

10. Büchi automaton Example 2
𝑞 0
𝑞 𝑓
𝐴 2
𝑥= 0|1
𝑥=1
Note that this is a nondeterministic Büchi automaton
𝐴 2 accepts 𝜌 if there exists a path along which a state in 𝐹 appears infinitely often
What is the language of 𝐴 2 ?
LTL formula 𝐅𝐆(𝑥=1)
𝑄: 𝑞 0 , 𝑞 𝑓 , Σ: 0,1 , 𝐹: { 𝑞 𝑓 }
Transitions: (as shown)
Fun fact: there is no deterministic Büchi automaton that accepts this language

11. Büchi automaton Example 3
𝑞 1
𝐴 3
𝑥= 0 |
𝑦=1
𝑥= 1 & 𝑦=0
What is the language of 𝐴 3 ?
LTL formula:
𝐆 𝑥=1 ⇒𝐅(𝑦=1)
I.e. always when 𝑥=1 , in some future step, (𝑦=1)
In other words, (𝑥=1) must be followed by (𝑦=1)
𝑦=0
𝑞 0
𝑦=1
𝑄: 𝑞 0 , 𝑞 1 , Σ: 0,1 , 𝐹: { 𝑞 𝑓 }
Transitions: (as shown)

12. Using Büchi monitors
Theoretical result: Every LTL formula 𝜑 can be converted to a Büchi monitor/automaton 𝐴 𝜑
Size of 𝐴 𝜑 is generally exponential in the size of 𝜑; blow-up unavoidable in general
Construct composition of the original process 𝑃 and the Büchi monitor 𝐴 𝜑
If there are cycles in the composite process that do not visit the states specified by the liveness property, then we have found a violation.
Reachable cycles in process composition correspond to counterexamples to liveness properties
Implemented in many verification tools (e.g. the SPIN model checker developed at NASA JPL)

13. Computation Tree Logic
LTL was a linear-time logic where we reason about traces
CTL is a logic where we reason over the tree of executions generated by a program, also known as the computation tree
We care about CTL because:
There are some properties that cannot be expressed in LTL, but can be expressed in CTL: From every system state, there is a system execution that takes it back to the initial state (also known as the reset property)
To understand pCTL (Probabilistic CTL), it’s good if you understand CTL 
Can express interesting properties for multi-agent systems

14. Computation Tree
nat x := 0; bool y:= 0
A: x := (x + 1) mod 2
B: even(x) → y: = 1-y
(0,0)
We saw computation trees when understanding semantics of asynchronous processes
Basically a tree that considers “all possibilities” in a reactive program
(0,1)
Process
(1,0)
(0,0)
(1,0)
(0,1)
(1,1)
(0,0)
(0,0)
(1,1)
Finite State machine

15. CTL Syntax Syntax of CTL 𝜑 ∷= 𝑝 ¬𝜑 𝜑∧𝜑 | 𝐄𝐗𝜑 𝐄𝐅𝜑 𝐄𝐆𝜑 𝐄 𝜑 𝐔 𝜑 𝐀𝐗𝜑 𝐀𝐅𝜑
𝑝 ¬𝜑 𝜑∧𝜑 |
Prop. in 𝐴𝑃, negation, conjunction
𝐄𝐗𝜑
Exists NeXt Step
𝐄𝐅𝜑
Exists a Future Step
𝐄𝐆𝜑
Exists an execution where Globally in all steps
𝐄 𝜑 𝐔 𝜑
Exists an execution where in all steps Until in some step
𝐀𝐗𝜑
In All NeXt Steps
𝐀𝐅𝜑
In All possible future paths, there is a future step
𝐀𝐆𝜑
In All possible future paths, Globally in all steps
𝐀 𝜑 𝐔 𝜑
In All possible future executions, in all steps Until in some step

16. CTL semantics 𝐀 𝐅 𝑝 For All executions Eventually/In Some Future step
Path properties: properties of any given path or execution in the program
Quantification over runs: Checking if a property holds over all paths or over some path
Example CTL operator:
𝐀 𝐅 𝑝
For All executions
Eventually/In Some Future step

17. CTL Semantics through examples
𝑞 0
𝑞 1
𝑞 2 𝑝
𝐄𝐗 𝑝
𝑞 0
𝑞 1
𝑞 2 𝑝
𝐀𝐗 𝑝∧𝐄𝐗 𝑞
𝑞 0
𝑞 1
𝑞 2 𝑝
𝐀𝐗 𝑝 𝑝 𝑞

18. CTL semantics through examples
𝑞 0 𝑝
𝑞 0 𝑝
𝐀𝐅 𝑝: Along all
Paths, There is some future step where 𝑝 holds
𝐄𝐅 𝑝: Along some
path, there is some future step where 𝑝 holds 𝑝

19. CTL semantics through examples
𝐀𝐆 𝑝: Across all
paths, and for every successor in the path, 𝑝 holds
𝑞 0
𝑞 0 𝑝 𝑝 𝑝 𝑝 𝑝 𝑝
𝐄𝐆 𝑝: Along some
path, 𝑝 always holds 𝑝 𝑝

20. CTL Operator fun
𝐀𝐆𝐄𝐅 𝑝
𝐀𝐆𝐀𝐅 𝑝
𝐄𝐆𝐀𝐅 𝑝
𝐀𝐆 (𝑝⇒𝐄𝐗 𝑞)

21. CTL advantages and limitations
Checking if a given state machine (program) satisfies a CTL formula can be done quite efficiently (linear in the size of the machine and the property)
Native CTL cannot express fairness properties
Extension Fair CTL can express fairness
CTL* is a logic that combines CTL and LTL: You can have formulas like 𝐀𝐆𝐅 𝑝
CTL: Less used than LTL, but an important logic in the history of temporal logic

22. Probabilistic CTL LTL Can be interpreted over individual executions
Can be interpreted over a state machine: do all paths satisfy property
CTL
Is interpreted over a computation tree
PCTL
Is interpreted over a discrete-time Markov chain
Encodes uncertainties in computation due to environment etc.

23. Probabilistic CTL Syntax of PCTL 𝜑 ∷= 𝑝 ¬𝜑 𝜑∧𝜑 | (State) 𝑃 ∼𝜆 𝜓 𝜓∷= 𝐗𝜑
𝑝 ¬𝜑 𝜑∧𝜑 |
Prop. in 𝐴𝑃, negation, conjunction
(State)
𝑃 ∼𝜆 𝜓
∼∈{<,≤,>,≥}, 𝜆∈[0,1] : Probability of 𝜓 being true
𝜓∷= 𝐗𝜑
NeXt Time
(Path)
𝜑 𝐔 ≤𝑘 𝜑
Bounded Until (upto 𝑘 steps)
𝜑 𝐔 𝜑
Until (Recall 𝐅𝜑=𝑡𝑟𝑢𝑒 𝐔 𝜑, and 𝐆𝜑=¬𝐅¬𝜑
PCTL formulas are state formulas, path formulas used to define how to build a PCTL formula

24. Semantics
Semantics of path formulas is straightforward (similar to LTL/CTL)
Semantics of state formula with Probabilistic operator:
𝑃𝑟𝑜𝑏 𝑞,𝐗𝜑 : 𝑞′⊨𝜑 𝑃 𝑞, 𝑞 ′
Does 𝑃 ≥0.5 𝐗 𝑝 hold in state q 0 ?
No, because 𝑃 𝑞 0 ,𝐗 𝑝 = =0.3
Semantics of state formula with Until 𝑃𝑟𝑜𝑏 𝑞,𝛼 𝐔 ≤𝒌 𝛽 :
1 if 𝑞⊨𝛽, otherwise
𝑞 ′ ⊨𝛼 ∧ 𝑞 ′ ⊭𝛽 𝑃 𝑞, 𝑞 ′ .𝑃𝑟𝑜𝑏( 𝑞 ′ ,𝛼 𝑈 𝑘−1 𝛽 ) for 𝑘≥0
𝑞 0
𝑞 2
𝑞 1
𝑞 3
0.4
0.1
0.2 𝑝 ¬𝑝

25. PCTL 𝑟: Checking cellphone 𝑝: Feeling sleepy 0.3 𝑝, 𝑟 ¬𝑝, 0.1 ¬𝑟 0.5
Does this formula 𝑃 ≥0.5 𝐗𝑝 hold in state Brake?
Yes
Value of 𝜖? 𝑃 ≥𝜖 ( 𝐅 ≤2 𝑟)
Compute 𝑃𝑟𝑜𝑏(𝑞, 𝐅 ≤2 𝑟) for all 𝑞, pick smallest
𝑃 𝐴,𝐵 +𝑃 𝐴,𝐶 +𝑃 𝐴,𝐴,𝐵 + 𝑃 𝐴,𝐴,𝐶
= * *0.2
= 0.91
𝜖=0.91
I.e. with probability ≥ 0.91, driver checks cell phone within 2 steps
0.2 1
Accelerate
Constant Speed
Idling
Brake
0.5
0.3
0.8
0.05
0.4
¬𝑝, ¬𝑟
𝑝,𝑟
𝑝, 𝑟
¬𝑝,𝑟
0.1

26. Quantitative in PCTL vs. Qualitative in CTL
Toss a coin repeatedly until “tails” is thrown
Is “tails” eventually thrown along all paths?
CTL: AF tails
Result: false
Why? 𝑞 0 𝑞 1 𝑞 0 𝑞 1 …
Is the probability of eventually thrown “tails” equal to 1?
PCTL: 𝑃 ≥1 ( 𝐅 𝑡𝑎𝑖𝑙𝑠 )
Result: true
Probability of path 𝑞 0 𝑞 1 𝑞 0 𝑞 1 … is zero! 1
𝑞 1
ℎ𝑒𝑎𝑑𝑠
0.5
𝑞 0
0.5 1
𝑞 2
𝑡𝑎𝑖𝑙𝑠

27. Bibliography
CTL: Automated Temporal Reasoning about Reactive Systems.
PCTL:
PCTL heads/tails example: prob%20logics.pdf