1. Metrics for real time probabilistic processes
Radha Jagadeesan, DePaul University
Vineet Gupta, Google Inc
Prakash Panangaden, McGill University
Josee Desharnais, Univ Laval

2. Outline of talk Models for real-time probabilistic processes
Approximate reasoning for real-time probabilistic processes

3. Discrete Time Probabilistic processes
Labelled Markov Processes
For each state s
For each label a
K(s, a, U)
Each state labelled
with propositional
information
0.3
0.5
0.2

4. Discrete Time Probabilistic processes
Markov Decision Processes
For each state s
For each label a
K(s, a, U)
Each state labelled
with numerical rewards
0.3
0.5
0.2

5. Discrete time probabilistic proceses
+ nondeterminism : label does not determine probability distribution uniquely.

6. Real-time probabilistic processes
Add clocks to Markov processes Each clock runs down at fixed rate r c(t) = c(0) – r t Different clocks can have different rates
Generalized SemiMarkov Processes Probabilistic multi-rate timed automata

7. Generalized semi-Markov processes.
Each state labelled
with propositional
Information
Each state has a set
of clocks associated
with it.
{c,d} s u
{c}
{d,e} t

8. Generalized semi-Markov processes.
Evolution determined by
generalized states
<state, clock-valuation>
<s,c=2, d=1> Transition enabled when a clock becomes zero
{c,d} s u
{c}
{d,e} t

9. Generalized semi-Markov processes.
<s,c=2, d=1> Transition enabled in
1 time unit
<s,c=0.5,d=1> Transition enabled in
0.5 time unit
{c,d} s u
{c}
{d,e} t
Clock c
Clock d

10. Generalized semi-Markov processes.
Transition determines: a. Probability distribution on next states b. Probability distribution on
clock values for new clocks
{c,d} s
0.2
0.8 u
{c}
{d,e} t
Clock c
Clock d

11. Generalized semi Markov proceses
If distributions are continuous and states are finite: Zeno traces have measure 0
Continuity results. If stochastic processes from <s, > converge to the stochastic process at <s, >

12. Equational reasoning Establishing equality: Coinduction
Distinguishing states: Modal logics
Equational and logical views coincide
Compositional reasoning: ``bisimulation is a congruence’’

13. Labelled Markov Processes PCTL Bisimulation [Larsen-Skou,
Desharnais-Panangaden-Edalat]
Markov Decision Processes
Bisimulation [Givan-Dean-Grieg]
Labelled Concurrent Markov Chains
PCTL [Hansson-Johnsson]
Labelled Concurrent Markov chains (with tau)
PCTL [Desharnais-Gupta-Jagadeesan-Panangaden]
Weak bisimulation
[Philippou-Lee-Sokolsky,
Lynch-Segala]

14. With continuous time Continuous time Markov chains
CSL [Aziz-Balarin-Brayton-Sanwal-Singhal-S.Vincentelli]
Bisimulation,Lumpability
[Hillston, Baier-Katoen-Hermanns]
Generalized Semi-Markov processes Stochastic hybrid systems
CSL
Bisimulation:?????
Composition:?????

15. Alas!

16. Instability of exact equivalenceVs Vs

17. Problem!
Numbers viewed as coming with an error estimate. (eg) Stochastic noise as abstraction Statistical methods for estimating numbers

18. Problem! Numbers viewed as coming with an error estimate.
Reasoning in continuous time and continuous space is often via discrete approximations. eg. Monte-Carlo methods to approximate probability distributions by a sample.

19. Idea: Equivalence metrics
Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell
Replace equality of processes by (pseudo)metric distances between processes
Quantitative measurement of the distinction between processes.

20. Criteria on approximate reasoning
Soundness
Usability
Robustness

21. Criteria on metrics for approximate reasoning
Soundness
Stability of distance under temporal evolution: ``Nearby states stay close '‘ through temporal evolution.

22. ``Usability’’ criteria on metrics
Establishing closeness of states: Coinduction.
Distinguishing states: Real-valued modal logics.
Equational and logical views coincide: Metrics yield same distances as real-valued modal logics.

23. ``Robustness’’ criterion on approximate reasoning
The actual numerical values of the metrics should not matter ``upto uniformities’’.

24. Uniformities (same)
m(x,y) = |x-y|
m(x,y) = |2x + sinx
-2y – siny|

25. Uniformities (different)
m(x,y) = |x-y|

26. Our results

27. Our results
For Discrete time models: Labelled Markov processes Labelled Concurrent Markov chains Markov decision processes
For continuous time: Generalized semi-Markov processes

28. Results for discrete time models
Bisimulation
Metrics
Logic
(P)CTL(*)
Real-valued modal logic
Compositionality
Congruence
Non-expansivity
Proofs
Coinduction

29. Results for continuous time models
Bisimulation
Metrics
Logic
CSL
Real-valued modal logic
Compositionality
???
Proofs
Coinduction

30. Metrics for discrete time probablistic processes

31. Bisimulation
Fix a Markov chain. Define monotone F on equivalence relations:

32. Defining metric: An attempt
Define functional F on metrics.

33. Metrics on probability measures
Wasserstein-Kantorovich
A way to lift distances from states to a distances on distributions of states.

34. Metrics on probability measures

35. Metrics on probability measures

36. Example 1: Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y) x y

37. Example 1: Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y) x y

38. Example 2: Metrics on probability measures

39. Example 2: Metrics on probability measures
THEN:

40. Lattice of (pseudo)metrics

41. Defining metric coinductively
Define functional F on metrics
Desired metric is maximum fixed point of F

42. Real-valued modal logic

43. Real-valued modal logic
Tests:

44. Real-valued modal logic (Boolean)q q

45. Real-valued modal logic

46. Results
Modal-logic yields the same distance as the coinductive definition
However, not upto uniformities since glbs in lattice of uniformities is not determined by glbs in lattice of pseudometrics.

47. Variant definition that works upto uniformities
Fix c<1. Define functional F on metrics
Desired metric is maximum fixed point of F

48. Reasoning upto uniformities
For all c<1, get same uniformity
[see Breugel/Mislove/Ouaknine/Worrell]
Variant of earlier real-valued modal logic incorporating discount factor c characterizes the metrics

49. Metrics for real-time probabilistic processes

50. Generalized semi-Markov processes.
Evolution determined by
generalized states
<state, clock-valuation>
: Set of generalized states
{c,d} s u
{c}
{d,e} t
Clock c
Clock d

51. Generalized semi-Markov processes.
{c,d}
Path:
Traces((s,c)): Probability distribution
on a set of paths. s u
{c}
{d,e} t
Clock c
Clock d

52. Accomodating discontinuities: cadlag functions
(M,m) a pseudometric space cadlag if:

53. Countably many jumps, in general

54. Defining metric: An attempt
Define functional F on metrics. (c <=1)
traces((s,c)), traces((t,d)) are distributions on sets of cadlag functions.
What is a metric on cadlag functions???

55. Metrics on cadlag functionsx y
are at distance 1 for unequal x,y
Not separable!

56. Skorohod metrics (J2)
(M,m) a pseudometric space. f,g cadlag with range M.
Graph(f) = { (t,f(t)) | t \in R+}

57. Skorohod J2 metric: Hausdorff distance between graphs of f,g
(t,f(t))
f(t)
g(t) t

58. Skorohod J2 metric
(M,m) a pseudometric space. f,g cadlag

59. Examples of convergence to

60. Example of convergence
1/2

61. Example of convergence
1/2

62. Examples of convergence
1/2

63. Examples of convergence
1/2

64. Examples of non-convergence
Jumps are detected!

65. Non-convergence

66. Non-convergence

67. Non-convergence

68. Non-convergence

69. Summary of Skorohod J2
A separable metric space on cadlag functions

70. Defining metric coinductively
Define functional on 1-bounded pseudometrics (c <=1)
a. s, t agree on all propositions b.
Desired metric: maximum fixpoint of F

71. Real-valued modal logic

72. Real-valued modal logic

73. Real-valued modal logic
h: Lipschitz operator on unit interval

74. Real-valued modal logic

75. Real-valued modal logic
Base case for path formulas??

76. Base case for path formulas
First attempt:
Evaluate state formula F on state
at time t
Problem: Not smooth enough wrt time since
paths have discontinuities

77. Base case for path formulas
Next attempt:
``Time-smooth’’ evaluation of state
formula F at time t on path
Upper Lipschitz approximation to evaluated
at t

78. Real-valued modal logic

79. Non-convergence

80. Illustrating Non-convergence
1/2
1/2

81. Results
For each c<=1, modal-logic yields the same distance as the coinductive definition
All c<1 yield the same uniformity. In this case, construction can be carried out in lattice of uniformities.

82. Proof steps Continuity theorems (Whitt) of GSMPs yield separable basis
Finite separability arguments yield closure ordinal of functional F is omega.
Duality theory of LP for calculating metric distances

83. Results
Approximating quantitative observables: Expectations of continuous functions are continuous
Continuous mapping theorems for establishing continuity of quantitative observables

84. Summary
Approximate reasoning for real-time probabilistic processes

85. Results for discrete time models
Bisimulation
Metrics
Logic
(P)CTL(*)
Real-valued modal logic
Compositionality
Congruence
Non-expansivity
Proofs
Coinduction

86. Results for continuous time models
Bisimulation
Metrics
Logic
CSL
Real-valued modal logic
Compositionality
???
Proofs
Coinduction

87. Questions?

88. Real-valued modal logic