1. CDC 2005 1 Relative Entropy Applied to Optimal Control of Stochastic Uncertain Systems on Hilbert Space Nasir U. Ahmed School of Information Technology and Engineering, and Department of Mathematics University of Ottawa, Ottawa Canada. and C. D. Charalambous ECE Department University of Cyprus, Nicosia, Cyprus. Also, School of Information Technology and Engineering, University of Ottawa, Ottawa Canada

2. 2 CDC 2005 Overview Notation and Problem Formulation Existence of Solution Examples Applications in Uncertain Stochastic Control System

3. CDC 2005 3 Notations and Problem Formulation

4. 4 CDC 2005 Notations is a Polish space. is the sigma algebra of Borel sets generated by the metric topology. is the space of countably additive regular probability measures defined on. is furnished with the weak topology. This topology is metrizable with the Prohorv metric which makes it complete. The relative entropy between two probability measures and is given by

5. 5 CDC 2005 Notations Let, be an extended real valued continuous function. Let denote the set of measures induced by a controlled stochastic system under the assumptions that the system is perfectly known. For each fixed and, define the set

6. 6 CDC 2005 Problem Formulation Basic Problem. The problem is to find a that minimizes the functional over the set. In other words we wish to find a solution to the min-max problem

7. CDC 2005 7 Existence of the Solution

8. 8 CDC 2005 Existence of the Solution Lemma 1. Let be an upper semi continuous function and strictly concave. Then for every, there exists a unique at which attains its supremum. Lemma 2. Suppose the assumptions of the above Lemma hold. Then, the graph of the multifunction is sequentially closed and the function is lower semi continuous.

9. 9 CDC 2005 Existence of the Solution Theorem 3. Consider the problem and suppose that is upper/lower semi continuous and strictly concave/convex and the set is compact with respect to Prohorov topology. Then the problem has a solution. Problem. Consider the following problem Define and find such that Corollary 4. Let be upper semi continuous and strictly concave,, and a compact subset of. Then, the above stated problem has solution.

10. CDC 2005 10 Examples

11. 11 CDC 2005 (E1) Target Seeking Problem Example 1. Let be a desired measure. we wish to find a that approximates as closely as possible. We consider the following objective functional Disregarding the uncertainty, and assuming is a compact set, attains its minimum on. Let be minimizer, that is,. But since the system is uncertain, the actual law in

12. 12 CDC 2005 (E1) Target Seeking Problem force may be any element from and one may face the worst situation, Clearly, with. So instead of choosing, we may choose one that minimizes the maximum distance. That is, we choose an element so that where Clearly

13. 13 CDC 2005 (E2) Evasion Problem Example 2. Let be a family of disjoint sets and suppose we wish to find a measure from the attainable set that has minimum concentration of mass on these sets. This represents an evasion problem. Let be any bounded continuous function, increasing in all its arguments. Define the functional as follows, where denotes the indicator function of the set

14. 14 CDC 2005 (E2) Evasion Problem Disregarding uncertainty, and noting that is continuous, and compact, there exists a at which attains its minimum,. Considering the uncertainty at we can find Such that

15. 15 CDC 2005 (E3) Hitting Problem Example 3. Let be a family of open subsets of and an increasing function of all its arguments. Consider the functional given by The objective is to find from the attainable set of measures that maximizes the concentration of mass on the given sets. This is equivalent to hitting the target with maximum probability.

16. 16 CDC 2005 (E3) Hitting Problem Due to the presence of uncertainty, we must maximize the minimum concentration giving rise to the sup-inf problem By virtue of mentioned theorems, it can be shown that the functional is lower semi continuous on in Prohorov topology.

17. CDC 2005 17 Application in Uncertain Stochastic Control Systems

18. 18 CDC 2005 Problem Formulation Perturbed System. The uncertain system is given by We assume throughout that is a measurable multifunction in the sense that for every the set. Nominal System. The nominal system is given by

19. 19 CDC 2005 Problem Formulation Theorem. Consider the perturbed system and suppose that is a measurable multifunction mapping to and there exists a finite positive number such that Then for each, the system has a nonempty set of martingale solutions denoted by Further, there exists a finite positive number such that

20. 20 CDC 2005 Problem Formulation (Pc): Find a control at which the following inf-sup is attained Defining Our problem can be compactly presented as follows. Find that minimizes the functional given by so that,

21. 21 CDC 2005 Solution For this problem one has to choose, and the multifunction as given below Then min-max problem (Pc) has a solution.

22. 22 CDC 2005 Solution Lemma. For a given, for some such that a measurable function bounded from below the following statement hold Moreover, if the supremum is attained at