2. What is Optimal Control Theory? Dynamic Systems: Evolving over time. Time: Discrete or continuous. Optimal way to control a dynamic system. Prerequisites: Calculus, Vectors and Matrices, ODE&PDE Applications: Production, Finance/Economics, Marketing and others.

3. Basic Concepts and Definitions A dynamic system is described by state equation: where x(t) is state variable, u(t) is control variable. The control aim is to maximize the objective function: Usually the control variable u(t) will be constrained as follows:

4. Sometimes, we consider the following constraints: (1) Inequality constraint (2) Constraints involving only state variables (3) Terminal state where X(T) is reachable set of the state variables at time T.

5. Formulations of Simple Control Models Example 1.1 A Production-Inventory Model. We consider the production and inventory storage of a given good in order to meet an exogenous demand at minimum cost.

6. Table 1.1 The Production-Inventory Model of Example 1.1

7. Example 1.2 An Advertising Model. We consider a special case of the Nerlove-Arrow advertising model.

8. Table 1.2 The Advertising Model of Example 1.2

9. Example 1.3 A Consumption Model. This model is summarized in Table 1.3:

10. Table 1.3 The Consumption Model of Example 1.3

11. History of Optimal Control Theory Calculus of Variations. Brachistochrone problem: path of least time Newton, Leibniz, Bernoulli brothers, Jacobi, Bolza. Pontryagin et al.(1958): Maximum Principle. Figure 1.1 The Brachistochrone problem

12. Notation and Concepts Used = “is equal to” or “is defined to be equal to” or “is identically equal to.” := “is defined to be equal to.”  “is identically equal to.”  “is approximately equal to.”  “implies.”  “is a member of.” Let y be an n-component column vector and z be an m-component row vector, i.e.,

13. when n = m, we can define the inner product

14. If is an m x k matrix and B={b ij } is a k x n matrix, C={c ij }=AB, which is an m x n matrix with components:

15. Differentiating Vectors and Matrices with respect to Scalars Let f : E 1  E k be a k-dimensional function of a scalar variable t. If f is a row vector, then If f is a column vector, then

16. Differentiating Scalars with respect to Vectors If F: E n x E m  E 1, n  2, m  2, then the gradients F y and F z are defined, respectively as;

17. Differentiating Vectors with respect to Vectors If F: E n x E m  E k, is a k – dimensional vector function, f either row or column, k  2; i,e; where f i = f i (y,z), y  E n is column vector and z  E m is row vector, n  2, m  2, then f z will denote the k x m matrix;

18. f y will denote the k x n matrix Matrices f z and f y are known as Jacobian matrices. Applying the rule (1.11) to F y in (1.9) and the rule (1.12) to F z in (1.10), respectively, we obtain F yz =(F y ) z to be the n x m matrix

19. and F zy = (F z ) y to be the m x n matrix

20. Product Rule for Differentiation Let x  E n be a column vector and g(x)  E n be a row vector and f(x)  E n be a column vector, then

21. Vector Norm The norm of an m-component row or column vector z is defined to be Neighborhood N zo of a point is where  > 0 is a small positive real number. A function F(z): E m  E 1 is said to be of the order o(z), if The norm of an m-dimensional row or column vector function z(t), t  [0, T], is defined to be

22. Some Special Notation left and right limits x k : state variable at time k. u k : control variable at time k. k : adjoint variable at time k, k=0,1,2,…,T. : difference operator.  x k := x k+1 - x k. x k *, u k *, and k, are quantities along an optimal path. discrete time (employed in Chapters 8-9)

23. sat function bang function

24. impulse control If the impulse is applied at time t, then we calculate the objective function J as

25. Convex Set and Convex Hull A set D  E n is a convex set if  y, z  D, py +(1-p)z  D, for each p  [0,1]. Given x i  E n, i=1,2,…,l, we define y  E n to be a convex combination of x i  E n, if  p i  0 such that The convex hull of a set D  E n is

26. Concave and Convex Function  : D  E 1 defined on a convex set D  E n is concave if for each pair y,z  D and for all p  [0,1], If  is changed to > for all y,z  D with y  z, and 0<p<1, then  is called a strictly concave function. If  (x) is a differentiable function on the interval [a,b], then it is concave, if for each pair y,z  [a,b],  (z)   (y)+  x (y)(z-y).

27. If the function  is twice differentiable, then it is concave if at each point in [a,b],  x x  0. In case x is a vector,  x x needs to be a negative definite matrix. If  : D  E 1 defined on a convex set D  E n is a concave function, then -  : D  E 1 is a convex function.

28. Figure 1.2 A Concave Function

29. Affine Function and Homogeneous Function of Degree One  : E n  E 1 is said to be affine, if  (x) -  (0) is linear.  : E n  E 1 is said to be homogeneous of degree one, if  (bx) = b  (x), where b is a scalar constant. Saddle Point  : E n x E m  E 1, a point  E n x E m is called a saddle point of  (x,y), if Note also that

30. Figure 1.3 An Illustration of a Saddle Point

31. Linear Independence and Rank of a Matrix A set of vectors a 1,a 2,…,a n  E n is said to be linearly dependent if  p i,, not all zero, such that If the only set of p i for which (1.25) holds is p 1 = p 2 = ….= p n = 0, then the vectors are said to be linearly independent. The rank of an m x n matrix A is the maximum number of linearly independent columns in A, written as rank (A). An m x n matrix is of full rank if rank (A) = n.