# What is Optimal Control Theory? Dynamic Systems: Evolving over time. Time: Discrete or continuous. Optimal way to control a dynamic system. Prerequisites:

## Presentation on theme: "What is Optimal Control Theory? Dynamic Systems: Evolving over time. Time: Discrete or continuous. Optimal way to control a dynamic system. Prerequisites:"— Presentation transcript:

What is Optimal Control Theory? Dynamic Systems: Evolving over time. Time: Discrete or continuous. Optimal way to control a dynamic system. Prerequisites: Calculus, Vectors and Matrices, ODE&PDE Applications: Production, Finance/Economics, Marketing and others.

Basic Concepts and Definitions A dynamic system is described by state equation: where x(t) is state variable, u(t) is control variable. The control aim is to maximize the objective function: Usually the control variable u(t) will be constrained as follows:

Sometimes, we consider the following constraints: (1) Inequality constraint (2) Constraints involving only state variables (3) Terminal state where X(T) is reachable set of the state variables at time T.

Formulations of Simple Control Models Example 1.1 A Production-Inventory Model. We consider the production and inventory storage of a given good in order to meet an exogenous demand at minimum cost.

Table 1.1 The Production-Inventory Model of Example 1.1

Example 1.2 An Advertising Model. We consider a special case of the Nerlove-Arrow advertising model.

Table 1.2 The Advertising Model of Example 1.2

Example 1.3 A Consumption Model. This model is summarized in Table 1.3:

Table 1.3 The Consumption Model of Example 1.3

History of Optimal Control Theory Calculus of Variations. Brachistochrone problem: path of least time Newton, Leibniz, Bernoulli brothers, Jacobi, Bolza. Pontryagin et al.(1958): Maximum Principle. Figure 1.1 The Brachistochrone problem

Notation and Concepts Used = “is equal to” or “is defined to be equal to” or “is identically equal to.” := “is defined to be equal to.”  “is identically equal to.”  “is approximately equal to.”  “implies.”  “is a member of.” Let y be an n-component column vector and z be an m-component row vector, i.e.,

when n = m, we can define the inner product

If is an m x k matrix and B={b ij } is a k x n matrix, C={c ij }=AB, which is an m x n matrix with components:

Differentiating Vectors and Matrices with respect to Scalars Let f : E 1  E k be a k-dimensional function of a scalar variable t. If f is a row vector, then If f is a column vector, then

Differentiating Scalars with respect to Vectors If F: E n x E m  E 1, n  2, m  2, then the gradients F y and F z are defined, respectively as;

Differentiating Vectors with respect to Vectors If F: E n x E m  E k, is a k – dimensional vector function, f either row or column, k  2; i,e; where f i = f i (y,z), y  E n is column vector and z  E m is row vector, n  2, m  2, then f z will denote the k x m matrix;

f y will denote the k x n matrix Matrices f z and f y are known as Jacobian matrices. Applying the rule (1.11) to F y in (1.9) and the rule (1.12) to F z in (1.10), respectively, we obtain F yz =(F y ) z to be the n x m matrix

and F zy = (F z ) y to be the m x n matrix

Product Rule for Differentiation Let x  E n be a column vector and g(x)  E n be a row vector and f(x)  E n be a column vector, then

Vector Norm The norm of an m-component row or column vector z is defined to be Neighborhood N zo of a point is where  > 0 is a small positive real number. A function F(z): E m  E 1 is said to be of the order o(z), if The norm of an m-dimensional row or column vector function z(t), t  [0, T], is defined to be

Some Special Notation left and right limits x k : state variable at time k. u k : control variable at time k. k : adjoint variable at time k, k=0,1,2,…,T. : difference operator.  x k := x k+1 - x k. x k *, u k *, and k, are quantities along an optimal path. discrete time (employed in Chapters 8-9)

sat function bang function

impulse control If the impulse is applied at time t, then we calculate the objective function J as

Convex Set and Convex Hull A set D  E n is a convex set if  y, z  D, py +(1-p)z  D, for each p  [0,1]. Given x i  E n, i=1,2,…,l, we define y  E n to be a convex combination of x i  E n, if  p i  0 such that The convex hull of a set D  E n is

Concave and Convex Function  : D  E 1 defined on a convex set D  E n is concave if for each pair y,z  D and for all p  [0,1], If  is changed to > for all y,z  D with y  z, and 0 { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/8/2313134/slides/slide_26.jpg", "name": "Concave and Convex Function  : D  E 1 defined on a convex set D  E n is concave if for each pair y,z  D and for all p  [0,1], If  is changed to > for all y,z  D with y  z, and 0

If the function  is twice differentiable, then it is concave if at each point in [a,b],  x x  0. In case x is a vector,  x x needs to be a negative definite matrix. If  : D  E 1 defined on a convex set D  E n is a concave function, then -  : D  E 1 is a convex function.

Figure 1.2 A Concave Function

Affine Function and Homogeneous Function of Degree One  : E n  E 1 is said to be affine, if  (x) -  (0) is linear.  : E n  E 1 is said to be homogeneous of degree one, if  (bx) = b  (x), where b is a scalar constant. Saddle Point  : E n x E m  E 1, a point  E n x E m is called a saddle point of  (x,y), if Note also that

Figure 1.3 An Illustration of a Saddle Point

Linear Independence and Rank of a Matrix A set of vectors a 1,a 2,…,a n  E n is said to be linearly dependent if  p i,, not all zero, such that If the only set of p i for which (1.25) holds is p 1 = p 2 = ….= p n = 0, then the vectors are said to be linearly independent. The rank of an m x n matrix A is the maximum number of linearly independent columns in A, written as rank (A). An m x n matrix is of full rank if rank (A) = n.

Download ppt "What is Optimal Control Theory? Dynamic Systems: Evolving over time. Time: Discrete or continuous. Optimal way to control a dynamic system. Prerequisites:"

Similar presentations