2. Chapter 10
Optimal Control
Homework 10
Consider again the control system as given before, described by
Assuming the linear control law
Determine the constants k1 and k2 so that the following performance index is minimized
Consider only the case where the initial condition is x(0)=[c 0]T and the undamped natural frequency (ωn) is chosen to be 2 rad/s.
Calculate the transfer function of the system if compensated with k
Determine the value of corresponding k (k1 or k2?) to obtain ωn as requested

3. Chapter 10
Optimal Control
Solution of Homework 10
Substituting the state feedback and finding the transfer function,

4. Chapter 10
Optimal Control
Solution of Homework 10

5. Algebraic Riccati Equation
Chapter 10
Optimal Control
Algebraic Riccati Equation
Consider again the n-dimensional state space equations:
but now with the following performance index to be minimized:
: symmetric, positive semidefinite
The control objective is to construct a stabilizing linear state feedback controller of the form u(t) = –K x(t) that at the same time minimizes the performance index J.
The state feedback equation u(t) = –K x(t) is also called the “control law.”

6. Algebraic Riccati Equation
Chapter 10
Optimal Control
Algebraic Riccati Equation
First, assume that there exists a linear state feedback optimal controller, such that the optimal closed-loop system:
is asymptotically stable.
Then, there exists a Lyapunov Function V = xT(t)P x(t) with a positive definite matrix P, such that dV/dt evaluated on the trajectories of the closed-loop system is negative definite.
The synthesis of optimal control law involves the finding of an appropriate Lyapunov Function, or equivalently, the matrix P.

7. Algebraic Riccati Equation
Chapter 10
Optimal Control
Algebraic Riccati Equation
The appropriate matrix P is found by minimizing:
For unconstrained minimization,
Optimal Solution
If u(t) = –K x(t) is so chosen that
min{f(u(t)) = dV/dt + xT(t)Q x(t) + uT(t)R u(t)} = 0
for some V = xT(t)P x(t),
Then the controller using u(t) as control law is an optimal controller.

8. Algebraic Riccati Equation
Chapter 10
Optimal Control
Algebraic Riccati Equation
The differentiation yields:
if P symmetric

9. Algebraic Riccati Equation: Control Law
Chapter 10
Optimal Control
Algebraic Riccati Equation: Control Law
Hence, incorporating the fact that P and R are symmetric, the optimal control law can be written as: or

10. Algebraic Riccati Equation: Minimum Test
Chapter 10
Optimal Control
Algebraic Riccati Equation: Minimum Test
We now need to perform the “Second Derivative Test” to find out whether u*(t) is a solution that minimizes f(u(t)).
Second Derivative Test
If f’(x) = 0 and f”(x) > 0 then f has a local minimum at x
If f’(x) = 0 and f”(x) < 0 then f has a local maximum at x
If f’(x) = 0 and f”(x) = 0 then no conclusion can be drawn
Performing the “Second Derivative Test”,
If the weight matrix R is chosen to be a positive definite matrix, then the optimal solution u*(t) is indeed a solution that minimizes f(u(t)).

11. Algebraic Riccati Equation: Finding P
Chapter 10
Optimal Control
Algebraic Riccati Equation: Finding P
Now, the appropriate matrix P must be found, in order to obtain the optimal closed-loop system in the form of:
The optimal controller with matrix P minimizes the cost function f(u(t)), and will yield: 
After some substitutions of x(t) and later u*(t),

12. Algebraic Riccati Equation (ARE)
Chapter 10
Optimal Control
Algebraic Riccati Equation: Finding P
After regrouping, we will obtain:
The equation above should hold for any x(t), which implies that:
Algebraic Riccati Equation (ARE)
After solving the ARE for P, the optimal control law given by:
can be applied to the linear system of
The solution of ARE does not depend on initial conditions

13. Solving Optimal Control Problem in Matlab
Chapter 10
Optimal Control
Solving Optimal Control Problem in Matlab
The solution of the ARE can be calculated easily in Matlab.
The command to be used is:
[K,P,E] = lqr(A,B,Q,R,N)
The performance index to be minimized is:
For the inputs: A and B are the system matrices, while Q, R, and N are the weight matrices. In our case, N is set to zero.
For the outputs, K is the optimal gain, P is the solution matrix of the ARE, while E is the eigenvalues of the optimal system.

14. Example 1: Optimal Control
Chapter 10
Optimal Control
Example 1: Optimal Control
Consider the following model:
along with the performance index:
Find the optimal control law for the system.
The matrices are:
The ARE is solved as:

15. Example 1: Optimal Control
Chapter 10
Optimal Control
Example 1: Optimal Control
The control law is:
The optimal closed-loop system is described by:
Some conclusions:
The system is a 1st order system with 2 inputs
The optimal location of the system’s pole is at s = –4/ 2r
Control effort u lightly weighted  r<<  pole location more to the left  system faster  energy expense increases
Control effort u heavily weighted  r>>  pole location more to the right  system slower  energy expense decreases

16. Give weight to x1(t) No restriction for x2(t)
Chapter 10
Optimal Control
Example 2: Optimal Control
Consider the following continuous-time system:
Design an optimal controller that minimizes
with
Give weight to x1(t) No restriction for x2(t)

17. Example 2: Optimal Control
Chapter 10
Optimal Control
Example 2: Optimal Control
P is found by solving the ARE:

18. Example 2: Optimal Control
Chapter 10
Optimal Control
Example 2: Optimal Control
Three equations can be obtained:
Thus, the optimal gain is given by:
The requested control law is:
Some conclusions: …

19. Example 2: Validation of Answer
Chapter 10
Optimal Control
Example 2: Validation of Answer
From the previous example, let us now set a = 1:
Taking the positive out of both plus-minus signs, P2 and P3 can be obtained. The solution of P will thus be:

20. Example 2: Validation of Answer
Chapter 10
Optimal Control
Example 2: Validation of Answer
The optimal feedback gain will be:
with:
The optimal closed-loop matrix is given by:

21. Example 2: Validation of Answer
Chapter 10
Optimal Control
Example 2: Validation of Answer
For r = 0.25, for example
Location of optimal poles

22. Example 2: Optimal State Feedback
Chapter 10
Optimal Control
Example 2: Optimal State Feedback
Changing the control effort weight r will change the optimal gain K and the location of the optimal poles E.

23. Example 2: Optimal State Feedback
Using x0 = [2;–1], the response of the system with 3 different weights r will now be compared.
Why x1, not x2?
Smaller r results in larger magnitude of u, since control effort is weighted less
Smaller r results in smaller magnitude of state x1, and faster convergence rate
: r = 1.4
: r = 0.05
: r = 0.25

24. Chapter 10
Optimal Control
Homework 11
The regulator shown in the figure below contains a plant that is described by
and has a performance index
Determine
The Riccati matrix P
The state feedback matrix k
The closed-loop eigenvalues

25. Homework 11A Consider the system described by the equations
Chapter 10
Optimal Control
Homework 11A
Consider the system described by the equations
(a) Determine the optimal control law u*(t) which minimizes the following performance index. (Hint: Use Algebraic Riccati Equation)
(b) Use ρ = 0.1·StID and calculate the numerical result of K. Verify your answer using Matlab. (Hint: Learn more about command “lqr”)
Deadline: 3 December 2014.