1. Chapter 4 The Maximum Principle: General Inequality Constraints

2. 4.1 Pure State Variable Inequality Constraints: Indirect Method It is common to require state variable to remain nonnegative, i.e., i.e., x i (t)  0, i=1,2,…,n. Constraints exhibiting (4.1) are called pure state variable inequality constraints. The general form is With (4.1): at any point where a component x i (t)>0, the corresponding constraint x i (t)  0 is not binding and can be ignored.

3. In any interval where x i (t)=0,we must have so that x i does not become negative. The control must be constrained to satisfy making f i  0, as a constraint of the mixed type (3.3) over the interval. We can add the constraint We associate multipliers η i with (4.3) whenever (4.3) must be imposed, i.e., whenever x i (t)=0. A convenient way to do this is to impose an “either or” condition η i x i =0. This will make η i =0 whenever x i >0.

4. We apply the maximum principle in (3.11) with additional necessary conditions satisfied by η and the modified transversality condition where  is a constant vector satisfying We can now form the Lagrangian where H is as defined in (3.7) and η=(η 1, η 2,…,η n ).

5. Since the constraints are adjoined indirectly (in this case via their first time derivative) to form the Lagrangian, the method is called the indirect adjoining approach. If on the other hand the Lagrangian L is formed by adjoining directly the constraints (4.1), i.e., where is a multiplier associated with (4.1), then the method is referred to as the direct adjoining approach.

6. Remark 4.1 The first two conditions in (4.5) are complementary slackness conditions on the multiplier η. The last condition is difficult to motivate. The direct maximum principle multiplier is related to η as The complementary slackness conditions for the direct multiplier are  0 and x*=0. Since  0, it follows that Example 4.1 Consider the problem:

7. Solution. The Hamiltonian is which implies the optimal control to be When x=0, we impose, in order to insure that (4.10) holds. Therefore, the optimal control on the state constraint boundary is Now we form the Lagrangian

8. where  1,  2, and  satisfy the complementary slackness conditions Furthermore, the optimal trajectory must satisfy From the Lagrangian we also get Let us first try (2)=  =0. Then the solution for is the same as in Example 2.2, namely,

9. Since (t)  -1 on [0,1] and x(0)=1>0, the original optimal control given by (4.11) is u*(t)=-1. Substituting this into (4.8) we get x(t)=1-t, which is positive for t<1. Thus In the time interval [0,1) by (4.14),  2 =0 since  *<1, and by (4.15)  =0 because x>0. Therefore,  1 (t)=- (t) =2-t >0 for 0  t<1, and this with u=-1 satisfies (4.13). At t=1 we have x(1)=0 so the optimal control is given by (4.12), which is u*(1)=0. Now assume that we continue to use the control u*(t)=0 in the interval 1  t  2.

10. Figure 4.1: State and Adjoint Trajectories in Example 4.1

11. With this control, we can solve (4.8) beginning with x(1)=0, and obtain x(t)=0 for 1  t  2. Since (t)  0 in the same interval, we see that u*(t)=0 satisfies (4.12) throughout this interval.To complete the solution, we calculate the Lagrange multipliers.since u*=0, we have  1 =  2 =0 throughout 1  t  2. Then from (4.16) we obtain  =- =2-t  0 which, with x=0, satisfies (4.15). This completes the solution. Remark 4.2 In instances where the initial state or the final state or both are on the constraint boundary, the maximum principle may degenerate I.e., there is no nontrivial solution of the necessary conditions,i.e., (t)  0, t  [0,T], where T is the terminal time.

12. 4.1.1 Jump Conditions There may be a piecewise continuous (t) satisfying a jump condition at a time at which the state trajectory hits its boundary value zero. Example 4.2 Consider Example 4.1 with T=3 and the terminal state constraint Clearly, the optimal control u* will be the one that keeps x as small as possible, subject to the state constraint (4.1) and the boundary condition x(0)=x(3)=1. Thus,

13. We only compute the adjoint function and multipliers that satisfy the optimality conditions. These are

14. 4.2 A Maximum Principle: Indirect Method Mixed constraints as in Chapter 3 and the pure state inequality constraints where h: E n x E 1  E p. By the definition of function h, (4.26) represents a set of p constraints h i (x,t)  0, i=1,2,…,p.It is noted that the constraint h i  0 is called a constraint of rth order if the rth time derivative of h i is the first time a term in control u appears in the expression by putting f(x,u,t) for after each differentiation.

15. In case of first order constraints, h 1 (x,u,t) is as follows: With respect to the ith constraint h i (x,t)  0, a sub Interval (  1,  2 )  [0,T] with  1 <  2 is called an interior interval if h i (x(t),t)>0 for all t  (  1,  2 ). If the optimal trajectory satisfies h i (x(t),t)=0 for for some i, then is called a boundary interval. An instant is called an entry time if there is an interior interval ending at t= and a boundary interval starting at. Correspondingly, is called an exit time if a boundary interval ends and an interior interval starts at.

16. If the trajectory just touches the boundary at time, i.e., and if the trajectory is in the interior just before and just after,then is called a contact time. Entry, exit and contact times are called junction times. Entry 11 11 ExitContact 11

17. Full rank condition on any boundary interval is as follows: where for and

18. To formulate the maximum principle for the problem with mixed constraints as well as first-order pure state constraints, we form the Lagrangian as H is defined in (3.7), u satisfies the complementary slackness conditions stated in (3.9), and  E q satisfies the conditions The maximum principle sates that the necessary conditions for u* to be an optimal control are that there exists multipliers, , , , ,and , and the jump parameters , which satisfy (4.29) that follows.

20. Note that the jump conditions on the adjoint variables in (4.29) generalize the jump condition on H in (4.29) requires that the Hamiltonian should be continuous at if h t = 0. Example 4.3 Consider the following problem with the discount rate   0:

21. Solution. As one can see from Figure 4.2 and 4.3, the optimal solution is: Figure 4.2: Feasible State Space and Optimal State Trajectory for Example 4.3

22. Figure 4.3: Adjoint Trajectory for Example 4.3

23. Note that at the entry time t=1 to the state constraint (4.34), the control u* and, therefore, h 1 *=u*+2(t-2) is discontinuous, i.e., the entry is non-tangential. on the other hand, u* and h 1 * are continuous at t=2 so that the exit is tangential. With u* and x* thus obtained, we must obtain,  1,  2, , and  so that the necessary optimality conditions (4.29) holds, i.e.,

24. From (4.41), we obtain (1 - )=e - . This with (4.38) gives:

25. and which, along with u* and x*, satisfy (4.29). Note, furthermore, that is continuous at the exit time t=2. At the entry time so that (4.30) also holds.

26. 4.3 Current-Value Maximum Principle: Indirect method With the Hamiltonian H as defined in (3.33), we can write The Lagrangian We can now state the current-value form of the maximum principle as given in (4.42)

28. 4.4 Sufficiency Conditions The sufficiency results can be stated in the indirect adjoining framework. In order to do so, let us define the Hamiltonian H and the Lagrangian L d in the direct method as where d,  d, and  d are multipliers in the direct formulation, corresponding to, , and  in the direct formulation.

29. Theorem 4.1 Let satisfy the necessary conditions in (4.29) and let If is concave in (x,u) at each t  [0,T], S in (3.2) is concave in x,g in (3.3) is quasiconcave in (x,u) h in (4.26) and a in (3.4) are quasiconcave in x, and b in (3.5) is linear in x, then (x*,u*) is optimal. It can be shown that:

30. Theorem 4.2 Theorem 4.1 remains valid if the concavity of in (x,u) at each t is replaced by the concavity of the maximized Hamiltonian Theorem 4.1 and Theorem 4.2 are written for finite horizon problems and remains valid if the transversality conditions on the adjoint variables (4.29) is replaced by the following limiting transversality condition in x, where

31. Example 4.1 (Continued) First we obtain the direct adjoint variable It is easy to see that is linear and hence concave in (x,u) at each t  [0,2].

32. Functions and are linear and hence quasiconcave in (x,u) and x, respectively. Functions S  0, a  0 and b  0 satisfy The conditions of Theorem 4.1 trivially.