1. 1 TTK4135 Optimization and control B.Foss Spring semester 2005 TTK4135 Optimization and control Spring semester 2005 Scope - this you shall learn Optimization - important concepts and theory Formulating an engineering problem into an optimization problem Solving an optimization problem - algorithms, coding and testing Course information Lectures are given by professor Bjarne A. Foss The course assistant is Mr. K. Rambabu.

2. 2 TTK4135 Optimization and control B.Foss Spring semester 2005 Course information All course information is provided on the web-pages for the course: www.itk.ntnu.no/fag/TTK4135. There will be no hand- out of material. Every student must access the course web-pages at least every week to keep updated course information (eg. changes in lecture times, information on mid-term exam) All students should subscribe to the email-list: 4135-optreg The deadlines for all assignments (“øvinger” and the helicopter lab. report) are absolute. There will be 1-2 “øvingstimer” with assistants present ahead of the deadline for every assignment. A minimum number of “øvinger” and the helicopter lab.report must be approved to enter the final examination. I will not cover the complete curriculum in my lectures; rather focus on the most important and difficult parts.

3. 3 TTK4135 Optimization and control B.Foss Spring semester 2005 Grading The final exam counts 70% on the final grade The mid-term exam is graded. It counts 15% on the final grade. Please note that only this semester’s mid-term exam counts. A mid-term grade from last year will not be acknowledged. The project report (based on the helicopter laboratory) is graded. It counts 15% on the final grade. Please note that only this semester’s report counts. A report grade from an earlier year will not be acknowledged. To ensure participation from all students 4 groups will be selected for an oral presentation of their laboratory work. This presentation will influence the grade on the report. Finally I welcome constructive criticism on all aspects of the course, including my lectures.

4. 4 TTK4135 Optimization and control B.Foss Spring semester 2005 Preliminary lecture plan The content of each lecture is specified in the following slides. All lectures are given in lecture halls EL 3 and EL 6. The mid-term examination is on 2004-03-11. The final examination is on 2004-05-23.

5. 5 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #1 - 2004-01-10 Optimization problems appear everywhere Stock portfolio management Resource allocation (airline companies, transport companies, oil well allocation problem) Optimal adjustment of a PID-controller Formulating an optimization problem: From an engineering problem to a mathematical description. Case: a realistic production planning problem Defining an optimization problem Definition of important terms Convexity and non-convexity Global vs. local solution Constrained vs. unconstrained problems Feasible region Reference: Chapter 1 in Nocedal and Wright (N&W)

6. 6 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #2 - 2004-01-14 Karush Kuhn-Tucker (KKT) conditions Sensitivities and Lagrange-multipliers Reference: Chapter 12.1, 12.2 in N&W

7. 7 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #3 - 2004-01-17 Linear algebra (App. A.2 in N&W) Norms of vectors and matrices Positive definit and indefinite matrices Condition number, well-conditioned and ill-conditioned linear equations Subspaces; null space and range space of a matrix Eigenvalue and singular-value decomposition Matrix factorization: Cholesky factorization, LU factorization Sequences (App.A.1, Ch.2.2 “Rates of …” in N&W) Convergence to some points; convergence rate; order notation Sets (App.A.1 in N&W) Open, closed, bounded sets Functions (App.A.1 in N&W) Continuity, Lipschitz continuity Directional derivatives

8. 8 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #4 + #5 - 2004-01-21/28 Linear programming - LP Mathematical formulation Condition for optimality - the Karush-Kuhn-Tucker (KKT) conditions Basic solutions - basis for the Simplex method The Simplex method Understanding the solution - Lagrange variables The dual problem Obtaining an initial feasible solution Efficiency of algorithms LP example - production planning Reference: Ch.12.2,13-13.5 in textbook

9. 9 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #6 - 2004-01-31 Quadratic programming - QP Mathematical formulation Convex vs. non-convex problems Condition for optimality - KKT conditions Special case: No inequality conditions Reduced space methods The active-set method for convex problems Understanding the solution - Lagrange variables The dual problem Obtaining an initial feasible solution Efficiency of algorithms QP example - production planning (varying sales price) Reference: Ch.12.2,16.1-16.4,(16.5),16.8 in textbook

10. 10 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #7 - 2004-02-04 Quadratic programming - QP The active-set method for convex problems The active-set method for non-convex problems QP example - production planning (varying sales price) Reference: 16.4,16.5,16.8 in textbook

11. 11 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #8 - 2004-02-07 Quadratic programming - QP The active-set method for non-convex problems Reference: 16.5,16.8 in textbook

12. 12 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #9 - 2004-02-11 Repetition of LP, QP --- Optimality conditions Necessary and sufficient conditions for optimality Iterative solution methods Starting point Search direction Step length Termination criteria Convergence Reference: 2.1, 2.2 in textbook

13. 13 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #10 - 2004-02-14 Line search methods Choice of Wolfe-conditions Back-tracking Curve-fit and interpolation Convergence of line-search methods - Theorem 3.2 Convergence rate Reference: 3.1-3.4 in textbook

14. 14 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #11 - 2004-02-18 Practical Newton-methods Approximate Newton-step Line search Newton Modified Hessian Reference: 6 - 6.3 in textbook Computing gradients Reference: 7 - 7.1 in textbook

15. 15 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #12 - 2004-02-21 Quasi Newton methods DFP and BFGS methods Rosenbrock example for illustration Reference: 8 - 8.1 in textbook Information on the mid.term examination

16. 16 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #13 - 2004-03-07 Mid-term examination

17. 17 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #14 - 2004-04-01 Mid-term examination - once again Model Predictive Control (MPC) The MPC principle Formulation of linear MPC Formulating the optimisation problem which is a QP-problem Reference: Ch.1 and 2 – Note on MPC by M.Hovd

18. 18 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #15 - 2004-04-04 Linear Quadratic Control (LQ-control) Formulation of the LQ-problem Finite horizon LQ-control Reference: Ch.1-1.2 - Note on LQ-control by B.Foss

19. 19 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #16 - 2004-04-08 Linear Quadratic Control (LQ-control) Infinite horizon LQ-control State-estimation (repetition from TTK4115) Reference: Ch.1.3-1.4 - Note on LQ-control by B.Foss

20. 20 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #17 - 2004-04-11 Model Predictive Control (MPC) Feasibility and constraint handling Target calculation Robustness Reference: Ch.4 – 6, 8, 9 – Note on MPC by M.Hovd

21. 21 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #18 - 2004-04-18 Nonlinear programming - SQP Line-search in nonlinear programming l 1 exact merit function Exact merit function Reference: 15.3,18.5,18.6 in textbook

22. 22 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #19 - 2004-04-25 Nonlinear programming - SQP Computing the search direction Solving nonlinear equtions Quasi-Newton method for computing the Hessian Reference: 11.1,18.1-18.4,18.6 in textbook

23. 23 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #20 - 2004-05-02 Nonlinear programming - SQP Reduced Hessian methods Convergence rate Maratos effect Reference: 18.7,18.10,18.11 in textbook

24. 24 TTK4135 Optimization and control B.Foss Spring semester 2005 Content of Lecture #21 - 2004-05-09 SQP – final remarks including examples Repetition Repetition of main topics Course evaluation ----------------------