1. Adaptive Optimal Control of Nonlinear Parametric Strict Feedback Systems with application to Helicopter Attitude Control OBJECTIVES  Optimal adaptive control of nonlinear parametric strict feedback systems in continuous-time is considered when the system internal dynamics has uncertainties.  A new approach to use the Hamilton-Jacobi-Bellman (HJB) equation online for obtaining the optimal control is developed. The closed loop stability of the overall system will be shown.  The forward-in-time optimal control approach is applied to a underactuated helicopter attitude control system by minimizing a cost function. Student: Hassan Zargarzadeh, PhD Student, ECE Department BACKGROUND  Several optimal control approaches for linear and nonlinear systems are presented with offline techniques since 1985.  In the field of optimal control of nonlinear affine systems, a neural network- based (NN) online approach (Dierks and Jagannathan 2010) is proposed without an initial stabilizing controller.  Lewis et al. (2010) have used an actor-critic approach with an initial stabilizing controller to attain an optimality. Full dynamics are required.  Motivated by the helicopter dynamics which has system uncertainties, this work assumes that the internal dynamics of the nonlinear system are unknown and without an initial stabilizing controller. The Problem Statement  The nonlinear parametric strict feedback system can be expressed as  The objective is to minimize the cost function.  To this end, the controller should guarantee the stability while the control input satisfies the HJB equation: Faculty Advisor: Prof. Jag Sarangapani, ECE Department Application to the Helicopter Dynamics  The helicopter system is an example of a parametric strict feedback system whose parameters are uncertain and are given by  A brief discretion of the system parameters are given below.  The simulation results are given in the following figures. DISCUSSION Discussion  For proving the overall closed-loop stability, Lyapunov analysis is utilized and demonstrated (not shown here).  Fig. 1 Provides a 3-D view of the helicopter performing a landing maneuver, but in this case shows the actual versus desired trajectory throughout the maneuver.  Fig. 2 Shows the convergence of the tracking error.  Fig. 3 Demonstrates the helicopter's ability to follow a trajectory in two dimensions while hovering.  Fig. 4 Shows the helicopter's ability to track a the desired trajectory.  Fig. 5 Shows a block diagram for the control scheme of the system. CONCLUDING REMARKS  Proposed online optimal control of uncertain parametric strict feedback system provides an accurate and acceptable control performance.  The stability of the closed loop system is shown under the assumption that an admissible controller not required whereas it is required in many previous works.  The optimal controller is proven to be stable both analytically and on Trex500 helicopter model. Two neural networks are employed: 1) the cost function approximation neural network; 2) the internal dynamics estimator.  The simulation results show a significant optimal performance when the optimal controller is applied. FUTURE WORK  The tracking performance should be improved by asymptotic results.  In order to have more applicable control scheme, improving the cost function estimation update law to simpler version is suggested.  The admissibility of the updated control law should be investigated as a possible future step. This can be a very challenging problem in adaptive dynamics programming.  The authors acknowledge the technical support and assistance of Dr. Dierks (Missouri S&T alumni 2009). Acknowledgements Control Signal Attitude: Position: NN-Based Optimal Adaptive Controller Design  The optimal adaptive controller in this approach is comprised of two neural networks, one for learning the cost function and the other one for estimating the unknown parameters. The unknown parameters are estimated  The cost function can be estimated through the following NN-based adaptive scheme:.  Therefore, the estimation of the optimal adaptive controller can be represented as:  The cost function parameters are estimated by using the update law  It is shown that the above control law guarantees the overall stability of system. Unknown Parameter Fig.1. 3-D Perspective of position during landing. Fig.2. The absolute attitude error Fig.3. 3-D Perspective of position during take-off and circular maneuver. Fig.4. Helicopter Position vs. time for tracking. Fig.5. The proposed online optimal controller. TREX500