65. 3. Applications
Function approximation: t  y t: target, y: actual output
Neural control r: reference u: control effort y: system output r  y
Ming-Feng Yeh

66. Neural Control Schemes
Supervised control
Hybrid control
Model reference control
Internal model control
Adaptive control
Reference: C.L. Lin & H.W. Su, “Intelligent control theory in guidance and control system design: an overview,” Proc. Natl. Sci. Counc. ROC (A), pp
Ming-Feng Yeh

67. Supervisory Control Neural network  inverse NN controller
The neural controller in the system is utilized as an inverse system model.
Ming-Feng Yeh

68. Hybrid Control
Generalized learning (off-line learning) A rough approximation to the desired control law  Drive the plant over the operating range and without instability
Specialized learning (on-line learning) Improve the control provided by the NN controller
Ming-Feng Yeh

69. Model Reference Control
Must define its input-output pair {r(t), yR(t)} in advance
Attempts to make the plant output y(t) match the reference model output asymptotically.
The error e(t) is used to adjust the weights of an neural controller.
Ming-Feng Yeh

70. Internal Model Control
The NN plant model is first trained off-line to emulate the controller plant dynamics directly. During on-line operation, the error is used as a feedback signal and passed to the NN controller.
The effect of the NN controller is to subtract the effect of the control signal from the plant output, i.e., disturbances.
The IMC plays a role as a feedforward controller and can cancel the influence due to unmeasured disturbances.
Ming-Feng Yeh

71. Adaptive Control
The tracking error cost is evaluated according to some performance index.
The result is then used as a basis for adjusting the connection weights of the neural networks.
The weights are adjusted on-line using basic backpropagation rather than off-line.
Ming-Feng Yeh

72. Practical Stability Issues in CMAC Neural Network Control Systems
Paper Study #1
Practical Stability Issues in CMAC Neural Network Control Systems
Fu-Chuang Chen & Chih-Horng Chang
IEEE Trans. on Control Systems Technology,
Vol. 4, No. 1, pp , 1996
Ming-Feng Yeh

73. Abstract
CMAC is a practical tool for improving existing nonlinear control systems.
CMAC can effectively reduce tracking error, but can also destabilize a control system which is otherwise stable.
Quantitative studies are presented to search for the cause of instability in the CMAC control system.
Ming-Feng Yeh

74. I. Introduction
CMAC is basically a look-up table method, very easy to implement, and at the same time it is a powerful and practical tool for nonlinear control.
There has been convergence result on the CMAC learning.
Ming-Feng Yeh

75. Main Purpose of This Paper
To introduce the CMAC control system from an industrial point of view.
To describe the unstable phenomenon.
To quantitatively study how the system parameters such as control gain, quantization, generalization, learning rate, etc., are related to the instability of the system.
To suggest ways to improve system stability.
To provide some experience evidence.
Ming-Feng Yeh

76. II. CMAC Control System CMAC controller Plant proportional controller
Y(k+1) = 0.5Y(k)+sin(Y(k))+U(k)
Use a workable traditional controller to stabilize the plant and to help the CMAC learn to provide precise control.
Ming-Feng Yeh

77. Functioning of CMAC Initially the CMAC table is empty.
In each time step k, the CMAC involves a recall and a learning process.
Ming-Feng Yeh

78. Recall Process
Uses Yd(k+1) and Y(k) as the address to generate the control signal from CMAC table, where Yd(k+1) is the desired system output for the next time step.
CMAC has two inputs and one output.
Ming-Feng Yeh

79. Leaning Process
U(k) is treated as the desired output to modified the content stored at location Y(k) and Y(k+1), where Y(k+1) is the actual system output for the next time step k+1.
To speed up the initial learning and to achieve better generalization, the generalization technique is employed, i.e., each input vector to CMAC for recall and learning will map to a number of memory locations instead of only one memory location.
Ming-Feng Yeh

80. Function Approximation
How precisely the CMAC can approximate a function is mainly determined by the quantization in each dimension of the input vector.
Reducing quantization would quickly increase the memory demand for storing the CMAC table.
Ming-Feng Yeh

81. Table Update Mechanism
Gradient-type learning rule
Wi(k+1) = Wi(k) +   [ U(k)  Uc(k) ] / g
g: the size of generation
Wi: the content of the ith memory location, there being q locations to be updated
: the learning rule
U: the correct (desired) data
Uc: the current (actual) data
Ming-Feng Yeh

82. III. A Typical Simulation Study
Proportional gain: P = 1.4
Learning rate:  = 0.1
Generalization = 50
Quantization = 5/500 (meaning five units divided into 500 divisions)
Reference command = sin(2k/200) with each sinusoidal cycle consisting of 400 time steps
Tracking error
Number of cycles
Ming-Feng Yeh

83. A Typical Simulation Study
In the first five cycles, the system is solely controlled by the P controller.
The CMAC is added at the 6th cycle, and then the error reduces quickly and significantly.
The error remains small for some time, and then it diverges around the 143th cycle.
Control output
Number of time steps
Ming-Feng Yeh

84. Discussion The CMAC can significantly reduce the tracking error.
CMAC can destabilize a control system which is otherwise stable. The unstable phenomenon certainly comes from the interactions between the proportional controller and the CMAC network.
Ming-Feng Yeh

85. Discussion
The proportional controller can not be removed even when the magnitude of the proportional control is very small compared with that of the CMAC (i.e., when the system output error has been significantly reduced). Otherwise, the good tracking can not be maintained.
Ming-Feng Yeh

86. Growth of Oscillation 8th cycle 90th cycle 155th cycle 40th cycle
Ming-Feng Yeh

87. V. Method for Improving System Stability
The continued learning of CMAC after the tracking error has reduced is the major cause of the instability.
Stopping the CMAC learning has two drawbacks. First, it can be difficult to determine when to stop the CMAC learning. Second, if the CMAC stops learning, then the CMAC control system cannot respond to any change in the reference command.
Ming-Feng Yeh

88. Modified Learning Rule
To effectively stop the CMAC learning when the tracking error is small, but at the same time allow the system to respond to any change in the reference command, a deadzone is added to the CMAC updating rule. Wi(k+1) = Wi(k) +   D[ U(k)  Uc(k) ] / g where
Ming-Feng Yeh

89. VI. Experiment
Ming-Feng Yeh

90. Paper Study #2
Intelligent Controller Using CMACs with Self-Organized Structure and Its Application for a Process System
T. Yamamoto, H. Yanagino & M. Kaneda
Proceedings of 1997 IEEE , pp
Ming-Feng Yeh

91. Abstract
This paper describes a design scheme of intelligent system consists of some CMACs.
Each of CMACs is trained for the specified command signal.
A new CMAC is generated for unspecificed command signals, and the CMAC whose command is nearest for the new command signal, is eliminated.
The proposed intelligent controller can be designed with relatively small memories.
Ming-Feng Yeh

92. 1. Introduction
The CMACs included in the intelligent controller are trained in both off-line and on-line learning process for each of the specified command signals.
For a unspecified command, a new CMAC is generated.
The initial weights are set by employing the linear interpolation to the trained weights included in two CMACs whose command signals are nearest for the new command signal.
The CMAC corresponding to the nearest command signal is eliminated.
The proposed intelligent controller can be designed with relatively small memories.
Ming-Feng Yeh

93. 3. Intelligent Controller Design
Reference Model
CMAC1
CMACi
CMACn 
System
controller
Ming-Feng Yeh

94. Outline
The input signals to each CMAC are the control error signal and the difference, that is, the two- dimensional CMACs are equipped in the intelligent control system.
By including the command signal as one of input signals in the CMAC, the intelligent control system can be constructed by using only a three-dimensional CMAC.
Ming-Feng Yeh

95. Off-line Learning Process
CMAC 
w(t): command signal u*(t): teacher signal
Updated rule: h = 1, 2, …, K: total number of the selected weights g1(t): the gradient to update the weights
a1, b1, c1: positive cont.
Ming-Feng Yeh

96. On-line Learning Process
The last weights obtained in the off-line learning are used as initial ones in the on-line learning.
Updated rule: k: the time-delay of the system g2(t): the gradient to update the weights
a2, b2, c2: positive cont.
Ming-Feng Yeh

97. Off-line vs On-line
In the off-line learning process, the teacher signal u*(t) is generated by a certain control law, e.g., PID control law and human experts.
u*(t) is utilized in order to determine the initial weights in the on-line learning of the CMAC.
In the on-line learning process, the teaching signal u*(t) can not be obtained.
The desired reference model output ym(t) is introduced, and the on-line learning is performed so that the system output y(t) approaches to ym(t).
Ming-Feng Yeh

98. Self-organized Structure
A new CMAC is generated for a new command signal
The initial weights includes in the new CMAC are set by employing the linear interpolation to the trained weights included two CMACs which are nearest for the new command signal
The CMAC whose command signal is nearest for the new one is eliminated.
Ming-Feng Yeh

99. 4. Experimental Results Air pressure control system
Control object: regulate the air pressure y to any desired values by manipulating the control value angle u.
In order to obtain u*(t), PID control law is employed for this control system.
Ming-Feng Yeh

100. Control Result Conventional PID control
Off-line learning (20 iterations)
Ming-Feng Yeh

101. Control Result (cont.) On-line learning (after 5 more iterations)
Unspecified command signal
Ming-Feng Yeh