Introduction to estimation theory Seoul Nat’l Univ.

Contents n What is estimator for signal models n estimator application n Signal models n Design objectives n Options of estimators n Objectives and design procedure n Options for estimator : smoothing, filtering, and predicting n FIR structure n Initial state dependency n Performance criterion n Extension to Control Seoul Nat’l Univ.

1.1 What is estimator for signal models (1/1) 1.Introduction Seoul Nat’l Univ. Parameter estimation State estimation estimator : Parameter : State as small as possible

 Other methodology  Fault detection  parameter estimation  state observer/estimation  signal separation  spectrum analysis  Output feedback control : state feedback control + estimator 1.2 estimator application (1/3) 1.Introduction Seoul Nat’l Univ.

Control Output feedback control = state feedback control + estimator 1.Introduction 1.2 estimator application (2/3) Seoul Nat’l Univ. plant estimator

 Practical areas  Speech - speech enhancement  Image - medical imaging - denoising  aerospace - target tracking - navigation - flight pass reconstruction  chemical process - distillation columns  mechanical system - motor system  biological area - cardiac arrhythmia detection 1.Introduction 1.2 estimator application ( 3/3) Seoul Nat’l Univ.

Modeled Unmodeled State space Generic linear model Linear Nonlinear Stochastic Deterministic Time invariant Time varying Discrete-time Continuous-time  Categories of signal models Seoul Nat’l Univ. 1.Introduction 1.3 Signal models (1/3)

 State space signal model  In case of stochastic model :  In case of deterministic model : and are random process and are deterministic signal  Choice of model is important for model-based signal processing Seoul Nat’l Univ. 1.Introduction 1.3 Signal models (2/3)

 Modelled vs unmodelled signal velocity Seoul Nat’l Univ. 1.Introduction 1.3 Signal models (3/3)  Unmodeled signal  Model based signal

 Stability of the filter  Estimation error ( often called performance ) unbiasedness : convergence : efficiency :  Robustness estimation error w.r.t signal model uncertainties  Computation load Seoul Nat’l Univ. 1.Introduction 1.4 Design objectives (1/1)

Estimator structure Performance Criterion Signal Models IIR (infinite horizon) FIR (receding horizon) Initial state dependent stochastic deterministic least square minimax Seoul Nat’l Univ. 1.Introduction Given Options Initial state independent Minimum variance Nonlinear Linear 1.5 Options for estimators (1/1) Filter Smoothing Prediction generic linear state space receding horizon infinite horizon

Signal modelsOptimal estimator Does it satisfy Estimator structure... Performance criterion... desired properties Desired properties Stability Robustness Small error Yes No 1.Introduction Seoul Nat’l Univ. 1.6 Objectives and design procedure (1/2)

Stability  FIR 1.Introduction Small error Robustness  w.r.t uncertainties  w.r.t disturbance  Performance criterion Objectives :Options 1.6 Objectives and design procedure (2/2)

1.Introduction 1.7 Options for estimator : smoothing, filtering, and predicting Seoul Nat’l Univ.  Categories of estimators Current time SmoothingFilteringPredicting

Which one do you think better ?  Case 1 (IIR)  Case 2 (FIR) 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (1/ 9)

Case 1 (FIR) Case 1 (IIR) 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (2/9)  BIBO stability of FIR estimators

Divergence of IIR filter (Kalman filter) Seoul Nat’l Univ. 1.Introduction 1.8 FIR structure (3/9)  Robustness to model uncertainty

 Robustness to round off error : comparison of error covariance  Observation :  Though rounding at the 4th digit are not serious, rounding of 3rd and 2 nd digit makes difference between the FIR filter and IIR filter. Round-off digit Filter Structure  Simulation environments  We assume that the filter gain is previously known by off-line calculation  Rounding off error is applied when updated  Model 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (4/9)

1.Introduction Seoul Nat’l Univ. 1.Stabilization the nominal system 2.Stabilization the disturbed systems In case of Control Nominal systems 1.Be sure to be deadbeat using FIR structure for nominal systems. 2.Small error for noise or disturbance corrupted systems. In case of Filter 1.8 FIR structure (5/9)  Require to be deadbeat using nominal systems  Nominal systems = zero disturbance / noise system  Deadbeat property

1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (6/9) Exact filter (deadbeat phenomenon) Noise State & estim. trajectory  Horizon size  Deadbeat property

 Original Filtered IIR filter 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (7/9) Magnitude Phase Time Frequency FIR filter Heavy distortion of phase at band gap

cf. Infinite impulse response(IIR) : Nonlinear phase Not always stable Easy to obtain from analog filter Suitable for sharp cutoff characteristic and high speed 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (8/9)  Advantage & disadvantage Advantage of FIR  Use of DFT  Robustness to round off error  Linear phase  Guaranteed stability  Good for adaptive filter Disadvantage of FIR  Computation load  H/W complexity

F I R I I R 1.Introduction Seoul Nat’l Univ. 1.8 FIR structure (9/9)

 Infinite impulse response (IIR) : dependent of IIR Linear Initial state dependent 1.Introduction Seoul Nat’l Univ. 1.9 Initial state dependency (1/2) FIR LinearInitial state free  Finite impulse response (FIR) : Independent of

 Filter is to estimate state The initial state is also a state It is not logical to assume the initial state 1.Introduction Seoul Nat’l Univ. 1.9 Initial state dependency (2/2)  Example : Try to guess who he is. 1. First case (our approach) 2. Second case (ex. Kalman filter), Given, then guess Original picture

 Performance criterion Minimum variance Least square Maximum Likelihood 1.Introduction 1.10 Performance criterion (1/3) Seoul Nat’l Univ.

 Performance criterion for deterministic models - filter - Minimax filter - Least squares 1.Introduction 1.10 Performance criterion (2/3) Seoul Nat’l Univ.  Performance criterion for Stochastic models - Minimum variance - Minimax variance - Minimum Entropy

1.Introduction 1.10 Performance criterion (3/3) Seoul Nat’l Univ. Stability  FIR Small error Robustness  w.r.t uncertainties  w.r.t disturbance  Minimization Objectives :Options  Minimization of maxima

1.Introduction 1.11 Extension to control : receding horizon control Seoul Nat’l Univ. Which one do you think better ? What is the receding horizon control?

1.Introduction Seoul Nat’l Univ.  Stability of the closed-loop systems  Small tracking error  Robustness stability tracking error 1.11 Extension to control : desired property

Control structure Performance Criterion Signal Models infinite horizon receding horizon output feedback stochastic deterministic LQ minimax Given Options state feedback LQG Finite memory control (including static control) Dynamic (IIR control) I/O model state space Seoul Nat’l Univ. 1.Introduction 1.11 Extension to control : options for controls

1.Introduction Seoul Nat’l Univ. Signal modelsOptimal control Does it satisfy Control structure Performance criterion LQG LQ Minimum entropy …… desired properties Desired properties Stability Robustness Small tracking error Yes No State feedback control Output feedback control Dynamic control Finite memory control …… 1.11 Extension to control : objectives and design procedures

1.Introduction Seoul Nat’l Univ extension to control : performance criterion with receding horizon LQLQ  LQG 

1.Introduction 1.11 extension to control : receding horizon output feedback control Seoul Nat’l Univ. State feedback receding horizon control LQC Control …… Filter Kalman filter filter Mixed filer …… + Question : Is it optimal ? FMC (finite memory control)  method 1  method 2 Global optimal output feedback control cf) LQG

1.Introduction Seoul Nat’l Univ.  Computation 1.11 extension to control : receding horizon output feedback control

Contents of standard textbook on optimal control and estimation 1.Introduction Seoul Nat’l Univ. n 1. LQ control u Finite horizon u Infinite horizon n 2. Kalman filter u Finite horizon u Infinite horizon n 3. LQG control u Finite horizon u Infinite horizon n 4. Full information control u Finite horizon u Infinite horizon n 5. filter u Finite horizon u Infinite horizon n 6. Output feedback control u Finite horizon u Infinite horizon u Receding horizon Covered in this class