Lectures 3&4: Non-Parametric System Identification: Impulse & Frequency Response ^ y(t) Model u(t) u(t) y(t) Controller Plant h(t), H(jw) Dr Martin Brown Room: E1k, Control Systems Centre Email: martin.brown@manchester.ac.uk Telephone: 0161 306 4672 http://www.eee.manchester.ac.uk/intranet/pg/coursematerial/ EE-M110 2006/7, EF L3&4

L3&4: Resources Main texts Chapter 2, Ljung Chapter 2, Norton Chapters 1 & 2 in on-line notes. Overview In these two lectures, we’re looking at how LTI systems can be described in a non-parametric fashion Look at system impulse response h(t) Look at frequency response H(jw) In both cases, we’ll investigate the representation, look at how you calculate the system response and consider how the system can be identified from data This is sometimes known as classical system identification We’re not doing an in-depth analysis for either case. EE-M110 2006/7, EF L3&4

Lecture 3: System Impulse Response Non-parametric system impulse response Impulse response d(t)→h(t) Properties of impulse response (causal, stable) Sifting property for signals System linearity & superposition property System response using convolution Examples of calculating the output using convolution Identifying the impulse response from data NB, we’ll be doing the main part of the analysis for discrete time signals and LTI systems, but the work carries over to continuous time signals and systems as well. EE-M110 2006/7, EF L3&4

System Impulse Response A very important way to analyse a system is to study the output signal when a unit impulse signal is used as an input Loosely speaking, this corresponds to giving the system a kick at t=0, and then seeing what happens This is so common, a specific notation, h(t), is used to denote the output signal, rather than the more general y(t) This impulse response signal can be used to infer properties about the system’s structure (LHS of difference equation or unforced solution) The system impulse response, h(t) completely characterises a linear, time invariant system x(t) d(t) System h(t) y(t) h(t) EE-M110 2006/7, EF L3&4

Properties of System Impulse Response Stable A system is stable if the impulse response is absolutely summable Causal A system is causal if h(t)=0 when t<0 Finite/infinite impulse response The system has a finite impulse response and hence no dynamics in y(t) if there exists T>0, such that: h(t)=0 when t>T Linear ad(t)  ah(t) Time invariant d(t-T)  h(t-T) h(t) t EE-M110 2006/7, EF L3&4

System Impulse Response Examples Causal, stable, finite impulse response y(t) = x(t) + 0.5x(t-1) + 0.25x(t-2) Looking at system impulse responses, allows you to determine certain system properties h(t) Causal, stable, infinite impulse response y(t) = x(t) + 0.7y(t-1) t h(t) h(t) t t Causal, unstable, infinite impulse response y(t) = x(t) + 1.3y(t-1) EE-M110 2006/7, EF L3&4

Introduction to Convolution Definition Convolution is an operator that takes an input signal and returns an output signal, based on knowledge about the system’s impulse response h(t). The basic idea behind convolution is to use the system’s response to simple input signals to calculate the response to more complex input signals This is possible for LTI systems because they possess the superposition property x(t) = d(t) LTI system y(t) = h(t) LTI system h(t) x(t) y(t) EE-M110 2006/7, EF L3&4

Sifting Property Basic idea: use a (infinite) set of of DT impulses to represent any DT signal. Consider any discrete input signal x(t). This can be written as the linear sum of a set of unit impulse signals: Therefore, the signal can be expressed as: In general, any discrete signal can be represented as: actual value Impulse, time shifted signal The sifting property EE-M110 2006/7, EF L3&4

Example: DT Signal Sifting The discrete signal x(t) Is decomposed into the following additive components … + x(-4)d(t+4) + x(-3)d(t+3) + x(-2)d(t+2) + x(-1)d(t+1) + … EE-M110 2006/7, EF L3&4

System Linearity Let y1(t) and y2(t) be the responses to x1(t) and x2(t), respectively A linear system must satisfy the two properties: 1 Additive: the response to x1(t)+x2(t) is y1(t) + y2(t) 2 Scaling: the response to ax1(t) is ay1(t) where aC Combined: ax1(t)+bx2(t)  ay1(t) + by2(t) Examples y(t) = 3x(t). Consider x(t) = ax1(t)+bx2(t), y(t) = 3(ax1(t)+bx2(t)) = a3x1(t) + b3x2(t) = ay1(t) + by2(t). The system is linear 2) y(t) = 3x(t)+2. Consider x(t) = 2x1(t) and use the scaling property y(t) = 3*2*x1(t)+2 = 6x1(t)+2  2y1(t). The system is not linear 3) y(t) = 3*x2(t). Consider x(t) = 2x1(t) and use the scaling property y(t) = 3*(2x1(t))2 = 12x1(t)2  2y1(t). The system is not linear EE-M110 2006/7, EF L3&4

Linear Systems and Superposition Generalizing the scaling/additive properties, suppose an input signal x(t) is made of a linear sum of other (basis) signals xk(t): then the response of a linear system is (where xk(t)yk(t)): The basic idea is that if we understand how simple signals get affected by the system, we can work out how complex signals are processed, by expanding them as a linear sum This is known as the superposition property which is true for linear systems in both CT & DT Important for understanding convolution EE-M110 2006/7, EF L3&4

Convolution Sum for LTI Systems As the system is time invariant, the response to the impulse signal d(t-k) is h(t-k). Then from the superposition property of linear systems, the system’s response to a more general input signal x(t) can be derived as follows. Input signal System output signal is given by the convolution sum i.e. it is the scaled sum of time shifted impulse responses. EE-M110 2006/7, EF L3&4

Convolution in a Nut-Shell For any LTI discrete time system, the response to an input signal x(t) is given as follows. Using the sifting property: h(t) is the system impulse response to d(t). Because the system is time invariant, Using the superposition property (linear): The system response of any LTI system can be calculated as a linear combination of impulse responses & written as x(t) t h(t) t y(t) t EE-M110 2006/7, EF L3&4

Interpreting the Convolution Sum Note that convolution can be interpreted in two ways: As a sum of scaled, shifted impulse response signals As an algebraic formula which is a function of t Method 1) works well when h(t) or x(t) has finite duration, but method 2) is more flexible and more widely used. See following examples x(t) h(t) y(t) t t t EE-M110 2006/7, EF L3&4

Example 1: LTI Convolution Calculate the DT, LTI system response when: h(t) = [0 0 1 1 1 0 0], x(t) = [0 0 0.5 2 0 0 0] Then: h(t) = u(t)u(2-t) h(t-k) = u(t-k)u(2-t+k) and the convolution sum: u(t) u(2-t) 2 t Very common trick! EE-M110 2006/7, EF L3&4

Example 2: LTI Convolution Consider a DT LTI system that has a step response h(t) = u(t) to the unit impulse input signal (integrator) What is the response when an input signal of the form x(t) = atu(t) where 0<a<1, is applied? h(t) t x(t) t y(t) t EE-M110 2006/7, EF L3&4

Example 3: LTI Convolution Find the system response when: x(t) = (1/2)tu(t+1) h(t) = (1/3)tu(t-1) Using DT, LTI convolution h(t) t x(t) t y(t) t EE-M110 2006/7, EF L3&4

System Identification and Prediction Note that the system’s response to an arbitrary input signal is completely determined by its response to the unit impulse. Therefore, if we need to identify a particular LTI system, we can apply a unit impulse signal, d(t), and measure the system’s response, h(t). That data can then be used to predict the system’s response to any input signal How to: Design impulse signal Collect the data (number of experiments and length) Make sure assumptions are satisfied (linearity) System: h(t) y(t) x(t) n(t)~N(0,s2) EE-M110 2006/7, EF L3&4

How to Identify the Impulse Response Impulse signal. Signal should be of large magnitude and small duration (unit area). How to reproduce with a physical device. NB, the system is generally continuous time Data collection. The number of impulse tests should be large enough to reduce mean estimate, affected by measurement noise, to acceptable accuracy. The length of the signal should be large enough compared to the input signal length (FIR/IIR system). The sample period should be short enough to capture high-order dynamics (Nyquist rate) Input amplitude. The input signal should be scaled to make the impulse response large compared to the magnitude of the noise signal. However, it should not be too large, otherwise the system may be non-linear (systems are generally only locally linear) NB could consider step response data instead u(t) t h(t) s(m)=s/n0.5 x x x x x x x x x x x x x x x x x x x x x x x x x x t ah(t) a<<1 x x x x x x x x x x t EE-M110 2006/7, EF L3&4

Lecture 4: System Frequency Response Non-parametric system frequency response System eigenfunctions System transfer functions (stable systems) Sinusoidal basis functions and Fourier transforms Convolution in the frequency domain Examples of calculating the output Bode plots Identifying the frequency response curve NB, we’ll be doing the main part of the analysis for continuous time signals and LTI systems, but the work carries over to discrete time signals and systems as well. EE-M110 2006/7, EF L3&4

What is a System Eigenfunction? Let’s imagine what (basis) signals fk(t) have the property that: i.e. the output signal is the same as the input signal, multiplied by the constant “gain” lk (which may be complex) For CT LTI systems, we also have that Therefore, to make use of this theory we need: 1) system identification is determined by finding {fk,lk}. 2) prediction, we also have to decompose x(t) in terms of fk(t) by calculating the coefficients {ak}. The {fk,lk} are analogous to eigenvectors/eigenvalues matrix decomposition x(t) = fk(t) y(t) = lkfk(t) System x(t) = Sk akfk(t) y(t) = Sk aklkfk(t) LTI System EE-M110 2006/7, EF L3&4

Sinusoidal Signals & LTI Systems Imaginary exponential (sinusoidal) signals and stable LTI systems are very closely related because: The signal is unchanged by any LTI system apart from a magnitude gain and a phase shift (multiply by complex number l) This means that by superposition, to calculate the system’s output, we can find the frequency content of each signal and sum the scaled responses, where fk(t)=ejkw0t Sinusoidal signals are so-called eigenfunctions of LTI systems, i.e. they remain unchanged by the system apart from a gain and a phase shift x(t) = ejwt y(t) = l(w)ejwt LTI system x(t) y(t) x(t) = Sk akfk(t) y(t) = Sk aklkfk(t) LTI system t EE-M110 2006/7, EF L3&4

Proof of Eigenfunction Property Consider a CT LTI system with impulse response h(t) and input signal x(t)=f(t) = est, for any value of sC: Assuming that the integral on the right hand side converges to H(s) (transfer function), this becomes (for any sC): Therefore f(t)=est is an eigenfunction, with eigenvalue l=H(s). When s=jw, this represents frequency response data, and the system must be stable (h(t) tends to zero) Very important for Fourier/Laplace transforms EE-M110 2006/7, EF L3&4

Fourier Series Fourier series theory says that nearly all continuous time, periodic signal, x(t), can be represented as complex sums of harmonic (sinusoidal) basis signals Examples (w0=1, T=2p) w0 is the fundamental frequency x(t) = sin(t)+0.2*sin(7*t) EE-M110 2006/7, EF L3&4

Fourier Transform When x(t) is not periodic we can calculate the Fourier transform as This is a continuous spectrum of frequencies Example NB the Fourier transform of periodic exists and it is a sum of delta signals, centred at the harmonic frequencies x(t) t T1 -T1 EE-M110 2006/7, EF L3&4

System Prediction using Frequency Response Taking Fourier transforms of the input and impulse response signals allows us to derive the frequency domain convolution equation Y(jw) = H(jw)X(jw) This can then be inverted to obtain the time-domain system response, y(t) to a input signal x(t): Calculate Fourier transform of input x(t) Calculate Fourier transform of impulse response h(t) Multiply together to obtain Y(jw) Invert the Fourier transform (generally by expressing as partial fractions) EE-M110 2006/7, EF L3&4

Example 1: Solving a First Order ODE Consider the response of an LTI system with impulse response: to the input signal: Transforming these signals into the frequency domain: and the frequency response: to convert this to the time domain, express as partial fractions: Therefore, the time domain response is: ba Try calculating using convolution EE-M110 2006/7, EF L3&4

Example 2: Designing a Low Pass Filter H(jw) wc -wc Lets design an ideal low pass filter in frequency domain: The impulse response of this filter is the inverse Fourier transform which is an ideal low pass filter in time domain Non-causal, so this cannot be manufactured exactly The time-domain oscillations may be undesirable How to approximate the frequency selection characteristics? Consider the 1st order LTI system impulse response: Causal and non-oscillatory time domain response and performs a degree of low pass filtering. Higher order filters (ODEs) are usually used. h(t) t EE-M110 2006/7, EF L3&4

Bode Plots A Bode Plot for a system is simply plots of log magnitude and phase against log frequency Both the log magnitude and phase effects are now additive Widely used for analysis and design of filters and controllers Example Low pass, unity filter Log mag v log freq Phase v log freq EE-M110 2006/7, EF L3&4

Example 1: Bode Plot 1st Order System Consider a LTI first order system described by: Fourier transfer function is: the impulse response is: and the step response is: Bode diagrams are shown as log/log plots on the x and y axis with t=2. EE-M110 2006/7, EF L3&4

Estimating Frequency Response Use sin() wave testing to estimate H(jw) over the whole range of frequencies necessary for prediction Y(jw) = H(jw)X(jw) If x(t) only contains low frequencies, it is not necessary to estimate H(jw) for high frequency components as the product is zero whatever the value With a bit of work, we can show that the system response to x(t) = sin(wt) where H(jw)=A+jB is y(t)=(A2+B2)0.5sin(wt+tan-1(B/A)) where gain=(A2+B2)0.5 phase advance=tan-1(B/A) These can be estimated from steady state response EE-M110 2006/7, EF L3&4

L3&4 Summary These lectures have concentrated on “conventional” system identification methods that can be applied to LTI systems. They are non-parametric and make few assumptions about the order of the underlying ODEs/difference equations. Impulse response A key concept for analyzing system properties and can be easily identified. System response is calculated using convolution Frequency response Derived as the Fourier transform of the impulse response signal. Provides information about the gain and phase of the system for different input frequencies. The introduction provided on this course has been very basic and glosses over many practical details. EE-M110 2006/7, EF L3&4

L3&4 Laboratory Theory Using convolution, calculate the DT response for h(t)=u(t-2), x(t)=0.5tu(t). Sketch all signals. Using convolution, calculate the DT response for h(t)=u(t), x(t)=2tu(-t). Sketch all signals. Using Fourier transforms, calculate the CT response for h(t)=e-3(t-1)u(t-1) and x(t)=e-(t+1)u(t+1). Sketch all signals. Matlab and Simulink Change to your P: directory and turn your diary on! Use the Matlab conv() function to verify the DT convolution in questions (1) and (2). Plot all the signals. Use the fourier() and ifourier() functions in the symbolic toolbox to verify the answer to question (3). By evaluating the equivalent transfer functions for each of the impulse responses in questions (1), (2) and (3), simulate the signals and systems in Simulink and verify that all the answers are the same. Simulate the frequency response of a range of first and second order systems in Simulink for a range of different frequencies EE-M110 2006/7, EF L3&4

Appendix A: DT Eigenfunctions/values In an analogous fashion, we can show the same for discrete-time systems, so consider a DT system with DT impulse response h(t) and input x(t). Again, we have assumed that the (infinite) summation on the right hand side converges to H(z). Therefore x(t)=f(t)=zt is a DT eigenfunction with eigenvalue l=H(z). Very important for z-transforms EE-M110 2006/7, EF L3&4

Appendix B: FT of Exponential Signal Consider the (non-periodic) signal (solution to first order ODE) Then the Fourier transform is: x(t) a = 1 If x(t) is a real signal 1) X(0) is real, 2) Im(X(-jw)) = -Im(X(-jw)) EE-M110 2006/7, EF L3&4