Digital Signal Processing II Lecture 3: Filter Realization Marc Moonen Dept. E.E./ESAT, K.U.Leuven marc.moonen@esat.kuleuven.be homes.esat.kuleuven.ac.be/~moonen/

PART-I : Filter Design/Realization Step-1 : define filter specs (pass-band, stop-band, optimization criterion,…) Step-2 : derive optimal transfer funcion FIR or IIR Step-3 : filter realization (block scheme/flow graph) direct form realizations, lattice realizations,… Step-4 : filter implementation (software/hardware) accuracy issues, … question: implemented filter = designed filter ? Lecture-2 Lecture-3 Lecture-4 Version 2005-2006 Lecture-3 Filter Realization

Lecture 3 : Filter Realizations FIR Filter Realizations IIR Filter Realizations PS: We will assume real-valued filter coefficients Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations =Construct (realize) LTI system (with delay elements, adders and multipliers), such that I/O behavior is given by.. Several possibilities exist… 1. Direct form 2. Transposed direct form 3. Lattice (LPC lattice) 4. Lossless lattice 5. Frequency-domain realization: see Part-2 Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations 1. Direct form u[k] u[k-4] u[k-3] u[k-2] u[k-1] x bo + b4 b3 b2 b1 y[k] Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations 2. Transposed direct form Starting point is direct form : `Retiming’ = select subgraph (shaded) remove delay element on all inbound arrows add delay element on all outbound arrows u[k] u[k-4] u[k-3] u[k-2] u[k-1] x bo + b4 b3 b2 b1 y[k] Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations `Retiming’ : results in... u[k] u[k-1] x bo + b1 y[k] u[k-3] u[k-2] b4 b3 b2 Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations `Retiming’ : repeated application results in... i.e. `transposed direct form’ u[k] x bo + y[k] b1 b2 b3 b4 different software/hardware, same i/o-behavior Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations 3. Lattice form : Derived from combined realization of with `flipped’ version of H(z) Reversed (real-valued) coefficient vector results in... i.e. - same magnitude response - different phase response Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Hence starting point is… u[k] u[k-1] u[k-2] u[k-3] u[k-4] bo b1 b2 b3 b4 x x x x x b4 b3 b2 b1 bo x x x x x y[k] + + + + y[k] ~ + + + + Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations With , this is rewritten as `sloppy notation’ Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations This is equivalent to... …Now repeat procedure for shaded graph (=same structure as the one we started from) u[k] u[k-3] u[k-2] x b’o + b’1 b’3 b’2 ko y[k] y[k] ~ Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Repeated application results in `lattice form’ u[k] bo x y[k] + + + + x x x x ko k1 k2 k3 y[k] ~ x x x x + + + + i.e. different software/hardware, same i/o-behavior Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Lattice form : Also known as `LPC Lattice’ (`linear predictive coding lattice’) Ki’s are so-called `reflection coefficients’ Every set of bi’s corresponds to a set of Ki’s, and vice versa. Procedure for computing Ki’s from bi’s corresponds to the well-known `Schur-Cohn’ stability test (from control theory): problem = for a given polynomial B(z), how do we find out if all the zeros of B(z) are stable (i.e. lie inside unit circle) ? solution = from bi’s, compute reflection coefficients Ki’s (following procedure on previous slides). Zeros are stable if and only if all reflection coefficients statisfy |Ki|<1 Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Lattice form : Procedure breaks down if |Ki|=1 is encountered. Means at least one root of B(z) lies on or outside the unit circle (cfr Schur-Cohn). Lattice form is often applied to `minimum phase’ filters, i.e. filter with only stable zeros (roots of B(z) strictly inside unit circle). Then design procedure never breaks down. Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations 4. Lossless lattice : Derived from combined realization of (possibly scaled) H(z) with which is such that (*) PS : interpretation ?… (see next slide) Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations PS : Interpretation ? When evaluated on the unit circle, this formula is equivalent to (for filters with real-valued coefficients) i.e. and are `power complementary’ (= form a 1-input/2-output `lossless’ system, see also below) 1 Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations PS : How is computed ? Note that if `a’ is a root of R(z), then `1/a’ is also a root of R(z), hence Note that ai’s can be selected such that all of them lie inside the unit circle. Then is a minimum-phase FIR filter. This is referred to as spectral factorization, =spectral factor. Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Hence starting point is… u[k] u[k-1] u[k-2] u[k-3] u[k-4] bo b1 b2 b3 b4 x x x x x x x x x x y[k] + + + + y[k] ~ + + + + Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations From (*) (page 16), it follows that Hence there exists a theta_0 such that `sloppy notation’ Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations This is equivalent to... …Now repeat procedure for shaded graph (=same structure as the one we started from!) (why?) u[k] u[k-3] u[k-2] x b’o + b’1 b’2 b’3 y[k] ~ Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Repeated application results in `lossless lattice’ u[k] x y[k] + x ~ + x + x + x x x x x x x + x + x + x Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Lossless lattice : also known as `paraunitary lattice form’ (see Lecture 6) each section is based on an orthogonal transformation, which preserves norm/energy/power (= forms a 2-input/2-output `lossless’ system) hence overall structure preserves the input energy, i.e. is `lossless’… Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations Orthogonality implies `losslessness’ : - assume initial internal state is x1[0], x2[0] (2nd order example) input sequence is u[0],u[1],u[2],…,u[L] corresponding output sequences are y[0],y[1],… and y[0],y[1],... then orthogonality implies that (up to scaling with alpha on page 22) `energy’ in initial states + inputs = `energy’ in final states + outputs - equivalent transfer function based property is… i.e. SISO transfer function H(z) is `embedded’ in 1-input/2-output lossless system (see page 17) ~ Version 2005-2006 Lecture-3 Filter Realization

FIR Filter Realizations PS : Relevance of lattice realizations : robustness example : o = original transfer function + = transfer function after 8-bit truncation of lattice filter parameters (angles theta) - = transfer function after 8-bit truncation of direct-form coefficients (bi’s) PS : A 2x2 orthogonal transformation (rotation) may be implemented in hardware based on a so-called `CORDIC’ (`COR’ for co-ordinate rotation’) architecture, which is more efficient (and more robust) than a multiply-add based implementation Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Construct LTI system such that I/O behavior is given by.. Several possibilities exist… 1. Direct form 2. Transposed direct form 3. Lattice-ladder form 4. Lossless lattice 5. Parallel realization 6. Cascade realization 7. Bi-quad coupled realization 8. State space realization Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 1. Direct form Starting point is … u[k] + + + + -a1 -a2 -a3 -a4 x x x x Direct form for B(z) x bo b4 b3 b2 b1 y[k] + + + + Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations which is equivalent to... PS : If all a_i=0 (hence H(z) is FIR), then this reduces to a direct form FIR realization. x bo b4 b3 b2 b1 + y[k] u[k] -a4 -a3 -a2 -a1 Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations -State space description: -N delay elements (=minimum number =max of numerator and denominator polynomial order = max(p,q)) -2N+1 multipliers, 2N adders u[k] x bo b4 b3 b2 b1 + y[k] -a4 -a3 -a2 -a1 x1[k] x2[k] x3[k] x4[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 2. Transposed direct form Starting point is… u[k] + x -a4 -a3 -a2 -a1 y[k] bo b4 b3 b2 b1 Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations which is equivalent to... u[k] x bo b4 b3 b2 b1 + + + + -a1 -a2 -a3 -a4 x x x x y[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Transposed direct form is obtained after retiming ... PS : If all a_i=0 (hence H(z) is FIR), then this reduces to a transposed direct form FIR realization. Transposed direct form B(z) x -a4 -a3 -a2 -a1 y[k] u[k] bo b4 b3 b2 b1 + Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations - State space description i.e. direct forms state space matrices are `transpose’ of each other, which justifies the name `transposition’. u[k] x -a4 -a3 -a2 -a1 y[k] bo b4 b3 b2 b1 + x1[k] x2[k] x3[k] x4[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 3. Lattice-ladder form Derived from combined realization of with... - numerator polynomial is denominator polynomial with reversed coefficient vector (see also page 9) - hence is an `all-pass’ (=`SISO lossless’) filter : Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Starting point is… u[k] + x b1 b2 b0 b3 b4 a3 a2 a1 1 a4 y[k] ~ x[k] - Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations This is equivalent to… (prove it!) + x x’[k-1] u[k] b0 y[k] ~ - Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Right-hand part has the same structure as what we started from, hence repeated application leads to `lattice-ladder form’ u[k] y[k] ~ x + b0 =lossless section Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Lattice-Ladder form : Ki’s are `reflection coefficients’ Procedure for computing Ki’s (=sin(theta_i) !!) from ai’s again corresponds to `Schur-Cohn’ stability test (see lecture-2): all zeros of A(z) are stable (i.e. lie inside unit circle) iff all reflection coefficients statisfy |Ki|<1 (i=1,…,N-1) (ps: procedure breaks down if |Ki|=1 is encountered) Orthogonal transformations correspond to `lossless’ sections which also explains `all-pass’ (lossless) characteristic of Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations State-space description EX: 1st-order system: EX: 2nd-order system: - R is `realization matrix’ (A-B-C-D-matrix) for u[k] -> y[k] (SISO) - Observe that R is an orthogonal matrix : R^T.R=I - Orthogonality of R again implies `losslessness’ (see next slide) ~ Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations - Orthogonality of R implies `losslessness’ : assume initial state is x1[0], x2[0] (2nd order example) input sequence is u[0],u[1],u[2],…,u[L] corresponding output sequence is y[0],y[1],y[2],…,y[L] then orthogonality of R implies that `energy’ in initial states + inputs = `energy’ in final states + outputs - In SISO-case, losslessness implies all-pass transfer function: (see also page 35) ~ Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations PS : `All-pass’ part (u[k]->y’[k]) is known as `Gray-Markel’ structure PS : Note that the all-pass part corresponds to A(z) (i.e. N angles theta_i correspond to N coefficients a_i), while the ladder section corresponds to B(z). If all a_i=0 (hence H(z) is FIR !), then all theta_i=0, hence the all-pass part reduces to a delay line, and the lattice-ladder form reduces to a direct-form FIR realization. x + in out Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 4. Lossless-lattice : Derived from combined realization of with... - numerator polynomial is C(z) is such that i.e. and are `power complementary’ Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations `Lossless-lattice’ : similar derivation (but more complicated -hence skipped) leads to… x + u[k] y[k] ~ =lossless section Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations Orthogonal transformations correspond to `lossless sections hence overall lossless characteristic, i.e. SISO transfer function H(z) is `embedded’ in 1-input/2-output lossless system Note that 2N+1 filter coefficients (a_i’s and b_I’s) define 2N+1 rotation angles. If all a_i=0 (hence H(z) is FIR), rotation angles will be such that all feedback paths are cancelled (in a magic way) Example: (draw it/try it!) Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations state-space description 1st-order system: etc…. R is `realization matrix’ (A-B-C-D-matrix) for u[k] ->y[k], y[k] (1-in/2-out) Observe that R is an orthogonal matrix : R^T.R=I Orthogonality of R again implies `losslessness’ ~ Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 5. Parallel Realization : partial fraction decomposition e.g. for simple poles: similar for the case of multiple poles each term realized in, e.g., direct form same number of delay elements, additions and multiplications (if p=q) as direct forms transmission zeros are realized iff signals from different sections exactly cancel out. =Problem in finite word-length implementation u[k] y[k] + Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 6. Cascade Realizations : pole-zero factorization of H(z) e.g. for N even: similar for N odd each section realized in, e.g., direct form second-order sections are called `bi-quads’ same number of delay elements, additions and multiplications as direct forms u[k] y[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations realization is non-unique: - multiple ways of pairing poles and zeros - multiple ways of ordering sections in cascade ``pairing procedure’’ : - pairing of little importance in high-precision (e.g. floating point implementation), but important in fixed-point implementation (with short word-lengths) - principle = pair poles and zeros to produce a frequency response for each section that is as flat as possible (i.e. ratio of max. to min. magnitude response close to unity) - obtained by pairing each pole to a zero as close to it as possible - procedure : start with pole pair nearest to the unit circle, and pair this to nearest complex zeros. remove pole-zero pair, and repeat, etc…. u[k] y[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 7. Bi-quad `coupled realization’ : - direct form realization of bi-quad may be unsatisfactory, e.g. for short word-lengths and poles near z=1 or z=-1 (see lecture-4) - alternative realization: + - y[k] u[k] state space description : poles : Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations 8. State-space Realization : State-space description: State-space realization = realization such that all the elements of A,B,C,D are the multiplier coefficients in the structure Example: bi-quad `coupled’ realiz. Example: (transposed) direct form is NOT a state-space realization, cfr. elements (bi-bo.ai) in B or C Example: lattice-ladder, lattice ? (not quite, but close enough…see p.55) + y[k] u[k] Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations State-space description/realization is non-unique: any non-singular matrix T transforms A,B,C,D into an alternative state-space description/realization with transfer function is (prove it) impulse response is Version 2005-2006 Lecture-3 Filter Realization

IIR Filter Realizations PS : orthogonal filters : state-space realization with orthogonal realization matrix: Relevance : see lecture-4 (scaling, quantization noise power, limit cycles, etc…) PS : lattice filters and lattice-part of lattice-ladder …. Strictly speaking, these are not state-space realizations (cfr. supra), but orthogonal R is realized as a product of matrices, each of which is again orthogonal, such that useful properties of orthogonal states-space realizations indeed carry over…. Version 2005-2006 Lecture-3 Filter Realization