1. EECS0712 Adaptive Signal Processing 1 Introduction to Adaptive Signal Processing
Assoc. Prof. Dr. Peerapol Yuvapoositanon
Dept. of Electronic Engineering
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

2. Course Outline Introduction to Adaptive Signal Processing
Adaptive Algorithms Families:
Newton’s Method and Steepest Descent
Least Mean Squared (LMS)
Recursive Least Squares (RLS)
Kalman Filtering
Applications of Adaptive Signal Processing in Communications and Blind Equalization
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

3. Evaluation Assignment= 20 % Midterm = 30 % Final = 50 % CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

4. Textbooks
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

5. http://embedsigproc. wordpress
/eecs0712-adaptive-signal-processing/
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

6. QR code
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

7. Adaptive Signal Processing
Definition: Adaptive signal processing is the design of adaptive systems for signal-processing applications. [
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

8. System Identification
Let’s consider a system called “plant”
We need to know its characteristics, i.e., The impulse response of the system
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

9. Plant Comparison CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

10. Error of Plant Outputs CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

11. Error of Estimation
Error of estimation is represented by the signal energy of error
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

12. Adaptive System We can do it adaptively CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

13. One-weight Adjust the weight for minimum error e CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

14. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

15. Error Curve Parabola equation CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

16. Partial diff. and set to zero
Partial differentiation
Set to zero
Result:
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

17. Multiple Weight Plants
We calculate the weight adaptively
Questions:
What is the type of signal “x” to be used, e.g. Sine, Cosine or Random signals ?
If there is more than one weight w0 , i.e., w0….wN-1, how do we calculate the solution?
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

18. Plants with Multiple Weight
If we have multiple weights
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

19. Two-weight In the case of two-weight CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

20. Input
From
We construct the x as vector with first element is the most recent
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

21. Plants with Multiple Weight (aka “Transversal Filter”)
If we have multiple weights
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

22. Regression input signal vector
If the current time is n, we have “Regression input signal vector”
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

23. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

24. Convolution Output of plant is a convolution Ex For N=2 CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

25. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

26. We can use a vector-matrix multiplication
For example, for n=3 we construct y(3) as
For example, for n=1 we construct y(1) as
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

27. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

28. Let us stop there to consider Random signal theory first.
The error squared is
Let us stop there to consider Random signal theory first.
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

29. Review of Random Signals
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

30. Wireless Transmissions
Ideal signal transmission 1 1 1 1 1 1 1
Information
Information is Random
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

31. Random variable CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

32. Random Variable Random variable is a function
For a single time Coin Tossing
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

33. Our signal x(n) is a Random Variable
For a series of Coin Tossing
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

34. Coin tossing and Random Variable
If random
We have random variable X
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

35. Random Digital Signal
If the random variable is a function of time, it is called a stochastic process
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

36. Probability Mass Function
We need also to define the probability of each random variable
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

37. Probability Mass Function
PMF is for Discrete distribution function
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

38. Time and Emsemble CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

39. Probability of X(2) CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

40. Probability Density Function
PDF is for Continuous Distribution Function
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

41. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

42. Probability Density Function
PDF values can be > 1 as long as its area under curve is 1 2 1
1/2 1
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

43. Cumulative Distribution Function
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

44. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

45. Expectation Operator CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

46. Expected Value Expected value is known as the “Mean” CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

47. Example of Expected Value (Discrete)
We toss a die N times and get a set of outcomes
Suppose we roll a die with N=6, we might get
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

48. Example of Expected Value (Discrete)
But, empirically we have Empirical (Monte Carlo) estimate as Expected Value
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

49. Theoretical Expected Value
But in theory, for a die
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

50. Ensemble Average 1 ensemble i ensembles CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

51. Ensemble Average CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

52. I) Linearity
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

53. II)
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

54. III)
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

55. Autocorrelation CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

56. CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

57. Autocorrelation n=m CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

58. Autocorrelation Matrix
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

59. Covariance
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

60. Stationarity (I) I) n1 n2 CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

61. Stationarity (II) II) CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

62. Expected Value of Error Energy
Let’s take the expected value of error energy
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

63. Vector-Matrix Differentiation
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

64. Partial diff. and set to zero
Differentiation
Result:
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

65. 2-D Error surface CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

66. Four Basic Classes of Adaptive Signal Processing
I) Identification
II) Inverse Modelling
III) Prediction
IV) Interference Cancelling
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

67. The Four Classes of Adaptive Filtering
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

68. System Identification
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

69. Inverse Modelling CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

70. Prediction
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

71. Interference Canceller
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon

72. What are we looking for in Adaptive Systems?
Rate of Convergence
Misadjustment
Tracking
Robustness
Computational Complexity
Numerical Properties
CESdSP
EECS0712 Adaptive Signal Processing Assoc. Prof. Dr. P.Yuvapoositanon