1. Lecture 6 Power spectral density (PSD)
Stochastic processes
Lecture 6
Power spectral density (PSD)

2. Random process

3. 1st order Distribution & density function
First-order distribution First-order density function

4. 2end order Distribution & density function
2end order distribution 2end order density function

5. EXPECTATIONS
Expected value
The autocorrelation

6. Some random processes Single pulse Multiple pulses
Periodic Random Processes
The Gaussian Process
The Poisson Process
Bernoulli and Binomial Processes
The Random Walk Wiener Processes
The Markov Process

7. Single pulse X (t) = A S(t −Θ)
Single pulse with random amplitude and arrival time: Deterministic pulse: S(t): Deterministic function. Random variables: A: gain a random variable Θ: arrival time. A and Θ are statistically independent
X (t) = A S(t −Θ)

8. Multiple pulses
Single pulse with random amplitude and arrival time: Deterministic pulse: S(t): Deterministic function. Random variables: Ak: gain a random variable Θk: arrival time. n: number of pulses Ak and Θk are statistically independent
x 𝑡 = 𝑘=1 𝑛 𝐴 𝑘 𝑆(𝑡− 𝛩 𝑘 )

9. Periodic Random Processes
A process which is periodic with T
x 𝑡 =𝑥 𝑡+𝑛𝑇
n is an integrer
x 𝑡 =𝑠𝑖𝑛 2𝜋𝑡 50 +Θ +𝑠𝑖𝑛 2𝜋𝑡 100 +Θ

10. The Gaussian Process
X(t1),X(t2),X(t3),….X(tn) are jointly Gaussian fro all t and n values
Example: randn() in Matlab

11. The Poisson Process
Typically used for modeling of cumulative number of events over time.
Example: counting the number of phone call from a phone
𝑃 𝑋 𝑡 =𝑘 = 𝜆 𝑡 𝑘 𝑘! 𝑒 −𝜆(𝑡)

12. Alternative definition Poisson points
The number of events in an interval
N(t1,t2)
𝑃 𝑁 𝑡1,𝑡2 =𝑘 =𝑃 𝑋 𝑡2 −𝑋 𝑡1 =𝑘 = 𝜆 𝑡2−𝑡1 𝑘 𝑘! 𝑒 −𝜆(𝑡2−𝑡1)
𝑃 𝑁 0,𝑡2 =𝑘 =𝑃 𝑋 𝑡 =𝑘 = 𝜆𝑡 𝑘 𝑘! 𝑒 −𝜆𝑡

13. Bernoulli Processes A process of zeros and ones
X=[ ]
Each sample must be independent and identically distributed Bernoulli variables.
The likelihood of 1 is defined by p
The likelihood of 0 is defined by q=1-p

14. Binomial process Summed Bernoulli Processes
Where X[n] is a Bernoulli Processes

15. Random walk
For every T seconds take a step (size Δ) to the left or right after tossing a fair coin

16. The Markov Process 1st order Markov process
The current sample is only depended on the previous sample
Density function
Expected value

17. The frequency of earth quakes
Statement the number large earth quakes has increased dramatically in the last 10 year!

18. The frequency of earth quakes
Is the frequency of large earth quakes unusual high?
Which random processes can we use for modeling of the earth quakes frequency?

19. The frequency of earth quakes
Data

20. Agenda (Lec 16) Power spectral density Definition and background
Wiener-Khinchin
Cross spectral densities
Practical implementations
Examples

21. Fourier transform recap 1
Transform between time and frequency domain
Fourier transform
Invers Fourier transform

22. Fourier transform recap 2
Assumption: The signal can be reconstructed from sines and cosines functions.
Requirement: absolute integrable
𝑒 −𝑗2𝜋𝑓𝑡 = cos 2𝜋𝑓𝑡 −𝑗sin 2𝜋𝑓𝑡

23. Fourier transform of a stochastic process
A stationary stochastic process is typical not absolute integrable
There the signal is truncated
Before Fourier transform

24. What is power?
In the power spectrum density power is related to electrical power
𝑃= 𝑉 2 𝑅

25. Power of a signal
The power of a signal is calculated by squaring the signal.
𝑥(𝑡) 2
The average power in e period is :

26. Parseval's theorem
The power of the squared absolute Fourier transform is equal the power of the signal

27. Power of a stochastic process
Thereby can the expected power can be calculated from the Fourier spectrum

28. Power spectrum density
Since the integral of the squared absolute Fourier transform contains the full power of the signal it is a density function.
So the power spectral density of a random process is:
Due to absolute factor the PSD is always real

29. PSD Example
Fourier transform
|X(f)|2

30. Power spectrum density
The PSD is a density function.
In the case of the random process the PSD is the density function of the random process and not necessarily the frequency spectrum of a single realization.
Example
A random process is defined as
Where ωr is a unifom distributed random variable wiht a range from 0-π
What is the PSD for the process and
The power sepctrum for a single realization
X 𝑡 =sin⁡( 𝜔 𝑟 𝑡)

31. PSD of random process versus spectrum of deterministic signals
In the case of the random process the PSD is usual the expected value E[Sxx(f)]
In the case of deterministic signals the PSD is exact (There is still estimation error)

32. Properties of the PSD Sxx(f) is real and nonnegative
The average power in X(t) is given by:
𝐸 𝑋 2 (𝑡) =𝑅𝑥𝑥 0 = −∞ ∞ 𝑆𝑥𝑥 𝑓 𝑑𝑓
If X(t) is real Rxx(τ) and Sxx(f) are also even
𝑆𝑥𝑥 −𝑓 =𝑆𝑥𝑥 𝑓
If X(t) has periodic components Sxx(f)has impulses
Independent on phase

33. Wiener-Khinchin 1
If the X(t) is stationary in the wide-sense the PSD is the Fourier transform of the Autocorrelation
Proof: page 175

34. Wiener-Khinchin Two method for estimation of the PSD
Fourier Transform
|X(f)|2
X(t)
Sxx(f)
Fourier Transform
Autocorrelation

35. The inverse Fourier Transform of the PSD
Since the PSD is the Fourier transformed autocorrelation
The inverse Fourier transform of the PSD is the autocorrelation

36. Cross spectral densities
If X(t) and Y(t) are two jointly wide-sense stationary processes, is the Cross spectral densities Or

37. Properties of Cross spectral densities
Since is
Syx(f) is not necessary real
If X(t) and Y(t) are orthogonal
Sxy(f)=0
If X(t) and Y(t) are independent
Sxy(f)=E[X(t)] E[Y(t)] δ(f)

38. Cross spectral densities example
1 Hz Sinus curves in white noise
Where w(t) is Gaussian noise
𝑋 𝑡 = sin 2𝜋 𝑡 +3 𝑤(𝑡)
𝑌 𝑡 = sin 2𝜋 𝑡+ 𝜋 𝑤(𝑡)

39. Implementations issues
The challenges includes
Finite signals
Discrete time

40. The periodogram The estimate of the PSD
The PSD can be estimate from the autocorrelation
Or directly from the signal
𝑆𝑥𝑥 ω = 𝑚=−𝑁+1 𝑁−1 𝑅𝑥𝑥 [𝑚] 𝑒 −𝑗ω𝑚
𝑆𝑥𝑥 ω = 1 𝑁 𝑛=0 𝑁−1 𝑥 [𝑛] 𝑒 −𝑗ω𝑛 2

41. The discrete version of the autocorrelation
Rxx(τ)=E[X1(t) X(t+τ)]≈Rxx[m]
m=τ where m is an integer
N: number of samples
Normalized version:
𝑅𝑥𝑥 𝑚 = 𝑛=0 𝑁− 𝑚 −1 𝑥 𝑛 𝑥[𝑛+𝑚]
𝑅𝑥𝑥 𝑚 = 1 𝑁 𝑛=0 𝑁− 𝑚 −1 𝑥 𝑛 𝑥[𝑛+𝑚]

42. Bias in the estimates of the autocorrelation
𝑅𝑥𝑥 𝑚 = 𝑛=0 𝑁− 𝑚 −1 𝑥 𝑛 𝑥[𝑛+𝑚]

43. Bias in the estimates of the autocorrelation
The edge effect correspond to multiplying the true autocorrelation with a Bartlett window
𝐸[𝑅𝑥𝑥 𝑚 ]=𝑤[𝑚]𝑅𝑥𝑥_𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑[𝑚]
𝑤𝑏 𝑚 = 𝑁−|𝑚| 𝑁 𝑚 <𝑁 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

44. Alternative estimation of autocorrelation
The unbiased estimate
Disadvantage: high variance when |m|→N
𝑅𝑥𝑥 𝑚 = 1 𝑁−|𝑚| 𝑛=0 𝑁− 𝑚 −1 𝑥 𝑛 𝑥[𝑛+𝑚]

45. Influence at the power spectrum
Biased version: a Bartlett window is applied
Unbiased version: a Rectangular window is applied
𝑆𝑥𝑥 ω = 𝑚=−∞ ∞ 𝑤𝑟[𝑚]𝑅𝑥𝑥_𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑[𝑚] 𝑒 −𝑗ω𝑚
𝑆𝑥𝑥 ω = 𝑚=−∞ ∞ 𝑤𝑏[𝑚]𝑅𝑥𝑥_𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑[𝑚] 𝑒 −𝑗ω𝑚
𝑤𝑟 𝑚 = 1 𝑚 <𝑁 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

46. Example
Autocorrelation biased and unbiased
Estimated PSD’s

47. Variance in the PSD
The variance of the periodogram is estimated to the power of two of PSD
𝑉𝑎𝑟 𝑆𝑥𝑥 𝜔 = 𝑆𝑥𝑥(𝜔) 2

48. Averaging Divide the signal into K segments of M length
𝑥𝑖=𝑥 𝑖−1 𝑀+1:𝑖 𝑀 ≤𝑖≤𝐾
Calculate the periodogram of each segment
𝑆𝑖𝑥𝑥 ω = 1 𝑀 𝑛=0 𝑀−1 𝑥 𝑖[𝑛] 𝑒 −𝑗ω𝑛 2
Calculate the average periodogram
𝑆 𝑥𝑥[ω]= 1 𝐾 𝑖=0 𝐾 𝑆𝑖𝑥𝑥[ω]

49. Illustrations of Averaging

50. Effect of Averaging The variance is decreased
But the spectral resolution is also decreased
𝑉𝑎𝑟 𝑆𝑥𝑥 𝜔 = 1 𝐾 𝑆𝑥𝑥(𝜔) 2

51. Additional options The Welch method
Introduce overlap between segment
𝑥𝑖=𝑥 𝑖−1 𝑄+1: 𝑖−1 𝑄+𝑀 ≤𝑖≤𝐾
Where Q is the length between the segments
Multiply the segment's with windows
𝑆𝑖𝑥𝑥 ω = 1 𝑀 𝑛=0 𝑀−1 𝑤[𝑛]𝑥 𝑖[𝑛] 𝑒 −𝑗ω𝑛 2

52. Example Heart rate variability
High frequency component related to Parasympathetic nervous system ("rest and digest")
Low frequency component related to sympathetic nervous system (fight-or-flight)

53. Agenda (Lec 16) Power spectral density Definition and background
Wiener-Khinchin
Cross spectral densities
Practical implementations
Examples