Presentation is loading. Please wait.

Presentation is loading. Please wait.

Environmental Data Analysis with MatLab Lecture 19: Smoothing, Correlation and Spectra.

Similar presentations


Presentation on theme: "Environmental Data Analysis with MatLab Lecture 19: Smoothing, Correlation and Spectra."— Presentation transcript:

1 Environmental Data Analysis with MatLab Lecture 19: Smoothing, Correlation and Spectra

2 Lecture 01Using MatLab Lecture 02Looking At Data Lecture 03Probability and Measurement Error Lecture 04Multivariate Distributions Lecture 05Linear Models Lecture 06The Principle of Least Squares Lecture 07Prior Information Lecture 08Solving Generalized Least Squares Problems Lecture 09Fourier Series Lecture 10Complex Fourier Series Lecture 11Lessons Learned from the Fourier Transform Lecture 12Power Spectral Density Lecture 13Filter Theory Lecture 14Applications of Filters Lecture 15Factor Analysis Lecture 16Orthogonal functions Lecture 17Covariance and Autocorrelation Lecture 18Cross-correlation Lecture 19Smoothing, Correlation and Spectra Lecture 20Coherence; Tapering and Spectral Analysis Lecture 21Interpolation Lecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-Tests Lecture 24 Confidence Limits of Spectra, Bootstraps SYLLABUS

3 purpose of the lecture examine interrelationships between smoothing, correlation and power spectral density

4 review

5 Autocorrelation and Cross-correlation

6 Autocorrelation Measure of correlation in time series at different lags t u(t)

7 Autocorrelation Measure of correlation in time series at different lags t t lag, t a(t) 0 lag, multiply and sum area no lag

8 Autocorrelation Measure of correlation in time series at different lags t t lag, t a(t) 0 lag, multiply and sum area small lag

9 Autocorrelation Measure of correlation in time series at different lags t t lag, t a(t) 0 lag, multiply and sum area large lag

10 Autocorrelation Measure of correlation in time series at different lags t t lag, t a(t) 0 lag, multiply and sum area

11 Autocorrelation Measure of correlation in time series at different lags t t lag, t a(t) 0 lag, multiply and sum area a(t)=u(t) ⋆ u(t)

12 crooss-correlation Measure of correlation between two time series at different lags t t u(t) v(t)

13 crooss-correlation Measure of correlation between two time series at different lags t t u(t) v(t) c(t)=u(t) ⋆ v(t)

14 important relationships c(t) = u(t) ⋆ v(t) = u(-t)*v(t) c( ω) = u*( ω ) v( ω ) a( ω) = |u( ω )| 2

15 rough time series frequency, ω 0 tu(t) lag, t a(t) 0 sharp autocorrelation wide spectrum |u( ω )| 2

16 smooth time series frequency, ω 0 tu(t) lag, t a(t) 0 wide autocorrelation narrow spectrum |u( ω )| 2

17 lag, t a(t) 0 rough timeseries lag, t a(t) 0 tv(t) t lag, t a(t) 0

18 Part 1 Smoothing a Time Series

19 smoothing as filtering (example of 3-point smoothing)

20 non-causal smoothing as filtering (example of 3-point smoothing)

21 fix-up allow for a delay

22 d smoothed and delayed = s * d obs causal filter, s

23 triangular smoothing filters sisi sisi index, i 3 points 21 points

24 smoothing if Neuse River Hydrograph

25 question how does smoothing effect the the autocorrelation of d

26 answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d

27 effect of smoothing on autocorrelation

28 autocorrelation of smoothed time series

29 effect of smoothing on autocorrelation autocorrelation of smoothed time series everything written as convolution

30 effect of smoothing on autocorrelation autocorrelation of smoothed time series everything written as convolution regrouped

31 effect of smoothing on autocorrelation autocorrelation of smoothing filter autocorrelation of time series convolved with *

32 answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d

33 Part 2 What Makes a Good Smoothing Filter?

34 then by the convolution theorem d smoothed (t) = s(t) * d obs (t)

35 then by the convolution theorem d smoothed (t) = s(t) * d obs (t) so what’s this look like?

36 example of a uniform or “boxcar” smoothing filter s(t) time, t T 0 1/T

37 take Fourier Transform where sinc(x) = sin(πx) / (πx)

38 A) T=3 B) T=21

39 falls off with frequency (good)

40 B) T=21 bumpy side lobes (bad)

41 a box car filter does not suppress high frequencies evenly the challenge find a filter that suppresses high frequencies evenly

42 Normal Function Fourier Transform of a Normal Function is a Normal Function (which has no side lobes)

43 A) L=3 B) T=21 B) T=3

44 but a Normal Function is non-causal (unless you truncate it, in which case it is not exactly a Normal Function)

45 Normal Function Box Car Triangle simplicity sidelobes

46 Part 3 Designing a Filter that Suppresses Specific Frequencies

47 General form of the IIR Filter, f

48 z-transform of the IIR filter

49 General form of the IIR Filter z-transform

50 General form of the IIR Filter z-transform ratio of polynomials

51 z-transform v(z) as a product of its factors u(z) as a product of its factors roots of u(z) roots of v(z) z-transform of the IIR filter ratio of polynomials

52 so designing a filter is equivalent to specifying the roots of the two polynomails u(z) and v(z)

53 at this point we need to explore the relationship between the Fourier Transform and the z-transform

54 Answer the Fourier Transform is the z-transform evaluated at a specific set of z’s

55 Relationship between Fourier Transform and Z-transform since

56 Relationship between Fourier Transform and Z-transform since Fourier Transform

57 Relationship between Fourier Transform and Z-transform since Fourier Transformdiscrete times and frequencies

58 Relationship between Fourier Transform and Z-transform since Fourier Transformdiscrete times and frequencies z-transform

59 Relationship between Fourier Transform and Z-transform since Fourier Transformdiscrete times and frequencies z-transform specific choice of z’ s

60 in words the Fourier Transform is the z-transform evaluated at a specific set of z’s

61 there are N specific z ’s zkzk or with θ

62 real z  imag z unit circle, |z| 2 =1 they plot as equally-spaced points around a “unit circle” in the complex z-plane zero frequency Nyquist frequency

63 Back to the IIR Filter roots of u(z) roots of v(z)

64 Back to the IIR Filter (z-z j u ) is zero at z=z j u produces a low amplitude region near z=z j u called a “zero”

65 Back to the IIR Filter 1/(z-z k v ) is infinite at z=z k u produces a high amplitude region near z=z k v called a “pole”

66 so build a filter by placing the poles and zeros at strategic points in the complex z-plane

67 Rules zeros suppress frequencies poles amplify frequencies all poles must be outside the unit circle (so v inv converges) all poles, zeros must be in complex conjugate pairs (so filter is real)

68 A) B)

69 A) B) zero near zero frequency suppresses low frequencies “high pass filter” zero near the Nyquist frequency suppresses high frequencies “low pass filter”

70 A) B)

71 A) B) poles near ± a given frequency amplify that frequency “band pass filter” poles and zeros near ± a given frequency attenuate that frequency “notch filter”

72 something useful a tunable band pass filter frequency, f -f ny +f ny 0 |f(ω)| 2 f1f1 f2f2 -f 2 -f 1

73 Chebychev band-pass filter: 4 zeros, 4 poles

74 2 zeros pole

75

76 not quite as boxy as one might hope …

77

78 In MatLab

79 Ground velocity at Palisades NY

80 Low pass filter

81 Ground velocity at Palisades NY high pass filter


Download ppt "Environmental Data Analysis with MatLab Lecture 19: Smoothing, Correlation and Spectra."

Similar presentations


Ads by Google