Presentation is loading. Please wait.

Presentation is loading. Please wait.

CMU SCS 15-826: Multimedia Databases and Data Mining Lecture #23: DSP tools – Fourier and Wavelets C. Faloutsos.

Similar presentations


Presentation on theme: "CMU SCS 15-826: Multimedia Databases and Data Mining Lecture #23: DSP tools – Fourier and Wavelets C. Faloutsos."— Presentation transcript:

1 CMU SCS : Multimedia Databases and Data Mining Lecture #23: DSP tools – Fourier and Wavelets C. Faloutsos

2 CMU SCS Copyright: C. Faloutsos (2012)2 Must-read Material DFT/DCT: In PTVF ch. 12.1, 12.3, 12.4; in Textbook Appendix B.PTVF Textbook Wavelets: In PTVF ch ; in MM Textbook Appendix CPTVFMM Textbook

3 CMU SCS Copyright: C. Faloutsos (2012)3 Outline Goal: ‘Find similar / interesting things’ Intro to DB Indexing - similarity search Data Mining

4 CMU SCS Copyright: C. Faloutsos (2012)4 Indexing - Detailed outline primary key indexing.. multimedia Digital Signal Processing (DSP) tools –Discrete Fourier Transform (DFT) –Discrete Wavelet Transform (DWT)

5 CMU SCS Copyright: C. Faloutsos (2012)5 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

6 CMU SCS Copyright: C. Faloutsos (2012)6 Introduction Goal: given a signal (eg., sales over time and/or space) Find: patterns and/or compress year count lynx caught per year

7 CMU SCS Copyright: C. Faloutsos (2012)7 What does DFT do? A: highlights the periodicities

8 CMU SCS Copyright: C. Faloutsos (2012)8 Why should we care? A: several real sequences are periodic Q: Such as?

9 CMU SCS Copyright: C. Faloutsos (2012)9 Why should we care? A: several real sequences are periodic Q: Such as? A: –sales patterns follow seasons; –economy follows 50-year cycle –temperature follows daily and yearly cycles Many real signals follow (multiple) cycles

10 CMU SCS Copyright: C. Faloutsos (2012)10 Why should we care? For example: human voice! Frequency analyzer speaker identification impulses/noise -> flat spectrum high pitch -> high frequency Freq.exe

11 CMU SCS Copyright: C. Faloutsos (2012)11 ‘Frequency Analyzer’

12 CMU SCS Copyright: C. Faloutsos (2012)12 DFT and stocks Dow Jones Industrial index, 6/18/ /21/2001

13 CMU SCS Copyright: C. Faloutsos (2012)13 DFT and stocks Dow Jones Industrial index, 6/18/ /21/2001 just 3 DFT coefficients give very good approximation freq Log(ampl)

14 CMU SCS Copyright: C. Faloutsos (2012)14 DFT: definition Discrete Fourier Transform (n-point): inverse DFT

15 CMU SCS Copyright: C. Faloutsos (2012)15 How does it work? Decomposes signal to a sum of sine (and cosine) waves. Q:How to assess ‘similarity’ of x with a wave? 0 1 n-1 time value x ={x 0, x 1,... x n-1 } Skip

16 CMU SCS Copyright: C. Faloutsos (2012)16 How does it work? A: consider the waves with frequency 0, 1,...; use the inner-product (~cosine similarity) 0 1 n-1 time value freq. f=0 0 1 n-1 time value freq. f=1 (sin(t * 2  n) ) Skip

17 CMU SCS Copyright: C. Faloutsos (2012)17 How does it work? A: consider the waves with frequency 0, 1,...; use the inner-product (~cosine similarity) 0 1 n-1 time value freq. f=2 Skip

18 CMU SCS Copyright: C. Faloutsos (2012)18 How does it work? ‘basis’ functions 0 1 n sine, freq =1 sine, freq = n cosine, f=1 cosine, f=2 Skip

19 CMU SCS Copyright: C. Faloutsos (2012)19 How does it work? Basis functions are actually n-dim vectors, orthogonal to each other ‘similarity’ of x with each of them: inner product DFT: ~ all the similarities of x with the basis functions Skip

20 CMU SCS Copyright: C. Faloutsos (2012)20 How does it work? Since e jf = cos(f) + j sin(f) (j=sqrt(-1)), we finally have: Skip

21 CMU SCS Copyright: C. Faloutsos (2012)21 DFT: definition Discrete Fourier Transform (n-point): inverse DFT

22 CMU SCS Copyright: C. Faloutsos (2012)22 DFT: definition Good news: Available in all symbolic math packages, eg., in ‘mathematica’ x = [1,2,1,2]; X = Fourier[x]; Plot[ Abs[X] ];

23 CMU SCS Copyright: C. Faloutsos (2012)23 DFT: definition (variations: 1/n instead of 1/sqrt(n) exp(-...) instead of exp(+...)

24 CMU SCS Copyright: C. Faloutsos (2012)24 DFT: definition (variations: … or even

25 CMU SCS Copyright: C. Faloutsos (2012)25 DFT: definition Observations: X f : are complex numbers except –X 0, who is real Im (X f ): ~ amplitude of sine wave of frequency f Re (X f ): ~ amplitude of cosine wave of frequency f x: is the sum of the above sine/cosine waves

26 CMU SCS Copyright: C. Faloutsos (2012)26 DFT: definition Observation - SYMMETRY property: X f = (X n-f )* ( “*”: complex conjugate: (a + b j)* = a - b j ) Skip

27 CMU SCS Copyright: C. Faloutsos (2012)27 DFT: definition Definitions A f = |X f | : amplitude of frequency f |X f | 2 = Re(X f ) 2 + Im(X f ) 2 = energy of frequency f phase  f at frequency f XfXf Re Im AfAf ff

28 CMU SCS Copyright: C. Faloutsos (2012)28 DFT: definition Amplitude spectrum: | X f | vs f (f=0, 1,... n-1) 0 1n-1 SYMMETRIC (Thus, we plot the first half only) Skip

29 CMU SCS Copyright: C. Faloutsos (2012)29 DFT: definition Phase spectrum |  f | vs f (f=0, 1,... n-1): Anti-symmetric (Rarely used) Skip

30 CMU SCS Copyright: C. Faloutsos (2012)30 DFT: examples flat time freq Amplitude

31 CMU SCS Copyright: C. Faloutsos (2012)31 DFT: examples Low frequency sinusoid time freq

32 CMU SCS Copyright: C. Faloutsos (2012)32 DFT: examples Sinusoid - symmetry property: X f = X * n-f time freq

33 CMU SCS Copyright: C. Faloutsos (2012)33 DFT: examples Higher freq. sinusoid time freq

34 CMU SCS Copyright: C. Faloutsos (2012)34 DFT: examples examples = + +

35 CMU SCS Copyright: C. Faloutsos (2012)35 DFT: examples examples Freq. Ampl.

36 CMU SCS Copyright: C. Faloutsos (2012)36 DFT: Amplitude spectrum year count Freq. Ampl. freq=12 freq=0 Amplitude:

37 CMU SCS Copyright: C. Faloutsos (2012)37 DFT: Amplitude spectrum year count Freq. Ampl. freq=12 freq=0

38 CMU SCS Copyright: C. Faloutsos (2012)38 DFT: Amplitude spectrum year count Freq. Ampl. freq=12 freq=0

39 CMU SCS Copyright: C. Faloutsos (2012)39 DFT: Amplitude spectrum excellent approximation, with only 2 frequencies! so what? Freq.

40 CMU SCS Copyright: C. Faloutsos (2012)40 DFT: Amplitude spectrum excellent approximation, with only 2 frequencies! so what? A1: compression A2: pattern discovery (A3: forecasting)

41 CMU SCS Copyright: C. Faloutsos (2012)41 DFT: Amplitude spectrum excellent approximation, with only 2 frequencies! so what? A1: (lossy) compression A2: pattern discovery

42 CMU SCS Copyright: C. Faloutsos (2012)42 DFT: Amplitude spectrum excellent approximation, with only 2 frequencies! so what? A1: (lossy) compression A2: pattern discovery

43 CMU SCS Copyright: C. Faloutsos (2012)43 DFT: Amplitude spectrum Let’s see it in action (defunct now…) (http://www.dsptutor.freeuk.com/jsanalyser/FFTSpectrumAnalyser.html)(http://www.dsptutor.freeuk.com/jsanalyser/FFTSpectrumAnalyser.html plain sine phase shift two sine waves the ‘chirp’ function

44 CMU SCS Copyright: C. Faloutsos (2012)44 Plain sine

45 CMU SCS Copyright: C. Faloutsos (2012)45 Plain sine

46 CMU SCS Copyright: C. Faloutsos (2012)46 Plain sine – phase shift

47 CMU SCS Copyright: C. Faloutsos (2012)47 Plain sine – phase shift

48 CMU SCS Copyright: C. Faloutsos (2012)48 Plain sine

49 CMU SCS Copyright: C. Faloutsos (2012)49 Two sines

50 CMU SCS Copyright: C. Faloutsos (2012)50 Two sines

51 CMU SCS Copyright: C. Faloutsos (2012)51 Chirp

52 CMU SCS Copyright: C. Faloutsos (2012)52 Chirp

53 CMU SCS Copyright: C. Faloutsos (2012)53 Another applet (seems virus-free – but scan, before you install) FFT_applet/index.html

54 CMU SCS Properties Time shift sounds the same –Changes only phase, not amplitudes Sawtooth has almost all frequencies –With decreasing amplitude Spike has all frequencies Copyright: C. Faloutsos (2012)54

55 CMU SCS Copyright: C. Faloutsos (2012)55 DFT: Parseval’s theorem sum( x t 2 ) = sum ( | X f | 2 ) Ie., DFT preserves the ‘energy’ or, alternatively: it does an axis rotation: x0 x1 x = {x0, x1}

56 CMU SCS Copyright: C. Faloutsos (2012)56 DFT: Parseval’s theorem sum( x t 2 ) = sum ( | X f | 2 ) Ie., DFT preserves the ‘energy’ or, alternatively: it does an axis rotation: x0 x1 x = {x0, x1}

57 CMU SCS Copyright: C. Faloutsos (2012)57 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

58 CMU SCS Copyright: C. Faloutsos (2012)58 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? 0 1 time value 1 Skip

59 CMU SCS Copyright: C. Faloutsos (2012)59 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 =? X 2 =? X 3 =? Skip

60 CMU SCS Copyright: C. Faloutsos (2012)60 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 = -1/2 j X 2 = - 1/2 X 3 = + 1/2 j Q: does the ‘symmetry’ property hold? Skip

61 CMU SCS Copyright: C. Faloutsos (2012)61 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 = -1/2 j X 2 = - 1/2 X 3 = + 1/2 j Q: does the ‘symmetry’ property hold? A: Yes (of course) Skip

62 CMU SCS Copyright: C. Faloutsos (2012)62 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 = -1/2 j X 2 = - 1/2 X 3 = + 1/2 j Q: check Parseval’s theorem Skip

63 CMU SCS Copyright: C. Faloutsos (2012)63 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 = -1/2 j X 2 = - 1/2 X 3 = + 1/2 j Q: (Amplitude) spectrum? Skip

64 CMU SCS Copyright: C. Faloutsos (2012)64 Arithmetic examples Impulse function: x = { 0, 1, 0, 0} (n = 4) X 0 =? A: X 0 = 1/sqrt(4) * 1* exp(-j 2  0 / n ) = 1/2 X 1 = -1/2 j X 2 = - 1/2 X 3 = + 1/2 j Q: (Amplitude) spectrum? A: FLAT!

65 CMU SCS Copyright: C. Faloutsos (2012)65 Arithmetic examples Q: What does this mean?

66 CMU SCS Copyright: C. Faloutsos (2012)66 Arithmetic examples Q: What does this mean? A: All frequencies are equally important -> –we need n numbers in the frequency domain to represent just one non-zero number in the time domain! – “frequency leak”

67 CMU SCS Copyright: C. Faloutsos (2012)67 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

68 CMU SCS Copyright: C. Faloutsos (2012)68 Observations DFT of ‘step’ function: x = { 0, 0,..., 0, 1, 1,... 1} t xtxt

69 CMU SCS Copyright: C. Faloutsos (2012)69 Observations DFT of ‘step’ function: x = { 0, 0,..., 0, 1, 1,... 1} t xtxt f = 0

70 CMU SCS Copyright: C. Faloutsos (2012)70 Observations DFT of ‘step’ function: x = { 0, 0,..., 0, 1, 1,... 1} t xtxt f = 0 f=1

71 CMU SCS Copyright: C. Faloutsos (2012)71 Observations DFT of ‘step’ function: x = { 0, 0,..., 0, 1, 1,... 1} t xtxt f = 0 f=1 the more frequencies, the better the approx. ‘ringing’ becomes worse reason: discontinuities; trends

72 CMU SCS Copyright: C. Faloutsos (2012)72 Observations Ringing for trends: because DFT ‘sub- consciously’ replicates the signal t xtxt

73 CMU SCS Copyright: C. Faloutsos (2012)73 Observations Ringing for trends: because DFT ‘sub- consciously’ replicates the signal t xtxt

74 CMU SCS Copyright: C. Faloutsos (2012)74 Observations Ringing for trends: because DFT ‘sub- consciously’ replicates the signal t xtxt

75 CMU SCS Copyright: C. Faloutsos (2012)75 original

76 CMU SCS Copyright: C. Faloutsos (2012)76 DC and 1st

77 CMU SCS Copyright: C. Faloutsos (2012)77 And 2nd DC and 1st

78 CMU SCS Copyright: C. Faloutsos (2012)78 And 3rd DC and 1st And 2nd

79 CMU SCS Copyright: C. Faloutsos (2012)79 And 4th DC and 1st And 3rd And 2nd

80 CMU SCS Copyright: C. Faloutsos (2012)80 Observations Q: DFT of a sinusoid, eg. x t = 3 sin( 2  / 4 t) (t = 0,..., 3) Q: X 0 = ? Q: X 1 = ? Q: X 2 = ? Q: X 3 = ?

81 CMU SCS Copyright: C. Faloutsos (2012)81 Observations Q: DFT of a sinusoid, eg. x t = 3 sin( 2  / 4 t) (t = 0,..., 3) Q: X 0 = 0 Q: X 1 = -3 j Q: X 2 = 0 Q: X 3 = 3j check ‘symmetry’ check Parseval

82 CMU SCS Copyright: C. Faloutsos (2012)82 Q: DFT of a sinusoid, eg. x t = 3 sin( 2  / 4 t) (t = 0,..., 3) Q: X 0 = 0 Q: X 1 = -3 j Q: X 2 = 0 Q: X 3 = 3j Observations Does this make sense? f AfAf 0 1 2

83 CMU SCS Copyright: C. Faloutsos (2012)83 Shifting x in time does NOT change the amplitude spectrum eg., x = { } and x’ = { }: same (flat) amplitude spectrum (only the phase spectrum changes) Useful property when we search for patterns that may ‘slide’ Property

84 CMU SCS Copyright: C. Faloutsos (2012)84 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

85 CMU SCS Copyright: C. Faloutsos (2012)85 DCT Discrete Cosine Transform motivation#1: DFT gives complex numbers motivation#2: how to avoid the ‘frequency leak’ of DFT on trends? t xtxt Skip

86 CMU SCS Copyright: C. Faloutsos (2012)86 DCT brilliant solution to both problems: mirror the sequence, do DFT, and drop the redundant entries! t xtxt Skip

87 CMU SCS Copyright: C. Faloutsos (2012)87 DCT (see Numerical Recipes for exact formulas) Skip

88 CMU SCS Copyright: C. Faloutsos (2012)88 DCT - properties it gives real numbers as the result it has no problems with trends it is very good when x t and x (t+1) are correlated (thus, is used in JPEG, for image compression)

89 CMU SCS Copyright: C. Faloutsos (2012)89 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

90 CMU SCS Copyright: C. Faloutsos (2012)90 2-d DFT Definition:

91 CMU SCS Copyright: C. Faloutsos (2012)91 2-d DFT Intuition: n1 n2 do 1-d DFT on each row and then 1-d DFT on each column

92 CMU SCS Copyright: C. Faloutsos (2012)92 2-d DFT Quiz: how do the basis functions look like? for f1= f2 =0 for f1=1, f2=0 for f1=1, f2=1

93 CMU SCS Copyright: C. Faloutsos (2012)93 2-d DFT Quiz: how do the basis functions look like? for f1= f2 =0 flat for f1=1, f2=0wave on x; flat on y for f1=1, f2=1~ egg-carton

94 CMU SCS Copyright: C. Faloutsos (2012)94 2-d DFT Quiz: how do the basis functions look like? for f1= f2 =0 flat for f1=1, f2=0wave on x; flat on y for f1=1, f2=1~ egg-carton

95 CMU SCS Copyright: C. Faloutsos (2012)95 DSP - Detailed outline DFT –what –why –how –Arithmetic examples –properties / observations –DCT –2-d DFT –Fast Fourier Transform (FFT)

96 CMU SCS Copyright: C. Faloutsos (2012)96 FFT What is the complexity of DFT? Skip

97 CMU SCS Copyright: C. Faloutsos (2012)97 FFT What is the complexity of DFT? A: Naively, O(n 2 ) Skip

98 CMU SCS Copyright: C. Faloutsos (2012)98 FFT However, if n is a power of 2 (or a number with many divisors), we can make it O(n log n) Main idea: if we know the DFT of the odd time-ticks, and of the even time-ticks, we can quickly compute the whole DFT Details: in Num. Recipes Skip

99 CMU SCS Copyright: C. Faloutsos (2012)99 DFT - Conclusions It spots periodicities (with the ‘amplitude spectrum’) can be quickly computed (O( n log n)), thanks to the FFT algorithm. standard tool in signal processing (speech, image etc signals) Freq.

100 CMU SCS Copyright: C. Faloutsos (2012)100 Detailed outline primary key indexing.. multimedia Digital Signal Processing (DSP) tools –Discrete Fourier Transform (DFT) –Discrete Wavelet Transform (DWT)

101 CMU SCS Copyright: C. Faloutsos (2012)101 Reminder: Problem: Goal: given a signal (eg., #packets over time) Find: patterns, periodicities, and/or compress year count lynx caught per year (packets per day; virus infections per month)

102 CMU SCS Copyright: C. Faloutsos (2012)102 Wavelets - DWT DFT is great - but, how about compressing a spike? value time

103 CMU SCS Copyright: C. Faloutsos (2012)103 Wavelets - DWT DFT is great - but, how about compressing a spike? A: Terrible - all DFT coefficients needed! Freq. Ampl. value time

104 CMU SCS Copyright: C. Faloutsos (2012)104 Wavelets - DWT DFT is great - but, how about compressing a spike? A: Terrible - all DFT coefficients needed! value time

105 CMU SCS Copyright: C. Faloutsos (2012)105 Wavelets - DWT Similarly, DFT suffers on short-duration waves (eg., baritone, silence, soprano) time value

106 CMU SCS Copyright: C. Faloutsos (2012)106 Wavelets - DWT Solution#1: Short window Fourier transform (SWFT) But: how short should be the window? time freq time value

107 CMU SCS Copyright: C. Faloutsos (2012)107 Wavelets - DWT Answer: multiple window sizes! -> DWT time freq Time domain DFT SWFT DWT

108 CMU SCS Copyright: C. Faloutsos (2012)108 Haar Wavelets subtract sum of left half from right half repeat recursively for quarters, eight-ths,...

109 CMU SCS Copyright: C. Faloutsos (2012)109 Wavelets - construction x0 x1 x2 x3 x4 x5 x6 x7

110 CMU SCS Copyright: C. Faloutsos (2012)110 Wavelets - construction x0 x1 x2 x3 x4 x5 x6 x7 s1,0 + - d1,0 s1,1d1, level 1

111 CMU SCS Copyright: C. Faloutsos (2012)111 Wavelets - construction d2,0 x0 x1 x2 x3 x4 x5 x6 x7 s1,0 + - d1,0 s1,1d1, s2,0 level 2

112 CMU SCS Copyright: C. Faloutsos (2012)112 Wavelets - construction d2,0 x0 x1 x2 x3 x4 x5 x6 x7 s1,0 + - d1,0 s1,1d1, s2,0 etc...

113 CMU SCS Copyright: C. Faloutsos (2012)113 Wavelets - construction d2,0 x0 x1 x2 x3 x4 x5 x6 x7 s1,0 + - d1,0 s1,1d1, s2,0 Q: map each coefficient on the time-freq. plane t f

114 CMU SCS Copyright: C. Faloutsos (2012)114 Wavelets - construction d2,0 x0 x1 x2 x3 x4 x5 x6 x7 s1,0 + - d1,0 s1,1d1, s2,0 Q: map each coefficient on the time-freq. plane t f

115 CMU SCS Copyright: C. Faloutsos (2012)115 Haar wavelets - code #!/usr/bin/perl5 # expects a file with numbers # and prints the dwt transform # The number of time-ticks should be a power of 2 # USAGE # haar.pl # the smooth component of the signal # the high-freq. component # collect the values into the = split ); } my $len = my $half = int($len/2); while($half >= 1 ){ for(my $i=0; $i< $half; $i++){ $diff [$i] = ($vals[2*$i] - $vals[2*$i + 1] )/ sqrt(2); print "\t", $diff[$i]; $smooth [$i] = ($vals[2*$i] + $vals[2*$i + 1] )/ sqrt(2); } print $half = int($half/2); } print "\t", $vals[0], "\n" ; # the final, smooth component

116 CMU SCS Copyright: C. Faloutsos (2012)116 Wavelets - construction Observation1: ‘+’ can be some weighted addition ‘-’ is the corresponding weighted difference (‘Quadrature mirror filters’) Observation2: unlike DFT/DCT, there are *many* wavelet bases: Haar, Daubechies- 4, Daubechies-6, Coifman, Morlet, Gabor,...

117 CMU SCS Copyright: C. Faloutsos (2012)117 Wavelets - how do they look like? E.g., Daubechies-4

118 CMU SCS Copyright: C. Faloutsos (2012)118 Wavelets - how do they look like? E.g., Daubechies-4 ? ?

119 CMU SCS Copyright: C. Faloutsos (2012)119 Wavelets - how do they look like? E.g., Daubechies-4

120 CMU SCS Copyright: C. Faloutsos (2012)120 Wavelets - Drill#1: Q: baritone/silence/soprano - DWT? t f time value

121 CMU SCS Copyright: C. Faloutsos (2012)121 Wavelets - Drill#1: t f time value Q: baritone/silence/soprano - DWT?

122 CMU SCS Copyright: C. Faloutsos (2012)122 Wavelets - Drill#2: Q: spike - DWT? t f

123 CMU SCS Copyright: C. Faloutsos (2012)123 Wavelets - Drill#2: t f Q: spike - DWT?

124 CMU SCS Copyright: C. Faloutsos (2012)124 Wavelets - Drill#3: Q: weekly + daily periodicity, + spike - DWT? t f

125 CMU SCS Copyright: C. Faloutsos (2012)125 Wavelets - Drill#3: Q: weekly + daily periodicity, + spike - DWT? t f

126 CMU SCS Copyright: C. Faloutsos (2012)126 Wavelets - Drill#3: Q: weekly + daily periodicity, + spike - DWT? t f

127 CMU SCS Copyright: C. Faloutsos (2012)127 Wavelets - Drill#3: Q: weekly + daily periodicity, + spike - DWT? t f

128 CMU SCS Copyright: C. Faloutsos (2012)128 Wavelets - Drill#3: Q: weekly + daily periodicity, + spike - DWT? t f

129 CMU SCS Copyright: C. Faloutsos (2012)129 Wavelets - Drill#3: Q: DFT? t f t f DWT DFT

130 CMU SCS Copyright: C. Faloutsos (2012)130 Wavelets - Drill: Let’s see it live: delta; cosine; cosine2; chirp Haar vs Daubechies-4, -6, etc

131 CMU SCS Copyright: C. Faloutsos (2012)131 Delta? x(0)=1; x(t)= 0 elsewhere t f

132 CMU SCS Copyright: C. Faloutsos (2012)132 Delta? x(0)=1; x(t)= 0 elsewhere t f

133 CMU SCS Copyright: C. Faloutsos (2012)133 2 cosines? x(t) = cos( 2 * pi * 4 * t / 1024 ) + 5 * cos( 2 * pi * 8 * t / 1024 ) t f

134 CMU SCS Copyright: C. Faloutsos (2012)134 2 cosines? x(t) = cos( 2 * pi * 4 * t / 1024 ) + 5 * cos( 2 * pi * 8 * t / 1024 ) t f Which one is for freq.=4?

135 CMU SCS Copyright: C. Faloutsos (2012)135 2 cosines? x(t) = cos( 2 * pi * 4 * t / 1024 ) + 5 * cos( 2 * pi * 8 * t / 1024 ) t f Which one is for freq.=4? f~4 f~8

136 CMU SCS Copyright: C. Faloutsos (2012)136 Chirp? x(t) = cos( 2 * pi * t * t / 1024 ) t f

137 CMU SCS Copyright: C. Faloutsos (2012)137 Chirp? x(t) = cos( 2 * pi * t * t / 1024 ) t f

138 CMU SCS Copyright: C. Faloutsos (2012)138 Chirp? x(t) = cos( 2 * pi * t * t / 1024 ) t f SWFT?

139 CMU SCS Copyright: C. Faloutsos (2012)139 Chirp? x(t) = cos( 2 * pi * t * t / 1024 ) t f SWFT

140 CMU SCS More examples (BGP updates) Copyright: C. Faloutsos (2012)140 BGP-lens: Patterns and Anomalies in Internet Routing Updates B. Aditya Prakash et al, SIGKDD 2009 time #updates

141 CMU SCS More examples (BGP updates) Copyright: C. Faloutsos (2012)141 freq. Low freq.: omitted t f

142 CMU SCS More examples (BGP updates) Copyright: C. Faloutsos (2012)142 freq. ??

143 CMU SCS More examples (BGP updates) Copyright: C. Faloutsos (2012)143 ‘Prolonged spike’ t f freq.

144 CMU SCS More examples (BGP updates) Copyright: C. Faloutsos (2012)144 freq. 15K msgs, for several hours: 6pm-4am

145 CMU SCS Copyright: C. Faloutsos (2012)145 Wavelets - Drill Or use ‘R’, ‘octave’ or ‘matlab’ – R: install.packages("wavelets") library("wavelets") X1<-c(1,2,3,4,5,6,7,8) dwt(X1, n.levels=3, filter=“d4”) mra(X1, n.levels=3, filter=“d4”)

146 CMU SCS Copyright: C. Faloutsos (2012)146 Wavelets - k-dimensions? easily defined for any dimensionality (like DFT, DCT)

147 CMU SCS Copyright: C. Faloutsos (2012)147 Wavelets - example Wavelets achieve *great* compression: ,000 # coefficients

148 CMU SCS Copyright: C. Faloutsos (2012)148 Wavelets - intuition Edges (horizontal; vertical; diagonal)

149 CMU SCS Copyright: C. Faloutsos (2012)149 Wavelets - intuition Edges (horizontal; vertical; diagonal) recurse

150 CMU SCS Copyright: C. Faloutsos (2012)150 Wavelets - intuition Edges (horizontal; vertical; diagonal) ml

151 CMU SCS Copyright: C. Faloutsos (2012)151 Advantages of Wavelets Better compression (better RMSE with same number of coefficients) closely related to the processing of the mammalian eye and ear Good for progressive transmission handle spikes well usually, fast to compute (O(n)!)

152 CMU SCS Copyright: C. Faloutsos (2012)152 Overall Conclusions DFT, DCT spot periodicities DWT : multi-resolution - matches processing of mammalian ear/eye better All three: powerful tools for compression, pattern detection in real signals All three: included in math packages (matlab, R, mathematica,... )

153 CMU SCS Copyright: C. Faloutsos (2012)153 Resources Numerical Recipes in C: great description, intuition and code for all three tools xwpl: open source wavelet package from Yale, with excellent GUI.

154 CMU SCS Copyright: C. Faloutsos (2012)154 Resources (cont’d) (defunct?)http://www.dsptutor.freeuk.com/j sanalyser/FFTSpectrumAnalyser.html : Nice java appletshttp://www.dsptutor.freeuk.com/j sanalyser/FFTSpectrumAnalyser.html : voice frequency analyzer (needs microphone – MSwindows only)http://www.relisoft.com/freeware/freq.html

155 CMU SCS Copyright: C. Faloutsos (2012)155 Resources (cont’d) www-dsp.rice.edu/software/EDU/mra.shtml (wavelets and other demos) R (‘install.packages(“wavelets”) )


Download ppt "CMU SCS 15-826: Multimedia Databases and Data Mining Lecture #23: DSP tools – Fourier and Wavelets C. Faloutsos."

Similar presentations


Ads by Google