Presentation is loading. Please wait.

Presentation is loading. Please wait.

EE645: Independent Component Analysis Elliot Taniguchi Advisor: Dr. Kuh May 16, 2003.

Similar presentations


Presentation on theme: "EE645: Independent Component Analysis Elliot Taniguchi Advisor: Dr. Kuh May 16, 2003."— Presentation transcript:

1 EE645: Independent Component Analysis Elliot Taniguchi Advisor: Dr. Kuh May 16, 2003

2 Presentation Overview ICA Motivation Mathematical Formulation Fast ICA Algorithm Applications Noise Separation and Feature Extraction Digital Watermarking

3 Motivation for ICA Cocktail Party Problem Suppose you are in a crowded room with many people. How do you understand what any one person is saying? Separation of Independent Signals Similar to Blind Source Separation Little knowledge of the signals Access to mixed signals only

4 Cocktail Party Problem ??

5 Cocktail Party Problem ICA Separation Algorithm Separation of Speech Signals Humans can separate multiple signals with only two ears/sensors ICA needs as many ears/sensors as message signals Here we assume he has four ears!

6 Recovered Messages

7 Mathematical Formulation Overview ICA Definition ICA Assumptions Independent Signals Non-Gaussian ICA Limitations Scaling Permutations No. of Sensors

8 ICA Definition Mixed Signals in Matrix Notation Variable Definitions

9 ICA Solution Signal Separation Find using the ICA Algorithm

10 ICA Block Diagram (2 Signals) Signal #1Signal #2 Multiplexed Signal #2 Multiplexed Signal #1 a 11 a 12 a 21 a 22

11 ICA Assumption #1: Independence Probability Density Definition Expected Value Definition

12 ICA Assumption #2: Non-Gaussian Property of Gaussian signals Addition of two independent Gaussian random variables is another single Gaussian random variable. Information Lost! Kurtosis Function Special Case: kurt(N) = 0

13 Limitation #1: Scaling ICA maximizes independence between signals. Assuming the Separated Signals are Independent Scaling Factor for Signal #1 Scaling Factor for Signal #2

14 Limitation #2: Signal Permutations The mixing matrix and independent components are unknown.

15 Limitation #3: No. of Sensors Sensor Requirement The number of separated signals cannot be larger than the number of inputs. Current research is being done to reduce this constraint.

16 ICA Separation Technique Central Limit Theorem If two random (non-Gaussian) signals added, the resulting signal will be more Gaussian than the original two random signals ICA Separation Concept Central Limit Theorem (in Reverse) Maximizing Non-Gaussianity Results in separating the two signals

17 Fast ICA Algorithm Overview “Fixed-Point” Algorithm Implementation Fast ICA algorithm Extensions Algorithm Speed & Performance Currently the fastest Most Commonly Used

18 Fast ICA Algorithm 1. Choose a random initial weight vector. 2. Let, 3. Let, 4. Repeat until converges.

19 Fast ICA Extensions Preprocessing Normalize mean to zero Pre-Whitening Activation Functions g(u)=u^3 g(u)=u^2 g(u)=tanh(a1*u) g(u)=u*exp(-a2*u^2/2)

20 Noise Separation Example Separation of Noise Impulsive Noise Additive White Gaussian Noise Implementation Two Sensor Setup Fast ICA Algorithm

21 Noise Generation AWGN Gaussian R.V. Impulsive Noise Poisson R.V. Gaussian R.V.

22 Physical Setup Receiver #1 Receiver #2 Impulsive Noise Gaussian Noise

23 Noise Separation Example

24

25

26 Noise Separation Performance

27

28 Noise Separation & Feature Extraction ICA performs well in Blind Source Separation ICA for Feature Extraction Reduce Complexity of the Neural Network Train only on the appropriate signal Detection and Estimation of Hidden Signals

29 Noise Detection and Estimation: Method #1 Fast ICA Sensor #3 Sensor #4 Sensor #2 Sensor #1 Feature #3 Feature #4 Feature #2 Feature #1 N.N. (BP, RBF, SVM)

30 Noise Detection and Estimation: Method #2 Fast ICA Sensor #3 Sensor #4 Sensor #2 Sensor #1 Feature #3 Feature #4 Feature #2 Feature #1 N.N. (BP, RBF, SVM) Decision Device

31 Noise Detection and Estimation: Conclusion Additional Preprocessing Segmentation of Impulsive Noise (Time-Limited) Possible Inputs to the Neural Network Statistical Moments Signal Samples Possible Neural Networks Back Propagation SVM Radial Basis Functions Problem Need to train Neural Network in Matlab

32 Digital Watermarking of Music Motivation Popularity of Digital Storage Devices Reliable, Fast, Ease of duplication, etc. How to protect copyrighted information? Leaving digital signatures of its artist Essential Properties for Watermarking Undetectable Irremovable Resilient

33 Detection & Estimation of Watermarks Detection of Watermark Authenticate copyrighted music Estimation of Watermark Authenticate copyrighted music Information on artist, producer, etc.

34 Watermarking Model Process Mix the original musical data with watermark Keep watermark Power relatively low Ensure high quality of the watermarked music Watermark is better hidden

35 Popular Digital Formats Wave FormatMp3 Format Bit Rate1411 [kbps]128 [kbps] Channels22 Audio Sample Rate 44 [kHz]

36 Detection of Watermark Watermarking Detection Algorithm 1. Use the ICA model to randomly mix the watermark and music file. 2. Save the watermarked music in the popular *.wav format 3. Read the saved *.wave file. Separate the watermark and the music file. 4. Identify the watermark using statistical methods (mean, std, etc.) Performance Statistic Correlation Coefficient (Absolute Value)

37 Detection Performance

38 Estimation of Watermark Watermarking Estimation Algorithm 1. Use the ICA model to randomly mix the watermark and music file. 2. Save the watermarked music in the popular *.wav format 3. Read the saved *.wave file. Separate the watermark and the music file. 4. Identify the watermark using statistical methods (mean, std, etc.) 5. Digitize the watermark signal. Performance Statistic Bit Error Rate

39 Estimation Performance

40 Is the Music Content Preserved? “Hero” – Mariah Carey Original Wave file from CD Hero.wav 29 [s] “Hero” – Mariah Carey Watermarked Wave file HeroWatermarked.wav 29 [s]

41 Resilience of Proposed Watermark Resiliency Previous Simulations show that wav format is resilient to 8-bit and 16-bit quantization. Can the watermark be detected after Mp3 compression and decompression?

42 Mp3 Compression/Decompression Actual Mp3 compression program used CDex Version 1.40 Release Mp3 (lossy) Compression Down Sampling Filter banks Mp3 Decompression Up Sampling Reconstruction Filter

43 Detection and Estimation Performance Bad News Correlation Coefficients 0 MSE 0.5 Possible Problems with Mp3 Compression Down Sampling Watermark information is lost Quantization Noise Watermark information absorbs into the quantization noise Lossy Compression 11:1 Compression Rate

44 How to Improve its Resilience? Alternative Approaches Synchronization of the music data Time shift in Mp3 compression? Storing watermark in certain frequencies (where less quantization occurs) Error Coding Hamming Reed-Solomon

45 Conclusion Wave to Wave Format Very good performance (even for 8-bit wave files) SNR is very low. Music Integrity is excellent Mp3 Compression Very bad performance Alternative methods need to be found! Need a greater understanding of current Mp3 Compression Algorithms

46 References [1] Araki and others. Suband Based Blind Source Separation with Appropriate Processing for Each Frequency Band. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. [2] Hoyer and Hyvarinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. August 2000. [3] Hyvarinen, Aapo. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. http://www.cis.hut.fi/~aapo/.http://www.cis.hut.fi/~aapo/ [4] Hyvarinen, Aapo. Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. http://www.cis.hut.fi/~aapo/. April 1999.http://www.cis.hut.fi/~aapo/ [5] Hyvarinen and Oja. Independent Component Analysis: A Tutorial. http://www.cis.hut.fi/projects/ica/. April 1999. http://www.cis.hut.fi/projects/ica/ [6] Introduction to Blind Source Separation. http://www.cnl.salk.edu/~tewon/Blind/.http://www.cnl.salk.edu/~tewon/Blind/ [7] Liu and others. A Digital Watermarking Scheme based on ICA Detection. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. [8] Mitra, Sanjit K. Digital Signal Processing: Second Addition. McGraw Hill, 1998. [9] Shen and others. A Method for Digital Image Watermarking Using ICA. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.


Download ppt "EE645: Independent Component Analysis Elliot Taniguchi Advisor: Dr. Kuh May 16, 2003."

Similar presentations


Ads by Google