Download presentation

1
**EE645: Independent Component Analysis**

Elliot Taniguchi Advisor: Dr. Kuh May 16, 2003

2
**Presentation Overview**

ICA Motivation Mathematical Formulation Fast ICA Algorithm Applications Noise Separation and Feature Extraction Digital Watermarking

3
**Motivation for ICA Cocktail Party Problem**

Suppose you are in a crowded room with many people. How do you understand what any one person is saying? Separation of Independent Signals Similar to Blind Source Separation Little knowledge of the signals Access to mixed signals only

4
**Cocktail Party Problem**

??

5
**Cocktail Party Problem**

ICA Separation Algorithm Separation of Speech Signals Humans can separate multiple signals with only two ears/sensors ICA needs as many ears/sensors as message signals Here we assume he has four ears!

6
Recovered Messages

7
**Mathematical Formulation Overview**

ICA Definition ICA Assumptions Independent Signals Non-Gaussian ICA Limitations Scaling Permutations No. of Sensors

8
ICA Definition Mixed Signals in Matrix Notation Variable Definitions

9
ICA Solution Signal Separation Find using the ICA Algorithm

10
**ICA Block Diagram (2 Signals)**

Multiplexed Signal #1 Multiplexed Signal #2

11
**ICA Assumption #1: Independence**

Probability Density Definition Expected Value Definition

12
**ICA Assumption #2: Non-Gaussian**

Property of Gaussian signals Addition of two independent Gaussian random variables is another single Gaussian random variable. Information Lost! Kurtosis Function Special Case: kurt(N) = 0

13
**Limitation #1: Scaling ICA maximizes independence between signals.**

Scaling Factor for Signal #1 Assuming the Separated Signals are Independent Scaling Factor for Signal #2

14
**Limitation #2: Signal Permutations**

The mixing matrix and independent components are unknown.

15
**Limitation #3: No. of Sensors**

Sensor Requirement The number of separated signals cannot be larger than the number of inputs. Current research is being done to reduce this constraint.

16
**ICA Separation Technique**

Central Limit Theorem If two random (non-Gaussian) signals added, the resulting signal will be more Gaussian than the original two random signals ICA Separation Concept Central Limit Theorem (in Reverse) Maximizing Non-Gaussianity Results in separating the two signals

17
**Fast ICA Algorithm Overview**

“Fixed-Point” Algorithm Implementation Fast ICA algorithm Extensions Algorithm Speed & Performance Currently the fastest Most Commonly Used

18
**Fast ICA Algorithm Choose a random initial weight vector. Let,**

Repeat until converges.

19
**Fast ICA Extensions Preprocessing Activation Functions**

Normalize mean to zero Pre-Whitening Activation Functions g(u)=u^3 g(u)=u^2 g(u)=tanh(a1*u) g(u)=u*exp(-a2*u^2/2)

20
**Noise Separation Example**

Separation of Noise Impulsive Noise Additive White Gaussian Noise Implementation Two Sensor Setup Fast ICA Algorithm

21
Noise Generation AWGN Gaussian R.V. Impulsive Noise Poisson R.V.

22
Physical Setup Impulsive Noise Gaussian Noise Receiver #1 Receiver #2

23
**Noise Separation Example**

24
**Noise Separation Example**

25
**Noise Separation Example**

26
**Noise Separation Performance**

27
**Noise Separation Performance**

28
**Noise Separation & Feature Extraction**

ICA performs well in Blind Source Separation ICA for Feature Extraction Reduce Complexity of the Neural Network Train only on the appropriate signal Detection and Estimation of Hidden Signals

29
**Noise Detection and Estimation: Method #1**

Fast ICA N.N. (BP, RBF, SVM) Sensor #1 Feature #1 Sensor #2 Feature #2 Sensor #3 Feature #3 Sensor #4 Feature #4

30
**Noise Detection and Estimation: Method #2**

Fast ICA N.N. (BP, RBF, SVM) Sensor #1 Feature #1 Decision Device Sensor #2 Feature #2 Sensor #3 Feature #3 Sensor #4 Feature #4

31
**Noise Detection and Estimation: Conclusion**

Additional Preprocessing Segmentation of Impulsive Noise (Time-Limited) Possible Inputs to the Neural Network Statistical Moments Signal Samples Possible Neural Networks Back Propagation SVM Radial Basis Functions Problem Need to train Neural Network in Matlab

32
**Digital Watermarking of Music**

Motivation Popularity of Digital Storage Devices Reliable, Fast, Ease of duplication, etc. How to protect copyrighted information? Leaving digital signatures of its artist Essential Properties for Watermarking Undetectable Irremovable Resilient

33
**Detection & Estimation of Watermarks**

Detection of Watermark Authenticate copyrighted music Estimation of Watermark Information on artist, producer, etc.

34
**Watermarking Model Process**

Mix the original musical data with watermark Keep watermark Power relatively low Ensure high quality of the watermarked music Watermark is better hidden

35
**Popular Digital Formats**

Wave Format Mp3 Format Bit Rate 1411 [kbps] 128 [kbps] Channels 2 Audio Sample Rate 44 [kHz]

36
**Detection of Watermark**

Watermarking Detection Algorithm Use the ICA model to randomly mix the watermark and music file. Save the watermarked music in the popular *.wav format Read the saved *.wave file. Separate the watermark and the music file. Identify the watermark using statistical methods (mean, std, etc.) Performance Statistic Correlation Coefficient (Absolute Value)

37
**Detection Performance**

38
**Estimation of Watermark**

Watermarking Estimation Algorithm Use the ICA model to randomly mix the watermark and music file. Save the watermarked music in the popular *.wav format Read the saved *.wave file. Separate the watermark and the music file. Identify the watermark using statistical methods (mean, std, etc.) Digitize the watermark signal. Performance Statistic Bit Error Rate

39
**Estimation Performance**

40
**Is the Music Content Preserved?**

“Hero” – Mariah Carey Original Wave file from CD Hero.wav 29 [s] “Hero” – Mariah Carey Watermarked Wave file HeroWatermarked.wav 29 [s]

41
**Resilience of Proposed Watermark**

Resiliency Previous Simulations show that wav format is resilient to 8-bit and 16-bit quantization. Can the watermark be detected after Mp3 compression and decompression?

42
**Mp3 Compression/Decompression**

Actual Mp3 compression program used CDex Version 1.40 Release Mp3 (lossy) Compression Down Sampling Filter banks Mp3 Decompression Up Sampling Reconstruction Filter

43
**Detection and Estimation Performance**

Bad News Correlation Coefficients MSE Possible Problems with Mp3 Compression Down Sampling Watermark information is lost Quantization Noise Watermark information absorbs into the quantization noise Lossy Compression 11:1 Compression Rate

44
**How to Improve its Resilience?**

Alternative Approaches Synchronization of the music data Time shift in Mp3 compression? Storing watermark in certain frequencies (where less quantization occurs) Error Coding Hamming Reed-Solomon

45
**Conclusion Wave to Wave Format Mp3 Compression**

Very good performance (even for 8-bit wave files) SNR is very low. Music Integrity is excellent Mp3 Compression Very bad performance Alternative methods need to be found! Need a greater understanding of current Mp3 Compression Algorithms

46
References [1] Araki and others. Suband Based Blind Source Separation with Appropriate Processing for Each Frequency Band. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. [2] Hoyer and Hyvarinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. August 2000. [3] Hyvarinen, Aapo. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. [4] Hyvarinen, Aapo. Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. April 1999. [5] Hyvarinen and Oja. Independent Component Analysis: A Tutorial. April 1999. [6] Introduction to Blind Source Separation. [7] Liu and others. A Digital Watermarking Scheme based on ICA Detection. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. [8] Mitra, Sanjit K. Digital Signal Processing: Second Addition. McGraw Hill, 1998. [9] Shen and others. A Method for Digital Image Watermarking Using ICA. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.

Similar presentations

OK

ERP DATA ACQUISITION & PREPROCESSING EEG Acquisition: 256 scalp sites; vertex recording reference (Geodesic Sensor Net)..01 Hz to 100 Hz analogue filter;

ERP DATA ACQUISITION & PREPROCESSING EEG Acquisition: 256 scalp sites; vertex recording reference (Geodesic Sensor Net)..01 Hz to 100 Hz analogue filter;

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on e waste recycling Ppt on water scarcity in uganda Ppt on new zealand culture past Training ppt on email etiquettes Ppt on transportation in human beings Ppt on world book day character Ppt on 21st century skills teacher Ppt on renewable resources of energy Ppt on three phase ac circuits Ppt on instrument landing system