3 Motivation for ICA Cocktail Party Problem Suppose you are in a crowded room with many people. How do you understand what any one person is saying?Separation of Independent SignalsSimilar to Blind Source SeparationLittle knowledge of the signalsAccess to mixed signals only
5 Cocktail Party Problem ICA Separation AlgorithmSeparation of Speech SignalsHumans can separate multiple signals with only two ears/sensorsICA needs as many ears/sensors as message signalsHere we assume he has four ears!
11 ICA Assumption #1: Independence Probability Density DefinitionExpected Value Definition
12 ICA Assumption #2: Non-Gaussian Property of Gaussian signalsAddition of two independent Gaussian random variables is another single Gaussian random variable.Information Lost!Kurtosis FunctionSpecial Case: kurt(N) = 0
13 Limitation #1: Scaling ICA maximizes independence between signals. Scaling Factorfor Signal #1Assuming theSeparated Signalsare IndependentScaling Factorfor Signal #2
14 Limitation #2: Signal Permutations The mixing matrix and independent components are unknown.
15 Limitation #3: No. of Sensors Sensor RequirementThe number of separated signals cannot be larger than the number of inputs.Current research is being done to reduce this constraint.
16 ICA Separation Technique Central Limit TheoremIf two random (non-Gaussian) signals added, the resulting signal will be more Gaussian than the original two random signalsICA Separation ConceptCentral Limit Theorem (in Reverse)Maximizing Non-GaussianityResults in separating the two signals
17 Fast ICA Algorithm Overview “Fixed-Point” AlgorithmImplementationFast ICA algorithmExtensionsAlgorithm Speed & PerformanceCurrently the fastestMost Commonly Used
18 Fast ICA Algorithm Choose a random initial weight vector. Let, Repeat until converges.
19 Fast ICA Extensions Preprocessing Activation Functions Normalize mean to zeroPre-WhiteningActivation Functionsg(u)=u^3g(u)=u^2g(u)=tanh(a1*u)g(u)=u*exp(-a2*u^2/2)
20 Noise Separation Example Separation of NoiseImpulsive NoiseAdditive White Gaussian NoiseImplementationTwo Sensor SetupFast ICA Algorithm
28 Noise Separation & Feature Extraction ICA performs well in Blind Source SeparationICA for Feature ExtractionReduce Complexity of the Neural NetworkTrain only on the appropriate signalDetection and Estimation of Hidden Signals
31 Noise Detection and Estimation: Conclusion Additional PreprocessingSegmentation of Impulsive Noise (Time-Limited)Possible Inputs to the Neural NetworkStatistical MomentsSignal SamplesPossible Neural NetworksBack PropagationSVMRadial Basis FunctionsProblemNeed to train Neural Network in Matlab
32 Digital Watermarking of Music MotivationPopularity of Digital Storage DevicesReliable, Fast, Ease of duplication, etc.How to protect copyrighted information?Leaving digital signatures of its artistEssential Properties for WatermarkingUndetectableIrremovableResilient
33 Detection & Estimation of Watermarks Detection of WatermarkAuthenticate copyrighted musicEstimation of WatermarkInformation on artist, producer, etc.
34 Watermarking Model Process Mix the original musical data with watermarkKeep watermark Power relatively lowEnsure high quality of the watermarked musicWatermark is better hidden
35 Popular Digital Formats Wave FormatMp3 FormatBit Rate1411 [kbps]128 [kbps]Channels2Audio Sample Rate44 [kHz]
36 Detection of Watermark Watermarking Detection AlgorithmUse the ICA model to randomly mix the watermark and music file.Save the watermarked music in the popular *.wav formatRead the saved *.wave file. Separate the watermark and the music file.Identify the watermark using statistical methods (mean, std, etc.)Performance StatisticCorrelation Coefficient (Absolute Value)
38 Estimation of Watermark Watermarking Estimation AlgorithmUse the ICA model to randomly mix the watermark and music file.Save the watermarked music in the popular *.wav formatRead the saved *.wave file. Separate the watermark and the music file.Identify the watermark using statistical methods (mean, std, etc.)Digitize the watermark signal.Performance StatisticBit Error Rate
40 Is the Music Content Preserved? “Hero” – Mariah CareyOriginal Wave file from CDHero.wav29 [s]“Hero” – Mariah CareyWatermarked Wave fileHeroWatermarked.wav29 [s]
41 Resilience of Proposed Watermark ResiliencyPrevious Simulations show that wav format is resilient to 8-bit and 16-bit quantization.Can the watermark be detected after Mp3 compression and decompression?
42 Mp3 Compression/Decompression Actual Mp3 compression program usedCDex Version 1.40 ReleaseMp3 (lossy) CompressionDown SamplingFilter banksMp3 DecompressionUp SamplingReconstruction Filter
43 Detection and Estimation Performance Bad NewsCorrelation CoefficientsMSEPossible Problems with Mp3 CompressionDown SamplingWatermark information is lostQuantization NoiseWatermark information absorbs into the quantization noiseLossy Compression11:1 Compression Rate
44 How to Improve its Resilience? Alternative ApproachesSynchronization of the music dataTime shift in Mp3 compression?Storing watermark in certain frequencies (where less quantization occurs)Error CodingHammingReed-Solomon
45 Conclusion Wave to Wave Format Mp3 Compression Very good performance (even for 8-bit wave files)SNR is very low.Music Integrity is excellentMp3 CompressionVery bad performanceAlternative methods need to be found!Need a greater understanding of current Mp3 Compression Algorithms
46 References Araki and others. Suband Based Blind Source Separation with Appropriate Processing for Each Frequency Band. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. Hoyer and Hyvarinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. August 2000. Hyvarinen, Aapo. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. Hyvarinen, Aapo. Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. April 1999. Hyvarinen and Oja. Independent Component Analysis: A Tutorial. April 1999. Introduction to Blind Source Separation. Liu and others. A Digital Watermarking Scheme based on ICA Detection. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003. Mitra, Sanjit K. Digital Signal Processing: Second Addition. McGraw Hill, 1998. Shen and others. A Method for Digital Image Watermarking Using ICA. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.