1. Real Time Digital Watermarking System for Audio Signals Yuval Cassuto and Michael Lustig Supervisor: Shay Mizrachi Technion - Israel Institute of Technology Department of Electrical Engineering Signal and Image Processing Laboratory

2. World Pirate Music Business US $4.2 Billion World Recording industry US $37 Billion Motivation - Music Piracy Internet music is virtually a 100% pirate medium According to IFPI report

3. The Watermarking Concept W Watermarking system owner signature: #1345234 Signature Embedding

4. Signature Detection You win Detection system Adversary W owner signature: #1345234

5. Embedded Inaudible Public Algorithm Damaged Signature  Damaged Audio Resolve Deadlock  Keep Original Human Auditory System Signature Requirements

6. The watermarking problem 1Generation of a unique, robust and hidden signature 2Find an appropriate embedding method and location 3Embedding

7. Signature Embedding Algroithm FFT Watermark Coloring Frequency Masking (Psychoacoustic model) Pseudo Random Noise Generation Local Key Calculation Output Original Signal Segment Owner ’ s Key + W #1345234 Local Key Calculation Pseudo Random Noise Generation Watermark Coloring Frequency Masking (Psychoacoustic model) FFT + Original Signal Segment Owner ’ s Key Output 1 1 0 1 0 0 0 1 1 0 + + + 1 0 1 0 0 0 1 1 0 1 + + + 0 1 0 0 0 1 1 0 1 0 + W W

8. Frequency Masking Hearing threshold Threshold in Quiet Signal Spectrum Finding local maximum Find Local Maxima Threshold in Quiet Find Local Maxima Signal Spectrum Tonal Components Finding Tonal Components Threshold in Quiet Find Local Maxima Tonal Components Signal Spectrum Finding Non-Tonal Components Atonal Components Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Signal Spectrum Tonal Masking components Tonal Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Signal Spectrum Tonal Masking components Tonal Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Signal Spectrum Non-Tonal Masking components Atonal Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Signal Spectrum Atonal Masking Total Masking components Total Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Signal Spectrum Atonal Masking Resulting Masking threshold Total Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Signal Spectrum Atonal Masking Resulting Masking threshold Total Masking Threshold in Quiet Find Local Maxima Tonal Components Atonal Components Tonal Masking Atonal Masking Total Masking Signal Spectrum 050100150200250 0 10 20 30 40 50 60 70 80 90 100 Frequency index Decibels Owner ’ s Key FFT Coloring Pseudo Random Noise Generation Local Key Calculation + Frequency Masking (Psychoacoustic model) FFT Freq. masking

9. Masking Threshold 0 10 20 -20 0 20 40 60 80 100 Frequency [KHz] embedding Freq. maskingFFT keyRandom noise FFT Noise Filtering Frequency Masking (Psychoacoustic model) Pseudo Random Noise Generation Local Key Calculation + Watermark Coloring Watermark Coloring Masking threshold Masking Threshold White Watermark Masking Threshold Colored Watermark Masking Threshold White Watermark Interception Raised Watermark Masking Threshold White Watermark Colored Watermark White Watermark Colored Watermark

10. #1345234 W Signature Calculation Correlation & Threshold Gain Matching Signature Calculation Original Signal Tested Signal Owner's Key Decision W Signature Detection Algorithm

11. Gain Matching #1345234 W Original Signal Tested Signal Owner's Key Signature Calculation Decision Correlation & Threshold W W Signature Detection Algorithm

12. #1345234 W Original Signal Tested Signal Owner's Key Signature Calculation Decision Correlation & Threshold Gain Matching W W Signature Detection Algorithm

13. Why Real-time? Portable Devices Cellular phones Real Time Applications Audio Streaming W W Live Broadcast

14. Real-Time Implementation The Problem:  In the Windows application, watermark embedding time is x8 longer than playing time (@44.1 [KHz] ). The Solution:

15. MATLAB TM simulation PC application: Embedding & Detection TMS320C54x DSP embedding implementation. TIGER 5410/PC real-time embedding application. Development Phases

16. DSP Implementation Challenges Fixed Point Speed Memory Accuracy Architecture Utilization Parallel Execution Optimization Capacity I/O Synchronization

17. Algorithm Specific Implementation Challenges Adaptive Masking Threshold  Log-Scale Spectrum Calculation  Identifying Masking Components.  Calculating Masking Curves. Watermark Embedding  Creating The Watermark.  Coloring The Watermark.

18. 050100150200250 0 10 20 30 40 50 60 70 80 90 100 Frequency index dB Adaptive Masking Threshold 115 [frames/sec]! {Non Linear Frequency Metric} (Bark Units) Logarithmic Scale

19. Masking Threshold Implementation - masking curves Challenge: Masking curves are Not Linear Not Shift-Invariant Implementation Static Bark Addressing Constructing efficient Look-Up Tables Result: No Conditional Operations! 050100150200250 0 10 20 30 40 50 60 70 80 90 100 Frequency index dB Masking Curves Conditional Operations  CPU time

20. Masking Threshold Exponential Model Challenge: Components and Spectrum are in Log-Scale Threshold filter is in Linear-Scale. Wide range of values. Vast usage of Exp  Log Transforms (more than 200,000 per second).

21. Masking Threshold Exponent Implementation Calculate e x, x is within a wide range. First Approach:  Taylor approximation. Second Approach:  Look-Up table Hybrid Approach:  exp(a+b) = exp(a)exp(b)  a  Integer, small LUT  b  fraction [0,1) Taylor app. 2 17 words, too Big, too slow

22. Implementation Challenges Watermark Coloring Challenge: Watermark Coloring 0 -20 0 20 40 60 80 100 Threshold Frequency [KHz] Log scale 512 Tap Filter Implementation: Time Domain  Convolution O(n 2 ) Frequency Domain  Multiplication O(n) ?

23. Watermark Coloring In Frequency Domain Facts: No Zero-Padding is done. Frequency Domain  Cyclic Convolution Explanation: Watermark is a Pseudo Random Noise Watermark requirements still achieved –Ability to regenerate the same colored WM using the source and a public key. –WM spectrum matches Psycho-Acoustic model (Inaudibility not affected). Filtering in frequency domain is appropriate!

24. Real-Time optimization Speed  Psycho acoustic model tables dB  Ln.  PN26: Averaging 2 x indices, Divisions  shift.  Optimized SQRT,EXP and LOG assembly implementations. Fixed Point  Optimal representation to each block.  32 bit operations: normalized and saved using dynamic Q 16bit representations. Memory  Optimization of memory  only fast internal memory used.

25. DSP Application software System Schematics PC Host Application software DSP board ISA BUS Data via HPI Sync. By interrupts and status/control registers

26. Modes of operation inputoutput Live mode : Audio input – Audio output DAC interrupt ADC interrupt

27. Modes of operation inputoutput Live mode : Audio input – Audio output DAC interrupt ADC interrupt sync

28. Conclusion Speed - 10 times faster than the PC application Quality - Strong inaudible watermark (-22dB) Completeness - Full DSP and HOST apps. Portability - Low power, low cost DSP (C54) Capacity - Easily upgradeable to a Multi- channel system Innovation - Non-Standardized field Commercial Value - Huge potential market

29. The Signal and Image processing lab staff headed by Prof. Malah and Mr. Peleg - for the technical assistance TI - for the equipment, support, encouragement and invitations to ICASSP2000 and the “3rd European DSP Education and Research Conference”. Acknowledgements

30. For more information visit the lab’s web site: http:\\www-sipl.technion.ac.il