1. PROPOSAL “EDGE DIGIT WATERMARK” A. Astapkovich, B. Krivosheev Saint-Petersburg State University of Aerospace Instrumentation State University of Saint-Petersburg 2011

2. Preliminary_1: digital fingerprinting formalizm Special notations will be used: In common case watermarking suppose using the encryptions and whole procedure is described with I x K x M → I” where I - container image K – encryption key M – watermark Attack is described with I” x A → I A ” container image after attack M A ” watermark image extracted from attacked image Number of pixel in image Im will be described as NP (Im)

3. Preliminary_2: robustness measurements Let us N (I 1,I 2 ) is some norm for difference between images I 1 and I 2 Robustness is described with the vector N ( I, I”) - quality of the watermarked image N (I, I A ”) - attack wildness N (M,M A ”) – quality of the extracted watermark image after the attack N (M,M A ”)/ N (I,I A ”) - relative robustness of the watermarking procedure;

4. Goal of the research THEORY Creating the knowledge base to support research activity in digital watermarking and digital fingerprinting fields; Developing and investigating the new concepts for robust watermarking and fingerprinting algorithm oriented for the digit video applications; PRACTICE The digital fingerprinting method for high quality video files has to be developed; The method has to be realized as the demo version of the soft tool for fingerprinting;

5. Basic requirements - practice Video file has format 1080p60 and is compressed with MPEG4; 1 min segment of video file can be used for the fingerprinting; Method has to withstand attacks: rotation, small enlargements with cropping, noising, small nonlinear distortions, down sampling up to 720p30, collusion attack with at least 100 copies ;

6. Basic requirements - theory Volume of the container (set of images) is the huge in comparizon with the marking information NP (M ) << NP( ∑ Im ) (C.1) Any watermark can be destroyed with severe attack and to provide the surviving of the fingerprint method has to meet the condition N (M,M A ”)/ N (I,I A ”) < 1 (C.2) Existence of the embedded watermark is more important than the quality of extracted image, but has to be good enough to be recognized N (M,M A ”) < prescribed level (C.3)

7. Edge watermark concept Any watermark can be destroyed by attack with strong enough wildness Attacked image has to have N (I, I A ”) < corrupted quality Watermarking method has to be build such way that N (M,M A ”)/ N (I,I A ”) < 1 It is reasonable to build the watermark to most fragile element of image, like edges Possibility to use edge region has to be investigated, also

8. Digital Fingerprinting Method (DFM) DFM includes: method to Generate the Fingerprint Set (GFS) method to generate the Set of Marking Positions (SMP) method to Embed the Marking Information at selected marking positions (EmMI) and method to Extract the Marking Information (ExMI) method to Interpretation Extracted Fingerprint (IEF)

9. Generating the Fingerprint Set (GFS) To provide robustness the fingerprinting information is converting to set of the images C0123 C0123… -> I0, I1, I2,I3,I4.. This images have to be embed to appropriate frames of video to survive against conclusion attack Images can be used directly or as wavelet decomposition components: LL,LH,HL,HH and so on

10. Generating the Set of Marking Positions (SMP) To provide robustness against collusion attack SMP has to provide possibility the Boneh-Shaw fingerprint scheme (code) For c+1 total users Boney-Shaw code uses O ( с 3 log (1/ε) ) bits to attain security against coalition of size c with error ε Boneh-Shaw code is used as building block for many sophisticated digital watermarking schemes Condition (C.1 ) provides possibility to build method with Boneh- Shaw code for C >> 100

11. Distortion against collusion Other possibilities have to be proposed and investigated As example : Collision attack based on difference of the images and creating the new version to eliminate the distributor ability to trace the object to any of them Little distortion of images with digital fingerprint destroy the simple collusion attack scheme Simple averaging will destroy the images and make collusion copy worthless

12. Example of the edge watermarking (I).bmp = 640x480 NP (EDGE) = 68698 NP(M) = NP( 100x100)= 10 4 бит (I x M ).bmp = 640x480 Marking Image was build to set of G and B components of edge pixels, generated with CANNY edge detector; Extracted M

13. Method to Embed the Marking Information (EmMI) Kutter algorithm for embedding provides defense against the noising attack Mi - bit of marking information I = {R,G,B} container p(x,y) – selected position for embedding ; Mi is embedded to B channel L(p)= 0.299 R(p) +0.587 G(p)+0.114 B(p) B(p) + q*L(p), if Mi =0 B(p)” = B(p) - q*L(p), if Mi =1 q – robustness parameter ( larger q leads to better robustness) In order to increase robustness every bit is embedded r times as the cross figure (7*7, с = 3), so total number is N = 3*r ;

14. Method to Extract the Marking Information (ExMI) ; Kutter algorithm for blind ExMI related with EmMI The bit value is determined by looking at the sign of the difference between the pixel under inspection and the estimated original Modification of ExMI + EmMI on base Kutter algorithm for edge pixel set has to be developed

15. Robustness of the wavelet decomposition As estimates the result of work Mohsen Ashourian, Peyman Moallem, Yo-Sung Ho “A Robust Method for Data Hiding in Color Images” can be used //PCM (2) 258-269, 2005; LL,HL,LH,HH Haar components of watermark (MASTER b/w image) is imbedded with modified Kutter algorithm ab cd (a+b+c+d)/4(a-b+c-d)/4 (a+b-c-d)/4(a-b-c+d)/4 LLLH HLHH

16. Eхамples of the wavelet decomposition approach LL LH HL HH Watermark MASTER image had the size ½ * ½ * 1/3 of the container image; Container images

17. Robustness against the compression attack PSNR for compression with JPEG The published results demonstrate the good robustness properties against compression with JPEG and JPEG2000 Extracted watermark image for Q=55 Robustness to MPEG4 has to be investigated ;

18. Robustness against down sampling attack Origin watermark Extracted watermark after ½ down sampling This is possible due to decomposition of the watermark image and the mixing of Haar components during embedding

19. Robustness against the filtering attack PSNR (dB) for extracted M MF GF Parrots 20.65 25.80 Boats 21.65 24.43 Median filtering (MF)Gauss filtering (GF)

20. Robustness against the cropping attack I A ” M A ” for 10% cutting Wavelet decomposition of the M provides high robustness for this type of attack ;

21. Open questions Robustness is described with the vector and some approaches have to be investigated: what norm has to be used ? is it the same norm has to be used for all components ? what level of norm meanings has to be defined from practical point of view ? For IEF stage norm has to be selected also ;

22. Brief norm review Classical approach is using the peak signal-to-noise ratio (PSNR) and MSE ; MAX I — maximal pixel meaning value (for 8-bit w/b image MAXI = 255); PSNR is no sign measure and useful for small distortions case; Original image Enhanced contrast PSNR=25 dB JPEG compression PSNR=25 dB

23. Attack wildness measurements  To measure the attack wildness the linear measure has to be used (at least) Salt and pepper noise attack Noise amplitude  PSNR can be modified (as example)

24. Modern structural norms Original image MSE=0 SSIM=1 CW-SSIM=1 MSE=306 SSIM=0.928 CW-SSIM=0.938 Enhanced contrastDistorted brightness MSE=309 SSIM=0.987 CW-SSIM=1 Gauss noise MSE=309 SSIM=0.576 CW-SSIM=0.814 Zhou Wang and Eero P. SimoncelliTranslation insensitive image similarity in complex wavelet domain / Proc. IEEE Inter. Conf. Acoustic, Speech & Signal Processing Volume II, Pages 573-576, March 2005  Some new ideas based on structural symmetry (SSIM) and complex wavelet structural symmetry (CW-SSIM) can be useful for the watermarking applications

25. Modern structural norms MSE=313 SSIM=0.73 CW-SSIM=0.811 Impulse noiseJPEG compression Enlargement MSE=309 SSIM=0.58 CW-SSIM=0.63 MSE=694 SSIM=0.505 CW-SSIM=0.925 MSE= 873 SSIM= 0.399 CW-SSIM=0.933 Rotation to left  Metrology benchmark image library and software tools have to be created  Structural similarity norm can be useful for estimation of the quality of the extracted watermark N (M,M A ”)

26. Noiseproof edge detector Edge watermarking withstand some attacks like cropping, rotation, displacement easily For blind watermarking this approach needs a noise proof edge detector Concept of the neuron like adaptive noise proof edge detector was proposed and investigated “The teaching by showing” methodology was used :  this approach is very flexible :  different samples can be used during initial learning  filter can increase experience due to additional learning with new samples This approach can be classified as “open algorithm approach”

27. Canny edge detector Clean image Noisy image edges Noisy image  Canny filter is the best edge detector, but ….. Clean image edges

28. Neuron adaptive linear filter Initial learning min II Sw – F II w Adding the new experience is not very expensive procedure Let us Se k = ∑ S k T S k - experience matrix for k samples Fek = ∑ Sk T F k – experience vector for k samples New filter weights W k+1 = (Se k + S k+1 T S k+1 +  E) –1 * (Fe k +S k+1 T F k+1 ) S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 image Filter 3*3 S1 S2 SNSEN 1 (S,W) TH min TH max W = (S T S +  E) –1 S T F

29. Neuron like adaptive noise proof edge detector TRmin = 40 TRmin = 30 Results of filtering for the clean and noisy test image with H51 and H52 filters with different low threshold (Trmin)  Methodology is universal and has no limits on size of edge filter  Example of 5*5 filter, learned with one and two sample noised images

30. Real image test  Two linear 5*5 filter were used to find the edges with different prescribed low thresholds  Artificial generated pictures with added noise were used to generate the learning samples  Different teacher samples were used Filter, learned with LAPLAS edge detector Filter, learned with hand pointed edge low threshold : 30 and 60

31. Overview of MPEG-4 MPEG-4 encodes natural video, synthetic video an still texture In addition MPEG-4 encodes a description of the scene for proper rendering of all objects At the decoding end the scene description and the individual media objects are decoded, synchronized and composed for presentation At rates below 1 Mbit/sec MPEG-4 demonstrates better performance than MPEG-1 and MPEG-2

32. Watermarking of Compressed Video Method for digital watermarking of MPEG-2 encoded video is presented. It consists of additions of an encrypted pseudo-noise signal to the video ( F.Hartung B. Girod “Watermarking of Uncompressed and Compressed Video” // Signal Processing Volume 66 Issue 3, May 1998, pp 283-301) The proposed scheme is applicable to MPEG-4, H.261,H.263. Compressed video is partly decoded to provide access to DCT

33. Hurtung-Girod method Just DCT encoded signal blocks are used For watermark pixel the 3D video sequence is regarded as one dimensional in T-domain Watermark signal is generated for each frame This signal is arranged into 2D signal, having the same dimensions as the video frames

34. Watermarking for low bit-rate MPEG-4 In work A. Alattar, M. Celik E.Lin Evaluation of Watermarking Low Bit-rate MPEG-4 Bit Stream the method for watermarking MPEG-4 was proposed. A watermark signal is inserted directly into MPEG-4 compressed bit-stream, while detection is performed using uncompressed video; A pair of templates is imposed on spread spectrum signal to combat synchronization loss due to rotation and scaling of the video after watermarking; To meet bit-rate constraints on watermarked video the special algorithm was proposed and investigated;

35. Experimental results Algorithm has benn tested with 5 sec of the standard sequences : Foreman, Flower Garden, Football and Salesmen, that were encoded with MPEG-4 at 128 Kb/s (QCIF 176*144),384 Kb/s and 768 Kb/s (CIF 352*288) at 15 fr/sec Average quality and increase in bit rate

36. Conclusion : result is the vector Method component The fingerprinting based on multiple description subband ( wavelet decomposition) image coding, that is embedded to edge domain with modified Kutter algorithm has to be investigated as the possible solution Some preliminary estimates demonstrate a good robustness against the various signal processing and geometrical attacks 2D embedding process (spatial + time) domains has to be investigated as the defense against the set of attacks (includes collusion) Theory component Vector approach for robustness measurements has to be developed and investigated Adaptive algorithm on base of neuron net approach for edge detector has to be developed and investigated Practice component  Metrology base has to be created The demo version of the digital fingerprinting tool has to be created

37. Conclusion : result is the vector Method component The fingerprinting based on multiple description subband ( wavelet decomposition) image coding, that is embedded to edge domain with modified Kutter algorithm has to be investigated as the possible solution Some preliminary estimates demonstrate a good robustness against the various signal processing and geometrical attacks 2D embedding process (spatial + time) domains has to be investigated as the defense against the set of attacks (includes collusion) Theory component Vector approach for robustness measurements has to be developed and investigated Adaptive algorithm on base of neuron net approach for edge detector has to be developed and investigated Practice component  Metrology base has to be created The demo version of the digital fingerprinting tool has to be created