1. FREE-VIEW WATERMARKING FOR FREE VIEW TELEVISION Alper Koz, Cevahir Çığla and A.Aydın Alatan

2. Outline Introduction Proposed Watermarking Method Robustness Results for the case of known external calibration. Analysis on the watermark transformations during Light Field Rendering Proposed solution for unknown external calibration Robustness results Summary and future works

3. Free View TV Rendered Video Cam. 0 Cam. 1 (Arbitrary view)

4. A New Problem: Free-view Video Watermarking Cam. 0 Cam. 1 METU EEE METU EEE METU EEE (?) Arbitrary View PROBLEM: How to embed the watermark such that the watermark can be extracted from a generated video for an arbitrary view?

5. Fundamental Case: One object static scenes Rendered Image Cam. 0 Cam. 1 Imagery Camera

6. Light Field Rendering (LFR) Light field: Static scene No occlusion Lambertian Surfaces

7. Light Field Rendering For each desired ray: - Compute intersection with uv and st plane. -Take closest ray Variants: -Bilinear in uv only -Bilinear in st only -Quadrilinear in (u,v,s,t)

8. Light Field Watermarking W W W W Direct Approach: Embed the same watermark into each frame of light field slab. uv plane (Camera plane) st plane (focal plane) I uv : Frame corresponds to the camera at (u,v). H uv : Output image after high pass filtering I uv.  : Watermark strength. W: Watermark sequence generated from a Gaussian distribution with zero mean unit variance.

9. Light Field Watermarking The watermark is inserted to the off-sheared images rather than embedding directly to camera frames. The same watermark is embedded to the pixels of different camera frames whose corresponding light rays intersect at the same point in the focal plane.

10. Watermark Detection Rendered watermark WW ren Rendered Image W I ren High pass filtering Normalized Correlation W ren Comparison to a threshold 1/0

11. Robustness Results 32x32 camera 256x256 pixels 24 Bits RGB Geometry of Buddha Light Field :

12. Robustness Results Camera position=[0.5 0 2]; Image plane normal: [0 0 1]; focal length=2 (default); Rendered image Rendered watermarked image

13. Robustness Results Camera position = [2.7 2.7 0]; Rotation : [0 0 0] Rendered image Rendered watermarked image

14. Transformation on Watermark Sequence in LFR  Bilinear Interpolation is utilized during LFR  Planar projective transformation between the original and rendered watermark.  Same transformation between imagery camera plane and focal plane.

15. Transformation on Watermark Sequence in LFR  How to find the projective planar transformation?  Find the points at which object surface and focal plane intersect.  Find the matches in the arbitrary view.  Determine the transformation between match pairs.

16. Transformation on Watermark Sequence in LFR  How to find the projective planar transformation?  Utilize two properties of these special points:  Light rays have same intensity  Observed at similar coordinates in LF images  Properties are related with Light Field parametrization.

17. Transformation on Watermark Sequence in LFR  How to find the projective planar transformation?  Find feature points between rendered and all LF images.  Determine images having higher number of correspondences (four images).  Select feature points at same coordinates with similar intensity values.  Fit a planar transformation model between match pairs.

18. Transformation on Watermark Sequence in LFR

19.  Apply the transformation to the original watermark.  Normalized correlation between the rendered watermark and the rendered image.  To handle small shidts in pixel locations :  utilize magnitude coefficients of the 2D Fourier transformation of the images

20. Robustness Results

21. Summary and future works A novel problem is introduced, Free-view watermarking. Analyses on watermark transformation are examined.  Known external calibrations.  Unknown external calibrations  Bilinear interpolation in u-v plane. Watermark is successfully detected from an aritrary view. Extension of the algorithm for multiple depths and static scenes