1 How Realistic is Photorealistic?

2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid

3 Introduction Among the set of all possible images, natural images only occupy a tiny subspace. For instance, there are totally 256^(n^2) d ifferent 8 -bit grayscale images of size nxn pixels. Natural images are sparsely distributed in the space of all possible images.

4 Image space e.g. when n = 10 pixels, it results in 1.3x10^154 different images !!!

5 Introduction (cont’) The regularities within natural images can be modeled statistically. Image statistical models are already in use by applications such as: Compression, de-noising, segmentation, texture synthesis, content-based retrieval and object/scene categorization.

6 Motivation 1 Identify Computer Graphics Sophisticated computer graphics software can generate highly convincing photorealistic images able to deceive the human eye. Differentiating these two types of images is an important task to ensure the authenticity and integrity of photographs.

7 Computer graphics example

8 Motivation 2 Identify Steg Images Image steganography hides messages in digital images in a non-intrusive way that is hard to detect visually. The task of generic steganalysis is to detect the presence of such hidden messages without the detailed knowledge of the embedding methods.

9 Steganography example Steg is the message image embedded into the original image. The rightmost image is the absolute value of the difference between the original and steg image, normalized into 8 bit for display purposes. Original message Steg |Original-Stego|

10 How ? Example

11 Motivation 3 Identify Re-broadcasting Biometrics-based (e.g., face, iris, or voice) authentication and identification systems are vulnerable to the “rebroadcast” attacks. (e.g. using a high-resolution photograph of a human face). We need to differentiate a “live” image (captured in real time by a camera) and a “rebroadcast” one (a photograph).

12 How to distinuish images ? Image properties ? –Image intensity histogram –Image frequency Other method ?

13 Using known methods

14 Why wavelets Image representations based on multi-scale image decomposition (e.g., wavelets) decompose an image with basis functions partially localized in both space and frequency - a compromise between these representations.

15 Quadrature Mirror Filter (QMF) Within the general framework of multi-scale image decomposition, there exist many different implementations, each having its own advantage and effective in different problems. In this work, two such decompositions, namely the quadrature mirror filter (QMF) pyramid decomposition and the local harmonic angular decomposition (LAHD), are employed for collecting image statistics characterizing natural images.

16 Wavelet (for example)

17 QMF - Quadrature Mirror Filter The QMF pyramid decomposition splits the image frequency space into three different scales, and within each scale, into three orientation subbands (Vertical, Horizontal and Diagonal).

18 QMF diagram

19 QMF The vertical, horizontal and diagonal subbands at scale i are denoted by Vi(x; y), Hi(x; y), and Di(x; y), respectively. Can be generated by convolving the image, I(x, y), with low-pass and high-pass filters.

20 Calculating the sub-bands The vertical subband is generated by convolving the image, I(x, y), with the low-pass filter in the vertical direction and the high-pass filter in the horizontal direction as: h(x)= High Pass Filter, l(x)= Low Pass Filter

21 Filters

22 The same for next levels (scales)

23 QMF decomposition – Example 1

24 QMF decomposition – Example 2

25 Example – QMF statistics

26 Example – Error Statistics

27 Is QMF enough ?

28 Add some magic … QMF coefficients Magic Box Error coefficients Simple but long (and out of scope) mathematical procedure

29 A Linear Predictor

30 Browsing Through

31 The same for all sub-bands

32 Technique Diagram Feature vector 

33 Computing the Feature Vector 3 – Sub-bands (vertical, horizontal, diagonal). 3 – Scales (levels of decompositions). 4 – First order statistics (mean, variance, skewness – asymmetry measure, kurtosis). 3 – Colors (RGB) 2 – marginal statistics (wavelet coefficients), error statistics. 216 = 3*3*4*3*2

34 Image examples

35 Feature vectors projected on 3D space Natural image – Blue. Synthetic images - noise (Green), fractal (Black), and discs (Red)

36 Learning and Testing CG\Steg\rebroadcast CG\steg\rebroadcast images are prepared. Statistics is collected over natural images and CG\steg\rebroadcast images (not using color). A Machine learning system (e.g. FLD, LDA, SVM) is then trained on some of the natural and some of the CG\steg\rebroadcast images. The remaining natural and CG\steg\rebroadcast images are used for testing.

37 Natural vs. CG results (SVM) All images TrainSucc [%] TestSucc [%] Natural CG

38 Training the system

39 Photorealistic (CG) images

40 Natural vs. CG results LDA - linear discrimination analysis SVM - Support vector machines

41 The Impact of Color

42 Correctly Classified Photorealistic

43 Incorrectly Classified Photographic

44 Natural vs. Steganography images A message consists of a 64x64 pixel region of a random image chosen from the same image database.

45 Natural vs. Steganography results All images TrainSucc [%] TestSucc [%] Natural Steg

46 Live vs. rebroadcast We collect statistics from natural images and the same images after having been printed on a laser printer and re-scanned with a scanner (printing and scanning are done at 72 dpi).

47 Live vs. rebroadcast results All images TrainSucc [%] TestSucc [%] Natural rebroad cast

48 Live vs. rebroadcast (cont’) Remark: It is not surprising that printing significantly disturbs the image statistics. Detecting a rebroadcast image will become more difficult with printers improvement.

49 Rebroadcasting example Shown is the original iris images (top row) and the images after being printed and scanned (bottom row).

50 Feature vectors projected on 3D space Results from a four-way classifier of 1000 natural, 1000 steg, 500 graphic, and 200 rebroadcast images.

51 More Applications how many different artists ?

52 More Applications Forgery detection.

53 What Next ? Can this technique be wrong - This model will not be immediately vulnerable to counter-attacks.

54 Finally Statistical model. capture regularities that are inherent to photographic images. Distinguish tampered \ CG images and natural images.