1. Forensic Detection of Image Manipulation Using Statistical Intrinsic Fingerprints Fernando Barros Filipe Berti Gabriel Lopes Marcos Kobuchi

2. Lab Seminar Series MO447 - Digital Forensics Prof. Dr. Anderson Rocha anderson.rocha@ic.unicamp.br http://www.ic.unicamp.br/~ro cha cha

3. Outline

4. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Outline ‣ Introduction ‣ System Model and Assumptions ‣ Statistical Intrinsic Fingerprints of Pixel Value Mappings ‣ Detecting Contrast Enhancement ‣ Detecting Additive Noise in Previously JPEG- Compressed Images ‣ Conclusion

5. Introduction

6. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Nowadays... ‣ In recent years, digital images have become increasingly prevalent through society.

7. * 2013 Seminar Series – Digital Forensics (MO447/MC919) © Daily Stormers (www.dailystomers.com) © Raíz da Vida (www.raizdavida.com.br) © http://topwalls.net/grand-canyon-united-states-2/ Real Pictures © www.wallpea.com

8. * 2013 Seminar Series – Digital Forensics (MO447/MC919) © Fan Pop (www.fanpop.com) Digital Images © johnnyslowhand.deviantart.com © www.highqualitywallpapers.eu

9. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Fake Pictures! © www.epicfail.com © www.hoax-slayer.com ©igossip.com © fashmark.files.wordpress.com

10. * 2013 Seminar Series – Digital Forensics (MO447/MC919) © www.vidrado.com Fake Pictures? © Veja (http://veja.abril.com.br/)© Telegraph (www.telegraph.co.uk)

11. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Fake Pictures? © http://www.buzzfeed.com/tomphillips/22-viral-pictures-that-were-actually-fake

12. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Fake Pictures? © http://10steps.sg/inspirations/photography/70-strange-photos-that-are-not-photoshopped/

13. * 2013 Seminar Series – Digital Forensics (MO447/MC919)

14. * Consequence ‣ At present, an image forger can easily alter a digital image in a visually realistic manner. ‣ As a result, the field of digital image forensics has been born.

15. * 2013 Seminar Series – Digital Forensics (MO447/MC919) State Of The Art ‣ Identification of images and image regions which have undergone some form of manipulation or alteration ‣ No universal method of detecting image forgeries exists ‣ Different techniques, with their own limitations

16. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Some techniques ‣ Lighting Angle Inconsistencies ‣ Inconsistencies in chromatic aberration ‣ Absence of Color Filter Array (CFA) interpolation induced correlations ‣ Classifier based approaches

17. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Fingerprints ‣ Most image altering operation leave behind distinct, traceable “fingerprints” in the form of image alteration artifacts ‣ Because these fingerprints are often unique to each operation, an individual test to catch each type of image manipulation must be designed

18. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Some works with fingerprints ‣ Resampling ‣ Double JPEG compression ‣ Gamma correction

19. * 2013 Seminar Series – Digital Forensics (MO447/MC919) This work ‣ Pixel value mapping leaves behind statistical artifacts which are visible in an image’s pixel value histogram ‣ By observing the common properties of the histogram of unaltered images, it’s possible to build a model of an unaltered image’s histogram

20. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ A number of operations are in essence pixel value mapping, it’s proposed a set of image forgery detection techniques which operate by detecting the intrinsic fingerprint of each operation This work

21. System Model and Assumptions

22. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Digital image ‣ In this work, digital images created by using an electronic imaging device to capture a real world scene ‣ Each pixel is assigned a value by measuring the light intensity reflected from a real world scene ‣ Inherent in this process is the addition of some zero mean sensor noise which arises due to several phenomena (shot noise, dark current, etc)

23. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ For color images, it is often the case that the light passes through a CFA so that only one color component is measured at each pixel location in this fashion ‣ In that case, the color component not observed at each pixel are determined through interpolation ‣ At the end of this process the pixel values are quantized, then stored as the unaltered image Color image

24. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ h(l) can be generated by creating L equally spaced bins which span the range of possible pixel values ‣ Tabulate the number of pixels whose value falls within the range of each bin ‣ Gray levels values in P = {0, …, 255}, Color values in P³ ‣ Pixel value histogram uses 256 bins Histogram

25. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histograms ‣ None of the histograms contains sudden zeros or impulsive peaks ‣ Do not differ greatly from the histogram’s envelope ‣ To unify these properties, pixel value are described as interpolatably connected

26. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Interpolatably connected ‣ Any histogram value h(l) can be aproximated by ĥ(l) ‣ Each value of ĥ has been calculated by removing a particular value from h then interpolating this value using a cubic spline ‣ Little difference from h and ĥ

27. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Fig. 1. Left: Histogram of a typical image. Right: Approximation of the histogram at left by sequentially removing then interpolating the value of each histogram entry.

28. * 2013 Seminar Series – Digital Forensics (MO447/MC919) System Model ‣ To justify this model, a database of 341 unaltered images captured using a variety of digital cameras ‣ Obtained each image’s pixel value histogram h and its approximated histogram ĥ ‣ The mean squared error between both along with the signal power of h to obtain an SNR~30.67dB

29. Statistical Intrinsic Fingerprints of Pixel Value Mappings

30. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Pixel Value Mapping ‣ A number of image processing operations can be specified entirely by a pixel value mapping ‣ Leave behind distinct, forensically significant artifacts, which we will refer as intrinsic fingerprint

31. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Intrinsic Fingerprint ‣ Intrinsic Fingerprint ‣ Original: x; Tampered: y

32. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Discrete Fourier Transform

33. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Example: Histogram Synthesized ImageReal World Image

34. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Example: DFT Synthesized ImageReal World Image

35. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Example: Frequency Domain Synthesized ImageReal World Image

36. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Example: Frequency Domain Synthesized ImageReal World Image

37. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ When examining a potentially altered image, if the histogram of unaltered pixel values is known, the tampering fingerprint can be obtained using Intrinsic Fingerprints

38. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ In most real scenarios, one has no a priori knowledge of an image’s pixel value histogram, thus the tampering fingerprint cannot be calculated But...

39. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ It’s possible to ascertain the presence of a tampering fingerprint by determining identifying features of a mapping’s intrinsic fingerprint ‣ Searching for their presence in the histogram of the image However

40. Detecting Contrast Enhancement

41. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Overview ‣ Contrast Enhancement operations seek to increase the dynamic range of pixel values within images.

42. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Detection of Globally Applied Contrast Enhancement ‣ Usually nonlinear mappings ‣ Consider only monotonic pixel value mappings. ‣ Thereby, disconsidering simple reordering mappings.

43. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Two significant mappings

44. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Euclidian norm increases! ‣ The energy of the DFT as well

45. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Therefore, all contrast enhancement mappings result in an increase in energy. ‣ This energy is related to the intrinsic fingerprint.

46. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Expected DFT’s to be strongly low-pass signal. ‣ Therefore, the presence of energy in the high frequency regions is indicative of contrast enhancement. ‣ Contrast enhancement will cause isolated peaks and gaps in the histogram.

47. * 2013 Seminar Series – Digital Forensics (MO447/MC919)

48. * Globally Applied Contrast ‣ Saturation Case

49. * 2013 Seminar Series – Digital Forensics (MO447/MC919) http://www.funnyjunk.com/funny_pictures/3306291/Houston+we+have+a+problem/

50. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Solution

51. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Measuring the energy

52. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Defining the best c with 244 images e N p = 4 ‣ γ = 1.1

53. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Results

54. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ Database of 341 unaltered images, taken in different resolutions and light conditions ‣ The green color layer created the grayscale images. ‣ γ ranging from 0.5 to 2.0 ‣ 4092 grayscale images

55. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast

56. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Globally Applied Contrast ‣ N p = 4 e c = 112 ‣ P d of 99% at a P fa approximately of 3% or less

57. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ Defined as applying a contrast mapping to a set of contiguous pixels within an image. ‣ Can identify cut-and-paste forgeries.

58. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ To detect it, the image is divided in smaller blocks and the global technique is applied to the blocks. ‣ Who small(and big!?) are these blocks?

59. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ Test the 341 unaltered images and use γ ranging from 0.5 to 0.9. ‣ Blocks of size 200x200, 100x100, 50x50, 25x25, and 20x20

60. * 2013 Seminar Series – Digital Forensics (MO447/MC919)

61. * Locally Applied Contrast ‣ The contrast enhancement can be reliably detected using testing blocks sized 100x100 pixels with a P d of at least 80% in every case at a P fa of 5%. ‣ When γ ranged from 1.0 to 2.0, the P d was of 95% at a P fa of 5%

62. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ In order to test the copy-and-paste, Photoshop was used to create the image (c).

63. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ The image was divided in 100x100 pixel blocks and tested local contrast enhancement on the red(d), green(e) and blue(f) color layers.

64. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ The image was divided in 50x50 pixel blocks and tested local contrast enhancement on the 3 the color layers.

65. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Locally Applied Contrast ‣ Applying a detection criteria.

66. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ Histogram equalization effectively increases the dynamic range of an image’s pixel values by subjecting them to a mapping such that the distribution of output pixel values is approximately uniform.

67. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ In order to identify it, we calculate the “uniformity” of the histogram. ‣ The process will introduce zeros into an image’s pixel value histogram, so mean absolute differences and mean square differences won’t work.

68. * 2013 Seminar Series – Digital Forensics (MO447/MC919) [2] http://www.funnyjunk.com/funny_pictures/3306291/Houston+we+have+a+problem/

69. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ For histogram equalized saturated images, the location of the impulsive component is often shifted. ‣ Suppose that the number of pixels in the lowest bin is greater than 2N/255.

70. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ If the lowest l such that h(l) > 0 is greater or equal to 1 and h(l) > 2N/255, the image is identified as saturated.

71. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ Test the 341 unaltered images and the 341 histogram equalized images.

72. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization

73. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ Analyze the frequency domain. ‣ α (k) is a weighting function used to deemphasize the high frequency regions in H(k)

74. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ Best conditions using α 1 (k) was with r 1 = 0.5, obtaining P d of 99% with a P fa of 0.5% and a P d of 100% with a P fa of 3%.

75. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Histogram Equalization ‣ Test 2046 images. ‣ r 2 = 4 ‣ P d of 100% with a P fa of 1%.

76. Detecting Additive Noise in Previously JPEG- Compressed Images

77. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise ‣ Additive noise can be used to mask previous modifications to images. ‣ Previous techniques has dealt with detection of localized fluctuations of SNR in an image. ‣ Fail on detection of globally added noises.

78. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise ‣ This technique applies a predefined mapping with a known fingerprint to a potentially altered image. ‣ If some noise was intentionally added, then an identifying feature of this fingerprint will be absent. ‣ We’ll be able to detect the presence of an additive noise if application of mapping does not introduce a fingerprint with this feature.

79. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Scale and Round mapping ‣ For additive noise detection it’ll be used scale and round mapping: ‣ And the set

80. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Scale and Round mapping ‣ Cardinality of U C (v) is periodic in v with period p. ‣ So, the intrinsic fingerprint of scale and round operation will contain a periodic component with period p.

81. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ JPEG compression/decompression schematics © Compressed Examples by JISC Digital Media. All files © University of Bristol, 2009 Compressed ExamplesJISC Digital Media

82. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ So, if a monotonically increasing mapping is applied to any color layer in the YC b C r color space, that mapping’s fingerprint will be introduced into the histogram of the color layer value.

83. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ Final stage of JPEG decompression: pixel transformation from YC b C r to RGB, mathematically described by this equation: ‣ Values less than 0 is set to 0 and greater than 255 is set to 255.

84. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ Defining: ‣ Detection of additive noise can be formulated as an statistical hypothesis testing problem:

85. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ The fingerprint left by the mapping: ‣ helps to rewrite both hypothesis as:

86. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ Under hypothesis H 0, z i can be expressed as: ‣ The term round(cx i ) dominates the behavior of z i and, so, the number of distinct x i values mapped to each z i value will occur in a fixed periodic pattern.

87. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ This will result in a periodic pattern discernible in the histogram, which corresponds to the intrinsic scale and round mapping.

88. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ Under hypothesis H 1, z i has a different behavior:

89. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario ‣ This hypothesis leads to 3 additional terms containing scale and round mapping, each with their own scaling constant. ‣ If this constants and the original scaling has no common period, no periodic pattern will be introduced into the histogram, as can be observed in the figures.

90. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Hypothesis Testing Scenario

91. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise Detection Images ‣ Detection of the addition of noise to a previously JPEG-compressed images is the same as detection of the periodic fingerprint within the normalized histogram. ‣ This detection is well suited for frequency domain and produces peaks with arbitrary location.

92. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise Detection Images ‣ Applying DFT in the normalized and pinched-off histogram we obtain G zi, it is possible to measure the strength of the peak introduced into it:

93. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise Detection Images ‣ Then it is used the following decision rule to determine presence of additive noise:

94. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ 227 unaltered images from 4 different digital cameras from unique manufacturers. ‣ Diversity of JPEG-compressed images using camera’s settings. ‣ Set of altered images created by decompression and addition of unit variance Gaussian noise to each pixel value. Additive Noise Detection Images - 1st performance test

95. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ All altered saved with original images resulting in a DB of 554 images. ‣ P d = 80% @ P fa = 0,4% (if P fa <= 6,5% P d goes to nearly 99%). Additive Noise Detection Images - 1st performance test

96. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise Detection Images - 1st performance test

97. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ 244 JPEG-compressed images at different quality. ratios. ‣ Q=90, 70, 50 and 30. ‣ Again, unit variance Gaussian noise added. Additive Noise Detection Images – 2nd performance test

98. * 2013 Seminar Series – Digital Forensics (MO447/MC919) ‣ For images with Q >= 50. ‣ P d = 99% @ P fa = 3,7%. Additive Noise Detection Images – 2nd performance test

99. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Additive Noise Detection Images – 2nd performance test

100. Conclusions

101. * 2013 Seminar Series – Digital Forensics (MO447/MC919) Conclusions ‣ Statistical Intrinsic Fingerprints of Pixel Value Mappings. ‣ Detecting Contrast Enhancement ‣ Detecting Additive Noise in Previously JPEG- Compressed Images

102. References

103. * 2013 Seminar Series – Digital Forensics (MO447/MC919) References 1. A. Swaminathan, M.Wu, and K. J. R. Liu, “Digital image forensics via intrinsic fingerprints,” IEEE Trans. Inf. Forensics Security, vol. 3, no. 1, pp. 101–117, Mar. 2008. 1. http://www.jiscdigitalmedia.ac.uk/guide/file-formats-and-compression/ http://www.jiscdigitalmedia.ac.uk/guide/file-formats-and-compression/

104. Thank You! Obrigado!