Seminar: Image Tampering MC919 - Prof. Anderson Rocha Arthur Espíndola Ribeiro Vinicius Dias de Oliveira Gardelli /11/2014

2014 Seminar Series - Digital Forensics (MO447/MC919) Outline [1/2] 1.Introduction 1. What is Image Tampering? 2. History 1.Tampering detection 1. Pixel-based techniques 2. Format-based techniques 3. Camera-based techniques 4. Physics-based techniques 5. Geometry-based techniques

2014 Seminar Series - Digital Forensics (MO447/MC919) Outline [2/2] 3.Selected Techniques 1. JPEG Ghosts Detection 2. Inconsistencies in Shadows 3.Conclusion References

2014 Seminar Series - Digital Forensics (MO447/MC919) Introduction

2014 Seminar Series - Digital Forensics (MO447/MC919) What is Image Tampering? From the dictionary: ➢ Tamper: Interfere with (something) in order to cause damage or make unauthorized alterations

2014 Seminar Series - Digital Forensics (MO447/MC919) What is Image Tampering? From Wikipédia: ➢ Image manipulation: It’s the application of image editing techniques to photographs in order to create an illusion or deception after the original photographing took place

2014 Seminar Series - Digital Forensics (MO447/MC919) History ●As old as photography itself

2014 Seminar Series - Digital Forensics (MO447/MC919) ~1860: Iconic Abraham Lincoln Photo was in fact a composition of his head and John Calhoun’s body. [Fourandsix/Hist]

2014 Seminar Series - Digital Forensics (MO447/MC919) History ●As old as photography itself ●Has been extensively used for political and artistic reasons

2014 Seminar Series - Digital Forensics (MO447/MC919) ~1930: Stalin had a commissar removed from the original photograph after the man fell out of favor with him. [Fourandsix/Hist]

2014 Seminar Series - Digital Forensics (MO447/MC919) 1970: Pullitzer prize winning photo had a pole removed from behind the screaming woman. [Fourandsix/Hist]

2014 Seminar Series - Digital Forensics (MO447/MC919) 1989: Oprah’s face was spliced onto actress Ann-Margaret’s body for the cover of a magazine. Neither women had agreed upon the montage beforehand. [Fourandsix/Hist]

2014 Seminar Series - Digital Forensics (MO447/MC919) History ●As old as photography itself ●Has been extensively used for political and artistic reasons ●More easily achieved over time

2014 Seminar Series - Digital Forensics (MO447/MC919) 2014: Recent forgery of a vote count report in an attempt to invalidate Brazilian presidential elections. [eFarsas/Dilma]

2014 Seminar Series - Digital Forensics (MO447/MC919) Tampering Detection

2014 Seminar Series - Digital Forensics (MO447/MC919) ●Techniques can be separated in roughly 5 categories [Farid 2009a]: o Pixel-based techniques o Format-based techniques o Camera-based techniques o Physics based techniques o Geometry-based techniques Tampering detection

2014 Seminar Series - Digital Forensics (MO447/MC919) Pixel-based techniques ●Pixels are the building blocks of images ●Image manipulation disrupts statistical properties of the pixels ●Directly or indirectly analyzes pixel-level correlations that arise from a specific form of tampering

2014 Seminar Series - Digital Forensics (MO447/MC919) Pixel-based techniques ●Cloning Cloned regions can be of any shape and location. ●Resampling Introduces specific periodic correlations between neighbouring pixels. ●Splicing Disrupts higher-order Fourier Statistics. ●Statistical Photographs contain specific statistical properties.

2014 Seminar Series - Digital Forensics (MO447/MC919) Format-based techniques ●Lossy compression introduces artifacts ●Rely on image compression specificities to detect forgery ●JPEG is the most common format

2014 Seminar Series - Digital Forensics (MO447/MC919) Format-based techniques ● Focus on three techniques o Double JPEG compression detection [Lukas & Fridrich 2003]

2014 Seminar Series - Digital Forensics (MO447/MC919) Histograms of four image quantizations. Double compression introduces periodic artifacts in the image histogram. [Farid 2009a]

2014 Seminar Series - Digital Forensics (MO447/MC919) Format-based techniques ● Focus on three techniques o Double JPEG compression detection [Lukas & Fridrich 2003] o JPEG Blocking artifacts [Luo et al. 2007]

2014 Seminar Series - Digital Forensics (MO447/MC919) Flower before and after heavy jpeg compression and resizing. [Wikipedia/CompArt]

2014 Seminar Series - Digital Forensics (MO447/MC919) Format-based techniques ● Focus on three techniques o Double JPEG compression detection [Lukas & Fridrich 2003] o JPEG Blocking artifacts [Luo et al. 2007] o JPEG Ghosts detection [Farid 2009b]

2014 Seminar Series - Digital Forensics (MO447/MC919) Camera-based techniques ●Cameras leave traces on generated images ●Chromatic aberration [Johnson & Farid 2006] o Variations in chromatic aberration patterns across an image may be used as evidence of image tampering

2014 Seminar Series - Digital Forensics (MO447/MC919) Color aberration generated on an image due to a camera lens’ color displacement. [Farid 2009a]

2014 Seminar Series - Digital Forensics (MO447/MC919) Camera-based techniques ●Cameras leave traces on generated images ●Chromatic aberration [Johnson & Farid 2006] ●Sensor noise o Distortions of sensor noise pattern

2014 Seminar Series - Digital Forensics (MO447/MC919) Camera-based techniques ●Cameras leave traces on generated images ●Chromatic aberration [Johnson & Farid 2006] ●Sensor noise ●Color-filter arrays o Color calculation from neighbour pixels introduces recognizable correlation patterns between pixels in an image

2014 Seminar Series - Digital Forensics (MO447/MC919) Color pattern created by a Bayer filter arrangement. [Wikipédia/Bayer]

2014 Seminar Series - Digital Forensics (MO447/MC919) Physics-based techniques ●2-D Lighting Considers only the two-dimensional (2-D) surface normals at the occluding object boundary. ●3-D Lighting Uses the model of the human eye, to determine the required 3-D surface normals. ●Light Enviroment Uses an aproximation of a Lambertian surface, simplified further to consider only the occluding boundary of an object.

2014 Seminar Series - Digital Forensics (MO447/MC919) Multiple lighting conditions for a single face. [Farid 2009a]

2014 Seminar Series - Digital Forensics (MO447/MC919) Geometric-based techniques ●Principal point estimation Principal point is the projection of the camera center onto the image plane. When a person or object is translated in the image, the principal point is moved proportionally. ●Metric measurements Tools from projective geometry that allow for the rectification of planar surfaces and, under certain conditions, the ability to make real-world measurements from a planar surface.

2014 Seminar Series - Digital Forensics (MO447/MC919) The result of planar rectification followed by histogram equalization. [Farid 2009a]

2014 Seminar Series - Digital Forensics (MO447/MC919) Selected Techniques

2014 Seminar Series - Digital Forensics (MO447/MC919) JPEG Ghosts Detection ●Explores double JPEG compression artifacts ●Detects lower-quality image patches spliced into higher quality images ●We need to understand JPEG compression

2014 Seminar Series - Digital Forensics (MO447/MC919) JPEG Compression Scheme ●Converts image from RGB to YCbCr color space ●Chroma channel subsampling o Usually 4:2:0 o Human eye less responsive to chromatic variations than to luminance variations ●Breaks Images in 8x8 pixel blocks

2014 Seminar Series - Digital Forensics (MO447/MC919) JPEG Compression Scheme ●Calculates 8x8 DCT coefficient matrices for each block: ●Quantizes the 8x8 DCT coefficient matrices using a quantizing matrix K for each channel:

2014 Seminar Series - Digital Forensics (MO447/MC919) DCT basis functions for a 8x8 image. [Wikipédia/JPEG]

2014 Seminar Series - Digital Forensics (MO447/MC919) JPEG Ghosts Detection ●Difference between initial compression and second compression is minimal when quality rate is the same ●Differences are calculated directly from pixel color values

2014 Seminar Series - Digital Forensics (MO447/MC919) Squared difference between coefficients originally quantized with factor q 0 =17 followed by quantization q 1 ∈ [1,30]. [Farid 2009b]

2014 Seminar Series - Digital Forensics (MO447/MC919) Squared difference between coefficients originally quantized with factor q 0 =23, followed by q 1 =17 and q 2 ∈ [1,30]. [Farid 2009b]

2014 Seminar Series - Digital Forensics (MO447/MC919) ●Problem: low-frequency regions (e.g: blue sky) may have lower difference values even for different quantization factors ●Solution: average differences over a square region and normalize values to fit in the interval [0,1] JPEG Ghosts Detection

2014 Seminar Series - Digital Forensics (MO447/MC919) ●Problem: low-frequency regions (e.g: blue sky) may have lower difference values even for different quantization factors ●Solution: average differences over a square region and normalize values to fit in the interval [0,1] JPEG Ghosts Detection

2014 Seminar Series - Digital Forensics (MO447/MC919) Top Left: Original image compressed at 85% quality with center re-saved at 65% quality. Rest: Differences between original image and re-saves at many qualities. [Farid 2009b]

2014 Seminar Series - Digital Forensics (MO447/MC919) ●JPEG Ghosts are usually visibly salient ●Still useful to quantify if a given region is different from the rest of the image o Two-factor Kolmogorov-Smirnov statistic JPEG Ghosts Detection

2014 Seminar Series - Digital Forensics (MO447/MC919) JPEG Ghosts Detection Results ●Threshold selected to yield less than 1% false positive rates ●KS statistics considered for each image difference Accuracy for different image sizes and quality variations. [Farid 2009b]

2014 Seminar Series - Digital Forensics (MO447/MC919) Inconsistencies in Shadows ●Technique for determining if cast and attached shadows in a photo are consistent with the model of a single distant or local point light source. ●Analyzing lighting and shadows are attractive. ●Relaxed constraints specify either angular wedges or half- planes in the image. ●Places no assumptions on the scene geometry.

2014 Seminar Series - Digital Forensics (MO447/MC919) Examples of cast and attached shadow constraints. [Farid 2013]

2014 Seminar Series - Digital Forensics (MO447/MC919) attached shadow: cast shadow: Cast and attached shadow constraints definitions. [Farid 2013]

2014 Seminar Series - Digital Forensics (MO447/MC919) Methods ●Combine the constraints into a single sistem of m inequalities: ●Account for errors or inconsistencies by introducing a set of m slack variables s i

2014 Seminar Series - Digital Forensics (MO447/MC919) Methods ●minimize the slack variables, while satisfying all of the cast and attached shadow constraints (linear programming). ●if the light is behind the center of projection: ●Greedily find an approximately minimal set of inconsistent constraints.

2014 Seminar Series - Digital Forensics (MO447/MC919) User Interface example. [Farid 2013]

2014 Seminar Series - Digital Forensics (MO447/MC919) Results

2014 Seminar Series - Digital Forensics (MO447/MC919) Results

2014 Seminar Series - Digital Forensics (MO447/MC919) Conclusion

2014 Seminar Series - Digital Forensics (MO447/MC919) Conclusion ●Many different techniques to detect image tampering ●Many ways to tamper with images

2014 Seminar Series - Digital Forensics (MO447/MC919) Questions?

2014 Seminar Series - Digital Forensics (MO447/MC919) ➢ [Fourandsix/Hist] “Photo Tampering throughout History”, available at access on 30/10/2014Photo Tampering throughout History ➢ [eFarsas/Dilma] “Fraude em urna eletrônica dá 400 votos para Dilma”, available at farsas.com/fraude-em-urna-eletronica-da-400-votos-para-dilma.html, access on 30/10/2014http:// farsas.com/fraude-em-urna-eletronica-da-400-votos-para-dilma.html ➢ [Wikipédia/JPEG] “JPEG”, available at access on 31/10/2014http://en.wikipedia.org/wiki/JPEG ➢ [Wikipédia/CompArt] “Compression artifact”, available at access on 31/10/ ➢ [Wikipédia/Bayer] “Bayer filter”, available at access on 31/10/2014http://en.wikipedia.org/wiki/Bayer_filter ➢ [Farid 2009a] Farid, H. “A Survey of Image Forgery Detection”. IEEE Signal Processing Magazine, 26(2):16-25, References

2014 Seminar Series - Digital Forensics (MO447/MC919) ➢ [Farid 2009b] Farid, H. “Exposing Digital Forgeries from JPEG Ghosts”, in Information Forensics and Security, IEEE Transactions on (Volume:4, Issue: 1 ), 2009Information Forensics and Security, IEEE Transactions on Issue: 1 ➢ [Farid 2013] Farid, H. “Exposing Photo Manipulation with Inconsistent Shadows”, ACM Transactions on Graphics, 2013 ➢ [Luo et al. 2007] W. Luo, Z. Qu, J. Huang, and G. Qiu, “A novel method for detecting cropped and recompressed image block”, in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, Honolulu, HI, 2007, pp. 217–220. ➢ [Lukas & Fridrich 2003] J. Lukas and J. Fridrich, “Estimation of primary quantization matrix in double compressed JPEG images”, in Proc. Digital Forensic Research Workshop, Cleveland, OH, Aug ➢ [He et al. 2006] J. He, Z. Lin, L. Wang, and X. Tang, “Detecting doctored JPEG images via DCT coefficient analysis”, in Proc. European Conf. Computer Vision, Graz, Austria, 2006, pp. 423– 435. References

2014 Seminar Series - Digital Forensics (MO447/MC919) ➢ [Johnson & Farid 2006] M. K. Johnson and H. Farid, “Exposing digital forgeries through chromatic aberration”, in Proc. ACM Multimedia and Security Workshop, Geneva, Switzerland, 2006, pp. 48–55 References