Summary Marie Yarbrough

Introduction History of Image Forgery Method Segmentation Classification Common-Sense Reasoning Conclusion

Images have been a powerful media of delivering and communicating information ever since their inception. The act of distorting and changing images has been around for the same amount of time. Detecting these image manipulations images has become and important problem. Even more so now that we have entered the digital age. This paper produces a way of detecting these falsified images.

People have been doing image manipulation since the beginning and these forgeries have been put to many uses. Such as Journalists who want to make up their own stories Photojournalists who want dramatic scenes Scientists who forge or repeat images in academic papers Politian's who try to direct public opinion by exaggerating or falsifying political events

There are 4 manipulation techniques that are used on images. Deletion of details: removing scene elements Insertion of detail: adding scene elements Photomontage: combining multiple images False Captioning: misrepresenting image content.

A series of photos of the “Devil’s Den” sniper photographed after the battle of Gettysburg. The first three photos show the soldier where he fell in battle. The fourth shows him “Posed” for dramatic effect.

The author proposes using Artificial Intelligence (AI) techniques of common-sense reasoning to detect duplicated and anomalous elements in a set of images. The basic premise: If they can detect a set of key elements in an image then they can detect if they have been moved, added, or deleted from a scene prior to image created.

The most important step of this process it to split the images into Regions of Importance (ROI). They use mean-shift image segmentation to decompose an image into homogeneous regions. Routine has inputs: Spatial radius: h s Color radius: h r Minimum number of pixels: M They then over segment the image using low values of h r and M and merge adjacent regions.

(Top)Results of mean-shift segmentation with parameters h s = 7, h r = 6, and M=50. (Bottom) Results of region merging.

After they segment the image the next step is to classify the ROI. They propose a segment based classification scheme. Brute force pixel comparisons take too long to do. Segment-wise classification reduces the size of the problem space significantly. Comparing the relationship of ROI across a corpus of images gives the ability to determine if a scene has been manipulated during the photo recording process.

They set about this classification by computing an importance map that assigns a scalar value to each pixel estimating the importance of that image location based on an attention model. They use measures of visual salience, image regions likely to be interesting to the low-level vision system, and high-level detectors for specific objects that are likely to be important.

(Top) Importance map. Saliency regions are outlined in magenta, face regions in cyan. (Bottom) Regions of importance.

They use two different approaches to assist digital forensics. To resolve local classification ambiguities within images, they query a knowledge base to resolve the proper relation A common-sense knowledge base such as Cyc and OpenMind is well suited for this task. For example, given two large horizontal blue regions many classifiers cannot distinguish which is ‘sky’ and which is ‘water’. A common-sense knowledge base can be queried to find the answer.

Then, they reason across a larger corpus of images to find unique or missing elements during an investigation. In many cases, the single image might not tell the complete story. A collection of photos, however, does show a narrative of a larger story. For example, a man-made object such as a plane in a field of grass should raise suspicion. Unless, all the photos in a corpus have a similar qualitative structure.

They suggest a software system based on a combination of existing tools to identify common objects across a corpus of images. By visualizing such objects, as in this figure, even a layperson can quickly determine whether a given image isfalsely captioned.

Through a series of segmentation, classification, and common sense reasoning they can find parts of images that might have been manipulated. These methods are limited by the performance of the components they use though.