Background Estimation Mehdi Ghayoumi, MD Iftakharul Islam, Muslem Al-Saidi Department of Computer Science Kent State University, Kent, OH

Objective Fill in the area of an image based on existing background User selects an area, which is then filled based on surrounding pixels Smooth transitions

Introduction Object Removal – Remove object(s) from image – Fill the hole with information extracted from the surrounding area. Filled region should look “realistic” to the human eyes

Example Source Image Target Final Image

Greedy Approach A Greedy Patch-based Image Inpainting Framework

Diffusion-based Approach The idea is to track perfectly the local geometry of the damaged image and allowing diffusion only in the isophotes curves direction.

Exemplar Based Approach Idea 1. Sample color values of the surrounding area 2. Generate textures with sampling result to fill the hole

Criminisi’s Algorithm Assign each pixel with a priority value Give linear structures higher priorities

Criminisi’s Algorithm P(p) = C(p)D(p) Confidence term Data term 1. Compute the filling priority

Criminisi’s Algorithm (a) The confidence term assigns high filling priority to out-pointing appendices (in green) and low priority to in-pointing ones (in red), thus trying to achieve a smooth and roughly circular target boundary. (b) The data term gives high priority to pixels on the continuation of image structures (in green) and has the effect of favoring in-pointing appendices in the direction of incoming structures. Effects of data and confidence terms

Criminisi’s Algorithm 2. Search for the best matching patch

Criminisi’s Algorithm In this step, the algorithm fills the region corresponding to Ψp∩Ω by replicating the corresponding region in the best matching patch Ψ ^q to the target patch Ψp. Besides, the boundary of the target region δΩ has to be renewed. 3. Copy the best matching patch information and refresh the boundary of target region

Criminisi’s Algorithm(cont.) Structure Propagation by exemplar-based texture synthesis

Criminisi’s Algorithm(cont.)

Improved Criminisi’s Algorithm(cont.)

Expected Results Input Output

Future Work Implementing Algorithms in JAVA Make and install its Plugin in Imagej

Future Work More accurate propagation of curve structures Solve the problems

References A. Criminisi, P. Perez, K. Toyama. Region filling and object removal by exemplar-based Inpainting, IEEE Transactions on Image Processing,2004. Christine Guillemot and Olivier Le Meur,Image Inpainting, Signal Processing Magazin,IEEE,2014. Jing Wang and et all, Robust object removal with an exemplar-based image inpainting approach,Neurocomputing, IEEE,2014.

Thanks!