A Closed Form Solution to Natural Image Matting

Slides:

Advertisements

Similar presentations

Bayesian Belief Propagation

Advertisements

L1 sparse reconstruction of sharp point set surfaces

SOFT SCISSORS: AN INTERACTIVE TOOL FOR REALTIME HIGH QUALITY MATTING International Conference on Computer Graphics and Interactive Techniques ACM SIGGRAPH.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Learning with Inference for Discrete Graphical Models Nikos Komodakis Pawan Kumar Nikos Paragios Ramin Zabih (presenter)

Spectral Matting Anat Levin 1,2 Alex Rav-Acha 1 Dani Lischinski 1 1 School of CS&Eng The Hebrew University 2 CSAIL MIT.

I Images as graphs Fully-connected graph – node for every pixel – link between every pair of pixels, p,q – similarity w ij for each link j w ij c Source:

1School of CS&Eng The Hebrew University

Image Matting with the Matting Laplacian

GrabCut Interactive Image (and Stereo) Segmentation Carsten Rother Vladimir Kolmogorov Andrew Blake Antonio Criminisi Geoffrey Cross [based on Siggraph.

A Gimp Plugin that uses “GrabCut” to perform image segmentation

Natural Video Matting using Camera Arrays Neel S. JoshiWojciech MatusikShai Avidan Mitsubishi Electric Research Laboratory University of California, San.

Graph-Based Image Segmentation

Jue Wang Michael F. Cohen IEEE CVPR Outline 1. Introduction 2. Failure Modes For Previous Approaches 3. Robust Matting 3.1 Optimized Color Sampling.

1 Roey Izkovsky Yuval Kaminka Matting Helping Superman fly since 1978.

Grayscale Image Matting And Colorization Tongbo Chen, Yan Wang, Volker Schillings, Christoph Meinel FB IV-Informatik, University of Trier, Trier 54296,

Stephen J. Guy 1. Photomontage Photomontage GrabCut – Interactive Foreground Extraction 1.

Image Matting and Its Applications Chen-Yu Tseng Advisor: Sheng-Jyh Wang

GrabCut Interactive Image (and Stereo) Segmentation Joon Jae Lee Keimyung University Welcome. I will present Grabcut – an Interactive tool for foreground.

Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1, Lehigh University.

Image Segmentation some examples Zhiqiang wang

The loss function, the normal equation,

10/11/2001Random walks and spectral segmentation1 CSE 291 Fall 2001 Marina Meila and Jianbo Shi: Learning Segmentation by Random Walks/A Random Walks View.

Boundary matting for view synthesis Samuel W. Hasinoff Sing Bing Kang Richard Szeliski Computer Vision and Image Understanding 103 (2006) 22–32.

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan, and Brian Kulis.

Light Mixture Estimation for Spatially Varying White Balance

Schedule Introduction Models: small cliques and special potentials Tea break Inference: Relaxation techniques:

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts

High-Quality Video View Interpolation

Abstract Extracting a matte by previous approaches require the input image to be pre-segmented into three regions (trimap). This pre-segmentation based.

Interactive Matting Christoph Rhemann Supervised by: Margrit Gelautz and Carsten Rother.

Stereo Computation using Iterative Graph-Cuts

An Iterative Optimization Approach for Unified Image Segmentation and Matting Hello everyone, my name is Jue Wang, I’m glad to be here to present our paper.

Lecture 8: Image Alignment and RANSAC

Understanding and evaluating blind deconvolution algorithms

A Trainable Graph Combination Scheme for Belief Propagation Kai Ju Liu New York University.

Bootstrapping a Heteroscedastic Regression Model with Application to 3D Rigid Motion Evaluation Bogdan Matei Peter Meer Electrical and Computer Engineering.

Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.

Image Segmentation Rob Atlas Nick Bridle Evan Radkoff.

06 - Boundary Models Overview Edge Tracking Active Contours Conclusion.

Tzu ming Su Advisor ： S.J.Wang MOTION DETAIL PRESERVING OPTICAL FLOW ESTIMATION 2013/1/28 L. Xu, J. Jia, and Y. Matsushita. Motion detail preserving optical.

A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting Jinlong Ju 1, Jue Wang 3, Yebin Liu 1, Haoqian Wang 2, Qionghai Dai.

1 Markov Random Fields with Efficient Approximations Yuri Boykov, Olga Veksler, Ramin Zabih Computer Science Department CORNELL UNIVERSITY.

Learning to Perceive Transparency from the Statistics of Natural Scenes Anat Levin School of Computer Science and Engineering The Hebrew University of.

Chapter 2-OPTIMIZATION G.Anuradha. Contents Derivative-based Optimization –Descent Methods –The Method of Steepest Descent –Classical Newton’s Method.

CS654: Digital Image Analysis Lecture 28: Advanced topics in Image Segmentation Image courtesy: IEEE, IJCV.

Improving Image Matting using Comprehensive Sampling Sets CVPR2013 Oral.

A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Statistics and Computing, Dec. 2007, Vol. 17, No.

Notes Over 3.1 Solving a System Graphically Graph the linear system and estimate the solution. Then check the solution algebraically.

Amir Yavariabdi Introduction to the Calculus of Variations and Optical Flow.

NATURAL IMAGE MATTING (Implementing Closed-form alpha matte solution as a python package) BTech. Project, I Sem Aniruddh Vyas CSE Dept, IIT Kanpur.

Hebrew University Image Processing Exercise Class 8 Panoramas – Stitching and Blending Min-Cut Stitching Many slides from Alexei Efros.

Image Blending : Computational Photography

Motion Segmentation with Missing Data using PowerFactorization & GPCA

Segmentation of Dynamic Scenes

Markov Random Fields with Efficient Approximations

Image Processing and Reconstructions Tools

Segmentation of Dynamic Scenes from Image Intensities

Single Image Haze Removal Using Dark Channel Prior

Xiang Liang Natural Image Matting Xiang Liang

Iterative Optimization

Learning to Combine Bottom-Up and Top-Down Segmentation

Spectral Clustering Eric Xing Lecture 8, August 13, 2010

Segmentation of Dynamical Scenes

3.5 Solving Nonlinear Systems

The loss function, the normal equation,

Mathematical Foundations of BME Reza Shadmehr

Systems of Equations Solve by Graphing.

More on natural image matting

Deblurring Shaken and Partially Saturated Images

Presentation transcript:

A Closed Form Solution to Natural Image Matting Anat Levin, Dani Lischinski and Yair Weiss School of CS&Eng The Hebrew University of Jerusalem, Israel

Matting and compositing +

The matting equations = x + x

Why is matting hard?

Why is matting hard?

Why is matting hard?

Matting is ill posed: 7 unknowns but 3 constraints per pixel Why is matting hard? Matting is ill posed: 7 unknowns but 3 constraints per pixel

Previous approaches The trimap interface: Scribbles interface: Bayesian Matting (Chuang et al, CVPR01) Poisson Matting (Sun et al SIGGRAPH 04) Random Walk (Grady et al 05) Scribbles interface: Wang&Cohen ICCV05

Problems with trimap based approaches Iterate between solving for F,B and solving for Accurate trimap required Input Scribbles Bayesian matting from scribbles Good matting from scribbles (Replotted from Wang&Cohen)

Wang&Cohen ICCV05- scribbles approach Iterate between solving for F,B and solving for Each iteration- complicated non linear optimization

Our approach Analytically eliminate F,B. Obtain quadratic cost in Provable correctness result Quantitative evaluation of results

Color lines Color Line: (Omer&Werman 04)

Color lines Color Line: B R G

Color lines Color Line: B R G

Color lines Color Line: B R G

Linear model from color lines Observation: If the F,B colors in a local window lie on a color line, then = Result: F,B can be eliminated from the matting cost

Evaluating an -matte ?

Evaluating an -matte ? ?

Evaluating an -matte ? ?

Theorem F,B locally on color lines Where local function of the image

Solving for using linear algebra Input: Image+ user scribbles

Solving for using linear algebra Input: Image+ user scribbles Advantages: Quadratic cost- global optimum Solve efficiently using linear algebra Provable correctness Insight from eigenvectors

Cost minimization and the true solution Theorem: Given: If: Then: locally on color lines Constraints consistent with

Matting and spectral segmentation Spectral segmentation: Analyzing smallest eigenvectors of a graph Laplacian L (E.g. Normalized Cuts, Shi&Malik 97)

Matting and spectral segmentation Spectral segmentation: Analyzing smallest eigenvectors of a graph Laplacian L (E.g. Normalized Cuts, Shi&Malik 97)

Comparing eigenvectors Input image Matting Eigenvectors Global- Eigenvectors

Matting results +

Quantitative results Experiment Setup: Randomize 1000 windows from a real image Create 2000 test images by compositing with a constant foreground using 2 different alpha mattes Use a trimap to estimate mattes from the 2000 test images, using the different algorithms Compare errors against ground truth

Quantitative results Smoke Matte Circle Matte Error Error Averaged gradient magnitude Averaged gradient magnitude Smoke Matte Circle Matte

Conclusions Analytically eliminate F,B and obtain quadratic cost . Solve efficiently using linear algebra. Provable correctness result. Connection to spectral segmentation. Quantitative evaluation. Code available: http://www.cs.huji.ac.il/~alevin/matting.tar.gz