Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Junhwan Kim and Fabio Pellacini Cornell University.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Junhwan Kim and Fabio Pellacini Cornell University."— Presentation transcript:

1 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Junhwan Kim and Fabio Pellacini Cornell University

2 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini GoalGoal Allow tile deformation Arbitrarily-shaped container and tiles Allow tile deformation Arbitrarily-shaped container and tiles 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

3 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Criteria Visually pleasing result Acceptable computational cost Little user effort Algorithm Criteria Visually pleasing result Acceptable computational cost Little user effort GoalGoal 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

4 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

5 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

6 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Giuseppe Arcimboldo 1527-1593 1527-1593 Motivation Fruit Face 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

7 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Fruit Face

8 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

9 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Related works 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions Mosaicing algorithms Photomosaics [Silvers,R and Hawley,M] (1997) (1997) Simulated Decorative Mosaics [Hausner] (siggraph2001) (siggraph2001) Escherization [Kaplan and Salesin] (siggraph2000) (siggraph2000) Packing algorithms Translational Polygon Containment and Minimal Enclosure using Mathematical Programming- [Milenkovic] (1999) Translational Polygon Containment and Minimal Enclosure using Mathematical Programming- [Milenkovic] (1999) Mosaicing algorithms Photomosaics [Silvers,R and Hawley,M] (1997) (1997) Simulated Decorative Mosaics [Hausner] (siggraph2001) (siggraph2001) Escherization [Kaplan and Salesin] (siggraph2000) (siggraph2000) Packing algorithms Translational Polygon Containment and Minimal Enclosure using Mathematical Programming- [Milenkovic] (1999) Translational Polygon Containment and Minimal Enclosure using Mathematical Programming- [Milenkovic] (1999)

10 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Related works

11 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

12 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Preparing Inputs Input=(container image + a set of tiles) of arbitrary shape Input=(container image + a set of tiles) of arbitrary shape Since we require a fairly high number of tiles, we need to be able to extract the shape of the tiles directly from the images themselves. We do so using active contours (Snake). Active contours are also used to extract the container shape.

13 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Preparing Inputs Following Hausner (Simulated Decorative Mosaics ‘01) approach, we segment the input image to generate a set of disjoint arbitrarily-shaped containers. Since we preserve the edge of each container, the final composite will preserve the important edges on the input image.

14 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

15 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini FrameworkFramework Energy minimization wCECwCECwCECwCEC wCECwCECwCECwCEC 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions Trying 3 rd tile…Trying 4 th tile… Container Container to be filled Container filled Tiles Legend Trying 5 th tile…Trying 2 nd tile… … high gap energy  discard Gap region …high overlap energy  discard Overlap region …high deformation energy  discard Shape mismatch …lowest energy  Accept Accepted tile Trying 1 st tile… Color mismatch …high color energy  discard + w G E G E = + w O E O + w D E D

16 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Framework (Energy terms evaluation) Framework (Energy terms evaluation) Ec : Which is estimated by taking the average of L^2 difference of the colors of the final image and the input container at random locations on the surface of the container. Ec : Which is estimated by taking the average of L^2 difference of the colors of the final image and the input container at random locations on the surface of the container. E G,E O : Each vertex of a tile is attached with a spring to the nearest edge of the other tiles or the container. If the signed distance d between the vertex and the anchor is positive (a gap between them), we add d^2/2 to E G. On the other hand, if d is negative, we add d^2/2 to Eo. E G,E O : Each vertex of a tile is attached with a spring to the nearest edge of the other tiles or the container. If the signed distance d between the vertex and the anchor is positive (a gap between them), we add d^2/2 to E G. On the other hand, if d is negative, we add d^2/2 to Eo. E D : E D :

17 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Advantages of Energy minimization Easy to control Easy to extend Photomosaic Photomosaic Simulated Decorative Mosaic Little user effort Advantages of Energy minimization Easy to control Easy to extend Photomosaic Photomosaic Simulated Decorative Mosaic Little user effort FrameworkFramework 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

18 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

19 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Basic Algorithm Overview 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions a) Initial container image b) Tile contours after tile placement c) Tile contours after tile refinement d) Final Jigsaw Image Mosaic Phase 3: Adjusting images Phase 1: Placing tiles Phase 2: Refining tiles In the third phase, we assemble the final mosaic by placing each tile in its position and warping each image to its final deformed shape using the image warping technique presented in (image warping by RBF,1994, Arad,N)

20 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini E = w C E C + w G E G + w O E O + w D E D PackingPacking Basic Algorithm Packing TileContainer with placed tile Container for next iteration abc Initial container 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

21 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Basic Algorithm Packing ContainerAvailable Tiles Place 1 st tileCannot place next Try again 1 st tile Backtrack Place next Done ab c 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

22 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Placing a tile When placing a tile in a container of arbitrary shape, it would be prohibitive to try every possible location. In order to reduce the branching overhead, we try those locations that are most likely to make the container shape easier to fill after updating. Unfortunately this depends on which tile we place. Nevertheless, we can guess how the container would look after we put an “average” tile. A container will be easier to fill if it does not have a protrusion and is as convex as possible.

23 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Placing a tile Before placing a new tile, we construct a CVD (centroidal voronoi diagram), where each site has an area roughly equal to the average size of tiles. We then select a random site among the ones that have the least number of neighbors, thus making the container as easy as possible to fill.

24 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini The refinement phase of our approach solves this issue by deforming the tiles obtained from the packing stage, while balancing between maintaining the original tile shape as closely as possible and minimizing the full energy function. E = w C E C + w G E G + w O E O PackingPacking RefinementRefinement Basic Algorithm Refinement + w D E D 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

25 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Basic Algorithm Refinement a) Initial contours b) Intermediate contours c) Converged contours 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

26 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

27 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Algorithm Optimization Placing a tile Branch-bound with Look-ahead Geometric hashing Placing a tile Branch-bound with Look-ahead Geometric hashing 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions V tile( V container) : the number of vertices per tile (container) N tile : the number of tiles in the database N tileInContainer : the number of tiles in the container b is the overhead due to branching in the search tree (backtracking).

28 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Placing a tile 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

29 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Look ahead (Russell and Norvig 1994) Algorithm Optimization Look ahead (Russell and Norvig 1994) Every time we cannot find a suitable tile to fill a container, we need to backtrack to the configuration that has minimal energy so far. To reduce branching overhead, we use a look-ahead technique. When placing a new tile, we penalize tiles that will make it harder to fill the container in the next iteration. To do this we add a term to the energy formulation that takes into account how the container will look after tile placement. The container shape term advocates for containers with a small area and short circumference.

30 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini E = w C E C + w G E G + w O E O PackingPacking E LA = w A area Look ahead Algorithm Optimization Look ahead (Russell and Norvig 1994) + w LA E LA + (1 - w A ) length 2 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions Where area is the container’s area, length is its boundary length, and W A controls the weight of area in relation to the weight of length.

31 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Container cleanup Algorithm Optimization Container cleanup After we place a tile in a container, we update the container by subtracting the tile from the container. However, the new container can be very jagged, or even have disjoint regions. If these fragments are shallower than the shallowest tile, we know it can never be filled with any existing tile. In that case, it is safe to separate those fragments and consider them as a gap. This cleanup process reduces the running time by cutting the number of vertices in the container. It also reduces the branching overhead, since it prevents the algorithm from wasting time attempting to fill unpromising fragments of the container.

32 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Geometric Hashing Algorithm Optimization Geometric Hashing We use geometric hashing to select a few tiles that will suit to a particular position in the container. We then evaluate the energy term for them and pick the best fitting one. Intuitively we use geometric hashing as a pruning technique to reject bad tiles. We create a grid of squares in the plane in a preprocessing phase. Each square corresponds to a hash table entry. If a shape boundary crosses a square, we will record the tile ID and its orientation as an entry in the list attached to that hash table entry.

33 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Geometric Hashing Algorithm Optimization Geometric Hashing Every time we need to place a new tile in a specific position in the container during the packing stage we register the container boundary segment to the hash table and access the hash table entries of the squares that the container passes through; for every entry found there, we cast a vote for the (tile ID, tile orientation) pair.

34 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Algorithm Optimization Geometric Hashing [Wolfson and Rigoutsos, 1997] 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

35 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

36 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini ResultsResults 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

37 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini ResultsResults a) Base case b) Lower color weight c) Lower overlap weight d) Tiles of finer scales 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

38 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini ResultsResults

39 ResultsResults

40 ResultsResults

41 ResultsResults

42 ResultsResults

43 ResultsResults

44 Motivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusionsMotivation Related Work Preparing Input Framework Basic Mosaic Algorithm Algorithm Optimization ResultsConclusions Outline 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

45 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Contributions New kind of mosaic General framework for mosaics Effective algorithm Future work Faster algorithm Extension to 3D Contributions New kind of mosaic General framework for mosaics Effective algorithm Future work Faster algorithm Extension to 3D ConclusionsConclusions 1.Motivation 2. Related Work 3. Framework 4. Algorithm 5.Results 6.Conclusions

46 Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini 報告完畢

Download ppt "Jigsaw Image Mosaics - Junhwan Kim and Fabio Pellacini Junhwan Kim and Fabio Pellacini Cornell University."

Similar presentations

Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France.

Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France.
Animosaics Kaleigh Smith, Yunjun Liu, and Allison Klein McGill University Eurographics Symposium on Computer Animation 2005.

Animosaics Kaleigh Smith, Yunjun Liu, and Allison Klein McGill University Eurographics Symposium on Computer Animation 2005.
Simulating Decorative Mosaics Alejo Hausner University of Toronto [SIGGRAPH2001]

Simulating Decorative Mosaics Alejo Hausner University of Toronto [SIGGRAPH2001]
Top-Down & Bottom-Up Segmentation

Top-Down & Bottom-Up Segmentation
Efficient access to TIN Regular square grid TIN Efficient access to TIN Let q := (x, y) be a point. We want to estimate an elevation at a point q: 1. should.

Efficient access to TIN Regular square grid TIN Efficient access to TIN Let q := (x, y) be a point. We want to estimate an elevation at a point q: 1. should.
MRI Brain Extraction using a Graph Cut based Active Contour Model Noha Youssry El-Zehiry Noha Youssry El-Zehiry and Adel S. Elmaghraby Computer Engineering.

MRI Brain Extraction using a Graph Cut based Active Contour Model Noha Youssry El-Zehiry Noha Youssry El-Zehiry and Adel S. Elmaghraby Computer Engineering.
November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.

November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.
Lecture 24 Coping with NPC and Unsolvable problems. When a problem is unsolvable, that's generally very bad news: it means there is no general algorithm.

Lecture 24 Coping with NPC and Unsolvable problems. When a problem is unsolvable, that's generally very bad news: it means there is no general algorithm.
Collision Detection CSCE 441. 2/60 What is Collision Detection?  Given two geometric objects, determine if they overlap.  Typically, at least one of.

Collision Detection CSCE /60 What is Collision Detection?  Given two geometric objects, determine if they overlap.  Typically, at least one of.
Techniques for Dealing with Hard Problems Backtrack: –Systematically enumerates all potential solutions by continually trying to extend a partial solution.

Techniques for Dealing with Hard Problems Backtrack: –Systematically enumerates all potential solutions by continually trying to extend a partial solution.
Ruslana Mys Delaunay Triangulation Delaunay Triangulation (DT)  Introduction  Delaunay-Voronoi based method  Algorithms to compute the convex hull 

Ruslana Mys Delaunay Triangulation Delaunay Triangulation (DT)  Introduction  Delaunay-Voronoi based method  Algorithms to compute the convex hull 
Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.

Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.
Force directed graph drawing Thomas van Dijk. The problem Given a set of vertices and edges, compute positions for the vertices. If the edges don’t have.

Force directed graph drawing Thomas van Dijk. The problem Given a set of vertices and edges, compute positions for the vertices. If the edges don’t have.
1 Minimum Ratio Contours For Meshes Andrew Clements Hao Zhang gruvi graphics + usability + visualization.

1 Minimum Ratio Contours For Meshes Andrew Clements Hao Zhang gruvi graphics + usability + visualization.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.

Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Computational Geometry and Spatial Data Mining

Computational Geometry and Spatial Data Mining
Nearest Neighbor. Predicting Bankruptcy Nearest Neighbor Remember all your data When someone asks a question –Find the nearest old data point –Return.

Nearest Neighbor. Predicting Bankruptcy Nearest Neighbor Remember all your data When someone asks a question –Find the nearest old data point –Return.
Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.

Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.
Hierarchical Region-Based Segmentation by Ratio-Contour Jun Wang April 28, 2004 Course Project of CSCE 790.

Hierarchical Region-Based Segmentation by Ratio-Contour Jun Wang April 28, 2004 Course Project of CSCE 790.
A New Force-Directed Graph Drawing Method Based on Edge- Edge Repulsion Chun-Cheng Lin and Hsu-Chen Yen Department of Electrical Engineering, National.

A New Force-Directed Graph Drawing Method Based on Edge- Edge Repulsion Chun-Cheng Lin and Hsu-Chen Yen Department of Electrical Engineering, National.

Similar presentations

Ads by Google