The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Jigsaw Image Mosaics Junhwan Kim, Fabio Pellacini (Cornell University) Siggraph 2002

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie What is it?

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie What does it do? The Jigsaw Image Mosaic (JIM) algorithm takes as input an arbitrarily- shaped container image and a set of image tiles of arbitrary shape and generates a mosaic. It then packs the container as compactly as possible with tiles of similar color to the container taken from the input set while optionally deforming them slightly to achieve a more visually-pleasing effect.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Related work (1) Photomosaics: square tiles on a rectangular grid. [Silvers, Hawley, 1997; Finkelstein, Range 1998]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Related work (2) Simulated Decorative Mosaics: square tiles with varying orientations. [Hausner 2001]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Related work (3) Regular tiling with slight distortions of tile shapes. [Kaplan, Salesin 2000] Random positions of tiles, gaps filled with samples from original image. [Haeberli 1990] Packing problem (NP-hard) heuristics. [Downsland 1995, Milenkovic 1999]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie The problem Given an arbitrarily-shaped container image and a set of arbitrarily-shaped tiles {T i }, find a set of shapes {S j } such that: the union over the S j resembles the container image as closely as possible; and each S j is a translated and rotated copy of one of the T i, possibly incorporating a small deformation.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Energy minimization framework (Color, Gap, Overlap, Deformation)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie The algorithm-overview Prepare and segment images using active contours. [Kass 1987] Place tiles roughly, ignoring deformation (“packing”). Refine placement of tiles and deform if necessary. Assemble the final image.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie The algorithm

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Packing (1) Place tiles one by one, ignoring the deformation term, such that the energy is minimized, and the tile is aligned to the container boundary.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Packing (2) Place tiles until the container is full. If there is no suitable tile, backtrack to the last minimal energy configuration.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Refining Use active contours whose vertices are subject to forces that maintain the initial shape, repulse two penetrating contours and attract two contours separated by a gap.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Optimizations (1) Packing: use centroidal Voronoi diagrams with sites the size of the medium tile size, and always pick a site with the minimal number of neighbors.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Optimizations (2) Branch-and-bound with look-ahead: to reduce backtracking overhead, penalize tiles that would make it harder to fill the container in the next iteration – favor tiles with a small area and short circumference. Container cleanup: separate fragments shallower than the shallowest tile.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Optimizations (3) Geometric hashing: create a grid of squares representing hash table entries, then for each possible position and orientation of a tile, record the tile in the hash table.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Results

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Adrian Ilie Summary General energy-based framework. New kind of mosaic. Effective algorithm.