1. Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France

2. Different Types of Mosaics A. Hausner [SIGGRAPH 01] K. Smith et. al [SIGGRAPH 05] J. Orchard et. al [NPAR 08] Jesus at Basilica of Sant' Apollinare Nuovo

3. Classic Mosaic: Main constraints Tile orientation should emphasize important scene edges Tiles should be packed tightly while preserving their completeness Bad TilingGood Tiling

4. Previous Work User Interaction – Hausner [SIGGRAPH2001] – Elber and Wolberg[Visual Computer 2003] Explicit Edge detection – Di Blasi et. al [Visual Computer 2005] – Battiato et. al [WSCG 2006] Implicit Methods – Battiato et. al [EG 2008] – Liu et. al [EMMCVPR 2007]

5. Previous Work Cont. Requires user interaction Discontinuities in tile orientation filed No control over tile overlap Local optimization for tile orientation Heuristics for tile packing Hausner [SIGGRAPH 2001] Battiato et. Al [EG 2008]

6. Our Method: Overview We address the problem in a global optimization framework – pixel labelling problem Formulate energy which encodes constraints for a good mosaic – Encourage tiles to align to image edges – Smoothly varying tile orientations – Tight tile packing – Overlapping tiles prohibited High EnergyLow Energy

7. Mosaic as Pixel Labelling Each pixel p is assigned a pairwise label ( v p, ϕ p ) ϕ p is tile orientation – if tile centered at p is visible, orientation is ϕ p v p is visibility – 1 means tile is present – 0 means tile is hidden 0 0 1 1 ϕpϕp

8. Energy Function encourages dense packing encourages tiles to align to image edges encourages smooth tile orientations and prohibits tile overlap high cost low cost high cost low cost high cost low cost

9. Optimization with Graph cut NP-hard to optimize all variables simultaneously We use step-wise optimization – First, ignore the visibility labels and optimize the orientation variables – Then, fix the orientation labels and optimize the visibility variables

10. Optimizing Orientation Variables Assign an orientation label ϕ p to each pixel – dont know where the tiles will be placed, so optimize for all possible locations approximately optimized by alpha- expansion [Boykov et.al. PAMI01]

11. Optimizing the visibility variables Heuristically build many mosaic layers according to the orientation labels – Each layer consists of non-overlapping tiles – Each layer is as dense as possible

12. Maintain current solution Randomly pick mosaic layer Find the smallest energy stitching 1 1 0 0 1 1 0 1 0 1 1 0 1 1 formulated as binary labelling problem energy roughly corresponds to the area of the gap space, therefore encouraging reduction in gaps tiles will not overlap since energy would be infinite Submodular energy, optimized with a graph cut [Kolmogorov et. al. PAMI2004] Stitching Layers

13. Layer Stitching Illustration Tiles may cross the image edges and create a blurry effect Current SolutionRandom Layer

14. Layer Stitching illustration Tiles overlapping intensity edges are removed Good stitching Bad stitching

15. Overview of Layer Stitching Start from a random layer In each iteration, stitch the current solution and a candidate mosaic layer together Repeat until the result is visually pleasing

16. Experimental Results Our Approach Battiato et. Al [EG 2008]

17. Our Approach Battiato et. Al [EG 2008] Experimental Results

20. Mosaic with Different Tile Size Input Image 10 by 10 20 by 20

21. Number of Iterations in Stitching Starting Layer 27 iteration 156 iteration

22. Edge Avoidance Small Weight Large Weight

23. Conclusion and future work We address classic mosaic problem in an energy optimization framework Our energy function encodes desired mosaic properties Tiles align to image edges, minimize gap space, prohibit tile overlap We eliminate monotonous user interaction of drawing feature edges Take advantage of powerful discrete optimization methods based on graph cuts Results are visually pleasing Extensions: tiles of different shapes and sizes video mosaics