Presentation on theme: "Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet."— Presentation transcript:
Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet Vision Workshop (CVPR 2008)
Online Virtual Globes
ClipMap pyramid. ServerClient Requests Tiles Problems: – Huge storage space required (one of the largest organized collections of imagery on the Internet). – Expensive acquisition and high transmission bandwidth. – Interpolation beyond available resolution: unnatural. User
Proposed Solution Proposition: – Synthesize, on the client, details for the lower pyramid levels. – Using super-resolution techniques. – Harnessing labels and textures samples from users (wiki model). Requirements: – Fast. – Seamless transition between layers.
User input 1: Labels Using interactive segmentation as in: Bai, X. and Sapiro, G., "A geodesic framework for fast interactive image and video segmentation and matting." ICCV, Original frame User provided labels Class 1 (grass) Class 2 (path)
User input 2: Keyframed Texture Keyframe 1 Keyframe 2 Keyscale 1 Keyscale 2 User provides the texture pyramid: Texture pyramid. e.g., in meters/pixel
Synthesis of a New Layer Input: from Server Output: New Layer Input: from Users ClipMap pyramid Labels Texture pyramid.
System Overview Undo Color Matching Texture Transfer Color Matching Interpolation Pyramid of Training Textures (from users) Inputs Last Layer (from server) Labels (from users) Outputs New Layer New Labels Selection of the Training Image
Texture transfer: 1 st pass Training Texture (from the texture pyramid) Color matched image, without high frequencies Wei, L. and Levoy, M., "Fast Texture Synthesis using Tree-structured Vector Quantization." SIGGRAPH, Efros, A. A. and Leung, T. K., "Texture Synthesis by Non-parametric Sampling." ICCV, Small contexts Fast Y Channel (luminance) I and Q Channels (chrominance) Mean and GradientOnly Mean Similarity between contexts considers Source Locations Produces 1-Pixel Patches: Contiguous areas copied verbatim from the training texture.
Texture transfer: 2 nd, 3 rd and 4 th passes Ashikhmin, M., "Synthesizing Natural Textures." ACM Symposium on Interactive 3D Graphics Few candidates Fast Training Texture Color Channel Pass 2 nd 3 rd and 4 th Y Channel (luminance) I and Q Channels (chrominance) Similarity between contexts considers Produces Bigger Patches
Texture transfer from the same texture Texture Pyramid 1 st synthetic frame 2 nd synthetic frame Patch interior (lilac and violet) directly copied. Patch boundaries (pink) synthesized in 4 passes as before. Doubles the patch size. ClipMap Pyramid