Presentation on theme: "Super-Resolution Texturing for Online Virtual Globes"— Presentation transcript:
1Super-Resolution Texturing for Online Virtual Globes Internet Vision Workshop (CVPR 2008)Super-Resolution Texturing for Online Virtual GlobesDiego Rother, Lance Williams and Guillermo SapiroUniversity of Minnesota and Google, Inc.
3Online Virtual Globes Requests Tiles Server Client User Problems: ClipMap pyramid.Problems:Huge storage space required (one of the largest organized collections of imagery on the Internet).Expensive acquisition and high transmission bandwidth.Interpolation beyond available resolution: unnatural.
5Proposed Solution Proposition: Requirements: Synthesize, on the client, details for the lower pyramid levels.Using super-resolution techniques.Harnessing labels and textures samples from users (wiki model).Requirements:Fast.Seamless transition between layers.
7User input 1: Labels Original frame User provided labels Class2 (path) (grass)Using interactive segmentation as in: Bai, X. and Sapiro, G., "A geodesic framework for fast interactive image and video segmentation and matting." ICCV, 2007.
8User input 2: Keyframed Texture User provides the texture pyramid:Keyscale1e.g., in meters/pixelKeyframe1Keyscale2Keyframe2Texture pyramid.
9Synthesis of a New Layer Input:from ServerInput:from UsersTexture pyramid.Output:New LayerClipMap pyramidLabels
10System Overview Inputs Outputs New Labels Labels (from server) Pyramid of Training Textures(from users)Inputs(from server)Last LayerLabelsOutputsNew LayerNew LabelsInterpolationColor MatchingTexture TransferUndo Color MatchingSelection of the Training Image
11Texture transfer: 1st pass Training Texture (from the texture pyramid)Color matched image, without high frequenciesY Channel(luminance)I and Q Channels (chrominance)Mean and GradientOnly MeanSimilarity between contexts considersSource LocationsProduces 1-Pixel Patches:Contiguous areas copied verbatim from the training texture.Small contexts → FastWei, L. and Levoy, M., "Fast Texture Synthesis using Tree-structured Vector Quantization." SIGGRAPH, 2000.Efros, A. A. and Leung, T. K., "Texture Synthesis by Non-parametric Sampling." ICCV, 1999.
12Texture transfer: 2nd, 3rd and 4th passes ColorChannelPass2nd3rd and 4thY Channel(luminance)I and Q Channels (chrominance)Similarity between contexts considersTraining TextureProduces Bigger PatchesFew candidates → FastAshikhmin, M., "Synthesizing Natural Textures." ACM Symposium on Interactive 3D Graphics
13Texture transfer from the same texture 1st synthetic frame2nd synthetic frameClipMap PyramidTexture PyramidPatch interior (lilac and violet) directly copied.Patch boundaries (pink) synthesized in 4 passes as before.Doubles the patch size.