1. Introduction Compression Performance Conclusions Large Camera Arrays Capture multi-viewpoint images of a scene/object. Potential applications abound: surveillance, special movie effects. Image-based rendering [Levoy ’96] Joint encoding of multiple views cannot be used Distributed Compression for Large Camera Arrays A distributed compression scheme for large camera arrays. Low-complexity Wyner-Ziv encoder Allows independent encoding of each camera view but centralized decoding to exploit inter- viewpoint image similarities. The existence of rendered side information and the use of shape adaptation techniques enhances compression efficiency. Experimental results show superior rate-PSNR performance over JPEG2000 and a JPEG-like SA- DCT coder, especially at low bit rates. Pixel domain coding and shape adaptation help to avoid blurry edges around the object (e.g., in JPEG2000) and blocky artifacts from block-based transform (e.g., in the SA-DCT coder). Xiaoqing Zhu, Anne Aaron and Bernd Girod Department of Electrical Engineering, Stanford University System Description Rendering of Side Information The geometry model is reconstructed from silhouette information of the conventional camera views Side information of the Wyner-Ziv camera views are rendered based on pixel correspondences derived from the geometry. Encoder Complexity CPU Execution Time milliseconds(ms) per picture Basic Operations The Wyner-Ziv encoder needs: 1 quantization step and 3 look-up-table procedures per pixel shape extraction and coding The JPEG2000 compressor needs: Multi-level 2-D DWT: ~ 5 multiplications per pixel Content-based arithmetic coding … WZ-ENC … Geometry Reconstruction Rendering Wyner-Ziv CamerasConventional Cameras Distributed Encoding Centralized Decoding WZ-ENC WZ-DEC Geometry Information Side Information Shape Adaptation Only encode pixels within the object shape Object shapes are obtained by chroma keying, compressed with JBIG, and then transmitted to the decoder. Wyner-Ziv Decoder Scaler Quantizer Turbo Coder Wyner-Ziv Encoder Turbo Decoder Reconstruction Buffer Parity Bits Request Bits Wyner-Ziv Codec The Wyner-Ziv coder in comparison with JPEG2000 and a SA-DCT coder, using the synthetic Buddha and the real-world Garfield data sets. Shape information is derived from perfect geometry for Buddha and coded at 0.0814 bpp for Garfield. The overhead of shape coding is counted in the Wyner-Ziv coder and the SA- DCT coder [ Aaron ’02] Shape Architecture Proposed Scheme Apply Wyner-Ziv coding to multi-viewpoint images Distributed encoding and joint decoding of the images, hence to benefit from the inter-viewpoint coherence. Stanford Camera Array, Courtesy of Computer Graphics Lab, Stanford Rate-PSNR Curve JPEG2000SA-DCT CoderWyner-Ziv Coder Reconstructed Images Rate = 0.11 bpp PSNR = 39.87 dBRate = 0.12 bpp PSNR = 38.89 dBRate = 0.11 bpp PSNR = 37.43 dB Rate = 0.13 bpp PSNR = 42.68 dBRate = 0.15 bpp PSNR = 41.86 dBRate = 0.13 bpp PSNR = 44.08 dB [Ramanathan ‘01] Contact: zhuxq@stanford.edu