Presentation is loading. Please wait.

Presentation is loading. Please wait.

Www.imgtec.com www.powervr.com PowerVR is a division of Imagination Technologies Ltd. Texture Compression using Low-Frequency Signal Modulation (PVR-TC)

Similar presentations


Presentation on theme: "Www.imgtec.com www.powervr.com PowerVR is a division of Imagination Technologies Ltd. Texture Compression using Low-Frequency Signal Modulation (PVR-TC)"— Presentation transcript:

1 PowerVR is a division of Imagination Technologies Ltd. Texture Compression using Low-Frequency Signal Modulation (PVR-TC) Simon Fenney Graphics Hardware 2003

2 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 2 Presentation Overview  Why do we want Texture Compression?  Previous Research  Aims, Observations, & Overview  Decompression (4bpp) – PVR-TC  Compression (4bpp)  2bpp mode  Results & Future Research  Summary

3 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 3 Texture Compression – Section 1 Why do we want texture compression?

4 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 4 Why do we want Texture Compression?  Bandwidth! Bandwidth!! Bandwidth!!!  Memory can become a bottleneck  A low-cost device will usually have a narrow bus  Workstations & consumer devices: Much wider bus but…  …can be flooded by complex filtering and multiple texture layers

5 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 5 Why? (pt. 2)  Smaller Texture Footprint  Portable devices (e.g. PDAs, 3D-Enabled phones) have relatively little memory  Reduces texture swapping – more data can stay resident  Benefit when transferring texture data

6 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 6 Texture Compression Issues  Beers et al. list four factors to consider when evaluating a texture compression scheme:  Speed of Decode: the faster the better!  Random Access - usually implies a fixed-rate encoding. (no JPEG/Huffman)  Compression Rate vs. Quality: Can usually tolerate loss of fidelity  Encoding time: Can be ‘asymmetric’

7 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 7 Texture Compression – Section 2 Previous Research

8 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 8 Previous Texture Compression Schemes:  Can be approximately divided into 3 main families:  Code-book based  e.g. Colour Palettes/VQ  ‘Global’ compression scheme  Block Truncation Coding (BTC) based  Delp & Mitchell  Independent blocks of pixels  Transform Coding Schemes  e.g. DCT or Wavelet based  Relate position to colour values

9 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 9 CodeBook-based systems  Colour Palettes – simple VQ  Replace each pixel with index into codebook  Widely employed from flight simulators through to game consoles  Pros and cons: Well researched compression schemes – e.g. Heckbert, Wu  Requires indirection  Modest compression rate (8bpp) or low quality (4bpp)

10 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 10 CodeBook-based systems - VQ  VQ: e.g. Beers, Agrawala, & Chaddha  Replace 2x2 or 4x4 pixels with index  Pros and Cons High compression ratio (e.g. ~2bpp) with reasonable quality  Still requires indirection  Larger codebook

11 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 11 BTC-based schemes -Introduction  Block Truncation Coding (BTC) - Delp & Mitchell  Not strictly intended for Texture compression…  Monochrome 2bpp  Two 8-bit intensities per 4x4 pixel block  Single bit per pixel choice of intensity  Pros and Cons No indirection Each block compressed/decompressed independently…  …Each block compressed/decompressed independently

12 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 12 BTC-based schemes - CCC  Color Cell Compression: Campbell et al.  Extends BTC to support colour.  8-bit intensities replaced by palette indices.  Meant for image transmission but could be used for texturing (Knittel et al)  Pros and Cons 2bpp colour compression  Requires indirection  Discontinuities at block boundaries  Some banding is evident

13 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 13 BTC-based schemes – S3TC  S3TC: Iourcha et al.  Eliminates palette indirection by storing 2 colours per 4x4 block  Two additional inferred colours  Four colour choices per pixel  Pros & Cons 4bpp (opaque), 8bpp (with alpha) Typically good quality results – less banding than CCC Widely supported (DirectX, OpenGL, Nintendo GameCube)  Artefacts at block boundaries  Sometimes blocks don’t reduce to line in colour space

14 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 14 Transform Coding schemes  Less popular  Hierarchical Texture compression – Pereberin  Haar Wavelets  Stores 3 levels of MIP map simultaneously  ‘Unusual’ compression rate  Talisman TREC – Torborg & Kajiya  JPEG-like  Expensive??

15 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 15 Texture Compression – Section 3 Aims, Observations, & Overview

16 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 16 Aims of Research  2~4bpp, good quality for (A)RGB  Simple decode  Simultaneous decompression of 2x2 pixels for filtering  Try to avoid discontinuities at block boundaries

17 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 17 Observations  Block schemes – no internal correlation of position & colour  Bicubic and Bilinear (low frequency) representations are often ‘close’ approximations of original  position/colour correlation  Delta + Low Frequency = original image  ‘Delta’ image often locally monochromatic  Idea:  Represent low frequency and ‘filtered delta’ images using bicubics  Modulate filtered delta using per-pixel scalar values – simple decode, better with graphics

18 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 18 ‘Initial’ Experiment  Compressed Format:  Two, independent, 1/4 or 1/8 resolution images, A & B.  2-bpp ‘modulation’ image  A&B and modulation data separate  4-bpp or 2½-bpp compression rate  Decompression  Bicubic upscale of the low- resolution images  Blended according to ‘modulation’ value

19 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 19 Initial Experiment – 2  Quality of results was very promising but…  …system ‘too expensive’ to implement in a hand-held device (e.g. PowerVR MBX)  2.5bpp mode used 128-bit modulation blocks  Bi-cubic upscale required 16 source colours + interpolation  Cost reduced by using Bilinear interpolation and 64 bit memory blocks

20 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 20 Texture Compression – Section 4 Practical Decompressor

21 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 21 Practical Decompressor - 4bpp  A & B images, 15~16bits / colour  A&B colours fully opaque (554, 555) or variable transparency (3443, 3444)  4x4 bilinear upscale  2 bits per pixel modulation  {0, 3/8, 5/8, 1} or  {0, ½, ½+Alpha=0, 1}  Alpha Mask/Punch-through decode scheme less prone to halos

22 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 22 Bilinear Upscale Cost Reduction  Keep decompression cost low – target bilinear upscale  Bilinear low-colour precision data  Bit replication standard method to ‘increase’ colour depth  Linear operation so…  …can be done after bilinear operation  Need 2x2 pixels from each upscale  Borrow from HOS techniques: Use ‘recursive patch subdivision’  Handful of additions and MUX operations

23 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 23 Texture Compression – Section 5 The tricky bit … The Compression Algorithm

24 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 24 Compression Algorithm  Still a ‘work in progress’  Current method uses two main phases:  Computation of initial A & B signals  Iterative improvement of A & B given ‘closest’ modulation values

25 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 25 Phase 1: Initial A & B Images  ‘Low pass filter’ of image  Obtain delta signal  Generate principal axis signal from delta  Obtain optimum end points, A and B

26 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 26 Low Pass Filter  Want best fitting bilinear approximation  Could use optimum FIR filter for bilinear reconstruction…  …but it’s much cheaper to use linear wavelets and almost as effective

27 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 27 Low Pass Filter & Delta (x2) (256x256 image)

28 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 28 Axis Generation  Want to find maximum axes through deltas - similar to VQ and S3TC compression approaches  To avoid discontinuities, ‘flip’ some deltas to get better alignment  Low pass filter to get ‘axis’ image Flipped Delta Image

29 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 29 Generate A & B Signals – part 1  Get position of delta image relative to axes  For each pixel: compute (delta dot axis) / (axis dot axis)  Gives signed scalar values  Take overlapping (Gaussian) weighted windows of pixels and sort by value…

30 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 30 Generate A & B Signals – part 2  Search for ‘ideal’ representative end points  Reminiscent of searching in VQ or S3TC  Exhaustive search too expensive  Uses adaptation of Generalised Lloyds Algorithm (GLA) (e.g. used in VQ) + Least squares regression line fit.

31 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 31 Raw A & B: Lena ‘the Goth’ & ‘In the Headlights’  Multiply ideal end point values by axes to give raw A & B images  Add original low pass signal

32 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 32 Filtered Initial A and B

33 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 33 Intermediate Modulation and Result before optimisation passes…

34 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 34 Iterative improvement  Make several passes of the image  Each pass:  Compute Modulation (dot product with scaling) from current A and B  Quantise Modulation  Examine 3x3 block regions - Optimise A and B colours for region  Quantise colours.

35 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 35 Optimising region colours  Select regions equivalent to 3x3 blocks  Colours affected by 16 AB rep pairs  Outer 12 assumed fixed  Modulation values used to update inner four A&B reps…

36 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 36 …Optimising A and B… “… a good dose of gratuitous differential equations is needed to meet the paper quota…” P. Heckbert. ‘Ray Tracing JELL-O’ No DE’s but…

37 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 37 Region optimisation  Remove content from pixels due to fixed outer 12 reps  Evaluation of 121 pixels equivalent to matrix multiply M w by AB reps  M w matrix set of 121x8 weights.  Each weight is combination of position (bilinear) and modulation value  Need to ‘solve’ for AB vector. 121 equations with 8 unknowns!!

38 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 38 Singular Value Decomposition  Allows us to ‘solve’ over- determined systems  Solution gives result with the Least Squares Error  SVD finds ‘equivalent’ of inverse of M w  Multiply M w -1 by pixel values to obtain ‘best’ solution  But…

39 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 39 Pitfalls…  Despite 121 equations and 8 unknowns…  System can be under-constrained! e.g. Simple Gradation  Gives null space in solution matrix – take care  Garbage in, sub-optimal out…  Poor choice of modulation value gives odd ‘optimum’ results  A and B colours can fly off to extremes, e.g., image B

40 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 40 Final Result and Error image (x4)

41 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 41 Texture Compression – Section 6 2bpp mode  Similar to 4bpp  ‘x’ dimension scaled by 8 rather than 4 for A and B  Modulation values either 1bpp (like CCC) or interpolated from neighbours (2bpp in chequerboard pattern)

42 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 42 Example Results: Lorikeet Original S3TC 4bpp 2bpp

43 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 43 Examples RMS ImageS3TC4bpp Lorikeet * Lena Kodak Image Kodak Image

44 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 44 Future research  Apply to 3D textures  Re-investigate better interpolation  Higher or lower ‘colour’ dimensions  Remove ‘texture wrap around’ assumption  Better A&B fitting in compressor

45 PowerVR is a division of Imagination Technologies Ltd. © Imagination Technologies 45 Summary  Presented new lossy texture compression method.  Inexpensive hardware decode  Avoids indirection  ‘Block-like’ decode behaviour  Optimised to feed bilinear texture filter ‘building block’  Typically good quality  Closer match to image behaviour  Avoids block boundary artefacts  Compressor open-ended research!

46 PowerVR is a division of Imagination Technologies Ltd. Questions?


Download ppt "Www.imgtec.com www.powervr.com PowerVR is a division of Imagination Technologies Ltd. Texture Compression using Low-Frequency Signal Modulation (PVR-TC)"

Similar presentations


Ads by Google