1. Introduction to Multimedia1 Introduction yCoding Requirements yEntropy Encoding xContent Dependent Coding Run-length Coding Diatomic Coding xStatistical Encoding Huffman Coding Arithmetic Coding ySource Encoding xPredictive Coding Differential Pulse Code Modulation Delta Modulation yAdaptive Encoding

2. Introduction to Multimedia2 Coding Requirements yStorage Requirements xUncompressed audio: 8Khz, 8-bit quantization implies 64 Kbits to store per second xCD quality audio: 44.1Khz, 16-bit quantization implies storing 705.6Kbits/sec xPAL video format: 640X480 pixels, 24 bit quantization, 25 fps, implies storing 184,320,000 bits/sec = 23,040,000 bytes/sec yBandwidth Requirements xuncompressed audio: 64Kbps xCD quality audio: 705.6Kbps xPAL video format: 184,320,000 bits/sec zCOMPRESSION IS REQUIRED!!!!!!!

3. Introduction to Multimedia3 Coding Format Examples yJPEG for still images yH.261/H.263 for video conferencing, music and speech (dialog mode applications) yMPEG-1, MPEG-2, MPEG-4 for audio/video playback, VOD (retrieval mode applications) yDVI for still and continuous video applications (two modes of compression) Presentation Level Video (PLV) - high quality compression, but very slow. Suitable for applications distributed on CD-ROMs Real-time Video (RTV) - lower quality compression, but fast. Used in video conferencing applications.

4. Introduction to Multimedia4 Coding Requirements yDialog mode applications xEnd-to-end Delay (EED) should not exceed 150-200 ms xFace-to-face application needs EED of 50ms (including compression and decompression). yRetrieval mode applications xFast-forward and rewind data retrieval with simultaneous display (e.g. fast search for information in a multimedia database). xRandom access to single images and audio frames, access time should be less than 0.5sec xDecompression of images, video, audio - should not be linked to other data units - allows random access and editing

5. Introduction to Multimedia5 Coding Requirements yRequirements for both dialog and retrieval mode applications xSupport for scalable video in different systems. xSupport for various audio and video rates. xSynchronization of audio-video streams (lip synchronization) xEconomy of solutions Compression in software implies cheaper, slower and low quality solution. Compression in hardware implies expensive, faster and high quality solution. xCompatibility e.g. tutoring systems available on CD should run on different platforms.

6. Introduction to Multimedia6 Classification of Compression Techniques xEntropy Coding lossless encoding used regardless of media’s specific characteristics data taken as a simple digital sequence decompression process regenerates data completely e.g. run-length coding, Huffman coding, Arithmetic coding xSource Coding lossy encoding takes into account the semantics of the data degree of compression depends on data content. E.g. content prediction technique - DPCM, delta modulation xHybrid Coding (used by most multimedia systems) combine entropy with source encoding E.g. JPEG, H.263, DVI (RTV & PLV), MPEG-1, MPEG-2, MPEG-4

7. Introduction to Multimedia7 Steps in Compression yPicture preparation analog-to-digital conversion generation of appropriate digital representation image division into 8X8 blocks fix the number of bits per pixel yPicture processing (compression algorithm) transformation from time to frequency domain, e.g. DCT motion vector computation for digital video. yQuantization Mapping real numbers to integers (reduction in precision). E.g. U-law encoding - 12bits for real values, 8 bits for integer values yEntropy coding compress a sequential digital stream without loss.

8. Introduction to Multimedia8 Compression Steps Picture Preparation Picture Processing Quantization Entropy Coding Compressed Picture Uncompressed Picture Adaptive Feedback Loop

9. Introduction to Multimedia9 Types of compression zSymmetric Compression Same time needed for decoding and encoding phases Used for dialog mode applications zAsymmetric Compression Compression process is performed once and enough time is available, hence compression can take longer. Decompression is performed frequently and must be done fast. Used for retrieval mode applications

10. Introduction to Multimedia10 Entropy Coding - Run-length Encoding (RLE) yContent dependent coding yRLE replaces the sequence of same consecutive bytes with the number of occurrences. The number of occurrences is indicated by a special flag - “!” yRLE Algorithm: If the same byte occurred at least 4 times then count the number of occurrences Write compressed data in the following format: “the counted byte!number of occurrences” yExample Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG Compressed sequence - ABC!9DEF!4GGG (from 20 to 13 bytes)

11. Introduction to Multimedia11 Variations of Run-length coding (Zero suppression) yAssumes that only one symbol appears very often - blank. yAlgorithm: single blanks are ignored Starting with a sequence of 3 blanks, they are replaced by an M-byte and a byte with the number of blanks in the sequence. E.g. 3 - 258 zero bytes can be reduced to 2 bytes. xSubsitution depends on relative position. xExtended definitions are possible If M4 == 8 zero blanks, M5==16 zero bytes, M4M5 == 24 zero bytes.

12. Introduction to Multimedia12 Variations of run-length coding - Text compression yPatterns that occur frequently can be substituted by single bytes. yE.g. “Begin”, “end”, “if”… yAlgorithm: xUse an ESC byte to indicate that an encoded pattern will follow. xThe next byte is an index reference to one of 256 words (patterns). yCan be applied to still images, audio, video. yNot easy to identify small sets.

13. Introduction to Multimedia13 Variation of Run-length coding: Zero Compression yUsed to encode long binary bit strings containing mostly zeros. yEach k-bit symbol tells how many 0’s occurred between consecutive 1’s. ye.g. 0000000 - 7 zeros to be encoded. x111 000 (3 bit symbol) ye.g. 000100000001101 (using 3 bit symbol) x011 111 000 001 (3-7-0-1 zeros between 1s)

14. Introduction to Multimedia14 Variation of run-length coding - Diatomic Coding yDetermined frequently occuring pairs of bytes ye.g. an analysis of the English language yielded frequently used pairs - “th”, “in”, “he” etc.. yReplace these pairs by single bytes that do not occur anywhere in the text (e.g. X)… ycan achieve reduction of more than 10%

15. Introduction to Multimedia15 Statistical Encoding (Frequency Dependent) yFixed length coding Use equal number of bits to represent each symbol - message of N symbols requires L >= log_2(N) bits per symbol. Good encoding for symbols with equal probability of occurrence. Not efficient if probability of each symbol is not equal. yVariable length encoding frequently occurring characters represented with shorter strings than seldom occurring characters. Statistical encoding is dependant on the frequency of occurrence of a character or a sequence of data bytes. You are given a sequence of symbols: S1, S2, S3 and the probability of occurrence of each symbol P(Si) = Pi.

16. Introduction to Multimedia16 Huffman Encoding (Statistical encoding technique) yCharacters are stored with their probabilities xNumber of bits of the coded characters differs. Shortest code is assigned to most frequently occurring character. xTo determine Huffman code, we construct a binary tree. Leaves are characters to be encoded Nodes contain occurrence probabilities of the characters belonging to the subtree. 0 and 1 are assigned to the branches of the tree arbitrarily - therefore different Huffman codes are possible for the same data. Huffman table is generated. xHuffman tables must be transmitted with compressed data

17. Introduction to Multimedia17 Example of Huffman Encoding P(A) = 0.16 P(B) = 0.51 P(C) = 0.09 P(D) = 0.13 P(E) = 0.11 P(A) = 0.16 P(B) = 0.51 P(C) = 0.09P(D) = 0.13P(E) = 0.11 P(CE) = 0.20P(DA) = 0.29 P(CEDA) = 0.49 P(CEDAB) = 1 0 0 00 1 1 1 1 w(A) = 011 w(B) = 1 w(C) = 000 w(D) = 010 w(E) = 001

18. Introduction to Multimedia18 Arithmetic Encoding yEach symbol is coded by considering prior data xencoded sequence must be read from beginning; no random access possible. xEach symbol is a portion of a real number between 0 and 1. yArithmetic vs. Huffman xArithmetic encoding does not encode each symbol separately; Huffman encoding does. xArithmetic encoding transmits only length of encoded string; Huffman encoding transmits the Huffman table. xCompression ratios of both are similar.

19. Introduction to Multimedia19 Source Encoding - Differential Encoding yCoding is lossy. yConsider sequences of symbols S1, S2, S3 etc. where values are not zeros but do not vary very much. xWe calculate difference from previous value -- S1, S2-S1, S3-S2 etc. yE.g. Still image Calculate difference between nearby pixels or pixel groups. Edges characterized by large values, areas with similar luminance and chrominance are characterized by small values. Zeros can be compressed by run-length encoding and nearby pixels with large values can be encoded as differences.

20. Introduction to Multimedia20 Differential Encoding example 00000 0255250253251 0255251254255 00000 Compressed sequence: M5, 0, 255, -5, 3, -2, 0, 255, -4, 3, 1

21. Introduction to Multimedia21 Differential Encoding (cont.) zVideo applications xIn a newscast or video phone, the background does not change often, hence we can use run-length encoding to compress the background. xIn movies, the background changes - use motion compensation. Compare blocks of 8X8 or 16x16 in subsequent pictures. Find areas that are similar, but shifted to the left or right. Encode motion using a “motion vector”.

22. Introduction to Multimedia22 Differential Encoding for Audio yDifferential Pulse Code Modulation(DPCM) When we use PCM, we get a sequence of PCM coded samples. Represent first PCM sample as a whole and all the following samples as differences from the previous one. Previous Sample Difference Sample + + Previous Sample difference + - DPCM Encoder DPCM Decoder

23. Introduction to Multimedia23 DPCM Example y0 0.25 0.5 0.75 0.25 0 -0.25 -0.5 y000 001 010 011 001 000 100 101 y0 0.25 0.25 0.25 -0.5 -0.25 -0.25 -0.25 y00 01 01 01 11 10 10 10 Samples Digital Code Difference Need only 2 bits to encode difference

24. Introduction to Multimedia24 Delta Modulation yModification of DPCM yUses only 1 bit to encode difference. xSets 1 if the difference increases xSets 0 if the difference decreases yLeads to inaccurate coding