1. 1/88 DCT Transform Decoder

2. 2/88 Image (512x512) Subsample (128x128) Manipulation Reposition : (256,256)-(384,384) Compress (JPEG) 167123 84200 2D array of data

3. 3/88 QUAN RUN-LEVEL CODING FINAL BITSTR DCT COMP & PART JPEG

4. 4/88 JPEG 2000 Framework WAVELET QUAN & PART BITPLANE CODING BITSTR ASSEMBLY FINAL BITSTR COMP & TILE COLOR IMAGE Y COMP CR COMP CB COMP

5. 5/88 Transform ORIGINAL 128, 129, 125, 64, 65, … TRANSFORM COEFFICIENTS 4123, -12.4, -96.7, 4.5, …

6. 6/88 2 LOW PASS ANALYSIS G(z) HIGH PASS ANALYSIS H(z) 2 X(z)X(z) LOW PASS COEFF γ(z) HIGH PASS COEFF  (z) 2 2 LOW PASS SYNTHESIS G’(z) HIGH PASS SYNTHESIS H’(z) Y(z)Y(z) + X(z)X(z) LOW PASS COEFF γ(z) HIGH PASS COEFF  (z) Y(z)Y(z) SPLIT P(z)P(z) P’(z) MERGE

7. 7/88 X(z)X(z) LOW PASS COEFF γ(z) HIGH PASS COEFF  (z) + sm(z)sm(z) + SPLIT tm(z)tm(z) + s0(z)s0(z) + t0(z)t0(z) K1K1 K2K2

8. 8/88 Y(z)Y(z) LOW PASS COEFF γ(z) HIGH PASS COEFF  (z) + sm(z)sm(z) + MERGE tm(z)tm(z) + s0(z)s0(z) + t0(z)t0(z) 1/K 1 1/K 2

9. 9/88 X(z)X(z) LOW PASS COEFF γ(z) HIGH PASS COEFF  (z) + f(a)f(a) + SPLIT f(b)f(b) + f(c)f(c) + f(d)f(d) K1K1 K2K2

10. 10/88 Lifting Implementation a=-1.586 b=-0.052 c= 0.883 d= 0.444 H0 H1 H2 H3 a a a a b b b c c c c d d d a a a a b b b b c c c c d d d d b d High LowOriginal x0 x1 x2 x3 x4 x5 x6 x7 x8 L0 L1 L2 L3 L4 a a 1 Just be saved in its own position Y = (x0+x2)*a + x1 x0 x1 x2............

11. 11/88 Inverse Transform

12. 12/88 Lifting Implementation Boundary Handling

13. 13/88 Inverse Transform -2d -d -c -2c -b -a -2a -a -b -2b INVERSE TRANSFORM L0 H0 L1 H1 x0 x1 x2 x3

14. 14/88 Wavelet Transform - 2D

15. 15/88 Wavelet Packet Structure mallat spaclpacket HL LHHH

16. 16/88 Line-based Lifting Scheme For 9-7 filter,the number of needed line in buffer is 6. High LowOriginal x0 x1 x2 x3 x4 x5 x6 x7 x8 L0 H0 L1 H1 L2 H2 L3 H3 L4 1 st Lift 2 nd Lift STEP 2 STEP 3... STEP 1

17. 17/88 Line-based Lifting Scheme Use of the partial character of wavelet transform No need to wait for all data Buffersize=linenumber ( depend on the filter’s length) * linesize LL HL LH HH Horizontal filter Vertical filter Lowpass of Vertical Highpass of Vertical

18. 18/88 Quantizer TRANSFORM COEFF 4123, -12.4, -96.7, 4.5, … QUANTIZE COEFF(Q=1) 4123, -12, -96, 4, …

19. 19/88 0 1 2 3 8 9 10 11 4 5 6 7

20. 20/88 Block Entropy Coding : Finally 45000 -74-1300 21040 14023 0000 3040 0350 0000 010 -4330 0010 0000 -4500 -180019 40230 000 BITSTREAM D R R-D CURVE

21. 21/88 A block of coefficients 45000 -74-1300 21040 14023 0000 3040 0350 0000 010 -4330 0010 0000 -4500 -180019 40230 000 01011010101101+ 1001010- 0010101+ 0001110+ 0000100- 0010010- 0000100+ 0000001- SIGN b 1 b 2 b 3 b 4 b 5 b 6 b 7 w0w1w2w3w4w5w6w7w0w1w2w3w4w5w6w7 ONE LINE OF COEF 45 -74 21 14 -4 -18 4

22. 22/88 010110101+ 1001010- 0010101+ 0001110+ 0000100- 0010010- 0000100+ 0000001- SIGN b 6 b 5 b 4 b 3 b 2 b 1 b 0 w0w1w2w3w4w5w6w7w0w1w2w3w4w5w6w7 45 -74 21 14 -4 -18 4 SIGNIFICANT IDENTIFICATION REFINEMENT PREDICTED INSIGNIFICANCE(PN) PREDICTED SIGNIFICANCE(PS) REFINEMENT (REF)

23. 23/88 Bits of Coefficients 01011010101101+ 1001010- 0010101+ 0001110+ 0000100- 0010010- 0000100+ 0000001- SIGN b 1 b 2 b 3 b 4 b 5 b 6 b 7 w0w1w2w3w4w5w6w7w0w1w2w3w4w5w6w7 COEFFICIENTS 45 -74 21 14 -4 -18 4

24. 24/88 Embedded Coding – The First Pass 0 1- 0 0 0 0 0 0 SIGN b 1 b 2 b 3 b 4 b 5 b 6 b 7 00 1- 0 0 0 0 0 0 w 0 *w 1 w 2 w 3 w 4 w 5 w 6 w 7 VALUE 0 RANGE -63..63 -96 -127..-64 -63..63 0 0 0 0 0 0

25. 25/88 Embedded Coding – The First Pass

26. 26/88 Embedded Coding – The First Pass

27. 27/88 Embedded Coding – The First Pass

28. 28/88 After Bitplane Scanning & Context Modeling 45000 -74-1300 21040 14023 0000 3040 0350 0000 010 -4330 0010 0000 -4500 -180019 40230 000 Bit: 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 …… Ctx: 0 0 9 0 0 0 0 0 0 7 10 0 0 0 0 0 0 0 0 ……

29. 29/88 h: number of significant horizontal neighbors (0-2) v: number of significant vertical neighbors (0-2) d: number of significant diagonal neighbors (0-4) v h d CURRENT

30. 30/88 Zero Coding Context

31. 31/88 Context in Sign Coding v h h, v: neighborhood sign status -1: one or both negative 0: both insignificant or both significant but opposite sign 1: one or both sig. & positive Expected Sign h v x context 1 1 0 13 1 0 0 12 1 -1 0 11 0 1 0 10 0 0 0 9 0 -1 1 -1 1 1 11 -1 0 1 12 -1 -1 1 13

32. 32/88 Magnitude Refinement ],[nm   h+vContext 1x16 0  115 0014 ],[nm   : Whether magnitude refinement has been applied h, v : Defined as in zero coding

33. 33/88 MQ-Coder BITS CTX BITSTREAM

34. 34/88 Arithmetic Coding - Illustration What is arithmetic coding 0 1 1-P 0 P0P0 1-P 1 P1P1 1-P 2 P2P2 S 0 =0 S 1 =1 S 2 =0 0.100 Coding result: (Shortest binary bitstream ensures that interval B=0.100 0000000 to D=0.100 1111111 is (B,D)  A ) A B D

35. 35/88 Probability estimation Assume within each context, the symbol is i.i.d (independently identically distributed) We can count the # of symbols and # of 1s and estimate the probability 1 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 1 0 0 0 0

36. 36/88 JPEG 2000 Bitstream Assembler

37. 37/88 Assemble the Bitstream Bitstream Rate-distortion optimized, for progressive by quality May be reordered Region with resolution access, progressive by quality Res1 Res2...... Encode each block separately & record a bitstream for each block... Abstraction Layer

38. 38/88 Rate-Distortion Optimized Assembling (1 st layer) D1D1 R1R1 D2D2 R2R2 D3D3 R3R3 D4D4 R4R4 D1D1 R1R1 D2D2 R2R2 D3D3 R3R3 D4D4 R4R4 r1r1 r2r2 r3r3 r4r4...

39. 39/88 Rate-Distortion Optimized Assembling (2 nd Layer) D1D1 R1R1 D2D2 R2R2 D3D3 R3R3 D4D4 R4R4 r’ 1 r’ 2 r’ 3 r’ 4 D1D1 R1R1 D2D2 R2R2 D3D3 R3R3 D4D4 R4R4 r1r1 r2r2 r3r3 r4r4...

40. 40/88 JPEG 2000 Bit Stream Syntax Packet Head Body Resync SOT marker SOC marker Global Header SOS marker Tile Header Packet Head Body Resync SOT marker SOS marker Tile Header EOI marker extra tiles Packet Head Body Resync Packet Head Body Resync Layer n Layer 1......

41. 41/88

42. 42/88 JPEG 2000 Bitstream What is in a packet? The compressed bitstream of  One component  One resolution  At a quality level Packet header :  length of a block bitstream in this packet Packet body :  The compressed bitstream

43. 43/88 Vmedia Interactive Image Browsing

44. 44/88 Vmedia Network Server Vmedia Media Program Media Network Vmedia Media Program... Client 1 Client n Vmedia Most work done at client end JPEG 2000 Compressed Images

45. 45/88 Virtual Media Concept Media Apps Network Service VMedia APIs Client Network Service Server BMP JPEG2000 Connect Request nid, nfid vfid Cache JPEG 2000 Virtual Media

46. 46/88 Render Apps Network Service VMedia APIs Client Cache Network Service Server BMP JPEG2000 Data Request Streaming with prioritization Data Cache Management

47. 47/88 Initial Stage Read filehead & media structure

48. 48/88 Initial Stage – JPEG 2000 JPEG 2000 image header & packet head are readin

49. 49/88 Entire Image – Low Res

50. 50/88 Zooming In x x

51. 51/88 Panning Around x x

52. 52/88 Operation Flow of the Vmedia Image Browser User Request Identify region of interest Identify related coefficient blocks Locate & access Associated bitstream Vmedia Interface Assemble bitstream Entropy decoding Inverse Quantization Inverse Lifting Shown to user

53. 53/88 Operation Flow of the Vmedia Client Access Request Availability Check Network Queues Cache Network Serve Requests Media database Vmedia client Vmedia server

54. 54/88 Vmedia in JPEG 2000 Viewer Without Vmedia Download entire file  Too slow A lot of http calls, or calls through imonitor  Lots of small network packets  No prioritization  No cache A lot of codes implement Vmedia functionality in similar media apps With Vmedia Provide all the features free Transparent to network Less codes

55. 55/88 Preemptive Loading Predict what the user wants next Zooming in and out, panning are the most common browse request Access data which is at the neighborhood of current view

56. 56/88 Demo ApplicationBrowser

57. 57/88 Summary What you should learn from the course Difference in file and image compression Key components of a modern image compression system New application and functionalities facilitated by the modern image compression system