1. Medical Image Compression EECE 541 Multimedia Systems Harjot Pooni Ashish Uthama Victor Sanchez

2. Department of Electrical Engineering and Computer Engineering What are medical images ? Some examples MRI / FMRI (Function Magnetic Resonance)

3. Department of Electrical Engineering and Computer Engineering Medical Images Dynamic 3D Ultrasound PET (Positron emission Tomography) CT (computerized Tomograhpy)

4. Department of Electrical Engineering and Computer Engineering Why compress medical images? Growing need for storage Efficient data transmission Telemedicine Tele-radiology applications Real time Tele-consultation. PACS (Picture archiving and communication systems)

5. Department of Electrical Engineering and Computer Engineering Challenges unique to medical images. Compression Algorithms Lossy / Lossless Medical Images should always be stored in lossless format. Erroneous Diagnostics and its legal implications.

6. Department of Electrical Engineering and Computer Engineering Techniques used Compression techniques may be classified into:  Lossy  Lossless Moreover, compression algorithms may be applied in the spatial domain or frequency domain Compressed image e.g. WinZIP Transform to frequency domain Compressed image e.g. JPEG, JPEG2000

7. Department of Electrical Engineering and Computer Engineering JPEG 2000 and JPEG-LS High compression efficiency Lossless color transformations Progressive by resolution and quality Multiple component images ROI coding (static and dynamic) Error resilience capabilities Object oriented functionalities (coding, information, embedding)

8. Department of Electrical Engineering and Computer Engineering Drawbacks of JPEG 2000 and JPEG-LS Only looks for redundancy in the frame. Does not exploit 3D and 4D redundancy 3D Redundancy 3D medical image

9. Department of Electrical Engineering and Computer Engineering 4D Redundancy Exploits temporal redundancy.... Time 1Time 2Time 3Time n 4D medical image

10. Department of Electrical Engineering and Computer Engineering Ordering the data to exploit redundancies Transform the problem domain: Convert 4D data to a sequence of 2D data Volume Time Volume 1Volume 2 Volume n..... Slice 1 Slice s Volume 1 Slice s Volume 2 Volume n..... Slice s Slice 1

11. Department of Electrical Engineering and Computer Engineering 3D-JPEG 2000 Part 10 – JP3D “Part 10 is still at the Working Draft stage. It is concerned with the coding of three-dimensional data, the extension of JPEG 2000 from planar to volumetric images” -http://www.jpeg.org/jpeg2000/j2kpart10.htmlhttp://www.jpeg.org/jpeg2000/j2kpart10.html Some commercial vendors have already come out with 3D extensions of JPEG 2000 http://www.aware.com/products/compression/J2K3D.html http://www.aware.com/products/compression/J2K3D.html Provides guidelines for the use of JPEG 2000 for 3D data

12. Department of Electrical Engineering and Computer Engineering 3D-JPEG 2000 The basic approach Wavelet transforms

13. Department of Electrical Engineering and Computer Engineering 3D-JPEG 2000 The basic approach Reorder the 4D data by time or volume For each set, apply a 1D wavelet transform along the z axis Apply JPEG 2000 on each transformed slice H1 H2 L2 1D wavelet transform + JPEG 2000 coding = 3D-JPEG 2000 HL1 HH1 LH1 HL2 HH2 LL2 LH2

14. Department of Electrical Engineering and Computer Engineering Drawbacks Does not effectively use the redundancy in the 4 th dimension (Temporal redundancy) Movement of object between two slices would adversely effect performance Object motion is significant in medical imaging Patient movement Organ movement (Heart, Lung)

15. Department of Electrical Engineering and Computer Engineering H.264/AVC Latest video coding standard  uses motion compensation and estimation. Source: www.vcodex.com

16. Department of Electrical Engineering and Computer Engineering Why use H.264? Better Intra frame compression  Medical images have comparatively more uniform areas Motion estimation and compensation  Address temporal redundancies Multiple frames may be used to predict a single frame.  Better performance Different block sizes for motion estimation (16x16, 16x8, 8x8)  Better performance! Improved entropy encoder  Better performance!!

17. Department of Electrical Engineering and Computer Engineering Approach One: H.264-VOL Volume 1 Slice s Volume 2 Volume n..... Slice s Slice 1 Apply H.264/AVC on slices arranged as shown above Results: Compression TechniqueCompression ratio JPEG20002.55:1 JPEG-LS3.06:1 3D-JPEG 2000(VOL)3.15:1 H.264-VOL3.89:1

18. Department of Electrical Engineering and Computer Engineering Approach Two: H.264-TIME Volume 1Volume 2 Volume n..... Slice 1 Slice s Apply H.264/AVC on slices arranged as shown above Results: Compression TechniqueCompression ratio JPEG20002.55:1 JPEG-LS3.06:1 3D-JPEG2000 (Time)7.37:1 H.264-TIME12.38:1

19. Department of Electrical Engineering and Computer Engineering H.264 applied across time  H.264-TIME Volume 1Volume 2 Volume n..... Slice 1 Slice s Best compression performance

20. Department of Electrical Engineering and Computer Engineering How to improve compression efficiency? Two ideas: Get the difference between consecutive image slices, then use H.264 Calculate the residual frames, then use H.264 Main objective: reduce the energy content of each image slice.

21. Department of Electrical Engineering and Computer Engineering Difference between slices Difference H.264 MC + entropy coder (CABAC)........ Difference Slice 1 Slice s Difference 2 Difference s Difference 2 Difference s Difference 2 Difference s Volume 1 Volume 2 Volume n Reference slice Slice 1 Slice s Reference slice Slice 1 Slice s s coded bit- streams Slice 1Slice 2Difference

22. Department of Electrical Engineering and Computer Engineering Residual frames H.264 MC H.264 MC + entropy coder (CABAC)..... MVs + + + Volume 1Volume 2Volume n Residual 2 Residual s s coded bit- streams Slice 1 Slice s Reference slice Slice 1 Slice s Reference slice Slice 1 Slice s Reference slice Original slicePredictedResidual

23. Department of Electrical Engineering and Computer Engineering Results Compression Technique Improvement H.264 Difference H.264 Residual 3D-JPEG2000 100% H.264-TIME20%27%

24. Department of Electrical Engineering and Computer Engineering Future improvements Contextual encoding  take into account characteristics of image High motion Low motion

25. Department of Electrical Engineering and Computer Engineering Low motion areas  lossy High motion areas  lossless Future improvements Lossless Lossy

26. Department of Electrical Engineering and Computer Engineering Encoding using “slices” (group of macroblocks):  First slice for high motion areas  Second slice for low motion areas Slices may be encoded at different rates Future improvements First slice Second slice

27. Department of Electrical Engineering and Computer Engineering Questions?