1. Multimedia communications EG-371& EE348Dr Matt Roach Multimedia Communications EG 371 and EE 348 Dr Matthew Roach Dr.matthew.roach@googlemail.com Lecture 1 Overview

2. Multimedia communications EG-371Dr Matt Roach Recommended texts F. Fluckiger, Understanding Networked Multimedia, Prentice Hall, 1995. R.A. Earnshaw and J.A. Vince, Multimedia Systems and Applications, Academic Press 1995. J. Ozer, Video Compression for Multimedia, AP Professional, 1995. F. Halsall Multimedia communications applications, networks, protocols and standards, Addison-Wesley, 2001.

3. Multimedia communications EG-371Dr Matt Roach Course Course notes online –Read up –Make own notes –http://galilee.swan.ac.uk/ Publications – teaching – EE371 10 lectures –Engineering E, 11:00 2 + Tutorials –Worksheets, last years exam questions

4. Multimedia communications EG-371Dr Matt Roach Course overview Visual signals Image definitions Image formats Video formats image information image compression

5. Overview of multimedia Multimedia communications EG-371Dr Matt Roach Multimedia Capture / generation Digital video signals Storage Data format JPEG MPEG4 & 7 Fundamentals of Compression Storage / transmission Transmission Data rates Request Internet search engines Relevance Consumption Human consumption

6. Multimedia communications EG-371Dr Matt Roach What is multimedia Media transferred over the net –Text Formatted, unformatted –Images Computer generated, digitised photographs –Audio Speech, sound –Video All of the above

7. Multimedia communications EG-371Dr Matt Roach Multimedia - video Digital Video –Visual signal Number of frames per second –A frame is a static image Spatial resolution Colour resolution –Acoustic signal Sampling rate Quantisation of the amplitude –Meta-data Text descriptions

8. Multimedia communications EG-371Dr Matt Roach Digital video signals What is a signal –Functions Scalar Vector Video capture Video storage

9. Multimedia communications EG-371Dr Matt Roach Fundamentals of compression Data information –Redundancy Compression ratio Types of redundancy –Coding –Inter-pixel –Psyco-visual –Data compression

10. Multimedia communications EG-371Dr Matt Roach Fundamentals of compression Design concerns –Trade-off quality loss Vs c compression ratio Measuring quality loss –Fidelity Criteria

11. Multimedia communications EG-371Dr Matt Roach Coding compression Histograms Entropy encoding –Run length encoding –Huffman encoding –Differential encoding

12. Multimedia communications EG-371Dr Matt Roach JPEG Static image compression standard –Image/block preparation –Forward DCT –Quantisation –Entropy encoding –Frame building

13. Multimedia communications EG-371Dr Matt Roach Tutorials & Exam questions Any questions?

14. Multimedia communications EG-371Dr Matt Roach Signals and Functions What is a signal Signal = function (variable with physical meaning) –one-dimensional (e.g. dependent on time) –two-dimensional (e.g. images dependent on two co-ordinates in a plane) –three-dimensional (e.g. describing an object in space) –higher-dimensional Scalar functions –sufficient to describe a monochromatic visual signals – black white video and sound. Vector functions –represent colour images - three component colours

15. Multimedia communications EG-371Dr Matt Roach Visual Functions image - continuous function of a number of variables Co-ordinates x, y in a spatial plane –for image sequences - variable (time) t Image function value = brightness at image points –other physical quantities –temperature, pressure distribution, distance from the observer Image on the human eye retina / TV camera sensor - intrinsically 2D 2D image using brightness points = intensity image Mapping 3D real world -> 2D image –2D intensity image = perspective projection of the 3D scene –information lost - transformation is not one-to-one –geometric problem - information recovery –understanding brightness info

16. Multimedia communications EG-371Dr Matt Roach Visual Acquisition & Manipulation Analogue camera –frame grabber –video capture card Digital camera / video recorder Capture rate 30 frames / second –Human Visual system (HVS) persistence of vision Computer, digitised image, software (example c) f(x,y) #define M 128 #define N 128 unsigned char f[N][M] 2D array of size N*M Each element contains an intensity value

17. Multimedia communications EG-371Dr Matt Roach Image definition Image definition: –A 2D function obtained by sensing a scene –F(x,y), F(x 1,x 2 ), F(x) –F- intensity, grey level –x,y - spatial co-ordinates No. of grey levels, L = 2 B B = no. of bits Why?

18. Multimedia communications EG-371Dr Matt Roach Brightness and 2D images Brightness dependent several factors –object surface reflectance properties surface material, microstructure and marking –illumination properties –object surface orientation with respect to a viewer and light source Some Scientific / technical disciplines work with 2D images directly –image of flat specimen viewed by a microscope with transparent illumination –character drawn on a sheet of paper –image of a fingerprint

19. Multimedia communications EG-371Dr Matt Roach Monochromatic images Image processing - static images - time t is constant Monochromatic static image - continuous image function f(x,y) –arguments - two co-ordinates (x,y) Digital image functions - represented by matrices –co-ordinates = integer numbers –Cartesian (horizontal x axis, vertical y axis) –OR (row, column) matrices Monochromatic image function range –lowest value - black –highest value - white Limited brightness values = gray levels

20. Multimedia communications EG-371Dr Matt Roach Chromatic images Colour –Represented by vector not scalar Red, Green, Blue (RGB) Hue, Saturation, Value (HSV) luminance, chrominance (Yuv, Luv) Red Green Hue degrees: Red, 0 deg Green 120 deg Blue 240 deg Green V=0 S=0

21. Multimedia communications EG-371Dr Matt Roach Use of colour space

22. Multimedia communications EG-371Dr Matt Roach Visual signal quality Quality of digital visual signal proportional to: –spatial resolution proximity of image samples in image plane –spectral resolution bandwidth of light frequencies captured by sensor –radiometric resolution number of distinguishable gray levels –time resolution interval between time samples at which images captured

23. Multimedia communications EG-371Dr Matt Roach Image summary F(x i,y j ) i = 0 --> N-1 j = 0 --> M-1 N*M = spatial resolution, size of image L = intensity levels, grey levels B = no. of bits

24. Multimedia communications EG-371Dr Matt Roach Digital Image Storage Stored in two parts –header width, height … cookie. –Cookie is an indicator of what type of image file –data uncompressed, compressed, ascii, binary. File types –JPEG, BMP, PPM.

25. Multimedia communications EG-371Dr Matt Roach PPM, Portable Pixel Map Cookie –Px Where x is: 1 - (ascii) binary image (black & white, 0 & 1) 2 - (ascii) grey-scale image (monochromic) 3 - (ascii) colour (RGB) 4 - (binary) binary image 5 - (binary) grey-scale image (monochromatic) 6 - (binary) colour (RGB)

26. Multimedia communications EG-371Dr Matt Roach PPM example PPM colour file RGB P3 # feep.ppm 4 15 0 0 0 0 0 0 0 0 0 15 0 15 0 0 0 0 15 7 0 0 0 0 0 0 0 0 0 0 0 0 0 15 7 0 0 0 15 0 15 0 0 0 0 0 0 0 0 0