2. 1 University of Canberra Advanced Communications Topics Television Broadcasting into the Digital Era by: Neil Pickford Lecture 1 Television Fundamentals, Analog TV and Formats

3. 2 Overview of Topics 1 - Fundamentals of Television Systems, Digital Video Sampling & Standards (2hr) 2 - Digital Audio/Video Stream Compression (2hr) 3 - Digital Modulation Systems for DTV 4 - Transmission System Error Protection 5 - Digital System Parameters, Planning and SI 6 - DTV Hardware 8 Hours total Fri 08:30-10:30 & Thu 12:30-13:30 1/2 Hour Multipart Question on Examination

4. 3 Digital Media n First media systems were Analog n Most media are converting to digital u Computer storage u Music (LP-CD) u Telecommunications u Multimedia u Internet Networking (TCPIP) u Radio (DAB) u Television (DTTB)

5. 4 What is Television n Images - Black and White Shades of Grey n Colour - Hue & Saturation n Sound - Audio Information n Data - Teletext & Other Data n Synchronisation - Specifies the Timing n Transport System - Gets the Above to your TV

6. 5 History - Ferdinand Braun - CRT n 1890 Ferdinand Braun developed the Cathode Ray Tube. n 1897 developed the Cathode Ray Oscillograph, the precursor to the radar screen and the television tube n 1907 First use of cathode ray tube to produce the rudiments of television images. n He shared the Nobel Prize for physics in 1909 with Guglielmo Marconi for his contributions to the development of wireless telegraphy.

7. 6 John Logie Baird - Basic TV n Oct 1923 John Logie Baird was the first person anywhere in the world to demonstrate true television in the form of recognisable images, instantaneous movement and correct gradations in light and shade. Scanning was done mechanically with a Nipkow disc. The first 30 line picture transmitted was a Maltese cross. n 1927 he also demonstrated video recording n 1928 transatlantic television n 1937 the broadcast of high definition colour pictures n 1941 stereoscopic television in colour n 1944 the multi-gun colour television tube, the forerunner of the type used in most homes today.

8. 7 Early Mechanical Approach to TV Mechanical Nipkow discs were used to scan the image and reconstitute the image at the receiver. PE cells were used to capture the image. The problem was synchronising the disks.

9. 8 30 Line Mechanical TV

10. 9 Electronic Television - Farnsworth n In 1922 at Age 14 Philo Farnsworth had the idea of how to make Electronic Television possible. n Sept. 7, 1927, Farnsworth painted a square of glass black and scratched a straight line on the centre. The slide was dropped between the Image Dissector (the camera tube that Farnsworth had invented earlier that year) and a hot, bright, carbon arc lamp. On the receiver they saw the straight-line image and then, as the slide was turned 90 degrees, they saw it move. This was the first all-electronic television picture ever transmitted.

11. 10 Vladimir Zworykin - Iconoscope n In 1923 Vladimir Zworykin of RCA made a patent application for a camera device, and by 1933 had developed a camera tube he called an Iconoscope. Although Zworykin submitted his patent application first after many years of legal battle Farnsworth was acknowledged as the inventor of electronic television. n By the end of 1923 he had also produced a picture display tube, the "Kinescope"

12. 11 Significant Television Inventions n These inventions were the underlying basis of the development of Television as we know it today

13. 12 Aspect ratio n First TV displays were Round n Rectangular Rasters easier to Generate n Television Developed using a 4:3 Aspect Ratio 4:3 (12:9) 16:9 n Cinematic formats are much wider n World now moving to 16:9 Aspect Ratio

14. 13 Film n Has been the highest Resolution storage format. n Various frame sizes used. 16mm, 35mm & 70mm n Difficult to produce, store, handle and display. n Easily degraded due to contamination and scratches. n Generally recorded at 24 fps. n Generally displayed at 72 fps (each frame 3x) to reduce flicker n Use a device called a Telecine to convert to television formats

15. 14 The Video Signal n First Television Pictures were Black & White Referred to as Luminance n Video refers to the linear base-band signal that contains the image information Front Back Porch Grey Background Sync Pulse White Stripe Black Stripe 0 mV 700 mV -300 mV

16. 15 Video Timing n SDTV 1 Line = 64 us Active Picture 52 us Sync 4.7 us Line Blanking 12 us u 64 us for each line (15.625 kHz) u 52 us Active Picture Area u 12 us Blanking and Synchronisation n Two level sync pulse 300 mV below blanking

17. 16 Frame Rate n A Frame represents a complete TV picture n Our analog TV Frame consists of 625 lines. n A Frame is usually comprised of 2 Fields each containing 1/2 the picture information n Our system has a Frame rate of 25 Hz n The Field rate is 50 Hz n Pictures displayed at 25 Hz exhibit obvious flicker n Interleaving the Fields reduces flicker.

18. 17 Flickerand Judder n Flicker and Judder are terms used to describe visual interruptions between successive fields of a displayed image. It affects both Film & TV. n If the update rate is too low, persistence of vision is unable to give illusion of continuous motion. n Flicker is caused by: u Slow update of motion Information u Refresh rate of the Display device u Phosphor persistence Vs Motion Blur n Judder usually results from Aliasing between Sampling rates, Display rates and Scene motion

19. 18 Interlace n To reduce the perceived screen flicker (25 Hz) on a television, a technique called 'interlacing' is employed. n Interlacing divides each video frame into two fields; the first field consists of the odd scan lines of the image, and the second field of each frame consists even scan lines. n Interlace was also used to decrease the requirement for video bandwidth. It is a form of Compression

20. 19 Interlaced Vs Progressive Scan n Interlaced pictures. - 1/2 the lines presented each scan 1,3,5,7,9,11,13...............623,625 field 1 2,4,6,8,10,12,14.............622,624 field 2 n Because the fields are recorded at separate times this leads to picture twitter & judder n Progressive pictures - all the lines sent in the one scan. 1,2,3,4,5,6,7,8................623,624,625 picture n No twitter or judder. n But twice the information rate.

21. 20 Progressive Scan n Simplifies the interpolation and filtering of images n Allows MPEG-2 compression to work more efficiently by processing complete pictures n Direct processing of progressively-scanned sources n 24 frame/second progressive film mode can be provided. n Assists video conversions with different: u numbers of scan lines u numbers of samples per line u temporal sampling (i.e., picture rate) Progressive Doubles Raw Data Requirement

22. 21ResolutionResolution n The number of picture elements resolved on the display n Resolution in TV is limited by: u Capture device u Sampling Rate u Transmission System / Bandwidth u Display Device F Dot Pitch, Phosphor F Focus & Convergence u Viewing distance / Display size u Human Eye n Typical SDTV systems attempt to transfer 720 pixels per line

23. 22 Colour Equations for PAL n For B&W only had to transmit Luminance (Y) n A Colour Image has Red, Green & Blue Components which need to be transmitted. n We already have the Y signal. n To remain compatible with Monochrome sets use Y, U & V to represent the Full Colour Picture Y = 0.299 R + 0.587 G + 0.114 B U = 0.564 (B - Y) V = 0.713 (R - Y) Colour Difference Signals

24. 23 A Compatible Colour System Y V U RGBRGBRGBRGB Y

25. 24 Colour Sub Carrier n Colour Sub- Carrier is added at 4.43361875 MHz n Frequency selected to interleave colour information spectra with Luma spectrum n More efficient use of spectrum.

26. 25 Adding Colour to B&W Video First TV signals were only Luminance In 1975 we added PAL Colour System A Colour Reference Burst on Back Porch And IQ modulated Colour Information

27. 26 Television Modulation - AM n TV uses Negative AM Modulation 0% 0% 100% 100%

28. 27 Amplitude Modulation

29. 28 TV Modulation - AM Min 20% n Peak White 20% Black 76% Syncs 100% 20% 20% 0% 0% 76% 76%100%100%

30. 29 TV Modulation - PAL AM n n Headroom prevents Colour Over/Under Modulating 20% 20% 0% 0% 76% 76%100%100%

31. 30 Frequency Modulation

32. 31 Intercarrier Sound n A FM subcarrier is added to the AM picture to carry the Audio information n FM Deviation 50 kHz used with 50 us Emphasis n PAL-B uses 5.5 MHz Sound subcarrier (L+R) u -10 dB wrt Vision for mono single carrier mode u -13 dB wrt Vision for Stereo & Dual mode n 2nd Sound subcarrier for Stereo (R) u 5.7421875 MHz (242.1875 kHz above main sound) u -20 dB wrt Vision carrier u 54.7 kHz Subcarrier Pilot tone added to indicate: Stereo (117.5 Hz) or Dual mode (274.1 Hz)

33. 32 FM Sound Emphasis dB Frequency (Hz)

34. 33 TV Modulation - Sound n n FM Sound Subcarriers Superimpose over the AM 20% 20% 0% 0% 76% 76%100%100%

35. 34 Vestigial Side Band - VSB n AM Modulation gives a Double Side Band signal u Each sideband contains identical information u 5 MHz of information means required BW > 10 MHz u Only one sideband is required for demodulation n To conserve spectrum Analog TV uses VSB u Only 1.25 MHz of the lower sideband is retained u VSB truncates the high frequency part of the lower sideband. n To implement Analog TV in 1950s with no lower sideband would have been very expensive because of the filtering required.

36. 35 PAL-B Spectrum -1.25+5.75 Sound -13 dB Sound -20 dB 0 dB Vision Carrier 4.433 Chroma -2 -1 0 1 2 3 4 5 6 -2 -1 0 1 2 3 4 5 6 Relative Frequency (MHz) Truncated Lower Sideband

37. 36 Frequencies Used n Australia uses 7 MHz Channels n VHF Band ICh 0-2 45 - 70 MHz n VHF Band IIICh 6-12174 - 230 MHz n UHF Band IV Ch 27-35520 - 582 MHz n UHF Band VCh 36-69582 - 820 MHz

38. 37 World TV Standards Australia is PAL NTSC PAL SECAM PAL/SECAM Unknown

39. 38 NTSC n National Television Systems Committee (NTSC) n First world wide Colour system Adopted (1966) n Generally used in 60 Hz countries n Predominantly 525 line TV systems n AM modulation of Luma & Syncs (4.2 MHz) n U & V Chroma AM Quadrature Modulated (IQ) n Chroma Subcarrier 3.579545 MHz n FM or Digital subcarrier modulation of Sound

40. 39 SECAM n Sequentiel Couleur Avec Memoire (SECAM) n Developed by France before PAL n 625 Line 50 Hz Colour system n Uses AM modulation for Luminance & Sync n Line sequentially sends U & V Chroma components on alternate lines n Receiver requires a 1H chroma delay line n Uses FM for Colour subcarrier 4.43361875 MHz n Uses FM for sound subcarrier

41. 40 PAL n Phase Alternation Line-rate (PAL) Colour System n Developed in Europe after NTSC & SECAM n Generally associated with 50 Hz Countries n Predominantly 625 Line system n AM modulation of Luma & Syncs (5 MHz) n U & V Chroma AM Quadrature Modulated with V (R-Y) component inverted on alternate lines n Chroma Subcarrier 4.43361875 MHz n FM or Digital subcarrier modulation of Sound

42. 41 Transmission Bandwidth - VHF Australia is one of a few countries with 7 MHz VHF TV 6 MHz 7 MHz 8 MHz Not in Use

43. 42 Transmission Bandwidth - UHF 6 MHz 7 MHz 8 MHz Not in Use Australia is Alone using 7 MHz on UHF

44. 43 U & V Components Y = 0.299 R + 0.587 G + 0.114 B B-Y = -0.299R - 0.587G + 0.866B R-Y = 0.701R - 0.587G + 0.114B U’ = B-Y V’ = R-Y

45. 44 Y, B-Y & R-Y Values B-Y = -0.299R - 0.587G + 0.866B B-Y Range is too large

46. 45 What makes a Colour Bar - RGB Red Green Blue Colour Bar

47. 46 Component Colour Bar - YUV Colour Bar V Y U

48. 47 Y, B-Y & R-Y Values R-Y = 0.701R - 0.587G + 0.114B R-Y Range is too large

49. 48 Y, U & V Values U = 0.564 (B-Y) V = 0.713 (R-Y)

50. 49 Component Video n Video distributed as separate Y U V Components n Y signal is 700 mV for Video Black-White n Y Signal carries Sync at -300 mV n U & V signals are 700 mV pk-pk. 350 mV at 0 350 mV 0 mV 700 mV -300 mV YVU

51. 50 Coax n Video Signals are transmitted on Coaxial Cable n 75 Ohm Coax - RG-59 or RG-178 n Video is usually 1 Volt Peak to Peak n Terminated with 75 Ohms at end of run n High impedance loop through taps are used n To split video must us a Distribution Amplifier n For Component signals all coaxes must be the same length otherwise mistiming of the video components will occur

52. 51 Standard Definition Television SDTV n The current television display system n 4:3 aspect ratio picture, interlace scan n Australia/Europe u 625 lines - 720 pixels x 576 lines displayed u 50 frames/sec 25 pictures/sec u 414720 pixels total n USA/Japan u 525 lines - 704 pixels x 480 lines displayed u 60 frames/sec 30 pictures/sec u 337920 pixels total

53. 52 Enhanced Definition Television EDTV n Intermediate step to HDTV n Doubled scan rate - reduce flicker n Double lines on picture - calculated n Image processing - ghost cancelling n Wider aspect ratio - 16:9 n Multi-channel sound

54. 53 High Definition Television - HDTV n Not exactly defined - number of systems n System with a higher picture resolution n Greater than 1000 lines resolution n Picture with less artefacts or distortions n Bigger picture to give a viewing experience n Wider aspect ratio to use peripheral vision n Progressive instead of interlaced pictures

55. 54 HDTV Parameters - AS 4599 n HDTV Defined as a MPEG-2 stream which is compliant with MP@HL encoding. n HDTV sample rate: u Less than 62 668 800 samples per second u Greater than 10 368 000 samples per second n Systems with less than 10 368 000 samples per second are defined as SDTV

56. 55 HDTV Have We Heard This Before? n The first TV system had just 32 lines n When the 405 line system was introduced it was called HDTV! n When 625 line black & white came along it was called HDTV! n When the PAL colour system was introduced it was called HDTV by some people. n Now we have 1000+ line systems and digital television - guess what? Its called HDTV!

57. 56 Do You Use A PC? All Current Generation PCs use Progressive Scan and display Pictures which match or exceed HDTV resolutions although the pixel pitch, aspect ratio and colorimetry are not correct. HDTV

58. 57 Video Formats - SDTV - 50 Hz All these formats are Interlaced

59. 58 Video Formats - HDTV - 50 Hz

60. 59 HD Video Formats 108019200 2,073,600 115214401,658,880 7201280921,600 576720414,720 480 345,600 1,552,200

61. 60 Common Image Format CIF n 1920 pixels x 1080 lines is now the world CIF. n All HDTV systems support this image format and then allow conversion to any other display formats that are supported by the equipment. n In Australia we have adopted the CIF for our HDTV production format. The Recommended Video format is 1920 x 1080 Interlaced at 50 Hz with a total line count of 1125 lines.

62. 61 Chromaticity u SDTV needs compatibility with legacy displays, so default SDTV chromaticity in DVB is: F same as PAL for 25Hz F same as NTSC for 30Hz u HDTV has unified world-wide chromaticity and no legacy displays F default is BT.709 for both 25Hz and 30Hz F simulcast allows mixture of legacy chromaticity for SDTV and BT.709 for HDTV

63. 62 BT-709 Colorimetry n HDTV uses a different colour space to SDTV n HDTV display Phosphors not same as SDTV n BT-709 defines the parameter values for HDTV n HDTV has a slightly different colour equation Y = 0.2126 R + 0.7152 G + 0.0722 B U = 0.539 (B - Y) V = 0.635 (R - Y) Colour Difference Signals

64. 63 Digital Television Why digital? To Overcome Limitations of Analog Television n Noise free pictures n Higher resolution images Widescreen / HDTV n No Ghosting n Multi-channel, Enhanced Sound Services n Other Data services.

65. 64 Digital Television - Types n Satellite (DBS) u DVB-S u Program interchange u Direct view / pay TV u SMATV Uplink Downlink

66. 65 Digital Television - Types n Cable u HFC - pay TV u MATV u DVB-C / 16-VSB Main Coax Fibre Tee Spur Tap

67. 66 Digital Television - Types n Terrestrial (DTTB) u DVB-T / 8-VSB u Free to air TV (broadcasting) u Narrowcasting/value added services u Untethered - portable reception

68. 67 Digital Terrestrial Television Broadcasting - DTTB n Regional free to air television n Replacement of current analog PAL broadcast television services n Operating in adjacent unused “taboo” channels to analog PAL service n Carries a range of services HDTV, SDTV, audio, teletext, data n Providing an un-tethered portable service

69. 68 Enabling Technologies n Source digitisation (Rec 601 digital studio) n Compression technology (MPEG, AC-3) n Data multiplexing (MPEG) n Transmission technology (modulation)

70. 69 Digitising Video - Rec BT-601 Output 27 MHz - Y Cr Y Cb Y Cr Y Cb ………….. 10 bit x 27 MHz = 270 Mbit/s

71. 70 Rec BT-601 - Sampling n Nyquist Rate for SDTV 11 MHz n 13.5 MHz base sampling rate. n Chrominance sample rate 6.75 MHz n 8 or 10 bit component samples

72. 71 Parallel BT-656 n 1st Rec 656 connection format used. n Uses 110 Ohm twisted pairs for data and clock n ECL level signalling @ 27 MHz n Width: 10 bits NRZ data + 1 clock pair n Uses standard DB-25 Female on Equipment n All cables are DB-25 Male to Male pin for pin n All cables have overall shield to prevent EMI n Max length without a DA 50 m, with EQ 200 m

73. 72 SDI - Serial BT-656 n Serial Data Interface - Current version of 656 n Uses standard 75 Ohm video coax Cabling n 1300 nm Optical fibre interface also defined n 270 Mb/s Serial data stream of 10 bit data n X 9 +X 4 +1 scrambling used for data protection n Encoding polarity free NRZI 800 mV pk-pk n 4 channel Audio can be encoded into ancillary data areas during the blanking period

74. 73 Sampling n Digital video requires sampling of the Analog image information. n Highest quality achieved when sampling Component video signals. n For SDTV a basic luminance sampling frequency of 13.5 MHz has been adopted. n Various methods exist to sample the complete colour image information 4:2:24:4:4 4:1:14:2:0

75. 74 YUV Y OnlyYUV 4:4:4 & 4:2:2 Sampling YUV Sampling Points 13.5 MHz 4:2:2 4:4:4

76. 75 YUV Y Only Y Only Y Only 4:1:1 & 4:2:0 MPEG-1 Sampling 4:2:0 YUV Sampling Points 13.5 MHz 4:1:1 Y V Y Y U Y JPEG/JFIF H.261 MPEG-1

77. 76 YUV Y Only Y Only Y Only 4:1:1 & 4:2:0 MPEG-2 Sampling 4:2:0 YUV Sampling Points 13.5 MHz 4:1:1 YV Y Only YU Y Only Co-sited Sampling MPEG-2

78. 77 Rec BT-601/656 n Digital Standard for Component Video n 27 MHz stream of 8 / 10 bit 4:2:2 Samples n 8 bit range 219 levels black to white (16-235) n Sync/Blanking replaced by SAV & EAV signals n Ancillary data can be sent during Blanking 128 16235 0 & 255 YVU

79. 78 Decoding Rec BT-601

80. 79 Rec BT-601 - Filtering Multiple A/D and D/A conversion generations should be avoided

81. 80 Enabling Technologies n Source digitisation (Rec 601 digital studio) n Compression technology (MPEG, AC-3) n Data multiplexing (MPEG) n Transmission technology (modulation)

82. 81 Video Bitrate - HDTV n 2 M pixels * 25 pictures * 3 colours * 8 bits = 1.24416 G bits / sec for Interlace Scan or = 2.4833 G bits / sec for Progressive We need to Compress this a bit!

83. 82 Compression Technology n When low bandwidth analog information is digitised the result is high amounts of digital information. 5 MHz bandwidth analog TV picture  170 - 270 Mb/s digital data stream. n 270 Mb/s would require a bandwidth of at least 140 MHz to transport n Compression of the information is required

84. 83 Compression - Types n Two types of compression available u Loss-less compression 2 to 5 times u Lossy compression 5 to 250 times

85. 84 Compression - Loss-less Types n Picture differences - temporal n Run length data coding - GIF u 101000100010001001101 = 1 + 4x0100 + 1101 F 21 bits source = 12 bits compressed u 01 11 31 31 31 21 01 11 F 21 symbols source = 16 symbols compressed n Huffman coding - PKZIP u Short codes for common blocks u Longer codes for uncommon blocks n Lookup tables

86. 85 Compression - Lossy Types n Quantisation - rounding n Motion vectors n Prediction & interpolation n Fractal coding n Discrete cosine transform (DCT)

87. 86 Approaches to Image Compression n Intraframe compression treats each frame of an image sequence as a still image. u Intraframe compression, when applied to image sequences, reduces only the spatial redundancies present in an image sequence. n Interframe compression employs temporal predictions and thus aims to reduce temporal as well as spatial redundancies, increasing the efficiency of data compression. u Example: Temporal motion-compensated predictive compression.

88. 87 MPEG-1: General Remarks - 1 n MPEG-1 standard simultaneously supports both interframe and intraframe compression modes. n MPEG-1 standard considers: u Progressive-format video only: u Luminance and two chroma channels representation where chroma channels are subsampled by a factor of 2 in both directions; u 8 bit/pixel video n Otherwise, appropriate pre- and post- processing steps should be carried out.

89. 88 MPEG-1: General Remarks - 2 n MPEG-1 standardises a syntax for the representation of encoded bit-stream and a method of decoding. n The standard syntax supports the operations of: u Discrete Cosine Transformation (DCT), u Motion-compensated prediction, u Quantisation, and u Variable Length Coding (VLC).

90. 89 MPEG-1 - I, P & B Frames Uncompressed SDTV Digital Video Stream - 170 Mb/s Picture 830kBytes 100 kBytes I Frame Picture 830kBytes B Frame 12-30 kBytes B Frame 33-50 kBytes P Frame n I - intra picture coded without reference to other pictures. Compressed using spatial redundancy only MPEG-2 Compressed SDTV Digital Video Stream - 3.9 Mb/s n P - predictive picture coded using motion compensated prediction from past I or P frames n B - bi-directionally predictive picture using both past and future I or P frames

91. 90 I Frames n Intraframe Compression u Frames marked by (I) denote the frames that are strictly intraframe compressed. u The purpose of these frames, called the "I pictures", is to serve as random access points to the sequence. IIIIIIIIIIII I

92. 91 I P P Frames n P Frames use motion-compensated forward predictive compression on a block basis. u Motion vectors and prediction errors are coded. u Predicting blocks from closest (most recently decoded) I and P pictures are utilised. P P Forward Prediction I

93. 92 I BB P B Frames n B frames use motion-compensated bi-directional predictive compression on a block basis. u Motion vectors and prediction errors are coded. u Predicting blocks from closest (most recently decoded) I and P pictures are utilised. BB P BB P Forward Prediction Bi-Directional Prediction BBI

94. 93 I BB P In Case of Poor Predictions n In both P and B pictures, the blocks are allowed to be intra compressed if the motion prediction is deemed to be poor. BB P BB P Forward Prediction Bi-Directional Prediction BBI

95. 94 IBBP Group of Pictures n Relative number of (I), (P), and (B) pictures can be arbitrary. n Group of Pictures (GoP) is the Distance from one I frame to the next I frame BBPBBPBBI 1 2 3 4 5 6 7 8 9 10 11 12 1 GoP = 12

96. 95 Some Other Frame Patterns n An I picture is mandatory at least once in a sequence of 132 frames (period_max= 132) IBBPBBIBBPBBI GoP = 6 IBIBIBIBIBIBI GoP = 2 IIIPIPIIIPIPI

97. 96 B8B7I2B6 Frame Transmission Sequence 1 2 3 4 5 6 7 8 9 10 11 12 1 B5P3B4B3P2B2B1P1 I2B8B7P3B6B5P2B4B3P1B2B1I1 I1 Source and Display Order Transmission Order

98. 97 MPEG Typical Frame Size GoP = 15

99. 98 Compression - DCT 8x8 Pixels

100. 99 Steps of Intra Frame Compression DCT Transform ( Efficient Representation) f(n,m) Quantise ( Reduce number of symbols) F(u,v) Segment Image into 8x8 Pixel Blocks Original Image F*(u,v) Symbol Coding (Minimise average length of symbols) Lossless Lossy Compressed Bit Stream Data

101. 100 Discrete Cosine Transformation (DCT) n DCT can be applied to various sample block sizes n For MPEG DCT is applied to 8 x 8 Blocks of Luminance and Chrominance data. and

102. 101 DCT - Original Spatial Pixels 5555 5555 109 55 109 109 55 5555 55 109109109109 109109109109109 109109109109109109 109109109109109109 555555555555 555555555555 555555555555 555555555555 55 55 55 55 55 55 55 m = 0 1 2 3 4 5 6 7 n = 0 n = 1 n = 2 n = 3 n = 4 n = 5 n = 6 n = 7 f(m,n) Spatial 8 x 8 Pixel Values

103. 102 NINT[ ] NINT = Nearest INteger Truncation DCT - Raw Values 602-69 -63147 -50 0 -24 -45 0 22-52 0 0162114 -220151912 000000 1680-5-7-4 000000 -1150343 000000 940-3-4-2 0 -15 0 12 34 0 -29 u = 0 1 2 3 4 5 6 7 v = 0 v = 1 v = 2 v = 3 v = 4 v = 5 v = 6 v = 7 F(u,v) Frequency Domain 8 x 8 Transform Values

104. 103 Why use transform Coding? n The purpose of transformation is to convert the data into a form where compression is easier n Transformation yields energy compaction u Facilitates reduction of irrelevant information n The transform coefficients can now be quantised according to their statistical properties. u This transformation will reduce the correlation between the pixels (decorrelate X, the transform coefficients are assumed to be completely decorrelated (Redundancy Reduction).

105. 104 How Do Transforms Work? n Basic Fourier Analysis u Waveform is composed of simpler sinusoidal functions u Providing enough of the waveform is sampled, component frequencies can be determined that approximate the original waveform n The component frequencies are the basis of the original waveform. u Basis waveforms change with each type of transform.

106. 105 2D DCT Basis Function

107. 106 1D DCT Basis Function x = 4 x = 5 x = 6 x = 7 x = 3 x = 2 x = 1 x = 0  = cos  nx X For Simplicity the 2D Basis function can be reduced to a 1D function that is applied in both x and y dimensions

108. 107 4 x 4 - DCT Basis Block Pattern u = 0 u = 1 u = 2 u = 3 v = 0 v = 1 v = 2 v = 3 Diagram Simplified by 1 bit Quantising the pattern

109. 108 DCT Block Scan Sequence u = 0 u = 1 u = 2 u = 3 v = 0 v = 1 v = 2 v = 3

110. 109 4 x 4 DCT Patterns

111. 110 Quantisation - DC Coefficient n The DCT coefficients are uniformly quantised. n DC and AC Coefficients are treated differently. n The DC Coefficient u The DC coefficient is divided by 8, and the result is truncated to the nearest integer in [-256 255] range. u F*(0,0) = NINT[F(0,0)/8]

112. 111 Quantisation - AC Coefficients n Each AC coefficient, F(U, V) is first multiplied by 16 and the result is divided by a weight. w(u, v). times the quantiser_scale. n F*(u,v) = NINT[16 * F(u,v)/w(u,v) * quantiser_scale]. n The result is then truncated to [-256,255] range. n The 8 x 8 array of weights, w(u,v), is called the quantisation matrix. n The parameter quantiser_scale facilitates adaptive quantisation.

113. 112 MPEG-1 Quantisation Matrix 816161619192222 22 2222 22262729342427293437 262729343438 262729343740 272932354048 293235404858 293438465669 353846566983 26 26 27 29 26 26 27 w(u,v) Weights are 8 bit integers Default Matrix Matrix can be downloaded

114. 113 DCT Example - Original Image 5555 5555 109 55 109 109 55 5555 55 109109109109 109109109109109 109109109109109109 109109109109109109 555555555555 555555555555 555555555555 555555555555 55 55 55 55 55 55 55

115. 114 Example - Raw DCT Coefficients 602-69 -63147 -50 0 -24 -45 0 22-52 0 0162114 -220151912 000000 1680-5-7-4 000000 -1150343 000000 940-3-4-2 0 -15 0 12 34 0 -29

116. 115 Example - Quantised DCT - QS=2 75-35 -3274 -21 0 -9 -16 0 8-19 0 0563 -70443 000000 520-2 000000 -310110 000000 210000 0 -4 0 3 10 0 -9 Quantiser_scale = 2

117. 116 Example - Quantised DCT - QS=7 75-10 -921 -6 0 -2 -5 0 2-5 0 0121 -20111 000000 110000 00000000000 000000 100000 0 0 1 3 0 -2 Quantiser_scale = 7

118. 117 Example - 8 x 8 Scan Sequence 75-10 -921 -6 0 -2 -5 0 2-5 0 0121 -20111 000000 110000 00000000000 000000 100000 0 0 1 3 0 -2 Quantiser_scale = 7

119. 118 Example - Inverse DCT - Result 5959 5956 105 56 107 105 62 6457 54 107110107107 108107108106107 105110107107106107 100108107106103102 615755565960 565854555755 565654545655 565954535655 50 52 55 55 55 55 52 Quantiser_scale = 7 Received 8 x 8 pixel block at the Decoder

120. 119 Assignment - Draw accurately the Full 8x8 DCT Basis block set Simplified Diagram by 1 bit Quantising the pattern

121. 120 Quantised Data Stream Quantiser_scale = 2 75 -35 74 0 -32 -21 -9 -16 0 -19 0 8 0 -7 0 5 0 0 5 0 10 0 -40 2 0 4 6 3 4 0 0 0 -3 0 -9 3 0 1 0 -1 0 3 0 -2 0 0 0 2 1 0 1 0 -1 0 1 0 0 0 0 0 0 0 0 Quantiser_scale = 7 75 -10 21 0 -9 -6 -2 -5 0 -5 0 2 0 -2 0 1 0 0 1 0 3 0 -1 0 1 0 1 2 1 1 0 0 -1 0 -2 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Quantiser_scale = 16 75 -4 9 0 -4 -3 -1 -2 0 -2 0 1 0 -1 0 1 0 0 1 0 1 0 -1 0 0 0 1 1 0 1 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Quantiser_scale = 4 75 -17 37 0 -16 -11 -4 -8 0 -9 0 4 0 -4 0 2 0 0 2 0 5 0 -2 0 1 0 2 3 2 2 0 0 0 -2 0 -4 2 0 1 0 -1 0 1 0 -1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

122. 121 Spatially-Adaptive Quantisation n Spatially-adaptive quantisation is implemented by the quantiser_scale, that scales the w(u,v) values u The quantiser_scale is allowed to vary from one "macroblock” to another within a picture to adaptively adjust the quantisation on a macroblock basis. u u The quantiser_scale is chosen from a specified set of values on the basis of spatial activity of the block (e.g., macroblocks containing busy, textured areas are quantised relatively coarsely), and on the basis of buffer fullness in constant bitrate applications.

123. 122 Coding: AC Coefficients n Coding is based on the fact that most of the quantised coefficients are zero and hence it is more efficient to represent the data by location and value of the non-zero coefficients. n The quantised AC coefficients are scanned in a zigzag fashion and ordered into symbol = [Run, level] pairs and then coded using variable length (Huffman) codes (VLC) (longer codes for less frequent pairs and vice versa). u (The VLC tables are standardised.)

124. 123 Example - Run Level Coding Quantiser_scale = 7 75 -10 21 0 -9 -6 -2 -5 0 -5 0 2 0 -2 0 1 0 0 1 0 3 0 -1 0 1 0 1 2 1 1 0 0 -1 0 -2 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Run Level 0-10 0-10 0 21 0 21 1-9 1-9 0-6 0-6 0-2 0-2 0-5 0-5 1-5 1-5 1 2 1 2 Run Level 1-2 1-2 1 1 1 1 2 1 2 1 1 3 1 3 1-1 1-1 1 1 1 1 0 2 0 2 Run Level 0 1 0 1 2-1 2-1 1-2 1-2 0 1 0 1 5 1 5 1 EOB 63 DCT coefficients represented by 47 symbols u Level: is the value of a non-zero coefficient; u Run: is the number of zero coefficients preceding it.

125. 124 Coding: DC Coefficients n Redundancy among quantised DC coefficients of 8 x 8 blocks is reduced via differential pulse coded modulation (DPCM). n The resultant differential signal ([-255, 255] range) is coded using variable length codes. u Standard VLC tables are specified. u These tables are the only standard tables in MPEG-1 that make a distinction between luminance and chrominance components of the data.)

126. 125 MPEG-1 Bit Stream Hierarchy

127. 126 MPEG Encoder n A typical MPEG encoder includes modules for: u Motion estimation u Motion-compensated prediction (predictors and framestores) u Quantisation and de-quantisation u DCT and IDCT u Variable length coding u a Multiplexer u a Memory buffer u a Buffer regulator

128. 127 Simplified MPEG Encoder  DCT Store Q IQ IDCT  MC Pred VLC Mux Bit Stream Digital Video Motion Vectors Side Info Audio, System & Other Data

129. 128 MPEG Decoder n The decoder basically reverses the operations of the encoder. n The incoming bit stream (with a standard syntax) is demultiplexed into: u DCT coefficients u Side information F Displacement vectors F Quantisation parameter, etc. n In the case of B pictures, two reference frames are used to decode the frame.

130. 129 MPEG Decoder  Store IQIDCT MC Pred VLC DeMux Motion Vectors + Side Information Image Data Program Stream Digital Video

131. 130 MPEG-2 - Formats ML & HL n MPEG-2 defines profiles & levels u They describe sets of compression tools n DTTB uses main profile. n Choice of levels n Higher levels include lower levels Level resolution Level resolution Low level (LL) 360 by 288SIF Main level (ML) 720 by 576SDTV High level (HL)1920 by 1152HDTV

132. 131 MPEG Profiles and Levels 422P@HL HP@HL HP@H14L SSP@H14L MP@H14L MP@HL HP@ML SNRP@ML SNRP@LL MP@LL SP@ML MP@ML 422P@ML 300 Mbit/s 100 Mbit/s 80 Mbit/s 60 Mbit/s 40 Mbit/s 20 Mbit/s HIGH HIGH-1440 MAIN LOW SIMPLE MAIN SNR SCALABLE HIGH SPATIALLY SCALABLE 4:2:2 LEVELS MAX. BIT- RATE PROFILES

133. 132 It is preferable that all decoders sold in Australia be MP@HL capable allowing all viewers access to HD resolution when it becomes commonly available MP@HLMP@ML

134. 133 Digital Audio - Multichannel n Two sound coding systems exist for Digital TV u MPEG 1 & 2 u Dolby AC-3 n Cover a wide variety of Audio Applications u DVB u VCD and S-VCD u DAB, DBS, DVD u Cinema (Film) u Computer Operating Systems (Windows) u Professional (ISDN codecs, tapeless studio, ….)

135. 134 Multichannel Sound TV C LFE L R RsLs

136. 135 Masking n Both use perceptual audio coding that exploits a psychoacoustic effect known as masking

137. 136 Multichannel Sound - MPEG 1/2 n MPEG Audio Layer II was developed in conjunction with the European DVB technology u Uses Musicam Compression with 32 sub bands u MPEG 1 is basic Stereo 2 channel mode u MPEG 2 adds enhancement information to allow 5.1 or 7.1 channels with full backwards compatibility with the simple MPEG 1 decoders u MPEG 1 is compatible with Pro-Logic processing. u Bitrate 224 kb/s MPEG 1 u Bitrate 480 - 512 kb/s MPEG 2 5.1

138. 137 MPEG Audio Encoder Subband Filter Bit Allocation Psycho- Acoustic Model Frame Packer Quantiser & Coder O/P Audio In 32 Sub-bands Coding of Side Information Audio Bit Stream 2 x 32-192 kb/s 2 x 768 kb/s

139. 138 MPEG Audio Decoder Frame Unpacker Inverse Subband Filter De-Quantiser Audio Bit Stream Audio Out Decoding of Side Information 2 x 768 kb/s 2 x 32-192 kb/s

140. 139 Multichannel Sound - Dolby AC-3 n Dolby AC-3 was developed as a 5.1 channel surround sound system from the beginning. u Compression Filter bank is 8 x greater than MPEG 2 (256) u Must always send full 5.1 channel mix One bitstream serves everyone u Decoder provides down-mix for Mono, Stereo or Pro-Logic u Listener controls the dynamic range, Audio is sent clean u Bitrate 384 kb/s or 448 kb/s u Dialogue level passed in bit-stream

141. 140 AC-3 Multichannel Coder 5.1-ch Encoder R L C LS RS LFE 5.1-ch Decoder R L C LS RS LFE Encoder Decoder

142. 141 AC-3 Stereo Decoder 5.1-ch Encoder R L C LS RS LFE Ro Lo Matrix 5.1-ch Decoder R L C LS RS LFE Encoder 2-channel Decoder

143. 142 MPEG-2 Multichannel Coder concept Down mix R L C LS RS LFE Ro Lo Ro Lo MPEG-1 Encoder MPEG-1 Decoder Extension Encoder Re matrix R L C LS RS LFE MPEG-2 Encoder MPEG-2 Decoder Extension Decoder

144. 143 Low cost 2-channel decoder Ro Lo R L C LS RS LFE LFE T2 T3 T4 MPEG-1 Encoder Extension Encoder Down mix MPEG-2 Encoder MPEG-1 Decoder Ro Lo  Low cost 2-channel decoder 2-channel Decoder

145. 144 Widely Available n All major MPEG-2 Video decoders incorporate 2-channel or 5.1 channel MPEG-2 Audio n Several dedicated MPEG-2 multichannel decoders n More than 100 Million decoders world-wide

146. 145 Enabling Technologies n Source digitisation (Rec 601 digital studio) n Compression technology (MPEG, AC-3) n Data multiplexing (MPEG) n Transmission technology (modulation)

147. 146 MPEG-2 n Compresses source video, audio & data n Segments video into I, P & B frames n Generates system control data n Packetises elements into data stream n Multiplexes program elements - services n Multiplexes services - transport stream n Organises transport stream data into 188 byte packets

148. 147 Digital Terrestrial TV - Layers... provide clean interface points.... Picture Layer Multiple Picture Formats and Frame Rates Multiple Picture Formats and Frame Rates 1920 x 1080 1280 x 720 50,25, 24 Hz Transmission Layer 7 MHz COFDM / 8-VSB VHF/UHF TV Channel Video Compression Layer MPEG-2 compression syntax ML@MP or HL@MP Data Headers Motion Vectors Chroma and Luma DCT Coefficients Variable Length Codes Transport Layer MPEG-2 packets Video packet Audio packet Aux data Packet Headers Flexible delivery of data

149. 148 Digital Television Encode Layers Delivery System Bouquet Multiplexer Program 2Program 3 Service Mux Other Data Control Data Program 1 Multiplexer MPEG Transport Stream Mux Control Data Picture Coding Audio Coding Data Coding MPEG-2 or AC-3 MPEG-2 Control Data VideoDataSound Modulator & Transmitter Error Protection Control Data 188 byte packetsMPEG Transport Data Stream

150. 149 Digital Television Decode Layers Audio Decoder Data Decoder Picture Decoder MPEG or AC-3 MPEG-2 Demodulator & Receiver Error Control Delivery System Data Mon Speakers MPEG Transport Stream De-Multiplexer MPEG DeMux Transport Stream

151. 150 Set top Box (STB) - Interfacing n Domestic and Professional interfaces still to be defined n Transport Stream via IEEE 1394 (Firewire) n Baseband Audio & RGB/YUV Video signals. n STB can convert between line standards so you do not have to have a HD display. n Display and transmitted information must be at same Frame/Field rate. (25/50)

152. 151 DTTB - Content & Services n DTTB was designed to carry video, audio and program data for television n DTTB can carry much more than just TV u Electronic program guide, teletext u Broadband multimedia data, news, weather u Best of internet service u Interactive services u Software updates, games n Services can be dynamically reconfigured

153. 152 DVB Data Containers n MPEG Transport Stream is used to provide DVB “data containers” which may contain a flexible mixture of: u Video u Audio u Data services n Streams with variable data rate requirements can be Statistically Multiplexed together. u Allows Six 2 Mb/s programs to be placed in a 8 Mb/s channel

154. 153 Examples of DVB Data Containers Single HDTV program HDTV 1 SDTV 1 SDTV 2 SDTV 3 SDTV 4 SDTV 5 Multiple SDTV programs SDTV 1 HDTV 1 Simulcast HDTV & SDTV Channel bandwidth can be used in different ways:

155. 154 Video Program Capacity è 1 HDTV service - sport & high action è 2 HDTV services - both film material è 1 HDTV + 1 or 2 SDTV non action/sport è 3 SDTV for high action & sport video è 6 SDTV for film, news & soap operas However you do not get more for nothing. n More services means less quality For a payload of around 19 Mb/s

156. 155 Fixed Bit Rate Multiplexing n Most early digital services used fixed data rates for each of the component streams. n The fixed rate had to allow for a high Quality of Service for demanding material. n Fixed Data Rate was set to a high value for QoS n Less demanding material is sent at a higher quality level. n Works well with systems having similar material on the transport channels.

157. 156 Spare Data Capacity n Spare data capacity is available even on a fully loaded channel. n Opportunistic use of spare data capacity when available can provide other non real time data services. n Example: 51 second BMW commercial The Commercial was shown using 1080 Lines Interlaced. 60 Mb of data was transferred during it. In the Final 3 seconds the BMW Logo was displayed allowing 3 Phone Books of data to be transmitted.

158. 157 Statistical Multiplexing - 1 n Increases efficiency of a multi-channel digital television transmission multiplex by varying the bit-rate of each of its channels to take only that share of the total multiplex bit-rate it needs at any one time. n The share apportioned to each channel is predicted statistically with reference to its current and recent-past demands. n Data rate control fed back to the encoders from the multiplexer.

159. 158 Statistical Multiplexing - 2 n More demanding material can request a higher data rate to maintain Quality of Service. n More channels can be multiplexed together than an equivalent fixed rate system. n Relies on demand peaks on only a few channels while other channels idle at a lower demand.