Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and.

Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and its Application in Consumer Products

Alain Bouffioux - Philips Leuven2A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Agenda Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio compression Video compression Audio/Video synchronisation The MPEG model and its situation in a communication context Application to DVD (Digital Versatile Disc) Application to DVB (Digital Video Broadcasting) Conclusion

Alain Bouffioux - Philips Leuven3A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio compression Video compression Audio/Video synchronisation The MPEG model and its situation in a communication context Application to DVD (Digital Versatile Disc) Application to DVB (Digital Video Broadcasting) Conclusion Agenda

Alain Bouffioux - Philips Leuven4A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Moore’s law Number of transistors per square inch doubles every 18 months

Alain Bouffioux - Philips Leuven5A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Moore’s law today Cost of a transistor divided by one million in 30 years

Alain Bouffioux - Philips Leuven6A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Moore’s law today (2) “Self-fulfulling prophecy” (not automatic) = roadmap for the semiconductor industry

Alain Bouffioux - Philips Leuven7A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Moore’s law today (3) Roadmap for semiconductor industry = only certainty in the current undefined future Moore’s law will continue to apply: 20 years –Economical limitation ? –Power consumption (Moore’s low in reverse direction) –Architectural gap between IP-blocks & application (middleware still more complex…) Progresses in semiconductors = fuel the innovation = fuel the software revolution = fuel the wireless revolution (WLAN, WPAN, WBAN, …) … Examples: WBAN & sensors, RFID applications, camera to swallow, flexible display…

Alain Bouffioux - Philips Leuven8A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The evolution of some CE products (1)Consumer Computer Communication CD DCC CD-i DVD STB

Alain Bouffioux - Philips Leuven9A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The evolution of our CE products (2) The Residential Gateway (Set-Top-Box) as the link between the home and the world-wide information infrastructure. STB Home Network World-wide communication infrastructure

Alain Bouffioux - Philips Leuven10A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The evolution of our CE products (3) The STB (in home) as the gateway to various services. Local Server provides 2 kind of services: –Broadcast Analogue & digital TV, NVOD, PPV –Point-to-point (Home to local server) Home shopping, VOD, e-mail, Web browsing, PC connection... Network Local server Internet Local server Up to 800 homes

Alain Bouffioux - Philips Leuven11A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The evolution of our CE products (4) The STB as a key element of the home network Home Network Residential Gateway To telephone Network To satellite Network To cable Network Computer DVD Jukebox Television Disk Recorder

Alain Bouffioux - Philips Leuven12A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The evolution of our CE products (5) 3C Convergence - Progressive New products combine all 3 functions Products always more and more complex Products have always new features Lifetime of products is always shorter

Alain Bouffioux - Philips Leuven13A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Compression is one among the various factors (all powered by semiconductor progresses) that enable multimedia. Factors enabling such evolution

Alain Bouffioux - Philips Leuven14A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 BUT !! Convergence of technologies (consumer, communication, computer) All products combine all three technologies BUT ! Divergence of applications –Home consumer, Multimedia phone, Camera, PDA, Office computer, Automotive… –High number of potential products Technology push  Market pull (user centric approach)

Alain Bouffioux - Philips Leuven16A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Compression in first A/V Products (1) First Audio/Video products made compression without knowing it was compression. How ? By removal of irrelevancies Audio and Video characteristics

Alain Bouffioux - Philips Leuven17A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Compression in first A/V Products (2) Audio products From 2 to 7.1 channels are enough to provide the spatial resolution. Video products Three colours (RGB) are enough to provide the spectral resolution.

Alain Bouffioux - Philips Leuven18A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio: Compression needed in spectral domain Bitrate of a stereo audio source (CD-DA encoding) Sampling frequency : 44.1 kHz Stereo 16-bit per sample Bitrate = 44100 * 2 * 16 = 1.41 Mbit/sec The need for more compression (1/5)

Alain Bouffioux - Philips Leuven19A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video: Compression needed in spatial domain Bitrate of a video source (CCIR 601 - 50 Hz countries) 25 images per second YUV coding (Y: luminance - U,V : Chrominance) Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixel Bitrate = (576*720)*25*16 = 166 Mbit/sec The need for more compression (2/5)

Alain Bouffioux - Philips Leuven20A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The need for more compression (3/5) Channels availables for AV transmission –Analog television channel (compatibility) Cable (bandwidth = 8 MHz) Satellite (Bandwidth = 30-40 MHz)  Capacity around 40 Mbit/sec –Compact disc (CD) For 74 min. play time : 1.41 Mbit/sec

Alain Bouffioux - Philips Leuven21A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The need for more compression (4/5) MPEG-1 target (Moving Picture Expert Group) (Video-CD : 74 min. constraints) But quality was judged too poor (about VHS quality)

Alain Bouffioux - Philips Leuven22A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The need for more compression (5/5) MPEG-2 target –Program stream (DVD) –Transport stream (DVB)

Alain Bouffioux - Philips Leuven23A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Principles of compression (1/2) Compression (or source coding) is achieved by suppressing information : –redundant information –irrelevant information Suppression of redundant information  lossless compression example: PCM to DPCM,DCT The original signal and the one obtained after encoding and decoding are identical

Alain Bouffioux - Philips Leuven24A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Principles of compression (2/2) Suppression of irrelevant information  lossy compression Example: bandwidth limitation, masking in audio The original signal and the one obtained after encoding and decoding are different but are perceived as identical

Alain Bouffioux - Philips Leuven25A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio Demonstration From “Borderline” Madonna - Stereo - 16 bit/channel Compression used AAC Compression Decompression Original - 32 kbps 128 kbps 64 kbps 16 kbps 705 kbps

Alain Bouffioux - Philips Leuven26A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MOS scale (1/2) Signal distortion is not a good measure of the performance of a lossy compression method  an other method is necessary: MOS scale (Mean Opinion Score) The five-grade CCIR impairment scale (Rec.562) 1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but not annoying), 5(Imperceptible) Example:Double blind test

Alain Bouffioux - Philips Leuven27A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MOS scale (2/2)

Alain Bouffioux - Philips Leuven28A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Compression to VBR or CBR CBR (Constant Bit Rate) vs VBR (Variable Bit Rate) Scene more complex  Higher bit rate for same quality CBR  variable quality (example : Video CD artefact) Constant quality  VBR necessary (e.g.: DVD-Video)

Alain Bouffioux - Philips Leuven29A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video demonstration

Alain Bouffioux - Philips Leuven30A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The compression trade-off Compression techniques are still making progress Trade-off Complexity/Quality/Bit Rate New technique may result in new trade-off Quality Bitrate Complexity MPEG Layer 1 MPEG Layer 2 MPEG Layer 3 MPEG AAC Other Technique Speech coding

Alain Bouffioux - Philips Leuven32A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio compression in MPEG (1/5) Based on psycho-acoustics Compress the bit rate without affecting the quality perceived by the human ears (based on the imperfection of human ears) Removal of irrelevancies 4 main principles : –Threshold of audibility –Frequency masking –Critical bands –Temporal masking

Alain Bouffioux - Philips Leuven33A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio compression in MPEG (2/5) Principle 1: Threshold of audibility  Not all frequency components need to be encoded with the same resolution. Nr_bit(f) = (signal/threshold) db /6

Alain Bouffioux - Philips Leuven34A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio compression in MPEG (3/5) Principle 2: Frequency masking  Analysis of the incoming signal

Alain Bouffioux - Philips Leuven35A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio compression in MPEG (4/5) Principle 3: Critical bands –Human ear may be modelled as a collection of narrow band filters –Bandwidth of these filters = critical band –critical band (<100 Hz) for lowest audible frequencies (  4 kHz) for highest audible frequencies –The human ear cannot distinguish between two sounds having two different frequencies in a critical band. Example : when we hear 50 & 60 Hz at the same time we cannot distinguish them. –Consequence : Noise masking threshold depends solely of the signal energy within a limited bandwidth domain. The largest sound is taken as the representative of the critical band. Necessity to analyse the signal at 100Hz resolution at low- frequency

Alain Bouffioux - Philips Leuven36A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio compression in MPEG (5/5) Principle 4: temporal masking  selection of the frame duration for frequency analysis and encoding.

Alain Bouffioux - Philips Leuven37A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 An enabling tool : the filter bank (1/2)

Alain Bouffioux - Philips Leuven38A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 An enabling tool : the filter bank (2/2) After decimation, same bit rate as original signal, but signal decomposed in various frequency ranges  possibility of frequency based compression Filter-bank: Aliasing occurs due to decimation It exists a class of filter-bank such that aliasing is compensated in synthesis filter : QMF (Quadrature Mirror Filter) but high complexity Pseudo-QMF (Polyphase filter bank) is used. Has good compromise between computation cost and performances Remark : Aliasing may occur if signal in a adjacent band is not reconstructed with an adequate resolution.

Alain Bouffioux - Philips Leuven39A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The MPEG encoder

Alain Bouffioux - Philips Leuven40A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The MPEG filter bank In MPEG, 32 equal-width subbands are used For each subband, necessity to define the maximum signal level and the minimum mask level. BUT, at low frequencies : bandwidth of subbands > critical bands  Necessity to rely on an FFT in order to compensate the lack of frequency selectivity of filterbank at low frequencies

Alain Bouffioux - Philips Leuven41A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Psychoacoustic model & Bit allocation(1/2) An FFT compensates the lack of frequency selectivity of filterbank at low frequencies FFT : 512 samples (layer 1) & 1024 samples (layer 2) resolution for layer 1 : Fs/512 < 100 Hz A psychoacoustic model based on the FFT computes the signal to mask ratio for each subband (1 bit = 6db) Ideally, after allocation, quantisation noise < masking level The scale factors are computed for each subband from the filterbank output (floating point representation of samples) The bit allocator adjust the bit allocation in order to meet the bitrate requirement. The bitstream syntax is dependent of the MPEG layer (See later)

Alain Bouffioux - Philips Leuven42A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Psychoacoustic model & Bit allocation(2/2)

Alain Bouffioux - Philips Leuven43A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The MPEG decoder Decoder is simple (Complexity is at encoder side) Remark 1: DCC is MPEG-1 but DCC encoder has no FFT, relies only on power in the 32 subbands  Higher bit rate (320 kbps) to reach transparent quality Remark 2: MPEG specifies bitstream syntax only. Encoder are given for information. Possibility of improvement.

Alain Bouffioux - Philips Leuven44A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio features in MPEG MPEG1 : –Mono/stereo/dual/joint stereo (Possibility Dolby surround) –Sampling frequencies : 32, 44.1 & 48 kHz –3 layers : trade-off complexity/delay versus coding efficiency of compression –Various bit rate : trade-off quality versus bitrate MPEG2 : –5.1 channels –Sampling frequencies extended to 16, 22.05 & 24 kHz

Alain Bouffioux - Philips Leuven45A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Dolby surround principles (1/5) 4 channels carried by stereo pair  same tools as for stereo Compatible with stereo installation

Alain Bouffioux - Philips Leuven46A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Dolby surround principles (2/5)

Alain Bouffioux - Philips Leuven47A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Dolby surround principles (3/5) Simple decoder provides only 3 dB channel separation (See previous equations)  Need for improvement  Dolby Surround pro-logic decoder (next slide)

Alain Bouffioux - Philips Leuven48A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Dolby surround principles (4/5) Dolby surround pro-logic decoder

Alain Bouffioux - Philips Leuven49A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Dolby surround principles (5/5) Performance of Dolby pro-logic decoder Channel separation larger than 35 dB

Alain Bouffioux - Philips Leuven50A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 5.1 surround sound MPEG-2 surround configurations (front/back) 3/2 3/0 + 2/0 3/1 2/2 2/0 + 2/0 3/0 2/1 2/0 1/0 + LFE (opt.) (Fs/96) 15-120Hz

Alain Bouffioux - Philips Leuven51A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Virtualisation Virtualisation has no direct relation with the MPEG standard. It is considered here only because it may be implemented in some of the future audio products (DVD, STB...) Virtualisation is a product feature. It allows reproduction of surround information (5.1, 3/1) on a stereo installation.

Alain Bouffioux - Philips Leuven52A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Virtualisation principle Virtualisation = processing of the signal in such a way the source of the signal is perceived at a selected position outside the loudspeaker axis (virtual loudspeaker). Drawback : very sensitive to listener position (stability) Remark : a mono signal coded in normal stereo is perceived between the two loudspeakers

Alain Bouffioux - Philips Leuven53A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Stereo widening Also called Q-sound , incredible sound, azimuth positionning... The stereo sources are positionned at virtual locations for improving the stereo effect (cheap analog solution exists) Real sound comes from real loudspeakers. Perceived sound is as if stereo signals were coming from virtual loudspeakers

Alain Bouffioux - Philips Leuven54A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Virtual surround Virtual surround gives on a stereo installation the subjective effect of a multichannel configuration. Each channels is virtually positionned at a location around the listener. The stereo installation performs the addition of the processed signals for each audio channel. Real sound comes from a stereo installation. Perceived sound is as if the various surround signals were coming from some virtually located loudspeakers.

Alain Bouffioux - Philips Leuven55A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Summary of surround aspects Remarks about Dolby surround pro-logic : Only carrier is stereo, source & presentation are multichannel Compatible with stereo installation (no surround effect except in the case of surround virtualisation)

Alain Bouffioux - Philips Leuven57A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (1/6) Principles –removal of intra-picture redundancy : Image is decomposed in 8*8 pixels sub-images. Each sub-image contains redundant information DCT transformation (in frequency domain) decorrelates the input signal.( most energy in low spatial frequencies) –removal of interpicture redundancy : coding of difference with an interpolated picture (moving vectors) –high frequent spatial frequencies quantized with lower resolution than low ones(remove irrelevancy) –zig-zag scan and VLC (remove redundancy)

Alain Bouffioux - Philips Leuven58A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (2/6) Result –4:2:2 CCIR 601 resolution : 166 Mbps (=25images/sec *576lines* 720pixels* 2(lum & chrom) *8bits)  ± 3-4 Mbps (mean) in MPEG2 –4:2:0 SIF resolution : 30 Mbps (=25 images/sec *288 lines *352pixels* 1.5(lum & chrom) *8bits)  ±1.2 Mbps (CBR) in video CD (MPEG1)

Alain Bouffioux - Philips Leuven59A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (3/6) Spatial redundancy reduction (DCT example)

Alain Bouffioux - Philips Leuven60A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (4/6) Temporal redundancy reduction

Alain Bouffioux - Philips Leuven61A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (5/6) Model of a possible encoder

Alain Bouffioux - Philips Leuven62A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Video compression in MPEG (6/6) MPEG1 en MPEG2 video features –MPEG1 sequential picture resolution : SIF format 288(240)*356*24,25 or 30 Hz –MPEG2 sequential or interlaced various levels : low level (SIF: 288*356), main level (CCIR601: 576 * 720), high 1440 level (HDTV: 1152*1440), high level (EQTV: 1152*1920) various profiles (toolboxes) : simple profile (No B picture), main profile (=MPEG1+interlaced), SNR scalable profile (allows graceful degradation (noise improvement at same resolution), spatial scalable profile (hierarchical coding : improvement at higher resolution), high profile.

Alain Bouffioux - Philips Leuven64A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Synchronisation Synchronisation in the multimedia context refers to the mechanism that ensures a temporal consistent presentation of the audio-visual information to the user

Alain Bouffioux - Philips Leuven65A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Intramedia synchronisation  T between capture & presentation = Constant  Same clock frequency & Data on time  Need for corresponding tools

Alain Bouffioux - Philips Leuven66A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Intermedia synchronisation  T_Audio =  T_Video  Sampled at the same time  Presented at the same time)  Possible tools : common time base and presentation control (media synchronisation with the common time base) Ex.: Lip_sync (requirement: |delay_difference| < 80msec)

Alain Bouffioux - Philips Leuven67A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Recovery of clock in CBR CBR = Constant Bit Rate if the clock to recover is synchronous with transport clock  Recovery of clock but not of common time base Remark : possibility to slave DSM to local clock

Alain Bouffioux - Philips Leuven68A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Recovery of clock and time base in VBR VBR = Variable Bit Rate Need for insertion of time stamps (OUTPUT TIME) Output time stamp says for example : “It is now 16h25” Receiver adjusts its own horloge to the received time stamp Recovery of clock & of common time base

Alain Bouffioux - Philips Leuven69A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Synchronisation with common time base Insertion of time stamp (=INPUT TIME) Input time stamp says : “Input has been sampled at 16h29”. Receiver presents the sample at (its input time stamp + maximum encoding and decoding delay). Alternative: transmission of presentation time stamp (input time+delay)

Alain Bouffioux - Philips Leuven70A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Getting data on time “On time”  Not too late, not too early No buffer over- or underflow Flow control : not applicable in broadcasting Common time base and Definition of a standard target decoder that describes the data consumption pattern of the receiver. Remark: Direct MPEG (Microsoft) does not use time information for clock recovery but relies on flow control

Alain Bouffioux - Philips Leuven71A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Streams Idea of continuity (pipelining) Carry time information for clock recovery No flow control (allows broadcasting) The emitter must have a precise knowledge of the receiver data consumption pattern (explicit in MPEG STD) Just-in-time Shorter delay and smaller buffer size than with flow control Two aspects in synchronisation : Clock recovery & timing control (model & buffering)

Alain Bouffioux - Philips Leuven72A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Requirement on for stream transport Data information  BER (Bit Error Rate) requirement No repetition of frame possible  FEC (Forward Error Correction) Time information  No jitter

Alain Bouffioux - Philips Leuven74A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 What is MPEG ? (1/2) Moving Picture Expert Group Still active (MPEG-21 is currently in development) International standard (ISO/IEC)  Interoperability & economy of scale Compression of audio and video and multiplexing in a single stream Definition of the interface not of the codecs  room for improvement MPEG-1 : until 1.5 Mbps, for DSM Progressive picture, stereo (Dolby surround)

Alain Bouffioux - Philips Leuven75A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 What is MPEG ? (2/2) MPEG-2 : Various bit rates (CBR & VBR) Program stream for DSM, transport stream for network Interlaced picture, 5.1 audio channels Definition of various video levels (e.g. CCIR601 resolution: 4-9 Mbps, HDTV:15-25 Mbps) and profiles MPEG-3 : Cancelled, integrated in MPEG-2 (Initially : for HDTV) MPEG-4 : standard for audio, video and graphics in interactive 2D and 3D multimedia communication. (Initially : low bit rate for real-time personal communication) MPEG-7 : Multimedia contents description interface MPEG-21 : Focus on multimedia distribution and on DRM aspects.

Alain Bouffioux - Philips Leuven76A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The MPEG model (1/2)

Alain Bouffioux - Philips Leuven77A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The MPEG model (2/2) Compression of audio & video and multiplexing in a single stream Guarantees intramedia and intermedia synchronisation. MPEG defines an interface –bitstream syntax –timing of the bitstream  STD specifying timing requirement (ideal model) Consequences: –Decoder should compensate deviations from STD –Network should correct jitter introduced by the channel (RTD- LJ) MPEG stream must be adapted to transmission channel formatting, error correction, channel coding (b.v.video-CD)

Alain Bouffioux - Philips Leuven78A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Components of the MPEG standard The MPEG standard is composed of 3 main parts : –Audio : Specifies the compression of audio signals –Video : Specifies the compression of video signals –System : specifies how the compressed audio and video signals are combined in the multiplexed stream (program stream or transport stream). Each part specifies : –The bitstream syntax –The timing requirement and the related information (bit rate, buffer needs)

Alain Bouffioux - Philips Leuven79A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Synchronisation Mechanism (1/2)

Alain Bouffioux - Philips Leuven80A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Synchronisation Mechanism (2/2) PCR for TS & SCR for PS (but same concept) Clock & time base recovery: Time-stamping at OUTPUT (PCR included in TS multiplex, SCR in pack header) Audio & video clock locked to STC  easy recovery (see next slide) Synchronisation of audio & video to common time base (Time stamping at Input) STD is defined (because of the absence of flow control) streams are such that STD buffers never over- or underflow In TS, many program in a single stream but unique clock per program. Time information  “No Jitter” requirement for transport

Alain Bouffioux - Philips Leuven81A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Clock recovery in receiver

Alain Bouffioux - Philips Leuven82A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MPEG program & transport streams Program streams: –Relatively error free environment –program stream packet may have variable and great length –Single time base Transport streams: –environment where errors are likely –many programs (independent time base) –Transport stream packet : fixed, 188 bytes –Contains tables

Alain Bouffioux - Philips Leuven83A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MPEG in a communication context (1) “Typical” communication system

Alain Bouffioux - Philips Leuven84A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MPEG in a communication context (2) MPEG : Source coding only (bit rate reduction) + multiplexing The MPEG stream must be adapted to the channel in what concern its physical characteristics and in order to get the required QoS (Quality of Service) & Security –Encryption –Channel coding (forward error correction, interleaving, modulation codes) –multiplexing & formatting –modulation (frequency allocation) –multiple access method Some channels : CD/DVD - satellite - cable - ATM - 1394

Alain Bouffioux - Philips Leuven85A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 MPEG in a communication context (3) A simple view of MPEG in the communication context

Alain Bouffioux - Philips Leuven86A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio compression Video compression Audio/Video synchronisation The MPEG model and its situation in a communication context Application to DVD (Digital Versatile Disc) Application to DVB (Digital Video Broadcasting) Conclusion Agenda

Alain Bouffioux - Philips Leuven87A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CD : Some concepts Hard disk vs compact disc : more differences than just storage technique. HD developed for data storage and recording, CD developed for stream storage (CD-DA)  their basic differences Questions –track form? –read direction? Why? –CAV or CLV? Why? –Access time : CD-ROM vs HD? –Data storage: on which face? –Production method? –Capacity? –Sensitivity to error? Diameter of a possible hole? –Standard = Interface definition : CD vs HD ?

Alain Bouffioux - Philips Leuven88A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CD-DA: Encoder model (1/3)

Alain Bouffioux - Philips Leuven89A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CD-DA: Encoder model (2/3) The CD-DA physical layer adapts the input stream (audio) to the requirements of the channel –Modulation : EFM (Eight to fourteen modulation + 3 merging bits) Pit & land length (number of successive 0 or 1 as written to disc): between 3 and 11 channel bits DC free code for adaptation to the channel bandwidth & for clock recovery considerations. –Error correction (Cross-interleaved Reed-Solomon code) Interleave placed between C1 & C2 ECC. Next slide presents only principles and not real CD implementation.

Alain Bouffioux - Philips Leuven90A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 –Error correction : addition of redundancy in order to be able to correct errors (e.g. RS(28,24,5)*RS(32,28,5)) Principle : –Interleaving : time diversity in order to deal with error burst. Successive erroneous channel bits (burst error) do not damage the same Reed-Solomon table. CD-DA: Encoder model (3/3)

Alain Bouffioux - Philips Leuven91A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CD-ROM encoder model

Alain Bouffioux - Philips Leuven92A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 From CD to DVD : the motivation Motivation = increase the capacity Why ? - Requirement of the motion picture industry –Playback time : more than 135 min. (duration of 90% of films) –Picture quality : superior to laser disc –Audio quality : 5.1 channels surround –Language/subtitles : 3 languages minimum.  capacity needs : more than 4.7 Gbytes Where ? - In physical layer DVD : developed specifically for audio/video (  video CD).

Alain Bouffioux - Philips Leuven93A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The DVD physical layer (1/2)

Alain Bouffioux - Philips Leuven94A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Objective was the storage of 2K sectors Error Correction Code (Reed-Solomon) - add redundancy Modulation - time diversity (Number of consecutive 0 : between 2 and 10) Pit and land length : between 3 and 11 (Idem CD) Synchronisation : for sector reconstruction. The DVD physical layer (2/2)

Alain Bouffioux - Philips Leuven95A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 DVD: the capacity improvement (1/4) Increase of channel bit density ( gain = 4.50) Min pit length : (0.83   0.4  ) Track pitch : (1.6   0.74  ) Diameter of laser spot (  wavelength/NA) Wavelength (780   640 nm)  gain = 1.5 NA (0.45  0.60)  gain = 1.78 reduced margin  gain = 1.68 Modulation: EFM (8 to 17 bit)  8 to 16  gain = 1.06 Error correction RS(32,28,5)*RS(28,24,5)  RS(182,172,11)*RS(208,192,17)  gain = 1,16

Alain Bouffioux - Philips Leuven96A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 DVD: the capacity improvement (2/4) No subcode  gain = 1.03 Sync pattern  gain = 1.03 Better sector formatting sector length (2352 bytes  2064)  gain = 1.14 Other (e.g. recorded area)  gain = 1.07 Total gain : 7.2 Capacity per side : 650 MBytes (mode 1)  4.7 Gbytes

Alain Bouffioux - Philips Leuven97A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 DVD: the capacity improvement (3/4)

Alain Bouffioux - Philips Leuven98A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 DVD: the capacity improvement (4/4) Capacity of the various types Single-layer single-side4.7 Gbytes Dual-layer single-side8.5 Gbytes Single-layer double-side9.4 Gbytes Dual-layer double-side17 Gbytes

Alain Bouffioux - Philips Leuven99A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The 3 components of the DVD-V standard DVD = DVD (= 3 random letters) (previously : Digital Versatile Disc, Digital Video Disc) DVD-V : DVD - Video

Alain Bouffioux - Philips Leuven100A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some DVD-V features (1/2) Presentation data = MPEG program stream, VBR, max peak bit rate = 10.08 Mbps) Video data1 streamMpeg1 Mpeg2 (ML@MP) 16:9 or 4:3 aspect ratio NTSC or PAL Audio datamax 8 streamsMpeg2 + 7.1 extension (50 Hz countries) AC-3 (60 Hz countries) Linear PCM (incl. 96 kHz - 24 bits) Sub picture datamax 32 streamsRun length encoded(subtitles) Bit map

Alain Bouffioux - Philips Leuven101A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some DVD-V features (2/2) Seamless playback Language parental lock Multi-angle camera Still picture Regional coding (6 regions) System menu Audio stream selection Subtitle selection Angle selection Encryption Decryption key hidden on the disc.

Alain Bouffioux - Philips Leuven102A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The DVD family of products

Alain Bouffioux - Philips Leuven103A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Recording on disk - principle Products: CD-R, CD-RW, DVD+/-R(W) … CD principle: reflectivity of pits & lands are different. Pits and lands are used to store 0 and 1. CD-RW principle: reflectivity of the two phases of the recording material (amorphous, crystalline) are different. Controlling the phase allows storage of 0 or 1. To Amorphous state (low reflectivity): T above melting point (600°C) & fast cooling To Crystalline state (high reflectivity): T above 200°C for a sufficient time Recording: by the laser heating the recording layer Reading: by laser as for CD (-> compatibility)

Alain Bouffioux - Philips Leuven104A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Blu-Ray DVD

Alain Bouffioux - Philips Leuven105A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 SACD = Super-Audio-CD = Response of Philips/Sony on the thread DVD-Audio brings on the revenues from CD portfolio Multi-layer hybrid scheme –One layer for playback in CD player at standard quality –One layer for playback in SACD player at enhanced quality (DVD-like, 4.38 Gbytes) Already on the market and in consumer homes (marketing argument) DSD technology (Direct Stream Digital) delta sigma DAC to decode the 2.82 Mbps PDM stream Lossless compression, 5.1 multichannel, encrypted 120 db (=20-bit), 100 kHz BW

Alain Bouffioux - Philips Leuven107A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Adaptation to the DVB channel Channel coding : transforms the TS in an other sequence of bits containing the same information than the input stream but more robust against the imperfections of the transmission on the physical channel cost : a higher bit rate Modulation : transforms an input sequence to an analog waveform for transmission over the physical channel

Alain Bouffioux - Philips Leuven108A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Channel coding (1/3) Unlike source coding that removes redundancy, channel coding adds redundancy in a structured way so that the decoder be able to detect and/or correct the errors introduced by the physical channel.

Alain Bouffioux - Philips Leuven109A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Channel coding (2/3) Channel coding may include : –Spectral modification of the signal for adaptation to the channel (e.g. remove DC, spectrum shaping like uniform distribution in the frequency space...) –FEC : Forward Error Correction Addition of redundancy in order to allow error detection and/or correction (example : The total of bought articles is similar to a parity byte)

Alain Bouffioux - Philips Leuven110A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Channel coding (3/3) –Interleaving Time diversity in order to deal with error bursts. The successive bytes of information are dispersed in time on the transmission channel in such a way that an error burst does not affect neighbouring bytes. Interleaving is often combined with FEC so that error bursts could be corrected by the FEC. Example :

Alain Bouffioux - Philips Leuven111A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Modulation in DVB (1/3) Different modulation techniques : –Cable : QAM –Satellite : QPSK –Terrestrial : OFDM Why ? Modulation technique depends on : –Physical characteristics of the channel –Compatibility constraints with actual analog transmission

Alain Bouffioux - Philips Leuven112A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Modulation in DVB (2/3) Example : influence of SNR on modulation technique selected  QPSK for satellite and QAM for cable

Alain Bouffioux - Philips Leuven113A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Modulation in DVB (3/3) Satellite Bandwidth : generally 27-36 MHz SNR low : about 10 db (power transmitted by satellite) direct path Cable Bandwidth : 8 MHz (50Hz countries) - 6 MHz (60Hz countries) SNR strong (about 25 db) Echoes from impedance mismatch in the network Terrestrial Bandwidth : idem as cable Multipath interference, signal level variation,...

Alain Bouffioux - Philips Leuven114A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 From TS to the DVB channel Some blocks are identical for all standards (Cable, Satellite & Terrestrial) Inner & outer : terminology is derived from the view of the quasi- error-free channel composed of a transmitter and a receiver. Satellite & Terrestrial : More sensitive to error  inner coder is added

Alain Bouffioux - Philips Leuven116A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Questions ?

Alain Bouffioux - Philips Leuven117A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 APPENDICES

Alain Bouffioux - Philips Leuven118A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Agenda Conditional access What is cryptography Symmetric & public-key cryptography Why cryptography for DVB ? Conditional access information in MPEG/DVB Conditional access mechanism Conditional access interfaces

Alain Bouffioux - Philips Leuven119A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Why cryptography ? –CONFIDENTIALITY - The message is not listened –INTEGRITY - The message is not modified –AUTHENTICITY - The message has been sent by Alice –NON-REPUDIATION - Alice cannot falsely deny she has sent the message What is cryptography (1/2)

Alain Bouffioux - Philips Leuven120A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 What is cryptography (2/2) Basic terminology

Alain Bouffioux - Philips Leuven121A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Symmetric cryptography Public-key cryptography Key1 = Key2Key 1  Key 2 Public-key cryptography One Public-key (known by everybody) : PK One Private-key or Secret-key (kept secret) : SK C = E Key1 (M)  M = D Key2 (C) = D Key2 (E Key1 (M)) In public-key cryptography, key1 may be PK or SK and key2 is the other key. Symmetric & public-key cryptography(1)

Alain Bouffioux - Philips Leuven122A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Symmetric & public-key cryptography(2) Example of symmetric cryptography –Key stream as long as message –Key stream = pseudo-random sequence (easy to break) –Low security should be compensated by frequent change of keys  necessity of secure channel  2 channels : one for the message & one for the key

Alain Bouffioux - Philips Leuven123A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Symmetric & public-key cryptography(3) Example of public-key cryptography

Alain Bouffioux - Philips Leuven124A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Symmetric & public-key cryptography(4) Symmetric cryptography example : DES Public-key cryptography example : RSA (1977) Symmetric versus public-key cryptography –Symmetric cryptography is faster (about 1000 times). –Low security of symmetric cryptography (due to the necessity of key transport) is improved by a frequent change of the key. –In Public-key cryptography the secret-key may be kept secret. It is never transported  High security. –Different usage : In DVB, symmetric key algorithm for encrypting data, public-key algorithm for key management (secure channel). Hybrid cryptosystem Example : DES for message and RSA for key encryption

Alain Bouffioux - Philips Leuven125A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Cryptography and DVB (1/2) Cryptography prevents unauthorised receiver from decoding the program. DVB compared with banking or military secret –high information rate –low information value –decryption must be cheap Cost of cracking the system should be higher than the benefits gained from the cracking Cryptography in DVB is a trade-off between cost/complexity versus piracy-proof. CA (Conditional Access) = very sensitive subject. Some service providers want their own CA system.

Alain Bouffioux - Philips Leuven126A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Cryptography and DVB (2/2) MPEG does not specify a conditional access (CA) system but defines a frame to support CA. DVB characterises some aspect left undefined by MPEG, It defines a CA interface. The broadcaster develops its CA system using a CA interface. DVB is based on –symmetric cryptography for audio-visual transmission –frequent key change to increase security –Public-key cryptography for key-exchange DVB relies on –stream of ECM’s (Entitlement Control Message) –stream of EMM’s (Entitlement Management Message)

Alain Bouffioux - Philips Leuven127A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CA information in MPEG TS (1/2)

Alain Bouffioux - Philips Leuven128A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 CA information in MPEG TS (2/2)

Alain Bouffioux - Philips Leuven129A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The CA mechanism : illustration Decryption Encrypted AV dataClear AV Data SMARTCARD Decryption ECM’s (Program related) EMM’s (CA system related) IK Entitlement SK Access control parameters SK CW’s PDK1PDK2 PDK

Alain Bouffioux - Philips Leuven130A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The CA mechanism (1/2) AV streams are scrambled with Control Words (CW) using symmetric cryptography CW are encrypted using Service Keys (SK), are placed in ECM’s and are securely transmitted to the receiver

Alain Bouffioux - Philips Leuven131A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 The CA mechanism (2/2) SK are encrypted using public-key cryptography - Keys are IK (unique key internal to the smartcard) or PDK (transmitted via EMM’s in order to define user’s group) ECM’s carries (informations related to a single program  PID of ECM’s in PMT) –enciphered CW –access parameters ECM’s are decoded to CW if the receiver contains the required entitlements EMM’s carries (information related to a conditional access system  PID of EMM’s in CAT) –New entitlements, SK’s (Service Keys) –Programmer distribution key

Alain Bouffioux - Philips Leuven132A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 About DVB scrambling Encryption occurs after compression (at the location in the stream where the redundancy is at its lowest value) in order to have a robust encryption system. Encryption may occur at PES level or at TS level. DVB scrambling is transparent (a valid TS remains valid after scrambling)  facilitates transport and manipulation. Synchronisation based on PCR’s  constant time required for scrambling/descrambling. Security device should authenticate EMM’s origin. CA is only one aspects of cryptography usage in DVB. An other may be copy protection by (watermarking) and authentication (by signature).

Alain Bouffioux - Philips Leuven133A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Agenda Some video format

Alain Bouffioux - Philips Leuven134A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some video formats (1) Max. component video signal bandwidth: 6 MHz. CCIR601 (CCIR is now ITU-R): Video sampling frequency: 13.5 MHz for 525 & 625 line standards (Shannon requirement) Synchronous with line (& image) sampling frequency F sampling = 864*F h for 625 line system (50Hz countries) F sampling = 858*F h for 525 line system (60Hz countries) Why synchronous? Points at the same place RGB format

Alain Bouffioux - Philips Leuven135A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some video formats (2) YC b C r format C b = B-Y, C r = R-Y Eye is more sensitive to luminance than to chrominance (lower resolution needed for chrominance)

Alain Bouffioux - Philips Leuven136A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some video formats (3) The 4:2:2 format –Y sampling @ 13.5 MHz –C sampling @ 6.75 MHz –8 bits per pixel –720 active points per line –576 lines active lines per image (2 fields) (625 lines) and 480 active lines (525 lines) –Pixels are not square (e.g. for 480 lines, only 640 active points are needed - VGA format) –Image size 720*576 or 720*480 The 4:2:0 format –Vertical luminance resolution reduced by a factor 2 (average on two successive lines)

Alain Bouffioux - Philips Leuven137A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Some video formats (4) SIF format (Source Intermediate Format) Half the vertical & horizontal resolution of 4:2:0 For 50Hz countries: –Luminance: 360*288 –Chrominance: 180*120 CIF format (Common Intermediate Format) –Intermediate format used in videoconferencing (communication between US & Europe) –resolution: 360*288 –Sampling frequency: 30 Hz QCIF (Quarter CIF) –Half the vertical & horizontal resolution of CIF.

Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and.

Similar presentations

Presentation on theme: "Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and.

Similar presentations

Presentation on theme: "Alain Bouffioux - Philips Leuven1A/V compression & Consumer Products - ULG Philips Digital Systems Laboratories 09/12/2004 Audio & Video Compression and."— Presentation transcript:

Similar presentations

About project

Feedback