1. Video Digital Multimedia, 2nd edition Nigel Chapman & Jenny Chapman
Chapter 7
This presentation © 2004, MacAvon Media Productions
Video
Digital Multimedia, 2nd edition
Nigel Chapman & Jenny Chapman
Chapter 7
This presentation © 2004, MacAvon Media Productions

2. Data Size PAL uncompressed 768x576 pixels per frame
191–192
Data Size
PAL uncompressed
768x576 pixels per frame
x 3 bytes per pixel (24 bit colour)
x 25 frames per second
≈ 31 MB per second
≈ 1.85 GB per minute
Data Size
PAL uncompressed
768x576 pixels per frame
x 3 bytes per pixel (24 bit colour)
x 25 frames per second
≈ 31 MB per second
≈ 1.85 GB per minute

3. Data Size NTSC uncompressed 640x480 pixels per frame
191–192
Data Size
NTSC uncompressed
640x480 pixels per frame
x 3 bytes per pixel (24 bit colour)
x 30 frames per second (approx)
≈ 26 MB per second
≈ 1.6 GB per minute
Data Size
NTSC uncompressed
640x480 pixels per frame
x 3 bytes per pixel (24 bit colour)
x 30 frames per second (approx)
≈ 26 MB per second
≈ 1.6 GB per minute

4. Digitization In the camera – DV + Firewire
193–195
Digitization
In the camera – DV + Firewire
In the computer – video capture card
Digitization in camera (DV) means less noise
Less noise allows better compression
Digitization
In the camera – DV + Firewire
In the computer – video capture card
Digitization in camera (DV) means less noise
Less noise allows better compression

5. 197
Streamed Video
Play back a video stream as it arrives over a network (like broadcast TV), instead of downloading an entire video clip and playing it from disk (like renting a DVD)
Streamed Video
Play back a video stream as it arrives over a network (like broadcast TV), instead of downloading an entire video clip and playing it from disk (like renting a DVD)

6. 198
HTTP Streaming
Start playing a downloaded clip as soon as enough of it has arrived
Starts when the (estimated) time to download the rest is equal to the duration of the clip
HTTP Streaming
Start playing a downloaded clip as soon as enough of it has arrived
Starts when the (estimated) time to download the rest is equal to the duration of the clip

7. 198
X axis, Time
Y axis, % remaining. Starting at 100% on Y axis download starts and runs at a 45 degree angle to the finish at 0% at distance along the X axis. A horizontal line parallel to X axis the proceeds from the 100% mark until the start playing time. At which point we have a line descendinfinsh at the same point on the time lineg at an angle of 26 degrees to the vertical to

8. Video Standards Digital video devices must conform to standards
199
Video Standards
Digital video devices must conform to standards
Digital standards must maintain compatibility with older analogue standards for broadcast TV
Video Standards
Digital video devices must conform to standards
Digital standards must maintain compatibility with older analogue standards for broadcast TV

9. Interlacing Required for TV, so encountered in captured footage
200
Interlacing
Required for TV, so encountered in captured footage
Each frame is divided into two fields
Field 1: odd lines; Field 2: even lines
Fields are transmitted one after the other
Frame is built out of the interlaced fields
Interlacing
Required for TV, so encountered in captured footage
Each frame is divided into two fields
Field 1: odd lines; Field 2: even lines
Fields are transmitted one after the other
Frame is built out of the interlaced fields

10. Analogue Standards PAL (Phase Alternating Line)
199
Analogue Standards
PAL (Phase Alternating Line)
Western Europe, Australia & New Zealand, China,…
NTSC (National Television Standards Committee)
North America, Japan, Taiwan, parts of South America,…
Analogue Standards
PAL (Phase Alternating Line)
Western Europe, Australia & New Zealand, China,…
NTSC (National Television Standards Committee)
North America, Japan, Taiwan, parts of South America,…

11. Analogue Standards SECAM (Séquential Couleur avec Mémoire)
199
Analogue Standards
SECAM (Séquential Couleur avec Mémoire)
France and former Soviet Union
Standard only used for transmission
Uses PAL cameras etc
Analogue Standards
SECAM (Séquential Couleur avec Mémoire)
France and former Soviet Union
Standard only used for transmission
Uses PAL cameras etc

12. PAL NTSC NTSC Frame has 625 lines, 576 are picture
202
PAL
Frame has 625 lines, 576 are picture
25 frames (50 fields) per second
NTSC
Frame has 525 lines, 480 are picture
29.97 frames (59.94 fields) per second
(Often quoted as 30 frames per second)
PAL
Frame has 625 lines, 576 are picture
25 frames (50 fields) per second
NTSC
Frame has 525 lines, 480 are picture
29.97 frames (59.94 fields) per second
(Often quoted as 30 frames per second)

13. CCIR 601 Digital video standard, properly called Rec. ITU-R BT.601
202–203
CCIR 601
Digital video standard, properly called Rec. ITU-R BT.601
720 luminance samples (Y), 2x360 colour difference samples (B−Y and R−Y) per line
PAL 720x576 pixels; NTSC 720x480 pixels
Pixels are not square
CCIR 601
Digital video standard, properly called Rec. ITU-R BT.601
720 luminance samples (Y), 2x360 colour difference samples (B−Y and R−Y) per line
PAL 720x576 pixels; NTSC 720x480 pixels
Pixels are not square

14. 4:2:2 sub-sampling Used in CCIR 601
203
4:2:2 sub-sampling
Used in CCIR 601
Twice as many Y samples as each of the colour difference samples
Co-sited: same pixel is used for all three samples
Reduces data rate to just over 20MB per second
4:2:2 sub-sampling
Used in CCIR 601
Twice as many Y samples as each of the colour difference samples
Co-sited: same pixel is used for all three samples
Reduces data rate to just over 20MB per second

15. DV Consumer format, also known as mini-DV
204 DV
Consumer format, also known as mini-DV
DVCAM, DVPRO use different tape formats, but generate the same data stream
4:1:1 chrominance sub-sampling
Data rate constant 25Mbits per second
Compression ratio 5:1 DV
Consumer format, also known as mini-DV
DVCAM, DVPRO use different tape formats, but generate the same data stream
4:1:1 chrominance sub-sampling
Data rate constant 25Mbits per second
Compression ratio 5:1

16. DV sub-sampling PAL DV 4:2:0 chrominance sub- sampling 210

17. DV sub-sampling NTSC DV 4:1:1 chrominance sub-sampling 210

18. MPEG ISO/IEC Motion Picture Experts Group
204–206
MPEG
ISO/IEC Motion Picture Experts Group
Series of standards including
MPEG-1 intended for video CD
MPEG-2 used in DVD and broadcast
MPEG-4 for low bitrate multimedia
ISO/IEC Motion Picture Experts Group
Series of standards including
MPEG-1 intended for video CD
MPEG-2 used in DVD and broadcast
MPEG-4 for low bitrate multimedia

19. 204–205
MPEG Profiles & Levels
Profiles define subsets of the features of the data stream
Levels define parameters such as frame size and data rate
Each profile may be implemented at one or more levels
Notation: e.g.
MPEG Profiles & Levels
Profiles define subsets of the features of the data stream
Levels define parameters such as frame size and data rate
Each profile may be implemented at one or more levels
Notation: e.g.

20. MPEG-2 Profiles & Levels
205
MPEG-2 Profiles & Levels
MPEG-2 Main Profile at Main Level used for DVD video
CCIR 601 scanning
4:2:0 chrominance sub-sampling
15 Mbits per second
Most elaborate representation of MPEG-2 compressed data
MPEG-2 Profiles & Levels
MPEG-2 Main Profile at Main Level used for DVD video
CCIR 601 scanning
4:2:0 chrominance sub-sampling
15 Mbits per second
Most elaborate representation of MPEG-2 compressed data

21. 205–206
MPEG-4
Designed to support a range of multimedia data at bit rates from 10kbps to >1.8Mbps
Applications from mobile phones to HDTV
Video codec becoming popular for Internet use, is incorporated in QuickTime, RealMedia and DivX
MPEG-4
Designed to support a range of multimedia data at bit rates from 10kbps to >1.8Mbps
Applications from mobile phones to HDTV
Video codec becoming popular for Internet use, is incorporated in QuickTime, RealMedia and DivX

22. MPEG-4 Profiles & Levels
205
MPEG-4 Profiles & Levels
Visual Simple Profile (SP), suitable for low bandwidth streaming over Internet
Visual Advanced Simple Profile (ASP) suitable for broadband streaming
(Level 1 of Simple Profile), 64 kbps, 176x144 pixel frame
kbps, full CCIR 601 frame
MPEG-4 Profiles & Levels
Visual Simple Profile (SP), suitable for low bandwidth streaming over Internet
Visual Advanced Simple Profile (ASP) suitable for broadband streaming
(Level 1 of Simple Profile), 64 kbps, 176x144 pixel frame
kbps, full CCIR 601 frame

23. Video Compression Spatial (intra-frame) compression
206–208
Video Compression
Spatial (intra-frame) compression
Compress each frame in isolation, treating it as a bitmapped image
Temporal (inter-frame) compression
Compress sequences of frames by only storing differences between them
Always some compression because of sub- sampling
Video Compression
Spatial (intra-frame) compression
Compress each frame in isolation, treating it as a bitmapped image
Temporal (inter-frame) compression
Compress sequences of frames by only storing differences between them
Always some compression because of sub-sampling

24. Spatial Compression Image compression applied to each frame
207
Spatial Compression
Image compression applied to each frame
Can therefore be lossless or lossy, but lossless rarely produces sufficiently high compression ratios for volume of data
Lossless compression implies a loss of quality if decompressed then recompressed
Ideally, work with uncompressed video during post-production
Spatial Compression
Image compression applied to each frame
Can therefore be lossless or lossy, but lossless rarely produces sufficiently high compression ratios for volume of data
Lossless compression implies a loss of quality if decompressed then recompressed
Ideally, work with uncompressed video during post-production

25. Temporal Compression Key frames are spatially compressed only
207–208
Temporal Compression
Key frames are spatially compressed only
Key frames often regularly spaced (e.g. every 12 frames)
Difference frames only store the differences between the frame and the preceding frame or most recent key frame
Difference frames can be efficiently spatially compressed
Temporal Compression
Key frames are spatially compressed only
Key frames often regularly spaced (e.g. every 12 frames)
Difference frames only store the differences between the frame and the preceding frame or most recent key frame
Difference frames can be efficiently spatially compressed

26. Motion JPEG Purely spatial compression Apply JPEG to each frame
209–210
Motion JPEG
Purely spatial compression
Apply JPEG to each frame
Used by most analogue capture cards
No standard, but MJPEG-A format widely supported
Motion JPEG
Purely spatial compression
Apply JPEG to each frame
Used by most analogue capture cards
No standard, but MJPEG-A format widely supported

27. DV Compression Starts with chrominance sub-sampling of CCIR 601 frame
210–211
DV Compression
Starts with chrominance sub-sampling of CCIR 601 frame
Constant data rate 25Mbits per second
Higher quality than MJPEG at same rate
Apply DCT, quantization, run-length and Huffman coding on zig-zag sequence – like JPEG – to 8x8 blocks of pixels
DV Compression
Starts with chrominance sub-sampling of CCIR 601 frame
Constant data rate 25Mbits per second
Higher quality than MJPEG at same rate
Apply DCT, quantization, run-length and Huffman coding on zig-zag sequence – like JPEG – to 8x8 blocks of pixels

28. 210–211
DV Compression
If little or no difference between fields (almost static frame), apply DCT to block containing alternate lines from odd and even fields
If motion between fields, apply DCT to two 8x4 blocks (one from each field) separately, leading to more efficient compression of frames with motion
DV Compression
If little or no difference between fields (almost static frame), apply DCT to block containing alternate lines from odd and even fields
If motion between fields, apply DCT to two 8x4 blocks (one from each field) separately, leading to more efficient compression of frames with motion

29. DV Compression Shuffling
210–211
DV Compression
Shuffling
Construct video segments by taking 8x8 blocks from five different areas of the frame, to ‘average’ amount of detail
Calculate coefficients for whole video segment, making more efficient use of available bytes
DV Compression
Shuffling
Construct video segments by taking 8x8 blocks from five different areas of the frame, to ‘average’ amount of detail
Calculate coefficients for whole video segment, making more efficient use of available bytes

30. 211–212
MPEG-1 Compression
Spatial compression based on quantization and coding of DCT coefficients
Temporal compression based on motion compensation
Record displacement of object plus changed pixels in area exposed by its movement
MPEG-1 Compression
Spatial compression based on quantization and coding of DCT coefficients
Temporal compression based on motion compensation
Record displacement of object plus changed pixels in area exposed by its movement

31. MPEG-1 Compression I-pictures purely intra-frame compressed
212–213
MPEG-1 Compression
I-pictures purely intra-frame compressed
P-pictures ‘predictive’
Difference frames based on earlier I- or P- pictures
B-pictures ‘bi-directionally predictive’
Difference frames based on preceding and following I- or P-pictures
MPEG-1 Compression
I-pictures purely intra-frame compressed
P-pictures ‘predictive’
Difference frames based on earlier I- or P-pictures
B-pictures ‘bi-directionally predictive’
Difference frames based on preceding and following I- or P-pictures

32. MPEG-1 Compression Group of Pictures (GOP)
213–214
MPEG-1 Compression
Group of Pictures (GOP)
Repeating sequence of I-, P- and B-pictures
Always begins with an I-picture
Display order – frames in order they will be displayed
Bitstream order – re-ordered so that every P- or B-picture comes after frames it depends on, allowing reconstruction of the complete frames
MPEG-1 Compression
Group of Pictures (GOP)
Repeating sequence of I-, P- and B-pictures
Always begins with an I-picture
Display order – frames in order they will be displayed
Bitstream order – re-ordered so that every P- or B-picture comes after frames it depends on, allowing reconstruction of the complete frames

33. MPEG-1 Compression Source Input Format (SIF) 4:2:0 chroma sub-sampled
214
MPEG-1 Compression
Source Input Format (SIF)
4:2:0 chroma sub-sampled
352x240 pixel frame
MPEG-1 compressed SIF video at 30 frames per second has data rate of 1.86Mbits per second (CD video – 40mins of video at that rate)
MPEG-1 can be scaled up to larger frames, but cannot handle interlacing
MPEG-1 Compression
Source Input Format (SIF)
4:2:0 chroma sub-sampled
352x240 pixel frame
MPEG-1 compressed SIF video at 30 frames per second has data rate of 1.86Mbits per second (CD video – 40mins of video at that rate)
MPEG-1 can be scaled up to larger frames, but cannot handle interlacing

34. 215
MPEG-4
Standard defines an encoding for multimedia streams made up of different sorts of object – video, still images, animation, 3-D models…
Higher profiles divide a scene into arbitrarily shaped video objects each one may be compressed and transmitted separately, scene is composed at the receiving end by combining them
SP and ASP restricted to rectangular objects, usually complete frames
MPEG-4
Standard defines an encoding for multimedia streams made up of different sorts of object – video, still images, animation, 3-D models…
Higher profiles divide a scene into arbitrarily shaped video objects each one may be compressed and transmitted separately, scene is composed at the receiving end by combining them
SP and ASP restricted to rectangular objects, usually complete frames

35. MPEG-4 Compression Refinement of MPEG-1 compression
215
MPEG-4 Compression
Refinement of MPEG-1 compression
I-pictures compressed by quantizing and Huffman coding DCT coefficients
Improved motion compensation leads to better quality than MPEG-1 at same bit rates
MPEG-4 Compression
Refinement of MPEG-1 compression
I-pictures compressed by quantizing and Huffman coding DCT coefficients
Improved motion compensation leads to better quality than MPEG-1 at same bit rates

36. MPEG-4 Compression Simple Profile P-pictures only
215–216
MPEG-4 Compression
Simple Profile
P-pictures only
Efficient decompression, suitable for PDAs etc
Advanced Simple Profile (ASP)
B-pictures
Global Motion Compensation
Sub-pixel motion compensation
MPEG-4 Compression
Simple Profile
P-pictures only
Efficient decompression, suitable for PDAs etc
Advanced Simple Profile (ASP)
B-pictures
Global Motion Compensation
Sub-pixel motion compensation

37. 216–219
Older Codecs
Cinepak – Longest established, high compression ratio, takes much longer to compress than to decompress
Intel Indeo – Similar to Cinepak, but roughly 30% faster compression
Sorenson – More recent, higher quality and better compression ratios than other two
All three based on vector quantization
Quality of all three inferior to MPEG-4
Older Codecs
Cinepak – Longest established, high compression ratio, takes much longer to compress than to decompress
Intel Indeo – Similar to Cinepak, but roughly 30% faster compression
Sorenson – More recent, higher quality and better compression ratios than other two
All three based on vector quantization
Quality of all three inferior to MPEG-4

38. 216
Vector Quantization
Divide each frame into small rectangular blocks (’vectors’)
Code Book – collection of constant vectors representing typical patterns (edges, textures, flat colour,…)
Compress by replacing each vector in image by index of vector from code book that most closely resembles it
Vector Quantization
Divide each frame into small rectangular blocks (’vectors’)
Code Book – collection of constant vectors representing typical patterns (edges, textures, flat colour,…)
Compress by replacing each vector in image by index of vector from code book that most closely resembles it

39. Editing Making a constructed whole from a collection of parts
223–230
Editing
Making a constructed whole from a collection of parts
Selection, trimming and organization of raw footage
Apply transitions (e.g. dissolves) between shots
Combination of picture with sound
No changes made to the footage itself
Editing
Making a constructed whole from a collection of parts
Selection, trimming and organization of raw footage
Apply transitions (e.g. dissolves) between shots
Combination of picture with sound
No changes made to the footage itself

40. Post-Production Changing or adding to the material
230–236
Post-Production
Changing or adding to the material
Most changes are generalizations of image manipulation operations (e.g. colour correction, blurring and sharpening,…)
Compositing – combining elements from different shots into a composite sequence
Animating elements and combining animation with live action
Post-Production
Changing or adding to the material
Most changes are generalizations of image manipulation operations (e.g. colour correction, blurring and sharpening,…)
Compositing – combining elements from different shots into a composite sequence
Animating elements and combining animation with live action

41. Preparing for Delivery
236–237
Preparing for Delivery
Compromises required to bring resource requirements of video within capabilities of delivery media (e.g. networks) and low-end machines
Reduce frame size (e.g. downsample to quarter frame)
Reduce frame rate (12fps is OK for smooth motion, flicker not a problem on computer)
Reduce colour depth
Preparing for Delivery
Compromises required to bring resource requirements of video within capabilities of delivery media (e.g. networks) and low-end machines
Reduce frame size (e.g. downsample to quarter frame)
Reduce frame rate (12fps is OK for smooth motion, flicker not a problem on computer)
Reduce colour depth