1. Video Processing & Communications
Basics of Video
Courtesy of Professor Yao Wang
Polytechnic University, Brooklyn, NY11201

2. Outline Video capture Analog video Digital video Video display
Photometric model
Geometric model
Analog video
Progressive vs. interlaced rasters in analog TV system
Different color representations: YUV/YIQ
Digital video
Sampling/quantization
Y’CbCr format
Video display
Spatial/temporal/bit-depth resolution
Adapted from Yao Wang, 2004
Video Basics

3. Photometric Model of Video Capture
Courtesy of Onur Guleryuz
Adapted from Yao Wang, 2004
Video Basics

4. Geometric Model of Video Capture
point
Camera
center
The image of an object is reversed from its 3-D position. The object appears smaller when it is farther away.
2-D
image
Image
plane
Adapted from Yao Wang, 2004
Video Basics

5. Implication of Models in Analog World
Miniature building Explosion from The Mummy
Lighting in Filmmaking
Adapted from Yao Wang, 2004
Video Basics

6. Progressive and Interlaced Raster Scans
Field 1
Field 2
Progressive Frame
Interlaced Frame
Horizontal retrace
Vertical retrace
Interlaced scan is developed to provide a trade-off between temporal and vertical resolution, for a given, fixed data rate (number of line/sec).
Adapted from Yao Wang, 2004
Video Basics

7. Color TV Broadcasting and Receiving
Adapted from Yao Wang, 2004
Video Basics

8. Why not using RGB directly?
R,G,B components are correlated
Transmitting R,G,B components separately is redundant
More efficient use of bandwidth is desired
RGB->YC1C2 transformation
Decorrelating: Y,C1,C2 are uncorrelated
C1 and C2 require lower bandwidth
Y (luminance) component can be received by B/W TV sets
Color transformation is a compromised solution, but the ultimate one
Adapted from Yao Wang, 2004
Video Basics

9. YIQ in NTSC I (in-phase): orange-to-cyan
Q (quadrature): green-to-purple (human eye is less sensitive)
Q can be further bandlimited than I
Phase=Arctan(Q/I) = hue, Magnitude=sqrt (I^2+Q^2) = saturation
Hue is better retained than saturation
Recall: Quadrature amplitude modulation (QAM) in digital communication
Adapted from Yao Wang, 2004
Video Basics

10. Color Image
Y image
I image (orange-cyan)
Q image (green-purple)

11. I and Q on the color circle
Q: green-purple
I: orange-cyan
Adapted from Yao Wang, 2004
Video Basics

12. Conversion between RGB and YIQ
RGB -> YIQ
Y = R G B
I = R G B
Q = R G B
YIQ -> RGB
R =1.0 Y I Q,
G = 1.0 Y I Q,
B =1.0 Y I Q.
Adapted from Yao Wang, 2004
Video Basics

13. YUV in PAL
Adapted from Yao Wang, 2004
Video Basics

14. YUV/RGB Conversion Numerical approximations
Y = (( ( 66 * R * G + 25 * B + 128) >> 8) + 16)
U = ( ( -38 * R - 74 * G * B + 128) >> 8) + 128
V = ( ( 112 * R - 94 * G - 18 * B + 128) >> 8) + 128
Adapted from Yao Wang, 2004
Video Basics

15. YIQ/YUV Comparison
Adapted from Yao Wang, 2004
Video Basics

16. Different Color TV Systems
Adapted from Yao Wang, 2004
Video Basics

17. Who uses what? From http://www.stjarnhimlen.se/tv/tv.html#worldwide_0
Adapted from Yao Wang, 2004
Video Basics

18. Digital Video Sampling Quantization
Taken from EE465: Image Acquisition
Adapted from Yao Wang, 2004
Video Basics

19. BT.601* Video Format * BT.601 is formerly known as CCIR601
Adapted from Yao Wang, 2004
Video Basics

20. RGB <--> Y’CbCr Analog video Digital video
Adapted from Yao Wang, 2004
Video Basics

21. YUV vs. Y’CbCr
Adapted from Yao Wang, 2004
Video Basics

22. Chrominance Subsampling Formats
Adapted from Yao Wang, 2004
Video Basics

23. Digital Video Formats
Adapted from Yao Wang, 2004
Video Basics

24. 4:2:0 YUV Video U: 144-by-176 V: 144-by-176 Y: 288-by-352
Adapted from Yao Wang, 2004
Video Basics

25. Tricky Photometric Situations
Shadow causes problem to background
extraction
Video enhancement
Adapted from Yao Wang, 2004
Video Basics

26. Geometric Invariance
Adapted from Yao Wang, 2004
Video Basics

27. Video Display High-end Low-end
If the resolution of display device is higher than that of video sequence, what can we do?
Tradeoff between quality and complexity
Subjective evaluation of video quality
Low-end
If the resolution of display device is lower than that of video sequence, what can we do?
What if the bit-depth resolution is lower? (e.g., display video on PDAs and portable DVDs)
It is the last and the least-researched component in visual
communication systems
Adapted from Yao Wang, 2004
Video Basics

28. Resolution, Resolution, Resolution
temporal
300fps
30fps 1M
10M
spatial
8bpp
32bpp
Bit-depth
Adapted from Yao Wang, 2004
Video Basics

29. High Dynamic Range Imaging
Q: Can we generate a HDR image (16bpp) by a standard camera?
A: Yes, adjust the exposure and fuse multiple LDR images together
Adapted from Yao Wang, 2004
Video Basics

30. HDR Display (after Toner Mapping)
Note that any commercial display devices we see these days are NOT HDR
Adapted from Yao Wang, 2004
Video Basics