1. "Digital Media Primer" Yue-Ling Wong, Copyright (c)2013 by Pearson Education, Inc. All rights reserved.

2. Chapter 6 Fundamentals of Digital Video
Part 6
MPEG Compression
Streaming Video and Progressive Download

3. In this lecture, you will learn:
Basic concepts of MPEG compression
Types of MPEG
Applications of MPEG
What GOP (group of pictures) in MPEG is
How GOP settings affect MPEG video file size
How GOP settings affect MPEG picture quality
What streaming video and progressive download are

4. MPEG
Moving Pictures Experts Group Committee who derives standards for encoding video
Allow high compression
MPEG-1, MPEG-2, MPEG-4

5. What Happend to MPEG-3? NOT MP3 (which is audio format)
Intended for HDTV
HDTV specifications was merged into MPEG-2

6. MPEG-1 Video quality comparable to VHS
Originally intended for Web and CD-ROM playback
Frame sizes up to 352  240 pixels
Video format for VCD (VideoCD) before DVD became widespread

7. MPEG-2 Supports DVD-video, HDTV, HDV standards
For DVD video production: Export video into DVD MPEG-2 format
For HDV video production: Export video into HDV's MPEG-2 format

8. MPEG-4 Newer standard of MPEG family
Different encoding approach from MPEG-1 and MPEG-2 (will discuss after MPEG-1 and MPEG-2 compression in this lecture)

9. How the Compression Works
MPEG-1 and MPEG-2
How the Compression Works

10. Properties of Typical Video
Neighboring frames are very similar
This property is called Temporal redundancy.
MPEG compression exploits temporal redundancy to reduce video file size
by looking for motion difference from one frame to the next
This technique is called motion compensation.

11. Basic Ideas of Motion Compensation
An video frame image is read in as a reference frame
Read the next video frame image
Compare the image content between this current frame (target frame) with the reference frame
one block of pixels at a time

12. Comparing Reference and Target Frames
Case 1:
If a pixel block is identical at the same location in both frames:
No need to encode the target frame's
Just save an instruction to refer to the block in the reference image—requires less space than encoding the whole pixel block

13. Comparing Reference and Target Frames
Case 2:
If a pixel block at the same location in both frames is not identical:
Search for the reference image for a match (because the content may be moved to another location)
Case 2a: No match is found
Case 2b: A match is found

14. Case 2a: No Match is Found
The whole pixel block is encoded
Thus, no saving in file size

15. Case 2b: A Match is Found
The displacement information of the block is saved.
Displacement information:
A 2-dimensional value, indicating how much the block moves horizontally and vertically from the reference frame to the target frame
Called motion vector
Much smaller size than encoding the whole pixel blocks (thus the saving in file size)

16. Illustration of the Ideas
Let's consider the first 4 frames of a video
Frame 1
Frame 2
Frame 3
Frame 4
The square grid cell in a frame represents a block of pixels.

17. Illustration of the Ideas
Frame 1 is read in.
Frame 1
Frame 2
Frame 3
Frame 4

18. Illustration of the Ideas
Frame 1 is used as a reference frame.
Frame 1
Frame 2
Frame 3
Frame 4
The red color highlights the pixel blocks that are encoded.
The whole frame of a reference frame is encoded. No saving in file size.

19. Illustration of the Ideas
Frame 2 is read in.
Frame 1
Frame 2
Frame 3
Frame 4

20. Illustration of the Ideas
Each pixel block in Frame 2 is searched in Frame 1 trying to find a match.
Frame 1
Frame 2
Frame 3
Frame 4

21. Illustration of the Ideas
Each pixel block in Frame 2 is searched in Frame 1 trying to find a match.
Frame 1
Frame 2
Frame 2
Frame 3
Frame 4
The blue color highlights some pixel blocks that are found in Frame 1.
The yellow color highlights some pixel blocks that are found in Frame 1 (but later in the process they are found in the next reference frame.)

22. Illustration of the Ideas
Frame 3 is read in.
Frame 1
Frame 2
Frame 2
Frame 3
Frame 4

23. Illustration of the Ideas
Each pixel block in Frame 3 is searched in Frame 1 trying to find a match.
Frame 1
Frame 2
Frame 2
Frame 3
Frame 4
The process is repeated similar to that for Frame 2.

24. Illustration of the Ideas about Motion Vector
Take Frames 1 and 2 as an example
Frame 1
Frame 2

25. Illustration of the Ideas about Motion Vector
Let's look at the pixel block showing the car's windshield
Frame 1
Frame 2

26. Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier
Frame 1
Frame 2

27. Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier
Frame 1
Frame 2

28. Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier
Frame 1
Frame 2

29. Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier
Frame 1
Frame 2
The red line indicates the displacement of the pixel block.

30. So how are reference frames chosen?
We will need to understand an important concept of group of pictures (GOP) first.

32. Motivation of Understanding GOP
To understand:
the terminology related to GOP because
it is used by MPEG which in turns used by DVD video and HDTV
it comes up in video production application dialog boxes when you export video into MPEG format
the impact of the GOP parameters on video file size and quality
why some video editing programs may not support frame accurate editing of MPEG

34. Group of Pictures (GOP)
Specifies the grouping structure of frames
Frame types in a structure:
I-frames
P-frames
B-frames
An MPEG video contains 1 or more repeating GOPs

35. Number of Frames in a GOP
Each GOP in a video has a fixed number of frames
The number is the N parameter
Example:
DVD-compliant MPEG-3: N = 15
HDV: N = 15

36. I-frames Stands for Intraframes
Encoded only using the information within the frame--intracoding.
Use spatial compression
Similar to JPEG compression
No temporal compression
Least compressed of the three types
Reference frame in the previous 4-frame example

37. I-frames
Each GOP starts with an I-frame
Each GOP has only one I-frame

38. P-frames Stands for predicted frames
Encoded using the information from the previous I- or P-frames as the reference frame if a match of a pixel block is found
The 2nd and 3rd frames in the previous 4-video example

39. B-frames Stands for bidirectional frames Between I- and P-frames
Encoded using information from the previous and subsequent I- and/or P-frame as the reference frames

40. Example GOP DVD-compliant MPEG-2: I B B P B B P B B P B B P B B N = 15
i.e., number of frames in a GOP = 15

41. Example GOP DVD-compliant MPEG-2: I B B P B B P B B P B B P B B M = 3
i.e., number of frames between non-B-frames plus one = 2+1 = 3

42. Revisit the 4-frame video example
Suppose N = 15, M = 3

43. Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1

44. Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1
I-frame:
Least compressed

45. Based on changes from Frame 1
Example GOP: N = 15, M = 3
I B B P B B P B B P B B P B B
Frame 1
Frame 2
Frame 3
Frame 4
P-frame:
Based on changes from Frame 1

46. Based on changes from Frames 1 and 4
Example GOP: N = 15, M = 3
I B B P B B P B B P B B P B B
Frame 1
Frame 2
Frame 3
Frame 4
B-frame:
Most compressed.
Based on changes from Frames 1 and 4

47. Based on changes from Frames 1 and 4
Example GOP: N = 15, M = 3
I B B P B B P B B P B B P B B
Frame 1
Frame 2
Frame 3
Frame 4
B-frame:
Most compressed.
Based on changes from Frames 1 and 4

48. How GOP Settings Affect File Size
Shorter GOP (i.e., lower N):
larger file size
Rationales:
A MPEG-2 consists of a repeating GOP structure
Each GOP contains one I-frame
I-frames are the least compressed among the three types of frames, and thus take up more storage space
Shorter GOP means more GOPs in a MPEG-2
More GOPs means more I-frames in a MPEG-2

49. How GOP Settings Affect Picture Quality
Shorter GOP (i.e., lower N):
better picture quality
Rationales:
I-frames are compressed based on the frame information, rather than predicted from other frames

50. How GOP Structure Affects Frame-Accurate Video Editing
Some video editing programs may not support frame-accurate video editing of MPEG-2 because it is more complex than other video format
Rationales:
The information for a P-frame depends on the information of its previous I-frame.
A B-frame depends on the information of its previous and subsequent I- or P-frames.
Thus, it is more complex to trim out frames from a MPEG-2 than other video format.

51. Applications of MPEG-4 Cover a wide range of data rate
Low end of the data rate: Video playback on mobile devices
High end of the data rate:
HDTV
Handheld and portable game devices (e.g., Sony PSP)

52. MPEG-4 Coding Approach Uses media objects
A scene may contain separate media objects
Not the frame-based coding used in MPEG-1 and MPEG-2
Conventional frame-based video can be converted to MPEG-4 because a frame can be treated as a media object--a degenearted case.

53. Ways of Playing Video

54. Two Ways of Playing Video
Play from disk
Play over a network

55. Play from Disk
An entire clip needs to be on disk before it can be played
Played from hard drive, CD, or DVD

56. Play over a Network
The video can be played while it is being downloaded
Can be played from disk
Streaming video
Progressive download

57. Streaming Video Play video as soon as enough data has arrived
Examples:
Streaming QuickTime
Real Video
Window Media Video (WMV)

58. Streaming Video Require a streaming server to stream video
Allow saving several different compression levels of a video in a single file
The server chooses the compression level to match the speed of network connection
Buffering: Wait time depends on network speed

59. Progressive Download Play video as soon as enough data has arrived
Does not require special servers
Example:
QuickTime fast-start
Created by saving the QuickTime movie as self-contained using QuickTime Pro

60. Review Questions Note to instructor:
Depending on your preference, you may want to go over the review questions at the end of this lecture as an instant review or at the beginning of next lecture to refresh students' memory of this lecture.

61. Review Question
True/False: The MP3 audio is a MPEG-3.
False

62. Review Question
___ provides a video quality comparable to VHS and is the file format for VCD.
MPEG-1
MPEG-2
MPEG-3
MPEG-4 A

63. Review Question ___ support the DVD-video, HDV, and HDTV standards.
MPEG-1
MPEG-2
MPEG-3
MPEG-4 B

64. Review Question
True/False: A typical MPEG-2 consists of a repeating GOP structure.
True

65. Review Question Motion compensation is a key technique in ___.
asymmetric
lossless
lossy
spatial
temporal E

66. Review Question
___ is encoded using only the information within that frame.
B-frame
I-frame
P-frame B

67. Review Question
___ is encoded using only the previous I- or P-frame as the reference frame.
B-frame
I-frame
P-frame C

68. Review Question
___ is encoded using the previous and subsequent I- and/or P-frame as the reference frames.
B-frame
I-frame
P-frame A

69. Review Question
___ is the least compressed among the three frame types.
B-frame
I-frame
P-frame B

70. Review Question The N parameter of the GOP refers to ___.
the number of B-frames in a GOP
the number of I-frames in a GOP
the number of P-frames in a GOP
the total number of frames in a GOP
one plus the number of frames between the I- and P-frame, the P- and P-frame, and the P- and next GOP's I-frame D

71. Review Question The M parameter of the GOP refers to ___.
the number of B-frames in a GOP
the number of I-frames in a GOP
the number of P-frames in a GOP
the total number of frames in a GOP
one plus the number of frames between the I- and P-frame, the P- and P-frame, and the P- and next GOP's I-frame E