1. Considerations for Video Scene Selection Margaret H. Pinson NTIA/ITS www.its.bldrdoc.gov/n3/video/

2. Scene Complexity Should evenly span a wide range of coding difficulty Major impact to quality Why use an objective metric? Uneven distribution in available scenes More easy-to-code scenes available since they are easy-to-film

3. Complexity Algorithm #1 ITU-T P.910 section 6.3 “Two Criteria for Video Test Scene Selection”, ITU-T Study Group 12, 1994 Spatial Information SI = max time {std space [Sobel (F n )]} Temporal Information TI = max time {std space [F n -F n-1 ]}

4. Complexity Algorithm #2 “Perceptual Effects of Noise in Digitial Video Compression,” SMPTE 1998 o = log 10 {mean time [SI n * TI n ]} SI n = rms space [Sobel (F n )] TI n = rms space [F n -F n-1 ]

5. Scene Complexity Criteria One very difficult to code scene One very easy to code scene One high spatial detail scene Evenly span full range of complexity Some low Some medium Some high

6. Wide Variety of Scene Content Pro Relieves viewer boredom Prevents viewer overtraining Analysis results more general Con More scene content required Less HRCs for given sized full-matrix test Could use partial block design

7. Wider Quality Spread at Lower Bit Rates

8. Scene Content Selection Criteria Avoid repetition of scene type e.g., only one basketball scene Use each scene as few times as possible to allow desired HRC comparisons e.g., pair each scene with 4 HRCs Divide experiment into 2 or more full matrixes (block design)

9. Scene Cuts Masking effect Hides impairments Few frames before, ≈0.25 sec after Philosophy No scene cuts Television ≈3 to 5 sec between scene cuts Opening credits rapid scene cuts GOP structure interactions MPEG-2 GOP 15F for 30fps H.264 / MPEG-4 part 10 GOP 33F for 30fps

10. Scene Cuts Criteria One scene with rapid scene cuts ≈½ source no scene cuts ≈½ source with scene cuts Avoid scene cuts at very beginning or end of edited scene

11. Camera or Original Quality High quality camera recording Many people don’t have Medium quality camera recording Mild coding noise, mild analog noise Impacts HRC response Low quality camera recording VQEG does not use Commonly available to end users

12. Impact of Scene Content Despite viewer instructions “Please do not base your opinion on the content of the scene or the quality of the acting.” Ratings include opinion of content “Susie effect” popular scene

13. Impact of Scene Content “Aspen” HDTV 50Mbps MOS=4.7 excellent (17 viewers) good (8 viewers) fair (0 viewer) “ControlledBurn” 25 Mbps HDTV MOS=4.5 excellent (15 viewers) good (6 viewers) fair (3 viewer) coding artifacts “TouchdownDay” 100 Mbps HDTV MOS=4.0 excellent (4 viewers) good (17 viewers) fair (3 viewers) professional cameraman, no scene cuts TouchdownDay ControlledBurn Aspen

14. Original Scene Content & Quality Criteria No objectionable or controversial content No politics, violence, scantily clad people Avoid scenes with technical issues Strong judder, awkward scene cuts, poor focus, motion blur HDTV test – viewers more discriminating CIF test – viewers more tolerant

15. Include Animation Coder/decoder response very different H.264 higher quality MPEG-2 lower quality Animation overlay increasingly common Scrolling text and edge noise Preferably, use at least one scene with animation or animation overlay

16. Unusual Scene Characteristics for Coding Difficulty & Viewer Perception Fine detail Gravel, grass, hair, rug texture, pinstripes Sensitive to blurring, but viewers may not notice or care Sharp black/white edges Blurred background, foreground in focus Night scene or dim lighting At least one

17. Unusual Scene Characteristics for Coding Difficulty & Viewer Perception Ramped color with graduated change of color (e.g., sunset) Water, fire & smoke contribute unusual shapes and shifting patterns Picture jiggles or bounces significantly Hand-held camera Flashing lights or other very fast events Emergency lights, camera flash, falling snow, breaking glass

18. Unusual Scene Characteristics for Coding Difficulty & Viewer Perception High motion E.g., object moves 20+ pixels per frame at VGA resolution Action is in a small portion of the total picture Attention focus Colorful scene Important

19. Unusual Scene Characteristics for Coding Difficulty & Viewer Perception Small amounts of analog noise Multiple objects moving in a random, unpredictable manner

20. Include Close-up of People Popular Skin tones Knowledge of how people ought to appear & move Internal reference Internal Reference No Internal Reference

21. Close-up People Criteria Filming Privacy issues Releases Faces large enough to be easily seen Several scenes with close-up people Close-up Crowd Scene

22. Contrast and Brightness One scene with low contrast Soft edges One scene with high contrast Hard edges One scene with low brightness Dim One scene with high brightness Mostly white

23. Obtaining Content TV & movie content strictly controlled Purchase for own use Typically cannot redistribute Hire someone to make content for you Shoot it yourself! Royalty-free distribution More flexible usage No time expiration

24. 2009, 4 th Quarter Consumer Digital Video Library Repository of high quality video User forums For quality measurement algorithms Subjectively rated databases For consumer video processing Coding, corrective/reconstructive processing, enhancement, re-formatting www.cdvl.org