1. Digital Media Dr. Jim Rowan ITEC 2110 Video

2. Works because of persistence of vision http://en.wikipedia.org/wiki/Frame_rate Fusion frequency –~ 40 frames per second –depends on the brightness of the image relative to the viewing environment Less than that –first flickering –then individual images appear losing the illusion of motion

3. Video Video vs Animation... –Video capture of frames in the world play them back –Animation create frames individually play them back

4. Video... Computationally demanding –Capture must be fast enough to capture sufficient frames to produce the illusion of motion –Transport across the web if there is a live feed across the web it must be fast enough to carry those captured frames at a rate fast enough to produce the illusion of motion –Playback the processor and all its internal parts must be fast enough to play those captured frames at a rate fast enough to produce the illusion of motion

5. Video Transport or playback not fast enough? –something’s got to give Video players (like quicktime) make compromises differently –Attempt to “degrade gracefully” Some drop frames holding the last image –effectively losing the illusion of motion but continuing the story as a slide show Some play lower resolution images Some continue to play audio

6. Video Standards NTSC PAL SECAM

7. Video Standards NTSC –America and Japan –http://en.wikipedia.org/wiki/NTSC –24 frames per second –525 scanlines, 60Hz (486 visible) chosen because of vacuum tube limitations –framesize different than PAL

8. Video Standards PAL –Western Europe and Australia –http://en.wikipedia.org/wiki/PAL –25 frames per second –625 (576 visible)-line/50 Hz –framesize different than NTSC

9. Video Standards SECAM –France, Eastern Europe –http://en.wikipedia.org/wiki/SECAM –frequency modulated (fm) –Difficult to edit in it’s native form

10. Video gets big (filesize) fast At a 640 X 480 framesize Using 3 byte color (24 bits, one byte per color) each frame ~ 2 megabytes One second of video (uncompresssed) is 26 Megabytes One minute is 1.85 gigabytes

11. The effects of large size... Uncompressed? –exceeds most home computer interface standards –strains the internal speed of the home computer –strains the storage capability of home computer –WAY exceeds what can be carried by the net

12. What to do? Apply compression! On the capture side –digitization & compression needs to be carried out by hardware to be fast enough –Can be done in the camera (VTR) –Can be done in the computer (iSight cam)

13. Compression in the camera Within the camera –3 (at least) different formats internally with differing error correction and compatibility –Recording on different media CD, Tape, Memory card Mini DV or DV format –All 3 media present the same stream of bits to the computer over a firewire connection Compression? Artifacts!

14. Compression in the computer? Analog is presented to the computer through a video capture card Compression is done (usually) in the video capture card Allows for a really small camera because the work is done elsewhere

15. Analog vs Digital An analog signal to the computer is susceptible to noise corruption Digital signal is not What’s the big deal? Consider compressing a video of a wall painted a solid color –Analog noise will cause small fluctuations from pixel to pixel –RLE can’t compress it because each pixel is a bit different

16. iMovie video capture card computer miniDV compression webCam analog signal digital signal 640 x 480 = 307,200 307,200 can be represented by < 24 bits, call it 3 bytes RLE: 307,200 (3bytes) + RGB (3 bytes) = 6 bytes Consider compressing this using RLE the scene 640 x 480 = 307,200 bytes Noise makes each pixel a little different RLE: 307,200 bytes x RGB (3bytes) = 921600 bytes !!!NOISE!!!

17. hardware vs software compression Hardware compression... user has no control over it... it is hardwired –It is in the camera –It can be in the video card Software conversion... is computationally expensive... it’s a slow process –Provides for the most flexibility –Can use different software coder-decoders (codec), picking and choosing what fits your needs better

18. Streaming Video Similar to Internet radio… but with images Video is transported across the web played as it arrives Video conferencing Network bandwidth is the chokepoint http://en.wikipedia.org/wiki/Streaming_media

19. Methods of Video delivery … NOT a mutually exclusive list …CAN be combined Streaming video Embedded video –embedded inside something else like a webpage –plays from the beginning –has user controls –usually does not allow download –youTube, google video

20. (more) Methods of Video delivery Podcast –download now and play later Progressive download starts playing before download is completed Pull Technology –user requested (requested from the client) Push Technology –server initiated (like chat)

21. Streaming video (like online tv shows) not stored on disk can be open-ended you don’t get a copy to play later –solves copyright issue! Advantages? –can collect viewer stats without Nielsen ratings –advertisers know how many viewers there are –on some, you can advance ahead of where the download is complete and the play will start again at the new spot

22. Progressive download downloads some and then starts playing quicktime does this –predicts when the download will complete and starts playing when there is enough downloaded to “safely” play without interruption you can get a copy to play later on some, you can advance ahead of where the download is currently completed and the download will start again at the new spot without waiting for the skipped-over part to load

23. The TV legacy Raster scan - 525 lines in US and Japan Vacuum tube limitations ==> couldn’t transmit 40 frames per second Resulted in Interlacing fields Play half the Frame (and half the data) then play the other half of the Frame

25. odd field

26. even field

27. The problem... The fields were played one after the other to avoid flicker BUT... The fields were also captured one after the other... The result? There is a time difference between when they were captured

28. Why is this a problem? When you play them on a computer, the computer can refresh much faster and can display the entire frame at the same time To play it can put both fields in a frame buffer and displays them at the same time If the object is moving fast, the second frame shows the object in a different place Results in a “comb effect”

30. Can we solve the problem? You can average the two frames and construct a single frame You can toss out one of the fields and interpolate between them Neither is very good...

31. Converting TV to Video is problematic (comb effect) Converting Film to Video is problematic So…

32. Film to Video Problematic –video is 30 frames per second –film is 24 frames per second How do you make 30 frames from 24? The 3-2 pull down…

33. http://en.wikipedia.org/wiki/Telecine Film to interlaced video:

34. Next The rest of chapter 6…

35. Questions?