1. MP4 help session Hoang Nguyen

2. Outline Scenarios Media Synchronization requirement FF/RWND vs. Seek Media synchronization GUI Demo steps (posted next Monday)

3. Scenario A Server 1 Dispatcher Client 1 Server 2 audio1 file video1 file audio2 file video2 file PLAY audio1/video1 or PLAY audio2/video2 Audio 2 streaming Video 2 streaming

4. Scenario B Server 1 Dispatcher Client 1 Client 2 Server 2 audio1 file video1 file audio2 file video2 file PLAY audio1/video1 PLAY audio2/video2

5. Scenario C (optional) Server 1 Dispatcher Client 1 Client 2 audio1 file video1 file audio2 file video2 file PLAY audio1/video1 PLAY audio2/video2

6. Scenario D (optional) Server 1 Dispatcher Client 1 Client 2 audio1 file video1 file PLAY audio1/video1 audio1 file video1 file

7. Media synchronization requirement Clips will be provided (1 week before the demo) –E.g. 1) An interview (lipsync), 2) A music clip Each clip consists of an audio stream and a video stream stored in separate files The demo will show how to –Upload/Manipulate those files (GUI or command line) –Play a synchronized clip per corresponding video file and audio file

8. How to create these clips? Find an video file (e.g. youtube) –E.g.: http://www.youtube.com/watch?v=AmUUYo9o9eg http://www.youtube.com/watch?v=AmUUYo9o9eg Download it (e.g. kissyoutube) –E.g.: http://www.kissyoutube.com/watch?v=AmUUYo9o9eghttp://www.kissyoutube.com/watch?v=AmUUYo9o9eg Extract audio –E.g.: ffmpeg -i obama.mp4 -vn -acodec pcm_s16le -ar 44100 -ac 2 obama.wav Extract video –E.g.: ffmpeg -i obama.mp4 -vcodec mjpeg obama.mjpeg

9. FF/RWND vs. Seek Only implement either FF/WRND or SEEK FF/Rewind: –requires skipping frames (similar to MP1) On PLAY  FF/RWND: –The client works on the local buffer first –The client sends the request to the server –The server sends 1-second frames for buffering 12345 FF/RWND: step of 2 67

10. FF/RWND vs. Seek Seek: –requires random access to frames ( av_seek ) –May require a faster/shorter buffering i Seek to frame ith

11. Media Synchronization Audio and video files must be able to clearly demonstrate synchronization –Speeches and singing are good examples requiring lip synchronization Audio is leading time synchronization stream –Video (and any other stream) should be synchronized with respect to audio Recommend synchronization point each second –Might need to speed up or slow down video relative to the audio

12. Media Synchronization 01234567891011121314151617 01234567891011 audio stream 3 samples/sec video stream 2 frames/sec Synchronization Points audio = 3 video = 2 audio = 6 video = 4 audio = 9 video = 6 audio = 12 video = 8 audio = 15 video = 10

13. Media Synchronization The server –May maintain a sequence number –Each sent frame is assigned with a sequence number The client –May maintain a counter –Each received frame is assigned with a counter

14. Media Synchronization Ensure the audio stream is not starved at the client Adjust the sending rate of the video stream according to the video –If the video is AHEAD => slow down the video server –If the video is BEHIND => Speed up the video server Drop frames arriving later than the desired presentation deadline

15. Media Synchronization audio stream 3 samples/sec video stream 2 frames/sec Synchronization Points audio = 3 video = 2 audio = 6 video = 4 audio = 9 video = 5 audio = 12 video = 8 audio = 15 video = 10 DROPPED VIDEO FRAME 6 TO CATCH UP 01234567891011121314151617 01234 6 57891011

16. Graphical User Interface GTK Requires creativity Groups are free to use their own original design Should support many operations common to video-on-demand services Should display meta-data –Dispatcher address and port –Audio/video characteristics Sample rate Frame rate

17. GUI Operations MENU –List all of the available A/V files –File management interface PLAY –Play the selected A/V stream STOP –Stop the selected A/V stream PAUSE –Pause the selected A/V stream FF and RWND (if implemented) –Fast-forward or rewind selected A/V stream SEEK (if implemented) –A textbox for user to enter or a slider