1. CS 414 - Spring 2012 CS 414 – Multimedia Systems Design Lecture 32 – Media Server (Part 2) Klara Nahrstedt Spring 2012

2. Administrative MP3 posted, April 10, 2012 MP3 deadline April 28, 5pm (Saturday) MP3 presentations  Monday, April 30, 5-7pm CS 414 - Spring 2012

3. Covered Aspects of Multimedia Image/Video Capture Media Server Storage Transmission Compression Processing Audio/Video Presentation Playback Audio/Video Perception/ Playback Audio Information Representation Transmission Audio Capture A/V Playback Image/Video Information Representation CS 414 - Spring 2012

4. Video Server Flickr Flickr – image and video hosting website In November 2007  Flickr hosted 2 Billion Photos In August 2009,  Flickr hosted 62 databases across 124 servers In September 2010,  Fickr hosted more than 5 billion images  Developed by Ludicorp, Vancouver, 2004, now owned by Yahoo! June 2011  51 Million registered members In August 2011  Flickr hosted 6 billion photos

5. Outline Media Server Disk Scheduling and Admission Control Media Server File System Issues CS 414 - Spring 2012

6. Media Server Architecture CS 414 - Spring 2011 Storage device Disk controller Storage management File System Memory Management Content Directory Network Attachment Incoming request Delivered data

7. Review - EDF Example CS 414 - Spring 2012 Note: Consider that block number Implicitly encapsulates the disk track number

8. 8 Review - Elevator (SCAN) Method  Take the closest request in the direction of travel  Real implementations do not go to the end (called LOOK) Pros  Bounded time for each request Cons  Request at the other end will take a while 0199 Arriving Requests in Request Queue 98, 183, 37, 122, 14, 124, 65, 67 Served Request at Disk Controller (37, 14, 0, 65, 67, 98, 122, 124, 183) 53

9. SCAN-EDF Scheduling Algorithm Combination of SCAN and EDF algorithms Each disk block request tagged with augmented deadline  Add to each deadline perturbation Policy:  SCAN-EDF chooses the earliest deadline  If requests with same deadline, then choose request according to scan direction CS 414 - Spring 2012

10. Implementation of SCAN-EDF Notation:  D i be deadline of disk block request ‘i’  N i be track (block) position on disk  N max be maximum number of disk tracks Deadline Modification:  D i + f(N i )  f(N i ) converts track number of ‘i’ into a small perturbation of deadline  Perturbation small enough so that D i + f(N i ) ≤ D j + f(N j ) for D i ≤ D j Possible f(N i ) = N i /N max CS 414 - Spring 2012

11. SCAN EDF Example (N max = 100) CS 414 - Spring 2012

12. Admission Control CS 414 - Spring 2011 Client 1 retrieves K1 blocks in one round Client 2 retrieves K2 blocks Client 3 retrieves K3 blocks Client 4 retrieves K4 blocks Server

13. Admission Control Disk block requests are timed  Media server must determine admit a stream serve (schedule) a stream without having negative effect on other streams already serviced. Deterministic Guarantees  Admission control considers worst case scenario when admitting new stream  Constrained Disk Placement Example: M - size of blocks, G – size of gabs, r dt – data transfer of disk CS 414 - Spring 2011

14. Media Server Architecture CS 414 - Spring 2011 Storage device Disk controller Storage management File System Memory Management Content Directory Network Attachment Incoming request Delivered data

15. Multimedia File System File Placement File allocation tables/Index tables Additional File System Operations  Fast forward  Rewind Block sizes CS 414 - Spring 2011

16. Multimedia File Systems Real-time Characteristics  Read operation must be executed before well-defined deadline with small jitter Additional buffers smooth data File Size  Can be very large even those compressed  Files larger than 2 32 bytes Multiple Correlated Data Streams  Retrieval of a movie requires processing and synch of audio and video streams CS 414 - Spring 2011

17. Placement of Multiple MM Files on Single Disk Popularity concept among multimedia content - very important Take popularity into account when placing movies on disk Model of popularity distribution – Zipf’s Law  Movies are k th ranked if their probability of customer usage is C/k,  C = normalization factor Condition holds: C/1 + C/2 + … C/N = 1,  N is number of customers CS 414 - Spring 2011

18. Example Assume N = 5 movies Problem: what is the probability that the next customer picks 3 rd ranked movie? Solution:  Solve C from the equation C/1 + C/2 + C/3 + C/4 + C/5 = 1  C = 0.437  Probability to pick 3 rd ranked movie is C/3 = 0.437/3 = 0.1456 CS 414 - Spring 2011

19. Placement Algorithm for Multiple Files on Single Disk Organ-Pipe Algorithms (Grossman and Silverman 1973) CS 414 - Spring 2011 Middle of disk (in case of traditional disk layout) 1 st rank (most popular movie) 2 nd ranked movie 3 rd 4 th 5 th 6 th 7 th 8 th 9 th Note: In case of ZBR disk layout, place most popular disks at the outer tracks

20. Placement of Mapping Tables Fundamental Issue: keep track of which disk blocks belong to each file (I-nodes in UNIX) For continuous files/contiguous placement  don’t need maps For scattered files  Need maps Linked lists (inefficient for multimedia files) File allocation tables (FAT) CS 414 - Spring 2011

21. Indexing and FAT CS 414 - Spring 2011 I Frame Higher Level Index Table Per File P Frame B Frame P Frame Block I1 Location PTR Block I2 Location PTR Block I3 Location PTR Block P11 Location PTR Block P12 Location PTR Block B1 Location PTR Block P21 Location PTR Block P22 Location PTR File Allocation Table ……….. …………..

22. Constant and Real-time Retrieval of MM Data Retrieve index in real-time Retrieve block information from FAT Retrieve data from disk in real-time Real-time playback  Implement linked list Random seek (Fast Forward, Rewind)  Implement indexing MM File Maps  include metadata about MM objects: creator of video, sync info CS 414 - Spring 2011

23. Fast Forward and Rewind (Implementation) Play back media at higher rate  Not practical solution Continue playback at normal rate, but skip frames  Define skip steps, e.g. skip every 3 rd, or 5 th frame  Be careful about interdependencies within MPEG frames Approaches for FF:  Create a separate and highly compressed file  Categorize each frame as relevant or irrelevant  Intelligent arrangement of blocks for FF CS 414 - Spring 2011

24. Block Size Issues in File Organization Small Block Sizes  Use smaller block sizes, smaller than average frame size Organization Strategy: Constant Time Length Need Metadata structure, called Frame Index  Frame means a time frame within a movie  Under the time frame read all blocks (audio, video, text) belonging to this time frame CS 414 - Spring 2011 AV VT Frame index Movie Time line AV VT ……… V A V

25. Block Size Issues Large Block Size  Use large blocks (e.g., 256 KB) which include multiple audio/video/text frames Organization Strategy: Constant Data Length Need Metadata structure, called Block Index  Each block contains multiple movie frames CS 414 - Spring 2011 AV V V AAA V VV Block Index

26. Tradeoffs Frame index : needs large RAM usage while movie is playing, however little disk wastage Block index (if frames are not split across blocks): need low RAM usage, but major disk wastage – internal disk fragmentation Block index(if frames are split across blocks): need low Ram usage, no disk wastage, extra seek times CS 414 - Spring 2011

27. Conclusion Designers of VOD systems strive to achieve low access latency for customers Challenges:  Handle large amount of customers (clients)  Maintain low cost of operation  Provide acceptable access latency CS 414 - Spring 2011