1 Multimedia Storage Issues

NUS.SOC.CS5248 OOI WEI TSANG 2 Media vs. Documents large file size write once, read many deadlines!

3 OS Review: Disk

NUS.SOC.CS5248 OOI WEI TSANG 4 Disk head, spindle, track, sector, cylinder seek time, rotational latency

NUS.SOC.CS5248 OOI WEI TSANG 5 Disk Scheduler disk sched I/O Request read/write command

6 Disk Scheduling Algorithm

NUS.SOC.CS5248 OOI WEI TSANG 7 Best Effort Scheduler FCFS: First come first serve SSTF: Shortest seek time first SCAN: Serve all requests in one direction, then in the other direction C-SCAN: Serve all requests in one direction only, then “jump” back to the other end and repeat.

NUS.SOC.CS5248 OOI WEI TSANG 8 Other Schemes EDF: Earliest deadline first EDF-SCAN: Use SCAN to schedule request with same deadline. FD-SCAN: SCAN towards direction with earliest feasible deadline SSEDO: Rank request by deadline and use formula (seektime* rank ) SSEDV: Use formula (lifetime* + seektime*(1- )

9 Data Placement

NUS.SOC.CS5248 OOI WEI TSANG 10 Contiguous

NUS.SOC.CS5248 OOI WEI TSANG 11 Fragmented

NUS.SOC.CS5248 OOI WEI TSANG 12 Stripping (RAID-0)

NUS.SOC.CS5248 OOI WEI TSANG 13 Mirroring (RAID-1)

NUS.SOC.CS5248 OOI WEI TSANG 14 Parity (RAID-5)

NUS.SOC.CS5248 OOI WEI TSANG 15 Parity

NUS.SOC.CS5248 OOI WEI TSANG 16 Summary of Concerns High Throughput Fault Tolerant Load balancing

NUS.SOC.CS5248 OOI WEI TSANG 17 Design Parameters unit of striping (block size) degree of striping (num of disks)

NUS.SOC.CS5248 OOI WEI TSANG 18 Stripe Unit

NUS.SOC.CS5248 OOI WEI TSANG 19 Stripe Unit

NUS.SOC.CS5248 OOI WEI TSANG 20 Effects of Block Size Block Size service time

NUS.SOC.CS5248 OOI WEI TSANG 21 Problem Find block size such that the service time for the most heavily loaded disk is minimize

NUS.SOC.CS5248 OOI WEI TSANG 22 Analysis Model: A Plan service time of the busiest disk as a function of block size expected num of blocks accessed on the busiest disk expected num of blocks accessed on any disk

NUS.SOC.CS5248 OOI WEI TSANG 23 Assumptions Non-redundant array Analyze read operation only

NUS.SOC.CS5248 OOI WEI TSANG 24 Variables Declaration N d : Number of Disks N c : Number of Clients B : Block size N b (i,j): Number of blocks client i access from disk j N b (i,*): Total number of blocks client i access

NUS.SOC.CS5248 OOI WEI TSANG 25 Analysis Suppose i request m blocks

NUS.SOC.CS5248 OOI WEI TSANG 26 Reading 1 block Prob(N b (i,j)=1)

NUS.SOC.CS5248 OOI WEI TSANG 27 Reading k block Prob(N b (i,j)=k)

NUS.SOC.CS5248 OOI WEI TSANG 28 Total Num of Blocks from Disk j N b (*,j) =

NUS.SOC.CS5248 OOI WEI TSANG 29 Busiest Disk N max =

NUS.SOC.CS5248 OOI WEI TSANG 30 Access Time T = N max (t s + t r + Bt t ) Depends on: block size B disk characteristic t s,t r,t t server design N d workload characteristic N c, N(i,j) _

NUS.SOC.CS5248 OOI WEI TSANG 31 Model Verification Service Time Block Size

NUS.SOC.CS5248 OOI WEI TSANG 32 Imbalance and Overhead I = 1 – T avg /T max O = 1 – N max BT t /T max

NUS.SOC.CS5248 OOI WEI TSANG 33 Effects of Block Size Block Size Imbalance Overhead ~optimum

NUS.SOC.CS5248 OOI WEI TSANG 34 Effects of Clients Block Size Imbalance Overhead N c incr, I decr, opt B incr

NUS.SOC.CS5248 OOI WEI TSANG 35 Effects of Clients Block Size Nc

NUS.SOC.CS5248 OOI WEI TSANG 36 Effects of Num of Disks N d incr, N(i,j) decr, I incr, opt B decr Block Size NdNd

NUS.SOC.CS5248 OOI WEI TSANG 37 Choosing Block Size Given N c, assume rest is fixed find B that minimize T if T < duration of a round incr N c and try again

NUS.SOC.CS5248 OOI WEI TSANG 38 There is more.. redundant array finding optimum degree of striping

39 Cello: Disk Scheduler

NUS.SOC.CS5248 OOI WEI TSANG 40 Problem How to co-exist with other applications?

NUS.SOC.CS5248 OOI WEI TSANG 41 Classes of Applications Real-time hard/soft periodic/aperiodic Best-effort interactive troughput-intensive

NUS.SOC.CS5248 OOI WEI TSANG 42 Attempt 1 Priority-based scheduler Always schedule real-time tasks ahead of best-effort tasks

NUS.SOC.CS5248 OOI WEI TSANG 43 Attempt 2 Partition time slots into real-time and best-effort

NUS.SOC.CS5248 OOI WEI TSANG 44 Attempt 3 Assign weight to application class, based on their priority Service based on weight

NUS.SOC.CS5248 OOI WEI TSANG 45 Attempt 3: Cello Two-level scheduling

NUS.SOC.CS5248 OOI WEI TSANG 46 Class Independent Scheduler

NUS.SOC.CS5248 OOI WEI TSANG 47 Allocating Disk Bandwidth Allocate in proportion to time Allocate in proportion to bytes

NUS.SOC.CS5248 OOI WEI TSANG 48 Notations P : Interval of a round I : Total idle time so far in this round U i : Time allocated so far to class i w i : Weight for class i W : Total weight for all class

NUS.SOC.CS5248 OOI WEI TSANG 49 Proportionate Time- Allocation U i < P-I

NUS.SOC.CS5248 OOI WEI TSANG 50 Proportional Time Allocation U i < (P-I)w i /W

NUS.SOC.CS5248 OOI WEI TSANG 51 Example Weights 1:1:2 P = 100 I =

NUS.SOC.CS5248 OOI WEI TSANG 52 Pick a class i which is underutilized, i.e. U i < (P-I)w i /W

NUS.SOC.CS5248 OOI WEI TSANG 53 r next prev Class specific schedule will return a request to be inserted, plus to location in the scheduled queue for insertion. (r, prev, next)

NUS.SOC.CS5248 OOI WEI TSANG 54 Need to Make Sure.. Does not exceed share for class i U i + T < (P-I)w i /W service time for new request

NUS.SOC.CS5248 OOI WEI TSANG 55 Need to Make Sure.. Total service time does not exceed available time  U i + T + T’ next – T next < P-I

NUS.SOC.CS5248 OOI WEI TSANG 56 Update and Repeat next prev

NUS.SOC.CS5248 OOI WEI TSANG 57 What if.. a class have no pending request? put in no_work group a class violates one of the constraints? put in too_much_work group

NUS.SOC.CS5248 OOI WEI TSANG 58 Improving Utilization When disk is idle, distribute service time among classes in too_much_work group Pick a request from this class and put into scheduled queue

NUS.SOC.CS5248 OOI WEI TSANG 59 Free Time Utilization

NUS.SOC.CS5248 OOI WEI TSANG 60 Class Specific Scheduler How to pick (r, prev, next)?

NUS.SOC.CS5248 OOI WEI TSANG 61 Slack Time l i = Latest time you must begin servicing request i e i = Earliest time you may begin servicing request i d i = Deadline for request i s i = Slack time for i = l i - e i

NUS.SOC.CS5248 OOI WEI TSANG 62 Slack Time l i = min(d i, l i+1 ) – T i e i = e i-1 + T i-1 Special case when request is at the beginning of queue or end of queue.

NUS.SOC.CS5248 OOI WEI TSANG 63 Slack Time i i

NUS.SOC.CS5248 OOI WEI TSANG 64 Class 1: Interactive Best Effort next prev pick r at front of pending queue, insert into earliest position in scheduled queue without violating deadlines of others.

NUS.SOC.CS5248 OOI WEI TSANG 65 Class 2: High Throughput Best Effort next prev pick r at front of pending queue, insert into tail of scheduled queue. Order consecutive best effort request by SCAN order.

NUS.SOC.CS5248 OOI WEI TSANG 66 Class 3: Real Time Application next prev pick r in EDF order. Insert into scheduled queue such that it will meet its deadline and will not violate deadlines of others

NUS.SOC.CS5248 OOI WEI TSANG 67 Evaluations response time of best effort app # video SCAN Cello

NUS.SOC.CS5248 OOI WEI TSANG 68 Evaluations %bandwidth time 50% No video text app video app

NUS.SOC.CS5248 OOI WEI TSANG 69 Cello Summary OS disk scheduler Support multiple classes Protect classes from each other Work conserving Use two-level scheduling