Download presentation

Presentation is loading. Please wait.

1
1 School of Computing Science Simon Fraser University, Canada Rate-Distortion Optimized Streaming of Fine-Grained Scalable Video Sequences Mohamed Hefeeda & ChengHsin Hsu 2 February 2007

2
2 Motivations Multimedia streaming over the Internet is becoming very popular -More multimedia content is continually created -Users have higher network bandwidth and more powerful computers Users request more multimedia content And they look for the best quality that their resources can support

3
3 Motivations (cont’d) Users have quite heterogeneous resources (bandwidth) -Dialup, DSL, cable, wireless, …, high-speed LANs To accommodate heterogeneity scalable video coding: Layered coded stream -Few accumulative layers -Partial layers are not decodable Fine-Grained Scalable (FGS) coded stream -Stream can be truncated at bit level

4
4 Motivations (cont’d) Goal: Optimize quality for heterogeneous receivers In general setting -FGS-coded streams -Multiple senders with heterogeneous bandwidth and store different portions of the stream Why multiple senders? -Required in P2P streaming: Limited peer capacity and Peer unreliability -Desired in distributed streaming environment: Disjoint network path Better streaming quality

5
5 Our Optimization Problem Assign to each sender a rate and bit range to transmit such that the best quality is achieved at the receiver. Consider a simple example to illustrate the importance of this problem

6
6 Example: Different Streaming Schemes Non-scalableLayered

7
7 Example: Different Streaming Schemes FGS ScalableOptimal FGS Scalable

8
8 Problem Formulation First: single-frame case -Optimize quality for individual frames Then: multiple-frame case -Optimize quality for a block of frames -More room for optimization

9
9 Input Parameters T : fixed frame period n : number of senders b i : outgoing bandwidth of sender i b I : incoming bandwidth of receiver s i : length of (contiguous) bits held by sender i Assume s 1 <= s 2 <= …… <= s n

10
10 Allocation: A = {(Δ i, r i ) | i=1, 2, ……, n} -Δ i : number of bits assigned to i -r i : streaming rate assigned to i Specifies: -Sender 1 sends range [0, Δ 1 -1] at rate r 1 -Sender 2 sends range [Δ 1, Δ 1 +Δ 2 -1] at rate r 2 -… -Sender i sends range at rate r i Outputs

11
11 Integer Programming Problem Minimize distortion Subject to: -on-time delivery -assigned range is available -assigned rate is feasible -Aggregate rate not exceeds receiver’s incoming BW

12
12 How do we Compute Distortion? Using Rate-Distortion (R-D) models -Map bit rates to perceived quality -Optimize quality rather than number of bits Approaches to construct R-D models -Empirical Models: Many empirical samples expensive -Analytic Models: Quality is a non-linear function of bit rate, e.g., log model [Dai 06] and GGF model [Sun 05] -Semi-analytic Models: A few carefully chosen samples, then interpolate, e.g., piecewise linear R-D model [Zhang 03] Detailed analysis of R-D models in [Hsu 06]

13
13 Within each bitplane, approximate R-D function by a line segment Line segments of different bitplanes have different slopes The Linear R-D Model

14
14 Visual Validation of Linear R-D Model Mother & Daughter, frame 110 Foreman, frame 100

15
15 Rigorous Validation of Linear R-D Model Average error is less than 2% in most cases

16
16 Let y i be number of bits transmitted from bitplane i Distortion is: -d : base layer only distortion -g i : slope of bitplane i -z : total number of bitplanes Using the Linear R-D Model

17
17 Integer Linear Programming (ILP) Problem Linear objective function Additional constraints -number of bits transmitted from bit plane h does not exceed its size l h -bits assigned to senders are divided among bitplanes

18
18 Solution of ILP is a Valid FGS Stream Lemma 1: -An optimal solution for the integer linear program produces a contiguous FGS-encoded bit stream with no bit gaps Proof sketch -minimizing -Since g 1 < g 2 < …… <g n <0 (line segment slopes), -the ILP will never assign bits to y i+1 if y i is not full

19
19 Solving ILP problem is expensive Solution: Transform it to Linear Programming (LP) problem -Relax variables to take on real values Objective function and constraints remain the same Linear Programming Relaxation

20
20 Solve LP -Result is real values Then, use the following rounding scheme for solution of the ILP Efficient Rounding Scheme

21
21 Correctness/Efficiency of Proposed Rounding Lemma 2 (Correctness) -Rounding of the optimal solution of the relaxed problem produces a feasible solution for the original problem Lemma 3 (Efficiency: Size of Rounding Gap) -The rounding gap is at most nT + n, where n is the number of senders and T is the frame period -Example: T=30 fps, n=30, the gap is 32 bits -Indeed negligible (frame sizes are in order of KBs)

22
22 FGSAssign: Optimal Allocation Algorithm Solving LP (using Simplex method for example) may still be too much -Need to run in real-time on PCs (not servers) Our solution: FGSAssign -Simple yet optimal allocation algorithm -Greedy: Iteratively allocate bits to sender with smallest s i (stored segment) first

23
23 Pseudo Code of FGSAssign Sort senders based on s i, s 1 ≤ s 2 ≤ …… ≤ s n ; x 0 = …… = x n = 0; Δ 1 = …… = Δ n = 0; r agg = 0; for i = 1 to n do x i = min(x i−1 + b i T, s i ); r i = (x i − x i−1 )/T ; if (r agg + r i < b I ) then r agg = r agg + r i ; Δ i = x i − x i−1 ; else r i = b I − r agg ; Δ i = T × r i ; return endfor

24
24 Optimality of FGSAssign Theorem 1 -The FGSAssign algorithm produces an optimal solution in O(n log n) steps, where n is the number of senders. Proof: see paper Experimentally validated as well.

25
25 Multiple-Frame Optimization Solve the allocation problem for blocks of m frames each Objective: minimize total distortion in block Why consider multiple-frame optimization? -More room for optimization -Less computation overhead: solve the problem less often

26
26 Multiple-Frame Optimization: Why? More room for optimization: higher quality and less quality fluctuation

27
27 Multiple-Frame Optimization Formulation (in the paper): -Straightforward extension to single-frame with lager number of variables and constraints -Computationally expensive to solve Our Solution: mFGSAssign algorithm -Heuristic (close to optimal results) -Achieves two goals: Minimize total distortion in a block Reduce quality fluctuations among successive frames

28
28 mFGSAssign: High-Level Description 1.Estimate a target distortion D that is feasible and achieves the two goals (binary search) 2.Compute for each frame f in the block its bit budget B f 3.For each frame f, call FGSAssign to allocate B f among senders

29
29 Computing Target Distortion Is bit budget enough to transmit all frames at distortion level D ? -D u : distortion upper bound -D l : distortion lower bound -D : distortion estimate

30
30 Efficiency of mFGSAssign Lemma 5 -mFGSAssign terminates in O(m n log n) steps, where n is the number of senders and m is the number of frames in a block Much more efficient than linear programming approach

31
31 Experimental Setup Software used -MPEG-4 Reference Software ver 2.5 Augmented to extract R-D model parameters Algorithms implemented (in Matlab) -LP solutions using Simplex for the single-frame and multiple-frame problems -FGSAssign algorithm -mFGSAssign algorithm -Nonscalable algorithm for baseline comparisons

32
32 Experimental Setup (cont’d) Streaming scenarios -Four typical scenarios for Internet and corporate environments Testing video sequences -Akiyo, Mother, Foreman, Mobile (CIF) -Sample results shown for Foreman and Mobile

33
33 Single Frame: Quality (PSNR) Foreman, Scenario I Mobile, Scenario III Quality Improvement: 1--8 dB FGSAssign is optimal

34
34 Multiple Frame: Quality (PSNR) Foreman, Scenario II Mobile, Scenario III Scalable: higher improvement than single frame mFGSAssign: almost optimal (< 1% gap)

35
35 Fluctuation Reduction Foreman, Scenario II Mobile, Scenario III Small quality fluctuations in successive frames

36
36 Running Time Foreman, Scenario I Foreman, Scenario IV Small and stable running times for mFGSAssign, unlike mOPT (Simplex) mFGSAssign can be used in real time

37
37 Reconstructed Pictures Nonscalable mFGSAssign

38
38 Conclusions Formulated and solved the bit allocation problem for single and multiple frame cases Nonlinear problem integer linear program -Using linear R-D model Integer linear program linear program -Using simple rounding scheme Efficient Algorithms -FGSAssign: Optimal and efficient -mFGSAssign: close to optimal in terms of average distortion, reduces quality fluctuations, runs in real time Significant quality improvements shown by our experiments

39
39 Thank You! Questions?? All programs/scripts/videos are available: http://www.cs.sfu.ca/~mhefeeda

Similar presentations

© 2024 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google