1. MPEG-21 Scalable Video Coding and the Stool (UniBS) system
Riccardo Leonardi, Alberto Signoroni,
Michele Brescianini, Nicola Adami
Signals and Communications Lab. – DEA
University of Brescia, Italy
Thessaloniki – October 7-8, 2004

2. R. Leonardi - A. Signoroni University of Brescia, Italy
Outline
Scenarios (why SVC is desirable)
MPEG Requirements (the “shall” and “should” of SVC)
Technologies which enable SVC
SVC architectures (t+2D, 2D+t, UniBS)
Experimental results
R. Leonardi - A. Signoroni University of Brescia, Italy

3. Video coding paradigms
Classical video coding
Code-decode at a single operating point (target frame resolution, spatial resolution and quality)
Code-transcode at operating points different from the target (complexity, performance loss, cost)
Scalability in current standards (few spatio-temporal operating points MPEG-2, coarse or fine grain quality scalability MPEG-4, performance loss)
Scalable video coding
Code once decode at S x T x Q operating points (full scalability in space, frame-rate and quality)
R. Leonardi - A. Signoroni University of Brescia, Italy

4. SVC scenarios and applications
Multi-channel content production and delivery:
the same stream will be viewed on a variety of devices with different spatial and/or temporal resolution
will be viewed and charged on the basis of delivered quality level
devices can decide to trade-off received quality for a longer battery life (complexity scalability)
Video streaming over heterogeneous IP networks
Advanced surveillance systems
Video conferencing
Mobile streaming video
Others...
R. Leonardi - A. Signoroni University of Brescia, Italy

5. MPEG-21 Ad Hoc Group on SVC
at Pattaya Meeting (Dec.2001) MPEG started an AHG to explore Interframe Wavelet Coding – activity successfully completed in Oct.2002
RPI software (prof. J.W.Woods) released based on MCTF+EZBC
MPEG started an AHG on Scalable Video Coding (MPEG-21)
Call for Evidence on SVC: 9 responses received by July 2003
Call for Proposals on SVC: 14 full proposals (Wavelet based, AVC based)
Wavelet based: best selected Microsoft Research Asia (software released)
AVC based: best selected HHI-Fraunhofer
Core Experiments started at the Monaco (Mar.2004) and Redmond (Jul.2004) meetings.
No reference architecture yet specified (MRSA, HHI, Samsung, UNIBS, DANAE-TOMPHSON, UNSW proposals at the next meeting Palma de Mallorca Oct.04)
R. Leonardi - A. Signoroni University of Brescia, Italy

6. Requirements shall should may
Last revision: MPEG output doc N6505, Redmond, July 2004
shall
should
may
Spatial
scalability
2 levels
(e.g. CIF+QCIF)
4 levels
From QQCIF to 3610x1536
Temporal scalability
(up to 60 Hz)
Non-dyadic
Quality (SNR) scalability
Coarse grain (25% bit-rate)
Medium grain
(10% bit-rate)
FGS or progressive
R. Leonardi - A. Signoroni University of Brescia, Italy

7. R. Leonardi - A. Signoroni University of Brescia, Italy
Requirements contd.
Combined Scalability (at least for a finite number of points in the spatial-temporal-quality space)
Challenging Reqs
Coding Efficiency performance (superior to H.264/AVC)
Base-layer compatibility (to H.264/AVC)
Complexity scalability (for power limited devices)
Others (less peculiar to SVC)
ROI and object scalability, robustness to transmission errors and on “best-efforts” networks, end-to-end delay, random access capability)
Figura a toni di grigio
R. Leonardi - A. Signoroni University of Brescia, Italy

8. Req. + Core Experiment (oct’04)
Multiple Adaptation Scalability: “decoding paths”
Figura a toni di grigio
R. Leonardi - A. Signoroni University of Brescia, Italy

9. A generic SVC encoder
Esempio con immagini e ricordo di contour preservation con nota critica sulla dinamica
Sergio p.55
In order to have an efficient scalable codec, it is necessary that
each one of the building blocks is operating in a scalable mode.
MPEG-21 doc N6520 Scalable Video Model 2.0 July 2004, Redmond, WA, USA
R. Leonardi - A. Signoroni University of Brescia, Italy

10. SVC Enabling Technologies
In the MPEG21-SVC framework a series of SVC enabling technologies have been identified
Spatial transform: DWT (Discrete Wavelet Transform) 9x7; IntegerDWT 5x3; Decimation/Interpolation; Block based transforms
Temporal transform: Haar MCTF (Motion Compensated Temporal Filtering); 5x3 MCTF; UnconstrainedMCTF; …
Motion coding: non-scalable; multires. quality scalable explicit (inside the method) / implicit (on the data structure); hierarchical B-frames; …
Texture coding: EBCOT, 3D-EBCOT; EZBC; layered H.264/AVC; SNR scalable CABAC based entropic coding; …others (e.g. embedded wavelet coders).
Figura p.58 sergio
R. Leonardi - A. Signoroni University of Brescia, Italy

11. R. Leonardi - A. Signoroni University of Brescia, Italy
SVC implementations
Following the Call for Proposal for SVC (MPEG doc.N5958 Oct.2003) a certain number of implementations have been presented (MPEG doc.M10569 Mar.2004).
On the base of the CfP inputs a SVC classification has been made which allows to distinguish some families of SVC implementations:
“t+2D” schemes
“2D+t” schemes
“2D+t+2D” schemes or “Multi-scale Pyramid t+2D”
Our STool scheme could be classified in the third category
Figura p.58 sergio
R. Leonardi - A. Signoroni University of Brescia, Italy

12. R. Leonardi - A. Signoroni University of Brescia, Italy
t+2D schemes
Estensione 3d e proprietà
Enables
temporal scalability
Enables
spatial scalability
Enables
quality scalability
R. Leonardi - A. Signoroni University of Brescia, Italy

13. R. Leonardi - A. Signoroni University of Brescia, Italy
2D+t schemes
Estensione 3d e proprietà
R. Leonardi - A. Signoroni University of Brescia, Italy

14. UniBS SVC scheme: “STool”
Estensione 3d e proprietà
2 SYSTEMS which differs in the EC part
3D-ESCOT (MSRA software)
GOF-EMDC (UNI-BS)
MEC (Motion Estimation and Coding)
+ MCTF (MSRA software)
R. Leonardi - A. Signoroni University of Brescia, Italy

15. Results: CITY QCIF 15 Hz 64 kbps
Original
AVC
STool
zoom
zoom
Criteri di filtraggio e esempio slices
R. Leonardi - A. Signoroni University of Brescia, Italy

16. R. Leonardi - A. Signoroni University of Brescia, Italy
Results:
CITY CIF
30 Hz 256 kbps
Original
AVC
Stool
Criteri di filtraggio e esempio slices
R. Leonardi - A. Signoroni University of Brescia, Italy

17. Results: AVC vs MSRA vs Stool
Play
QCIF 15Hz 96kbps
CIF 30Hz 384kbps
HARBOUR
Criteri di filtraggio e esempio slices
QCIF 15Hz 64kbps
CIF 30Hz 256kbps
CITY
R. Leonardi - A. Signoroni University of Brescia, Italy

18. R. Leonardi - A. Signoroni University of Brescia, Italy
Conclusions
SVC systems
Multipoint spatial-temporal-SNR scalability
Comparable or superior in coding performance with respect to AVC
A lot of work is currently made on SVC (AVC is mature, SVC is young… :-)
STool
STool: a new 2D+t+2D system with good performance (UNIBS submission to the next MPEG meeting)
Rivedere i prodotti di e la rappresentazione ad albero, commentare i vari utilizzi, inserire immagini (e.g. sclerosi multipla nell’object analysis, reni nella semplificazione…)
R. Leonardi - A. Signoroni University of Brescia, Italy