Video over the Internet Introduction Video & Internet: the problems Solutions & Technologies in use Discussion
Video & Internet: the problems Video Bandwidth needs Internet Delays Packet loss Delay jitter Congestion control
Internet Delays Heavy traffic load and internal transmission problems can cause delays. using RTP to handle the time- related media delivery issues. video has high tolerance to packet losses.
Packet loss Pack losing is inevitable ALF --A well-established principle for loss-resilient transmission of video over the Internet. Scalable or layered coding base layer enhancement layers
Delay Jitter The queuing delay may vary significantly during a session creating delay jitter. Play out buffers initial minimum buffer size. change the buffer size in real time
Congestion Control A vital parameter is the bottleneck bandwidth Rate control: control encoding rate in a predictable way in adapting its output network condition. No globally accepted solution.
Solutions & Technologies in use Video stream properties for network use Video compression
Video stream properties for network use Natural breakpoints for packetization Adjustable packet sizes No bit level shifts during packetization Well defined high-priority information Flexible rate control Ease of transcoding Layered coding Resilience to error propagation
Video compression Introduction Video compression techniques– RTP Payload formats -- H.261 -- Motion-JPEG -- MPEG-1 & MPEG-2 -- H.263+
H.261 Coding is organized as a hierarchy of groupings: Frame (picture) level : specifies the delay from the previous frame, the image format, and various indicators. GOB level: specifies the GOB number and the default quantifier that will be used for the MBs. MB level: specifies which blocks are present and which did not change, and optionally a quantifier and motion vectors. MB is taken as the unit of fragmentation
H.261 payload specific header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |SBIT |EBIT |I|V| GOBN | MBAP | QUANT | HMVD | VMVD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ♪ Start bit position (SBIT): 3 bits ♪ End bit position (EBIT): 3 bits ♪ INTRA-frame encoded data (I): 1 bit ♪ Motion Vector flag (V): 1 bit ♪ GOB number (GOBN): 4 bits ♪ Macro block address predictor (MBAP): 5 bits ♪ Quantizer (QUANT): 5 bits ♪ Horizontal motion vector data (HMVD): 5 bits ♪ Vertical motion vector data (VMVD): 5 bits
Enhancing the packet loss resilience 3 ways to mitigate packet loss (1) Use only INTRA-frame encoding and MB level conditional replenishment. (2) Adjust the INTRA-frame encoding refreshment rate according to the packet loss observed by the receivers. (3) The fastest way is to request an INTRA-frame coded image refreshment after a packet loss is detected.
Use of optional H.261-specific control packets Two H.261-specific RTCP control packets (RTP’s companion control protocol). Full INTRA-frame Request Negative Acknowledgement The purpose: speed up refreshment of the video. Caution of usage: Those control packets are sent directly via unicast from the decoder to the coder. effective only when the number of receivers is small.
H.261 One of the earliest RTP payload formats for packet video transport over the internet. Demonstrates a successful approach to packetizing a video payload that was not originally designed for packet networks. It is a constant-bit-rate encoding and effective only when the number of receivers is small.
Motion JPEG What is Motion-JPEG Review of JPEG Payload format
Overview of JPEG The image =>one or more frames=> one or more scans=> one to four components. (3 color components). The scan contains an ordered sequence of MCUs. Each MCU defines the image data for a small rectangular block of the output image. Each frame and scan is preceded with a header containing optional definitions for compression parameters.
Three Compressed data formats Interchange format: Contains definitions for all the tables used by the entropy-coded segments. JPEG File Interchange Format (JFIF) is a standard that provides extra information. Abbreviated format: Might omit some assuming they were defined out-of-band or by a "previous" image. Table-specification format
Restart markers in the JPEG Denote a point where the decoder should reset its state. restart interval: a block of data containing a restart marker followed by some fixed number of MCUs.
JPEG over RTP The RTP payload format for MJPEG: Optimized for—JPEG codec parameters change rarely from frame to frame. Assumes JPEG’s sequential DCT operating mode and single-scan, interleaved images. RTP/JPEG data format: abbreviated format
RTP/JPEG Packet Format Each packet contains RTP header Special JPEG header Main JPEG header A Restart Marker header and/or Quantization Table header
Main JPEG header Type-specific: 8 bits Interpretation depends on the value of the type field. Fragment Offset: 24 bits The offset in bytes of the current packet in the JPEG frame data. Type field: 8 bits Defines the abbreviated table-specification and additional JFIF-style parameters not defined by JPEG data stream. Determines whether restart markers are present. Q field: 8 bits Defines the quantization tables. Determines whether a Quantization Table header appears. Width & Height : 8 bits/ each Encodes the width and height of the image in 8-pixel multiples.
Restart Marker header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Restart Interval |F|L| Restart Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Restart Interval field: 16 bits Specifies the number of MCUs that appear between restart markers. F (first) and L (last) fields: 1 bit/ each Restart Count bit : 14 bits The value of F, L and Restart Count bit can be used to indicate that a receiver MUST reassemble the entire frame before decoding it.
Discussions Codecs designed for the Internet : greater scalability lower computational complexity greater resiliency to network losses lower encode/decode latency The network properties must be considered jointly.