Slice Interleaving in Compressed Video Packetization Victor Varsa and Marta Karczewicz Visual Communications Lab. Nokia Research Center

Outline IntroductionIntroduction Previous presented schemePrevious presented scheme –Adaptive packetization Proposed SchemeProposed Scheme –Slice interleaving pattern –Intra frame interleaving pattern Packetizer operationPacketizer operation Simulation ResultsSimulation Results ConclusionConclusion

Introduction Over a packet oriented transport system, it ’ s possible to lose packets.Over a packet oriented transport system, it ’ s possible to lose packets. Without packet loss re-transmission, to packet a bitstream properly is important.Without packet loss re-transmission, to packet a bitstream properly is important. –trade-off between packet loss effect and packet header overhead –Independency between all packet –Error isolation and data partition

Adaptive Packetization Reference picture current picture packet

Adaptive Packetization GOB is packeted with it ’ s reference GOBsGOB is packeted with it ’ s reference GOBs Every packet contains at least one GOB which is INTRA modeEvery packet contains at least one GOB which is INTRA mode If one packet is lost, error will restrict in specific regionIf one packet is lost, error will restrict in specific region Information between GOB??Information between GOB??

Proposed Scheme The lost MBs in an inter frame can be reconstructed using the approach of temporal or/and spatial concealment.The lost MBs in an inter frame can be reconstructed using the approach of temporal or/and spatial concealment. Packetization SchemePacketization Scheme –The frame area (number of MBs) covered by one slice is small –Puts several slices in 1 transport packet SliceSlice –GOB

Proposed Scheme (cont.) 11 MBs …… 8 9MBs QCIF (176 * 144) 9 slices Slice structure:Slice structure: –1 row of MBs of a frame Only slices are put into the same packetOnly full independently slices are put into the same packet Slice will not divide into 2 or more packetSlice will not divide into 2 or more packet

No neighbor condition Slices in a packet should be provided with no neighbor condition:Slices in a packet should be provided with no neighbor condition: –Spatial neighbors –Temporal neighbor –Spatial neighbors of temporal neighbors There shouldn ’ t be slices from consecutive frames lost at the same positionThere shouldn ’ t be slices from consecutive frames lost at the same position

No neighbor condition (cont.) T-1 T+1 T Current Slice Spatial neighbor (1) Temporal neighbor (2) Spat-temp neighbor (3)

Interleaving Pattern BinBin –Buffer to store the compressed bitstream of the slice –Packets for transmission are formed from bins Interleaving PatternInterleaving Pattern –Order of slices to put into one bin –Number of interleaving patterns depend on “ no neighbor condition ” we used

Interleaving Pattern (cont.) ExampleExample T0 T1T2 B0 B1 B2 ……

Interleaving Pattern (cont.) Example 2:Example 2: –Temporal neighbor is included in the same bin –Only Two Bins needed

Proposed Slice Interleaving Use consecutive 9 frame as a periodUse consecutive 9 frame as a period One bin contains slices from these 9 framesOne bin contains slices from these 9 frames Slice pattern of B0 :Slice pattern of B0 : –T0S0, T1S3, T2S6, T3S1, T4S4, T5S7, T6S2, T7S5, T8S8

Intra frame interleaving pattern Packets containing slices of different frame types have different priorityPackets containing slices of different frame types have different priority The interleaving pattern should collect slices from only same frame types.The interleaving pattern should collect slices from only same frame types. The concealment algorithm use only spatial information for concealing lost Intra frame MBsThe concealment algorithm use only spatial information for concealing lost Intra frame MBs The intra interleaving pattern creates 2 bins for a frame with odd and even slices separately.The intra interleaving pattern creates 2 bins for a frame with odd and even slices separately.

Packetization Packetizer design constraintsPacketizer design constraints –Target size of transport packets –Maximum allowed delay –Methods to combat loss of consecutive packet No mention in this paperNo mention in this paper

Packetization (cont.) Packetization AlgorithmPacketization Algorithm 1.Take next slice S from the encoder 2.Select the bin B where to put S according the interleaving pattern 3.If “ complete ” condition of bin B is fulfilled, from packet out of B and send 4.If not end of stream, go to 1, else empty all bins by forming packets and send

Simulation Result

Simulation Result (cont.)

Conclusion One new slice interleaving packetization scheme for transmitting compressed video over internet.One new slice interleaving packetization scheme for transmitting compressed video over internet. This Scheme is robust against packet loss well.This Scheme is robust against packet loss well.