1 Single Reference Frame Multiple Current Macroblocks Scheme for Multiple Reference IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY Tung-Chien Chen, Chuan-Yung Tsai, Yu-Wen Huang, and Liang-Gee Chen, Fellow, IEEE

2 Outline Introduction Fundamentals and Problem Statement Proposed Data Reuse Scheme Proposed Framework for SRMC Scheme Simulation Results and Performance Evaluation

3 Introduction The H.264/AVC can save 25%-45% and 50%-70% of bitrates when compared with MPEG-4 advanced simple profile and MPEG-2, respectively, but higher computation and memory bandwidth. The inter prediction occupies over 95% of the computational resource, which is mainly caused by multiple reference frames motion estimation (MRF-ME).

4 Introduction We propose a new frame-level DR (data reuse) scheme. With the frame-level rescheduling, the data of one loaded SW (search window) can be reused by multiple current MBs in different original frames for MRF-ME, and the system bandwidth and local memory size is greatly reduced.

5 Fundamentals and Problem Statement A. Inter Prediction in H.264/AVC For variable block size ME (VBS-ME), there are 41 different blocks within one MB and gives rise to a large number of possible combinations. Lagrangian mode decisions : The Lagrangian cost function considers both the distortion and the rate parts. Distortion : sum of absolute differences (SAD) Rate : the number of bits required to code the reference frame number and the motion vectors (MVs).

6 Fundamentals and Problem Statement B. Conventional Data Reuse Scheme and Problem Statement The bandwidth between system memory and ME core is very heavy if all required pixels are loaded from system memory. A common solution is to design local buffers to store reusable data.

7 Fundamentals and Problem Statement B. Conventional Data Reuse Scheme and Problem Statement When the ME of MB-a is finished, and the MB-b will be processed, only the reference pixels in D are loaded to replace A in the local memory.

8 Fundamentals and Problem Statement B. Conventional Data Reuse Scheme and Problem Statement There are four SW memories, and each SW memory will be independently loaded and updated.

9 Fundamentals and Problem Statement B. Conventional Data Reuse Scheme and Problem Statement The hardware cost is almost proportional to the maximum reference frame number. The more system bandwidth requirement, the more power consumption. The more memory size, the more silicon area and cost.

10 Proposed Data Reuse Scheme A. Frame-Level Data Reuse

11 Proposed Data Reuse Scheme A. Frame-Level Data Reuse In MRSC (multiple reference frames single current macroblock) scheme, one current MB is loaded only one time, and one reference SW is loaded several times. In SRMC (single reference frame multiple current macroblocks) scheme, one current MB is loaded several times while one reference SW is only loaded once. Since the SW is much larger than one MB, both the bandwidth and memory size can be largely reduced.

12 Proposed Data Reuse Scheme B. Frame-Level Rescheduling It is assumed that there are six MBs in each frame and five P-frames to be coded. The maximum reference frame number is four.

13 Proposed Data Reuse Scheme B. Frame-Level Rescheduling The first, second, third and fourth ME cubes in one column represent the step-1, step-2, step-3, and step-4 searching processes in Fig.3.

14 Proposed Data Reuse Scheme B. Frame-Level Rescheduling The first, second, third, and fourth ME cubes in one vertical column represents the step-1, step-2, step-3, and step-4 searching processes in fig. 4.

15 Proposed Framework for SRMC Scheme A. Mode Decision for SRMC Scheme The problem of inaccurate mode decision will occur after the block-level data reuse with the parallel hardware and frame- level rescheduling with the proposed SRMC scheme.

16 Proposed Framework for SRMC Scheme A. Mode Decision for SRMC Scheme In the reference software (JM8.5), the Lagrangian cost function takes MV costs into consideration. The MV of each block is generally predicted by the medium value of MVs from the left, top, and top-right neighboring blocks. The exact MVPs of variable blocks are changed to the medium of MVs of the left, top, and top-right MBs in order to facilitate the parallel processing with block-level data reuse.

17 Proposed Framework for SRMC Scheme A. Mode Decision for SRMC Scheme

18 Proposed Framework for SRMC Scheme A. Mode Decision for SRMC Scheme The mode decision flow is divided into partial mode decision (PMD) and final mode decision (FMD) as Table II. The MVs and the distortion costs of these suboptimal results are written to the external memory. After the PMD results of all reference frames are generated for a certain current MB, the FMD decides the best combination of variable blocks in different reference frames with system RISC.

19 Proposed Framework for SRMC Scheme B. Architecture Design The ME core computes the candidates ’ distortion value, and the PMD engine on-line decides the MVs of variable blocks according to the estimated MVPs. The PMD results are buffered at system memory, and then the RISC performs FMD.

20 Proposed Framework for SRMC Scheme B. Architecture Design

21 Proposed Framework for SRMC Scheme B. Architecture Design The SW at the frame t-4 is loaded to SW buffer first. Then, the ME task of the current MB in the frame t-3 will be performed. After that, the FMD of this current MB is then done by RISC after the PMD results are written out. At the same time, the current MBs at the same location of the following frames as t-2, t-1, t are processed one after another.

22 Simulation Results and Performance Evaluation A. Simulation Results Four sequences: Foreman, Mobile, Akiyo, and Stefan. The encoding parameters are Baseline profile, IPPP … structure, four reference frames, 16-pel search range, and low complexity mode decision.

23 Simulation Results and Performance Evaluation A. Simulation Results

24 Simulation Results and Performance Evaluation B. Performance Evaluation MRSC: SRMC: The includes the MVs and the matching costs of variable blocks. So it is relatively small.

25 Simulation Results and Performance Evaluation B. Performance Evaluation The increases with larger search range. Therefore, the proposed SRMC scheme has better performance for the videos the larger frame sizes that inherently require larger search range.

26 Conclusion By frame-level rescheduling, the procedures for multiple current MBs in different original frames can simultaneously utilize the data of single SW. In the proposed framework, the SRMC scheme reduces not only 63% of external system bandwidth but also 75% of internal memory size for HDTV specifications.