1. Motion Estimation and Prediction
Present by :fakewen

2. Advanced motion vector prediction
AMVP
It allows the selection of the best predictor from
three spatially adjacent motion vectors
their median
temporal motion vector.

4. #if PART_MRG
Bool bTestNormalMC = true;
if (pcCU->getWidth( 0 ) > 8 && iNumPart == 2 && iPartIdx == 0)
bTestNormalMC = false;
if (bTestNormalMC) {
#endif

5. Motion Estimation and Prediction
主要分布位置
Tcomprediction.cpp
TEncSearch.cpp

6. predInterSearch

7. I frame
xMotionEstimation ( pcCU, pcOrgYuv, iPartIdx, eRefPicList, &cMvPred[iRefList][iRefIdxTemp], iRefIdxTemp, cMvTemp[iRefList][iRefIdxTemp], uiBitsTemp, uiCostTemp );
I frame自己重算motion vector
I frame
I frame

8. P frame I frame pcCU->getSlice()->getNoBackPredFlag()==1 P frame
enum RefPicList {
REF_PIC_LIST_0 = 0, ///< reference list 0
REF_PIC_LIST_1 = 1, ///< reference list 1
#if DCM_COMB_LIST
REF_PIC_LIST_C = 2, ///< combined reference list for uni-prediction in B-Slices
#endif
REF_PIC_LIST_X = 100 ///< special mark };
如果很smooth且cost不高
->SKIP MODE
直接拿之前的Motion vector來用
否則再從他的ref frame list裡找

9. B frame I frame P frame 如果很smooth且cost不高 ->SKIP MODE
直接拿之前的Motion vector來用
否則再從他的ref frame list裡找

11. Advanced motion vector prediction
AMVP
It allows the selection of the best predictor from
three spatially adjacent motion vectors(a,b,c)
their median
temporal motion vector.
xCheckBestMVP
計算哪種的cost較低

12. xEstimateMvPredAMVP Step 1 check
AMVP mode還有number of motion vector predictor candidates
for ( i = 0 ; i < pcAMVPInfo->iN; i++)
算uiBestCost
rcMvPred = pcAMVPInfo->m_acMvCand[pcCU->getMVPIdx(eRefPicList,uiPartAddr)];
從裡面選一個最像的當Mvpredict
並記錄cost還有選到的鄰居

13. Merge xMergeEstimation 算出最小的cost的合併方式 xMergeEstimation(…);
UInt uiMRGCost = uiMRGError + m_pcRdCost->getCost( uiMRGBits );
if ( bMergeValid && uiMRGCost < uiMECost )
合併!

14. xMotionEstimation if ( !m_iFastSearch || bBi ) xPatternSearch
xPatternSearchFast
Mv初始值用傳入參數
xTZSearch
xPatternSearchFracDIF( pcCU, pcPatternKey, piRefY, iRefStride, &rcMv, cMvHalf, cMvQter, ruiCost )

15. xMotionEstimation
xSetSearchRange ( pcCU, cMvPred, iSrchRng, cMvSrchRngLT, cMvSrchRngRB );
先設定一下range(P frame)
if ( !m_iFastSearch || bBi )
xPatternSearch
xPatternSearchFast
Mv初始值用傳入參數
xTZSearch
xPatternSearchFracDIF( pcCU, pcPatternKey, piRefY, iRefStride, &rcMv, cMvHalf, cMvQter, ruiCost )
rcMv <<= 2;
rcMv += (cMvHalf <<= 1);
rcMv += cMvQter;
紀錄結果

16. xTZSearch
range
test whether one of PRED_A, PRED_B, PRED_C MV is better start point than Median predictor PK
raster search if distance is too big
idist
raster refinement
P.s. 若idist==1 則只考慮上下左右

17. raster refinement
Idist too big

18. motionCompensation Step 1 Check iPartIdx >= 0 Step 2
eRefPicList!= REF_PIC_LIST_X(B frame)
xPredInterLumaBlk
xPredInterBi
xPredInterUni
xPredInterChromaBlk
xPredInterLumaBlk_ha 畫圖
Motion compensation
可以處理到subpixel(1/4)
用filter處理
xPredInterChromaBlk_ha

19. motionCompensation
Void motionCompensation ( TComDataCU* pcCU, TComYuv* pcYuvPred, RefPicList eRefPicList = REF_PIC_LIST_X, Int iPartIdx = -1 );初始
Motion
Compensation
pcCU->getPartIndexAndSize( iPartIdx, uiPartAddr, iRoiWidth, iRoiHeight );
iPartIdx >= 0?
getPartIndexAndSize
For(pcCU->getNumPartInter())
eRefPicList != REF_PIC_LIST_X ?
eRefPicList看是I還是P frame
eRefPicList不在REF裡
xPredInterUni
xPredInterBi

20. // set motion
cMv[iRefList] = cMvTemp[iRefList][iRefIdxTemp];
iRefIdx[iRefList] = iRefIdxTemp;
pcCU->getCUMvField(eRefPicList)->setAllMvField( cMv[iRefList], iRefIdx[iRefList], ePartSize, uiPartAddr, iPartIdx, 0 );
紀錄最後結果
接著做AMVP
xEstimateMvPredAMVP( pcCU, pcOrgYuv, iPartIdx, eRefPicList, iRefIdxTemp, cMvPred[iRefList][iRefIdxTemp]);
aaiMvpIdx[iRefList][iRefIdxTemp] = pcCU->getMVPIdx(eRefPicList, uiPartAddr);
aaiMvpNum[iRefList][iRefIdxTemp] = pcCU->getMVPNum(eRefPicList, uiPartAddr);
uiBitsTemp += m_auiMVPIdxCost[aaiMvpIdx[iRefList][iRefIdxTemp]][aaiMvpNum[iRefList][iRefIdxTemp]];

21. predInterSearch SPS: sequence parameter set predInterSearch
pcCU->getPartIndexAndSize( iPartIdx, uiPartAddr, iRoiWidth, iRoiHeight );得到這個block的info
if (pcCU->getWidth( 0 ) > 8 && iNumPart == 2 && iPartIdx == 0)
bTestNormalMC = false;
For(iNumPredDir)
FOR(pcCU->getSlice()->getNumRefIdx(eRefPicList)
iNumPredDir = pcCU->getSlice()->isInterP() ? 1 : 2;
P是1 B是2
RefPicList 標示現在是IPB
enum RefPicList {
REF_PIC_LIST_0 = 0, ///< reference list 0
REF_PIC_LIST_1 = 1, ///< reference list 1
#if DCM_COMB_LIST
REF_PIC_LIST_C = 2, ///< combined reference list for uni-prediction in B-Slices
#endif
REF_PIC_LIST_X = 100 ///< special mark };
RefPicList eRefPicList = ( iRefList ? REF_PIC_LIST_1 : REF_PIC_LIST_0 );
iRefList//若是I frame則0
pcCU->getSlice()->getNoBackPredFlag()…
判斷式
判斷式
pcCU->getSlice()->getNoBackPredFlag() || pcCU->getSlice()->getSPS()->getUseLDC()
cMvTemp[1][iRefIdxTemp] = cMvTemp[0][iRefIdxTemp];
SKIP MODE
xMotionEstimation ( pcCU, pcOrgYuv, iPartIdx, eRefPicList, &cMvPred[iRefList][iRefIdxTemp], iRefIdxTemp, cMvTemp[iRefList][iRefIdxTemp], uiBitsTemp, uiCostTemp );
I frame自己重算motion vector
uiCostTemp = uiCostTempL0[pcCU->getSlice()->getRefIdxOfL0FromRefIdxOfL1(iRefIdxTemp)];
用相近的block

22. predInterSearch(cont.)
// set motion
cMv[iRefList] = cMvTemp[iRefList][iRefIdxTemp];
iRefIdx[iRefList] = iRefIdxTemp;
pcCU->getCUMvField(eRefPicList)->setAllMvField( cMv[iRefList], iRefIdx[iRefList], ePartSize, uiPartAddr, iPartIdx, 0 );
紀錄最後結果
接著做AMVP
xEstimateMvPredAMVP( pcCU, pcOrgYuv, iPartIdx, eRefPicList, iRefIdxTemp, cMvPred[iRefList][iRefIdxTemp]);
aaiMvpIdx[iRefList][iRefIdxTemp] = pcCU->getMVPIdx(eRefPicList, uiPartAddr);
aaiMvpNum[iRefList][iRefIdxTemp] = pcCU->getMVPNum(eRefPicList, uiPartAddr);
uiBitsTemp += m_auiMVPIdxCost[aaiMvpIdx[iRefList][iRefIdxTemp]][aaiMvpNum[iRefList][iRefIdxTemp]];
for( UInt ui = 0; ui < MRG_MAX_NUM_CANDS; ui++ ) {
pcCU->setNeighbourCandIdxSubParts( ui, ucNeighCand[ui], uiPartAddr, iPartIdx, pcCU->getDepth( uiPartAddr ) ); }
if ( bMergeValid && uiMRGCost < uiMECost )
// 可以merge且merge cost較低>>set Merge result
else
// set ME result記錄自己的motion estimation

23. xEstimateMvPredAMVP
if( pcCU->getAMVPMode(uiPartAddr) == AM_NONE || (pcAMVPInfo->iN <= 1 && pcCU->getAMVPMode(uiPartAddr) == AM_EXPL) )
no AMVP mode或是number of motion vector predictor candidates小於一
來亂的
if (pcCU->getAMVPMode(uiPartAddr) == AM_EXPL && bFilled)
for ( i = 0 ; i < pcAMVPInfo->iN; i++)
算uiBestCost
rcMvPred = pcAMVPInfo->m_acMvCand[pcCU->getMVPIdx(eRefPicList,uiPartAddr)];
從裡面選一個最像的當Mvpredict
並記錄cost還有選到的鄰居

24. xMotionEstimation
xSetSearchRange ( pcCU, cMvPred, iSrchRng, cMvSrchRngLT, cMvSrchRngRB );
先設定一下range(P frame)
if ( !m_iFastSearch || bBi )
xPatternSearch
xPatternSearchFast
Mv初始值用傳入參數
xPatternSearchFracDIF( pcCU, pcPatternKey, piRefY, iRefStride, &rcMv, cMvHalf, cMvQter, ruiCost )
rcMv <<= 2;
rcMv += (cMvHalf <<= 1);
rcMv += cMvQter;
紀錄結果

25. motionCompensation
Void motionCompensation ( TComDataCU* pcCU, TComYuv* pcYuvPred, RefPicList eRefPicList = REF_PIC_LIST_X, Int iPartIdx = -1 );初始
Motion
Compensation
pcCU->getPartIndexAndSize( iPartIdx, uiPartAddr, iRoiWidth, iRoiHeight );
iPartIdx >= 0?
getPartIndexAndSize
For(pcCU->getNumPartInter())
eRefPicList != REF_PIC_LIST_X ?
eRefPicList看是I還是P frame
eRefPicList不在REF裡
xPredInterUni
xPredInterBi

26. combine

27. taco

28. Inter Block Coding Extended block size motion partitions
Geometric motion partitions
Higher precision MV representation
Improved sub-pixel interpolation
他口Taco

29. Extended block size motion partitions
Advantage to allow for motion partition sizes larger than 16x16
Set largest motion partition size is 64x64
=> 64x64 64x32 32x64 32x32
=> 32x32 32x16 16x32 16x16
=> Existing motion partition sizes in H264
每一個block 都要傳一個MV 所以說 若允許越大的block size motion partition 則傳的bit也可以減少
最上面那一層就是從 64*64的block size 開始 會比 如果是選32x32 就會 繼續下一層
最後 就會跟AVC/H.264一樣 (16*16 16*8 8*16 8*8 8*4 4*8 4*4)
那真正用的時候 是bottom-up search 如果原本H264是16*16 那就會再往上看32x32 32x16 16x32 16x16
看哪個RD cost比較小
他口Taco

30. Inter Block Coding Extended block size motion partitions
Geometric motion partitions
Higher precision MV representation
Improved sub-pixel interpolation
他口Taco

31. Geometric motion partitions
Divides the block into two regions by a straight line.
Define by the angle subtended by the perpendicular line with x axis (θ) and the distance (ρ)
Motion model 都假設是長方形或正方形的 block 移動 但是其實不是
而且當我們做了剛剛的事情 extend block size motion partition 此問題會更嚴重
那一個方法當然是把block切小一點 但是MV會變多!~
所以 他這邊提出 Geometric motion partitions
而這條線的方程式如下
而這邊用兩個 32 bit lookup table 一個存-1/tano 另一個存 1/sino 所以用查表就行了
而teta 和饕 各有這麼多種可能 所以在16*16 有256種可能……
θ : 32 different values
ρ : 8 values (16x16) =>256
16 values (32x32) =>512
32 values (64x64) =>1024
他口Taco

32. Geometric motion partitions
Reuse rectangular partitions
Find the largest rectangular block that lies entirely inside the region => MV for the partition region
Motion cost : SAD
For example:
16x16 blocks
256 possible geometric partition
Calculate motion cost
16 best motion costs
Full enhanced predictive zonal search(EPZS) to find the lowest RD cost one
而這種 歪歪斜斜用線切的 geometric motion partitions 的motion search又要怎麼做呢
那如果用這種奇怪形狀直接去找的話 很expensive =>
Reuse rectangular partitions
就找整個躺在partition region的最大rectangular block 其MV為整partition region的MV
而 32*32 和64*64 的blocks 為了減少計算量 會將 可能的geometric partitions數量減少
他口Taco

33. Geometric motion partitions
Reuse rectangular partitions
Find the largest rectangular block that lies entirely inside the region => MV for the partition region
For 32x32 and 64x64
Sub sampling ρ and θ by 2
ρ =8 θ =16 (32x32) 128
ρ =16 θ =16 (32x32) 256
而 32*32 和64*64 的blocks 為了減少計算量
他口Taco

34. Geometric motion partitions
Deblocking may not reduce this.
Define boundary pixels
Not a boundary pixels
Normal motion compensation
Boundary pixel
=> Weighted sum of 2/3 for belonging region MV 1/3 other region MV
在同一個block 卻有兩個region 有不同的MV 所以在region分界的地方
應該是有超大的discontinuities 就很像block effect一樣 對人眼產生副作用
很直觀的
假如此pixel上下左右鄰近的pixels 是另外一個region的 => boundary pixels
他口Taco

35. Inter Block Coding Extended block size motion partitions
Geometric motion partitions
Higher precision MV representation
Improved sub-pixel interpolation
他口Taco

36. Motion accuracy
Separate filters to perform 1/4th and 3/4th pixel interpolation
And have higher accuracy MV(1/8th pixel)
For every block in a motion partition, first a 1/4th pixel accuracy MV is found using EPZS
比H264用bilinear interpolation 準多了
且準確度達到 1/8 個pixel
所以對每個block 做motion partition時 一開始用EPZS 來算所有1/4th的MV 看誰最準
在最準的那個點附近再長1/8的點 成為最後的MV
他口Taco

37. Inter Block Coding Extended block size motion partitions
Geometric motion partitions
Higher precision MV representation
Improved sub-pixel interpolation
他口Taco

39. Simple QuadTree Motion and Transform Structure
The quadtree based prediction representation was common among many proposals to HEVC.
But it is configurable so that it can operate in the simplified low complexity mode.

40. Simple QuadTree Motion and Transform Structure
Large macroblocks are employed in addition to traditional 16x16 macroblocks.
Restricts the quadtree structure of large macroblocks to inter-picture coding modes and only one motion vector.(16x16  16x8 8x16 8x8)
Motion partition sizes smaller than 8x8 are disabled, reducing decoder complexity.

41. Simple QuadTree Motion and Transform Structure
Higher motion and detailed texture are coded using smaller macroblocks.
Smooth areas are either coded with SKIP macroblocks of size 16x16 pixels or with macroblocks of size 32x32 64x64 pixels.順的，沒啥變化的地方用SKIP
For transform sizes larger than 8x8, the proposal utilizes truncated transforms where only the 8x8 low frequency coefficients are calculated.

42. Simple QuadTree Motion and Transform Structure
Gray skip
原來顏色 找motion
Green intra 自己壓

43. fakewen

44. Improved Techniques for Motion Representation
For fractional-sample motion-compensated prediction, a fixed-point implementation of the maximal-order-minimum-support 有用到嗎?algorithm is presented that uses a combination of infinite impulse response and FIR filtering.
用arithmetic coding?
infinite impulse response and FIR filtering?

45. Motion-Compensated Prediction
Fractional-Sample Interpolation Using MOMS
Generalized interpolation using MOMS
Choice of MOMS basis functions
Implementation aspects of cubic and quintic O-MOMS
Interleaved Motion-Vector Prediction
Merging of Motion-Compensated Predicted Blocks

46. Interleaved Motion-Vector Prediction
In order to reduce the bit rate required for transmitting the motion vectors
first step, the vertical motion vector component is predicted using conventional median prediction
Then, only those motion vectors of neighboring blocks for which the absolute difference between their vertical component and the vertical component for the current motion vector is minimized are used for the prediction of the horizontal motion-vector component
first step, the vertical motion vector component is predicted
using conventional median prediction (as in [1]), and the
corresponding prediction residual, i.e., the difference between
the actual vertical component and its prediction, is coded.
Then, only those motion vectors of neighboring blocks for
which the absolute difference between their vertical compo-
nent and the vertical component for the current motion vector
is minimized are used for the prediction of the horizontal
motion-vector component. Interleaving the prediction of the
motion-vector components and the actual coding of related
residuals in such a way leads to an overall increased coding
efficiency.

47. Merging of Motion-Compensated Predicted Blocks
However, in general, quadtree-based block partitioning may result in an over-segmentation due to the fact that, without any further provision, at each interior node of a quadtree, four subblocks are generated while merging of blocks is possible only by pruning complete branches consisting of at least four child nodes in the parent-child relationship within a quadtree.

48. skill

49. Inter Prediction Motion vector resolution: 1/4, 1/8
Interpolation Filter
AVC filter, separable AIF filter, bilinear
Motion sharing
Motion vector competition
Block-based illumination compensation
那 motion vector它的resolution可以到多少呢?
那大家已經看到答案了 1/8
在 inter prediction這邊
1/8的 resolution並不會用在B frame
因為b frame已經算
技巧QQ|||