Presentation is loading. Please wait.

Presentation is loading. Please wait.

AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Similar presentations


Presentation on theme: "AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,"— Presentation transcript:

1 AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology, Peking University Oct.12, 2010

2 Video coding… Compress high bit-rate digital video signal as low bit-rate stream –SD: from ~250Mbps to < 5Mbps –HD: for >1Gbps to < 20Mbps is the reason makes DTV possible at 1994 and always a core concern for Digital TV system and any other digital video applications

3 3 Video coding Standards ISO/IEC recommended MPEG series standards –MPEG-1, MPEG-2, MPEG-4 ITU-T recommended H.26X series standards –H.261, H.262, H.263, H.263+, H MPEG-China AVS H.264 / MPEG-4 AVC China 1G standards 2G standards

4 MPEG-2: the 1 st Generation ISO/IEC / ITU-T 1994 –SDTV at ~5Mbps –HDTV at ~20Mbps Makes DTV became reality –Europe DVB at 1997 –USA ATSC at 1997 –Japan ISDB at 1998 –China DTMB at 2006

5 5 5 AVS – one of the 2 nd Generation In Feb2006, AVS video issued as Chinese national standard –GB/T –Effective from Mar2006 In Aug2006, DTMB issued as Chinese national standard –GB/T –Mandatory from Aug2007

6 nd G Video – ITU-T IPTV Selection ITU-T IPTV FG decided to recommended the following video format at May2007 –H.264/ MPEG-4 AVC –MPEG-2 (for Legacy) –AVS – GB/T –SMPTE VC-1 ITU-T Q6-WP3-SG16 meeting at last week (10July09) decided to release upper document as the ITU Technical Paper on “Media coding toolbox for IPTV”

7 HD: 20 Mbps SD:4-6Mbps HD: 6-8Mbps SD:1-2.5Mbps 7 Video Compression Performance Improvement Compression Ratio AVS MPEG-4 AVC/H.264 VC-1 MPEG-1 AVS-?/MPEG-?/others-? MPEG-2

8 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

9 Higher Performance, High Cost Gold age is far away… –MPEG-1,MPEG-2 –H.261,H.262, H.263,H.263+/++ Face to complicated situation nowadays –H.264,VC-1

10 10 MPEG Standards 1991 : MPEG-1 –Royalty Free 1994 : MPEG-2 –DVD, DTV, Emmy Award –2.5$ per device 1996 : MPEG LA—A licensing agency 1999 : MPEG-4 SP (Part 2) –0.25 $/device, coding time based charge (2 cents/ hours) –Objected by AOL-Times Warner 2003 : H.264/MPEG-4 Part 10 AVC –Nov. 17,2003: Licensing policy – title based licensing –Objected by EBU ( EBU Technical Statement-D ) –May.20,2004: Policy finalized, little revision

11 Leonardo Chiariglione 《 Patents and standards 》 –http://www.chiariglione.org/ride/patents_and_standards.htmhttp://www.chiariglione.org/ride/patents_and_standards.htm If Gutenberg had lived in a culturally more advanced city like Venice or Florence, which had already started issuing litterae patentes ("open letters", whence the word "patent" comes from) since a few decades, he could have protected his invention with a patent. Some time later, MPEG had to deal with MPEG-2 patents. Many companies had participated in the MPEG-1 development not so much because they necessarily had plans to make products in that space, but because they were keen to be around and help tune up the MPEG machine for the big MPEG-2 deal. In January 1993 I took the initiative to invite to the Rome meeting Mike Smith, head of the Information Technology Task Force (ITTF), the group inside the ISO Central Secretariat in Geneva looking after JTC1 matters. Mike provided advice on how this difficult issue could be treated without trespassing the limits set by the ISO/IEC directives. In the event, an MPEG-2 Patent Pool was established, managed by MPEG LA. This organisation acts as a "one-stop shop" for MPEG-2 Video and Systems patents (Audio is not part of the licensing), but there are a few companies licensing their technology separately. First Effort to deal with patent issue for MPEG-2 The result is reasonable (4$ - 2.5$/device)

12 网址

13 http://www.avs.org.cn13 AVC/H.264 licensing structure Source:AVC/H.264 license terms Codec Manufacturers Subscription Title-by-Title Internet Broadcast Free Television Participation Fees Where End User pays Participation Fees where Remuneration is from Other sources

14 http://www.avs.org.cn14 AVC/H.264 License Terms: Participation Fees Where End User pays for AVC Video –Subscription (not limited by title) – 100,000 or fewer subscribers/yr = no royalty; > 100,000 to 250,000 subscribers/yr = $25,000; >250,000 to 500,000 subscribers/yr = $50,000; >500,000 to 1M subscribers/yr = $75,000; >1M subscribers/yr = $100,000 –Title-by-Title - 12 minutes or less = no royalty; >12 minutes in length = lower of (a) 2% or (b) $0.02 per title. Where remuneration is from other sources –Free Internet Broadcast (not title-by-title, not subscription) – no royalty during first term (through 2010; not greater than free television thereafter) –Free Television - (a) one-time $2,500 per transmission encoder or (b) annual fee starting at $2,500 for > 100,000 HH rising to maximum $10,000 for > 1,000,000 HH Enterprise cap: $3.5M/yr , $4.25M/yr , $5M/yr 2010 Royalties begin January 1, 2006 Source:AVC/H.264 license terms

15 Effort for 2G: RF JVT baseline AVS Strategy turned in 2003 For JVT baseline + essential enhanced tools To a dependent, compact, patent clear standard

16 MPEG LA 专利池构成与风险分析

17

18 AVS Mission To develop a 2 nd Generation video coding standard with same/better coding performance than others Avoids licensing risk based on clearly analysis of related patents in last 50 years To help DTV, IPTV and new media operators in China and out of China leave from more and more dangerous licensing mire AVS was born at Mar.2002

19 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

20 AVS Working Group Set up by MII (Ministry of Information Industry) in June of 2002 In charge of national standardization of audio-video coding for Chinese A/V industry Also in charge of organizing the delegation of China National Body to MPEG(ISO/IEC JTC1/SC29/WG11)

21 AVS membership

22 国内企业 Local Members

23 MNC or Out of China 外资企业和国外企业

24 AVS Subgroups System Video Audio Digital Media Rights Management Requirement Conformance IPR Implementation

25 AVS IPR Experts Group formation Late 2003 the President of AVS asked that a group of experts be assembled to create a recommendation for an IPR policy for AVS Create policies that: –reflect and respect China law and culture –reflect WTO requirements –reflect successful global practices –balance the rights of the inventor and needs of the implementers –innovative and forward looking –evolves with practice and law in China

26 Mission of AVS IPR Group Make a friendly environment for technical society (standard body), industry, and end user Key issue to work out –Reforming the IPR policy making by only IP holder, to by IP holder + industry + consumer

27 AVS IPR Experts Group Experienced members: 经验丰富的成员 –Patent Attorneys 专利律师 –Technical Experts 技术专家 –Contract Attorneys 处理合同的律师 –Previous experience in setting up patent pools 具有建立专利池相关经 验 的成员 Members are from: 成员来自 –Asia 亚洲 –Europe 欧洲 –North America 北美 Members represent: 成员代表了 –Consumer Electronics 消费电子产品界 –Information Technology 信息技术界 –Microelectronics 微电子产业界

28 AVS IPR Experts group Members –Broadcom –IBM –Intel –Matsushita/Panasonic –Microsoft –Nokia –Sony –Sun Microsystems

29 AVS Standards Participant by SUBGROUP (Audio, Video, …) Contributor Member default IP selection Non- Contributor Patent PoolLicensing RAND Royalty Free RAND (RoW only) Non-subgroup participant Non-participant elected default applies Voluntary Disclosure no Elected option yes Participant elected default applies Disclosure no Elected option yes Member IPR Obligations 成员知识产权义务 Participant: Patent Pool RAND RF RAND Non- Participant: Patent Pool RAND RF RAND No-license

30 Licensing Obligations Join AVS Sign Member Agreement Declare default option Participate in Subgroup Commit to license License options Submit Contribution Disclose patents Declare license option Non-Participant: RAND RF POOL RAND No License RAND RF POOL RAND China Patent: RAND RF POOL Worldwide Patent: RAND RF POOL RAND Participant: RAND RF POOL RAND

31 http://www.avs.org.cn31 Disclosure Obligations When making a contribution –Unpublished, published and granted patents Ongoing –When a patent is discovered –When a Patent Application is published During Final Draft Standard Review –90 days to declare licensing option, OR: –Default applies

32 http://www.avs.org.cn32 AVS Licensing 1 RMB concept –Only one charge, to device/software/chip –No charge to operator –No charge to end-user Will be handled by AVS PPA –PPA is a non-profit organization –Founded By #1 institute of MII and ICT of CAS, in 2005

33 Patent : - Royalty Free -AVS Pool -RAND Public knowledge AVS Patent Pool AVS Stand ards AVS Working Group AVS Patent Pool Council AVS Industry Alliance AVS Industry Chain ICs,software Device,System Operating,Service AVS Industry Cluster HDTV IPTV DTH MobileTV …… R&D IPR Industry R&D IPR Industry EssentialevaluationOne-Stoplicense AVS Licensing Technical Evaluation Licensing principle IPRPrinciple Cooperation & Development Development ApplicatioStandardBetterImplem. Three Independent Organizations of AVS

34 http://www.avs.org.cn34 AVS Industry Alliance –Amoisonic –Broadcom –Changhong –Haier –Huaju –Huawei –Langchao –NSCC –Panstar Semi –SVA –Skyworth –Spectrum Communication –Sunnywell –TCL –USstarcom –ZTE

35 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

36 AVS1 Family AVS Parts 国家标准计划号 STD Number 小组草案 ( WD ) 工作组 草案 (CD) 最终草 案 (FCD) 标准送审 稿( FD ) 国家标准发布 实施( GB ) AVS1-P1 (系统 - 广播) T AVS1-P2 ( J-profile ) T AVS1-P2 ( X-profile ) T AVS1-P2 ( S-profile ) T AVS1-P3 (Audio-Stereo) T AVS1-P3 ( M-profile ) T AVS1-P4 ( Conformance ) T AVS1-P5 ( Ref. Software ) T AVS1-P6 ( DRM ) T AVS1-P7 ( Mobility Video) T AVS1-P8.1 ( System-IP ) T AVS1-P8.2 ( System-IP ) T AVS1-P9.1 ( File Format ) T AVS1-P9.2 ( File Format ) T AVS1-P10 ( Mobile Speech and Audio ) TBD

37 AVS TutorialOct.20, MPEG-2 Video Entropy Coding IQ/IT Motion compensation Control Data Coeff. MV Inter/Intra Control Decoder Motion estimation Trans./ Quant - 0

38 AVS TutorialOct.20, Entropy Coding IQ/IT Motion compensation Control Data Coeff. MV Inter/Intra Control Decoder Motion estimation Trans./ Quant - Deblock Intra- frame Prediction AVS Video

39 AVS TutorialOct.20, Entropy Coding IQ/IT Motion compensation Control Data Coeff. MV Inter/Intra Control Decoder Motion estimation Trans./ Quant - Deblock Intra- frame Prediction AVS Video

40 http://www.avs.org.cn40 Periods of key video coding tools contributed ~ DPCM Interlace B-frames P-frames Scene Adaptive Coder Block Motion Estimation DCT Macroblocks Hybrid Coding Huffman Coding Transform Coding Motion Vector Prediction Video Object Planes Object- Based Scalabilit y Error Resilienc e Generic B-pictures Advanced Deblockin g Filter Modified from C Reader, 2002 Multiple Reference Integer Transfor m Variable Block- size Motion Compensation

41 Coding tools and its contribution Color space Conversion Transform and bit allocation Entropy coding ME and MC 2 * 3 * 3 * 3 ≈ 50 1st G 2 * 3 * 6 * 3 ≈ 100 2nd G

42 AVS TutorialOct.20, AVS Video Tools AVS1 video accepted 42 technical proposals from more than 200. Major tools –Transform – 16bit-implemented 8x8 integer transform –Quantization and scaling - scaling only in encoder –Intra prediction – 5 modes –Motion compensation – 16x16/16x8/8x16/8x8 modes –Quarter-pel interpolation – 4-taps interpolation filter –Deblocking –Entropy coding Minor tools –Motion vector prediction –Skipped mode and Coded block pattern –Adaptive scan

43 Spatial Coding Tools Fourier Transform Block Quantization Adaptive Block Coding Slant Transform Hadamard Transform Scene Adaptive Coder Zonal Coding Threshold Coding Zigzag Scan ZZ-Run- Length Coding Non-Uniform Quantizer Uniform Quantizer Block Classification Cosine Transform Dead-Zone Quantizer Rate Buffer 1960s 1970s Statistical Coding Andrews, 1968 Ahmed, 1974 Pratt, 1969 Max, 1960 Tescher, 1975 VLC Video Variable- Length Coding VLC Huffman 1952 Tescher, 1977 Fano 1949 Chen, 1973 Habibi, 1971 Chen 1981 Tasto 1971 Huang 1963 Seyler 1962 Pratt 1969 Woods 1969 Tescher, 1976 Huang 1966 Run-Length Coding Pratt, 1970 C Reader, 2002

44 DCT C(h,v)=C(v)C(h)ΣΣ[P(x,y)COS{πv(2y+1)/16}COS{πh(2x+1)/16}] C(0)=1/2√2C(v)=C(h)=1/2 [v,h≠0] x=0y=0 77 Quant VLC/ CABAC Y U V RGB YUV DCT-based Compression

45 AVS Innovation Example(1) DCT Transform Matrix in AVS Differs from the one in MPEG-2 (floating value) and H.264 (4X4 and 8X8) that was patented with similar performance (or even better in some cases )

46 46 Temporal Coding Tools Conditional Update (Spatial; Pixel) Conditional Update (Transform) 3D (Temporal) DPCM Transform Coding (Intra) DPCM Hybrid Coding Hybrid Coding (Transform) Harrison 1952 Pratt 1968 Reader 1973 Schroeder 1970 Roese 1975 Temporal Statistics Seyler 1962 Motion Modeling Brofferio 1970 Interlaced Field/Frame Coding Jones 1978 Transform in DPCM Loop with MC Jain 1979 PRA Motion Estimation Netravali 1979 Variable – Size BMA Motion Est. Giorda 1975 Motion Compensated DPCM (Pixel) Motion Measurement Limb 1975 Rocca 1969 Conditional Update (Spatial; Block) Seyler 1962 Candy 1971 Motion Compensated Transform Motion Compensated DPCM (Block) Koga 1981 Netravali 1979 BMA Motion Estimation Taki 1974 MPEG2 H.261 MPEG1 MPEG2 Etc. H.120 C Reader, s1970s

47 47 Skip, Direct and Symmetry Modes in B-frame Same: –Use both forward and backward reference frame to predict Different: –Skip: no coded MVD, no coded coefficient –Direct: no coded MVD, with coded coefficient –Symmetry: with 1 coded MVD and coded coefficient

48 48 AVS B-frame Compared with Other Standards AVS –Bi-prediction Skip, direct –No motion vector Symmetric –1 coded motion vector Higher compression ratio Other Standards –Bi-prediction Skip, direct –No motion vector Other cases –2 coded motion vector

49 AVS vs. H.264 vs. MPEG-2 toolsAVS1-P2H.264MPEG-2 Intra- interpolation 8x8, 5 modes for Y prediction, 4 modes for UV prediction 4x4, 9 modes for Y prediction, 4 modes for UV prediction Prediction only to DC coefficient Reference frame <=2<=16<=2 Block-size for MC 16x16, 16x8, 8x16 8x8 16x16, 16x8, 8x16 8x8,8x4,4x8,4x4 16x16, 16x8(interlace) B frame micro-block bi-direction prediction Coding front motion vector, and computing back motion vector Coding two motion vectors

50 toolsAVS1-P2H.264MPEG-2 ¼ pixel MC ½ pixels 4-tap ¼ pixels 4-tap ½ pixels 6-tap ¼ pixels 2-tap ½ pixels 2-tap Transform and quantizatio n 8x8 integer transform , encoding site normalization only 4x4 integer transform, both encoding and decoding sites need to normalize 8x8 float DCT Entropy coding Adaptive 2D VLCCAVLC CABAC VLC Loop filter 8x8 based Less boundaries Less BS-levels (0..2), Less pixels filtered (p0, p1,q0, q1) 4x4 based More boundaries More BS-levels (0..4), More pixels filtered (p0..p3,q0..q3) N/A AVS vs. H.264 vs. MPEG-2 (cont ’ d)

51 AVS Tutorial51 AVS Video Profile and Level JiZhun ( 基准 ) Profile (Dec. 2003) –4 levels 4.0 :up to Standard Definition with 4:2:0 4.2 :up to Standard Definition with 4:2:2 6.0 :up to High Definition with 4:2:0 6.2 :up to High Definition with 4:2:2 AVS Zengqiang Profile (2008) –for Storage media such as movie AVS Shensuo Profile (2008) –for video surveillance

52 AVS1 Video Subjective Testing April-Sept., 2005, AVS subjective testing has been executed by SARFT testing center The testing is based on GY/T standard as well as related ITU testing standard Compressed testing video stream bit-rate SD resolution (625/50i) HD resolution (1125/50i) MPEG-2 typical bit-rate (Mbps) 4~620 AVS bit-rate (Mbps) Testing resultABAA-

53 Performance- HD (1280 x720)

54

55 Performance – SD (720x576)

56 Performance – SD (720x576i)

57 Performance – CIF (352x288)

58

59 http://www.avs.org.cn59 Complexity analysis ToolsEstimated increasing AVSH.264 Multiple reference12 Variable block-size MC12 Quarter pixel33 Entropy coding0.51 Deblocking filter0.51 Total69

60 Review: MPEG-4 AVC/ H.264(2001~09)

61 Review: AVS ( 2002 ~) At Jan.26-Feb.5 , Geneva, ITU-T SG16 Q6 ( VCEG ) Meeting During the discussion on the requirements of H.265/H.NGC, Multiple MNC insisted on the new standard show be compact at technology side and lower complexity at implement side. Their proposal on new standard is very similar with AVS This is the road AVS selected 8 years ago.

62 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

63 AVS DTV End-End Solution

64 Professional AVS Encoders in the Market PowerCoder AE200MC PowerCoder AE100SD/HD CoderStar AE1001 CoderStar AC1001 4Caster C4 SUMAVISION 9210 Telarity

65 SVA Coderstar Series Codestar AC1001 for SDTV (1,2,4 channel) Codestar AC1001-L for MobileTV (1,2,4 channels) Codestar AE1001-H for HDTV Professional AVS Encoders from SVA (Shanghai)

66 UniAV (Beijing) Powercoder AVS Encoders for CIF,SD and HD Envivio (USA) 4caster For CIF and SD Professional AVS Encoders

67 AVS decoding Chips in the Market China Companies –Spreadtrum SV6111 / SV6100 / SV6600V –Penstar DS-1000 HD / DS-1000 SD –NationalChip GX3203 / GX3101 –Celestial CSM1208 –Haier Hi2830(HD)/Hi1260(SD) MNC –Broadcom BCM7405B / BCM7466 –ST STi520x/710x –NXP STB222 –SigmaDesign SMP8654

68 AVS decoder Chips in the Market AVS + MPEG-2 SD&HD (Beijing) AVS SD&HD (Shanghai) AVS + AVC + MPEG-2 SD&HD (Beijing) AVS + MPEG-2 SD & CTTB (Hangzhou) AVS+MPEG-2+AVC+VC-1 SD&HD (USA) AVS+MPEG-2+AVC+VC-1 SD&HD (Europe) AVS+MPEG-2+AVC+VC-1 SD (USA)

69 AVS decoding Chips in the Market

70 USB Dangle + PC STB and TV Set AVS End Devices

71 AVS STBs

72 AVS receivers in car and bus AVS End Devices

73 SVA AVS TV Set 获得 2008 年上海国际工业博览会创新奖 AVS TV Set

74 车载移动终端 >7” 屏 AVS in Car

75 AVS in CBHD (China Blu-ray High-definition Disc player) AVS in CBHD (China Blu-ray High-definition Disc player)

76 双拼双视 ( Stereo Interleaving Video ) 时分双拼 – 左视和右视以奇偶场双拼 空分双拼 – 左右 – 上下 与现有标准关系 – 现有视频编码标准可以支持这种双拼视频 – 增加信令,沟通采集表示环节与立体显示器 * Images from

77 AVS 3D TV Plan to live broadcast the Asia Games at Guangzhou at Nov. of this year

78 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

79 AVS Terrestrial TV Deployment Commercial Operating Testing before Operating

80 AVS Terrestrial TV in Operating Shanghai Oriental Pearl digital TV Co., Ltd.16 SD in 1 Channel Sichuan Province StarMedia Broadcasting Co.,LTD36 SD in 3 Channels Hang Zhou City Culture Radio Television Group21 CIF in 3 Channels Shanxi Province Mobile Television20 SD & 10 CIF in 2 Channels Qingdao City Mobile Digital TV Co., LTD,9 CIF in 1 Channel Shaanxi Province Mobile Digital Television20 CIF in 1 Channel Hebei Province Mobile Digital Television Co. LTD20 CIF in 1 Channel

81 The State Development and Reform Commission (Responsible for DTV Policy) Digital TV Development Project (2009) (1) Digital Terrestrial TV Standards Family (2) Single Frequency Digital Terrestrial TV System Deployment (3) Key DTV Products: AVS, DRA and DTMB Chips, STB, Encoder, etc

82 Outline Licensing problem for AV standards AVS approach for dealing with IPR AVS Video technologies AVS Products AVS Applications nowadays Conclusion and Discussion

83 Conclusion 2 nd Generation Video coding standards can double the efficiency of terrestrial frequency There are three choices for 2 nd G video coding standards –ISO/IEC MPEG-4 AVC/ITU-T H.264 (2004) –AVS (China, Feb2006) –SMPTE VC-1 (leads by Microsoft, Apr2006)

84 ISO/IEC MPEG-4 AVC/ITU-T H.264 Already being adopted in some countries and area But high royalty cost and risk –Operators should pay MPEG LA for commercial services (e.g. PPV), about 5 million USD per year –Patent holders outside of MPEG LA Patent Pool are waiting for collecting money (the licensing policy still under table)

85 AVS Technologies Much compact designed 2 nd G standard –based on analysis on method & patents in last 50 yrs –without many complicated but small performance contribution tools in international standards –Technologies Free methods and tools in public domain Self-innovation patents (~50 Chinese patents for video) –Similar performance with lower complexity Very low cost –1 RMB ( 1/6 USD) per device –Royalty free for operators

86 AVS Products and Deployments Most international video decoder chip vendors provide AVS-compatible SD&HD chips 4 China local companies provide low cost chips –e.g. Current price for a AVS+MPEG-2 SD Terrestrial Box is about 30USD Enough AVS products in the market Shanghai deploys first AVS based Terrestrial broadcasting system with 16 AVS SD programs, 7 provinces/cities provide commercial broadcasting AVS will be the unique mandatory format for all TV set and STB from next year

87 Comments 2 nd G video coding standards is correct choice, but please consider –MPEG LA licensing –Potential patent holders especially who in MPEG-2 pool but not in MPEG-4 AVC pool AVS is a better choice for long-term commercial operating –Commercial operating experience from multiple cities and provinces –Cooperation on development on AVS1 and research on AVS2

88 Thanks ! AVS Working Group : Tiejun Huang :


Download ppt "AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,"

Similar presentations


Ads by Google