1. QoE Standardization Issues QoMex September 9, 2011 Mechelen, Belgium Arthur Webster, NTIA/ITS Co-Chair of VQEG Chair of ITU-T SG9 1

2. Context VQEG/Standardization Process ITU-T & SG9, SG12, SG16 ITU-R WP6C Other Standards Orgs VQEG Results Industry & Academia Standards & Reports 2

3. Present: VQEG and ITU Recs VQEG Project ITU-T SG9 Recommendation ITU-R SG6 Recommendation FRTV-I J.143 (2000), J.144 (2001), J.149 (2004) FRTV-IIJ.144 revised (2004)BT-1683 (2004) Multimedia J.148 (2003), P.910 revised (2008), J.246 (2008), J.247 (2008), J.340 (2010) BT-1866 (2010), BT- 1867 (2010) RRNR-TVJ.244 (2008), J.249 (2010)BT-1885 (2011) HDTV-IJ.341 (2011), J.342 (2011) QARTP.912 (2008) Hybrid and JEG-Hybrid J.bitvqm, J.mm-noref, J.noref 3DTV J.3D-disp-req, J.3D-fatigue, P.3D-sam MM2J.av-dist 3

4. Open Issues and Challenges for Standardization of QoE Need for Resources Testing is expensive & labor intensive Industry wants No-Reference Models Difficult technical problem High Quality Test Material (esp. AV & 3DTV) 3DTV QoE Subjective methods Objective methods Telepresence QoE (Question in SG12) Joint Efforts (JEG and JEG-Hybrid) vs Competitive See VQEG JEG-Hybrid Call for Proposals and Participation: http://wiki.vqeg-jeg.orghttp://wiki.vqeg-jeg.org 4

5. Future and Trends VQEG Timeline and Future Activities 2000200420061996199820022008 FRTV-I FRTV-II HDTV-I RRNR-TV Multimedia VGA QCIF CIF HYBRID 20102012 3DTV JEG-Hybrid HDTV2 QART MM2 5

6. SG9 & WP6C Recommendations resulting from VQEG projects--List J.143: User requirements for objective perceptual video quality measurements in digital cable television J.144: Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference J.148: Requirements for an objective perceptual multimedia quality model J.149: Method for specifying accuracy and cross-calibration of Video Quality Metrics (VQM) J.244: Full reference and reduced reference calibration methods for video transmission systems with constant misalignment of spatial and temporal domains with constant gain and offset J.246: Perceptual visual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference J.247: Objective perceptual multimedia video quality measurement in the presence of a full reference J.249: Perceptual video quality measurement techniques for digital cable television in the presence of a reduced reference J.340: Reference algorithm for computing peak signal to noise ratio of a processed video sequence with compensation for constant spatial shifts, constant temporal shift, and constant luminance gain and offset J.341: Objective perceptual multimedia video quality measurement of HDTV for digital cable television in the presence of a full reference J.342: Objective multimedia video quality measurement of HDTV in the presence of a reduced reference signal P.910 (revised) Subjective video quality assessment methods for multimedia applications BT.1683 Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a full reference BT.1866 Objective perceptual video quality measurement techniques for broadcasting applications using low definition television in the presence of a full reference signal BT.1867 Objective perceptual visual quality measurement techniques for broadcasting applications using low definition television in the presence of a reduced bandwidth reference BT.1885 Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a reduced bandwidth reference J.bitvqm Hybrid perceptual bitstream video quality assessment J.mm-noref Perceptual audiovisual quality measurement techniques for multimedia services over digital cable television networks in the absence of a reference J.noref Perceptual video quality measurement techniques for digital cable television in the absence of a reference J.3D-disp-req Display requirements for 3D video quality assesment J.3D-fatigue Assessment methods of visual fatigue and safety guideline for 3D video P.3D-sam Subjective assessment methods for 3D video quality J.av-dist Methods for subjectively assessing audiovisual quality of internet video and distribution quality television, including separate assessment of video quality and audio quality 6

7. Questions and Feedback? Thank you! webster@its.bldrdoc.gov www.vqeg.org www.itu.int 7