Presentation on theme: "1 Case study of PESQ performance in live wireless mobile VoIP environment Zizhi Qiao (Motorola UK) Dr. Lingfen Sun (University of Plymouth, UK) Prof. Emmanuel."— Presentation transcript:
1 Case study of PESQ performance in live wireless mobile VoIP environment Zizhi Qiao (Motorola UK) Dr. Lingfen Sun (University of Plymouth, UK) Prof. Emmanuel Ifeachor (University of Plymouth, UK) 15/09/2008 PIMRC2008
2 Aim and Objectives Aim is to investigate PESQ algorithms performance in wireless mobile VoIP environment Objectives include: A live wireless mobile VoIP speech quality test platform and its calibration PESQs performance investigation, especially time alignment related performance issues
3 We find out that in some cases, PESQ readings are significantly different from MOS scores. Here are a few examples: A PESQ performance issue – why we do this research Yes, PESQ is wrong in this case. Time alignment error is the main issue. Ref. Speech PESQ=3.09 Deg. #23-38 PESQ=3.71 Deg. #25-46 But lets have a look of the test platform and its calibration before we go to PESQs accuracy issues, because the test platform forms the basis of the PESQ performance investigation PESQ scores 3.09 but MOS score is 3.6 Is PESQ wrong in this case? If so, Why?
4 A live wireless VoIP mobile test platform
5 Calibration of the live test platform Volume settings (hardware / software) affect PESQ score Calibrations are needed for each hardware or path settings
6 Case studies of PESQs accuracy issue Two cases are discussed in details Silence gap caused wrong delay reading Time alignment caused wrong PESQ reading They were all caused by time alignment method in PESQ
7 Silence gap caused wrong delay reading PESQ score on the top Relative delay on the bottom Wrong delay readings at the spike as pointed out A waveform analysis shows the PESQ delay reading is not correct
8 Silence gap caused wrong delay reading (cont.) Degraded wave on the top Wrong PESQ delay plot as shown on last slide Silence gap caused wrong delay reading in PESQ
9 Time alignment caused wrong PESQ score Positive shift in the waveform Negative shift in the waveform
10 Time alignment caused wrong PESQ score (cont.) PESQs reading for sample #23-38 And its wrong… Next slide shows why PESQ was wrong
11 Time alignment caused wrong PESQ score (cont.) We cut the sample and compared the chunks, each chunk has higher score than the original score, so we can conclude that PESQ was wrong and the error was caused by the delay shift.
12 Conclusions A live wireless mobile VoIP speech quality test platform is established and calibrated. A few PESQ performance cases has been discussed. PESQs time alignment can not deal with some silence gaps and time shifting situations correctly. More reliable time alignment method is necessary for PESQ to be used in new environment. Before the new improved are standardised, more detailed investigation of PESQ results are necessary for each test, especially in new environment.