2. ACM Multimedia October 4, 2001 Real-time Voice Communication over the Internet Using Packet Path Diversity Yi Liang, Eckehard Steinbach, and Bernd Girod Image, Video, and Multimedia Systems Group Information Systems Laboratory Department of Electrical Engineering Stanford University

3. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 2 Overview Requirements of VoIP Packet path diversity for low delay VoIP Adaptive multi-stream playout scheduling Internet experiments Demo

4. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 3 Requirements of VoIP Small end-to-end delay for conversational services ( <150ms ) Delay variations (jitter) have to be smoothed using receiver buffer Late packets are lost, no time for retransmissions Small residual packet loss rate is ok Trade-off between end-to-end delay and late loss rate Sender Receiver Playout Time  12345678 12345678 Missed deadline Receiver buffer Time Packetization time

5. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 4  Motivation for Packet Path Diversity for VoIP  In up to 80% better alternative path [Savage ‘99]  Multi-path routing [Sidhu ‘91, Bahk ‘92]  Uncorrelated packet loss on independent paths [Apostolopoulos ‘01]  Efficiency of FEC limited by packet loss and delay correlation [Bolot ‘93, Bolot ‘99]  Delay jitter is the major killer of delay sensitive applications  Our contributions:  Exploitation of statistically independent jitter behavior for VoIP  Adaptive multi-stream playout scheduling technique R S Second path 12 Speech Packet Path Diversity Default path

6. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 5 Two-path speech coding  Redundant description of voice stream (MDC)  Two bitstreams for two paths Stream 1: [Jiang 2000]  Even samples: (8-bit, PCM)  Odd samples: (2-bit, ADPCM) Stream 2: Vice versa  25% Overhead  Loss of one packet  small reduction in speech quality  Loss of both packets  error concealment E s1 s2 O E O E O O E O E O E  E O Packet length Time

7. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 6 Packet Path Diversity for Low Delay VoIP Time 123456 Sending on path 1 Receiving on path 1 Packet Path Diversity reduces effective delay jitter and therefore late loss rate Packet Path Diversity reduces effective delay jitter and therefore late loss rate Playout 12 34 6 5  123456 Sending on path 2 Receiving on path 2

8. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 7 Adaptive Playout Scheduling Adaptive playout scheduling and speech scaling allow us to use more packets for playout at given mean target delay Adaptive playout scheduling and speech scaling allow us to use more packets for playout at given mean target delay Time 123456 Sending on path 1 Receiving on path 1 1 Constant Playout stretchingcompressing Adaptive Playout 123465 If past delay values indicate congestion  delay playout of next packet(s) by stretching speech signal If past delay values are small  advance playout of next packet(s) by compressing speech signal 23465 

9. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 8 Modification of Playout Speed Based on time-domain interpolation algorithm WSOLA [Verhelst et al., 1993, Liang 2001] Output packet 1/20/12/33 4 Original packet Pitch- period 02134 Template

10. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 9 Adaptive Two-stream Playout Scheduling playout deadline estimate of loss probability Histogram of past delay values Combination of Packet Path Diversity and Adaptive Playout Minimization of Lagrangian cost function Delay Variation of

11. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 10 Qwest Internet Experiment Explicit path selection using relay server [Apostolopoulos ‘01] UDP packets with payload of 240 bytes Exodu s Comm. BBN Planet Netergy Networks 192.84.16.176 MIT 18.184.0.50 Harvard 140.247.62.110 (5ms) (45ms) (40ms)(5ms) Sender Relay Server Receiver

12. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 11 Measured Packet Delay Trace Delay in ms Packet number

13. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 12 Adaptive Two-stream Playout Scheduling Delay in ms Packet number

14. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 13 Comparison: Single-path Transmission with FEC 2 3 41 123 2 41 13 2341 Stream received with packet loss Stream reconstructed 3 Stream sent Packets protected with FEC  FEC: adds redundancy by sending one or more copies of the source signal in the following packet(s) [Bolot ‘96] FEC protected single-stream For fair comparison Primary copy: quantized at fine resolution (8-bit) Secondary copy quantized at coarser resolution (2-bit) Same data rate as transmission with Packet Path Diversity Same adaptive playout scheduling technique

15. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 14 Results delay (ms) Packet loss rate in %

16. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 15 Demo Original  Average total end-to-end delay: 84 ms  Error concealment: speech segment repetition  Average total end-to-end delay: 84 ms  Error concealment: speech segment repetition Path DiversitySingle-stream with FEC at same data rate

17. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 16 Conclusions  Packet Path Diversity for real-time voice communication over IP  Multiple Description Coding of speech signal  Quality improves with each description received  Exploitation of statistically independent jitter behavior  Improvement of delay versus speech quality trade-off  Adaptive Playout Scheduling  Flexible playout deadline using time-scale modification  Reduction of late loss rate for given target delay  Lagrangian cost function for two-path transmission  Internet experiments  Implementation of path diversity using relay server  Observation of largely independent delay jitter behavior  Significant speech quality improvement in comparison to single-path transmission with FEC at the same data rate and end-to-end delay

18. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 17 PESQ Results  Perceptual Evaluation of Speech Quality (ITU-T Rec. P.862, Feb. 2001)  PESQ can be used for end-to-end quality assessment  Ranges from –0.5 to 4.5 but usually produces MOS-like scores between 1.0 and 4.5

19. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 18 Internet Experiment II VBNS IP Backbone Service DANTE Operations UUNE T Tech. Erlangen 131.188.130.136 Harvard 140.247.62.110 (7ms) (40ms) AT&T (5ms) (10ms) New Jersey 165.230.227.81  Path 1 (direct): N. J. – Erlangen  Path 2 (alternative): N. J. – Harvard – Erlangen

20. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 19 Result Path 1 (direct): N. J. – Germany Path 2 (alternative): N. J. – Harvard – Germany Mean delay 61.3/65.0 ms link loss 0.6% / 1.1% Significant reduction of late loss and end-to-end delay by packet path diversity

21. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 20 Constant Playout Trade-off between packet loss and delay Constant playout deadline late loss

22. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 21 Adaptive playout deadline Adaptive Playout Adaptation to delay variation (jitter)

23. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 22 Late Loss  Mean Delay

24. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 23 Speech Scaling

25. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 24 Demo 2 Original Average total end-to-end delay: 108 ms Path Diversity (PESQ: 4.1) Single-stream with FEC (PESQ: 3.5)

26. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 25 Voice over IP (VoIP) VoIP is rapidly growing 900% 1998-1999 5000% 1999-2004 2004 1999 1998 135 billion minutes 2.7 billion minutes 310 million minutes [Source: IEEE Spectrum, Mai 2000]

27. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 26 Limits of Speech Stretching More than 25% is annoying OriginalStretching: s=1.3

28. Liang, Steinbach, Girod: Real-time Voice Communication over the Internet Using Packet Path Diversity 27 Speech and Audio Scaling Speech scaling Audio scaling originalstretched: s=1.3compressed: f=0.7