2. Yi Liang Department of Electrical Engineering Stanford University April 19, 2000 Loss Recovery and Adaptive Playout Control for Packet Voice Communications over IP

3. Yi Liang Department of Electrical Engineering, Stanford University Outline Background and Motivation Network Delay and Jitter Characteristics Voice Concealment Using Time Scale Modification and Wave Substitution Adaptive Playout Delay Adjustment Future Work and More Loss Reconstruction/Concealment Techniques

4. Yi Liang Department of Electrical Engineering, Stanford University Voice Over IP Increased network bandwidth Universal presence of IP, easier and faster Internet access Availability of supporting hardware IP Gateway PSTN H.323

5. Yi Liang Department of Electrical Engineering, Stanford University Voice Over IP - Challenges The Internet protocol delivers service on a best-effort basis, with no guarantee of quality of service The transmission mechanism is unreliable and connectionless Transport layer – TCP guarantees reliable transmission by retransmission protocols; but not acceptable for live audio due to end-to- end latency

6. Yi Liang Department of Electrical Engineering, Stanford University Improving QoS – Efforts and Implementations Problems End-to-End Delay Delay Jitter Packet Loss Efforts Minimize Delay Absorb Jitter Reconstruct/ Conceal Loss Implementations Optimize Algorithms Buffer Management Many Loss Recovery Techniques

7. Yi Liang Department of Electrical Engineering, Stanford University Delay Jitter Local Host: cass.stanford.edu Packet Size=200 bytes Packet Interval=20 ms

8. Yi Liang Department of Electrical Engineering, Stanford University Jitter Absorption and Buffer Management Sender Receiver Playout time Packet sequence number kK+1K+3K+4K+2 d a : buffer absorption time or allowed max delay kK+1K+3K+4K+2 K+5 kK+1K+3K+4K+2K+5 Packet dropped

9. Yi Liang Department of Electrical Engineering, Stanford University Delay and Loss Rate d: delay, a non negative random variable The p.d.f. of d: f(d) The delay distribution varies as time - The need to modify playout speed d a : Absorption Time

10. Yi Liang Department of Electrical Engineering, Stanford University Time Scale Voice Modification 23467 5 1 Time Scale Voice Modification Conceal Lost Packet Modify Playback Speed t 2371 t 46 a) Error Concealment 2341 2341 2341 b) Playback Speed Modification Acceleration Slowing Down Original

11. Yi Liang Department of Electrical Engineering, Stanford University Waveform Similarity Overlap Add (WSOLA) Algorithm [Verhelst, Roelands]

12. Yi Liang Department of Electrical Engineering, Stanford University Search for Similar Waveform by Pattern Matching Search for Synchronized Overlap Segment S k Positions of Search Regions

13. Yi Liang Department of Electrical Engineering, Stanford University WSOLA Implementation To conceal one lost packet: Introduced delay=2 packet time [Stenger, Younes, Reng, Girod]

14. Yi Liang Department of Electrical Engineering, Stanford University Delay-optimized Time Scale Modification 23467 5 1 2371 t 46 a) Conventional Method Delay = 2 packet time Packet length extension = 50% 245671 671 t 5 4 b) Improved Method 3 2 Delay = 1 packet time Packet length extension = 1/3 What if double-packet loss?

15. Yi Liang Department of Electrical Engineering, Stanford University Waveform Substitution Using Pattern Matching 1 234 template sliding template L L best match L replaced packet Can only conceal a limited number of lost packet using prior information Advantage: Delay=0 search window [Goodman, Wasem]

16. Yi Liang Department of Electrical Engineering, Stanford University Conceal Consecutive Lost Packets Using Hybrid Method 45671 671 t 5 4 3 1 2 Delay is reduced to one packet time; voice quality is good even for double-packet loss Can conceal more consecutive lost packets; but voice quality degrades as the number of consecutive lost packets goes up Waveform Substitution Time-scale Modification

17. Yi Liang Department of Electrical Engineering, Stanford University Adaptive Playout Delay Adjustment [Ramjee, Moon] Average Playout Delay = 5.47ms Late Loss Rate = 10.82% Average Playout Delay = 5.76ms Late Loss Rate = 5.63% Less Bursty Loss

18. Yi Liang Department of Electrical Engineering, Stanford University Audio Demos II. Playback Speed Modification III. Adaptive Playout Delay Adjustment Example 1: 20% Loss OriginalScaled OriginalLossPlayout I. How Well WSOLA-Based Concealment Works? Voice Stream Suffered from Simulated Random Packet Loss Concealed Voice Using Delay-optimized Time-scale Modification Introduced Delay = 1 packet time OriginalLossConcealed Example 2: 30% Loss

19. Yi Liang Department of Electrical Engineering, Stanford University Conclusions

20. Yi Liang Department of Electrical Engineering, Stanford University Future Work Architecture and Protocols Receiver- based Loss Recovery and Concealment Scalable (layered) Encoding Scheme Error Protection and Data Reconstruction Joint Source and Channel Coding Loss Avoidance at Network Layer Protect Real-time Voice from Data, Reserving the Bandwidth (Token Bus, RSVP, DiffServ) Audio Signal Processing Loss-resilient Real-time Data Transmission VoIP

21. Yi Liang Department of Electrical Engineering, Stanford University Combating Loss

22. Yi Liang Department of Electrical Engineering, Stanford University Forward Error Correction Various Encoding Schemes 2341 t Original Stream 2 3 41 123 2 41 13 2341 FEC Encoded Packet Loss Reconstructed 3 2 3 41 0 Protecting from Burst Loss 11223 t

23. Yi Liang Department of Electrical Engineering, Stanford University References For references and more voice samples: http://www-ise.stanford.edu/~yiliang/voip.htm