1. 1 Dude, Where is My Packet?

2. NUS.SOC.CS5248 OOI WEI TSANG 2 Overview Characteristics of the Internet General techniques Error recovery for audio Effect of loss on MPEG Error recovery for MPEG

3. 3 Loss Characteristics of The Internet

4. NUS.SOC.CS5248 OOI WEI TSANG 4 Characteristics of Internet 60-70% of paths do not show any loss Those with loss have an average of 4.5 – 6% packet loss [Paxson97] End-to-end Internet packet dynamics

5. NUS.SOC.CS5248 OOI WEI TSANG 5 Packet Loss Pattern 1000 30

6. NUS.SOC.CS5248 OOI WEI TSANG 6 Characteristic of Internet Bursts of loss are typically short (2-3 consecutively loss packets) Long burst do occur Burst may occur periodically

7. NUS.SOC.CS5248 OOI WEI TSANG 7 Wireless Link Loss rate measured in my office 10 ~ 20% up to 50% reported!

8. 8 General Error Recovery Techniques Perkins, Hodson and Hardman IEEE Network Magazine 1998

9. NUS.SOC.CS5248 OOI WEI TSANG 9 Retransmission 1234 124 3 3

10. NUS.SOC.CS5248 OOI WEI TSANG 10 Redundant Data 1234 124 2 23 345 53

11. NUS.SOC.CS5248 OOI WEI TSANG 11 24 Error Concealment 1234 1

12. 12 Retransmission for audio

13. NUS.SOC.CS5248 OOI WEI TSANG 13 Retransmission is Bad Need scalable feedback NACK consumes bandwidth May need to retransmit frequently Retransmission maybe useless

14. NUS.SOC.CS5248 OOI WEI TSANG 14 Retransmission is Good Only retransmit when needed Exact recovery

15. NUS.SOC.CS5248 OOI WEI TSANG 15 Scalable Retransmission On packet loss T = random(0, RTT) wait for T multicast NACK On receiving NACK from others suppress own NACK

16. NUS.SOC.CS5248 OOI WEI TSANG 16 Retransmit when group size is small loss rate is low large latency acceptable

17. 17 Redundant Data for audio

18. NUS.SOC.CS5248 OOI WEI TSANG 18 Parity FEC 1011001 1000010 0001001 XOR 0010010

19. NUS.SOC.CS5248 OOI WEI TSANG 19 Parity FEC 1011001 1000010 0001001 0010010

20. NUS.SOC.CS5248 OOI WEI TSANG 20 Parity FEC Ordering

21. NUS.SOC.CS5248 OOI WEI TSANG 21 Parity FEC Ordering

22. NUS.SOC.CS5248 OOI WEI TSANG 22 Parity FEC Ordering

23. NUS.SOC.CS5248 OOI WEI TSANG 23 Parity FEC Ordering

24. NUS.SOC.CS5248 OOI WEI TSANG 24 Parity FEC Ordering

25. NUS.SOC.CS5248 OOI WEI TSANG 25 Reed-Solomon Code RS(n,k) RS

26. NUS.SOC.CS5248 OOI WEI TSANG 26 Reed-Solomon Code

27. NUS.SOC.CS5248 OOI WEI TSANG 27 Media Specific FEC 1234 124 2 23 345 53

28. NUS.SOC.CS5248 OOI WEI TSANG 28 Interleaving

29. NUS.SOC.CS5248 OOI WEI TSANG 29 Pro and Cons Media Independent FEC Overhead for redundant data Exact recovery Could be computationally costly Media Specific FEC Approximate recovery only

30. 30 Error Concealment

31. NUS.SOC.CS5248 OOI WEI TSANG 31 Effect of Loss on Audio Speech Human ears can interpolate Loss up to length of phoneme can still be tolerable

32. NUS.SOC.CS5248 OOI WEI TSANG 32 Insertion-based Repair Splice Silence Substitution Noise Substitution Repetition

33. NUS.SOC.CS5248 OOI WEI TSANG 33 Other Repair Methods Interpolation Regeneration

34. NUS.SOC.CS5248 OOI WEI TSANG 34 Pro and Cons Could be computationally costly Approximate recovery only Does not work well for long burst of packet loss

35. 35 Colin’s Recommendations

36. NUS.SOC.CS5248 OOI WEI TSANG 36 Non-Interactive Apps Interleaving FEC Retransmission for unicast only

37. NUS.SOC.CS5248 OOI WEI TSANG 37 Interactive Applications Media Specific FEC

38. NUS.SOC.CS5248 OOI WEI TSANG 38 Error Concealment Repeat

39. 39 Effects of Packet Loss on MPEG Jill Boyce and Robert Gaglianello ACM Multimedia 1998

40. NUS.SOC.CS5248 OOI WEI TSANG 40 Measurement-based Study Need to understand the problem before proposing solution

41. NUS.SOC.CS5248 OOI WEI TSANG 41 Data Gathering Method From: NYC 13, Austin 21, London 18 To: Holmdel, NJ

42. NUS.SOC.CS5248 OOI WEI TSANG 42 Data Gathering Method Video: Two 5-mins MPEG 30 fps 384 kbps and 1 Mbps IBBPBBPBBPBBPBB QSIF 176x112 and SIF 352x240 One row per slice

43. NUS.SOC.CS5248 OOI WEI TSANG 43 Average Packet Loss

44. NUS.SOC.CS5248 OOI WEI TSANG 44 Packet Loss vs Time

45. NUS.SOC.CS5248 OOI WEI TSANG 45 How to Fill Packets B 1 frame 1 packet P 1 slice 1 packet I 1 slice 1 packet

46. NUS.SOC.CS5248 OOI WEI TSANG 46 Packet Size Distributions

47. NUS.SOC.CS5248 OOI WEI TSANG 47 Packet Size vs Loss Rate

48. NUS.SOC.CS5248 OOI WEI TSANG 48 Lesson 1 slice per packet even if larger than MTU If smaller packets means higher loss rate: fill packet until MTU Else one slice per MTU Alt: Change size of slice

49. NUS.SOC.CS5248 OOI WEI TSANG 49 Frame Effected by Errors

50. NUS.SOC.CS5248 OOI WEI TSANG 50 Correlation of Packet Loss

51. 51 Overview of Error Recovery for Video

52. NUS.SOC.CS5248 OOI WEI TSANG 52 Methods Retransmission Error Concealment FEC Limiting Error Propagation Reference Frame Selection

53. NUS.SOC.CS5248 OOI WEI TSANG 53 Reference Frame Selection I P B B P X I

54. NUS.SOC.CS5248 OOI WEI TSANG 54 Methods Retransmission Error Concealment FEC Limiting Error Propagation Reference Frame Selection Changing Temporal Pattern

55. 55 Changing Temporal Patterns Injong Rhee SIGCOMM ‘98

56. NUS.SOC.CS5248 OOI WEI TSANG 56 Basic Idea “Better Late Than Never!”, or Late packet is still useful

57. NUS.SOC.CS5248 OOI WEI TSANG 57 MPEG Frame Pattern IBBPBBP IPBBPBB

58. NUS.SOC.CS5248 OOI WEI TSANG 58 H.261 Error Propagation IPPPPPP XXXXXX IPPPPPP XXXX retransmission loss

59. NUS.SOC.CS5248 OOI WEI TSANG 59 H.261 Frame Pattern IPPPPPP X loss

60. NUS.SOC.CS5248 OOI WEI TSANG 60 IPPPPPP H.261 Frame Pattern X loss X X retransmission

61. NUS.SOC.CS5248 OOI WEI TSANG 61 PTDD Periodic Temporal Dependency Distance Large PTDD Later Deadlines Longer Error Propagations Less Temporal Dependencies

62. NUS.SOC.CS5248 OOI WEI TSANG 62 QAL IPPP IPPP Base Layer Enhancement Layer

63. NUS.SOC.CS5248 OOI WEI TSANG 63 QAL IPPP IPPP Base Layer + FEC Enhancement Layer X

64. NUS.SOC.CS5248 OOI WEI TSANG 64 QAL + PTDD IPPP IPPP Base Layer Enhancement Layer

65. 65 SR-RTP Feamster and Balakrishnan PV’02

66. NUS.SOC.CS5248 OOI WEI TSANG 66 Overview An analytical model for packet loss SR-RTP Post Processing

67. NUS.SOC.CS5248 OOI WEI TSANG 67 Analytic Model p: prob. packet loss f : observed frame rate : frame drop rate S i : mean num. of packets in frames of type i P(i) : prob. frame type is i P(F) : prob. frame is useless

68. NUS.SOC.CS5248 OOI WEI TSANG 68 p: prob. packet loss f : observed frame rate : frame drop rate S i : mean num. of packets in frames of type i P(i) : prob. frame type is i P(F) : prob. frame is useless

69. NUS.SOC.CS5248 OOI WEI TSANG 69 Is it accurate?

70. NUS.SOC.CS5248 OOI WEI TSANG 70 SR-RTP Extensions to RTP for selective reliability

71. NUS.SOC.CS5248 OOI WEI TSANG 71 SR-RTP 0Length ADU Sequence Number ADU Length ADU Offset PriorityLayer Number

72. NUS.SOC.CS5248 OOI WEI TSANG 72 Feamster & Balakrishnan’s ADU 1 ADU = 1 Frame

73. NUS.SOC.CS5248 OOI WEI TSANG 73 Loss Detection LEN: 50 ADU: 0 ADU Size: 100 ADU Off: 0 LEN: 20 ADU: 0 ADU Size: 100 ADU Off: 80 LEN: 50 ADU: 1 ADU Size: 150 ADU Off: 40 LEN: 50 ADU: 3 ADU Size: 50 ADU Off: 0

74. NUS.SOC.CS5248 OOI WEI TSANG 74 Retransmission Request Based on priority Example: I-Frame: highest priority P-Frame: higher priority if closer to prev I B-Frame: no retransmition

75. NUS.SOC.CS5248 OOI WEI TSANG 75 Error Concealment for Video Repeat pixels from previous frame Interpolate pixels from neighbouring region Interpolate motion vectors from previous frame

76. NUS.SOC.CS5248 OOI WEI TSANG 76 Inferred MV for P Frame IP

77. NUS.SOC.CS5248 OOI WEI TSANG 77 Inferred MV for I Frame PPI

78. NUS.SOC.CS5248 OOI WEI TSANG 78 Evaluation Peak Signal to Noise Ratio (PSNR) 255 2 over average square of differences in pixel values Calculated in dB (10 log 10 )

79. NUS.SOC.CS5248 OOI WEI TSANG 79 Performance of SR-RTP

80. 80 Today’s Summary

81. NUS.SOC.CS5248 OOI WEI TSANG 81 How to recover packet loss Retransmission FEC Error Concealment

82. NUS.SOC.CS5248 OOI WEI TSANG 82 How to limit the damage of error Interleaving Key frame selection Change reference frame