An Experimental Evaluation of Voice Quality over the Datagram Congestion Control Protocol H. Balan International Univeristy Bremen L. Eggert Nokia Research Center S. Niccolini NEC Network Laboratories M. Brunner NEC Network Laboratories IEEE INFOCOM 2007 Presented by: Te-Yuan Huang

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  Modified TFRC  Conclusion

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  Modified TFRC  Conclusion

Goal  To evaluate Voice Quality over DCCP

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  Modified TFRC  Conclusion

Background  Why not TCP? Reliability is unnecessary for voice Retransmission can be harmful  Why not UDP? Not responsive to network congestion Negatively impact competing traffic

Background (Cont.)  What ’ s DCCP? Datagram Congestion Control Protocol IETF transport protocol framework  RFC 4340 (March, 2006)  Congestion-controlled but not reliable Offers different congestion control scheme (CCIDs)  CCID2 – TCP-like windowing scheme  CCID3 – TCP-Friendly Rate Control (TFRC)

Background (Cont.)  What ’ s DCCP? CCID3 – TCP-Friendly Rate Control (TFRC) Initial slow start:  Initial rate ： 4 packets/RTT  Double every RTT Linear increase / decrease

Background (Cont.)  What ’ s DCCP? CCID3 – TCP-Friendly Rate Control (TFRC)  No window size  Regulate Sending rate by a Markov Model The model is based on TCP Reno X is the transmit rate in bytes/second s is the packet size in bytes p is the loss event rate T 0 is the TCP retransmission time in seconds

Background (Cont.)  What ’ s DCCP? CCID3 – TCP-Friendly Rate Control (TFRC)  The transmission rate is upper bounded by X(p) - the computed maximal rate 2 * X_recv - the rate reported by receiver  Several modifications based on TFRC TFRC SP  TFRC for small packets TFRC FR  TFRC for fast restart

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Results  Modified TFRC  Conclusion

Goal Refresh  OK … TCP not good UDP not good Now IETF propose a thing called DCCP  Show me how well they would perform! TCP UDP TFRC TFRC SP TFRC FR+SP

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  Modified TFRC  Conclusion

Experimental Setup  Voice Call Synthesis Interleaving talk spurts & pauses (On/Off Model)  Length : decaying exponential distribution Average Length of talk spurt : 1 sec Average Length of pause : 1.5 sec  Audio of Talk spurt Extracted out of a speech recording (Bush on Creation of Homeland Security Dept.)  Each call is 100 talk spurt/pause cycle Average call length is 250 seconds

Experimental Setup  Voice Encoding Using Speex codec to emulate  G.711 codec  G.729 codec  Both with voice activity detection

Experimental Setup  Data Transport Transmit audio frames over  UDP  TCP  TFRC (DCCP CCID3)  TFRC small packet variant (TFRC SP)  TFRC SP with “ faster restart ” optimization (TFRC SP+FR)

Experimental Setup  Test Network Transmit over a one-hop network DummyNet to emulate  Delay: 0 – 400 ms  Loss Rate: 0.01% - 10%  No bandwidth limitation

Experimental Setup  Playout Buffering Get Time-Seq. trace of incoming audio frames  By using ttcp toolttcp Compute BEST possible playout seq.  Offline, dynamic-programming-based algo.  Result: Highest possible audio quality for received voice frame

Experimental Setup  Voice Quality Metric Use E-Model to calculate R score Quality degradation due to codec Quality degradation due to loss Overall packet loss rate Unity Step Function End-to-end delay

Experimental Setup  Voice Quality Metric Mapping of R score & Perceived Quality

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  Modified TFRC  Conclusion

Outline - Experimental Evaluation  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation Varying Delay, No Loss Varying Delay, Fix Loss Fix Delay, Varying Loss  Modified TFRC  Conclusion

Experimental Evaluation – Varying Delay, No Loss (G.729) 1.UDP & TCP outperform TFRC 2.TFRC and SP behave similarly 3.TFRC SP and SP+FR behave similarly 4.UDP and TCP are identical

Experimental Evaluation – Varying Delay, No Loss (G.711) 1.UDP & TCP outperform TFRC 2.TFRC and SP behave similarly 3.TFRC SP and SP+FR behave similarly 4.UDP and TCP are NOT identical

Experimental Evaluation – Varying Delay, No Loss (G.729) 1.UDP & TCP outperform TFRC According to TFRC SPEC: B = number of bytes received since last receive rate report T = time since last receive rate report was sent X_recv = B/T T includes idle period X_recv is UNDER-ESTIMATED ◎ TFRC send rate is bounded by 2*X_recv

Experimental Evaluation – Varying Delay, No Loss (G.729) 2. TFRC and SP behave similarly Due to miscalculated receive rate 3. TFRC SP and SP+FR behave similarly Due to no loss 4.UDP and TCP are identical large initial window size > G.729 ’ s frame size large initial window size < G.711 ’ s frame size 。 X(p) - the computed maximal rate 。 2 * X_recv - the rate reported by receiver

Outline - Experimental Evaluation  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation Varying Delay, No Loss Varying Delay, Fix Loss Fix Delay, Varying Loss  Modified TFRC  Conclusion

Experimental Evaluation – Varying Delay, 0.1% Loss (G.729) 1.Basic TFRC is worse than TCP or UDP calculate X(P) with actual packet size 2. TFRC and SP behave similarly Due to miscalculated receive rate

Experimental Evaluation – Varying Delay, 0.1% Loss (G.729) 1.TFRC SP+FR quadruples the send rate during slow-start-restart

Experimental Evaluation – Varying Delay, 0.1% Loss (G.711) TCP Performs worse in G delay by window size 2. delay by retransmission

Outline - Experimental Evaluation  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation Varying Delay, No Loss Varying Delay, Fix Loss Fix Delay, Varying Loss  Modified TFRC  Conclusion

Experimental Evaluation – 50ms Delay, Varying Loss (G.729) TCP outperform all TFRC variants Reason: A.Freebsd enable 1. selective acknowledgement 2. limited transmit B.TCP: degrade by delay DCCP: degrade by loss

Experimental Evaluation – 50ms Delay, Varying Loss (G.711)

Analysis of Results Lars Eggert ®

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  TFRC Improvement Scheme Evaluation  Conclusion

Still one issue …  TFRC is based on TCP Reno  Modern TCP is different from Reno Large initial windows Selective acknowledgement Limited transmit extension  Modern TCP more aggressive than Reno  Goal of TFRC Use same bandwidth as TCP Use modern TCP instead of TCP Reno in the future

Outline - TFRC Improvement  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  TFRC Improvement Scheme Evaluation  Conclusion

Experimental Result: Varying Delay, No Loss (G.729)

Experimental Result: Varying Delay, No Loss (G.711) TCP Still Outperform all TFRC variants Due to delayed feedback pronounced impact on high-bit rate codec

Experimental Result: Varying Delay, 0.1% Loss (G.729)

Experimental Result: Varying Delay, 0.1% Loss (G.711)

Experimental Result: 50ms Delay, Varying Loss (G.729)

Experimental Result: 50ms Delay, Varying Loss (G.711)

Outline  Goal  Background  Goal Refresh  Experimental Setup  Experimental Evaluation  TFRC Improvement  Conclusion

Conclusion  Current TFRC variants perform Not only worse than UDP But also worse than TCP  TFRC assumptions don ’ t fit voice Large packets Continuous transmission  TFRC is less aggressive than modern TCP  Design an improved TFRC variants Contributed to IETF DCCP design process

My Conclusion  I like the paper Especially, the experiment setup  Answered my question Found why TFRC is so conservative  As a master thesis Impressed by the quality and quantity  Quality of Modified TFRC same as UDP Would like to know its reaction when competing with TCP