1. Chapter 6 Multimedia Networking
A note on the use of these ppt slides:
We’re making these slides freely available to all (faculty, students, readers). They’re in powerpoint form so you can add, modify, and delete slides (including this one) and slide content to suit your needs. They obviously represent a lot of work on our part. In return for use, we only ask the following:
If you use these slides (e.g., in a class) in substantially unaltered form, that you mention their source (after all, we’d like people to use our book!)
If you post any slides in substantially unaltered form on a www site, that you note that they are adapted from (or perhaps identical to) our slides, and note our copyright of this material.
Thanks and enjoy! JFK / KWR
All material copyright
J.F Kurose and K.W. Ross, All Rights Reserved
Computer Networking: A Top Down Approach Featuring the Internet, 2nd edition. Jim Kurose, Keith Ross Addison-Wesley, July 2002.

2. Multimedia, Quality of Service: What is it?
Multimedia applications: network audio and video
(“continuous media”)
network provides application with level of performance needed for application to function.
QoS

3. Chapter 6: Goals Principles Classify multimedia applications
Identify the network services the apps need
Making the best of best effort service
Mechanisms for providing QoS
Protocols and Architectures
Specific protocols for best-effort
Architectures for QoS

4. Chapter 6 outline 6.1 Multimedia Networking Applications
6.2 Streaming stored audio and video
RTSP
6.3 Real-time Multimedia: Internet Phone Case Study
6.4 Protocols for Real-Time Interactive Applications
SIP

5. MM Networking Applications
Fundamental characteristics:
Typically delay sensitive
end-to-end delay
delay jitter
But loss tolerant: infrequent losses cause minor glitches
Antithesis of data, which are loss intolerant but delay tolerant.
Classes of MM applications:
1) Streaming stored audio and video
2) Streaming live audio and video
3) Real-time interactive audio and video
Jitter is the variability
of packet delays within
the same packet stream

6. Streaming Stored Multimedia
media stored at source
transmitted to client
streaming: client playout begins before all data has arrived
timing constraint for still-to-be transmitted data: in time for playout

7. Streaming Stored Multimedia: What is it?
streaming: at this time, client
playing out early part of video,
while server still sending later
part of video
3. video received,
played out at client
Cumulative data
2. video
sent
1. video
recorded
network
delay
time

8. Delay Jitter constant bit rate transmission client constant bit
variable
network
delay
(jitter)
client
reception
constant bit
rate playout
at client
client playout
delay
Cumulative data
buffered
data
time

9. Jitter: Definitions Input: t0, …, tn Delay Jitter: Rate Jitter
Delay jitter J
For every k: |t0 + kX - tk |  J
X = (tn - t0) / n
Rate Jitter
Rate Jitter A
Ik = tk - tk-1
For every k and j: |Ij - Ik|  A
Jitter and buffering
delay versus jitter

10. Streaming Stored Multimedia: Interactivity
VCR-like functionality: client can pause, rewind, FF, push slider bar
10 sec initial delay OK
1-2 sec until command effect OK
RTSP often used (more later)
timing constraint for still-to-be transmitted data: in time for playout

11. Streaming Live Multimedia
Examples:
Internet radio talk show
Live sporting event
Streaming
playback buffer
playback can lag tens of seconds after transmission
still have timing constraint
Interactivity
fast forward impossible
rewind, pause possible!

12. Interactive, Real-Time Multimedia
applications: IP telephony, video conference, distributed interactive worlds
end-end delay requirements:
audio: < 150 msec good, < 400 msec OK
includes application-level (packetization) and network delays
higher delays noticeable, impair interactivity
session initialization
how does callee advertise its IP address, port number, encoding algorithms?

13. Internet Multicasting

14. Multicast: Connecting multiple users
Unicast: connecting two users
Multicast: connecting multiple users
Conference call
Event broadcast
Chat room
Supporting multicast
Multiple unicast
Inefficient
Build a multicast tree
Assume cost on links

15. The Multicast Problem ? GIVEN: Network
GIVEN: User Requests (and quality of service requirements)
ESTABLISH: Efficient distribution (trees)

16. What is The Need for Multicasting?
Use bandwidth efficiently
Replace many unicast connections with one multicast connection
Duplicate packets only when necessary
Theoretical model
Links have cost
Find the minimum cost multicast tree
3 times the
traffic

17. Steiner Tree Problem Input: Output: Optimization: Minimum cost
Given a graph G(V,E)
Costs on edges: Cost(e)
Set of terminal nodes S
Output:
Tree T,
Includes all the nodes in S
Optimization: Minimum cost
Complexity: NP-complete

18. Steiner Tree: MST Heuristic
define GS = complete graph on terminals
edge cost = shortest path cost
find TMST = MST of GS
replace each edge of TMST with the path in G
output TMST

19. MST Approximation ratio
MST is at least 2-approximation
TMST = output of MST algo (on GS)
T* = optimal Steiner Tree in G
Cost(TMST,G) ≤ Cost(TMST,GS)
Cost(T*,GS) ≤ 2 Cost(T*,G)
Cost(TMST,GS) ≤ Cost(T*,GS)
Conclusion: Cost(TMST,G) ≤ 2C ost(T*,GS)
MST is at most 2-approximation
Edges cost black = 2 red =1
MST cost = 2k-2, OPT cost = k
Ratio 2-(1/k) 1 2 3 k 5 4

20. Multicast: Online setting
Online Steiner problem
Terminal arrive one at a time
Need to allocate a tree to current terminals
Online Greedy :
At time t:
Current tree Tt-1
arriving terminal vt
Find shortest path Pv from vt to Tt-1
Let Tt be the union of Pv and Tt-1

21. Performance of Online Greedy
Sort terminal by their cost
cost(u1) ≥ cost(u2) ≥ … ≥ cost(un)
CLAIM: cost(uk) ≤ 4 OPT/k (need to prove it …)
Assuming the claim:
cost(T) = cost(u1) + … + cost(un)
≤ 4 OPT/1 + … + 4OPT/n
≤ 4 OPT(1 + ½ + ⅓+¼+⅛… + 1/n)
≈ 4 ln(n) OPT

22. Proving the claim Consider Uk= {u1 , … , uk}
Suppose cost(uk) > 4 OPT /k,
Then cost(ui) > 4 OPT/k, for i<k (they where sorted)
Every ui and uj are distance at least 4 OPT/k
Let GS be the graph on Uk
with cost = shortest distance
Let TU be the MST of GS
Cost (Tu) > 4(k-1) OPT/k ≥ 2 OPT
Contradiction to the MST heuristic
which is at 2-approximation!
QED

23. Chapter 6 outline 6.1 Multimedia Networking Applications
6.2 Streaming stored audio and video
RTSP
6.3 Real-time Multimedia: Internet Phone Case Study
6.4 Protocols for Real-Time Interactive Applications
SIP

24. Streaming Stored Multimedia
Application-level streaming techniques for making the best out of best effort service:
client side buffering
use of UDP versus TCP
multiple encodings of multimedia
Media Player
jitter removal
decompression
error concealment
graphical user interface w/ controls for interactivity

25. Internet multimedia: simplest approach
audio or video stored in file
files transferred as HTTP object
received in entirety at client
then passed to player
audio, video not streamed:
no, “pipelining,” long delays until playout!

26. Internet multimedia: streaming approach
browser GETs metafile
browser launches player, passing metafile
player contacts server
server streams audio/video to player

27. Streaming from a streaming server
This architecture allows for non-HTTP protocol between server and media player
Can also use UDP instead of TCP.

28. Streaming Multimedia: Client Buffering
constant bit
rate video
transmission
variable
network
delay
client video
reception
constant bit
rate video
playout at client
client playout
delay
Cumulative data
buffered
video
time
Client-side buffering, playout delay compensate for network-added delay, delay jitter

29. Streaming Multimedia: Client Buffering
constant
drain
rate, d
variable fill
rate, x(t)
buffered
video
Client-side buffering, playout delay compensate for network-added delay, delay jitter

30. Streaming Multimedia: UDP or TCP?
server sends at rate appropriate for client (oblivious to network congestion !)
often send rate = encoding rate = constant rate
then, fill rate = constant rate - packet loss
short playout delay (2-5 seconds) to compensate for network delay jitter
error recover: time permitting
TCP
send at maximum possible rate under TCP
fill rate fluctuates due to TCP congestion control
larger playout delay: smooth TCP delivery rate
HTTP/TCP passes more easily through firewalls

31. Streaming Multimedia: client rate(s)
1.5 Mbps encoding
28.8 Kbps encoding
Q: how to handle different client receive rate capabilities?
0.5+ Kbps ADSL
100Mbps Ethernet
A: server stores, transmits multiple copies of video, encoded at different rates

32. User Control of Streaming Media: RTSP
HTTP
Does not target multimedia content
No commands for fast forward, etc.
RTSP: RFC 2326
Client-server application layer protocol.
For user to control display: rewind, fast forward, pause, resume, repositioning, etc…
What it doesn’t do:
does not define how audio/video is encapsulated for streaming over network
does not restrict how streamed media is transported; it can be transported over UDP or TCP
does not specify how the media player buffers audio/video

33. Chapter 6 outline 6.1 Multimedia Networking Applications
6.2 Streaming stored audio and video
RTSP
6.3 Real-time, Interactive Multimedia: Internet Phone Case Study
6.4 Protocols for Real-Time Interactive Applications
SIP

34. Real-time interactive applications
Going to now look at a PC-2-PC Internet phone example in detail
PC-2-PC phone
instant messaging services are providing this
PC-2-phone
videoconference with Webcams

35. Interactive Multimedia: Internet Phone
Introduce Internet Phone by way of an example
speaker’s audio: alternating talk spurts, silent periods.
64 kbps during talk spurt
pkts generated only during talk spurts
20 msec chunks at 8 Kbytes/sec: 160 bytes data
application-layer header added to each chunk.
Chunk+header encapsulated into UDP segment.
application sends UDP segment into socket every 20 msec during talkspurt.

36. Basic IP Telephony Audio Packet Transfer…
Digitization (e.g., sampling at 8kHz, 16 bits per sample, i.e, 128 kb/s or 320 bytes per 20 ms)
Real-time compression/encoding (e.g., G.729A at 8 kb/s)
Transport to remote IP address and port number over UDP (Why not TCP?)
Processing on receiver side is the reverse

37. Basic IP Telephony: Sampling, Quantization, Encoding
+127 … +0
-127
Sample at twice the highest voice frequency
2 x 4000=8000
Round off samples to one of 256 levels (introduces noise)

38. Internet Phone: Packet Loss and Delay
network loss: IP datagram lost due to network congestion (router buffer overflow)
delay loss: IP datagram arrives too late for playout at receiver
delays: processing, queueing in network; end-system (sender, receiver) delays
typical maximum tolerable delay: 400 ms
loss tolerance: depending on voice encoding, losses concealed, packet loss rates between 1% and 10% can be tolerated.

39. Basic IP Telephony: Problems with UDP
Sender 1 2 3 4 5 6 7
(a)
(b) 1 2 3 5 7 6
timeline
Receiver
Unreliable UDP
Packet loss
Out-of-order (very rarely)
Jitter (delay variation)

40. Basic IP Telephony: Receive buffer
Sequence number: to detect packet loss; Just ignore the loss!
Receive buffer: to absorb jitter
Sender 1 2 3 4 5 6 7 8 9 1 2 3 4 1 2 3 5 7 6 8 9 2 3 4
Receiver 1 2 7 1 2 5 7 8 9 2 3 1 2 3 5 6 7 8 9 2

41. Basic IP Telephony: Timestamp Vs sequence number
Silence suppression
Variable length packets
Sender t1 t2 t3 t4 t5 t6 t7 t8 t9 1 2 3
Silence … 4 5 6 7 1 2 3 4 5 6 7
Receiver
Playout time vs packet loss detection

42. Basic IP Telephony : Real-time Transport Protocol - RTP
IP header V CC M
Payload type
Sequence number P X
UDP header
Timestamp (proportional to sampling time)
RTP Header
Source identifier (SSrc)
Encoded
Audio
Optional contributors’ list (CSrc)
msg
Ver, Version. 2 bits. RTP version number. Always set to 2.
P, Padding. 1 bit. If set, this packet contains one or more additional padding bytes at the end which are not part of the payload. The last byte of the padding contains a count of how many padding bytes should be ignored. Padding may be needed by some encryption algorithms with fixed block sizes or for carrying several RTP packets in a lower-layer protocol data unit.
X, Extension. 1 bit. If set, the fixed header is followed by exactly one header extension.
CC, CSRC count. 4 bits. The number of CSRC identifiers that follow the fixed header.
M, Marker. 1 bit. The interpretation of the marker is defined by a profile. It is intended to allow significant events such as frame boundaries to be marked in the packet stream. A profile may define additional marker bits or specify that there is no marker bit by changing the number of bits in the payload type field.
PT, Payload Type. 7 bits. Identifies the format of the RTP payload and determines its interpretation by the application. A profile specifies a default static mapping of payload type codes to payload formats. Additional payload type codes may be defined dynamically through non-RTP means. An RTP sender emits a single RTP payload type at any given time; this field is not intended for multiplexing separate media streams.
sendto(…, msg, …)
recvfrom(…, msg, …)

43. Internet Phone: Fixed Playout Delay
Receiver attempts to playout each chunk exactly q msecs after chunk was generated.
chunk has time stamp t: play out chunk at t+q .
chunk arrives after t+q: data arrives too late for playout, data “lost”
Tradeoff for q:
large q: less packet loss
small q: better interactive experience

44. Fixed Playout Delay First packet received at time r
Sender generates packets every 20 msec during talk spurt.
First packet received at time r
First playout schedule: begins at p
Second playout schedule: begins at p’

45. Adaptive Playout Delay, I
Goal: minimize playout delay, keeping late loss rate low
Approach: adaptive playout delay adjustment:
Estimate network delay, adjust playout delay at beginning of each talk spurt.
Silent periods compressed and elongated.
Chunks still played out every 20 msec during talk spurt.
Dynamic estimate of average delay at receiver:
where u is a fixed constant (e.g., u = .01).

46. Adaptive playout delay II
Also useful to estimate the average deviation of the delay, vi :
The estimates di and vi are calculated for every received packet, although they are only used at the beginning of a talk spurt.
For first packet in talk spurt, playout time is:
where K is a positive constant.
Remaining packets in talkspurt are played out periodically

47. Adaptive Playout, III
Q: How does receiver determine whether packet is first in a talkspurt?
If no loss, receiver looks at successive timestamps.
difference of successive stamps > 20 msec -->talk spurt begins.
With loss possible, receiver must look at both time stamps and sequence numbers.
difference of successive stamps > 20 msec and sequence numbers without gaps --> talk spurt begins.

48. Recovery from packet loss (1)
forward error correction (FEC): simple scheme
for every group of n chunks create a redundant chunk by exclusive OR-ing the n original chunks
send out n+1 chunks, increasing the bandwidth by factor 1/n.
can reconstruct the original n chunks if there is at most one lost chunk from the n+1 chunks
Playout delay needs to be fixed to the time to receive all n+1 packets
Tradeoff:
increase n, less bandwidth waste
increase n, longer playout delay
increase n, higher probability that 2 or more chunks will be lost

49. Recovery from packet loss (2)
2nd FEC scheme
“piggyback lower quality stream”
send lower resolution audio stream as the redundant information
for example, nominal stream PCM at 64 kbps and redundant stream GSM at 13 kbps.
Whenever there is non-consecutive loss, the receiver can conceal the loss.
Can also append (n-1)st and (n-2)nd low-bit rate chunk

50. Recovery from packet loss (3)
Interleaving
chunks are broken up into smaller units
for example, 4 -5 units per chunk
Packet contains small units from different chunks
if packet is lost, still have most of every chunk
has no redundancy overhead
but adds to playout delay

51. Summary: Internet Multimedia: bag of tricks
use UDP to avoid TCP congestion control (delays) for time-sensitive traffic
client-side adaptive playout delay: to compensate for delay
server side matches stream bandwidth to available client-to-server path bandwidth
chose among pre-encoded stream rates
dynamic server encoding rate
error recovery (on top of UDP)
FEC, interleaving
retransmissions, time permitting
conceal errors: repeat nearby data

52. Chapter 6 outline 6.1 Multimedia Networking Applications
6.2 Streaming stored audio and video
RTSP
6.3 Real-time, Interactive Multimedia: Internet Phone Case Study
6.4 Protocols for Real-Time Interactive Applications
SIP

53. SIP Session Initiation Protocol Comes from IETF SIP long-term vision
All telephone calls and video conference calls take place over the Internet
People are identified by names or addresses, rather than by phone numbers.
You can reach the callee, no matter where the callee roams, no matter what IP device the callee is currently using.

54. SIP Services Setting up a call Determine current IP address of callee.
Provides mechanisms for caller to let callee know she wants to establish a call
Provides mechanisms so that caller and callee can agree on media type and encoding.
Provides mechanisms to end call.
Determine current IP address of callee.
Maps mnemonic identifier to current IP address
Call management
Add new media streams during call
Change encoding during call
Invite others
Transfer and hold calls

55. Setting up a call to a known IP address
Alice’s SIP invite message indicates her port number & IP address. Indicates encoding that Alice prefers to receive (PCM ulaw)
Bob’s 200 OK message indicates his port number, IP address & preferred encoding (GSM)
SIP messages can be sent over TCP or UDP; here sent over RTP/UDP.
Default SIP port number is 5060.

56. Setting up a call (more)
Codec negotiation:
Suppose Bob doesn’t have PCM ulaw encoder.
Bob will instead reply with 606 Not Acceptable Reply and list encoders he can use.
Alice can then send a new INVITE message, advertising an appropriate encoder.
Rejecting the call
Bob can reject with replies “busy,” “gone,” “payment required,” “forbidden”.
Media can be sent over RTP or some other protocol.

57. Example of SIP message INVITE sip:bob@domain.com SIP/2.0
Via: SIP/2.0/UDP
From:
To:
Call-ID:
Content-Type: application/sdp
Content-Length: 885
c=IN IP
m=audio RTP/AVP 0
Notes:
HTTP message syntax
sdp = session description protocol
Call-ID is unique for every call.
Here we don’t know
Bob’s IP address.
Intermediate SIP servers will be necessary.
Alice sends and receives SIP messages using the SIP default port number 5060.
Alice specifies in Via: header that SIP client sends and receives SIP messages over UDP

58. Name translation and user location
Caller wants to call callee, but only has callee’s name or address.
Need to get IP address of callee’s current host:
user moves around
DHCP protocol
user has different IP devices (PC, PDA, car device)
Result can be based on:
time of day (work, home)
caller (don’t want boss to call you at home)
status of callee (calls sent to voic when callee is already talking to someone)
Service provided by SIP servers:
SIP registrar server
SIP proxy server

59. SIP Registrar
When Bob starts SIP client, client sends SIP REGISTER message to Bob’s registrar server
(similar function needed by Instant Messaging)
Register Message:
REGISTER sip:domain.com SIP/2.0
Via: SIP/2.0/UDP
From:
To:
Expires: 3600

60. SIP Proxy Alice send’s invite message to her proxy server
contains address
Proxy responsible for routing SIP messages to callee
possibly through multiple proxies.
Callee sends response back through the same set of proxies.
Proxy returns SIP response message to Alice
contains Bob’s IP address
Note: proxy is analogous to local DNS server

61. Example Caller jim@umass.edu with places a call to keith@upenn.edu
(1) Jim sends INVITE message to umass SIP proxy. (2) Proxy forwards request to upenn registrar server. (3) upenn server returns redirect response, indicating that it should try
(4) umass proxy sends INVITE to eurecom registrar. (5) eurecom registrar forwards INVITE to , which is running keith’s SIP client. (6-8) SIP response sent back (9) media sent directly
between clients.
Note: also a SIP ack message, which is not shown.

62. Comparison with H.323
H.323 is another signaling protocol for real-time, interactive
H.323 is a complete, vertically integrated suite of protocols for multimedia conferencing: signaling, registration, admission control, transport and codecs.
SIP is a single component. Works with RTP, but does not mandate it. Can be combined with other protocols and services.
H.323 comes from the ITU (telephony).
SIP comes from IETF: Borrows much of its concepts from HTTP. SIP has a Web flavor, whereas H.323 has a telephony flavor.
SIP uses the KISS principle: Keep it simple stupid.