1. University of Delaware
Real-Time Protocols
RTP/RTCP/RTSP
Amit Hetawal
University of Delaware
CISC 856 -Fall 2005
Thanks to Professor Amer

2. Overview History of streaming media Streaming performance requirements
Protocol stack for multimedia services
Real-time transport protocol (RTP)
RTP control protocol (RTCP)
Real-time streaming protocol (RTSP)

3. Brief history of streaming media

4. Real-time multimedia streaming
Real-time multimedia applications
Video teleconferencing
Internet Telephony (VoIP)
Internet audio, video streaming
(A-PDUs)

5. Streaming performance requirements
Sequencing
– to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment –covers up errors from lost PDU by using redundancy in most-adjacent-frame
Quality of Service (QoS) feedback – from receiver to sender for operation adjustment
Rate control –sender reduces sending rate adaptively to network congestion

6. Ideal Timing – no jitter
30 seconds
First RTP-PDU
application
Second RTP-PDU
Third RTP-PDU
Send time
Play time

7. Reality – jitter Send time Play time delay 00.00.00 00.00.10
First RTP-PDU
Second RTP-PDU
Third RTP-PDU
Fourth RTP-PDU
Send time
Play time

8. Jitter (contd.) Send time Play time 00.00.00 00.00.10 First RTP-PDU(0)
Second RTP-PDU(10)
Third RTP-PDU(20)
Fourth RTP-PDU (30)
Send time
Play time

9. Jitter (contd.) Playback buffer At time 00:00:18 At time 00:00:28

10. How does Sequence number and Timestamp help ?
Audio silence example:
sender
receiver
Seq no.1, Tmpst 100
Seq no.2, Tmpst 200
Seq no.3, Tmpst 300
Seq no.4, Tmpst 600
Seq no.5, Tmpst 700
Consider audio data
What should the sender do during silence?
silence
Not send anything
Why might this cause problems?
Receiver cannot distinguish between loss and silence
Solution:
After receiving no PDUs for a while, next PDU received at the receiver will reflect a big jump in timestamp, but have the correct next seq. no. Thus, receiver knows what happened.

11. Streaming performance requirements
Sequencing
– to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment –covers up errors from lost PDU by using redundancy in most-adjacent-frame
Quality of Service (QoS) feedback – from receiver to sender for operation adjustment
Rate control –sender reduces sending rate adaptively to network congestion

12. Support from transport layers
TCP is not used because:
TCP does retransmissions  unbounded delays
No provision for time stamping
TCP does not support multicast
TCP congestion control (slow-start) unsuitable for real-time transport
RTP + UDP usually used for multimedia services

13. Protocol stack for multimedia services
RTSP
RTP
RTCP
TCP
(till now)

14. RTP: Introduction Application RTP RTCP UDP IP Data Link Physical
Provides end-to-end transport functions for real-time applications
Supports different payload types
All RTP and RTCP PDUs are sent to same multicast group (by all participants)
All RTP PDUs sent to an even-numbered UDP port, 2p
All RTCP PDUs sent to UDP port 2p+1
Does NOT provide timely delivery or other QoS guarantees
Relies on other protocols like RTCP and lower layers
Does NOT assume the underlying network is reliable and delivers PDUs in sequence
Uses sequence number
Application
RTP RTCP
Transport
layer
UDP IP
Data Link
Physical

15. RTP Session
RTP session is sending and receiving of RTP data by a group of participants
For each participant, a session is a pair of transport addresses used to communicate with the group
If multiple media types are communicated by the group, the transmission of each medium constitutes a session.

16. RTP Synchronization Source
synchronization source - each source of RTP PDUs
Identified by a unique,randomly chosen 32-bit ID (the SSRC)
A host generating multiple streams within a single RTP must use a different SSRC per stream

17. RTP Basics of Data Transmission
RTP PDUs

18. RTP PDU Header (used by mixers) Sampling instant of first data octet
multiple PDUs can have same timestamp
not necessarily monotonic
used to synchronize different
media streams
Incremented by one for each RTP PDU:
PDU loss detection
Restore PDU sequence
Payload type
Identifies synchronization source
(used by mixers)
Identifies contributing sources

19. Mixer
RTP mixer - an intermediate system that receives & combines RTP PDUs of one or more RTP sessions into a new RTP PDU
Stream may be transcoded, special effects may be performed.
A mixer will typically have to define synchronization relationships between streams.Thus…
Sources that are mixed together become contributing sources (CSRC)
Mixer itself appears as a new source having a new SSRC

20. Translator end system 1 end system 2 transl.1 transl.2
An intermediate system that…
Connects two or more networks
Multicasting through a firewall
Modifies stream encoding, changing the stream’s timing
Transparent to participants
SSRC’s remain intact
end system 1
end system 2
transl.1
from ES1: SSRC=6
from ES2: SSRC=23
transl.2
authorized tunnel
firewall

21. RTP Control Protocol (RTCP)
RTCP specifies report PDUs exchanged between sources and destinations of multimedia information
receiver reception report
sender report
source description report
Reports contain statistics such as the number of RTP-PDUs sent, number of RTP-PDUs lost, inter-arrival jitter
Used by application to modify sender transmission rates and for diagnostics purposes

22. RTCP message types
Typically, several RTCP PDUs of different types are transmitted in a single UDP PDU

23. Sender/Receiver report PDUsRC
PT=200/201  SR/RR
Length (16 bits)
Header
SSRC of Sender
RTP Timestamp
Sender’s PDU Count
NTP Timestamp, most significant word
NTP Timestamp, least significant word
Sender’s Octet Count
Sender Info
SSRC_1 (SSRC of the 1st Source)
Fraction Lost
Cumulative Number of PDU Lost
Extended Highest sequence Number Received
Report Block 1
Interarrival Jitter
Last SR (LSR)
Delay Since Last SR (DLSR)
Report Block 2
SSRC_2 (SSRC of the 2nd Source)
… …
Profile-Specific Extensions

24. Ethereal capture for RTP-PDU
Basic header

25. Ethereal capture for RTCP-PDU
header of SR report
sender info
receiver report block
SDES items

26. Synchronization of streams using RTCP
RTP audio
RTCP audio
RTP video
RTP video
Internetwork
Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks
timestamps are not tied to the wall-clock time, or each other!
Each RTCP sender-report PDU contains (for most recently generated PDU in associated RTP stream):
The timestamp of RTP PDU
The wall-clock time for when PDU was created
Receivers can use this association to synchronize the playout of audio and video

27. RTCP bandwidth scaling
Problem
What happens when there is one sender and many receivers?
RTCP reports scale linearly with the number of participants and would match or exceed the amount of RTP data! More overhead than useful data!
Example
Suppose one sender, sending video at a rate of 2 Mbps. Then RTCP attempts to limit its traffic to 100 Kbps.
The 75 kbps is equally shared among receivers:
With R receivers, each receiver gets to send RTCP traffic at 75/R kbps.
Sender gets to send RTCP traffic at 25 kbps.
Solution
RTCP attempts to limit its traffic to 5% of the session bandwidth to ensure it can scale!
RTCP gives 75% of this rate to the receivers; and the remaining 25% to the sender.

28. Real-Time Streaming Protocol (RTSP)
Application layer protocol (default port 554)
Usually runs on RTP for stream & TCP for control
Provides the control channel
Uses out-of-band signaling
Usable for Live broadcasts / multicast
Also known as “Network remote control” for multi-media servers.

29. presentation descriptor
RTSP Overview
web browser
media player
Web Server
Web Server/Media server
HTTP
presentation descriptor
Presentation
descriptor
RTSP
pres. desc,streaming commands
RTP/RTCP
audio/video content

30. RTSP Methods OPTIONS C  S determine capabilities of server/client
DESCRIBE
get description of media stream
ANNOUNCE
C  S
announce new session description
SETUP
create media session
RECORD
start media recording
PLAY
start media delivery
PAUSE
pause media delivery
REDIRECT
redirection to another server
TEARDOWN
immediate teardown
SET_PARAMETER
change server/client parameter
GET_PARAMETER
read server/client parameter

31. RTSP Session RTSP server RTSP TCP client data source UDP media server
Default port 554
media server
RTSP SETUP
media player
RTSP
server
RTSP OK
TCP
UDP
RTSP PLAY
RTSP
client
RTSP OK
RTSP TEARDOWN
get UDP port
RTSP OK
data
source
choose
UDP port
RTP VIDEO AV
subsystem
RTP AUDIO
RTCP

32. Example:Media on demand (Unicast)
Media server A
audio.example.com
Media server V
video.example.com
Web server W
-holds the media descriptors
Client C

33. RTSP Message sequence W V C A C -> W : GET/Twister.sdp HTTP/1.1
Host:
Accept: application/sdp
W-> C : HTTP/ OK
Content-Type: application/sdp C W V A
C-> A : SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=
A-> C : RTSP/ OK
Session:
server_port=
C->V : SETUP rtsp://video.example.com/twister/video.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=
A-> C : RTSP/ OK
Session:
server_port=

34. RTSP Message sequence (contd.)
C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 2
Session:
V->C: RTSP/ OK
RTP-Info: url=rtsp://video.example.com/twister/video;
seq= ; C W V A
C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 2
Session:
A->C: RTSP/ OK
RTP-Info: url=rtsp://audio.example.com/twister/audio.en;
seq=876655;

35. RTSP Message sequence (contd.)
C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 3
Session:
A->C: RTSP/ OK C W V A
C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 3
Session:
V->C: RTSP/ OK

36. References [1] B. A. Forouzan, “TCP/IP Protocol Suite”, Third edition,
[2] H. Schulzrinne, S. Casner, R. Frederick and V.
Jacobson, "RTP: a transport protocol for real-time
applications", RFC 3550, July 2003.
[3] H. Schulzrinne, A. Rao and R. Lanphier, "Real Time
Streaming Protocol (RTSP)", RFC 2326, April 1998.

37. RTCP compound PDU SR SDES compound PDU (single UDP datagram) sender
report
receiver
SSRC
source 2
source 3
RTCP PDU 1
SDES
CNAME PHONE
RTCP PDU 2
compound PDU
(single UDP datagram)

38. Example source 1 reports, there are 2 other sources SR sender report
receiver
SSRC
source 2
source 3
RTCP PDU

39. RTCP processing in Translators
SR sender information : Does not generate their own sender information(most of the times), but forwards the SR PDUs received from one side to other
RR reception report blocks : Does not generate their own RR reports (most of the times), but forwards RR reports received from one side to another. SSRC are left intact
SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A translator about to cease forwarding, send a BYE PDU to each connected nodes

40. RTCP processing in Mixers
SR sender information : Generates its own SR info. Because the characteristics of source stream is lost in the mix. The SR info is sent in same direction as the mixed stream
RR reception report blocks : Generates its own reports for sources in each cloud and sends them only to same cloud
SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A mixer about to cease forwarding, send a BYE PDU to each connected nodes

41. Source description PDUs
May contain:
a CNAME item (canonical identifier/name)
a NAME item (real user name)
an item
a PHONE item
a LOC item (geographic location)
a TOOL item (application name)
a NOTE item (transient msg, e.g. for status)
a PRIV item (private extension)
Value 1 2 3 4 5 6 7 8
CNAME=1
length
user and domain name