1. MULTIMEDIA SYSTEMS IREK DEFEE MULTIMEDIA SYSTEMS NETWORKING

2. MULTIMEDIA SYSTEMS IREK DEFEE SERVERCLIENTNETWORK SOFTWARE MULTIMEDIA SYSTEM

3. MULTIMEDIA SYSTEMS IREK DEFEE WE CONSIDER MULTIMEDIA SYSTEMS BUILT-UP ON LARGE SCALE: REACHING PRACTICALLY EVERYBODY, LIKE TV, RADIO AND TELEPHONE TODAY, THAT IS THOUSANDS AND MILIONS OF USERS MULTIMEDIA MEANS BROADBAND STREAMING AND INTERACTIVITY

4. MULTIMEDIA SYSTEMS IREK DEFEE MAIN PROBLEMS IN NETWORKING NETWORKS FOR MULTIMEDIA REQUIRE: -SUFFICIENT/CONTROLLED BANDWIDTH -RESERVATION OF NETWORK RESOURCES (WE CAN NOT LOSE DATA) -STREAMING -FAST INTERACTIVITY DELIVERY CAN BE WIRED OR WIRELESS: WIRED - USES PHYSICAL LINKS WIRELESS – USES RADIO

5. MULTIMEDIA SYSTEMS IREK DEFEE A MODEL OF INTERACTIVE SYSTEM IS INTERNET A MODEL OF BROADBAND STREAMING ARE TV, RADIO MULTIMEDIA SYSTEMS WOULD BE SOMETHING LIKE INTEGRATION OF THEM BOTH SOMETHING LIKE BECAUSE WE CAN NOT PREDICT IN DETAIL WHAT IT MIGHT BE

6. MULTIMEDIA SYSTEMS IREK DEFEE COMMUNICATION SERVICE TYPES -BROADCAST: SINGLE SOURCE, MANY USERS – LIKE TV, 1-WAY DISTRIBUTION - MULTICAST: LIKE BROADCAST BUT USERS SIGN TO A SESSION, ONE WAY CONNECTION -RETRIEVAL, UNICAST – ASYMMETRIC CONNECTION, USERS RECEIVE MUCH MORE THAN SEND, E.G. WWW

7. MULTIMEDIA SYSTEMS IREK DEFEE COMMUNICATIVE SERVICE – SYMMETRIC 2-WAY CONNECTION LIKE TELEPHONE IN MULTIMEDIA SYSTEMS WE ARE MOSTLY INTERESTED IN THE RETRIEVAL AND COMMUNICATIVE SERVICES WITH HIGH BIT RATE

8. MULTIMEDIA SYSTEMS IREK DEFEE STREAMING STREAMING MEANS DATA ARE SEND SYNCHRONOUSLY IN TIME AND WILL REACH USER PRESERVING THEIR TIME ORGANIZATION. NO DATA LOSS OR DELAYS ARE ALLOWED. EXAMPLE: TELEPHONE NETWORK IN MULTIMEDIA STREAMING IS ABSOLUTELY NECESSARY

9. MULTIMEDIA SYSTEMS IREK DEFEE THIS IS VERY MUCH RELATED TO THE PROBLEM OF NETWORK RESOURCE MANAGEMENT AND CONTROL THERE ARE TWO TYPES OF NETWORKS -CONNECTION-ORIENTED -CONNECTIONLESS -IN CONNECTION-ORIENTED NETWORK RESOURCES ARE ALLOCATED BEFORE

10. MULTIMEDIA SYSTEMS IREK DEFEE THE DATA TRANSFER IS STARTED. IN CONNECTIONLESS NETWORK RESERVATION IS NOT DONE: DATA MAY BE TRANSFERRED WITHOUT ANY GUARANTEE OF RESOURCE ALLOCATION, OR WITH SOME GUARANTEE ONLY.

11. MULTIMEDIA SYSTEMS IREK DEFEE HOW STREAMING CAN BE ENSURED IN NETWORKING? THERE ARE THREE BASIC TYPES OF NETWORKING -CIRCUIT SWITCHING - CONNECTION IS ’SWITCHED’ BEFORE SENDING DATA (”PHONE”) -PACKET SWITCHING – DATA ARE ORGANIZED IN PACKETS. EACH PACKET CARRIES ADDRESSES

12. MULTIMEDIA SYSTEMS IREK DEFEE - CELL SWITCHING – BETWEEN CIRCUIT AND PACKET SWITCHING ”PACKETS ARE SWITCHED” (TECHNOLOGY CALLED ATM – ASYNCHRONOUS TRANSFER MODE) ALL THESE SYSTEMS HAVE VARIOUS POSIIVE AND NEGATIVE ASPECTS FOR MULTIMEDIA

13. MULTIMEDIA SYSTEMS IREK DEFEE IN CIRCUIT SWITCHING NETWORK RESOURCES ARE FULLY RESERVED FOR STREAM TRANSMISSION IN PACKET SWITCHING PACKETS ARE FLOWING IN LITTLE CONTROLLED WAY MAKING STREAMING PROBLEMATIC IN CIRCUIT SWITCHING INTERACTIVITY IS SLOW (TIME FOR CONNECTION NEEDED) IN PACKET SWITCHING INTERACTIVITY IS FAST

14. MULTIMEDIA SYSTEMS IREK DEFEE EXAMPLES: -TELEPHONE SYSTEM – SWITCHING (connection establishment) IS DONE IN EXCHANGES, SWITCHBOARDS OR SWITCHING CENTERS -INTERNET - PACKETS ARE DIRECTED IN ROUTERS WHICH READ THEIR ADDRESSES AND DIRECT THEM The difference is that only circuit switching fully guarantees streaming but it is expensive

15. MULTIMEDIA SYSTEMS IREK DEFEE ADVANTAGES OF CIRCUIT SWITCHING: -GUARANTEED DATA DELIVERY -STREAMING AND TIMING OF DATA NO PROBLEM -DISADAVANTAGES: -CIRCUIT ESTABLISHMENT (”call”) TAKES TIME - EXPENSIVE INFRASTRUCTURE

16. MULTIMEDIA SYSTEMS IREK DEFEE PACKET SWITCHING EXAMPLE: INTERNET DATA ARE ORGANIZED IN PACKETS PACKETS CARRY HEADERS WITH SOURCE AND DESTINATION ADDRESS PACKETS ARE ROUTED IN THE NETWORK BASING ON THE ADDRESSESS

17. MULTIMEDIA SYSTEMS IREK DEFEE ADVANTAGES OF PACKET SWITCHING - SIMPLICITY, EVEN A PC CAN BE ROUTER -PACKETS CAN BE ROUTED IMMEDIATELY -PACKETS CAN BE SEND USING DIFFERENT ROUTES, OR STORED EASY ADAPTATION TO THE AMOUNT OF TRAFFIC - IDEAL FOR TIME NON-CRITICAL DATA

18. MULTIMEDIA SYSTEMS IREK DEFEE DISADVANTAGES: -NO GUARANTEE FOR PACKET DELIVERY -NO GUARANTEE FOR RESOURCES NO GUARANTEE FOR PACKET DELIVER ON-TIME, NO STREAMING

19. MULTIMEDIA SYSTEMS IREK DEFEE EXAMPLE – WWW. WE GET INSTANT RESPONSE FROM THE PS INTERNET IF THE INTERNET WOULD BE CS, WE WOULD NEED TO WAIT FOR CONNECTION ESTABLISHMENT. BUT THERE IS NO RESOURCE RESERVATION FOR STREAMS IN PS A SOLUTION FOR THIS PROBLEM COULD BE INTELLIGENT PACKET SWITCHING AND NETWORKING

20. MULTIMEDIA SYSTEMS IREK DEFEE IN INTELLIGENT PS PACKETS HAVE PRIORITIES ASSIGNED, MULTIMEDIA PACKETS GET HIGH PRIORITY SO THEY WILL NOT BE DISTURBED BY OTHER PACKETS IF THE NETWORK WOULD HAVE ENOUGH CAPACITY, MULTIMEDIA PACKETS WILL FLOW IN STREAMS (if there is not enough capacity, no help)

21. MULTIMEDIA SYSTEMS IREK DEFEE THUS PACKET SWITCHING CAN BE IN PRINCIPLE BETTER BECAUSE OF STREAMING AND FAST INTERACTIVITY CURRENT NETWORKING IS THUS EVOLVING IN THIS DIRECTION (INTERNET IS BECOMING VERY POWERFUL)

22. MULTIMEDIA SYSTEMS IREK DEFEE CELL SWITCHING OVERVIEW THIS TECHNOLOGY IS REALIZED IN ONE PARTICULAR FORM CALLED ATM – ASYNCHRONOUS TRANSFER MODE IT IS A COMBINATION OF CIRCUIT SWITCHING AND PACKET SWITCHING PACKETS HAVE CONSTANT, VERY SHORT LENGTH – 48DATA+5HEADER=53BYTES/PACKET

23. MULTIMEDIA SYSTEMS IREK DEFEE ATM ENABLES COMBINATION OF CIRCUIT AND PACKET SWITCHING BASIC CONCEPTS ARE VIRTUAL CHANNEL AND VIRTUAL PATH HPAYL.H H H FLOW OF ATM CELLS H – HEADER 5 BYTES PAYLOAD 48 BYTES

24. MULTIMEDIA SYSTEMS IREK DEFEE HEADERS ESTABLISH WHERE THE PACKETS ARE TRANSMITTED GFC VPI VCI PT CL HEC STRUCTURE OF ATM CELL HEADER: GFC - GENERIC FLOW CONTROL VPI - VIRTUAL PATH IDENTIFIER VCI - VIRTUAL CIRCUIT IDENTIFIER PT - PAYLOAD TYPE CL - CALL LOSS PRIORITY HEC – HEADER ERROR CHECK 4 8 16 3 1 8 bits

25. MULTIMEDIA SYSTEMS IREK DEFEE FOR MULTIMEDIA DATA ONE VP COULD BE ALLOCATED AND SEVERAL VC FOR E.G. VC=1 VIDEO VP=5 VC=2 AUDIO VC=3 TEXT

26. MULTIMEDIA SYSTEMS IREK DEFEE IN THE ATM NETWORK THERE ARE SWITCHES WHICH DIRECT PACKETS ACCORDING TO THEIR VP AND VC SWITCH 1 SWITCH 2 VP=1 VC=5 VP=6 VC=13 VP=4 VC=2 VP AND VC BETWEEN THE SWITCHES (THERE CAN BE MANY OF THEM HAS TO BE ASSIGNED TO ENABLE END-TO-END TRANSMISSION

27. MULTIMEDIA SYSTEMS IREK DEFEE HOW THE ASSIGNMENT IS DONE? IT CAN BE DONE BY HAND OR IT CAN BE DONE AUTOMATICALLY USING SPECIAL SYSTEM FOR MAPPING END POINT NUMBERS TO VP AND VC OF INTERMEDIATE SWITCHES

28. MULTIMEDIA SYSTEMS IREK DEFEE IN ATM THIS IS SOLVED IN A WAY WHICH IS SIMILAR TO TELEPHONE NETWORKS IT IS POSSIBLE TO ESTABLISH CONNECTION BETWEEN ANY END-POINTS BECAUSE THEY HAVE UNIQUE NUMBERS

29. MULTIMEDIA SYSTEMS IREK DEFEE THE ATM CALLS CAN SPECIFY BANDWIDTH AND OTHER PARAMTERS THERE ARE SEVERAL CLASSES OF SERVICES AAL – ATM ADAPTATION LAYER AAL1- CONSTANT BITRATE WITH TIMING

30. MULTIMEDIA SYSTEMS IREK DEFEE AAL2 – VARIABLE BIT RATE WITH TIMING AAL3/4- VARIABLE BIT RATE WITH NO TIMING AAL5 VARIABLE BIT RATE WITH NO TIMING, CONNECTIONLESS FOR THESE AAL’S IT IS SPECIFIED HOW DIFFERENT PROTOCOLS ARE MAPPED ON THEM

31. MULTIMEDIA SYSTEMS IREK DEFEE FOR EXAMPLE MPEG-2 TRANSPORT STREAM IS MAPPED INTO 2 X188 MPEG-2 PACKETS -> 8 ATM CELLS AAL2 IS IDEAL FOR MULTIMEDIA BUT IT IS DIFFICULT TO IMPLEMENT (TIMING) LARGE ATM NETWORKS ONLY RARELY ARE IMPLEMENTED

32. MULTIMEDIA SYSTEMS IREK DEFEE BUT IN NEW THIRD GENERATION WIRELESS NETWORKS - WIRELESS LAN - WIRELESS CELLULAR ATM AND AAL2 (WIRELESS) IS SPECIFIED AND IMPLEMENTED THUS, THEY MAY OPERATE IN THE ATM MODE

33. MULTIMEDIA SYSTEMS IREK DEFEE PACKET SWITCHING – IP INTERNET IN STANDARD PACKET SWITCHING THERE IS NO NETWORK RESOURCE RESERVATION BEFORE THE PACKETS ARE SEND. THE BASIC METHOD OF TRANSFERRING PACKETS ON THE INTERNET IS THE UDP – USER DATAGRAM PROTOCOL

34. MULTIMEDIA SYSTEMS IREK DEFEE IPv4 Header Structure Ip version Header_length Tos Total length IdentificationFlags and fragmentation TTL ProtocolHeader check sum 32 bit Source Ip address 32 bit Destination address Options(if any) Data

35. MULTIMEDIA SYSTEMS IREK DEFEE UDP/IP PACKETS ARE ROUTED ACCORDING TO THEIR ADRESSES, ONCE SENT, THEY ARE UNCONTROLLED THIS CAN BE VERY UNRELIABLE BUT ON RELIABLE AND HIGH BANDWIDTH LINKS IT CAN BE SUFFICIENT FOR MULTIMEDIA

36. MULTIMEDIA SYSTEMS IREK DEFEE THE TCP/IP PROTOCOL IS USED TO PROVIDE INFORMATION THAT PACKETS REACHED THEIR DESTINATION, THERE IS CONFIRMATION SEND ABOUT THEIR RECEPTION IF THERE IS NO CONFIRMATION, PACKETS ARE RESEND TCP/IP IS GOOD FOR NON-TIME CRITICAL DATA (EMAIL)

37. MULTIMEDIA SYSTEMS IREK DEFEE IP NETWORKING FOR MULTIMEDIA FOR MULTIMEDIA DATA PACKET SWITCHING INTERNET CAN BE VERY BAD ON THE OTHER HAND INTERNET IS CHEAP, AVAILABLE AND UBIQUITOUS

38. MULTIMEDIA SYSTEMS IREK DEFEE FOR SOLVING PROBLEMS WITH MULTIMEDIA DATA OVER IP PACKET NETWORK, THERE ARE THREE APPROACHES: -INTRODUCING RESOURCE RESERVATION, PROTOCOL CALLED RSVP – RESOURCE RESERVATION PROTOCOL AIMS TO DO IT - INTRODUCE PACKET PRIORITY LABELLING (DONE IN ROUTERS)

39. MULTIMEDIA SYSTEMS IREK DEFEE BOTH WOULD REQUIRE IMPLEMENTATION IN ROUTERS AND ARE COMPLICATED THIRD APPROACH - SIMPLER -TRYING TO FOLLOW HOW PACKETS ARE FLOWING THROUGH THE NETWORK AND SEND INFORMATION TO THE SENDING SIDE

40. MULTIMEDIA SYSTEMS IREK DEFEE REAL-TIME PROTOCOL RTP IN THIS PROTOCOL EACH PACKET GETS TIME STAMP AND NUMBER WHEN IT IS SEND. ON THE RECEIVING SIDE IT IS POSSIBLE TO DETECT IF PACKETS ARE LOST AND WHAT ARE THE PACKETS DELAYS. THECOMPLETE PACKET LOOKS LIKE THIS

41. MULTIMEDIA SYSTEMS IREK DEFEE THERE IS A PROTOCOL CALLED RTCP – REAL TIME CONTROL PROTOCOL WHICH COLLECTS STATISTICAL INFORMATION ABOUT PACKET DELAYS AND LOSS AND SENDS IT TO THE SENDING SIDE WHICH MAY REACT IN SOME WAY

42. MULTIMEDIA SYSTEMS IREK DEFEE THE RTP/RTCP COMBINATION IS SIMPLE AND REALISTIC FOR THE CURRENT INTERNET: -DOES NOT CHANGE ANYTHING IN THE SYSTEM -PROVIDES CONCISE REPORTING -SENDING SIDE MAY ESTIMATE QUALITY OF CONNECTION AND REACT E.G. BY REDUCING BITRATE

43. MULTIMEDIA SYSTEMS IREK DEFEE RTP INTRODUCTION  provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio or video.  does not address resource reservation and does not guarantee quality-of-service for real-time services.  independent of the underlying transport and network layers.  RTP packet consists fixed RTP header, a possibly empty list of contributing sources and the payload data

44. MULTIMEDIA SYSTEMS IREK DEFEE RTP header T TIMESTAMP SYNCHRONIZATION SOURCE IDENTIFIER (SSRS) VPMCCPTXSEQUENCE NUMBER CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...

45. MULTIMEDIA SYSTEMS IREK DEFEE RTP header Version (2 bits) identifies the version of RTP Padding (1 bit) packet contains one or more additional padding octets Extension (1 bit) the fixed header must be followed by one header extension CSRC Count (4 bits) contains the number of CSRC identifiers that follow the fixed header. VPCCX

46. MULTIMEDIA SYSTEMS IREK DEFEE RTP header Marker (1 bit) the packet contains marker bits Payload Type (7 bits) identifies the format of the RTP payload Sequence number (16 bits) increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence. MPTSEQUENCE NUMBER

47. MULTIMEDIA SYSTEMS IREK DEFEE RTP header Timestamp (32 bits) reflects the sampling instant of the first octet in the RTP data packet SSRC (32 bits) field identifies the synchronization source. CSRC list (0 to 15 items, 32 bits each) identifies contributing sources for the payload. The number of identifiers is given by the CC field. TIMESTAMP SYNCHRONIZATION SOURCE IDENTIFIER (SSRS) CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...

48. MULTIMEDIA SYSTEMS IREK DEFEE RTP Applications Designed for multiparticipant multimedia conferences –Storage of continuos data –Interactive distributed simulation –Control and measurement applications

49. MULTIMEDIA SYSTEMS IREK DEFEE Implementation of RTP Typically integrated into application processing No direct coupling at the RTP level between different media –For example, audio and video are in different sessions –Enables the choice of receiving only audio signal With UDP, RTP uses an even port number and the corresponding RTCP stream uses the next higher odd port number

50. MULTIMEDIA SYSTEMS IREK DEFEE RTP/UDP/IP

51. MULTIMEDIA SYSTEMS IREK DEFEE Application dependent protocol Complete implementation of RTP for a particular application requires a profile and a payload format (e.g., media encodings) specification

52. MULTIMEDIA SYSTEMS IREK DEFEE Profile Defines –Mapping of a set of payload formats to payload type values –Extensions into the RTP and RTCP headers RTP Profile for –Audio and Video Conferences with Minimal Control (IETF 1890) –Source Authentication and Non-Repudiation of Audio and Video Conferences (Internet Draft) –Interactive media (proposed in journal paper)

53. MULTIMEDIA SYSTEMS IREK DEFEE Payload Types RFCs –H.261 and H.263 video streams; Redundant Audio Data; MPEG1/MPEG2 Video; Bundled MPEG; JPEG- compressed Video; Generic Forward Error Correction –MPEG-4 IN PREPARATION Internet Drafts: –MPEG-4 Streams; Interleaved Media; DV Format Video; MPEG-2 AAC Streams; DTMF Digits, Telephony Signals; Text Conversation; Real-Time Pointers; DV Audio; MP3 Audio, Shared Multicast Virtual Worlds (SMVW) Also proprietary streams can be used

54. MULTIMEDIA SYSTEMS IREK DEFEE RTCP RTP control protocol (RTCP) Periodic transmission of control packets to all participants in session

55. MULTIMEDIA SYSTEMS IREK DEFEE Features of RTCP Feedback on the quality of data distribution –Evaluate whether problems are local or global –Transmission rate can be changed according to network situation Persistent transport-level identifier for an RTP source called the canonical name or CNAME –Keep track of participants –Associate multiple data streams from a given participant in a set of related RTP sessions

56. MULTIMEDIA SYSTEMS IREK DEFEE Features of RTCP (2) Observe the number of participants Scaling of control information transmission rate in order for RTP to suit also to a large number of participants –Maximum of 5 % of session bandwidth allocated to RTCP Transport minimal session control information –For example display participant identification in the user interface

57. MULTIMEDIA SYSTEMS IREK DEFEE RTCP Packet Format SR: Sender report, for transmission and reception statistics from participants that are active senders RR: Receiver report, for reception statistics from participants that are not active senders SDES: Source description items –describes participant using ASCII text BYE: Indicates end of participation APP: Application specific functions

58. MULTIMEDIA SYSTEMS IREK DEFEE Compound RTCP Packet Typically multiple RTCP packets are sent together as a compound RTCP packet In the following format: –Encryption prefix if packet is encrypted –SR or RR –SDES –Other RTCP packet types

59. MULTIMEDIA SYSTEMS IREK DEFEE Sender report (SR) First section of SR –Version, padding, reception report count, packet type, and SSRC –Similar to the beginning of RTP header

60. MULTIMEDIA SYSTEMS IREK DEFEE Second section of SR NTP timestamp: 64 bits –Real time when report was sent –Used in combination with timestamps returned in reception reports to measure round-trip propagation to those receivers RTP timestamp: 32 bits –Same time as the NTP timestamp but with the same random offset as the RTP timestamps in data packets –Used by media- independent receivers to estimate the nominal RTP clock frequency

61. MULTIMEDIA SYSTEMS IREK DEFEE Second section of SR (2) Sender's packet count: 32 bits –Total number of RTP data packets transmitted by the sender since starting transmission Sender's octet count: 32 bits –Total number of payload octets transmitted in RTP data packets by the sender since starting transmission –Used to estimate the average payload data rate

62. MULTIMEDIA SYSTEMS IREK DEFEE Third section of SR SSRC_n (source identifier): 32 bits –SSRC identifier of the source to which the information in this reception report block pertains Fraction lost: 8 bits –Fraction of RTP data packets from source SSRC_n lost since the previous SR or RR packet was sent Cumulative number of packets lost: 24 bits –Total number of RTP data packets from source SSRC_n that have been lost since the beginning of reception

63. MULTIMEDIA SYSTEMS IREK DEFEE Third section of SR (2) Extended highest sequence number received: 32 bits –Low 16 bits contain the highest sequence number received in an RTP data packet from source SSRC_n –Most significant 16 bits extend sequence number with the corresponding count of sequence number cycles Interarrival jitter: 32 bits –Estimate of the statistical variance of the RTP data packet interarrival time –J=J+(|D(i-1,i)|-J)/16

64. MULTIMEDIA SYSTEMS IREK DEFEE Third section of SR (3) Last SR timestamp (LSR): 32 bits –NTP timestamp received as part of the most recent RTCP sender report (SR) packet from source SSRC_n Delay since last SR (DLSR): 32 bits –Delay between receiving the last SR packet from source SSRC_n and sending this reception report block

65. MULTIMEDIA SYSTEMS IREK DEFEE Sender Report Profile may define extensions to SR Format of the receiver report (RR) packet is the same as that of the SR packet except sender information is omitted

66. MULTIMEDIA SYSTEMS IREK DEFEE Why RRs and SRs? Sender may modify its transmission rate based on the feedback Receivers can determine whether problems are local, regional or global Network managers may use profile-independent monitors that receive only RTCP packets to evaluate the performance of their networks for multicast distribution Cumulative counts are used to make measurements over both short and long time periods, and to provide resilience against the loss of a report

67. MULTIMEDIA SYSTEMS IREK DEFEE Why RRs and SRs? (2) Since timestamp is independent of the clock rate for the data encoding, it is possible to implement encoding- and profile-independent quality monitors A third-party monitor can calculate the average payload data rate and the average packet rate over an interval without receiving the data Jitter measure may indicate congestion before it leads to packet loss –necessary to analyze a number of reports from one receiver over time or from multiple receivers

68. MULTIMEDIA SYSTEMS IREK DEFEE SDES: Source description RTCP packet CNAME: Canonical end-point identifier SDES item –CNAME should be fixed for a given participant in order to provide a binding across multiple media tools NAME: User name, EMAIL,PHONE,LOC: Geographic user location, TOOL: Application or tool name NOTE: Notice/status SDES item –Intended for transient messages describing the current state, e.g., "on the phone, can't talk" PRIV: Private extensions SDES item

69. MULTIMEDIA SYSTEMS IREK DEFEE RTP Translators and Mixers Firewalls and low bandwidth connections Connects transport-level "clouds" –Each cloud is defined by a common network and transport protocol, multicast address or pair of unicast addresses, and transport level destination port May add or removes encryption May change encoding of data or underlying protocols

70. MULTIMEDIA SYSTEMS IREK DEFEE Translators Translator passes through the data streams from different sources separately Forwards RTP packets with their SSRC identifier intact Receivers cannot detect the presence of a translator Replicator from multicast to unicast Application-level filter in firewalls May, for example, filter non-CNAME SDES information if bandwidth is limited

71. MULTIMEDIA SYSTEMS IREK DEFEE Mixer Intermediate system that receives streams of RTP packets from sources, combines them and then forwards the combined stream Makes timing adjustments among the streams and generates its own timing for the combined stream Packets marked with the mixer's own SSRC identifier Generates its own reception reports for sources in each cloud and sends them out only to the same cloud

72. MULTIMEDIA SYSTEMS IREK DEFEE Pros and cons of mixers Output bandwidth is limited to that of one source even when multiple sources are active on the input side Receivers on the output side don't have any control over which sources are passed through or muted Receivers can't do inter-media synchronization of the original streams

73. MULTIMEDIA SYSTEMS IREK DEFEE Loop detection Translators and mixers may create transmission loops A loop of data packets to a multicast destination can cause severe network flooding. –All mixers and translators are required to implement a loop detection algorithm SSRCs are kept globally unique for each RTP session, they can be used to detect loops that may be introduced by mixers or translator

74. MULTIMEDIA SYSTEMS IREK DEFEE RTSP (The Real Time Streaming Protocol)

75. MULTIMEDIA SYSTEMS IREK DEFEE RTSP introduction application level protocol for control over the delivery of data with real-time properties. provides an extensible framework to enable controlled, on demand delivery of real-time data, such as audio and video. protocol provides means for choosing delivery channels such as UPD, multicast UDP and TCP

76. MULTIMEDIA SYSTEMS IREK DEFEE RTSP introduction provides means for choosing delivery mechanisms based on RTP does not depend on the mechanism used to carry continuous media. similar in syntax and operation to HTTP control may occur on TCP connection while the data flow via UDP.

77. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message ANNOUNCE DESCRIBE GET_PARAMETER OPTIONS PAUSE PLAY RECORD REDIRECT SETUP SET_PARAMETER TEARDOWN

78. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message (describe) DESCRIBE method retrieves the description or a presentation or media object identified request URL from the server. –DESCRIBE rtsp://server.example.com/foo RTSP/1.0 –Cseg: 312 –Accept: application/sdp, application/rtsl, application/mheg response –RTSP/1.0 200 OK –Cseg: 312 –Date: 23 Jan 1999 15:35:06 GMT –Content-Type: application/sdp –Content-Length: 376 –SDP part

79. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message (announce) two purposes: 1. When sent from client to server, ANNOUNCE posts the description of a presentation or media object by the request URL to a server. 2. When sent from server to client, ANNOUNCE updates the session description in real-time. The whole presentation description should be sent, rather than just the additional components.

80. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message (options) query the server about options it may or may not support OPTIONS * RTSP/1.0 Cseq: 1 Require: implicit-play Proxy-Require: gzipped-message Response could be RTSP/1.0 200 OK cseg: 1 Public: DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE or RTSP/1.0 551 Option not supported cseq:1 unsupported: implicit-play

81. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message (setup) specifies the transport mechanism to be used for the streamed media change transport parameter SETUP rtsp://example.com/foo RTSP/1.0 Cseg: 302 Session: 47112344 Transport: RTP/AVP;unicast;client_port=… If the change is not allowed response is RTSP/1.0 455 Method Not Valid In This State Cseq: 302

82. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message (play) tells the server to start sending data PLAY request may be queued PLAY rtsp://example.com/foo RTSP/1.0 Cseg: 832 Session: 47112344 Range: npt=10-15 PLAY rtsp://example.com/foo RTSP/1.0 Cseg: 833 Session: 47112344 Range: npt=20-25

83. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message The PAUSE request causes the stream delivery to be interrupted temporarily. The TEARDOWN request stops the stream delivery freeing the resource associated with it. The REDIRECT request informs the client that it must connect to another server location.

84. MULTIMEDIA SYSTEMS IREK DEFEE RTSP request message The GET_PARAMETER request retrieves the value of a parameter of presentation or stream. The SET_PARAMETER requests to set the value of a parameter for a presentation or stream. The RECORD request initiates recording a range of media data according to the presentation description.

85. MULTIMEDIA SYSTEMS IREK DEFEE RTSP example DESCRIBE 200 OK SETUP (video) 200 OK SETUP (audio) 200 OK PLAY 200 OK PAUSE 460 (error message) PAUSE 200 OK

86. MULTIMEDIA SYSTEMS IREK DEFEE HIGHER LEVEL PROTOCOLS  DEVELOPED BY MMUSIC GROUP OF IETF INCLUDE SDP AND SIP PROTOCOLS IN ADDITION TO RTP/RTCP/RTSP

87. MULTIMEDIA SYSTEMS IREK DEFEE Introduction. MMUSIC (Multiparty Multimedia Session Control) Internet standard track protocols to support Internet teleconferencing sessions.

88. MULTIMEDIA SYSTEMS IREK DEFEE SIP (Session Initiation Protocol) application-layer control protocol for creating, modifying and terminating sessions with one or more participants Internet multimedia conference, Internet telephone calls and multimedia distribution.

89. MULTIMEDIA SYSTEMS IREK DEFEE SDP (Session Description Protocol) describing multimedia sessions session announcement, session invitation, and other forms of multimedia session initiation.

90. MULTIMEDIA SYSTEMS IREK DEFEE SIP introduction supports name mapping and redirection services allows implementation of ISDN and Intelligent Network telephony subscriber service. Internet telephony gateways that connect Public Switched Telephone Network (PSTN) parties can also use SIP to set up calls.

91. MULTIMEDIA SYSTEMS IREK DEFEE SIP introduction cont. an application-layer control protocol for creating, modifying and terminating sessions with one or more participants. invite both persons and ”robots”, such as a media storage service. can be used to initiate sessions as well as invite members to sessions that have been advertised and established by other means.

92. MULTIMEDIA SYSTEMS IREK DEFEE SIP introduction does not offer conference control services such as floor control or voting doesn’t prescribe how a conference is to be managed can be used to introduce conference control protocol does not reserve resources can convey to the invited system the information necessary to do this.

93. MULTIMEDIA SYSTEMS IREK DEFEE SIP Addressing The SIP URL similar to a mailto or telnet URL i.e. user@host User part is a user name or a telephone number the host part is either a domain name or a numeric network address Sip:jorma.ollila@nokia.com

94. MULTIMEDIA SYSTEMS IREK DEFEE SIP entities SIP client is an application program that is able to send SIP methods (invite). SIP server is an application that accepts SIP methods and replies with status code. SIP proxy is an intermediary program that acts as both a server and client for the purpose of making requests on behalf of the other client

95. MULTIMEDIA SYSTEMS IREK DEFEE SIP entities SIP registrar is a server that accepts REGISTER requests. SIP redirect server is a server that accepts a SIP request, maps a address into zero or more new addresses and returns these addresses to the client. Does not initiate or accepts call by itself. Location server is used by SIP request or proxy server to obtain information about a callee’s possible locations

96. MULTIMEDIA SYSTEMS IREK DEFEE SIP methods INVITE (call user) ACK (confirm connection) CANCEL (cancel operation) BYE (disconnect) REGISTER (sign up with server) OPTIONS (capability query)

97. MULTIMEDIA SYSTEMS IREK DEFEE SIP invite The INVITE method indicates that the user or service is being invited to participate in a session. message body contains a description of the session type of media it is able to receive media it is willing to send parameters such as network destination.

98. MULTIMEDIA SYSTEMS IREK DEFEE SIP invite INVITE sip:watson@boston.bell-tel.com SIP/2.0 Via: SIP/2.0/UDP kton.bell-tel.com From: A. Bell To: T. Watson Call-ID: 3298420296@kton.bell-tel.com Cseq: 1 INVITE Subject: Mr. Watson, come here. Content-type: application/sdp Content-length: … (SDP) v=0 o=bell 53655765 2353687637 IN IP4 128.3.4.5 s=Mr. Watson, come here. c=IN IP4 kton.bell-tel.com m=audio 3456 RTP/AVP 0 3 4 5

99. MULTIMEDIA SYSTEMS IREK DEFEE SIP ack confirms that the client has received a final response to an INVITE request ACK is used only with INVITE request. ACK request may contain final session descriptions to be used by callee. If the ACK message body is empty, the callee uses the session description in the INVITE request.

100. MULTIMEDIA SYSTEMS IREK DEFEE SIP ack ACK sip:watson@boston.bell-tel.com SIP/2.0 Via: SIP/2.0/UDP kton.bell-tel.com From: A. Bell To: T. Watson Call-ID: 3298420296@kton.bell-tel.com Cseq: 1 ACK

101. MULTIMEDIA SYSTEMS IREK DEFEE SIP cancel, bye and options cancels a pending request with the same header field values, but doesn’t affect a completed request. release the call may be issued by either caller or callee. find out the capabilities of the potential callee without ”ringing” the designated address

102. MULTIMEDIA SYSTEMS IREK DEFEE SIP register register the address listed in the To header field with the SIP server. REGISTER sip:bell-tel.com SIP/2.0 Via: SIP/2.0/UDP saturn.bell-tel.com From: sip:watson@bell-tel.com To: sip:watson@.bell-tel.com> Call-ID: 70710@saturn.bell-tel.com Cseq: 1 Register Contact: Expires: 7200

103. MULTIMEDIA SYSTEMS IREK DEFEE SIP status messages (responses) 1xx:Informational - request received, continuing to process the request 2xx:Success - the action was successfully received, understood and accepted 3xx:Redirection - further action needs to be taken in order to complete the request. 4xx:Client error - request contains bad syntax or cannot be fulfilled at this server. 5xx:Server error - the server failed to fulfill an apparently valid request. 6xx:Global failure - the request cannot be fulfilled at any server.

104. MULTIMEDIA SYSTEMS IREK DEFEE SIP status messages (responses) 100 Trying 180 Ringing 181 Call is being forwarded 182 Queued 200 OK 300 Multiple choices 301 Moved permanently 302 Moved Temporarily 303 See other 304 Use proxy 380 Alternative Service 400 Bad request 401 Unauthorized 402 Payment required 403 Forbidden 404 Not Found... 480 Temporarily not available …. 500 Internal server error 501 Not implemented 502 Bad gateway 503 Service unavailable 504 Gateway time-out 505 SIP version not supported 600 Busy everywhere 603 Decline 604 Does not exist anywhere 605 Not acceptable

105. MULTIMEDIA SYSTEMS IREK DEFEE SIP status messages (responses) SIP/2.0 180 Ringing Via: SIP/2.0/UDP kton.bell-tel.com From: A. Bell To: Call-ID: 3298420296@kton.bell- tel.com Cseq: 1 INVITE Content-length:0 SIP/2.0 200 OK Via: SIP/2.0/UDP kton.bell- tel.com From: A. Bell To: Call-ID: 3298420296@kton.bell- tel.com Cseq: 1 INVITE Contact: sip:watson@boston.bell- tel.com Content-type: application/sdp Content-length: … v=0 o=bell 53655765 2353687637 IN IP4 128.3.4.5 s=Mr. Watson, come here. c=IN IP4 kton.bell-tel.com m=audio 3456 RTP/AVP 0 3 4 5

106. MULTIMEDIA SYSTEMS IREK DEFEE SIP operation in proxy mode pulver.com Proxy server nortel.com user@nortel.com 1. INVITE sip:jeff.pulver@pulver.com SIP/2.0 From: sip:user@nortel.com Location Server jeff.pulver pulver@von1 2. INVITE sip:pulver@von1 SIP/2.0 From: sip:user@nortel.com 3. SIP/2.0 200 ok From: sip:pulver@von1 pulver@von1 4. SIP/2.0 100 OK From: sip:jeff.pulver@pulver.com 5. ACK sip:jeff.pulver@pulver.com SIP/2.0 From: sip:user@nortel.com 6. ACK sip:pulver@von1 SIP/2.0 From: sip:user@nortel.com

107. MULTIMEDIA SYSTEMS IREK DEFEE SIP operation in redirect mode 1. INVITE sip:jeff.pulver@pulver.com From: sip:user@nortel.com 2. SIP/2.0 320 Moved temporarily Contact: sip:pulver@von1.pulver.com nortel.com user@nortel.com pulver.com Redirect Server Location Server Jeff.pulver pulver@von1 4. INVITE sip:pulver@von1.pulver.com From: user@nortel.com 3. ACK sip:jeff.pulver@pulverr.com From: sip:user@nortel.com 5. SIP/2.0 200 OK To: user@nortel.com 6. ACK sip:pulver@von1.pulver.com From: sip:user@nortel.com Pulver@von1

108. MULTIMEDIA SYSTEMS IREK DEFEE SIP and H.323 SIP 60+ pages Firewall - friendly SIP address = email address Personal mobility, IN services H.323 200+ pages complex, multiple protocols yet another address Not (yet)

109. MULTIMEDIA SYSTEMS IREK DEFEE SDP introduction describing multimedia sessions for the purposes of session announcement, session invitation, and other forms of multimedia session initiation advertise multimedia conference addresses and conference tool-specific information necessary to participation. does not incorporate a transport protocol, and is intended to use with other protocols like SIP and RTSP.

110. MULTIMEDIA SYSTEMS IREK DEFEE SDP includes: Session name and purpose Time(s) the session is active (start, stop, repeat time and so on The media comprising the session (type, transport protocol, format and so on) Information to receive those media (addresses, ports, formats and so on) Bandwidth information Contact information

111. MULTIMEDIA SYSTEMS IREK DEFEE SDP example –v=0 –o=mhadley 2890844526 2890842807 IN IP4 126.16.64.4 –s=SDP seminar –i=A seminar on the session description protocol –u=http://www.cs.ucl.ac.uk/staff/M.Hadley/dsp.03.ps –e=mjh@isi.edu (Mark Hadley) –p=+44 171 380 7777 –c=IN IP4 224.2.17.12/127 –b=CT:128 –t=2873397496 2873404696 –a=recvonly –m=audio 49170 RTP/AVP 0 –m=video 51372 RTP/AVP 31 –m=application 32461 udp wb –a=orient:portrait V,O,S,T & M tags are required

112. MULTIMEDIA SYSTEMS IREK DEFEE APPLICATION TO MULTIMEDIA – MPEG-4 EXAMPLE  MPEG-4 SYSTEM FITS AND REQUIRES SOMETHING LIKE THE RTP/RTCP/RTP AND SDP/SIP  IN MPEG-4 GENERIC NETWORKING SCHEME IS PROVIDED, ABOVE PROTOCOLS CAN BE ADAPTED TO IT  THIS IS ONGOING WORK

113. MULTIMEDIA SYSTEMS IREK DEFEE Streaming Media Formats

114. MULTIMEDIA SYSTEMS IREK DEFEE... Scene Description Stream Object Descriptor Stream Visual Stream Audio Stream Interactive Scene Description MPEG-4 Systems Principle STREAMS WHICH CAN BE TRANSPORTED OVER NETWORK

115. MULTIMEDIA SYSTEMS IREK DEFEE Example of MPEG-4 scene

116. MULTIMEDIA SYSTEMS IREK DEFEE Object-based compression and delivery

117. MULTIMEDIA SYSTEMS IREK DEFEE... Decoding MPEG-4 Interactive Scene Composition and Rendering Primitive AV Objects Scene Description Information... Elementary Streams FlexMux NetworkNetwork TransMux... Object Descriptor Display and Local User Interaction MPEG-4: An integrated Multimedia System

118. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 NETWORKING

119. MULTIMEDIA SYSTEMS IREK DEFEE Example of MPEG-4 over IP

120. MULTIMEDIA SYSTEMS IREK DEFEE Carriage of MPEG-4 contents over IP Networks MPEG-4 is a standard designated for the representation and delivery of multimedia information over a variety of transport protocols; MPEG-4 includes: –Interactive scene management; –Visual & Audio representations; –Functional systems (multiplexing, synchronization); –Object descriptor framework; Transfer method for MPEG-4 over IP: –In context with specific IP packets; –Multiplexed in MPEG-4 Flexmux (carried in IP packets); –Multiplexed in MPEG-2 TS (carried in IP packets);

121. MULTIMEDIA SYSTEMS IREK DEFEE HTTP Transport Protocol It is a very simple way to stream media files; The wire format is the same as the file-format storage; Arbitrary MPEG-4 Intermedia files cannot be streamed directly using HTTP, because of the “loose” in ordering the objects, and possible cross-references within the media file; Live streaming for Intermedia files can also be supported using proprietary methods;

122. MULTIMEDIA SYSTEMS IREK DEFEE Media Control Protocols In order to enable full streaming systems, a media control protocol needs to be defined to support the following features: 1.Seeking: –Forward; –Rewind; –Skip. 2.Bandwidth Scalability 3.Live Streaming

123. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 Systems In MPEG-4 Systems, the transport of streams is divided in 4 layers: –Compression Layer: includes elementary (raw) media streams (audio, video, etc.). –Synchronization Layer (SL): Adds a header to each unit of an elementary stream, which includes time stamps, reference to a clock elementary stream, and identification of key frames (RandomAccessPoint). This is similar to the task of RTP in IP networks. However, SL does not contain payload type (like RTP), and does not contain the Elementary Stream. In addition, an SL packet does not contain an indication of its length, so it must be framed by a lower-level protocol such as FlexMux or RTP. –FlexMux Layer: Groups elementary streams according to common attributes, such a QoS requirements. This is very simple multiplexing protocol, but also very low overhead. –TransxMux Layer: This is the actual transport protocol, such as RTP/UDP, MPEG- 2, etc. MPEG-4 does not define its own transport protocol, but assumes the application relies on an existing transport protocol. The FlexMux Layer is optional, but the Synchronization Layer is always present.

124. MULTIMEDIA SYSTEMS IREK DEFEE TCP/IP & SL-packet over UDP/IP TCP/IP: –Not suitable for real-time transfers (high delays and causes jitter, it was created for reliable transmission over timely delivery); –Does not support multicast; –Congestion control mechanism not suitable for AV media; SL-packet over UDP/IP: –SL provides: Timing & Sequence numbering; –UDP provides: Multiplexing, Length field, Checksum service; –SL+UDP may be used like a transport protocol for AV media;

125. MULTIMEDIA SYSTEMS IREK DEFEE Problems of SL/UDP/IP No other media stream can be synchronized with MPEG-4 data carried directly over UDP; The dynamic scene and session control concepts cannot be extended to non MPEG-4 data; No defined technique for synchronizing MPEG-4 streams from different servers; Streams from different servers may collide (their sources may become unresolvable at destination) in a multicast session; Mechanisms need to be defined to protect sensitive MPEG-4 data (RTP supports FEC); A feedback channel must be defined for quality control;

126. MULTIMEDIA SYSTEMS IREK DEFEE RTP & RTCP RTP = Real-time Transport Protocol; RTCP = Real-time Transport Control Protocol; A session consists of an RTP/RTCP pair of channels Usually works over UDP/IP (can work with other protocols also); RTP supports: –Multicasting; –Payload type identification; –Time stamping; –Sequence numbering; –Delivery monitoring; –From UDP (Multiplexing, Length field, Checksum service);

127. MULTIMEDIA SYSTEMS IREK DEFEE RTP RTP Problems: –It does not support the timely delivery of data or any other QoS guarantees; –It does not guarantee delivery, so packets may be delivered out of order or get lost (no mechanism to recover from packet loss); Time Stamp (TS): –Place incoming packet in correct timing order –Initial values are picked randomly and independently for each RTP stream; –Increase in time indicated by each packet; Sequence Number (SN): –Detect packet loss; –Increase by one for each packet; For video frame that is split into multiple RTP packets: they share the same TS but use different SN;

128. MULTIMEDIA SYSTEMS IREK DEFEE RTP Characteristics RTP can map only one source stream onto RTP session: –Multiplexing causes problems; –For scale coding, each layer must have its own RTP session and multicast group; Within each RTP session, source can change its data format over time; It allows FEC (Forward Error Correction); Synchronize across different media streams; Provide feedback on the quality of data using packet counts; Identify and keep track of participants;

129. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 over RTP/UDP/IP Easiest is to wrap the MPEG-4 SL packet in an RTP packet: –High overhead: two full headers; –RTCP may not provide enough control for the MPEG-4 stream; Several types of MPEG-4 payloads are being defined by the IETF for different ESs;

130. MULTIMEDIA SYSTEMS IREK DEFEE RTP ES & SL payload restrictions RTP packetization is not media aware: –No unique scheme can be defined, need a payload definition per payload type (not practical); –This may require the definition of new session and scene description mechanisms to deal with all the flows; Common restrictions: –RTP timestamp corresponds to composition time stamp (CTS) of MPEG-4 stream; –Packets should be sent in decoding order; –Streams can be synchronized using RTCP; Map SL stream onto RTP session: –SL header is optional; Reduced SL headers does not include: –Sequence number (mapped to RTP header); –Composition Time Stamp (mapped to RTP header);

131. MULTIMEDIA SYSTEMS IREK DEFEE Streams & Media Control in MPEG-4 Multiple Strems inMPEG-4: –Port consuming: Each AV contains one or more streams; Each stream needs one RTP session; –Potential solution: Selective bundling of ESs-FlexMux -> define a multiplexed MPEG-4 payload (may have to be defined for multiple payload types); Generic RTP multiplexing for use with MPEG-4, under development by IETF. MPEG-4 Media Control: –Remote interactivity: add or delete a stream, etc. –Media control channel allows renegotiating in time the available network and processing resources; –Must have an efficient signaling protocol that can handle such messages;

132. MULTIMEDIA SYSTEMS IREK DEFEE Media Control Framework To allow a client and one or more servers to exchange types of control messages and also allow for peer to peer exchange between two or more clients, the framework requires several components: –A description of the stored or live presentation; –A set of protocols that can provide proper services for the back channel message delivery; –A set of protocols that can allocate resources for the involved hosts and networks;

133. MULTIMEDIA SYSTEMS IREK DEFEE Components of media control (1) Presentation Description: –The client needs to refer to the description of a presentation that expresses the temporal and static properties of the presentation itself; –Must include information about the media, location of the media, etc. –Should provide multiple description instances of the same presentation so that the client can specify a given combination that fits its needs/capabilities – the client is the orchestrator of the presentation and the server is participating;

134. MULTIMEDIA SYSTEMS IREK DEFEE Components of media control (2) Client and Server State Representations: –Out of band signaling is used: the data streams and the control information are carried over separate channels using different protocols, each best suited to their needs and modes of operation; –As the media streams may be modified by the end user, the server needs to a state if the streams status for each client it is serving; –The client has to keep track of all the participating streams;

135. MULTIMEDIA SYSTEMS IREK DEFEE Components of media control (3) Basic Media Control Messages: –A multimedia system should have access to control messages ranging from remote VCR functions such as stop, play, fast forward and fast reverse, to messages generated in response to user actions to modify the presentation of a given object stream such as add or remove or alter, etc. –The basic control functionality relates to presentation and stream set-up: play, stop, pause, teardown and recording;

136. MULTIMEDIA SYSTEMS IREK DEFEE Real Time Streaming Protocol (RTSP) It is an application level protocol that provides an extensible framework to enable controlled delivery of real-time data, such as audio & video; It can be transported over UDP, TCP and is designated to work with RTP and HTTP; Provide support for bidirectional communications (frame level timing for remote video editing); RTSP does: –Control the transmission of media stream; –Use out-of-band signaling; RTSP does not: –Define compression schemes; –Define how AV is encapsulated; –Define how to buffer;

137. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 and RTSP From DMIF’s perspective, RTSP is an application alongside MPEG-4 systems; The RTSP client and server interact with the MPEG-4 systems; The RTSP client and server control the streams through the DAI by an RTSP- DMIF interface; The interface is kept very simple by limiting it to field mapping between the RTSP fields and the DAI primitive parameters; The RTSP client server interactions are used to control the MPEG-4 elementary streams;

138. MULTIMEDIA SYSTEMS IREK DEFEE Session Initiation Protocol (SIP) IETF Family of Session Protocols: –Session Description Protocol (SDP); –Session Announcement Protocol (SAP); –Session Initiation Protocol (SIP); SIP: –Two basic components: user agent & network server; –Independent of lower layer protocols; –Extensible to be application specific; –Gaining widespread use in IP telephony; MPEG-4 and SIP: –Unique ability to control different media types within a single session  Multiple stream transmission in one network session; –User Agent model fits in well with an MPEG-4 Client/Server model (point-to-point communication);

139. MULTIMEDIA SYSTEMS IREK DEFEE Toolboxes for transmitting MPEG-4 over internet RTP  transport of audio/video/… data, quality-of-service feedback; RTSP  very simple media control of streams; SIP  inviting people, media servers to sessions – telephony and streaming audio/video; HTTP, SDP  retrieve media descriptions;

140. MULTIMEDIA SYSTEMS IREK DEFEE Issues Encapsulation of MPEG-4 Sync layer packetized stream: –IEC/ISO 14496-8, framework still in revision; –Lots of issues still remain: time axis, buffer management, etc… Interactivity between application and End User: –Description of MPEG-4 content; –Initialization of an MPEG-4 session; SIP and IPC (Inter Process communication): –How to describe the dynamic process of channel (stream) setup and release? –What control information is necessary and how to transport it? Transport and IP QoS: –Must define a mapping mechanism among the different QoS mechanisms: transport QoS (not available yet), network QoS, etc. And more…e.g. DMIF

141. MULTIMEDIA SYSTEMS IREK DEFEE DMIF Delivery Multimedia Integration Framework DMIF is networking part of the MPEG-4 standard DMIF specifies the Delivery Layer of MPEG-4 (mostly in generic form)

142. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 Standard Structure DAI ESI Delivery Aware, media unaware : DMIF FlexMux Media Aware, delivery unaware : Audio and Visual Compression Scene Description and Object Descriptor Protocol Media and Delivery unaware : Systems Delivery Layer FlexMux Tool Sync. Layer Compression Layer Composition Related to networking

143. MULTIMEDIA SYSTEMS IREK DEFEE Why DMIF? many different delivery technologies, each with its own peculiarities different APIs for different environments (Local Files, Broadcast sources, Interactive servers through a variety of transports) no consolidated solution for real time multimedia streaming at certain QoS the introduction of QoS support in networks also impacts on charging policies

144. MULTIMEDIA SYSTEMS IREK DEFEE DMIF goals Separate the delivery aspects from the multimedia application issues Guarantee permanence of multimedia application in presence of new delivery technologies Allow the concurrent usage of different delivery technologies (e.g. broadcast + local retrieval) Allow resource logging to support flexible charging policies

145. MULTIMEDIA SYSTEMS IREK DEFEE DMIF DMIF addresses the following scenarios: –multicast/broadcast –local storage –remote interactive Broadcast source Local Storage Network

146. MULTIMEDIA SYSTEMS IREK DEFEE The DMIF reference architecture supports the streaming of multimedia content avoids embedding delivery dependency in a multimedia application allows the concurrent usage of multiple delivery technologies enables a plug-in approach to add new modules –to follow technical evolution in content streaming –to allow ad-hoc delivery implementations (e.g. for charging, admission control, Intelligent Networks …)

147. MULTIMEDIA SYSTEMS IREK DEFEE The DMIF reference architecture Sig map Target DMIF Target App DNIDAI Origin. App DAI DMIF Filter Origin. DMIF for Remote srv DNI Sig map NetworkOrigin. DMIF for Broadcast Target DMIF Target Application Broadcast source Origin. DMIF for Local Files Target DMIF Target Application Local Storage

148. MULTIMEDIA SYSTEMS IREK DEFEE MPEG-4 synchronisation, mux & transport (MPEG-4 own Sync Layer + Flexmux approach ) DAI ESI Synchronisation Layer FlexMux SL FlexMux SL HTML page JPG GIF Video Audio Java class OD BIU BIA DMIF DNI Delivery Layer Control Plane Applic. Signal. IP TCP HTTP UDP RTP FlexMux Channels ES Channels TransMux Channels RTCP QoS Manager

149. MULTIMEDIA SYSTEMS IREK DEFEE DNI MPEG-4 synch., mux & transport (IETF PROPOSAL, RTP CHANNEL MULTIPLEX ) IP TCP HTTP UDP RTP/RTP Mux Generic MP4 ES (& SL-PDU) to RTP mapping HTML page JPG GIF Video Audio Java class OD BIU BIA Delivery Layer DMIF Control Plane Synchronisation and Protection TransMux Channels ES Channels Applic. Signal. DAI ESI RTCP

150. MULTIMEDIA SYSTEMS IREK DEFEE Conclusions MPEG-4 is here in a limited fashion; We will see a marked growth in MPEG-4 products as chip sets become more readily available; Simple basic MPEG-4 service is not a problem; There is a lot of ongoing research on how to deliver MPEG-4 content over IP- based networks;