1. 1 CMSCD1011 Introduction to Computer Audio Lecture 10: Streaming audio for Internet transmission Dr David England School of Computing and Mathematical Sciences http://java.cms.livjm.ac.uk/homepage/staff/cmsdengl/ Teaching/cmscd1011.html Email: d.england@livjm.ac.uk

2. 2 In this session... How information is delivered via the Internet Characteristics of audio transmission over the Internet Streaming technologies –Real Networks’ RealAudio –Microsoft’s Advanced Streaming Format Streaming is a technology that lets you present multimedia data as it arrives at your computer

3. 3 How information is delivered via the Internet The Internet is built on a protocol called TCP/IP –Transmission Control Protocol / Internet Protocol Information is sent in the form of small packets –Each packet has a sending and receiving address (IP) There are two forms of transport mechanism for packets: TCP and UDP –TCP guarantees that each packet will arrive and that it will arrive in the right order –UDP does not guarantee that each packet will arrive or get there in the correct order –TCP is slower but more reliable, UDP is much faster

4. 4 How information is delivered via the Internet Suppose we want to send some information over the Internet to a friend on the Greek island of Crete. The information may hop across many countries en route (shown by the various paths on the diagram.) There are no guarantees that the information will arrive in the order that we sent it due to the numerous routes that it can take. TCP rearranges out of sequence information. UDP does not.

5. 5 Protocols for real-time information Because of the problems associated with TCP and UDP when dealing with multimedia data, other protocols have been developed which sit alongside or on top of the existing system: –RTP (Real time Transport Protocol) Usually sites on top of UDP adding extra features for synchronisation, ensuring the correct sequence and identifying different data types (e.g. audio or video) –RTSP (Real Time Streaming Protocol) Often described as an “Internet VCR remote control”, it is used to co-ordinate how the actual media data transmission

6. 6 Unicasting and Multicasting

7. 7

8. 8 Characteristics of audio transmission Keywords: Delay, Jitter and Quality of Service Audio is one medium where we can tolerate a small amount of data loss –If we miss a word in a sentence, we can usually infer what the speaker means –If a note is missing in some music, we don’t mind This describes our minimum Quality of Service (QoS) There will inevitably be delay in the transmission of audio data via networks and the variation in delay is known as jitter

9. 9 The effect of delay and jitter on synchronisation Sender Video Audio Time Receiver Video Audio Time Packets containing compressed data If the jitter is pronounced, audio may play back at the wrong time (earlier or later than expected). If the audio has to be synchronised to video, you will lose the synchronisation. This usually has to be corrected at the receiving end.

10. 10 Downloading audio data Bytes to download Time Playback cannot begin until this point Assume that you wish to download a 10 MB digital audio file in wav format. The time it will take to download the data is a function of the size of the data and the speed of the network connection.

11. 11 Streaming audio data Bytes to download Time Start playing Download Playback Buffer The client software must download a certain amount of data first. This is to compensate for the differences in required data rate and actual network transmission speed. The size of the buffer will increase if the required data rate is higher than the actual network speed. If they are equal the buffer is zero. Note that the whole data is not kept on the disk

12. 12 Points to note about streaming There needs to be a data buffer –Unless the network speed is greater than or equal to the required data rate (which is unlikely as more users request data at the same time) Playback commences when the buffer is initially full During playback, the client software continues to download the remaining data in the background –Filling the buffer (a FIFO data structure) The size of the buffer may need to be changed dynamically during playback if the client detects that the network speed has changed The entire audio is never stored on the local disk, only a small amount (equal to the size of the buffer) is ever stored

13. 13 Streaming data: The process Encoder Data files The streaming media encoder converts the existing data into a format suitable for streaming. Server UNIX/Linux Windows A fast server is required to serve these files by sending small chunks of information over the network. Fast network connection and hard drive are important factors. Player Network TCP/IP UDP HTTP RTP RTSP Various network protocols may be employed to deliver the actual data.

14. 14 Real Networks’ RealAudio RealAudio was the first major player in the field of streaming audio (and video and now images) The latest version uses a technology called SureStream™ to encode different quality settings in the same data –The higher your bandwidth, the better the quality RealPlayer is one of the most widely used multimedia applications on the Internet today http://www.realnetworks.com/getstarted/index.html http://www.realnetworks.com/products/servers/bguide/index.html http://www.realnetworks.com/getstarted/index.html http://www.realnetworks.com/products/servers/bguide/index.html

15. 15 The effects of bandwidth on data transmission Server Client 28.8 Kbps modem 56 Kbps ISDN 56 Kbps of data over a 28.8 Kbps modem Playback interrupted

16. 16 Microsoft’s Advanced Streaming Format (ASF) ASF is part of the Windows Media protocols The data is broken up into small packets with each numbered packet containing part of the media data The media server knows which packets to send to the user at any given time –If, for example, network congestion requires the server to send less data, it will not send packets that the user won't receive –Also, if the user starts skipping around in the file, by fast forwarding, rewinding, or skipping to markers within the file, the server sends the user the data that corresponds to what the user has requested http://www.microsoft.com/windows/windowsmedia/ http://msdn.microsoft.com/workshop/imedia/windowsmedia/tourwmtools.asp http://www.microsoft.com/windows/windowsmedia/ http://msdn.microsoft.com/workshop/imedia/windowsmedia/tourwmtools.asp

17. 17 ASF tools Windows Media Author

18. 18Summary Because in its current form, the Internet is so slow, we need different methods to deliver multimedia data –This is changing now with the introduction of broadband communications, e.g. ADSL, Cable modems Streaming is one technique for delivering multimedia data in real time over the Internet Streaming can be used for live broadcasts or for transmitting pre-recorded information Streaming does not store the whole presentation on disk (which is good if you sell media!)