1. xStreamer Modular Multimedia Streaming Nicolas Staelens nicolas.staelens@intec.ugent.be www.ibcn.intec.ugent.be INTEC Broadband Communication Networks (IBCN) Department of Information Technology (INTEC) Ghent University - IBBT

2. xStreamer  In-house developed modular multimedia streamer  Alexis Rombaut (alexis.rombaut@intec.ugent.be)alexis.rombaut@intec.ugent.be  Written in C++  Uses libraries:  libavformat/libavcodec (parsing/encoding/decoding)  live555 (RTSP)  jrtplib (RTP)  Released under General Public License (GPL)  Freely available at http://xstreamer.atlantis.ugent.be/http://xstreamer.atlantis.ugent.be/ 2

3. 3 Modular Multimedia Streaming Inspired by Click Modular Router & DirectShow Offers different components  Performs basic functions  Readers, packetizers, multiplexers, schedulers, transmitters, receivers, writers, classifiers, analyzers Streamer is directed graph of components

4. 4 Modular Multimedia Streaming Supports audio and video Using RTP packetization:  MPEG-1/2/4 Video & Audio Using MPEG-2 Transport Streams:  MPEG-1/2/4 Video & Audio  H.264 AVC/SVC

5. 5 Modular Multimedia Streaming Multitude of supported protocols RTP/UDP RTSP/RTP/UDP UDP TCP

6. What can xStreamer do? Advanced streaming server  Own MPEG-2 TS multiplexer  SVC streaming  Differentiated streaming using classifiers Proxy/client  Proxy: convert differentiated stream into a single stream  Client: save captured stream to file Video tool  No ‘real’ streaming involved  Simulate packet loss  Collect tracefiles during streaming 6

7. Create xStreamer configuration Configuration saved in XML-based file  Describes directed graph of components and connections between components Graphical User Interface  Visualize directed graph  Drag components and draw connections  Configure components 7

8. Example: Differentiated SVC streaming 8 Read raw H.264 video stream Packetize frames into packets as defined in RFC3984 Avoid bursts by smoothing packets over time Classify NAL units depending on SVC layer Stream different layers over different connections

9. Example: Proxy/Client 9 ProxyClient

10. xStreamer as video tool Offline simulator  No ‘real’ streaming  Simulate packet loss using Classifier component  Random, Gilbert-Elliott  Write resulting packet stream back to file Tracefile generation  Packetizer : video trace  Transmitter : sender trace  Receiver : receiver trace  Classifier : sender & receiver trace 10

11. xStreamer – HDTV PVS creation HDTV PVS Creation  H.264/AVC  Packet loss Toolchain 1.Encoding 2.Streaming 3.Decoding & syncing 11

12. HDTV PVS Creation – xStreamer config 12 Packet loss simulator using Gilbert-Elliott model

13. HDTV PVS Creation – Tracefiles 13

14. HDTV PVS Creation – Tracefiles 14 Video tracefile generation 0H53911.066667 0H2411.066667 0H411.066667 0I2568351801.066667 1P111970791.133333 2B52428371.100000 3P43918311.200000 4B30967221.166667 5P62890451.266667 6B20660151.233333 7P69342491.300000 8P65287461.333333 9P78591561.400000 10B31715231.366667 11P78028551.466667 12B34809251.433333 13P71582511.533333 …

15. HDTV PVS Creation – Tracefiles 15 Sender tracefile 0.00id 1udp 24 0.00id 2udp 4 0.00id 3udp 1430 0.00id 4udp 1430 0.00id 5udp 1430 0.00id 6udp 1430 0.00id 7udp 1430 0.00id 8udp 1430 0.00id 9udp 1430 0.00id 10udp 1430 0.00id 11udp 1430 0.00id 12udp 1430 0.00id 13udp 1430 0.00id 14udp 1430 0.00id 15udp 1430 0.00id 16udp 1430 0.00id 17udp 1430 …

16. HDTV PVS Creation – Tracefiles 16 Receiver tracefile 0.00id 0udp 539 0.00id 1udp 24 0.00id 2udp 4 0.00id 3udp 1430 0.00id 4udp 1430 0.00id 5udp 1430 0.00id 6udp 1430 0.00id 7udp 1430 0.00id 8udp 1430 0.00id 9udp 1430 0.00id 10udp 1430 0.00id 11udp 1430 0.00id 12udp 1430 0.00id 15udp 1430 0.00id 16udp 1430 0.00id 17udp 1430 0.00id 18udp 1430 … 2 packets lost: id 13 & 14

17. Combining tracefiles 17 0HCORRECT video tracesender tracereceiver trace 0HCORRECT 0IDAMAGED …

18. PVS Creation: decoding & syncing Decode streamed raw H.264/AVC file  avc-decoder based on libavcodec (ffmpeg)  Only decodes correctly received frames Align decoded and original sequence  Decoded sequence misses pictures  Lost & damaged  Tracefile analysis  Locate missing pictures  Conceal missing pictures  Copy previous frame, gray, black, green, … 18

19. Conclusion Modular architecture  Logical structure  Easily extendable Advanced streaming server  SVC Support  Differentiated streaming Video tool  Network simulation  Tracefile generation 19

20. Questions ?