1. Πολυμεσικό Υλικό στο Internet: Συγχρονισμός, Επεξεργασία και Διακίνηση Συγχρονισμός Πολυμεσικών Εφαρμογών & Σχετικά Εργαλεία 13/11/2014 Β. Μάγκλαρης Μ. Γραμματικού mary@netmode.ntua.gr Δ. Καλογεράς www.netmode.ntua.gr

2. Outline Real-Time Protocols (RTP, RTCP, RTSP) ITU – H.323 Session Initiation Protocol (SIP) Synchronized Multimedia Integration Language (SMIL) A Rough History of Web Standards – HTML5 – PopcornJS - HTML5 Media Framework – Timesheets.js

3. Real-Time Protocol (RTP) Real-Time Control Protocol (RTCP) http://en.wikipedia.org/wiki/Real-time_Transport_Protocol http://tools.ietf.org/html/rfc1889 https://tools.ietf.org/html/rfc3611 http://en.wikipedia.org/wiki/RTP_Control_Protocol http://en.wikipedia.org/wiki/Real-time_Transport_Protocol http://tools.ietf.org/html/rfc1889 https://tools.ietf.org/html/rfc3611 http://en.wikipedia.org/wiki/RTP_Control_Protocol RTP: first published in 1996 by IETF RTP: Defines a standardized packet format for delivering audio and video over IP Networks Real-Time Protocol (RTP) – a framing protocol for real-time applications – there is no QoS in real time based on UDP protocol RTCP: RTCP protocol monitors transmission statistics and Quality of Service (QoS) and aids synchronization of multiple streams

4. Real Time Streaming Protocol (RTSP) http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol http://www.ietf.org/rfc/rfc2326.txt http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol http://www.ietf.org/rfc/rfc2326.txt RTSP was developed by RealNetworks, Netscape and Columbia University, with the first draft submitted to IETF in 1996 RTSP: – an application-level protocol for control over the delivery of data with real- time properties – controls streaming media servers – is used for establishing and controlling media sessions between end points – RTSP servers use the Real-time Transport Protocol (RTP) in conjunction with Real-time Control Protocol (RTCP) for media stream delivery

5. ITU – H.323 http://www.h323forum.org, http://en.wikipedia.org/wiki/H.323 www.rfc-editor.org/info/rfc4123 http://www.h323forum.org http://en.wikipedia.org/wiki/H.323 www.rfc-editor.org/info/rfc4123 First version of H.323 was published by the ITU in November 1996 Defines the protocols to provide audio-visual communication sessions in any IP (Internet) infrastructure – Point-to-point – Multi-point (Multi-Conference Unit - MCU) Addresses call signalling and control, multimedia transport and control, and bandwidth control for point-to-point and multi-point conferences H.323 call signalling – is based on the ITU-T Recommendation Q.931 protocol – is suited for transmitting calls across networks using a mixture of IP, PSTN, ISDN, and QSIG over ISDN A Gatekeeper is an optional component in the H.323 network that provides a number of services to terminals, gateways, and MCU devices.

6. Session Initiation Protocol (SIP) http://en.wikipedia.org/wiki/Session_Initiation_Protocol https://www.ietf.org/rfc/rfc3261.txt http://en.wikipedia.org/wiki/Session_Initiation_Protocol https://www.ietf.org/rfc/rfc3261.txt SIP: a signalling communication protocol, widely used for controlling multimedia communication sessions such as voice and video calls over Internet Protocol (IP) networks SIP was originally designed by Henning Schulzrinne and Mark Handley in 1996 Text Based (like HTTP) SIP address like : user @ host Same services with Η.323 but : – H.323 MCU as a special purpose component – SIP: server-based multi conference SDP (Session Description Protocol) for media (audio, video) description

7. Synchronized Multimedia Integration Language (SMIL) http://en.wikipedia.org/wiki/Synchronized_Multimedia_Integration_Language http://www.ietf.org/rfc/rfc4536.txt http://en.wikipedia.org/wiki/Synchronized_Multimedia_Integration_Language http://www.ietf.org/rfc/rfc4536.txt SMIL version 1 : introduced 1999 SMIL: a World Wide Web Consortium (W3C) recommended Extensible Markup Language (XML) to describe multimedia presentations Defines markup for : – timing – layout – animations – visual transitions – media embedding Presenting media items (text, images, video, audio, links) to other SMIL presentations, and files from multiple web servers

8. A Rough History of Web Standards (ref) 2004WHATWG started 2008W3C Working Draft 2012 (2010)W3C Candidate Rec 2022W3C Rec 1996 – CSS 1W3C Rec 1998 – CSS 2W3C Rec 1999 – CSS 3Proposed 2001 – CSS 3W3C Working Draft 2005 – CSS 2.1W3C Candidate Rec 91-9293-9495-9697-9899-0001-0203-0405-0607-0809-1011-1213-14 HTML 1HTML 2HTML 4XHTML 1 HTML 5 CSS 1CSS 2T-less DWeb 2.0CSS3 JSECMA, DOM DOM 2AjaxDOM, APIs HTML CSS

9. HTML5: more than HTML HTML5 Spec CSS3 (Cascading Style Sheet) Spec JavaScript to tie it all together

10. What’s new to HTML5 Simplified and Loose Syntax New Elements and Attributes Embedded Media (audio, video) Canvas Offline Storage Drag and Drop Geo-Location

11. Browser Support http://www.w3.org/TR/html5/ HTML5 is not yet an official standard - browsers have not full HTML5 support But all major browsers (Safari, Chrome, Firefox, Opera, Internet Explorer) continue to add new HTML5 features to their latest versions

12. HTML5 Video http://en.wikipedia.org/wiki/HTML5_video http://www.w3.org/TR/html5/embedded-content-0.html#the-video-element HTML5 specification introduced the element, partially replacing the element. HTML5 video is intended to become the new standard way to show video on the web without plug-ins The element was proposed by Opera Software in February 2007

13. HTML5 Video Benefits End users have full control over the multimedia. – HTML5 video and audio are keyboard-enabled by default, which is a great accessibility benefit. End users do not need to install a plug-in to play them. – Browser already has everything it needs to play movies/sound. Video and audio content on the page can be manipulated. – Two new elements that can be styled, moved, manipulated, stacked and rotated. You can build your own controls using HTML, CSS and JavaScript. – No new skills or development environment needed. Simple interaction with the rest of the page. – Multimedia API gives you full control over the video, and you can make the video react both to changes in the video itself and to the page around it.

14. Using HTML5 Media Elements https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Using_HTML5_audio_and_video https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats Your browser does not support the audio element. I'm sorry; your browser doesn't support HTML5 video in WebM with VP8 or MP4 with H.264.

15. MP4 or H.264: codec supported by Apple and Microsoft. Provides good quality video and small file sizes ogg/Theora: free open standard for video encoding. It generates high quality videos with comparatively larger file sizes. Hard to find tools to convert to ogg/Theora. WebM: uses the VP8 codec, owned by Google. High video quality, longer to encode than in H.264. Very difficult to encode, especially compared to the many free & commercial tools available for H.264. HTML5 Video - Codecs that Browsers Support http://en.wikipedia.org/wiki/HTML5_video

16. As of April 2010, in the wake of Apple iPad launch, a number of high-profile sites have started to serve H.264 HTML5 video instead of Flash for user-agents identifying as iPad On November 8, 2011, Adobe announced that it was ceasing development of the Flash Player "plug-in" for browsers on mobile devices, in an effort to shift its focus on using the ActionScript programming language and Adobe AIR to develop fully functional native apps for mobile Adobe – Apple Controversy http://en.wikipedia.org/wiki/Apple_and_Adobe_Flash_controversy https://www.apple.com/hotnews/thoughts-on-flash/

17. Popcorn.js is an HTML5 media framework written in JavaScript for filmmakers, web developers, and anyone who wants to create time-based interactive media on the web: – For media makers: Popcorn.js allows video, audio and other media to control elements of a webpage. Combining our library with a mixture of standard HTML + JavaScript, authors can let their media be the “conductor” of interactive and immersive experiences. – For developers: Popcorn.js is an event framework for HTML5 media. Popcorn.js utilizes the native HTMLMediaElement properties, methods and events, normalizes them into an easy to learn API, and provides a plug-in system for community contributed interactions. PopcornJS - HTML5 Media Framework http://popcornjs.org/ https://popcorn.webmaker.org/

18. A Javascript library to rely on declarative W3C standards (namely, SMIL Timing and SMIL Timesheets) to synchronize HTML content Focusing on two developments: – a Timesheet Scheduler, i.e. a JavaScript implementation of SMIL Timing and SMIL Timesheets – a Timesheet Editor, i.e. a wysiwyg, Mozilla-based, multimedia page authoring tool Timesheets.js http://wam.inrialpes.fr/timesheets/

19. https://dev.opera.com/articles/introduction-html5-video/ https://dev.opera.com/blog/everything-you-need-to-know-about- html5-video-and-audio-2/ https://dev.opera.com/blog/everything-you-need-to-know-about- html5-video-and-audio-2/ https://developer.mozilla.org/en- US/docs/Web/Guide/HTML/Using_HTML5_audio_and_video https://developer.mozilla.org/en- US/docs/Web/Guide/HTML/Using_HTML5_audio_and_video http://html5video.org/wiki/HTML5_Player_Comparison http://www.w3.org/2010/05/video/mediaevents.html http://www.html5rocks.com/en/tutorials/video/basics/ https://popcorn.webmaker.org/ http://blog.teamtreehouse.com/building-custom-controls-for- html5-videos http://blog.teamtreehouse.com/building-custom-controls-for- html5-videos References