CMAF COMMON MEDIA APPLICATION FORMAT FOR INTERNET DELIVERY KILROY HUGHES, MICROSOFT ALBERT KOVAL, DECE

MPEG COMMON MEDIA APPLICATION FORMAT (CMAF) Apple, Microsoft, and other companies started standardizing the Common Media Application Format (CMAF) in MPEG February 2016 MPEG calls this type of specification a Media Application Framework (MAF), and the purpose is primarily to combine and apply MPEG standards The initial spec was proposed by several companies and written over the last 2 years by Apple and Microsoft, based upon work done by DECE, DASH, MPEG Common Encryption, ISOBMFF, MPEG-4 Part 30, etc.

OPPORTUNITY FOR CONVERGENCE Microsoft and Apple recognized the growing opportunity for industry convergence on a single media format for internet video, driven by the growth of adaptive streaming An opportunity to unify and codify industry practice that is already converging around adaptive streaming Industry consortia could normatively reference a single ISO MPEG media format standard, rather than maintain several slightly different specifications for adaptive streaming A single format would be more efficient for content providers, streaming services, player implementers, browsers, apps, devices, testing, etc. A single format is more efficient for CDN caching, multicast delivery, network efficiency, and quality of user experience

ENCODE ONCE, ADAPTIVELY STREAM EVERYWHERE All screens All operating systems All networks All screens All operating systems All networks

COMMON MEDIA APPLICATION FORMAT PRIORITIES Combine broadly adopted MPEG technologies Use proven industry profiles and practices Support Adaptive Streaming (live and on demand) Enable DRM interoperability Enable efficient and interoperable encoding, caching, and delivery Make presentation composition and decoding simple Design for extensibility to make it future-proof (UHD/HDR, VR, etc.)

CMAF APPLICATION MODEL AND SPEC SCOPE The normative scope of the CMAF spec is the media format, its encoding and decoding process, and its composition into synchronized and adaptive multimedia presentations. Manifest format and delivery protocol are not specified, but assumed in the hypothetical application model. The intent is to specify a media format that is compatible with DASH and HLS manifests, Common Encryption, adaptive delivery, file download, unicast, multicast, broadcast, and other delivery methods.

CMAF APPLICATION MODEL SCOPE (DARK BOXES NOT SPECIFIED BY CMAF) Sample Encoding and Encryption in CMAF Media Profiles and Track Sets in a CMAF Presentation Profile Packaging CMAF Fragments, Tracks, and Resources, Resource Identification Manifest generation and Resource Reference Manifest Delivery and Resource location, sync and delivery of CMAF Fragments CMAF Fragment Splicing, Parsing and Sync Sample Decryption Decoding and Adaptive Scaling CMAF Media Profiles, CMAF Presentation Profile CMAF Container and Track brand CMAF Media Profiles, CMAF Presentation Profile

REFERENCED MPEG TECHNOLOGIES Widely adopted MPEG technologies for internet video expected to be referenced by the Common Media Application Format include: ISO Base Media File Format (MPEG-4 Part 12) MPEG codecs for audio and video MPEG-4 file and track specifications for the delivery of audio, video, and subtitles/captions derived from the Base format (MPEG-4 Part 15, Part 30, etc.) Common Encryption (ISO/IEC )

“CMFHD” CMAF PRESENTATION PROFILE ISOBMFF segments (CSF)Common Encryption 2ch AAC AudioHD AVC Video Subtitles (ttml/vtt)

CMAF INTEROP AND CONFORMANCE CODE POINTS CMAF Presentation Profile Defines the minimum required Media Profiles and optional encryption schemes required in a Presentation of this Profile CMAF Container and Track Compatibility Brand File type brand for every CMAF Header indicating container and Track structural conformance CMAF Media Profile Compatibility Brand File type brand indicating conformance to the indicated Media Profile sample, CMAF Fragment, Track, Track Set, and encryption mapping

CMAF PROFILES AND BRANDS CMAF Profiles and brands are structured so non-MPEG codecs can specify and register CMAF compatible Media Profiles and Track bindings for use in CMAF Presentations. Also, external orgs and specs can define Presentation Profiles with different requirements for which Tracks, encryption, etc. are required or optional. It’s expected that ATSC, DVB, CTA WAVE, etc. will define Presentation Profiles for their ecosystem. The CMAF defined Presentation Profile at present is “CMFHD”, which requires: an AVC HD profile (believed to be equal to DECE HD Profile) AAC Stereo audio both VTT and IMSC1 subtitles

MEDIA OBJECT MODEL, FOR EFFICIENCY AND FLEXIBILITY Presentation (manifest) Audio Selection Set Audio Switching Set English 1+ Tracks containing Segments Audio Switching Set Spanish 1+ Tracks containing Segments Video Selection Set HD AVC Switching Set ABR Tracks containing Segments UHD HEVC Switching Set ABR Tracks containing Segments Selection Set Switching Set Alternative Content Tracks Seamless Track Switching Segments Independent movie fragments audio, video, or subtitle

HYBRID DELIVERY, “LATE BINDING” OF SEGMENTS

COMPARISON TO DECE COMMON FORMAT - 1 A brand and profile system for: a) the CMAF container, b) each CMAF Track and its CMAF Media Profile, c) Presentation Profiles in Manifests that define required and optional Tracks and their Media Profiles CMFHD Profile equivalent to CFF HD track requirements in a file.

COMPARISON TO DECE COMMON FORMAT - 2 CMAF Fragments encoded and decoded regardless of delivery method (CMAF Fragments are ISOBMFF movie fragments with complete CVSs, like CFF/CSF) CMAF Fragments are contained in Resources for delivery, which include Chunks, Segments, and Track Files. a) Track Files are what CSF specified. b) Segments can be one or more CMAF Fragments, live or VOD. c) Chunks can deliver a CMAF Fragment as short ISOBMFF segments containing a few samples for low latency delivery during live streaming.

ENCRYPTION There are two CENC 3 rd Edition common encryption schemes that are both optional in CMAF CMFHD Presentations: ‘cenc’ - same as CFF, and ‘cbcs’ which is compatible with FairPlay

POTENTIAL CF DIFFERENCES - SUMMARY Aside from the foregoing, CMAF encoding is believe to fit within the DECE CFF spec’s CSF “delivery target”. CMAF may specify more constraints between tracks in a Switching Set in order to determine how browsers and decoders using the MSE interface can do seamless switching -- CSF is pretty good in that regard as well. CMAF defines optional file extensions and MIME parameters, and CSF does also; may not be conflicting requirements, only different options. Details like the above are likely to change before a CMAF Committee Draft is issued for MPEG national body voting.

MPEG SCHEDULE CMAF is expected to be technically frozen later this year and published around the middle of next year, but that depends on MPEG process Dependency: MPEG requires conformance tests and vectors for all standards DECE could contribute to this through our Liaison with MPEG

QUESTIONS? Contacts for further information: Microsoft: Iraj Sodagar CMAF AHG Reflector CMAF Ad Hoc Group open to all participants, MPEG meetings for MPEG members only

BACKUP SLIDES

REQUIREMENTS The Common Media Application Format (CMAF) shall be based on existing MPEG standards and industry practices; and shall support the following requirements: Shall define specific profiles based on the ISO Base Media File Format (ISOBMFF), each defining a conformance point that provides interoperability between CMAF conformant devices (CMAF players) and CMAF presentations that support that profile. The highest priority is to specify the minimum requirements both CMAF presentations and CMAF players must support for interoperability, and to specify an easily adopted baseline profile. Shall be capable of carrying encoded audio, video, and subtitles, including signaling to support track selection and common accessibility use cases.

CMAF PROFILE CMAF profile shall define: A video codec, its profile and level, associated color space, EOTF and other rendering constraints; An audio codec, its profile, features, and channel configuration; Optional closed caption and subtitle formats; An optional encryption scheme and key management constraints for Common Encryption (CENC) of media samples; Track and segment encryption metadata that enables the use of any DRM systems that conform to MPEG Common Encryption for key management and decryption; A set of constraints for ISOBMFF files, tracks, movie fragments, samples, and elementary streams bindings; CMAF profile identifier(s) for the conformance points supported by a presentation or device.

DELIVERY MECHANISM Shall be delivery mechanism agnostic, but support at least the following delivery mechanisms: Adaptive streaming with seamless adaptive switching of CMAF tracks encoded with different bitrates, frame rates, and video resolutions. Late binding of independently created and/or delivered tracks for combined playback.

BINDING AND PLAYBACK Early binding of independently created and/or delivered tracks for combined delivery, i.e. playable delivery segments that contain media segments of differing media types but the same approximate time-range Multicast and broadcast delivery Hybrid network delivery (broadcast + unicast) Physical media delivery (memory sticks, discs, etc.) Low latency live streaming Multitrack file progressive download and playback, Efficient CDN delivery with uniform identification of each media segment by URI (and possible byte range) for efficient caching and reuse. Shall support random access of the content Shall support signaling of various random access points and their types Shall support delivery and playback of selected tracks of the content.

USE CASES Adaptive bitrate streaming of Media Segments using HTTP(S) and any Presentation Description, such as DASH MPD, Smooth Streaming Manifest, Apple HTTP Live Streaming Manifest (m3u8), etc. Note: Specific CMAF bindings to MPEG DASH and other presentation descriptions are expected to be defined separately. Broadcast/multicast streaming of Media Segments over one-way networks such as terrestrial broadcast, satellite, or cellular network. Hybrid network streaming of live content via broadcast/multicast or cellular, and unicast for time shifted or individualized content, and Segments lost during broadcast delivery. Download of streaming files for local playback, or local playback of downloaded content combined with streamed Segments. Server-side and Client-side ad insertion with media segment and manifest-level signaling of messages for ad insertion and other purposes, such as SCTE-35, VAST, VMAP, etc.