1. An Overview of MXF and the Search for the Video File Wrapper.
NDIIPP
Preserving Digital Public Television
JTS 2007
Toronto

2. Universal Preservation Format
Self describing storage
Must know the coding of the file.
Must know the coding of the essence within the file.
At JTS 2000

3. Universal Preservation Format
Wrapper contains both essence and metadata
Must know the location of the essence and metadata within the file.

4. Open Source Codecs Report on File Formats and Packages VLAN FFMPEG
VLAN
FFMPEG
VLAN supports H.264

5. Source Code Links http://www.free-codecs.com/
Dirac licensed under Mozilla Public License.
Dirac now just uses raw planar YUV.
MJPEG 2000 covered elsewhere.

6. MXF Background & MXF - AS/PBS
Thomas Edwards
PBS

7. MXF – What is It? Material eXchange Format
Influenced by the “EBU/SMPTE Joint Task Force for Harmonised Standards for the Exchange of Television Programmes as bit streams”
A “wrapper format” that ties together essence and metadata elements into a coherent whole

8. What can you do with MXF? Store simple completed works with metadata
Store files in a streamable format
Wrap up a playlist/EDL and essence with synchronization information
Wrap a wide variety of compression formats

9. KLV Coding (SMPTE 336M) K= Key: a unique identifier
L= Length: how long is the field
Encoded with ASN.1 BER, so can be as big as needed
< 0x80 is actual length (0-127 decimal)
0x8Y implies next Y bytes contain actual length
Example: 0x , length is 0x1000 = 4096 decimal
V= Value: what is the value of the field

10. UL – SMPTE Universal Label
KLV Using SMPTE MDD
06.0E.2B 14
47 6F 6E …
SMPTE Designator
SMPTE Item
Length
Value
“Version Title”
“Colonial House”
(in ISO-7 bit code)
UL – SMPTE Universal Label
(16 bytes)

11. Simple Versus Complex MXF files can be very simple
“Just wrap up my essence”
E-VTRs, Video Servers
MXF files can be very complex
EDL’s, alternate packages
Self-referential clips
Software implementations

12. Simple MXF File Partitions Header Body Footer
Logical division of a file for easier parsing
Header
Has metadata about whole file
Body
Contains essence
Footer
Terminates file & can have updated metadata

13. Metadata in MXF Structural Metadata Descriptive Metadata Dark Metadata
Capability of a file & how it is constructed
Descriptive Metadata
Editorial aspects of the file
Several “schemes” available, such as DMS-1
Dark Metadata
Metadata unknown by some applications
Static Metadata
Doesn’t change over entire output
e.g. Show title, producer, etc.
Time-Dependent Metadata
Timeline or Event
e.g. “This part was shot by Bob”

14. MXF Object Model Packages Material Package File Package
A logical grouping of metadata sets
Material Package
Represents output timeline
Owns Tracks
File Package
Represents actual essence in the file

15. Physical vs Logical
physical - how the bytes are stored (essence container)
logical - what the bytes are intended to represent (packages)

16. MXF Object Model cont.

17. KLV Wrapping of Essence
“Frame Wrapped”
“Clip Wrapped”

18. MXF Standards SMPTE 377M MXF File Format Specification
Essence Containers:
SMPTE 385M SDTI-CP
SMPTE 381M MPEG (including long-GOP)
SMPTE 382M AES3 and Broadcast Wave Audio
SMPTE 383M DV 25/50
SMPTE 386M D-10 (IMX 50 Mbps I-frame MPEG2)
SMPTE 387M D-11 (HDCAM)
Descriptive Metadata
SMPTE 380M MXF Descriptive Metadata Scheme - 1 (DMS-1)

19. Why Bother? MXF is a good way to wrap up video essence into files
MXF is a good way to link rich metadata to video essence
MXF has support from many vendors for interoperation (but this is a work-in-progress)
MXF and file transfer may provide significant benefits for an IT-oriented workflow

20. PBS File Delivery Project (NGIS)
Distribute long-form, broadcast-quality video as files
To 180 independent public television organizations
Five major video server vendors, and some smaller vendors

21. NGIS in Operation

22. MXF Format Woes
MXF can wrap many different types of audio and video essence
There are a number of ways that variable size edit unit essence can be indexed
MXF has ten different “Operational Patterns” (OPs)
MXF can carry a wide range of descriptive metadata schemes

23. MXF – AS/PBS An MXF Application Specification for PBS
Base Requirements:
Single file per program distribution mechanism.
Direct play back (or import) into MXF compatible servers
Otherwise easy to translate into non-MXF compatible servers

24. AS-PBS Development Began in 2004 based on “AS-CNN”
Five key server vendors to public television initially involved
Expanded to include other server and translation vendors
Iterative process w. teleconferences and distribution
SD targeted first

25. MXF/AS-PBS Essence Video essence type: Video bitrate:
MPEG-2 4:2:0
Video bitrate:
8 Mbps
Audio essence type:
Uncompressed, 16-bit PCM audio in BWF
VBI (CC/XDS):
SMPTE 334M ANC carried in SMPTE 436M for MXF

26. MXF/AS-PBS MXF Features
OP1a
Partitions no longer than 30 minutes
Index Table segments in partition after the Essence they index

27. MXF/AS-PBS Future
To be defined:
HD Version
Archive version

28. MXF/AS-PBS tedwards@pbs.org Short “Golden Files” available
Actual full-length program example to be distributed shortly (MPEG pre-charge issue being worked out)
AS-PBS specification is available to those who want to see it,

29. Library of Congress Stake
Carl Fleischhauer
National Digital Information Infrastructure and Preservation Program
What is the Library of Congress stake in this topic, or perhaps better put, what are the Library of Congress stakes in this topic? There are two main activities for which the digital formatting of moving image content is important to us.

30. Video Reformatting at the National Audio-Visual Conservation Center
First is our own reformatting of historical collections. You may know that the major building-construction phase for the new National Audio Visual Conservation Center at Culpeper has been completed, and the staff is starting to move in. Culpeper is the new home of the Motion Picture, Broadcasting, and Recorded Sound Division. (Ken Weissman is representing the division at this meeting.)

31. The plan for video reformatting in the new Center is to implement the SAMMA system from the Media Matters company. One element of the SAMMA approach--the encoding of individual video frame images using lossless JPEG is being covered here at JTS by Ian Gilmour, part of the Media Matters team. The SAMMA device also wraps the frame images, sound track, and timecode data.

32. From SMPTE 390M
The wrapper being implemented for the Library of Congress is MXF, the same wrapper selected by the public television group. What is inside will differ, however. The Library’s image essences will be JPEG 2000 instead of MPEG-2. And this makes for another difference, pertaining to SMPTE’s MXF Generic Container. You saw Thomas Edwards diagram of the object model; there are separate containers for each element in the wrapper, for example, for picture and sound. The specs vary according to the specific essence contained.

33. Meanwhile, some aspects of metadata have been settled--for example, logging data is supplied by the SAMMA device--while other aspects are still being pinned down.

34. Concern for born-digital content
Second, there is the matter of born digital content that is being acquired by the Library. This topic is not only of interest to the Motion Picture, Broadcasting, and Recorded Sound Division but also to my unit, the Office of Strategic Initiatives, and our National Digital Information Infrastructure and Preservation Program, aka NDIIPP.

35. On the M/B/RS side, two important acquisitions with digital elements began to blaze this trail for the Library: the Coca-Cola advertising collection and the Vanderbilt television news collection. Both projects have been active since the 1990s and before and, in both cases, Library staff conferred with the donors about formatting.

36. The outcome was appropriately conservative for that time: MPEG-2 files at varying (but moderate) levels of resolution. Since MPEG-2 is publicly specified and widely employed (and carried into the digital broadcast era by the ATSC specification), it is a good bet for sustain-ability for at least the next decade. In the 1990s, the specifications for wrappers like MXF were just being published and it was premature to adopt them.

37. Born digital examples (not moving image) Web harvest GIS Still image
The 1990s and the current decade have also seen other parts of the Library acquiring digital content: geospatial data sets in the map division, graphic arts in the Prints and Photographs Division, to say nothing of eBooks and eJournals for the General Collections. Some arrives via the same path as the Coke and Vanderbilt collections, as carefully negotiated gifts, some arrives through copyright. In addition, we began harvesting the Web with the help of the Internet Archive.
Still image
Mark Yankus

38. In the face of these first digital arrivals, we found ourselves asking about infrastructure for ingestion, protocols for access, and so on. Some of us had a special worry: “How are we to plan for its preservation?” We also saw huge quantities of digital content out there and recognized that no one institution can preserve it all. Thus, another question: “Can we help nurture a national plan for distributed preservation? What might be the relationship between public archives and libraries and the creators of content?”

39. These latter questions underlie our partnership with public television
These latter questions underlie our partnership with public television. (We also have partnerships with entities for eJournals, geospatial data, social science data sets, and more.) The synergy with public television is especially appropriate, since the Motion Picture, Broadcasting, and Recorded Sound Division has been archiving PBS content--in conventional videotape form--for twenty or more years.

40. The public television team’s exploration has been enriched by outreach to organizations like the National Geospatial-Intelligence Agency of the Department of Defense, active promoters of the AAF specification (a close cousin to MXF), and the former AAF Association, now renamed the Advanced Media Workflow Association, promoters of increased implementation and adoption of the AAF-MXF cousins.

41. Preservation Formatting Working Scorecard
Interoperability very important
From-the-hip scorecards for
Wrapper
Essence
Profiles/Application Specs
Metadata
Distributed preservation and outreach to others raises the stakes for sorting out interoperability--which bring us back to formatting. Shooting from the hip, I have a quartet of scorecards.

42. Wrapper Scorecard No-wrapper vs. with-wrapper
Proprietary vs. open specification
Sizing up ISO/IEC Motion JPEG 2000
AAF/MXF is attractive
How settled in the community? Moderately
Wrapper. There is more than one tug of war here. As our own earlier acceptance of “naked” MPEG-2 files indicates, there is a no-wrapper practice that contrasts with a with-wrapper practice. There is also the proprietary-wrapper practice, based on typical uses of applications like AVID or FinalCutPro, in a tug of war with the interoperability camp, represented by organizations like the Advanced Media Workflow Association. And then, a little off to one side, is ISO/IEC specification 15444, part 3, better known as Motion JPEG It is a wrapper intended for use with JPEG 2000 frame encoding, thus a competitor for MXF-based implementations from the Digital Cinema Initiative and SAMMA. I say “off to one side” because I have not encountered any actual use of Motion JPEG 2000; my occasional visits to NAB suggest that, for broadcasters, SMPTE rather than ISO wrapper specs are winning the day. The Library has no official position on this but we are drawn to the pro-AAF/MXF forces. Degree of settled-ness? Moderate, the tugs of war continue.

43. Picture Essence Scorecard
MPEG (ITU-T) family
MPEG-2 (H.262)
MPEG-4 (H.263 but mostly H.264)
All I-frame possible (preservation value)
JPEG 2000 family
Frame image encoding
Possible wrapper
Less adoption in still photography than some wish
How settled in the community? Not bad, no need for one size to fit all
Essence encoding (in this short talk, picture essence encoding). Here two specification families seem to be in play. One is the MPEG family--perhaps we should also say the ITU-T family, since this European body is an important contributor to the encodings proper: H.261, 262, 263, and H.262 (MPEG-2) is very widely adopted; adoption of the newer H.264 group (MPEG-4's AVC) varies--more on the Web and less in broadcast production--and carries with it concerns about the impact of patents on long-term preservation. The other family is associated with JPEG 2000, still less widely adopted, in both its still- and moving-image implementations, than some would wish. It is worth saying that both MPEG and JPEG 2000 have variants with frame integrity. For MPEG, this is the all-I-frame structure. JPEG 2000 is inherently frame-based but it seems to have a special edge on the lossless front. (We are still waiting to learn more about H.264's so-called Fidelity Range Extensions.)
Degree of settled-ness? Not bad, all things considered.

44. Profile/App Spec Scorecard
Important part of MPEG family from an early day
Several ISO/IEC profiles and levels
SMPTE added one for high level
MXF is complex
Will benefit from application specification
Good example from PBS discussed today
JPEG 2000 family
Digital Cinema Initiative has established two profiles
LC has one for its newspaper scanning project
How settled in the community? In the early stages
Metadata, especially content-management and preservation metadata. My focus is, first, on the technical characteristics of the video object at hand, the sort of thing you would describe using the parametrics terminology in the SMPTE RP-210 data dictionary, or the instantiation segment of public broadcasting’s PB Core specification. Second is special metadata to support long-term preservation, the types of data associated with library community project called PREMIS. And some experts, like Dave MacCarn, see the need to go even further, seeking ways to incorporate into digital objects elements like essence encoding specifications and content-playing applications (open source preferred!). Degree of settled-ness? Less than we’d like.

45. Metadata Scorecard
Metadata for management, support long-term preservation
Technical characteristics
SMPTE RP-210 data dictionary (“parametrics”)
Public Broadcasting PB Core (“instantiation”)
Preservation support
Includes preceding
Library community PREMIS project, e.g., fixity
Specifications, source code for readers/transformers
How settled in the community? Not very
Profiles aka Application Specifications. This is critically important. The specifications for all of the wrappers and encodings I mentioned admit of so much variation that they must be constrained for true interoperability. Profiles and levels have been part of the MPEG specifications from an early day. The ISO/IEC specified a number of MPEG-2 and -4 profiles and levels, but omitted one that was important to broadcasters. Then, in 1998, SMPTE published the MPEG-2 4:2:2 profile at “high level” to serve this need. Meanwhile, you just heard from Thomas Edwards about PBS’s work on MXF application specifications. And on the JPEG 2000 front, the importance of profiles is suggested by the fact that the Digital Cinema Initiative has worked with ISO/IEC to develop a pair of profiles for their specification. But I have not seen anything comparable for JPEG 2000 as applied to video, including the reformatting of historical footage. Degree of settled-ness? In its early stages.

46. Preserving Digital Public Television