1. Captioning Digital Multimedia Geoff Freed Carl and Ruth Shapiro Family National Center for Accessible Media (NCAM) WGBH Educational Foundation http://ncam.wgbh.org

2. What to expect  Part I: Brief history; current state  Part II: How it’s done — editors — style  speed; convenience; quality  Part III: What’s next — formats — standards — recommendations — regulations

3. 3 About NCAM  Carl and Ruth Shapiro Family National Center for Accessible Media at the WGBH Educational Foundation (NCAM); http://ncam.wgbh.org http://ncam.wgbh.org  Part of the Media Access Group Media Access GroupMedia Access Group  The Caption Center (1972)  Descriptive Video Service (1990)  NCAM (1991)

4. 4 About NCAM  R&D facility with the mission to make electronic media of all types accessible to people with sensory impairments  Work funded by federal grants, private foundations and strategic partners large and small  Expertise in on-line accessibility of all kinds (Web, multimedia, PDF, captioning, description, etc.)  Expertise in standards and guidelines (Section 508, WCAG, SMIL, ATSC, SMPTE, TTML, PDF, Flash, e-books/textbooks, image description, etc.)

5. Part I: General Information

6. What are captions?

9.  A visual representation of spoken narration or dialogue  Indicate important non-speech information: — sound effects, music, laughter — speaker identification  Synchronized to appear simultaneously with audio  Displayed in either pop-on or roll-up styles  In some countries, captions are called subtitles

10. What are captions?  Captions and foreign-language subtitles are not the same thing — captions contain information in addition to narration and dialog; subtitles do not — captions are frequently positioned on the screen to indicate who is speaking; subtitles are not

11. What are captions?  Captions can be closed or open: — closed captions can be turned on and off by the user — open captions are visible to everyone and cannot be turned off  QuickTime Player, iTunes, Apple mobile devices, RealPlayer, Flash, Silverlight and Windows Media Player all provide caption controls — some are custom, some are not  HTML5 introduces browser playback and control

12. What about transcripts?  A transcript provides a text version of the audio track — a transcript is useful for creating captions — a transcript is a by-product of the captioning process  Transcripts should be considered a supplement to, not a replacement for, synchronized captions

13. Captions can be displayed by all major multimedia players  QuickTime (embedded or external track/QTtext format) below the video region translucent overlay transparent overlay

14. Captions can be displayed by all major multimedia players  QuickTime or iTunes (embedded track/SCC)

15. Apple devices: iTunes

17.  RealPlayer (external track; RealText) Captions can be displayed by all major multimedia players below the video region transparent overlay

18. Captions can be displayed by all major multimedia players  Windows Media Player (external track; SAMI)

19. Captions can be displayed by all major multimedia players  Flash (ccPlayer; TTML) Flash

20. Apple devices/SCC captions iPod nano iPhone/iPod touch/iPad

21. Captions can be displayed by all major multimedia players  Some BlackBerry smartphones

23. More and more on-line programming is captioned  ABC.com  Hulu.com  Hulu desktop  MTV  NBC.com  Netflix Instant Play  YouTube  others

24. On-line customization  Some players allow customized views — YouTube (no account required) — Hulu (account required to save preferences) Hulu

25. Part IIa: How It’s Done/General Rules CC University (the abbreviated course)

26. Authoring captions  The most important aspect of caption writing is not… — software — technical format — delivery — UI

27. Authoring captions  The most important aspect of caption writing is… accuracy — accurate transcription — accurate spelling — accurate editing — accurate formatting — accurate timing — accurate reviewing  Speed, convenience/quality

28. Authoring captions  Example 1  Example 2

29. Authoring captions  Most caption-authoring applications follow the same basic procedure — transcribe audio  external transcription/import is usually easier (if permitted) — format and edit the text  divide text into discrete captions  divide rows within captions  edit if/as necessary — time the captions  verbatim vs edited — review; export

30. Authoring captions: transcription  Accurately represent what is spoken — spelling, spelling, spelling — don’t add information — don’t edit unless there is reason to do so  reading level; special vocabulary  “there’s three things…” vs “going to/gonna”  fillers  don’t censor — indicate different speakers when necessary — indicate sound effects when necessary  Generally speaking, it’s faster to transcribe into a text editor and import the text into the caption editor

31. Authoring captions: formatting  Make the captions easy to read — use appropriately sized fonts — use fonts that are easy to read  sans serif vs serif  open characteristics — break rows in logical places — break captions in logical places  end punctuation  natural pauses  Formatting is especially important for small-screen readability

32. Authoring captions: timing  Time the captions to appear when corresponding words are spoken — lead/lag +/- one second if it is appropriate for speed — take advantage of pauses (to an extent) — align with shot changes (+/- one second) for a cleaner appearance  Verbatim timing is expected unless there is a reason to do otherwise — language level/comprehension

33. Authoring captions: timing  Timing example 1  Timing example 2

34. Authoring captions: reviewing and exporting  Always review carefully — correct/edit/re-time as necessary — if a long video has been captioned by multiple authors, ensure that everyone has followed the same style rules  spelling, timing, editing, presentation conventions  Export to the appropriate target format

35. Part IIb: How It’s Done/Caption Editors

36. Various editors  Annotation Edit Annotation Edit  CapScribe OpenCapScribe Open  CPC CPC  DIVX DIVX  Gnome Subtitles Gnome Subtitles  Jubler Jubler  MAGpieMAGpie  MovieCaptionerMovieCaptioner  Subtitle Workshop Subtitle Workshop  vSync format converter vSync  SubPLY, Subtitle Horse (on-line editors; export captions in various formats) SubPLYSubtitle Horse

37. (Cautiously) Using YouTube  YouTube can generate a complete caption file (transcribed and timed) for you (aka auto-caption) — upload video; wait for caption file to be generated — download caption file, clean up and re-upload  edit with a text editor  use a caption editor (required if re-timing is necessary) — demo: not bad but still requires clean-up and correction  You don’t have to do any clean-up, but…  In most cases, you must correct the auto-generated file

38. Other YouTube options  Upload a plain-text transcript — YouTube will generate a timed script (movie) — download caption file; correct timing; re-upload  Upload your own complete caption file (movie) — in most cases, this is the most accurate option — TTML, SRT formats; others  File-creation guidelines for YouTube caption files File-creation guidelines

39. Let others write captions for you  Professional captioning agency, such as the Media Access Group at WGBHMedia Access Group at WGBH  Crowdsourcing — free labor — loss of quality control  YouTube Subtitler YouTube Subtitler  CaptionTube CaptionTube  Universal Subtitles Universal Subtitles  Overstream Overstream  dotSUB dotSUB

40. Part III: What’s Next?

41. New rules  21 st Century Video Communications and Video Accessibility Act 21 st Century Video Communications and Video Accessibility Act — programs that were originally captioned for broadcast must retain captions when distributed over IP  does not govern mobile television — FCC now considering final rules — distribution format for captions/subtitles under consideration (or not)

42. Formats  Old way — each multimedia player/device used its own text-display format  New way — all players and devices use a single non-proprietary format (e.g., TTML)  The real way… — no single format will be used by all devices  FCC ruling on formats for IP distribution will have big impact –VPAAC working-group recommendation is SMPTE-TTVPAAC

43. Formats  Existing open formats — TTML TTML  BBC, Netflix, Flash video, others  TTML community group at the W3C TTML community group — SMPTE-TT SMPTE-TT  convert broadcast captions for IP delivery  UltraViolet UltraViolet  Coming soon — WebVTT (WHAT-WG) WebVTT — WebVTT (W3C) WebVTT — WebVTT (W3C community group) WebVTT  Prediction — no agreement on a single contribution format — TTML, SMPTE-TT and WebVTT will be the primary contribution formats

44. Viewing captions the new way: Web  HTML5 makes it much easier to embed video/audio into Web pages —, ; no plug-ins — to identify and synchronize external caption/subtitle file(s)  currently no agreed-upon baseline format  (no agreed-upon video format, for that matter) — no public support today, but soon  What it might look like

45. Viewing captions the new way: mobile  Apple, BlackBerry devices  Mobile TV (OTA) — ATSC M/H (A/153) supports CC carriage — some LG and RCA receivers decode captions if availableLGRCA — receivers also available to build into cars/buses  watch television while traveling at speeds up to 120 mph — currently no regulations mandating ATSC M/H captions

46. Resources  List available at http://tinyurl.com/coa9ykkhttp://tinyurl.com/coa9ykk