School of Informatics CG087 Time-based Multimedia Assets Timecodes & SyncP. Vickers/J. Edwards/A. Watson1 Time codes and Synchronisation Synchronising video and audio with time codes such as SMPTE and SMIL

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson2 Time codes A time code allows each frame of video to be numbered. Numbering frames allows precise identification when editing. It also facilitates synchronisation of audio with video An 80-bit word is stored on each frame to represent hours, minutes, seconds, frames and other synchronizing information. There is also room for ‘user bit’

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson3 Time code example 03:10:45:18 Hours Minutes Seconds Frames

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson4 Starting time codes 2 ways of starting a time code –Record run –Free run

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson5 Record run Only stamps a time code when camera is recording. Time code set to zero at the start of the day’s operation and a continuous record is produced on each tape covering all takes Customary to record the tape number in place of the hour section on the time code –E.g. the first cassette of the day would start , and the second cassette would start

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson6 Free run Time code is set to the actual time of day and when synchronized is set to run continuously. Whether the camera is recording or not, the internal clock will continue to operate. When the camera is recording, the actual time of day will be recorded on each frame. –Useful for editing day-long events (e.g. sport) Action can be logged by time as it occurs and can be quickly found by reference to the time code on the recording

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson7 Free run: gaps Shot 1 (8 seconds) Shot 2 (5 seconds) Shot 3 (7 seconds) 200 frames125 frames175 frames 04:12:45:0114:44:22:0515:02:44:03

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson8 Free run: gaps A change in shot produces a gap in time code corresponding to the elapsed time between recordings Shot 1 (8 seconds) Shot 2 (5 seconds) Shot 3 (7 seconds) 200 frames125 frames175 frames 04:12:45:0114:44:22:0515:02:44:03

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson9 Time code problems Missing time code numbers can cause problems with edit controller when it rolls back from intended edit point and is unable to find time code number it expects there (i.e. the time code of the frame to cut on, minus the pre-roll time).

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson10 Synchronisation To fit a sound track to a video sequence you need to syncrhonise the audio with the video When a system is synchronized, all the video and audio decks, computer sequencers, hard disks and so forth will play back exactly together. This allows you to compose layers of sound, secure in the knowledge that once you get something in place, it will stay there. (see mpte/SMPTE.html) mpte/SMPTE.html

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson11 SMPTE This section shamelessly plagiarised from The Society for Motion Picture and Television Engineers (SMPTE) has defined its own time code SMPTE format allows different devices to be synchronised in a master/slave arrangement –Master device must be a video deck –Other devices follow the master –Multiple video decks are ‘genlocked’ to a clock device Genlocking is a line-by-line (rather than frame-by-frame) synchronisation system

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson12 SMPTE overview SMPTE code recorded on one audio track Actually a modulated tone that is decoded into data 80-bit word for each frame to store hh:mm:ss:ff + subcode information

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson13 SMPTE gadgets SMPTE generator –Creates the time code SMPTE reader –Displays the time Regenerator –Reads the code and recreates it for recording on another device

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson14 SMPTE gadgets continued MTC generator –Converts SMPTE into MIDI time code which can be sent over MIDI cables SPS converter –Turns SMPTE into song pointer and MIDI clock data to control sequencers FSK –System for recording MIDI song pointer and MIDI clock data directly on tape. Simpler than SMPTE but less flexible

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson15 SMPTE gadgets continued Controller –Tape deck remote control –Uses SMPTE to monitor tape position and can cue tape to any desired location –More accurate than a mechanical tape counter Synchroniser –Controls >=2 tape decks

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson16 SMPTE frame rates 24 fps –Film 25 fps –European video (PAL, SECAM) 30 fps –American & Japanese B&W video fps –American & Japanese colour video. Can lock 30fps audio deck to this, but pitch will drop from A=440 to Hz

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson17 SMPTE numbering schemes Each frame contains hh:mm:ss:ff SMPTE time code can only increment in whole frame numbers How to deal with odd division of frames in US/Jap colour format (29.97 fps)? At each new frame (now lasting a little more than 1/30th of a sec.), the Time Code still increments by a whole 1/30th and drags behind real time

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson18 SMPTE DF and NDF In Non Drop Frame mode we slow down the code generator to match the frame rate –This makes a 1-hour programme actually last 59:56.4! –29.97/30 * 3600 = s In Drop Frame mode we skip two frame numbers a the start of each minute except those ending in zero (00, 10, 20, 30, 40,50) –54 mins that have 2 frame no.s added = 108 –3600s *29.97 fps = frame no.s = = 3600 * 30

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson19 SMPTE and MIDI MIDI time code is a specialised SMPTE code that can be transmitted over MIDI cables Most sequencers now support SMPTE control Look at DOP’s Options|Sync Settings

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson20 SMIL Synchronized Multimedia Integration Language. Pronounce it "smile" ! An XML abstraction for Synchronisation of Multimedia Originally proposed around 1998 At present v2.0 Can be used in IE6 and RealPlayer GRINS and SOJA also example implementations Synchronisation reflects position and time

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson21 A simple SMIL document …… looks familiar ??? SIMILAR STRUCTURE TO HTML

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson22 Positioning Not really very exciting (link region to data)

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson23 Positioning - Relative

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson24 OTHER MM TAGS AVAILABLE MediaTagG2GRiNSSoja GIFimgOK JPEGimgOK Microsoft WavaudioOK - Sun AudioaudioOK Sun Audio Zippedaudio--OK MP3audioOK-- Plain texttextOK Real texttextstreamOK-- Real movievideoOK-- AVIvideoOK - MPEGvideoOK - MOVvideoOK--

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson25 The meat of SMIL is timings!! This is really what sets it apart from DHTML Allows the specification of –Duration – Dur = “3s” –Sequence - –Synchronisation – id referencing

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson26 Duration

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson27 Sequence

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson28 Sychronisation

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson29 Synchronisation synchronisation specified us id(previous sources)(time duration)

CG087 Time-based Multimedia Assets School of Informatics Timecodes & SyncP. Vickers/J. Edwards/A. Watson30 SMIL Summary FLASH for the open source world !!! SMIL 2.0 augments the above framework to incorporate more media handling –More animation –More event handling –Multiple Windows Reference : All examples from – Reference: SMIL 2.0 –