WLAP: Improving acquisition Workshop on digital video archiving 22 June 2001, CERN Hector Sanchez San Martin Universitat Jaume I Ing. Informatica CERN ETT-DH

Hector Sanchez22 June CERN Contents Background introduction Status report Possible improvements for the acquisition Unsolved problems & future work

Hector Sanchez22 June CERN Background introduction Lots of courses, seminars and tutorials are given at CERN WLAP: Web Lecture Archive Project Need for keeping an archive Internet allows making them world wide available Web Lectures (WL) Made of video, slides and timing information Nowadays: Set of HTML files produced by SyncOmat Future: Standard for describing WLLecture Object Started at CERN in 1999 Aim: Implement an e-archival system for slide based lectures on the Internet

Hector Sanchez22 June CERN For having a persistent archive of WL is necessary to keep the data in a format which allows to obtain any desired output Don’t keep WLs but “raw” data (video, slides, timing) The way of storing this data has to be consistent We will be able to generate any kind of output: syncOmat, lecture objects, only video,... We will focus on the acquisition of data Need of a real system of WL production Current archive is not scalable (syncOmat WL are static) Acquiring data is the first step towards an archive Which are the current needs? A time persistent archive

Hector Sanchez22 June CERN Initial scenario (I) WLAPer Contact the speaker SpeakerOrganiser Contact the speaker*Give transparenciesRecord lectureDigitalise videoEdit videoScan slidesProduce WLPublish WLPublish Info LectureObtain lecture info BulletinWeb Main Page... Timing

Hector Sanchez22 June CERN Initial scenario (II) The process is carried out in a quite manual way Some tools have been developed in the production part Acquisition is too heavy Some tasks are performed more than once Contacting the speaker, publishing info about the lecture,… Dependence on one person: WLAPer Actors making technical tasks in which they are not specialised Contacting the speaker, recording, making web pages… Loss of quality in the final product

Hector Sanchez22 June CERN Current scenario (I) WLAPer Contact the speaker SpeakerOrganiser Contact the speaker*Give transparenciesRecord lecture & timingDigitalise videoEdit videoScan slidesProduce WLPublish WLPublish Info LectureObtain lecture info BulletinWeb Main Page... Video service Scanning service

Hector Sanchez22 June CERN Current scenario (II) Specialised professionals dealing with technical tasks Video & scanning Better quality “raw” data Process is more distributed STILL dependence on WLAPer to archive “raw” data STILL: Manual process, repeated tasks New techniques tested and currently in production More people involved means more channels of communication This could make the process more inaccurate and slower The way for transmitting information between the actors is manual

Hector Sanchez22 June CERN Future scenario (I) WLAPer Introduce Lecture Info SpeakerOrganizer Contact the speakerSubmit transparenciesScan slidesProduce WLPublish WLPublish Info Lecture BulletinWeb Main Page... Scanning service Obtain lecture infoRecord lectureDigitalis videoEdit video Video service Timing ?

Hector Sanchez22 June CERN Future scenario (II) The archival system is automatic and transparent to actors Actors only do their work informing the system about the lecture The system will archive the files as necessary The system is autonomous Communication between actors is carried on inside the system No need for human co-ordination Ex: WLAPer will only notice that all files concerning a lecture are avaliable Some work is needed to achieve this situation Re-organising the process Develop SW for supporting it

Hector Sanchez22 June CERN How? Organiser CDS Agenda Maker Speaker Lecture data Library DB Video service CDS Submit Mail Transp. Scanning service Upload script Lecture Id & Scanned trasp. Upload script E-documents Doc FS Webcast FS Lecture data Video [Timing] Link Manager Creates a bibliographic record *CDS Search *Bulletin *Organiser web page *Webcast pages... Notice

Hector Sanchez22 June CERN Editing the video: The basis Once the video of the lecture is encoded it needs to be edited Adjust it to the real duration of the lecture The timing file needs to be resynchronised Normally the timing tool is started before the lecture starts This two adjusts need human intervention The main idea is taking some data during the recording of the lecture that allows to do the editing automatically

Hector Sanchez22 June CERN Editing the video: Example time Starts live encoding Starts recording tape Starts timing Starts lecture Ends lecture Ends recording tape First slide change Ends live encoding How has the video to be cut? Between 00:10:00 and 01:10:00 Tsl-Tse Tel-Tse 10:50:00 00:00:00 Tse 11:00:00 00:10:00 Tsl 12:00:00 01:10:00 Tel

Hector Sanchez22 June CERN Editing the video: How? Lecture recorder Taketime Webcast FS Editvideo File with global times Lecture Data & events Edited video Script that will run in batch

Hector Sanchez22 June CERN Editing the video: Final notes This can be used also to synchronise the timing file Technology used: Python+Tk (tkinter) Can be run in different platforms: Win, UNIX, MAC Some tests have been made but more testing is needed Very easy to use: No need for a computer specialist We could reduce human intervention to do some clicks

Hector Sanchez22 June CERN Upload scripts: The basis The archival system is based on the Talk Number Id Numbers are always difficult to handle The process has to be transparent for people involved Transfer of files to servers Lecture Id vs. Lecture data This is the sw support for achieve an automated archive of raw data It’s a kind of interface for archive “raw” data The user selects the lecture his material is related to The script will obtain the talk number and archive the files

Hector Sanchez22 June CERN Upload scripts: Architecture MySQL Db pymod MySQL Client Agenda DB GUI Config File CORE RealProducer smtplib pymod Organiser mail ftplib pymod Webcast FS User Lectures data & id notification Encoded video encoded video Lecture data Upload script for encoding&uploading video

Hector Sanchez22 June CERN Upload scripts: Final notes Technology used: Python+Tk (tkinter) + MySQL This will be extended to other parts of the process Submission of scanned transparencies Submission of timing files First version of the script will be tested next week Ready for Summer Student lectures For now ASCII interface Almost automatic publication of videos The user only selects the lecture to record The video of the lecture will be available a few minutes after it is finished

Hector Sanchez22 June CERN Unsolved problems & future development Timing acquisition Agreement between people involved Master copy of the digital video: MPEG??? Better tool integration A lot of different tools could be integrated (timing, encoding, editing,…) RealMedia is not the better master format Very high quality video file which can be converted to any format When not using MS PWPT another person is needed