1. Automatic Video Authoring with Media Analysis 2003/11/25 Chen-hsiu Huang Advisor: Dr. Ja-Ling Wu

2. Outline  Problem formulation  Solutions and framework  Expectations  Status report  Demonstration  Questions and discuss

3. Problem Formulation  The digital video capture devices such as DVs are made more affordable for end users.  It’s interesting to shoot videos but frustrating for editing them.  There’s still a tremendous barrier between amateurs (home users) and the powerful video editing software.  Finally people leave their precious shots in piles of DV tapes without editing and management.  Type I home video: memorial  Type II home video: recreational

4.  According to a survey on DVworld*, the relations between the video length and how many times will user review them after days:  People are inpatient for videos without scenario or voice-over, especially for those with no music.  Video clips with no more then 5 minutes are best for human’s concentration. Video lengthReview times >= 1 hr1 or 0 30 min ~ 1 hr2 ~ 3 15 ~ 30 min5 ~ 10 5 ~ 15 min>= 10 <= 5 minYou take it out and watch it when you think about! *http://www.DVworld.com.tw/http://www.DVworld.com.tw/

5. Solutions and Framework  A consumer product called “muvee autoProducer” has been announced to ease the burden of professional video editing.  The application scenario is quite simple:  Pickup a video  Choose your music  Produce musical video!  Although there are commercial products in the market, only few academic publications related.  Goal: To achieve the near or beyond quality in the similar application scenario with the content-analysis technologies developed in multimedia domain. Jump!

6. Input video Input music Shot change Scene change Audio segment cutting Alignment Output Video Volume ZCR Brightness Bandwidth … Human face Flash light Motion strength Color variance Edgeness... Scene selection Key shot selection Audio rhythm & Video motion/color synchronization Framework of AVAMA system Currently finished

7. Expectations  Quality results  Time & space complexity consideration  Must not take too longer to make a video  Must not consume too much memory  Complete consumer software implementation  Deal with MPEG-1/2 videos and MPEG-1 Layer 1/2/3 audios directly without pre-processing  Simple application scenario to produce videos  The need of different profile?  Is semi-automatic necessary?

8. Status Report  Audio Cutting:  Cutting with zero-crossing rate (not so good)  Cutting with dramatic volume change (great results)  Brightness, bandwidth,...  Video Segmentation:  Shot change detection by pixel MAD  Detect scenes with flash light event  Detect fast motion scenes  Longer fast motion scenes means author’s tracing some objects  Short fast motion scenes may due to  Detect human face information (time complexity?)

9. Demonstration

10. Questions and Discuss  Any comments are welcomed.  Special thanks for Mr. 劉嘉倫, for his videos and suggestions.  Thanks friends in DVworld who provide lots of ideas and comments.

11. Create Music Videos using Automatic Media Analysis, ACM Multimedia, 2002

12.  Let’s see its demo! Back