1. From Semantically Annotated Media To Video Documentaries Stefano Bocconi CWI Amsterdam, The Netherlands

2. Talk Outline Motivation Example Scenarios Technical details – Annotations – Editing Process Conclusions Talk Outline

3. Philips, 31-05-2006 Case Study Subject: the opinion of American people after 9-11 Shooting: from 27-10- 2001 to 01-11-2001 in Stamford (CT), New York (NY), Boston (MA) and Cleveland (OH) Editing: approximately 3 months http://www.interviewwithamerica.com/

4. Philips, 31-05-2006 The problems Only one final version: what do we show? Who is pro and who is against? What are the numbers? Biased versus objective trade-off: a choice must be made

5. Philips, 31-05-2006 Authoring Documentaries Traditional video editing: there is only one final version, what is shown is the choice of the author/editor Proposed video authoring: ● Annotate the video material semantics ● Show automatically what the user asks to see, using presentation forms a film editor would use

6. Philips, 31-05-2006 Example “I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing

7. Claim Concession Claim contradict support Claim I am not a fan of military actions War has never solved anything Two billions dollar bombs on tents I cannot think of a more effective solution weaken Analysis of the Example

8. Scenarios Vox Populi: Augmenting one interview with man- on-the-street opinion Overview of the content of video footage: ● Example: trailers ● Browse the content by opinion Voices of Iraq

9. Philips, 31-05-2006 What question was asked Who answered (social categories) Filmic (e.g. location/time/framing/gaze) Possible query: show me all the answers to question “what do you think of the war in Afghanistan” given by black men Descriptive Annotations

10. Statements formally annotated: – – E.g. “war best solution” A thesaurus containing: – Terms (155) – Relations between terms: similar (72), opposite (108), generalization (10), specialization (10) – E.g. war opposite diplomacy Annotations: 1 hour annotated, 15 interviews, 60 interview segments, 120 statements Rhetorical Annotations

11. ClaimData Qualifier WarrantBackingConditionConcession 57 Claims, 16 Data, 4 Concessions, 3 Warrants, 1 Condition Toulmin Model

12. Using the thesaurus, generate related statements and query the repository – E.g. from “war best solution” “diplomacy best solution”, “war not solution” Create a graph of related statements – Nodes are the statements (video segments), edges are either support or contradict The Generation Process

13. Partially tunable: examining the Segment graph gives feedback on the quality of the annotations and the thesaurus The annotator can revise the annotations in a guided manner S1 S2 S3 S5 S4 S7 S6 S8 S9 S10 = support = contradict The Semantic Graph

14. Video documentaries are not neutral account of reality: the selection and editing of the footage expresses a point of view Editing strategy: Balanced, Pro opinion X and Against opinion X Vox Populi uses: – Logos (the statements) – Ethos (based on user profile) – Film editing (framing, transitions, counterpoint editing) Controlling Bias

15. Vox Populi automatically presents video interviews augmented with supporting and/or contradicting material The user can determine the subject and the bias of the presentation Extensive annotations are needed Conclusions This presentation and a Demo available at: http://www.cwi.nl/~media/demo/IWA

16. Philips, 31-05-2006 Discussion

17. Philips, 31-05-2006 Issues Advantages Reuse the reasoning of the humans, no need for a very complex system like Cyc Disadvantages Extensive annotation is needed Need for a knowledge base Validation User studies? Mimicking the reality

18. This presentation and a Demo available at: http://www.cwi.nl/~media/demo/IWA/ This research was funded by the Dutch national ToKeN2000 I 2 RP and CHIME projects.

19. Philips, 31-05-2006 Using the Pathos How and what emotions can we use to persuade? Cognitive structure of emotions (Ortony) The expressions for happiness, sadness, disgust, surprise, anger and fear are culture-, gender- and age independent (Ekman & Friesen) Open Documentary PROPRO AGAINSTAGAINST

20. The annotations Descriptive – Question asked – Interviewee (social) – Filmic (e.g. location/time/framing/gaze) Rhetorical – Rhetorical Statement (mostly verbal, but visual also possible) – Argumentation model: Toulmin model Annotations

21. Philips, 31-05-2006 Rhetorical Tools According to Aristotle, rhetoric is "the ability, in each particular case, to see the available means of persuasion” Three main forms of rhetoric: ● Ethos: appeal based on the character of the speaker. ● Logos: appeal based on logic or reason. ● Pathos: appeal based on emotion. Open Documentary