FhG Bonn, Generating Multimedia Presentations Stefano Bocconi Visual material taken from the film Minority report, TM and © 2002 Twentieth Century Fox and Dreamworks

FhG Bonn, Information Systems Multimedia and Human Computer Interaction Group We are currently working on four aspects: modeling of domain-dependent and domain-independent discourse to steer the presentation generation process, investigate dependencies of the user and domain models in the generation process, investigate characteristics of media types for presenting information to the user. investigate to what extent graphic design knowledge can be included in the generation process

FhG Bonn, The Genre of Documentaries A documentary purports to present factual information about the world The viewer is led to think that the facts are filmed as they happened Video Documentary

FhG Bonn, Categorical Documentaries Conveys information in a simple fashion over a specific category (or more) Examples are scientific documentaries (national geography) The category and its subcategories provide a scheme for the narrative Video Documentary

FhG Bonn, Rhetorical Documentaries The goal is to persuade the audience to adopt an opinion Arguments from the source Subject-centered arguments Viewer-centered arguments Video Documentary The River, Pare Lorentz (1936)

FhG Bonn, Interview with America Subject: the opinion of American people after 9-11 Shooting: from to in Stamford (CT), New York (NY), Boston (MA) and Cleveland (OH) Editing: approximately 3 months Video Documentary

FhG Bonn, The problems Only one final version: what do we show? Who is pro and who is against? What are the numbers? Biased versus objective trade- off: a choice must be made Video Documentary

FhG Bonn, Providing Access to the Source Annotate the material: What question was asked Who answered (social categories) Where Possible query: show me all the answers to question “what do you think of the war in Afghanistan” given by black men Open Documentary

FhG Bonn, Rhetorical Tools According to Aristotle, rhetoric is "the ability, in each particular case, to see the available means of persuasion” Three main forms of rhetoric: Ethos, Logos, and Pathos Open Documentary

FhG Bonn, Ethos Ethos is appeal based on the character of the speaker. An ethos-driven document relies on the reputation of the author Ethos is used in what Bordwell calls Arguments from the source Open Documentary

FhG Bonn, Logos Logos is appeal based on logic or reason. Documents distributed by companies or corporations are logos-driven. Scholarly documents are also often logos-driven Logos is used in what Bordwell calls Subject-centered arguments Open Documentary

FhG Bonn, Pathos Pathos is appeal based on emotion. Advertisements tend to be pathos- driven Emotions have a functional role as constraints on our decision making (Damasio) Ethos is used in what Bordwell calls Viewer-centered arguments Open Documentary

FhG Bonn, Example “I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing, I suppose there is a point in which certain people play by certain rules and you have to go to their level, I do not think there is any way to resolve this conflict diplomatically” Open Documentary Subject is Controversial The position is Pro or Against

FhG Bonn, Using the Logos Concession: I do not like A, but B A = war in Afghanistan (military action) B = only solution reasoning A is the only solution because B0) A is cruel B1) the attack was cruel B2) the response has to be like the attack Open Documentary PROPRO AGAINSTAGAINST

FhG Bonn, Using the Logos Different rhetorical figures, in general enthymemes: War is the best thing to do because it will solve the problem Unique : only A is possible Antithesis : only B and A possible and B not a solution Optimal : A is the best solution Unconditional : A is true Evidence : (A if B) and B is true The reasoning can be explicitly expressed in the interview or belong to the “common sense” Open Documentary

FhG Bonn, Using the Pathos How and what emotions can we use to persuade? Cognitive structure of emotions (Ortony) The expressions for happiness, sadness, disgust, surprise, anger and fear are culture-, gender- and age independent (Ekman & Friesen) Open Documentary PROPRO AGAINSTAGAINST

FhG Bonn, Strategies We saw some “local” strategies More high level strategies are possible, like increasing / decreasing or hyperbole To form a longer presentations, higher level strategies are needed Open Documentary Rhetor ic PathosLogos ReplaceAugment

FhG Bonn, Issues Advantages Reuse the reasoning of the humans, no need for a very complex system like Cyc Disadvantages Extensive annotation is needed Need for a knowledge base Validation User studies? Mimicking the reality

FhG Bonn, Previous Work “Media Streams is a system for annotating, retrieving, repurposing, and automatically assembling digital video. It uses a stream- based, semantic representation of video content with an iconic visual language interface of hierarchically structured, composable, and searchable primitives.” AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing

FhG Bonn, Related Work `Terminal Time is a history "engine:" a machine which combines historical events, ideological rhetoric, familiar forms of TV documentary, consumer polls and artificial intelligence algorithms to create hybrid cinematic experiences for mass audiences that are different every single time’

FhG Bonn, References Damasio, Antonio - Descartes' Error: Emotion, Reason, and the Human Brain, Avon Books, Bordwell, D. (1989). Making Meaning - Inference and Rhetoric in the Interpretation of Cinema. Cambridge, Massachusetts: Harward University Press. Davis, M. Media Streams: Representing Video for Retrieval and Repurposing, Massachusetts Institute of Technology, Cambridge, Massachusetts, Nack, F. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing, Lancaster University, Ortony, A., Clore, G. L. and Collins, A. - The Cognitive Structure of Emotions. Cambridge University Press, Ekman, P. and Friesen, W.V., Facial Action Coding System. Consulting Psychologists Press Inc., Terminal Time

FhG Bonn, Discussion

FhG Bonn, Using the Logos Different rhetorical figures, in general enthymemes: War is the best thing to do because it will solve the problem Unique : only A is possible Antithesis : only B and A possible and B not a solution Optimal : A is the best solution Unconditional : A is true Otherwise : (B if not A) and B not good Evidence : (A if B) and B is true Condition : (B if A) and B good Open Documentary