When a picture and its words don't match: Irony and Integration in Comics Shweta Narayan University of California, Berkeley April 8 th 2006

Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Irony and Integration in Comics ● Integration? – Mental Spaces ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Mental Spaces ● Theoretical framework – e.g. Fauconnier and Turner, 2002* ● Used to explain a wide variety of linguistic and cognitive phenomena * Fauconnier, Gilles and Mark Turner The Way We Think: Conceptual Blending and the Mind's Hidden Complexities. New York: Basic Books.

Mental Spaces ● Spaces are cognitive domains – There are mappings between spaces.

Mental Spaces ● Spaces are cognitive domains – There are mappings between spaces. ● Let's look at “trashcan basketball” – How can we understand this at all? – What do trashcans have to do with basketball?

Mental Spaces ● Trashcan Basketball

Mental Spaces ● Trashcan Basketball Domain 1: Trashcans * Trash * Container

Mental Spaces ● Trashcan Basketball Domain 1: Trashcans Domain 2: Basketball * Trash * Container * Ball * Hoop * Points

Mental Spaces ● Trashcan Basketball * Trash * Container * Ball * Hoop * Points Domain 1: Trashcans Domain 2: Basketball Mappings

Integration ● Mental Spaces can be integrated ● Creating new spaces with – inferences mapped from the input spaces – and new inferences that emerge from the process of integration.

Integration ● Trashcan Basketball – Inference from Trashcans: You don't retrieve the ball – Inference from Basketball: It's a game – Emergent structure: Slam dunks aren't cool TrashcansBasketball * Trash * Container * Ball * Hoop * Points

Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Comics as data

Time 1 Static image

Comics as data Time 1 Static image Time 2 Static image Panel boundary

Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion

Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion Time 3 Inferred motion

Comics as data Time 1 Static image Time 2 Static image Panel boundary Inferred motion Time 3 Static image Speech bubbles Inferred motion

Comics as data ● Linguistic and Pictorial component

Comics as data ● Linguistic and Pictorial component ● How do they relate?

Comics as data ● Linguistic and Pictorial component ● How do they relate? Type: Entity Property: can-talk

Comics as data ● Linguistic and Pictorial component ● How do they relate? Speech Content Type: Entity Property: can-talk

Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning Type: Entity Property: is-talking Speech Content

Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning ● Complementary information

Comics as data ● Linguistic and Pictorial component ● How do they relate? – Emergent meaning ● Complementary information ● Conflict?

Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

Conflict between characters' belief spaces

Picture Space Depicted objects: Political signs Speech space Content: implies lack of understanding of ch2's cognitive viewpoint

Conflict between characters' belief spaces Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs

Conflict between characters' belief spaces Depiction space Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects

Conflict between characters' belief spaces * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs Professor (off-screen) Type: Entity Property: has beliefs Property: can see Depiction space

Conflict between characters' belief spaces * Ch1 * Ch2 (Tajel) (Prof) * Speech * Depicted bubble objects Type: Entity Property: has beliefs Property: is-talking Speech Content implies beliefs Professor (off-screen) Type: Entity Property: has beliefs Property: can see Content informs beliefs Depiction space

Conflict between characters' belief spaces * Ch1 * Ch2 (Agatha) (Adam) * Speech * Depicted bubble object Type: Entity Property: cannot see Property: has beliefs Property: is-talking Speech Content implies beliefs Type: Entity Property: has beliefs Property: can see Content informs beliefs Depiction space

Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

Concept: “digital age” Information: - content: implies ink and paper are outdated and unnecessary Entity: US Constitution Information: - content: depiction of entity Narrative-level Conflict

Concept: “digital age” Information: - content: implies ink and paper are outdated and unnecessary - style: register shift to informal rhetorical question hand-lettering Entity: US Constitution Type: Ink-on-paper Evokes democracy as American cultural ideal Information: - content: depiction of entity - style: evokes action hand-drawing Narrative-level Conflict

Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – cues: ● Text space in middle panel – register: factual ● US Constitution – evokes: democracy ● art style – evokes: hand drawing; ink on paper ● Ink and Paper don't matter implies: Voting Machines are a good idea – cues: ● Text space in final panel – register: informal – style: rhetorical question

Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – Majority of the cues support this – Elaboration: Democracy ● Ink and Paper don't matter implies: Voting Machines are a good idea – Cues stylistically marked as sarcastic

Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – Majority of the cues support this – Elaboration: Democracy ● Ink and Paper don't matter implies: Voting Machines are a good idea – Cues stylistically marked as sarcastic Emergent reasoning: Voting machines are undemocratic

Picking a viewpoint ● Integrated spaces have – Multiple characters – Each with a belief space ● But spaces are not all equal – Readers know which viewpoint to side with.

Conflict between spaces ● Conflict between characters' belief spaces ● Narrative-level conflict ● Discourse-level conflict

All the tech belongs to ME! I can make them say whatever I want, and Phil can't stop me. Mwaahahahahaha!

* Phil * World * Kaja * Ch1 (Gil) * Drawn Comic * Ch2 (Wooster) * Tech * Ch3 (Agatha) * Muppets * Mad science Domain 1: real lifeDomain 2: Comic Comic Creator1: artist Creator2: tech-person Property: is depiction of narrative world

Without Integration ● Kaja is not a mad scientist ● Kaja has a causal relation to the words on the page (she controls the tech) ● Kaja knows about muppets ● Characters do not know about muppets

Integration ● The words on the page are what characters say. ● Changing the written words changes the characters' speech. Emergent structure: Kaja can make the characters say things. – Which makes her a mad scientist

Irony and Integration in Comics ● Integration? ● Comics as data ● Conflict between spaces ● What do Comics do that's different?

Narrative Simultaneity ● Pictorial and linguistic information are presented “at the same instant” ● Conflicting spaces have to be

Characteristics of the Image ● Backgroundedness ● Truth ● Viewpoint

The importance of Inference ● The irony is not “in” the comic ● It comes from inferences – Made by the reader – Based on minimal cues – Involving the relations between mental spaces. ● Understanding is an active process.

Conclusions ● Multimodal narratives are different from single- modality narratives – Integration of pictorial and linguistic cues – Narrative simultaneity – Backgroundedness – Viewpoint ● Tools from Cognitive Linguistics are useful for analyzing this ● As well as for pure linguistic analysis.

Text Space Interaction Entity: voting machine (category)

Text Space Picture Space Interaction Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity

Text Space Picture Space Interaction Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing

Text Space Picture Space Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing

Text Space Picture Space Entity: voting machine (category) Type: touch-screen Information: Coming into more common usage (inc. here). Entity: voting machine (single item) Type: touch-screen Information: - content: depiction of entity - style: evokes action hand-drawing Generic space blend?

Emergent meaning ● Emergent meaning – Complementary information

Voting Machines: viewpoints ● Ink and Paper matter implies: Voting Machines are a bad idea – cues: ● Text space in middle panel – register: factual ● US Constitution – evokes: democracy ● art style – evokes: hand drawing; ink on paper ● Ink and Paper don't matter implies: Voting Machines are a good idea – cues: ● Text space in final panel

Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network

Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex)

Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex) – Conflict

Emergent meaning ● Emergent meaning – Complementary information ● Mirror Network – Input spaces are structured by a shared frame – But they are not just adding independent information (Not simplex) – Conflict – Conflicting information between spaces – All of which does get projected into the blend.

Voting Machines: Conflict

Base Narrator's Belief Counterfactual Ink and paper matter Voting machines: bad - lead to fraud. Ink and paper don't matter Voting machines: good Narrator Voting machines Fraud

Base Narrator's Belief Counterfactual Ink and paper matter - for self- expression (e.g. this comic) Voting machines: bad - lead to fraud. - undemocratic Ink and paper don't matter Voting machines: good Narrator/artist Voting machines Fraud US Constitution Democracy

Base Narrator's Belief Counterfactual Ink and paper matter - for self- expression (e.g. this comic) Voting machines: bad - lead to fraud. - undemocratic Ink and paper don't matter Voting machines: good Narrator/artist Comic Voting machines Fraud US Constitution Democracy