sound, image, text Lecture One Wed, 23rd July 2008

KMB107: overview Weeks 1-3: How do sound, image & text function across a range of media key concepts in listening & basic analytical techniques for audiovisual media Weeks 5-9: Creation of meaning through our interactions with sound, image & text narrative, identity and imagination as constructed through sound, performance, film, other media Weeks 10-12: guest lectures and audiovisual screenings / performances Week 13: review

What is sound? –A sensation caused in the ear by the vibration of the surrounding air or other medium –What is or may be heard –An idea or impression conveyed in words Source: The Australian Concise Oxford Dictionary, fourth edition (2004)

auditive phenomena “ By sound I mean sounds, voices, and aurality - all that might fall within or touch on auditive phenomena, whether this involves actual sonic or auditive events or ideas about sound or listening; sounds actually heard or heard in myth, idea, or implication; sounds heard by everyone or imagined by one person alone; or sounds as they fuse with the sensorium as a whole.” From Douglas Kahn: Noise, Water, Meat (1999, p3)

Our definition of SOUND … includes natural sound, voice, speech, dialogue, music, effects, soundscapes, atmospheres as well as musical thoughts and ideas about sound It refers to: –PERCEIVED SOUNDS (ie the things we hear) and –IMAGINED SOUNDS, (ie our mental representations of sound)

defining image –A representation / the external form of an object –The character or reputation of a person or thing as generally perceived –An optical appearance... refracted through a lens –A person or thing that closely resembles another –An idea or concept / an imagination…

image Our definition of IMAGE includes: –pictures, written text, visual objects, graphic representations etc (ie all things we can see) –and ALSO mental representations, ideas or concepts (ie the things we imagine). This relates closely to our definition of sound that includes all the sounds we can perceive and imagine.

defining text From Latin, texture, context, from texere to weave –the original words and form of a written work –the printed score of a musical composition –a source of information or authority –Anything considered as an object to be examined, explicated, or deconstructed Source: Mirriam-Webster Online Dictionary:

listening strategies When we listen to sound, we adopt many techniques that help make sense of what we hear. Listening strategies rely on our EXPERIENCE of the world, where we make judgements about the identity and meaning of a sound based on previous experience: What is making that sound? What is it trying to communicate? Other listening strategies rely on our ability to IMAGINE and create NEW MEANINGS, where we allow sounds become metaphors for other things.

modes of listening French theorist Michel Chion suggests that there are at least 3 listening modes: CAUSAL SEMANTIC REDUCED The three modes refer to a sound’s source, meaning and content. Source - Michel Chion (1994). Audio-vision

Causal listening –listening to the SOURCE OF THE SOUND. –can be precise or general –often we identify indices (categories): mechanical, human, environmental sounds –a sound may have more that one source –causal listening is related to our experience of the world. When we hear a sound we try to associate it with a memory or prior knowledge of that sound. Our ability to name that sound is often dependent on our previous experience. –causal listening can be manipulated, in film, for example, to make us believe that the sound of an object is ‘real’

example Can you identify the source of the sound in the following example...

Semantic listening –The purpose of semantic listening is to gain information about what is communicated in the sound. –a code or language that conveys information: spoken language, –we often use causal and semantic modes simultaneously, eg listening to the sound of my voice and what I’m saying

example What are you listening for in this piece of music? What do you think it means?

Reduced listening –Pierre Schaeffer: acousmatic listening –listening that focuses on the characteristics of the sound itself, ie a sound’s actual content (pitch, duration, colour, intensity etc) –listening to a sound independent of its cause or meaning –the sound (speech, noise, instrumental music etc) as an object or event to be observed rather than a vehicle for something else

example What characteristics of these sounds do you hear? Can you listen without listening for the source or meaning of the sounds? _on_a_.html

audiovision We can learn more about a sound or an image by asking some simple questions. What do I hear? What do I see? Then, we enrich our knowledge of sound image relationships, by asking even more questions: What do I hear of what I see? What do I see of what I hear? What do I imagine when I see or hear?

The audiovisual contract “The audiovisual relationship is not natural but rather a sort of symbolic pact to which the audio-spectator agrees to forget that sound is coming from loudspeakers and picture from the screen. The audio- spectator considers the elements of sound and image to be participating in one the same entity or world. The result of the audio-visual contract is that one perception influences the other and transforms it. So that... We never see the same thing when we also hear. We don't hear the same thing when we see as well.” (Edited excerpt: Michel Chion, Audio-Vision, Source:

introducing audiovisual analysis What do I hear? Methods of listening: reduced: the quality of the sound causal: the source of the sound semantic: the meaning of the sound What do I see? Methods of observation: masking: cutting out the screen image or sound from an excerpt to see the inherent qualities in each.This is generally done several times, eg what the sequence with both image & sound, then mask the image to hear the soundtrack alone, then cut the sound to see the image alone, then watch the sequence again with image & sound together.

preparation for tutorials next week Spend 5 mins listening to an everyday environment. What do you hear? What do you see? How are you listening to this environment? Does your listening change in different environments? and/ or bring in a short excerpt of a work (a song, a video etc) that you feel relates to the themes of this lecture. Has the way you listen to this work changed? You are encouraged to bring in your own creative work. Readings: Chion, Michel. (1994). Chapter 2: The Three Listening Modes in Chion, Michel, Audio vision : sound on screen, New York: Columbia University Press, pp (On CMD) Kahn, Douglas. “Noise Water Meat: A history of sound in the arts”. Cambridge, Mass. : MIT Press, c1999. pp1-13 & (QUT Lib e- book)