1. School of Computer Science & Information Technology G6DPMM - Lecture 11 Media Design II – Sound

2. Sound & Multimedia Do not underestimate its importance Do not underestimate its importance Consider the five senses Consider the five senses Smell & Taste Smell & Taste Never implemented, except in research Never implemented, except in research Touch Touch Rarely implemented in mainstream multimedia – starting to become used in games and for specialised applications Rarely implemented in mainstream multimedia – starting to become used in games and for specialised applications Sight & Hearing Sight & Hearing The basis of most multimedia The basis of most multimedia Sound is both underused and badly used Sound is both underused and badly used

3. Sound vs Still Images Sound is active, images are passive Sound is active, images are passive Users are not usually in immediate control of sound Users are not usually in immediate control of sound Sound interrupts other activities Sound interrupts other activities Sound is time-based Sound is time-based May be finite or looping May be finite or looping Images remain until erased Images remain until erased Sound is non-localised Sound is non-localised Hard to annotate Hard to annotate Stereo effects can give only a rough idea of location Stereo effects can give only a rough idea of location Notation is difficult Notation is difficult

4. Purposes of Sound Most ideas developed for early radio Most ideas developed for early radio Background sound Background sound “Noise” - usually ignored, but can create a strong impression of an environment “Noise” - usually ignored, but can create a strong impression of an environment “Atmos” – term from radio f/x “Atmos” – term from radio f/x Pleasant sounds Pleasant sounds Communicate emotions rather than meaning Communicate emotions rather than meaning Natural (e.g. birdsong, wind, sea, rain, waterfall) Natural (e.g. birdsong, wind, sea, rain, waterfall) Artificial (e.g. music) Artificial (e.g. music) Unpleasant sounds Unpleasant sounds Harsh, grating sounds Harsh, grating sounds Effects are often personal, and hard to reproduce Effects are often personal, and hard to reproduce

5. Purposes of Sound (cont.) Emotive sounds Emotive sounds Human voices (e.g. anger, pain, child crying) Human voices (e.g. anger, pain, child crying) Danger (e.g. animal growling, brakes squealing) Danger (e.g. animal growling, brakes squealing) Speech Speech Meaning as in text Meaning as in text More subtleties than text (tone of voice, accent etc.) More subtleties than text (tone of voice, accent etc.) Subtext can be conveyed (e.g. emotions, state of health etc.) Subtext can be conveyed (e.g. emotions, state of health etc.) Silence Silence The absence of sound – can be very powerful The absence of sound – can be very powerful

6. Designing with Sound Sound may have meaning Sound may have meaning Multimedia designers can develop complex vocabularies of sounds Multimedia designers can develop complex vocabularies of sounds Sounds may be coupled with various actions and fulfil various purposes in a multimedia system. Sounds may be coupled with various actions and fulfil various purposes in a multimedia system. Changing states Changing states Provision of user-feedback about a state Provision of user-feedback about a state Not generally good design (states are continuous) Not generally good design (states are continuous) Reinforcement of events Reinforcement of events User events – e.g. clicking on a button User events – e.g. clicking on a button Mirrors real-world association between action and sound Mirrors real-world association between action and sound Should be subtle and appropriate Should be subtle and appropriate

7. Designing with Sound (cont.) Sound as a navigation aid Sound as a navigation aid Most common in virtual environments – especially games Most common in virtual environments – especially games Various sound-cues can provide navigational aids (e.g. mimicking acoustics of a building) Various sound-cues can provide navigational aids (e.g. mimicking acoustics of a building) Sound to reinforce design metaphor Sound to reinforce design metaphor Sounds taken from an object used as a metaphor Sounds taken from an object used as a metaphor Ringing phone, clicking of a camera etc Ringing phone, clicking of a camera etc Can be extremely effective, but avoid overuse Can be extremely effective, but avoid overuse Sound for emotion and realism Sound for emotion and realism Music / atmos etc. Music / atmos etc.

8. Physical aspects of sound Volume Volume Loud sounds are shocking (especially if emotive) Loud sounds are shocking (especially if emotive) Very occasionally, this is useful Very occasionally, this is useful More often this is confusing and distracting – so they are usually to be avoided More often this is confusing and distracting – so they are usually to be avoided Lead-in Lead-in A period of build-up makes a loud sound far less shocking A period of build-up makes a loud sound far less shocking Usually it is good practice to provide warning of loud sounds Usually it is good practice to provide warning of loud sounds Length Length Sound is extremely distracting Sound is extremely distracting Hence – sounds should be as short as possible Hence – sounds should be as short as possible If they accompany visual cues they should be no longer than this If they accompany visual cues they should be no longer than this

9. Sound Sets Sound sets should be consistent Sound sets should be consistent Same issues as icon sets Same issues as icon sets Physical characteristics Physical characteristics Pitch, length, volume etc Pitch, length, volume etc NB exceptions (e.g. warnings, or a long animation) NB exceptions (e.g. warnings, or a long animation) Associations Associations Sound “themes” are commonly used Sound “themes” are commonly used Physical source (e.g. water, metallic etc.) Physical source (e.g. water, metallic etc.) Environment (e.g. factory, forest etc.) Environment (e.g. factory, forest etc.) Combinations of sounds Combinations of sounds Atmos will probably co-exist with feedback sounds Atmos will probably co-exist with feedback sounds Must be distinguishable Must be distinguishable

10. Speech Parallel channels of communication Parallel channels of communication e.g. images illustrated by speech e.g. images illustrated by speech can be very effective or confusing, depending upon how well it is designed. Beware distraction! can be very effective or confusing, depending upon how well it is designed. Beware distraction! Slows interaction Slows interaction Users often wait until the end of speech Users often wait until the end of speech This can be frustrating – make sure it is easy to interrupt (and make sure the user knows this!) This can be frustrating – make sure it is easy to interrupt (and make sure the user knows this!) Interactions can take a long time Interactions can take a long time Context Context Speech is very contextual – the system must provide the context Speech is very contextual – the system must provide the context

11. Speech (cont.) Text rules Text rules Most of the writing guidelines also apply for speech Most of the writing guidelines also apply for speech Choices Choices In exclusive speech systems, choice puts a higher demand onto short-term memory than with graphics. In exclusive speech systems, choice puts a higher demand onto short-term memory than with graphics. Hence choice should be limited (e.g. consider voicemail) Hence choice should be limited (e.g. consider voicemail) Voice Fragments Voice Fragments Dynamic systems often assemble words (or even phonemes) Dynamic systems often assemble words (or even phonemes) This always sounds very artificial (machine-like!) This always sounds very artificial (machine-like!) Scripted speech sounds far better, but is much harder to make dynamic Scripted speech sounds far better, but is much harder to make dynamic

12. Using sound in multimedia Sound is powerful Sound is powerful Emotionally Emotionally Informative Informative Sound may not be available or desirable Sound may not be available or desirable Accessibility issues – consider deaf users Accessibility issues – consider deaf users Hardware (e.g. may be no speakers for public computers) Hardware (e.g. may be no speakers for public computers) Extraneous sound (noisy environments) Extraneous sound (noisy environments) Privacy issues (for public computers) Privacy issues (for public computers) Summary Summary Make good use of sound Make good use of sound Design interfaces that have an alternative to sound Design interfaces that have an alternative to sound