Game Audio Mark Peskir November 14, 2005 ITCS 5010

Overview Terms/Fundamentals Computers and Sound Parameters, Filters, Effects Types of Sound Production Planning For Game Audio

Definitions Sound: compression waves transmitted through a medium Wave: cycle of compression and rarefaction Amplitude: measure of sound wave pressure Coming soon: cool stuff

Definitions Frequency: speed of wave oscillation Measured in Hertz (Hz) Pitch: a particular constant frequency Hertz (Hz): number of oscillations per second decibels (dB): measure of the degree of amplitude Note: a predefined pitch used in music

Common Frequency Ranges Human Hearing: 20Hz – 20kHz Human Voice: 90 – 300 Hz Middle C: 261 Hz

Digital Sound A “unit” of digital sound is called a sample The sampling rate is the number of samples taken per second Bit depth: size of sound sample data See Figure 5.5.2, pg 598

Nyquist Limit A sampling rate can only accurately record sounds of frequencies at half its value 48 kHz sampling rate: can record a 24 kHz max frequency That’s really high, so who cares?!

Common Sampling Rates “CD Audio” quality: 16 bit, 44.1 k IP Voice Chat (BattleCom, TeamSpeak): 8 – 16 bit, 16k – 24k sample rate Burger King drive-thru: ??? High-End sound cards support: 24/96, 24/192 The Tradeoff: Performance vs. Quality

Flow of Tone Production Tone is generated Amplitude (volume) set Filters, effects, etc. added Sound is mixed with other sounds Sound signal sent to speakers

Sound Formats Sample: short recording of live sound, with frequency adjusted to produce other pitches/notes Streaming Audio: a full soundtrack or complete sound effect, loaded into a buffer and played while the rest of sound loads MechWarrior 2: used hybrid CD for music!

Audio Compression The Goal: reduce the amount of data needed to produce quality sound (like JPEG images) Two methods: Bit reduction: reduce bit depth Psychoacoustic Compression: discard audio not generally heard (MP3)

ADSR Envelope Attack, Decay, Sustain, Release Could define a variety of sound aspects (pitch, volume, rate of repetition) What would the envelopes look like for: Revving a car engineRevving a car engine Cymbal crashCymbal crash John Williams choirJohn Williams choir

Other Sound Parameters Pan: the “left/right” location of the sound Volume: ratio of amplitude played Pitch/Frequency: can be assigned to override a sound’s actual pitch 3DGS: sound mySound = ; snd_tune(mySound, 50, 100, -100); play mySound at 50% vol, 100% freq, left

Pass Filters Low Pass Filter: only allow sounds below a certain frequency High Pass Filter: only allow sounds above a certain frequency Band Pass Filter: only allow sounds in a given frequency range used for cutting “highs” and/or “lows”

3D Sound Methods Head Relative Transfer Function (HRTF) used in stereo speaker config dynamically changes balance for L/R reduces high frequencies if behind Today: Dolby Surround Sound – 5.1, 6.1, 7.2!

Environmental Effects Echo: early reflections Reverberation (reverb): late reflections Diffusion: the “breaking up” of a sound reflecting off of a rough surface (carpet) Obstruction: the blocking of a sound’s direct path to the ear Occlusion: high freqs being blocked by a material, while low freqs pass through (car bass with windows up, underwater)

Environmental Effects Two current standards: I3DL2: established by the Interactive Audio Special Interest Group (IASIG) EAX: created by Creative Labs for SoundBlaster audio cards – multiple versions exist (1.0, 2.0, 3.0) Difference in sound: ???

Half Life 2: Soundscape DSP

Audio Systems MIDI: Musical Instrument Digital Interface file contains sound “instructions” metaphor: sheet music for a computer uses preloaded sound banks Digital Audio: actual recording generally much better sound quality more “real” downside: much bigger file sizes

Audio Systems MIDI: Digital Audio:

Advanced Audio Programming Programmers should not have to deal with audio!Programmers should not have to deal with audio! Audio scripts handle production of sound (HL2 DSP presets) Common problems: sound variation, repetition, looping, ambience

Advanced Music Programming HALO 2: Quantum Audio – music has starts, ends, middles, transitions Interactive Music: “musical boxcars”

Voice Interaction Phoneme: individual voice sound Lip Synching: matching phonemes to mouth animation Voice Playback: generating words using phonemes Voice Recognition: decoding voice input to understood text of some kind Dragon Naturally Speaking

Half Life 2: Phoneme Extractor

When you’re Project Lead: Create an audio development process Pre-production:Production: audio engineSFX workflow game styleMusic workflow file systemVO workflow Foley workflow

When you’re Project Lead: Create an audio asset list “How many sounds can this make?” “How often will we hear this?” “How much music can/should we use?” Excel spreadsheets are commonly used

When you’re Project Lead: Understand audio budget needs Audio Designer Audio engineers Composer Voice actors Recording equipment Audio software Audio hardware Music gear Licenses (sounds, middleware) Field Recording Hired musicians Music composition Canned music, SFX, foley SFX research Sound processors Sound file storage

When you’re Project Lead: Integrate your audio team Audio team works with art Audio team works with animation Audio team works with design Audio team works with story writers In short: COMMUNICATE!

The End Thanks for listening