4.3. S OUND Use of sound within games

In lecture exploration of answers to frequently asked student questions

Overview of the different uses of sound within games

The use of sound within games includes: Music - establishing scene mood, e.g. uplifting, urgent, peaceful, etc. Sound effects - dependent sounds tied to in-game events, e.g. collecting power-ups, jumping, etc. Dialog – recorded/generated speech used within cut-scenes (furthering plot) or in-game (improving realism, emersion, etc.).

Sound plays a very important role in setting the mood for the game as well as, increasingly, how engaging the game is. For example: A pinball game without any sound (ball hits, accumulating score) will appear dull and lifeless (the player will feel more detached). In a 3D game, character movement without footfall sounds will reduce the believability/emersion (consciously, or not, the player will feel movement is not ‘right’).

Basic principles behind managing sound playback

Sound hardware typically supports a maximum number of audio output channels (simultaneous sounds). Additionally, sound APIs may support a maximum number of output channels. The number of channels defines the maximum number of simultaneously playing sounds. Aside: “The Java Sound API provides support for the capture, processing, and rendering of audio and MIDI data. Java Sound … is a 16-bit, 32-channel audio rendering and MIDI controlled sound synthesis engine.”

Background music Foot falls Water sounds Constraint: You cannot have more individual sounds playing than you have channels. People talking Actually, 32+ channels is ample for most games…

What happens if every projectile from a ship is to make a sound when it hits something? This could be a problem…

Question: Given a finite number of channels, how can we deal with a potentially large number of simultaneous sounds? Start 10 mins9 mins8 mins7 mins6 mins5 mins4 mins3 mins2 mins 1 min 30 secFinished

Approach 1: Allocate a maximum number of sound channels that can be used for a particular type of sound. Develop a manager that dynamically monitors channel usage (imposing sound type channel restriction if all channels are in use) Explosions (max 6 channels) Projectile hits (max 10 channels) Total 32 channels Issues: When the playback limit is reached, how are new requests handled? Are they ignored? Is playback of the longest playing sound in that class cut short? If other classes have exceeded their maximum, which one will contract?

Approach 2: As sound waveforms are additive, software/hardware can be used to combine separate waveforms to be then played via a single output channel. This effectively provides an unlimited number of channels, but with the additional cost of combining the individual sounds. Aside: Approach 1 and 2 can be successfully combined together.

Overview of sound usage in Java

Sound playback is achieved using the Java sound API, using: SourceDataLine – referring to a source of data for a mixer (i.e. output from a program) TargetDataLine – referring to a source of output from a mixer Clip – a loaded (not streamed) sound file. LineListener –attached to lines to monitor LineEvents (i.e. audio opened, closed, started, or stopped)

See the following classes within the core code repository: ● SoundAsset ● SoundAssetClip ● SoundAssetAssembly To do: Explore this

Overview of sound usage in XNA

XNA audio is created using the Cross- Platform Audio Creation Tool (XACT). XACT supports WAV, AIFF, XMA formats and stero or 5.1 output XACT provides excellent audio integration features and supports streaming support and audio event notification Aside: To play MP3s (and similar formats) it will be necessary to convert them into a WAV format). This applies to XNA and Java

Audio within XACT is structured as follows: Wave Banks – a collection of wave files Sound Banks – instructions and cues for the playback of sounds in a wave bank Cues – triggering audio playback from a sound bank. A cue can contain a sequence of ordered sounds to play, or a list from which to select a random sound to play. audioEngine = new AudioEngine (audioParameters); waveBank = new WaveBank soundBank = new SoundBank soundBank. PlayCue (Name); Cue cue = soundBank. GetCue (Name); cue. Play (); To do: Explore this

To do: Think about the role of sound in your game (how important do you want it to be?) Continue developing visual/aural design of game Continue writing code to display graphics or play sound Today we explored: Overview of the use of sound within games and how sounds can be managed Overview of Java/XNA sound classes