CSE 380 – Computer Game ProgrammingAudio Engineering

Audio Engineering

• For games:– Sound Effects and Music

• Sometimes overlooked

• If not done right, can ruin a game

• If done right, enhances experience– seamless with gameplay– not too obvious

What do you need sound effects for?

• Discernable Game Events– Collisions

• tied to collision response

– Movement• footfalls, rockets,

• tied to sprite state

– Ambiance• background noise

• tied to game state

– Gameplay Events• tied to victory, area triggers, power-ups, death, etc

• to help give your game meaningful play

Recording & Exporting Sounds

• Lots of tools to use

• Recommendation: Audacity– free– easy to learn to use– exports to many formats

• Ex: mp3

– easy to mix sounds• to make complex sound effects

– http://audacity.sourceforge.net/

Sound Properties

• Frequency– measures speed of sound vibrations– measured in cycles per second (Hz)– audible range for human ear: 20Hz – 20,000Hz

• Intensity– amount of air molecules pushed around by sound– pressure applied by sound wave– measured in decibels (dB), a log scale

• sound of whispering 5ft. Away: 20 dB

• sound of thunder: 120dB

Sound File Formats

• WAV– stores raw sound data, is the largest format

– easy on CPU

• MP3/M4A, OGG, & WMA– compressed formats

~10:1 compression over WAV

– negligible quality difference from WAV

– decompression CPU hit

• MIDI– programmed sound

Long vs. Short sound files

• For short sound effects:– store as MP3/OGG/etc.– decompress data in memory at load time– play as needed in decompressed format– heavy on memory, light on CPU

• For long sound effects or music:– store as MP3/OGG/etc.– play in compressed format– again, heavy on CPU usage, light on memory

Music

• What role does music play?– emotion– recognition– themes– fill the void

• Sound & Music:– should compliment the gameplay, not overwhelm it

For the programmer

• Lots of libraries

• Keep game code platform independent

• Options:– FMod and tools (FMod Designer, etc.)

– DirectSound (deprecated)

– XAudio/XACT (soon to be deprecated

– XAudio2/XACT3 (soon part of Windows)• See DirectX Documentation for C++

– lots of other libraries as well (Miles, WWise, etc.)

Microsoft Audio Technologies

For us?

• XACT3– designer controlled content– easy audio tweaking

• XAudio2– API to build sound engine

What’s so great about that?

• The audio designer can focus on audio

• Easy to:– test many different sound effects & songs– choose the right sounds & music– alter/improve sounds & music

• All without having to bother a programmer

XBox 360• 3 CPU cores

– each with 2 hardware threads– Common approach:

• run XAudio2 and XACT on their own threads

• Has real-time XMA decoder hardware– XMA is XBox format for WMA– Windows uses ADPCM

• Windows is more complicated– How many cores?

• GetLogicalProcessorInformation

XAudio Tools/APIs

• XACT• XACT Command-Line Tool• AdpcmEncode• Wave Merge Tool• XMA Command-Line Encoder• XMA2 Command-Line Encoder• xWMA Command-Line Encoder• X3DAudio• XAPO• XAPOFX• XMAEncoder

My Recommendation

• Each team pick an audio engineer for Benchmark 3

• That person will become the XACT3/XAudio2 expert– start with the DirectX Audio manual

• The other team members can continue to concentrate on gameplay

• Go to DirectX install directory

• Go to Samples/

XACT Terminology

• Start by reading about:– wave– wave bank– sound– sound bank– track– events– cues

IXAudio2

• Core of the engine– enumerate available audio devices– configure properties– create & destroy voices– monitor performance

• It’s a COM object– create after CoInitializeEx

• One can create multiple instances– each operates in own thread

Voices

• Used to process, manipulate, and play audio data– Source Voices

• used to stream audio data

– Submix Voices• manipulate audio data

– Mastering Voices• send data to audio hardware

Audio Graph

• A collection of voices

To Play a Sound

1. Initialize XAudio2

2. Load Audio Data– init a WAVEFORMATEX & XAUDIO2_BUFFER

3. Create a Source Voice

4. Pass data to the voice

5. Start the voice

For the Designer

• Determine all the sound effects for your game

• Produce such a list right now

• This is a to-do list of recordings to make

A very good reference to start with

http://www.ultimategameprogramming.com/BeginningDirectX11%5C58958_App_B_rev01_lores.pdf

References• Programming with DirectX : Sound in DirectX – XAudio 2

– http://programming4.us/multimedia/3830.aspx

• Game Coding Complete, 4th Edition

– McShaffry/Graham

• DirectX Software Development Kit Documentation