CSE 380 – Computer Game ProgrammingAudio Engineering
Audio Engineering
• For games:– Sound Effects and Music
• Sometimes overlooked
• If not done right, can ruin a game
• If done right, enhances experience– seamless with gameplay– not too obvious
What do you need sound effects for?
• Discernable Game Events– Collisions
• tied to collision response
– Movement• footfalls, rockets,
• tied to sprite state
– Ambiance• background noise
• tied to game state
– Gameplay Events• tied to victory, area triggers, power-ups, death, etc
• to help give your game meaningful play
Recording & Exporting Sounds
• Lots of tools to use
• Recommendation: Audacity– free– easy to learn to use– exports to many formats
• Ex: mp3
– easy to mix sounds• to make complex sound effects
– http://audacity.sourceforge.net/
Sound Properties
• Frequency– measures speed of sound vibrations– measured in cycles per second (Hz)– audible range for human ear: 20Hz – 20,000Hz
• Intensity– amount of air molecules pushed around by sound– pressure applied by sound wave– measured in decibels (dB), a log scale
• sound of whispering 5ft. Away: 20 dB
• sound of thunder: 120dB
Sound File Formats
• WAV– stores raw sound data, is the largest format
– easy on CPU
• MP3/M4A, OGG, & WMA– compressed formats
~10:1 compression over WAV
– negligible quality difference from WAV
– decompression CPU hit
• MIDI– programmed sound
Long vs. Short sound files
• For short sound effects:– store as MP3/OGG/etc.– decompress data in memory at load time– play as needed in decompressed format– heavy on memory, light on CPU
• For long sound effects or music:– store as MP3/OGG/etc.– play in compressed format– again, heavy on CPU usage, light on memory
Music
• What role does music play?– emotion– recognition– themes– fill the void
• Sound & Music:– should compliment the gameplay, not overwhelm it
For the programmer
• Lots of libraries
• Keep game code platform independent
• Options:– FMod and tools (FMod Designer, etc.)
– DirectSound (deprecated)
– XAudio/XACT (soon to be deprecated
– XAudio2/XACT3 (soon part of Windows)• See DirectX Documentation for C++
– lots of other libraries as well (Miles, WWise, etc.)
Microsoft Audio Technologies
For us?
• XACT3– designer controlled content– easy audio tweaking
• XAudio2– API to build sound engine
What’s so great about that?
• The audio designer can focus on audio
• Easy to:– test many different sound effects & songs– choose the right sounds & music– alter/improve sounds & music
• All without having to bother a programmer
XBox 360• 3 CPU cores
– each with 2 hardware threads– Common approach:
• run XAudio2 and XACT on their own threads
• Has real-time XMA decoder hardware– XMA is XBox format for WMA– Windows uses ADPCM
• Windows is more complicated– How many cores?
• GetLogicalProcessorInformation
XAudio Tools/APIs
• XACT• XACT Command-Line Tool• AdpcmEncode• Wave Merge Tool• XMA Command-Line Encoder• XMA2 Command-Line Encoder• xWMA Command-Line Encoder• X3DAudio• XAPO• XAPOFX• XMAEncoder
My Recommendation
• Each team pick an audio engineer for Benchmark 3
• That person will become the XACT3/XAudio2 expert– start with the DirectX Audio manual
• The other team members can continue to concentrate on gameplay
• Go to DirectX install directory
• Go to Samples/
XACT Terminology
• Start by reading about:– wave– wave bank– sound– sound bank– track– events– cues
IXAudio2
• Core of the engine– enumerate available audio devices– configure properties– create & destroy voices– monitor performance
• It’s a COM object– create after CoInitializeEx
• One can create multiple instances– each operates in own thread
Voices
• Used to process, manipulate, and play audio data– Source Voices
• used to stream audio data
– Submix Voices• manipulate audio data
– Mastering Voices• send data to audio hardware
Audio Graph
• A collection of voices
To Play a Sound
1. Initialize XAudio2
2. Load Audio Data– init a WAVEFORMATEX & XAUDIO2_BUFFER
3. Create a Source Voice
4. Pass data to the voice
5. Start the voice
For the Designer
• Determine all the sound effects for your game
• Produce such a list right now
• This is a to-do list of recordings to make
A very good reference to start with
http://www.ultimategameprogramming.com/BeginningDirectX11%5C58958_App_B_rev01_lores.pdf
References• Programming with DirectX : Sound in DirectX – XAudio 2
– http://programming4.us/multimedia/3830.aspx
• Game Coding Complete, 4th Edition
– McShaffry/Graham
• DirectX Software Development Kit Documentation