audio engine programming

59
Audio Engine Programming Lecture 4 Introduction to FMOD programmers API

Upload: xanti

Post on 04-Feb-2016

47 views

Category:

Documents


0 download

DESCRIPTION

Audio Engine Programming. Lecture 4 Introduction to FMOD programmers API. 3D Sound. Sounds have position and velocity There is a listener component Relationship between the two Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities) - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Audio Engine Programming

Audio Engine Programming

Lecture 4

Introduction to FMOD programmers API

Page 2: Audio Engine Programming

3D Sound

Sounds have position and velocity There is a listener component Relationship between the two

Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities)

Lots of “psycho-acoustic” options

Page 3: Audio Engine Programming

Fmod audio development

There are three primary components: The Sound Designer The Sandbox The API

We’ll use all three Start with Sound Designer

Page 4: Audio Engine Programming

Data Driven vs. Low Level API Low-Level API

Everything is done using programming The programmer has complete control

The benefits are lower memory overhead

Page 5: Audio Engine Programming

Continued…

Data-Driven API Takes assets from FMOD Designer

Sound Designer has control over the audio Integrates them into the code using the

EventSystem object

Page 6: Audio Engine Programming

Key Features

The data-driven approach of the EventSystem: Support’s event logic

Allows the Sound Designer to fine-tune or ‘tweak’ the final mix without the need for Programmer intervention

Creates a logical separate of concerns for the Programmer and Sound Designer

Page 7: Audio Engine Programming

Overview of event model

FMOD Low Level

FMOD EventSystem

FMOD FMOD EXEX

FEVevent data

FSBaudio data

Page 8: Audio Engine Programming

Assets from the sound designer

The build process creates the following assets: .FEV file - contains the event meta data .FSB file(s) - contains the audio data The ‘Programmer report ‘- a human readable list of

event names, sound definitions [Optional] A ‘header’ file - A source code header file [Optional] Cache files - Files used to speed up subsequent

builds (These files are not required by the Programmer)

Page 9: Audio Engine Programming

Linking the library

Include the EventSystem header file #include "fmod_event.hpp“

Declare a pointer to an EventSystem FMOD::EventSystem *eventsystem;

Create the object (allocate memory) FMOD::EventSystem_Create(&eventsystem);

Initialize the object eventsystem->init(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL);

Page 10: Audio Engine Programming

Loading files

Set the media path (location of files) eventsystem->setMediaPath((char *)MEDIA_PATH);

Load the .FEV file eventsystem->load("examples.fev", 0, 0);

Load an event group eventsystem->getGroup("examples/examples", FMOD_EVENT_DEFAULT,

&eventgroup);

Load an individual event eventgroup->getEvent(“car", FMOD_EVENT_DEFAULT, &car);

Page 11: Audio Engine Programming

Loading Audio

The required .FSB file(s) are listed in .FEV file Both the .FEV and .FSB files should be located in the

directory specified with setMediaPath() The location of audio within the .FSB files is

stored in the .FEV FMOD will load the audio data from the .FSB

files as required for an event, automatically

Page 12: Audio Engine Programming

Loading Options

FMOD_EVENT_DEFAULT Specifies default loading behaviour Event data for the whole group is NOT cached Loading will block any other task until complete.

FMOD_EVENT_NONBLOCKING FMOD will use a thread to load the data Use Event::getState to find out when loading is

complete.

Page 13: Audio Engine Programming

Continued...

FMOD_EVENT_INFOONLY Does not allocate instances or load data Creates a ‘handle’ that allows the programmer to

get information from the event

Page 14: Audio Engine Programming

EVENT Playback

Each events have a number of methods and attributes

To start playback of an event: example->start();

To stop an event: Example->stop();

Page 15: Audio Engine Programming

Updating

Updating the event system should be called once per 'game' tick (or once per frame) eventsystem->update();

This allows the FMOD Engine to refresh data such as: the position of 3D sounds changes to volume, pitch, effects as well as internal mechanisms

Page 16: Audio Engine Programming

Demonstration

Simple Event Demonstrates the necessary code to start

playback of an event

Page 17: Audio Engine Programming

PROVIDING Parameter DATA

Get the parameter called ‘test’ example->getParameter(“test", &test); test will be a EventParameter object

To get the range of the parameter test->getRange(&rangemin, &rangemax);

To set the value of the parameter test->setValue(value);

Page 18: Audio Engine Programming

Demonstration Parameter

Demonstrates the necessary code to provide parameter data to an event

Page 19: Audio Engine Programming

PROVIDING a KEY-OFF If an event parameter is currently sustaining

on a ‘sustain point’,  triggering a keyoff will release it and allow the parameter to continue test->keyoff()

Page 20: Audio Engine Programming

Demonstration

Keyoff Demonstrating the necessary code to leave a

sustain point

Page 21: Audio Engine Programming

Selecting sounds at run-time

The sound to be played can be selected at runtime This is required for situations such as ‘colour

commentary’ in sport games or other situations that require dynamic dialogue

The are two methods: Programmer Sounds Programmer Selected

Page 22: Audio Engine Programming

Demonstration

Programmer Sound When a sound definition containing a ‘ProgrammerSound’ is to

be played, a callback is made The code must then provide a sound object at runtime just

before it's played

Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSound’ method

Page 23: Audio Engine Programming

Demonstration

Programmer Selected When a sound definition with the playmode

‘ProgrammerSelected’ is to be played, a callback is made The code must then provide the index number at runtime just

before it's played The index number identifies which file within the sound definition

is to be played

Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSelected’ method

Page 24: Audio Engine Programming

Closing the Event System

Unload event data eventsystem->unload();

Release audio data in memory fsb->release();

Release memory and close eventsystem memory eventsystem->release();

Page 25: Audio Engine Programming

Setting up a Project

Take the advice of the video tutorials

Have a separate folder Copy sounds into a “sounds” directory

Keeps the safe Can have sub-directories

Create an “output” directory

Page 26: Audio Engine Programming

Designer Interface

Can delete this if you want, but need to have atleast one group

Page 27: Audio Engine Programming

Events

Used to define sound properties for an event Can be comprised of

One sound Several sounds randomly chosen Monophonic vs Polyphonic Can randomize pitch and attenuation

Events can be Simple Multi-track

Page 28: Audio Engine Programming

A Simple Event

Page 29: Audio Engine Programming

Granular Sounds

These sounds aren’t looping

Allows for a sounds to occur between time ranges

Allows for polyphony

Page 30: Audio Engine Programming

Event Options

Can vary attenuation (dropoff with distance)

Can vary pitch Creates a sound

considerably smaller than a “soundtrack”

Plays forever and is always random!

Page 31: Audio Engine Programming

Sound Definitions

Required for multi-track events

Page 32: Audio Engine Programming

Multi-track Events

Comprised of sound defs in layers

Page 33: Audio Engine Programming

Multi-track Events

Careful when adding sounds Choose sound def Ambient noise is looping Other sounds are granular,

so choose “Oneshot” Have parameters Not based on time!

Page 34: Audio Engine Programming

Multi-track Events

Can cross-fade and set fade out time

Page 35: Audio Engine Programming

Effects

Page 36: Audio Engine Programming

Parameter Properties

Can define range and velocity (to simulate time)

Page 37: Audio Engine Programming

Engine Designer

fmod can specifically work with engine sounds Need for Speed 2

Based on the “load” of the engine Right-click on a

sound instance->properties Auto-pitch

Window->fmod Engine Designer

Page 38: Audio Engine Programming

The Build Process

Know which platform you’re targeting Changes are applied only to that platform

Project->Clean, Project->Build

Page 39: Audio Engine Programming

Interactive Music

Comprised of several short segments of music Intro music Darkening or discovery Fighting/intense fighting Release

Before you begin, you must know Tempo of music (beats per minute – bpm) Beats per measure (time signature)

Page 40: Audio Engine Programming

Cues

Page 41: Audio Engine Programming

Themes and Auditioning

Page 42: Audio Engine Programming

Transitioning via Parameters

Page 43: Audio Engine Programming

Other Things

Supports deployment of multiple languages Can deploy different builds based on

Platform Language

Page 44: Audio Engine Programming

A Look at the API

Basic API Designer API

Built on basic API Can read .fev files

Written for C/C++ #include <fmod.h>// C #include <fmod.hpp> // C++

Read the documentation fmodex.chm

Page 45: Audio Engine Programming

API Parts A System is the fmod engine A Sound is the raw resource

2D uses FMOD_2D 3D uses FMOD_3D

result = system->createSound(“music.wav", FMOD_2D, 0, &sound);

Page 46: Audio Engine Programming

API Parts

A Channel is an instance of a Sound Each time you play a sound, you get a new

channel Channels can start out paused You can set the

Volume (0.0f – 1.0f) Pan (-1.0f – 1.0f) Frequency (in Hz as a float)

Always use FMOD_CHANNEL_FREE to pick for you

Page 47: Audio Engine Programming

#include <iostream>#include <fmod.hpp>#include <fmod_errors.h>

using namespace std;using namespace FMOD;

void main () {FMOD_RESULT result;System* system;// Create the fmod system. We only need one of theseresult = System_Create(&system);// Specify a max number of simultaneous channelsresult = system->init(100, FMOD_INIT_NORMAL, 0);

Sound* sound;// Decompress the entire mp3 into 16-bit PCM in memoryresult = system->createSound("winning.mp3", FMOD_DEFAULT, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel; // Used for setting volume, pan, pausing...// You have the option of passing that channel as the last parameterresult = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency (44100.0f);

if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}while (true) {

cout << "1"; system->update(); // Now required in fmod}

}

Page 48: Audio Engine Programming

Virtual Voices Supports “virtual voices” when hardware is

limited Set from System::init() System::getCPUUsage() Try to keep < 1000

May need to set the priority of a channel if it’s important

channel->isVirtual()

Page 49: Audio Engine Programming

3D Sound in fmod

Sound attenuation Logarithmic Set the mindistance of a sound channel to start

attenuation Bee = 0.1f Jet = 50.0f

Leave max distance alone (default is 10,000) Set 3D settings with System::set3DSettings()

Doppler Distance factor (in cm, m, feet) Rolloff scale (attenuation models)

Page 50: Audio Engine Programming

3D Sound

fmod uses a left-handed coordinate systemresult=system->init(100,

(FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D),0);

With velocity, you must pass it as metres per second

velx = (posx-lastposx) * 1000 / timedelta;

vel = 0.1 * 1000 / 16.67 = 6 meters per second

vel = 0.2 * 1000 / 33.33 = 6 meters per second

Page 51: Audio Engine Programming

void main () {FMOD_RESULT result;FMOD_VECTOR soundPos, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp;

System* system;result = System_Create(&system);result = system->init(100, (FMOD_MODE)(FMOD_3D), 0);

int numHC = 0;result = system->getHardwareChannels(&numHC);cout << "Hardware channels: " << numHC << endl;

Sound* sound;result = system->createSound(“train.mp3", FMOD_3D, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel;result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency(44100.0f); if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}float dsp, stream, geometry, update, total;soundPos.y = soundVel.x = soundVel.y = 0.0f;soundPos.z = -100.0f; soundPos.x = 5.0f;soundVel.z = 6.0f;channel->set3DMinMaxDistance(10, 10000);listenerPos.x = listenerPos.y = listenerPos.z = 0.0f;listenerVel.x = listenerVel.y = listenerVel.z = 0.0f;listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f;listenerForward.z = listenerUp.y = 1.0f;system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp);

while (true) {system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total);cout << total << endl; soundPos.z+=0.01f;channel->set3DAttributes(&soundPos, &soundVel);

}}

Page 52: Audio Engine Programming

Interfacing with Sound Designer

You should have created .fev file (Designer file – event data) .fsb file (raw audio data) .txt file (describes the events and parameters)

May want to start by creating a helper function

void checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout << "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }}

Page 53: Audio Engine Programming

Imagine That This is our Project

Page 54: Audio Engine Programming

#include <iostream>#include <fmod.hpp>#include <fmod_event.hpp>#include <fmod_errors.h>using namespace std;using namespace FMOD;

void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar;

FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0); result = eventSystem->getGroup("fmodTest/beeps", false, &eventGroup); result = eventGroup->getEvent("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer", &eventPar); myEvent->start();

float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue(0.1f); while (true){

cout << currentParVal << endl;eventPar->getValue(&currentParVal);currentParVal+=dir;eventPar->setValue(currentParVal);if ((currentParVal >= 1.0)||(currentParVal <= 0.0)) {

dir = -dir;}

}}

Page 55: Audio Engine Programming

Interactive Music

Page 56: Audio Engine Programming

void main () {EventSystem* eventSystem = NULL;MusicSystem* musicSystem = NULL;MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt;FMOD_MUSIC_ITERATOR cueIter;FMOD_MUSIC_ITERATOR paramIter;FMOD_MUSIC_ID intensityParID = -1.0f;

FMOD_RESULT result = EventSystem_Create(&eventSystem);result = eventSystem->init(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL);result = eventSystem->load("acid_test.fev", 0, 0);result = eventSystem->getMusicSystem(&musicSystem);musicSystem->setVolume(1.0f);result = musicSystem->getCues(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fightPrompt);result = musicSystem->getNextCue(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fight2Prompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &introPrompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &releasePrompt);

musicSystem->getParameters(&paramIter);intensityParID = paramIter.value->id;cout << "paramIter.value->name is " << paramIter.value->name << endl;musicSystem->setParameterValue(intensityParID, 7.5f);result = introPrompt->begin();

while (true){eventSystem->update();

}}

Page 57: Audio Engine Programming

Other Things You Should Look Into…

The 3D Reverb API Asynchronously loading data (by default) Memory management for non-PC platforms

FMOD::Memory_Initialize() // fix size of fmod Using compressed samples

Built-in DSP

Page 58: Audio Engine Programming

Configuration Notes

You’ll be working with Header files

C/C++ -> General->Additional Include Directories Lib files

Linker->General->Additional Library Directories Linker->General->Additional Dependencies

DLLs (which you should put into Windows/System32)

There’s also a FMOD.NET site…

Page 59: Audio Engine Programming

Configuration Notes

You have the option of 64-bit and 32-bit libraries You might have to create a new platform