dynamic sound for ios apps and games

49
Dynamic Audio For Apps and Games

Upload: tack-mobile

Post on 02-Jun-2015

2.271 views

Category:

Technology


0 download

DESCRIPTION

A talk given at http://360idev.com/ topic, covering sound design, Pure Data, libpd and examples integrating Pure Data into iOS.

TRANSCRIPT

Page 1: Dynamic Sound for iOS Apps and Games

Dynamic Audio For Apps and Games

Page 2: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

About Me

• @thillerson, +thillerson (if you’re into that)

• Developer at Tack Mobile (tackmobile.com)

• iOS, Android, and Mobile Web

• Rails, Node, maybe Elixir (one day)

Page 3: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Rawk Star

About Me

Page 4: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

About Me

• Bass

• Electronic Music

• Some day I hope to play this thing --------->

Page 5: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Design Work: Start

• Alarm App

• Alarm Sounds

• App Store: “Start by Tack”

Page 6: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Design Work: Adrift

• Puzzle Game - “Adrift by Tack” in App Store

• Background Music

• Sound Effects

Page 7: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Programming Sound

• In Beta Soon*(*this week)

• http://pragprog.com/

• What we’re talking about today...

• ... and much more

Page 8: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

What We’re Going To Do Today

• Answer: What Is Sound Design?

• Discuss UX Principles of Sound Design

• Introduce Pure Data

• Introduce libpd

• Tour Two iOS Apps Embedding Pd

Page 9: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

When We Are Done, You Will

• Understand Sound Design

• Understand Procedural Audio

• Know what Pd is, and how libpd makes it awesome

• Be excited about Dynamic Audio in your apps and games

Page 10: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

What is Sound Design?

Page 11: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

A Sound Designer

Ben Burtt

Page 12: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

A Sound Designer...

• Conceptualizes

• Captures or Designs

• Records

• Engineers

• Produces

• ... the sound for an experience

Page 13: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Designer Skills

• Understand Sound

• Intuitive vs. Theoretical

• Music vs. Sound

• Audio Engineering

Page 14: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Design in Mobile Apps

• Developers/Designers wear lots of hats

• Control

• Efficiency

• Asset size

Page 15: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Design Principles

Page 16: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

A Sound Designer’s Goals

• Audible Feedback

• Fulfill Expectations

• Immersion

• Emotional Response

Page 17: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound Design and Good UX

• Your app doesn’t need sound

• Expect to be turned off

• Don’t make me listen to you

• Listen on lots of devices/environments

• Engineer for small speakers

• Make your sound mean something!

Page 18: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Always

• Test with phone calls

• Test while playing music

• Have fallbacks - e.g. No music, just effects

• Provide control to the user

Page 19: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

A Good Example: Clear

• Realmac Software

• Josh Mobley

• Meaningful Sound

• demo

Page 20: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Sound

Page 21: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Longitudinal Waves

Page 22: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Of Pressure

Page 23: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Through A Medium

Page 24: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Transverse Wave

Longitudinal Wave

Page 25: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

• Longitudinal Waves

• Of Pressure

• Through a Medium

Sound

Page 26: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Properties of Waves

• A - Wavelength (Distance), Period (Time)

• B - Amplitude

• C - Frequency - 1/Period (measured in Hertz)

A

B

B

C 1 2 3

Page 27: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Fourier Series

• Based on his work ------->

• Periodic signals composedof simple oscillatingfunctions

• Sine/Cosine waves arefundamental

Page 28: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Oscillators

• Something that moves according to a periodic function

• A physical body

• An electronic device

Page 29: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Basic Electronic Sound

oscillator

voltage

magnet

pressurewaves

speaker

Page 30: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Digital Audio

• Stream of Numbers

• Signal

• Frequency and Amplitude

• DAC - Digital to Analog Converter

Page 31: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Dynamic Audio

Page 32: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Procedural Audio

• Functions

• Objects

• Controls and Components

• Produce a Signal

• “Building Sound from First Principles”

Page 33: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Pure Data

• Procedural Audio

• Visual Programming Environment

• Free and Open Source

• pure-data.info

Page 34: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Pure Data 101

Page 35: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Signals and Values

• Signals are streams of numbers, usually audio data

• Values are numbers or symbols

Page 36: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

• Pd’s class

• or, kind of like a command line tool

Objects

osc~ 440

signal inlet inlet

signal outlet

class name arguments

Page 37: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Hot vs Cold Inlets

• Hot inlets make objects immediately act

• Cold inlets usually store a value

• Execution is right to left

Page 38: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

• Signal connections are thick

• Others are thin

Connections

osc~ 440

*~

dac~

1

Page 39: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Messages

• Used for sending values without a connection

• [send] and [receive] objects

• Message boxes

Page 40: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

libpd

• http://libpd.cc

• Peter Brinkmann - Google Engineer

• Enables you to embed an instance of Pd in your app

• iOS and Android

Page 41: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Examples

Page 42: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Source Code

• github.com/thillerson/dynamic-sound-ios

• Examples of capabilities, not necessarily the best architecture

• Goal: understand capabilities, have a place to start

Page 43: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Example 1: A Simple Keyboard

• Sine wave oscillator

• Single voice

• Note on/off

Page 44: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Example 1: A Simple Keyboard Patch

osc~

dac~

receive midinote

mtof receive gate

line~

*~

;

midinote 60;

gate 1;

;

gate 0$1 1

print

Page 45: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Example 2: A Game

• Top down perspective space shooter

• Code by Lucas Jordanfrom iOS 5 Game Dev ->

• Now with sound!

Page 46: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Example 2: A Game Patchload a square waveat startup

Thruster

noise~

*~

samphold~

dac~

hip~ 200

phasor~ 800

line~

$1 200

*~ 0.1

receive thruster

BulletShip_HitAsteroid_Hit

samphold~

tabosc4~ square

dac~

line~

phasor~ 8000

tabosc4~ square

line~

unpack f f

0 $1

tabosc4~ square

line~

unpack f f

0 $1

hip~ 1000

unpack f f

0 $1

*~ 0.5

receive asteroid receive ship receive bullet

8-bit-ifythese sounds

cut low frequencies

commentcomment

Test_Messages;bullet 25 200;

;thruster 0

;thruster 1

pd wavesetup

;ship 1000 50;

;asteroid 200 100;

Page 47: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Further Reading

• puredata.info && libpd.cc

• youtube.com/user/cheetomoskeeto

• http://designingsound.org

• mitpress.mit.edu/books/designing-sound

Page 48: Dynamic Sound for iOS Apps and Games

360 iDev tackmobile.com

Questions

Page 49: Dynamic Sound for iOS Apps and Games

Thank you!Dynamic Audio • Tony Hillerson

• We’re Hiring!

[email protected]

• Excellent Team

• Awesome Projects

• Great Office