SoundscapesSoundscapes

James MartinJames Martin

OverviewOverview

Problem StatementProblem Statement

Proposed SolutionProposed Solution

Solution Created (Modules, Model, Pics)Solution Created (Modules, Model, Pics)

TestingTesting

Looking BackLooking Back

See It in ActionSee It in Action

Q&AQ&A

Problem StatementProblem Statement

Create a music visualization program that Create a music visualization program that renders the visual display in real time and renders the visual display in real time and reacts to the music that is playing. reacts to the music that is playing.

Proposed SolutionProposed Solution

Processing - PlatformProcessing - Platform

Jsyn - SoundJsyn - Sound

Jogl – 3D Visuals in OpenGLJogl – 3D Visuals in OpenGL

L-Systems – Data structure for VisualsL-Systems – Data structure for Visuals

Solution CreatedSolution Created

Processing – PlatformProcessing – Platform

Three ModulesThree Modules InterfaceInterface Sound – ESSSound – ESS Graphics – Uses Spectrum Analysis DataGraphics – Uses Spectrum Analysis Data

System ModelSystem Model

Input

Music file(s) read in from a default

directory

Sound Module

Control audio playback and analysis

Output

Music is output from program through sound

card to speakers

Interface

Acts as input to sound module for user

controls

Visualization Module

Graphics are generated based on input

Output

Graphics are output from program through video card to display

Wav or

MP3

Global Variables

DigitalSignal

Function calls

DigitalSignal

Interface ModuleInterface Module

SimpleSimple Cultural CluesCultural Clues

DragableDragable

Able to be hiddenAble to be hidden

Sound ModuleSound Module

Uses ESS – Plug-in for ProcessingUses ESS – Plug-in for Processing

Load and Play MusicLoad and Play Music

Spectrum AnalysisSpectrum Analysis Uses Fast Fourier Transform AlgorithmUses Fast Fourier Transform Algorithm

Extracts Frequency Data from WaveformExtracts Frequency Data from Waveform

Graphics ModuleGraphics Module

Draws Images to screenDraws Images to screen

Uses Data from Spectrum AnalysisUses Data from Spectrum Analysis

Look at Look at CodeCode…?…?

PicturesPictures

PicturesPictures

PicturesPictures

PicturesPictures

PicturesPictures

PicturesPictures

PicturesPictures

PicturesPictures

TestingTesting

Interface ModuleInterface Module Button FunctionalityButton Functionality

Play and Stop work as describedPlay and Stop work as described DragabilityDragability

Dragable Dragable

Stays intactStays intact

Cannot be dragged off screenCannot be dragged off screen ConcealabilityConcealability

Interface can be hidden with any keystrokeInterface can be hidden with any keystroke

Interface can be shown with any keystrokeInterface can be shown with any keystroke

Interface is not interactive when hiddenInterface is not interactive when hidden

TestingTesting

Sound ModuleSound Module Song PlaybackSong Playback

Proper song playsProper song plays

Song plays to length (< 2:51)Song plays to length (< 2:51)

Song plays smoothSong plays smooth

TestingTesting

Graphics ModuleGraphics Module VisualizationsVisualizations

Look as desiredLook as desired

Play smoothPlay smooth

TestingTesting

SystemSystem Interface & SoundInterface & Sound

Buttons and sound playback are linkedButtons and sound playback are linked Sound & GraphicsSound & Graphics

Visualization matches beatVisualization matches beat Graphics and InterfaceGraphics and Interface

Buttons and visualization are linkedButtons and visualization are linked

Looking BackLooking Back

ConclusionsConclusions

Learn (Positive and Negative)Learn (Positive and Negative)

Do the Same?Do the Same?

Do Differently?Do Differently?

Open ProblemsOpen Problems

Extension(s)Extension(s)

PrototypePrototype

Behold!Behold!