M I C H A E L M U S I C K

T I M B R A L H A U N T I N G S

S O N I C S P A C E N O . 6

( 2014 )

T I M B R A L H A U N T I N G S ( Sonic Space No. 6 – Iteration No. 1) Program Notes Timbral Hauntings (Sonic Space No. 6 – Iteration No. 1) is an interactive installation work that borrows ideas from soundscape analysis and the idea that echoes from the past effect the present and future. Timbral Hauntings ‘listens’ to the soundscape of the space, analyzing the timbre of each acoustic event; cataloguing the eight most commonly occurring timbral phrases throughout the life of the system. As a critical number of events are collected, the system selects the most commonly occurring timbral phrase. This phrase is used to shape the timbre of the ‘present’ in the hauntings of the past. At the same time the selected phrase from the past is analyzed for near timbral matches from the present. When a match occurs, those near moments from the past are played back in an attempt to influence the future in repeating the past. As the composition progresses, new phrases from the past are selected, allowing for the constant progression in the emergent composition. Participants to the either space are welcome to wander the space, simply sit and listen, or contribute to the composition, in any way they feel comfortable, including playing the instruments laid out. Timbral Hauntings is part of Michael Musick’s Sonic Spaces Project.

Technical Description Equipment Provided By Host:

1. A 6.1 audio system – With audio cables to run from control station. a. 6 full range studio speakers

i. 6 Power cables, 1 for each speaker, long enough to reach power drops ii. Suspension mounts from the ceiling for each speaker, OR stands that would

put the speakers at head height (approximately 5 feet 8 inches for the tweater.)

b. 1 or 2 Subwoofers (depending on the needs of the space you assign me to) i. Power cable for subwoofer, long enough to reach power drop

ii. The subwoofer/s will sit on the floor, in the ring of speakers. c. 7 balanced audio cables. (Likely XLR). Long enough to safely and discreetly run

from the control station/room where the computer and audio interface is to each of the powered speakers.

d. 7 adapter Male Balanced TRS ¼” -> XLR Male. The audio interface I will bring has ¼” outputs.

e. 2. 2-4 microphone stands to put microphones on. OR permission to hang/suspend

microphones from the ceiling. a. Microphone cables to reach mics from control station or control room.

3. 3 pedestals to put “toy instruments” on which visitors/participants may use. OR permission to hang/suspend small toy instruments.

4. 2-4 benches for participants/visitors to sit on. 5. If this is set up from the floor, versus suspended from the ceiling, I will need gaffe tape to

secure cables. 6. A secure space for the control station. Either a near by room or corner of the presentation

room where equipment could be locked to a physical structure. Equipment Provided by Composer

1. 2-4 microphones 2. Toy instruments for participants/visitors to engage the system with. 3. Computer 4. Multi-Channel Audio Hardware interface, with balanced ¼” outputs for the 6.1 system.

Technical Setup

The 6.1 speaker system is driven with discrete outputs (including the LFE channel) from the audio interface. The speaker array should be placed in a regular distribution around a central listening area. The microphones should be suspended from the ceiling, or installed on stands within this listening area. All cables should be neatly run to a control station (computer, audio interface, power supply), which, if possible should be in a separate room or hidden in a corner as much as is possible. Room Needs The room can range in size from about 15’ X 15’ to much larger. Larger is better. This composition has been comfortably installed and tested in a variety of spaces. The only specific need, is that enough space exist for participants/visitors to comfortably walk and explore the sonic space of the composition. Likewise, higher ceilings are preferred. MORE IMPORTANTLY: As this piece does touch on notions of the past’s influence on the present, and future, as well as the interconnected energy of unique locations; this work would be ideally situated in a location of some historical reverence. Art museum galleries, public corridors, sound/audio studios, and concert halls have been successful locations for the presentation of these works. This encourages participants to consider the lasting impact the past has on us today, as well as the energy they will leave for generations to come. It is also important that there will be a decent amount of foot traffic by the installation, as this piece is best experienced when more than one person is in the room. Setup time One full day will be required for setup (up to 5 hours.) It usually takes:

1. 1-2 hours to setup the room and test all connections 2. 1 hour to dress cables (tape down if needed), and tighty up any other messes in the

room 3. 2 hours to ‘tune the system’ for this particular room. Because these pieces are so

reliant on the physical space, there is an extensive tuning process involved in order to bring cohesion between the digital system and physical space.

Technical Diagram

Previous setup example

 

Timbral Hauntings Code

The following Pages contain the code. This code is the composition and should be viewed as such.

Following in the idea of Agostino Di Scipio, this is a piece in which “wanted interactions [among

the interacting agents] have been composed.”

In addition, this code is provided for posterity and musical analysis.

1 /******************************************************************************* 2 * 3 * TIMBRAL HAUNTINGS 4 * 2014 - New York City, NY 5 * 6 * Last Edited: 7.2.14 7 * Using: SC 3.7.alpha 8 * 9 * 10 * Dependencies: 11 * sc3-plugins - http://sourceforge.net/projects/sc3-plugins/ 12 * MathLib Quark - Quarks.gui 13 * KDTree Quark - "" 14 * Python with Numpy installed 15 * Swing OSC - http://www.sciss.de/swingOSC/ 16 * (This can be commented out. and currently is) 17 * 18 *******************************************************************************/ 19 20 // call this if the server refuses to start 21 Server.killAll; 22 ServerOptions 23 24 /******************************************************************************/ 25 /* START THE SERVER AND ALLOCATE MEMORY */ 26 ( 27 o = Server.default.options; 28 o.memSize = 2**18; 29 o.memoryLocking = false; 30 31 // o.device = "Avid HDX"; 32 // o.device = "Avid 002 Rack"; 33 // o.device = "Digidesign HW ( 002 )"; 34 // o.device = "Digidesign Mbox 2 Mini"; 35 // o.inDevice = "Built-in Microph"; o.outDevice = "Built-in Output"; 36 // o.inDevice = "Built-in Microph"; o.outDevice = "Digidesign Mbox 2 Mini"; 37 o.device = "MOTU UltraLite mk3"; 38 o.numInputBusChannels = 2; 39 o.numOutputBusChannels = 10; 40 41 // GUI SPECIFIC 42 // GUI.swing; 43 // g = SwingOSC.default; 44 // g.reboot; 45 Buffer.freeAll; 46 s.reboot; 47 48 ) 49 50 51 /******************************************************************************/ 52 /* INITIALIZE GLOBAL VARIABLES AND ALLOCATE MEMORY TO BUFFERS */ 53 ( 54 // SET THE GLOABL VOLUME LEVEL 55 // s.volume = -6.0; 56 57 /* GLOBAL VARIABLES */ 58 // These are adjustable for tuning 59 ~numCoeff = 13; 60 ~numCoeff_slope = 10; 61 ~kNum = 14; 62 ~classyCountGoal = 2; 63 ~classyLearnSize = 2; 64 // ~mic = 0;

65 ~mic = [0, 1]; 66 ~numOfSpeakers = 6; 67 // ~numOfSpeakers = 4; 68 ~maxCurrentSynths = 4; 69 // ~maxCurrentSynths = 2; 70 71 72 // important -- tuning values! 73 ~mfccDiffThresh = 0.4; 74 75 ~onsetThresh = 0.2; 76 ~grainLength = 0.05; // 50ms 77 ~minBuffLength = 2; 78 ~maxRecordLength = 20; 79 ~recordDelay = 0.7; 80 ~chromaSize = 12; 81 82 // Python file location 83 ~pythonFileLocation = ( 84 // "/Volumes/MUSICK_LOVE/TimbralHauntings"++ 85 "/Users/Musick/GoogleDrive/Compositions/TimbralHauntings"++ 86 // "/Users/michaelmusick/GoogleDrive/Compositions/TimbralHauntings"++ 87 // Document.current.dir++ 88 "/pythonPolyfit.py" 89 ); 90 "Please check that Python File Location is correct...".postln; 91 ~pythonFileLocation.postln; 92 93 // Buses which are used to pass audio and control signals around the environment 94 ~micBus = Bus.audio( s, 1 ); 95 ~micDelayBus = Bus.audio( s, 1 ); 96 ~micRecordBus = Bus.audio( s, [~mic.size, 1].maxItem ); 97 ~rmsBus = Bus.audio( s, 1 ); 98 ~mfccBus = Bus.control( s, ~numCoeff ); 99 ~mfccDelayBus = Bus.control( s, ~numCoeff ); 100 ~smoothedMFCC = Bus.control( s, ~numCoeff ); 101 ~mfccClassify = Bus.control( s, ~numCoeff ); 102 ~slopeBus = Bus.control( s, 1 ); 103 ~slopeClassify = Bus.control( s, ~numCoeff_slope ); 104 ~fluxBus = Bus.control( s, 2 ); 105 ~trigBus = Bus.control( s, 1 ); 106 ~onsetBus = Bus.control( s, 1 ); 107 ~chromaBus = Bus.control( s, ~chromaSize*2 ); 108 ~fftSlopeBus = Bus.control( s, 1 ); 109 ~systemOutputBus = Bus.audio( s, ~numOfSpeakers ); 110 111 // Holding Arrays 112 // used to compare onsets for segmentation 113 ~mfccValueArray = Array.fill( 2, { 0 ! (~numCoeff-1) }; ); 114 // used by the record synth to write values to 115 ~tempRecBuffs = Array.fill( 4, 116 { Buffer.alloc( s, s.sampleRate * ~maxRecordLength, 1); }); 117 ~tempSlopeBuffs = Array.fill( 4, 118 { Buffer.alloc( s, 119 ((s.sampleRate/s.options.blockSize)*~maxRecordLength).ceil, 1); }); 120 ~tempMFCCBuffs = Array.fill( 4, 121 { Buffer.alloc( s, 122 (~maxRecordLength/~grainLength).round, 13 )}; ); 123 ~tempRhythmBuffs = Array.fill( 4, 124 { Buffer.alloc( s, 125 ((s.sampleRate/s.options.blockSize)*~maxRecordLength).ceil, 1); }); 126 127 // used to hold audio after classification 128 ~audioArray = Array.newClear( ~kNum ); 129 ~mfccArray = Array.newClear( ~kNum );

130 ~slopeArray = Array.newClear( ~kNum ); 131 ~rhythmArray = Array.newClear( ~kNum ); 132 133 // hold state values about the recordSynth 134 ~tempRecSynths = Array.newClear( 5 ); 135 ~tempRecSynths[4] = 0; 136 137 // classifier variables and buffers 138 ~kbuf_mfcc = Buffer.alloc(s, ~kNum, ~numCoeff+1); 139 ~timbreDecisions = Array.fill( ~kNum, {0}); 140 ~timbreCount = 0; 141 ~kbuf_slope = Buffer.alloc(s, ~kNum, ~numCoeff_slope+1); 142 ~slopeDecisions = Array.fill( ~kNum, {0}); 143 ~slopeCount = 0; 144 145 // new storage structure 146 // make a temp storage structure of dictionaries 147 ~tempData = Array.fill( 4, { (); }; ); 148 149 // create an array for the Synths 150 ~curSynths = Array.fill( ~maxCurrentSynths, { ('lpc': nil); }); 151 ~cs = 0; 152 153 // Chroma variables 154 ~chromaArray = Array.fill( ~chromaSize*2, {0} ); 155 156 // get the local address for OSC communication amongst components 157 ~addr = NetAddr.localAddr; 158 159 // create information windows 160 s.makeWindow; 161 s.plotTree(); 162 s.meter; 163 ); 164 165 166 /******************************************************************************/ 167 /* GRACEFUL FINISH */ 168 // sets all synths 0.0 gain allowing for a smooth finish 169 // Call this if the system needs to be cut with people around 170 ( 171 Task({ 172 ~time.postln; 173 ~sysIsClassy.free; 174 ~curSynths.do( { |val, idx| 175 if( ~curSynths[idx].lpc != nil, 176 { ~curSynths[idx].lpc.set(\attenuationThresh, 0); 177 }); 178 }); 179 4.wait; 180 ~curSynths.do( { |val, idx| 181 // check if a synth needs to be cleared out 182 if( (~curSynths[idx].lpc != nil), { 183 // clear the array pointer out 184 ~curSynths[idx].lpc.set(\envTrig, 0); 185 ~curSynths[idx].lpc = nil; 186 ~curSynths[idx].grain.stop; 187 ~curSynths[idx].audio.free; ~curSynths[idx].audio = nil; 188 ~curSynths[idx].slope.free; ~curSynths[idx].slope = nil; 189 ~curSynths[idx].audioReady = false; 190 ~curSynths[idx].slopeReady = false; 191 }); 192 }); 193 194

195 ~timbreDecisions.do({|val, idx| ~timbreDecisions[idx] = 0;}); 196 ~slopeDecisions.do({|val, idx| ~slopeDecisions[idx] = 0;}); 197 198 199 200 201 ~classyCountGoal = 8; 202 }).start; 203 ) 204 205 206 207 208 /******************************************************************************/ 209 /******************************************************************************/ 210 /* MAIN BLOCK */ 211 ( 212 213 /*TAKE ME OUT!!!!!!!!!!!!!!******************/ 214 // ~classyCountGoal = 2; 215 // ~classyLearnSize = 2; 216 /*TAKE ME OUT!!!!!!!!!!!!!!******************/ 217 218 // Reset storage buffers at start of new execution 219 ~curSynths = Array.fill( ~maxCurrentSynths, { ('lpc': nil); }); 220 ~audioArray.do({|val, id| 221 if(~audioArray[id].notNil, {~audioArray[id].free; ~audioArray[id]=nil;}) }); 222 ~slopeArray.do({|val, id| 223 if(~slopeArray[id].notNil, {~slopeArray[id].free; ~slopeArray[id]=nil;}) }); 224 225 // keep track of time running 226 ~startTime = Main.elapsedTime; 227 ~timeClock = { 228 loop({~time = (Main.elapsedTime - ~startTime).asTimeString; 0.5.wait}) 229 }.fork; 230 231 // reset task every 30 minutes 232 ~taskTimer = ~timeClock = { 233 loop({ 234 ~resetTask.start; 235 (60*380).wait}) 236 }.fork; 237 238 239 240 ~resetTask = Task({ 241 ~time.postln; 242 243 ~curSynths.do( { |val, idx| 244 if( ~curSynths[idx].lpc != nil, 245 { ~curSynths[idx].lpc.set(\attenuationThresh, 0); 246 }); 247 }); 248 4.wait; 249 ~curSynths.do( { |val, idx| 250 // check if a synth needs to be cleared out 251 if( (~curSynths[idx].lpc != nil), { 252 // clear the array pointer out 253 ~curSynths[idx].lpc.set(\envTrig, 0); 254 ~curSynths[idx].lpc = nil; 255 ~curSynths[idx].grain.stop; 256 ~curSynths[idx].audio.free; ~curSynths[idx].audio = nil; 257 ~curSynths[idx].slope.free; ~curSynths[idx].slope = nil; 258 ~curSynths[idx].audioReady = false; 259 ~curSynths[idx].slopeReady = false;

260 }); 261 }); 262 263 264 ~timbreDecisions.do({|val, idx| ~timbreDecisions[idx] = 0;}); 265 ~slopeDecisions.do({|val, idx| ~slopeDecisions[idx] = 0;}); 266 267 268 269 270 ~classyCountGoal = 2; 271 ~resetTask.stop; 272 }); 273 274 275 276 // get the MFCC values and send them out a 277 // 1. real-time bus for phrase detection and visualization 278 // 2. delayed bus for recording 279 ~getMFCCs = SynthDef(\getMFCCs, { 280 arg outBus = 0, outDelayBus = 1, input = 0, delayLength = 0.5, 281 fftSize = 2048; 282 var inSig, fft, mfccArray; 283 284 // make sure ~numCoeff is set outside SynthDef 285 if(~numCoeff==nil, { 286 ~numCoeff = 13; 287 "13 coefficients will be used for mfcc".postln; 288 }, { 289 (~numCoeff.asString + "coeff will be used for mfcc").postln; 290 "~numCoeff was previously set".postln; 291 }); 292 293 // get input signal 294 inSig = Mix.new( In.ar( input ) ); 295 // place that in an FFT 296 fft = FFT( 297 buffer: LocalBuf(fftSize), 298 in: inSig, 299 hop: 0.5, 300 wintype: 1 /*hann*/ 301 ); 302 // compute MFCC 303 mfccArray = MFCC.kr( fft, numcoeff: ~numCoeff ); 304 305 // moving mean filter 306 mfccArray = MeanTriggered.kr( 307 mfccArray, 308 trig: 1, 309 // length: one second long 310 length: (s.sampleRate/s.options.blockSize*0.25).round 311 ); 312 313 // send the values out a control bus 314 Out.kr(outBus, mfccArray); 315 316 mfccArray = DelayC.ar( mfccArray, delayLength, delayLength ); 317 Out.kr(outDelayBus, mfccArray); 318 319 }).play( 320 args: [ \outBus, ~mfccBus, \outDelayBus, ~mfccDelayBus, \input, ~micBus, 321 \trig, ~trigBus.asMap, 322 \delayLength, ~recordDelay 323 ]); 324

325 326 // get the change in slope values and send them out a 327 // 1. real-time bus for phrase detection and visualization 328 // 2. delayed bus for recording 329 ~getSlope = SynthDef(\getSlope, { 330 arg outBus = 0, input = 0, meanFilterSize = 512, delayLength = 0.5; 331 var inSig, freq, hasFreq, avgFreq, slope; 332 333 // get input signal 334 inSig = Mix.new( In.ar( input ) ); 335 // inSig.poll; 336 337 // get the freq 338 # freq, hasFreq = Pitch.kr( inSig, 339 minFreq: 80, // min expected freq 340 maxFreq: 3000, // max expected freq 341 median: 200, // length of median filter 342 // (used for smoothing output pitch 343 ampThreshold: 0.01, // min amplitude thresh to perform calc 344 peakThreshold: 0.5, // min peak threshold 345 // (this helps in eliminting overtones ) 346 clar: 1 // produce a confidence measure 347 ); 348 349 // only take frequencies that are "confident" 350 freq = Gate.kr( freq, hasFreq-0.9 ); 351 // check freq and confidence measure values 352 // freq.poll(label:\freq); 353 // hasFreq.poll(label:\freqCondfidence); 354 355 // pass the frequency through a moving mean filter 356 // this prevents sudden mis-labels 357 avgFreq = (RunningSum.kr( freq, meanFilterSize ) / meanFilterSize); 358 slope = Slope.kr( avgFreq ).lag(0.1); 359 // slope.poll(label:\slope); 360 361 // delay it before sending it to the recorder 362 slope = DelayC.kr(slope, delayLength, delayLength); 363 // send the values out a control bus 364 Out.kr(outBus, slope); 365 }).play( args: 366 [\outBus, ~slopeBus, \input, ~micBus, \delayLength, ~recordDelay]); 367 368 // get the onset triggers which are used to check for a new phrase 369 // also get the flux value between FFT frames 370 // this is used as a gate in the recorder to now when to sum MFCC values 371 // summed MFCC values are used to classify the timbre of a phrase 372 ~onsetTrigger = SynthDef(\onsetTrigger, { 373 arg outBus = 0, outBusTrig = 1, outBusOnsets = 2, input = 0, 374 onsetThresh = 0.2, fftSize = 2048; 375 var inSig, fft, flux, maxFlux, trig, env, envGen, envTrig, onsetSig; 376 377 inSig = In.ar( input ); 378 379 fft = FFT( 380 buffer: LocalBuf(fftSize), 381 in: inSig, 382 hop: 0.5, 383 wintype: 1 /*hann*/ 384 ); 385 386 flux = FFTFlux.kr( fft ); 387 388 // trig this 389 // this is used to set a variables onset threshold limit

390 // that lowers after 2 seconds 391 envTrig = LocalIn.kr(1); 392 env = Env.new( 393 [0, 2, 1, 0], 394 [0.01, 0.8, 0.2] 395 ); 396 envGen = EnvGen.kr(env, gate: envTrig, doneAction: 0); 397 398 onsetThresh = onsetThresh + envGen; 399 // onsetThresh.poll; 400 trig = Onsets.kr( 401 chain: fft, 402 threshold: onsetThresh, // min threshold 403 odftype: \wphase, // analysis type 404 relaxtime: 2, // time it takes for the detector 405 // to forget the spectrum 406 floor: 0.05, // expected min rms 407 mingap: 10, // brute force min in frames 408 medianspan: 11 // median filter length in num of frames 409 ); 410 LocalOut.kr( trig ); 411 412 // send an OSC message to the server alerting it to the trigger 413 SendTrig.kr( trig, 0 ); 414 415 maxFlux = RunningMax.kr( flux, TDelay.kr(trig, 0.01) ); 416 417 // send the flux values and onset trigger signal out buses 418 Out.kr( outBus, [ flux, maxFlux ] ); 419 Out.kr( outBusTrig, trig ); 420 421 // Poll.kr(trig, flux, "flux"); 422 // Poll.kr(trig, maxFlux, "max flux value"); 423 // Poll.kr(trig, onsetThresh, "thresh"); 424 }).play( args: [\outBus, ~fluxBus, \outBusTrig, ~trigBus, \input, ~micBus, 425 \outBusOnsets, ~onsetBus, \onsetThresh, ~onsetThresh], 426 addAction: \addToTail 427 ); 428 429 // mic signals 430 ~delayedMic = SynthDef(\delayInput, { 431 arg input = [8, 9], outMicBus = 0, outDelayBus = 0, outRMS = 1, 432 delayLength = 0.5, feedbackIn = 0, feedbackScale = 0, outRecord = 0; 433 var inSig, delaySig, rmsSig, fbSig; 434 435 inSig = SoundIn.ar( ~mic ); 436 437 Out.ar(outRecord, inSig); 438 439 if( inSig.size > 1, 440 { 441 inSig[0] = inSig[0] * SinOsc.kr( 0.05, 0, 1, 4 ).clip(0, 1); 442 inSig[1] = inSig[1] * SinOsc.kr( 0.05, 3.14, 1, 4 ).clip(0, 1); 443 inSig = Mix( inSig ); 444 }); 445 446 fbSig = DelayC.ar( 447 in: InFeedback.ar( feedbackIn ) * feedbackScale, 448 maxdelaytime: 0.01, 449 delaytime: 0.01 450 ); 451 fbSig = LPF.ar( fbSig, 8000 ); 452 453 inSig = Mix([inSig, Mix(fbSig)]); 454

455 delaySig = DelayC.ar(inSig, delayLength, delayLength); 456 rmsSig = RunningSum.rms(inSig, 2048); 457 458 459 460 Out.ar(outMicBus, inSig); 461 Out.ar(outDelayBus, delaySig); 462 Out.ar(outRMS, rmsSig); 463 }).play( args: [\input, ~mic, \outMicBus, ~micBus, \outDelayBus, ~micDelayBus, 464 \outRMS, ~rmsBus, \delayLength, ~recordDelay, 465 \feedbackIn, ~systemOutputBus, \outRecord, ~micRecordBus 466 ]); 467 // ~delayedMic.set(\micScale, 1.5); 468 // ~delayedMic.set(\feedbackScale, 0.3); 469 470 // CHROMAGRAM TRACKER OF MIC 471 ~getChroma = SynthDef(\chroma, { 472 arg input = 0, outBus = 0, fftSlopeOut = 1, tuningBase = 32.703195662575, lowOctSize =

4; 473 var in, fft, chroma, chromaLow, chromaHigh, cutoff, slope; 474 475 cutoff = tuningBase*(2**lowOctSize); 476 477 in = In.ar( input ); 478 in = Mix(in); 479 480 fft = FFT(LocalBuf(2048), in); 481 slope = FFTSlope.kr(fft).lag(0.2); 482 483 484 fft = FFT(LocalBuf(2048), BLowPass.ar( in, cutoff, 0.07 ) ); 485 chromaLow = Chromagram.kr(fft, 2048, 486 n: ~chromaSize, 487 tuningbase: tuningBase, 488 octaves: lowOctSize, 489 integrationflag: 1 ); 490 491 fft = FFT(LocalBuf(2048), BHiPass.ar( in, cutoff, 0.07 ) ); 492 chromaHigh = Chromagram.kr(fft, 2048, 493 n: ~chromaSize, 494 tuningbase: cutoff, 495 octaves: 4, 496 integrationflag: 1 ); 497 chroma = chromaLow ++ chromaHigh; 498 // chroma.poll; 499 500 Out.kr( outBus, chroma ); 501 Out.kr( fftSlopeOut, slope ); 502 }).play( args: 503 [\input, ~micDelayBus, \outBus, ~chromaBus, \fftSlopeOut, ~fftSlopeBus], addAction:

\addToTail ); 504 505 // TASK TO TRACK THE 4 HIGHEST CHROMA VALUES 506 ~chromaTask = ( 507 Task({ 508 inf.do{ 509 ~chromaBus.get( { |chroma| 510 //get the chroma values 511 ~chromaArray.do( 512 { arg val, idx; ~chromaArray[idx]=val+chroma[idx]}); 513 }); 514 // ~chromaArray.postln; 515 516 0.1.wait; 517 }

518 }); 519 ); 520 ~chromaTask.start; 521 522 // CHROMA PEAK PICKER FUNCTION 523 // This function peak picks the chroma 524 // and uses these to assign the harmonic pitches to the LPC Synths 525 ~lastChromaBase = 0; 526 ( 527 ~chromaToLPC_harmony = { 528 arg base = 32.703195662575*2; 529 var chroma, size, maxVal, peaks, octave, bass, bassIdx, slope; 530 // get the current state of the chroma array 531 chroma = Array.newFrom( ~chromaArray[0..~chromaSize*2-1] ); 532 size = chroma.size; 533 slope = ~fftSlopeBus.getSynchronous; 534 535 536 // normalize both halves to 1 537 maxVal = chroma.maxItem; 538 chroma.do( {|val, idx| chroma[idx]=val/maxVal; }); 539 chroma = ( chroma[0..(size/2-1).asInteger].normalize 540 ++ chroma[(size/2-1).asInteger..(size-1).asInteger].normalize ); 541 542 // get the 4 highest values 543 peaks = Array.newClear( ~maxCurrentSynths ); 544 // make sure we get a new root for the chroma basec chord 545 if( chroma.maxIndex != ~lastChromaBase, 546 { peaks[0] = chroma.maxIndex; chroma[peaks[0]] = 0; }, 547 { 548 chroma[chroma.maxIndex] = 0; 549 peaks[0] = chroma.maxIndex; 550 chroma[peaks[0]] = 0; 551 } 552 ); 553 ~lastChromaBase = peaks[0]; 554 555 peaks[1..3].do( { |val, idx| 556 peaks[idx+1] = chroma.maxIndex; 557 chroma[peaks[idx+1]] = 0; 558 }); 559 // peaks.postln; 560 561 // find the fundamental bass freq 562 bassIdx = peaks[0]; 563 // convert to a frequency based off C2 564 if( bassIdx < 12, 565 { bass = base * ( (bassIdx+12) / 12 ); }, 566 { bass = base * ( bassIdx / 12 ); } 567 ); 568 569 peaks.do( { |degree, idx| 570 var octMul; 571 // "degree: ".post; degree.post; " octave: ".postln; 572 // pick an octave based on range 573 if( (degree<12).and( slope > (-4) ), 574 { octMul = rrand(0, 2); }, 575 { octMul = rrand(3, 6); } 576 ); 577 octave = 12 * (2 ** octMul); 578 // octave.postln; 579 // make the rest of the set relative to the bass 580 degree = (degree-bassIdx) % 12; // mod12 to keep it positive 581 // degree.postln; 582 // translate it to the equivalent size of the octave

583 degree = ((2 ** octMul) - (2 ** (octMul-1))) * degree; 584 // degree.postln; 585 586 // get the frequency 587 peaks[idx] = ( (bass * (degree+octave)) / 12 ); 588 }); 589 590 // POSTER 591 // "Spectral Slope: ".post; slope.postln; 592 // "Bass Freq: ".post; bass.postln; 593 // "frequency set: ".post; peaks.postln; 594 595 // reset the tracker chroma array; 596 ~chromaArray.fill(0); 597 598 // change the LPC bandpass 599 peaks.do( { |freq, idx| 600 if( ~curSynths[idx].lpc != nil, { 601 ~curSynths[idx].lpc.set(\filtLag, rrand(1, 12)); 602 ~curSynths[idx].lpc.set(\filtFreq, freq); 603 }); 604 }); 605 606 }; 607 ); 608 // ~chromaToLPC_harmony.() 609 610 // record all signals for each phrase. 611 // at the end of this synth, send the values out for classification 612 // This is called by an OSC Function when new 'phrases' are discovered 613 ( 614 SynthDef(\audioRecorder, { 615 // bufnum is the buffer to write to 616 // input is the audio input channel 617 // delaycomp compensates for any delays introduced during pre-processing 618 arg bufnum_audio = 0, bufnum_slope = 1, bufnum_mfccs, 619 input_audio = 0, input_slope = 1, input_mfcc = 2, 620 grainLength = 0.05, flux = 0, fluxThresh = 1.0, 621 bus_smooth = 0, fluxDelayComp = 0.5, 622 id = 0, t_reset = 0, gate = 1; 623 var inSig, slopeSig, mfccSig, playhead_ar, playhead_kr, playhead_grain; 624 var recorder, countdown, env, envSig, mfccTrig, timestamp, averageMFCC; 625 var avgCount, avgGate, envTrig; 626 var onsetSig; 627 628 629 // an envelope 630 envTrig = (gate * (1 - Trig1.kr( (LocalIn.kr(1)+(s.sampleRate*0.5)), 10 ))); 631 env = Env.new([0,0,1,1,0,0], [0.001, 0.05, 0, 0.05, 0.01], releaseNode: 3 ); 632 envSig = EnvGen.ar(env, envTrig ); 633 634 // input signals 635 inSig = Delay2.ar(Delay2.ar( In.ar(input_audio) )) * envSig ; 636 slopeSig = In.kr(input_slope) * envSig; 637 mfccSig = In.kr(input_mfcc, ~numCoeff); 638 639 640 641 // track playheads for each signal 642 // keeping track of these values allow for variable length written signals 643 playhead_ar = Phasor.ar( 644 trig: t_reset, 645 rate: 1, 646 start: 0, 647 end: BufFrames.kr(bufnum_audio),

648 resetPos: 0 649 ); 650 playhead_kr = Phasor.kr( 651 trig: t_reset, 652 rate: 1, 653 start: 0, 654 end: BufFrames.kr(bufnum_slope), 655 resetPos: 0 656 ); 657 playhead_grain = (playhead_ar / (s.sampleRate * grainLength)).floor; 658 659 // use the flux value as a gate for the mean MFCC value set calculation 660 flux = Mix(In.kr(flux, 1)) + Line.kr(3, 0, 0.5); 661 // flux.poll(label: "flux"); 662 flux = DelayC.kr(flux, fluxDelayComp, fluxDelayComp); 663 flux = ((flux-fluxThresh)*(-1)); 664 // flux.poll(label: "fluxThresh"); 665 666 // TRACK THE MFCC VALUES FOR THIS PHRASE 667 // THIS IS USED FOR TIMBRE CLASSIFICATION. 668 // only let MFCCs through when spectral flux is stable 669 avgGate = Clip.kr(flux, 0, 1).ceil; 670 // avgGate.poll(label: "gate"); 671 averageMFCC = if(avgGate, mfccSig, 0!13); 672 // get the number of readings counted 673 avgCount = 1 + RunningSum.kr(avgGate, 674 (s.sampleRate/s.options.blockSize*60).ceil); 675 // insure that division by 0 will not occur 676 avgCount = Clip.kr(avgCount, 677 lo: 1, 678 hi: (s.sampleRate/s.options.blockSize*60).ceil); 679 // avgCount.poll(label: "sumCount"); 680 // averageMFCC = AverageOutput.kr(averageMFCC); 681 // find the average for MFCCs 682 averageMFCC = RunningSum.kr(averageMFCC, 683 (s.sampleRate/s.options.blockSize*60).ceil); 684 // averageMFCC[3].poll(label: "testCoeff0"); 685 averageMFCC = averageMFCC / avgCount; 686 // averageMFCC[3].poll(label: "testCoeff1"); 687 Out.kr(bus_smooth, averageMFCC); 688 689 // apply smoothing to capture the nature of each grain 690 mfccSig = MeanTriggered.kr(mfccSig[1..~numCoeff-1], 1, 691 grainLength/(s.options.blockSize/s.sampleRate)); 692 // capture the first time stamp for each grain and append to the list 693 mfccTrig = Changed.ar(playhead_grain); 694 timestamp = Latch.ar(playhead_ar, mfccTrig); 695 // bundle into an array 696 mfccSig = mfccSig++[timestamp]; 697 698 // copy it all in! HUZZAH 699 BufWr.kr( mfccSig, bufnum_mfccs, playhead_grain, 0 ); 700 BufWr.kr( slopeSig, bufnum_slope, playhead_kr, 0 ); 701 recorder = BufWr.ar( inSig, bufnum_audio, playhead_ar, 0 ); 702 703 // keep track of space remaining in each buffer 704 countdown = (playhead_ar-BufFrames.kr(bufnum_audio)+100); 705 LocalOut.kr(countdown); 706 707 // if at the end, then let client know and relay the playhead positions 708 SendReply.ar([K2A.ar(Done.kr(envSig)), countdown], '/metaData', 709 values: ( [playhead_ar, playhead_kr, playhead_grain] ++ averageMFCC ), 710 replyID: id 711 ); 712

713 FreeSelf.kr(Done.kr(recorder) + K2A.ar(Done.kr(envSig))); 714 715 // mfccSig.poll; 716 // (playhead_ar / s.sampleRate).poll; 717 // [playhead_ar, playhead_kr, playhead_grain].poll; 718 // countdown.poll(label:"countdown"); 719 720 }).add; 721 ); 722 723 724 // classifier for timbre values 725 ~mfccClassifier = SynthDef(\mfccClassifier, { 726 arg input_mfcc = 0, temp_id = 0, kmeansBuf = 0, kNum = ~kNum; 727 var classificationCount; 728 var mfccSig, id, trig, classification, centers, resetTrig; 729 730 // get the mean MFCC values 731 // these are '.setn()' so changes will only occur when this happens 732 mfccSig = In.kr( input_mfcc, ~numCoeff ); 733 734 // if a change occurs, then classify 735 trig = Changed.kr( mfccSig[~numCoeff-1] ); 736 resetTrig = 0; 737 738 // the classifier 739 classification = KMeansRT.kr( 740 bufnum: kmeansBuf, 741 inputdata: mfccSig, 742 k: kNum, 743 gate: trig, 744 reset: resetTrig, 745 learn: 1 746 ); 747 748 // number of classifications 749 classificationCount = PulseCount.kr( trig, resetTrig ); 750 751 // send the classification out 752 // SendTrig.kr(trig, 5, classification); 753 SendReply.kr(trig, '/mfccClassification', classification, temp_id); 754 755 // post info 756 // classification.poll(trig, label: "timbre classification"); 757 // classificationCount.poll(trig, label: "classification count"); 758 // mfccSig.poll(trig, label: "mfcc values received"); 759 // centers = KMeansRT.getCentroid( 760 // bufnum: kmeansBuf, 761 // classif: classification, 762 // ndims: ~numCoeff 763 // ).poll(trig, label: "centers"); 764 765 }).play( 766 args: [\input_mfcc, ~mfccClassify, \kmeansBuf, ~kbuf_mfcc, \kNum, ~kNum], 767 addAction: 'addToTail' 768 ); 769 770 ~slopeClassifier = SynthDef(\slopeClassifier, { 771 arg input_slope = 0, temp_id = 0, kmeansBuf = 0, kNum = ~kNum; 772 var classificationCount; 773 var slopeSig, trig, classification, centers, resetTrig; 774 775 // get the slope coeffs 776 slopeSig = In.kr( input_slope, ~numCoeff_slope ); 777

778 // trig when changed 779 trig = Changed.kr( slopeSig[~numCoeff_slope-1] ); 780 resetTrig = 0; 781 782 classification = KMeansRT.kr( 783 bufnum: kmeansBuf, 784 inputdata: slopeSig, 785 k: kNum, 786 gate: trig, 787 reset: resetTrig, 788 learn: 1 789 ); 790 791 classificationCount = PulseCount.kr( trig, resetTrig ); 792 793 // send the classification out 794 // SendTrig.kr( trig, 6, classification ); 795 SendReply.kr(trig, '/slopeClassification', classification, temp_id); 796 797 // post info 798 // classification.poll(trig, label: "slope classification"); 799 // classificationCount.poll(trig, label: "classification count"); 800 // centers = KMeansRT.getCentroid( 801 // bufnum: kmeansBuf, 802 // classif: classification, 803 // ndims: ~numCoeff_slope 804 // ).poll(trig, label: "centers"); 805 }).play( 806 args: [\input_slope, ~slopeClassify, \kmeansBuf, ~kbuf_slope, \kNum, ~kNum], 807 addAction: 'addToTail' 808 ); 809 810 // THESE STRINGS BECOME TOO BIG. 811 // NEED TO USE OSC 812 // function to get poly coefficients from python 813 ~getPolyCoeff = { 814 // the slope to use 815 arg slope = [], id; 816 // the degree of the polynomial 817 var polyDegree = ~numCoeff_slope; 818 // convert the slope signal to a string 819 var sigString = slope.asCompileString; 820 var unixRet, coeffs, string, line, pipe; 821 // id.postln; 822 // slope.shape.postln; 823 // if( ~pitchPlot.notNil, {~pitchPlot.parent.close} ); 824 // ~pitchPlot = slope.plot(name: ("slope pre-classify id: "++id)); 825 // sigString.postln; 826 sigString = (sigString)[10..]; 827 828 // call the python script 829 string = ( "python "++ 830 ~pythonFileLocation++ 831 " \""++sigString++"\" "++polyDegree 832 ); 833 834 ~sigString = string; 835 // string.postln; 836 // string[0..100].postln; 837 838 // get the values from python 839 unixRet = string.unixCmdGetStdOut; 840 841 // strip the return string unneccessary chars 842 // unixRet.postln;

843 // r = unixRet.postln; 844 unixRet.remove($[); 845 unixRet.remove($]); 846 unixRet.remove($ ); 847 unixRet = unixRet.stripWhiteSpace; 848 unixRet = unixRet.split($,); 849 // convert to a float array 850 // ignore the last coefficient (this is the y-intercept) 851 coeffs = (unixRet.collect( { 852 |string| 853 string.asFloat; 854 })[0..(unixRet.size-2)]); 855 x = coeffs; 856 // check that we didn't get a bad reading 857 if( coeffs.sum!=0, 858 { 859 // ("got slope coefficients and sent to slope classifier.").postln; 860 ~slopeClassifier.set(\temp_id, id); 861 ~slopeClassify.setn(coeffs); 862 } , { 863 // POSTER 864 // ("00 !!!!! no slope detected").postln; 865 } 866 ); 867 // increase gesture count 868 coeffs; 869 }; 870 871 872 // reset counters (needed if restarting) 873 ~timbreCount = 0; 874 ~slopeCount = 0; 875 ~tempRecSynths[4] = 0; 876 877 878 // identify new "phrases" by large differences of MFCC sets at Onset Moments 879 ~onsetMsgs = OSCFunc( { 880 arg msg, time, addr, recvPort; 881 var id; 882 883 // pull the '/tr' id 884 id = msg[2]; 885 886 // [time, addr, recvPort, msg].postln; 887 // [time, msg].postln; 888 // ("1 "++~tempRecSynths[4]).postln; 889 890 if( id==0, { 891 var mfccDiff; 892 893 // wait 0.4 seconds to move past transient and let MFCCs calm 894 0.4.wait; 895 // grab those MFCC values 896 ~mfccBus.getn( ~numCoeff, { |val| 897 // get the coefficients (try all coeffs and the last 12 only) 898 {~mfccValueArray[1] = val[1..(~numCoeff-1)];}.defer; 899 // {~mfccValueArray[1] = val;}.defer; 900 }); 901 902 // get the euclidean distance from the last 'marked' phrase 903 mfccDiff = (~mfccValueArray[1] - ~mfccValueArray[0]); 904 mfccDiff = (mfccDiff**2).sum.sqrt; 905 // ("MFCC Distance: " ++ mfccDiff).postln; 906 // ~mfccValueArray.postln; 907

908 // check if the mfcc exceeds the thresh 909 if(mfccDiff >= ~mfccDiffThresh, { { 910 911 // update the harmony of the lpcSynths 912 ~chromaToLPC_harmony.(); 913 914 // stop the last recording (if it is still going) 915 if(~tempRecSynths[~tempRecSynths[4]].notNil, { 916 ~tempRecSynths[~tempRecSynths[4]].set(\gate, 0); 917 ~tempRecSynths[~tempRecSynths[4]] = nil; 918 }); 919 920 // make sure we are caught up 921 0.1.wait; 922 923 // change the tempRecorder Synth Array Pointer 924 ~tempRecSynths[4] = ( (~tempRecSynths[4]+1) % 4 ); 925 926 // POSTER 927 // ("MFCC Distance: " ++ mfccDiff).post; 928 // ". Active recorder: ".post; ~tempRecSynths[4].postln; 929 930 // reset smoothedMFCC Bus 931 ~smoothedMFCC.setn( 0!13 ); 932 933 // move the mfcc arrays down for testing against the next onset triggers 934 ~mfccValueArray[0] = ~mfccValueArray[1]; 935 936 // start a new recorder 937 ~tempRecSynths[~tempRecSynths[4]] = Synth(\audioRecorder, 938 args: [ 939 \input_audio, ~micDelayBus, 940 \input_slope, ~slopeBus, 941 \input_mfcc, ~mfccDelayBus, 942 \bufnum_audio, ~tempRecBuffs[~tempRecSynths[4]], 943 \bufnum_slope, ~tempSlopeBuffs[~tempRecSynths[4]], 944 \bufnum_mfccs, ~tempMFCCBuffs[~tempRecSynths[4]], 945 \bus_smooth, ~smoothedMFCC, 946 \flux, ~fluxBus, 947 \fluxDelayComp, ~recordDelay, 948 \id, ~tempRecSynths[4] 949 ], 950 addAction: 'addToTail' 951 ); 952 953 }.fork; } // end of if() true function 954 ); // end of if( mfccThresh > ) 955 }.fork; // end of if() true function .fork makes it a scheduled function 956 ); // end of "if( id== 0 )" 957 958 }, // end of the OSCFunc function definition 959 '/tr', // the message to match 960 s.addr // get messages from the server on this machine 961 ); 962 963 // "End Of Recording Messages" 964 ~recMsgs = OSCFunc( { 965 arg msg, time, addr, recvPort; 966 var id; 967 // [time, msg].postln; 968 969 // get the ~tempRecSynths[] instance id 970 id = msg[2]; 971 972 // copy over the data!

973 ~tempData[id].audiohead = msg[3]; 974 ~tempData[id].slopehead = msg[4]; 975 ~tempData[id].mfcchead = msg[5]; 976 ~tempData[id].meanMFCC = msg[6..18]; 977 978 // must be longer than 2" 979 if( ( ~tempData[id].audiohead >= (s.sampleRate*~minBuffLength) ), { 980 // "recording is long enough. copy it and get classifications".postln; 981 982 // set the mfcc classification 983 ~mfccClassifier.set(\temp_id, id); 984 ~mfccClassify.setn( ~tempData[id].meanMFCC ); 985 986 // get the slope classification 987 ~tempSlopeBuffs[id].loadToFloatArray( count: 988 ~tempData[id].slopehead, 989 action: { |array| { ~getPolyCoeff.(array, id) }.defer; } 990 ); 991 }, { 992 // POSTER 993 // "phrase was too short to classify".postln; 994 } ); 995 996 // release the pointer so another process doesn't try to free a dead Synth 997 ~tempRecSynths[id] = nil; 998 999 }, // end of the OSCFunc function definition1000 // the message to match1001 '/metaData'1002 );1003 1004 1005 // TIMBRE CLASSIFICATION MESSAGES1006 ~mfccMsgs = OSCFunc( {1007 arg msg, time, addr, recvPort;1008 var id, class, copyData = true;1009 // [time, msg].postln;1010 1011 // get the ~tempRecSynths[] instance id1012 id = msg[2].asInteger;1013 1014 // get the classification number1015 ~tempData[id].timbreClass = class = msg[3].asInteger;1016 1017 // up the timbre classification counts1018 ~timbreCount = ~timbreCount+1;1019 1020 // POSTER1021 // ("Timbre Classification: "++~tempData[id].timbreClass).post;1022 // (" -- Timbre Count: "++~timbreCount).postln;1023 1024 // have we learned enough?1025 if( ~timbreCount > ~classyLearnSize,1026 { ~timbreDecisions[class] = ~timbreDecisions[class]+1; },1027 { copyData = false; }1028 );1029 1030 // check lengths, in order to keep the longer recording1031 if( ~audioArray[class].notNil, {1032 if( ( ( ~tempData[id].audiohead >= ~audioArray[class].numFrames )1033 .or( ~tempData[id].audiohead > (s.sampleRate * 4) ) ),1034 // long enough so do nothing1035 {},1036 // if it is longer, than keep the old data1037 { copyData = false; }

1038 ); });1039 1040 // if copyData is true then create a random chance that the last recording will stay in

the buffer1041 if( copyData.and([0, 1].wchoose( [0.65, 0.35]

).booleanValue).or(~audioArray[class].isNil),1042 // else free the buffer and prepare for the new recording1043 { ~audioArray[class].free; ~audioArray[class] = nil; /*"copy the audio in".postln;*/

},1044 { copyData = false; /*"dont copy the audio in".postln;*/ }1045 );1046 1047 1048 // if we are replacing1049 if( copyData, {1050 // allocate a new buffer of the correct length.1051 ~audioArray[class] = Buffer.alloc(1052 s,1053 ~tempData[id].audiohead,1054 1,1055 // send a message which calls a copy function once its allocated1056 completionMessage: { ~addr.sendMsg('/audioBuffReady', id); }1057 );1058 1059 // copy the MFCC data to an array1060 ~tempMFCCBuffs[id].loadToFloatArray(1061 action: {1062 arg array;1063 ~mfccArray[class] = array.reshape(1064 (~tempData[id].mfcchead+1).asInt, ~numCoeff);1065 // ~tempMFCCBuffs[id].plot(1066 // name: ("mfcc -- id:"++id++"class:"++1067 // ~tempData[id].timbreClass));1068 });1069 1070 });1071 1072 1073 },1074 '/mfccClassification'1075 );1076 1077 // COPY THE AUDIO INTO THE audioArray1078 ~audioBuffMsgs = OSCFunc( {1079 arg msg, time, addr, recvPort;1080 var id;1081 // [time, msg].postln;1082 1083 // get the ~tempRecSynths[] instance id1084 id = msg[1];1085 1086 ~tempRecBuffs[id].copyData(~audioArray[~tempData[id].timbreClass]);1087 // plot the saved signal1088 // fork{1089 // if((~audioWindow1 != nil), {~audioWindow1.parent.close;});1090 // 0.01.wait;1091 // ~audioWindow1 = ~audioArray[~tempData[id].timbreClass].plot;1092 // ~audioWindow1.parent.onClose = { ~audioWindow1 = nil };1093 // };1094 },1095 '/audioBuffReady'1096 );1097 1098 // SLOPE CLASSIFICATION MESSAGES1099 ~slopeMsgs = OSCFunc( {

1100 arg msg, time, addr, recvPort;1101 var id, class, copyData = true;1102 // [time, msg].postln;1103 1104 // get the ~tempRecSynths[] instance id1105 id = msg[2].asInteger;1106 1107 // get the classification number1108 ~tempData[id].slopeClass = class = msg[3].asInteger;1109 1110 // up the slope classification counts1111 ~slopeCount = ~slopeCount+1;1112 1113 // POSTER1114 // ("Slope Classification: "++~tempData[id].slopeClass).post;1115 // (" -- Slope Count: "++~slopeCount).postln;1116 1117 // have we learned enough?1118 if( ~slopeCount > ~classyLearnSize,1119 { ~slopeDecisions[class] = ~slopeDecisions[class]+1; },1120 { copyData = false; }1121 );1122 1123 // check lengths, in order to keep the longer recording1124 if( ~slopeArray[class].notNil, {1125 if( ( ~slopeArray[class].numFrames >= ~tempData[id].slopehead ),1126 // if it is longer, than keep the old data1127 { copyData = false; },1128 // else free the buffer and prepare for the new recording1129 { ~slopeArray[class].free; ~slopeArray[class] = nil; }1130 ); });1131 1132 // if we are copying1133 if( copyData,1134 {1135 // allocate a new buffer of the correct length.1136 ~slopeArray[class] = Buffer.alloc(1137 s,1138 ~tempData[id].slopehead,1139 1,1140 // send a message which calls a copy function once its allocated1141 completionMessage: { ~addr.sendMsg('/slopeBuffReady', id); }1142 );1143 },1144 {1145 // if copy data was false, check if it is time for new synth set1146 if( ( ~slopeDecisions.sum >= ~classyCountGoal )1147 .and( ~timbreDecisions.sum >= ~classyCountGoal ),1148 {1149 ~classyCountGoal = ~kNum/2;1150 // then call the transition function1151 // "system is classified!".postln;1152 1153 // POSTER1154 // "".postln;1155 // "-------------------------------------------------------------------

".postln;1156 // ~time.postln;1157 1158 1159 ~addr.sendMsg('/systemIsClassified');1160 }, {1161 // POSTER1162 // "Sums: ".post;1163 // ~slopeDecisions.sum.post;

1164 // " ".post;1165 // ~timbreDecisions.sum.post;1166 // " -- Classify Count Goal: ".post;1167 // ~classyCountGoal.postln; "".postln;1168 });1169 }1170 );1171 },1172 '/slopeClassification'1173 );1174 1175 // ~classyCountGoal = 2;1176 1177 // COPY THE SLOPE INTO THE slopeArray1178 ~slopeBuffMsgs = OSCFunc( {1179 arg msg, time, addr, recvPort;1180 var id;1181 // [time, msg].postln;1182 1183 // get the ~tempRecSynths[] instance id1184 id = msg[1];1185 1186 ~tempSlopeBuffs[id].copyData(~slopeArray[~tempData[id].slopeClass]);1187 // ~slopeArray[~tempData[id].slopeClass].plot(1188 // name: ("slope -- id:"++id++"class:"++~tempData[id].slopeClass))1189 1190 fork{1191 // if((~slopeWindow1 != nil), {~slopeWindow1.parent.close;});1192 // 0.01.wait;1193 // ~slopeWindow1 = ~slopeArray[~tempData[id].slopeClass].plot(1194 // name: ("Classified Slope: classID: "++~tempData[id].slopeClass++" tempID:

:"++id),1195 // bounds: Rect(10, 10, 600, 300)1196 // );1197 // ~slopeWindow1.parent.onClose = { ~slopeWindow1 = nil };1198 0.01.wait;1199 1200 // classify slope always comes after classify timbre1201 // so check at the end of this function if the minimum has been met1202 if( ( ~slopeDecisions.sum >= ~classyCountGoal )1203 .and( ~timbreDecisions.sum >= ~classyCountGoal ), {1204 ~classyCountGoal = ~kNum/2;1205 // then call the transition function1206 // "system is classified!".postln;1207 1208 // POSTER1209 // "".postln;1210 // "-------------------------------------------------------------------

".postln;1211 // ~time.postln;1212 1213 1214 ~addr.sendMsg('/systemIsClassified');1215 }, {1216 // POSTER1217 // "Sums: ".post;1218 // ~slopeDecisions.sum.post;1219 // " ".post;1220 // ~timbreDecisions.sum.post;1221 // " -- Classify Count Goal: ".post;1222 // ~classyCountGoal.postln; "".postln;1223 });1224 };1225 1226 },

1227 '/slopeBuffReady'1228 );1229 1230 1231 // CALL AUDIO SYNTHESIS AND FILTERING SYNTHS1232 // ONCE SYSTEM HAS BEEN TRAINED AND CLASSIFIED MIN NUMBER OF TIMES1233 ~cs = 0;1234 ~sysIsClassy = OSCFunc( {1235 arg msg, time, addr, recvPort;1236 var timbreClass, slopeClass, id;1237 1238 // pull in the current ID1239 // to attach to this 'EVENT' dictionary while processing and preparing1240 id = ~cs;1241 1242 // choose the winners1243 // "choosing winners".postln;1244 // return the class with the highest number of classifications1245 // (in the case of a tie, the first index is returned)1246 ~curSynths[id].timbreClass = timbreClass = ~timbreDecisions.maxIndex;1247 ~curSynths[id].slopeClass = slopeClass = ~slopeDecisions.maxIndex;1248 // reset the decision arrays1249 ~timbreDecisions.do({|val, idx| ~timbreDecisions[idx] = 0;});1250 ~slopeDecisions.do({|val, idx| ~slopeDecisions[idx] = 0;});1251 1252 // get that sucker going~1253 // POSTER1254 ("Starting new synths. ID: "++id++" TimbreClass: "++1255 timbreClass++" SlopeClass: "++slopeClass).postln;1256 1257 // check if a synth needs to be cleared out1258 if( (~curSynths[id].lpc != nil), {1259 // clear the array pointer out1260 ~curSynths[id].lpc.set(\envTrig, 0);1261 ~curSynths[id].lpc = nil;1262 ~curSynths[id].grain.stop;1263 ~curSynths[id].audio.free; ~curSynths[id].audio = nil;1264 ~curSynths[id].slope.free; ~curSynths[id].slope = nil;1265 ~curSynths[id].audioReady = false;1266 ~curSynths[id].slopeReady = false;1267 });1268 1269 // allocate Buffers1270 ~curSynths[id].audio = Buffer.alloc( s, ~audioArray[timbreClass].numFrames,1271 completionMessage: { ~addr.sendMsg('/synthAudioReady', id); }1272 );1273 ~curSynths[id].slope = Buffer.alloc( s, ~slopeArray[slopeClass].numFrames,1274 completionMessage: { ~addr.sendMsg('/synthSlopeReady', id); }1275 );1276 // create a K-D-Tree1277 ~curSynths[id].mfccTree = KDTree( ~mfccArray[timbreClass], lastIsLabel: true );1278 1279 // load the granulator function in1280 ~curSynths[id].grain = ~grainSearch.(id);1281 1282 // update pointer to "current synth"1283 ~cs = (~cs + 1) % ~maxCurrentSynths;1284 1285 // "allocating space and copying".postln;1286 1287 1288 // lower the attenuationLimiterThreshhold for each LPC synth for a brief time1289 // This gives the new voice an opportunity to establish itself1290 // and change the texture of the composition.1291 // Especially if the texture is particualrily 'thick'

1292 ~curSynths.do( { |val, idx|1293 if( ~curSynths[idx].lpc != nil, {1294 Task({1295 ~curSynths[idx].lpc.set(\attenuationThresh, 0.04);1296 1.wait;1297 ~curSynths[idx].lpc.set(\attenuationThresh, 0.17);1298 }).start;1299 });1300 });1301 1302 1303 },1304 '/systemIsClassified'1305 );1306 1307 1308 1309 ~synthAudio = OSCFunc( {1310 arg msg, time, addr, recvPort;1311 var id;1312 // [time, msg].postln;1313 1314 // get the ~tempRecSynths[] instance id1315 id = msg[1].asInteger;1316 1317 ~audioArray[~curSynths[id].timbreClass].copyData( ~curSynths[id].audio );1318 // normalize audio1319 ~curSynths[0].audio.normalize(1);1320 // set ready flag1321 ~curSynths[id].audioReady = true;1322 1323 // "audio copied!".postln;1324 1325 // try starting the LPC Synth1326 // each buffer copy calls this and when both are true the synth will start1327 ~addr.sendMsg('/startLPCsynth', id);1328 1329 // clear audio array1330 ~audioArray.do({|val, idx| if(~audioArray[idx].notNil,1331 {~audioArray[idx].free; ~audioArray[idx]=nil;});1332 });1333 // "audio copied exit.".postln;1334 },1335 '/synthAudioReady'1336 );1337 1338 ~synthSlope = OSCFunc( {1339 arg msg, time, addr, recvPort;1340 var id;1341 // [time, msg].postln;1342 1343 // get the ~tempRecSynths[] instance id1344 id = msg[1].asInteger;1345 1346 ~slopeArray[~curSynths[id].slopeClass].copyData( ~curSynths[id].slope );1347 // normalize to 41348 ~curSynths[id].slope.normalize(4);1349 // set ready flag1350 ~curSynths[id].slopeReady = true;1351 1352 // "slope copied".postln;1353 1354 // try starting the LPC Synth1355 // each buffer copy calls this and when both are true the synth will start1356 ~addr.sendMsg('/startLPCsynth', id);

1357 1358 // clear slope array1359 ~slopeArray.do({|val, idx| if(~slopeArray[idx].notNil,1360 {~slopeArray[idx].free; ~slopeArray[idx]=nil;});1361 });1362 // "slope copy exited".postln;1363 },1364 '/synthSlopeReady'1365 );1366 1367 ~startSynths = OSCFunc( {1368 arg msg, time, addr, recvPort;1369 var id;1370 // [time, msg].postln;1371 1372 // get the ~tempRecSynths[] instance id1373 id = msg[1];1374 1375 // "trying to start synths ".postln;1376 if( (~curSynths[id].audioReady)1377 .and(~curSynths[id].slopeReady).and(~curSynths[id].lpc.isNil), {1378 // ("YAYAYAYAYAY Starting the Synths. ID: "++id).postln;1379 1380 // plot the winners1381 // fork{1382 // if(~audioCandPlot.notNil, {~audioCandPlot.parent.close;});1383 // if(~slopeCandPlot.notNil, {~slopeCandPlot.parent.close;});1384 // 0.01.wait;1385 // ~audioCandPlot = ~curSynths[id].audio.plot(1386 // name: ("Audio Candidate -- class: "++~curSynths[id].timbreClass),1387 // bounds: Rect( 410, 780, 600, 200 ));1388 // ~audioCandPlot.parent.onClose = { ~audioCandPlot = nil };1389 // ~slopeCandPlot = ~curSynths[id].slope.plot(1390 // name: ("Slope Candidate -- class: "++~curSynths[id].slopeClass),1391 // bounds: Rect( 410, 540, 600, 200 ));1392 // ~slopeCandPlot.parent.onClose = { ~slopeCandPlot = nil };1393 // };1394 1395 var mic;1396 if(~mic.size > 1, {mic = ~mic.choose}, {mic = ~mic} );1397 1398 1399 // start the LPC Synth1400 ~curSynths[id].lpc = Synth(\lpcSynth,1401 [ \audioBuff, ~curSynths[id].audio,1402 \slopeBuff, ~curSynths[id].slope,1403 \input, mic,1404 \instance, id,1405 \out, ~systemOutputBus,1406 \outPos, (id % ~maxCurrentSynths).asInteger;1407 ],1408 target: ~sysOut,1409 addAction: \addBefore1410 );1411 // "And the Task".postln;1412 // start the grain finder task1413 ~curSynths[id].waitTime = 0.1;1414 ~curSynths[id].grain.start;1415 1416 });1417 1418 // POSTER1419 // ("Audio Buffer Length: "++1420 // (~curSynths[id].audio.numFrames/s.sampleRate)).postln;1421 // ("Synths Succesfully Started. ID: "++id).postln;

1422 },1423 '/startLPCsynth'1424 );1425 1426 1427 1428 1429 ~grainSearch = {1430 arg id = 0;1431 Task({1432 arg lookAhead = 0.1, fluxThresh = 0.3, waitTime = 0.25;1433 var timestamp, lastTimestamp, distance, dur = 0, nodeDistanceThresh = 0.2;1434 // "Grain Search Function Started".postln;1435 inf.do{1436 ~fluxBus.get({1437 arg fluxVal;1438 if( fluxVal[1] < fluxThresh, {1439 ~mfccBus.get({1440 arg mfccSet;1441 var node, updown, out, pitchShift, level;1442 node = ~curSynths[id].mfccTree.nearest( mfccSet[1..12] );1443 distance = node[1];1444 timestamp = (node[0].label);1445 out = ~systemOutputBus;1446 updown = (2.rand).asInteger;1447 if(updown==0,1448 { pitchShift = distance.expexp(0.05, 0.4, 1, 1.5).clip(1, 2);1449 level = distance.explin(0.1, 0.4, 1, 0.7 ).clip(0.3, 1) },1450 { pitchShift = distance.expexp(0.05, 0.4, 1, 0.5).clip(0.5, 1);1451 level = distance.explin(0.1, 0.4, 1, 0.3 ).clip(0.3, 1) }1452 );1453 1454 if( (distance < nodeDistanceThresh).and(timestamp != lastTimestamp),

{1455 // make sure cpu is low enough1456 if( s.peakCPU<90, {1457 dur = distance.expexp(0.05, nodeDistanceThresh, 3, 0.4) *

(nodeDistanceThresh * 20).clip(1, 4);1458 // ("ID: "++id++" DistanceThresh: "++nodeDistanceThresh++"

Distance: "++distance.round(0.01)++"- Dur: "++dur.round(0.01)++"- Level: "++level.round(0.01)++"- pitchShift: "++pitchShift.round(0.01)++

1459 // "- Timestamp: "++timestamp++"- Last Timestamp: "++lastTimestamp++"- Flux: "++fluxVal.round(0.01)).postln;

1460 {Synth(\grainPB,1461 args: [\startFrame, timestamp, \dur, dur, \level, level,1462 \out, out, \pitchShift, pitchShift, \rmsSig,

~rmsBus,1463 \bufnum, ~curSynths[id].audio,1464 ],1465 target: ~sysOut,1466 addAction: \addBefore1467 );1468 }.value;1469 lastTimestamp = timestamp;1470 nodeDistanceThresh = 0;1471 }/*, {"cpu too high".postln;}*/);1472 }, { dur = 0; } );1473 1474 });1475 });1476 nodeDistanceThresh = nodeDistanceThresh + 0.02;1477 });1478 // choose an appropriate wait time1479 if( (dur*0.25) > 0, {(dur*0.15).wait;}, {~curSynths[id].waitTime.wait});1480 };

1481 });1482 };1483 1484 1485 1486 1487 (1488 // lpc synth1489 SynthDef(\lpcSynth, {1490 arg input = 0, out = 0, outPos = 0, audioBuff = 0, instance = 0,1491 slopeBuff = 0, envTrig = 1, filtLag = 10, filtFreq = 120,1492 attenuationThresh = 0.17;1493 var rateSig, inputSig, filteredSig, analysisSig, analysisAmp,1494 analysisAmp_mic, sigToFilter, sourceChain, lpcOutput, hpfOutput,1495 bandPOutput, limitedOutput, sig, env, delay,1496 onsetSig, cutSig, quarter, eigth, sixt, tempo, rms, envLimit, bandPassQ;1497 1498 // finishing env1499 env = EnvGen.kr(1500 Env.new( [0, 0, 1, 1, 0], [0.5, 5, 0.2, 3], releaseNode: 3 ),1501 gate: envTrig,1502 doneAction: 21503 );1504 1505 // get the playback rate signal for the analysisSig Buff from the slope data1506 rateSig = PlayBuf.ar( 1,1507 bufnum: slopeBuff,1508 rate: BufRateScale.kr(slopeBuff)/4,1509 loop: 11510 ) + 1.0; // add 1 to set likely speed to 11511 rateSig = rateSig.lag(0.2);1512 1513 1514 // analysis signal for LPC1515 // inputSig = DelayC.ar( SoundIn.ar(0)*0.8, 10, 2 );1516 inputSig = PlayBuf.ar( 1,1517 audioBuff,1518 rate: BufRateScale.kr(audioBuff) * rateSig.lag(3), //.poll(label: \playBackScalar),1519 loop: 11520 );1521 1522 1523 // get a delayed onset signal to send out for rhythm processing1524 onsetSig = Onsets.kr(1525 chain: FFT( LocalBuf( 1024, 1 ), inputSig, 0.25, 1 ),1526 threshold: 0.30, // min threshold1527 odftype: \rcomplex, // analysis type1528 relaxtime: 2, // time it takes for the detector1529 // to forget the spectrum1530 floor: 0.15, // expected min rms1531 mingap: 20, // brute force min in frames1532 medianspan: 11 // median filter length in num of frames1533 );1534 // onsetSig.poll( onsetSig, "onset sig" );1535 // Out.ar(0, Mix([inputSig, SinOsc.ar(200, mul: onsetSig.lag(0.05 ) )]));1536 1537 // filter the sig (this is a historical process ala Lansky)1538 // although this should occur in actual sample rate of the algorithm1539 // i need to keep the system at 44.1/48 so filtering work better than nothing1540 filteredSig = BLowPass.ar( inputSig, freq: 17000, rq: 0.5 );1541 // get the rms value, this is used as a simple gate1542 analysisAmp_mic = RunningSum.rms( filteredSig, numsamp: 512 );1543 // analysisAmp_mic.poll( label: \rmsMic);1544 // shift the sig1545 filteredSig = PitchShift.ar( filteredSig,

1546 windowSize: 0.1,1547 // use the same control signal as the playback speed,1548 // but take abs and place a 2 second lag on the signal1549 // take the signal as (2-signal) to provide some balance to time stretch1550 pitchRatio: ((2-rateSig).abs).lag(3), //.poll(label: \pitchScalar),1551 pitchDispersion: 0.001,1552 timeDispersion: 0.0011553 );1554 // Normalize the signal to 0.91555 // this has the effect of squashing the signal,1556 // this makes it easier to analyze and produces a strong output signal1557 // the analysisAmp_mic is used as a gate for this signal1558 // analysisSig = Normalizer.ar( filteredSig, level: 0.9, dur: 1 )1559 // analysisSig = filteredSig1560 analysisSig = CompanderD.ar(1561 filteredSig,1562 thresh: 0.5,1563 slopeBelow: 0.5,1564 slopeAbove: 0.5,1565 clampTime: 0.002,1566 relaxTime: 0.5,1567 mul: 1.11568 )1569 * EnvGen.kr(1570 Env.asr( attackTime: 0.2,1571 releaseTime: 0.3 ),1572 gate: (analysisAmp_mic - 0.025)1573 .clip( lo: 0, hi: 1 )1574 );1575 1576 analysisSig = HPF.ar( analysisSig, freq: 92 );1577 1578 1579 1580 // signal to filter with LPC1581 sigToFilter = SoundIn.ar(input);1582 delay = SinOsc.ar( 0.01, mul: 2, add: 2.5 ).clip(1, 5);1583 sigToFilter = CombC.ar( sigToFilter, 5, delay, delay*1.1 );1584 // delay and verb act to "whiten" the signal just a bit1585 // sigToFilter = CombC.ar( sigToFilter,1586 // maxdelaytime: 0.1,1587 // delaytime: 0.1,1588 // decaytime: 11589 // );1590 sigToFilter = FreeVerb.ar( sigToFilter,1591 mix: 0.1,1592 room: 0.2,1593 damp: 0.11594 );1595 1596 // get the LPC output1597 // the analysis and filtering are both done in real time1598 lpcOutput = LPCAnalyzer.ar(1599 input: analysisSig,1600 source: sigToFilter,1601 n: (2**11), // window size1602 // p: 128, // number of polls1603 // p: 96, // number of polls1604 p: 10,1605 testE: 0, // test for error up to num of polls1606 windowtype: 1, /*triangular double window*/1607 mul: 0.4 // the output of this process tends to be1608 ); // quite hot, so scale down accordingly1609 1610 hpfOutput = HPF.ar( lpcOutput,

1611 freq: 801612 );1613 1614 1615 // A BandPass filter to add harmonic content as the piece progress1616 rms = LocalIn.kr( 1, 0 );1617 rms = ( rms - 0.1 );1618 // rms.poll( trig: 0.5, label: \bpfRMS );1619 bandPassQ = EnvGen.kr(1620 Env.new( [ 0.002, 0.4, 0.002 ], [ 10, 10, 10 ] ),1621 gate: rms.clip( 0, 1 )1622 );1623 bandPassQ = bandPassQ + Line.kr( 1, 0, 45 );1624 bandPassQ = bandPassQ + SinOsc.kr(1625 SinOsc.ar(0.1, mul: 0.1, add: 0.1),1626 mul: 0.3,1627 add: 0.251628 );1629 bandPassQ = bandPassQ.clip(0.002, 1);1630 bandPOutput = Resonz.ar( hpfOutput,1631 freq: filtFreq.lag(filtLag),1632 bwr: bandPassQ1633 )*1.1;1634 // filtLag.poll( trig: 0.5 );1635 // bandPassQ.poll( trig: 0.5 );1636 1637 analysisAmp = RunningSum.rms( bandPOutput, numsamp: 4096*2 );1638 // analysisAmp.poll( label: \lpcRMS);1639 limitedOutput = bandPOutput1640 * EnvGen.kr(1641 Env.asr( 0.01, 1, 0.01 ),1642 gate: (analysisAmp.lag(0.0011) - 0.008).clip( 0, 1 )1643 );1644 1645 limitedOutput = CompanderD.ar( limitedOutput,1646 thresh: 0.4,1647 slopeBelow: 0.8,1648 slopeAbove: 0.8,1649 clampTime: 0.005,1650 relaxTime: 0.2,1651 mul: 11652 );1653 1654 1655 1656 1657 1658 # quarter, eigth, sixt, tempo = BeatTrack.kr( FFT( LocalBuf( 1024, 1 ), inputSig ));1659 // onsetSig = quarter.lag2(0.01, ((1/tempo) * (1-analysisAmp) ) );//.poll;1660 // onsetSig = quarter.lag2(0.01, ((60/tempo) * 0.9 ) );//.poll;1661 1662 1663 onsetSig = ( 0.4 + onsetSig.lag2(1664 analysisAmp.clip(0.001, 1).explin(0.001, 0.2, 0.4, 0.01).clip(0.01, 0.4), //.poll(

label: \onsetRise),1665 (( (60/(tempo*60) )*0.8)1666 + (1 - (analysisAmp*10)).linexp(0.00000001, 1, 0.2, 1)1667 ).clip(0.3, 2).lag(0.001) //.poll( label: \onsetFall)1668 )1669 ).clip(0.4, 2);1670 1671 // tempo.poll;1672 // onsetSig.poll( label: \onsetSig );1673 1674

1675 cutSig = (1 - Decay.kr( quarter.lag2(0.01, 0),1676 (analysisAmp_mic*5).lag(0.1, 1) )1677 ).clip(0, 1);1678 // cutSig.poll( label: \cutSig );1679 1680 1681 1682 1683 limitedOutput = Limiter.ar(1684 limitedOutput ,1685 level: 0.98,1686 dur: 0.21687 // )*env * onsetSig.lag2(0.05, (1-analysisAmp) );1688 )*env * 0.7 * onsetSig * cutSig; // this is the 'finishing' env1689 1690 1691 1692 // TRACK THE RMS1693 // if it is too high for too long, then knock down for a while1694 // (self-regulating limiter)1695 rms = RunningSum.rms( limitedOutput, numsamp: 2048 ).lag( 20, 5 );1696 LocalOut.kr( rms );1697 // (rms - the rmsLimit that is not to be crossed)1698 rms = (rms - attenuationThresh.lag(10, 0) );1699 // rms.poll( label: \lpcRMSder );1700 // a limiter env1701 envLimit = EnvGen.kr(1702 Env.new( [1, 0, 0, 1], [2, 40, 8]),1703 gate: rms.clip( 0, 1 )1704 );1705 SendTrig.ar( rms, id: 10, value: instance );1706 // envLimit.poll( label: \envLimit);1707 1708 limitedOutput = limitedOutput * envLimit;1709 1710 outPos = (outPos*2)/~maxCurrentSynths;1711 outPos = SinOsc.kr( (rateSig*0.05).abs.clip(0.0001, 4), mul: 0.5, add: outPos );1712 // outPos.poll(label: "out position");1713 1714 sig = PanAz.ar(1715 numChans: ~numOfSpeakers,1716 in: limitedOutput,1717 pos: outPos,1718 width: 2.1,1719 orientation: 0.21720 );1721 1722 Out.ar( out, sig );1723 // Out.ar( 1, inputSig*0.2 );1724 1725 // free the synth when done1726 // FreeSelfWhenDone.kr(env);1727 1728 }).add;1729 );1730 1731 // identify a new "segment" by large differences between MFCC sets at Onset Moments1732 ~lpcAttenuationEnvMsg = OSCFunc( {1733 arg msg, time, addr, recvPort;1734 var id, instance;1735 1736 // pull the '/tr' id1737 id = msg[2];1738 instance = msg[3];1739

1740 if( id==10, {1741 1742 // POSTER1743 // "".postln;1744 // "".postln;1745 // "".postln;1746 // ("Synth "++instance++" has had the attentuation env triggered").postln;1747 1748 // alter the grainsearch wait time1749 Task({1750 ~curSynths[instance.asInt].waitTime = 20;1751 4.wait;1752 ~curSynths[instance.asInt].waitTime = 0.1;1753 }).start;1754 });1755 },1756 '/tr', // the message to match1757 s.addr // get messages from the server on this machine1758 );1759 1760 SynthDef(\grainPB, {1761 arg startFrame = 0, dur = 0.6, level = 1, bufnum = 0,1762 pitchShift = 1, out = 0, rmsSig = 0;1763 var sig, env, rms;1764 1765 rms = (In.ar(rmsSig, 1)*2);1766 1767 // env = Env.sine( dur, level );1768 env = Env.new(1769 [ 0, 0.5, 1, 0.5, 0, 0 ],1770 [0.25, 0.25, 0.25, 0.25, 0.25],1771 [1, -1, 1, -1, 0]*51772 );1773 env = EnvGen.kr( env, levelScale: level, timeScale: dur, doneAction: 2 );1774 1775 sig = PlayBuf.ar(1776 numChannels: 1,1777 bufnum: bufnum,1778 rate: 1,1779 trigger: 1,1780 startPos: startFrame,1781 loop: 1,1782 doneAction: 21783 ) * 0.707;1784 1785 // pitch shift1786 sig = PitchShift.ar( sig,1787 windowSize: 0.023,1788 pitchRatio: pitchShift,1789 pitchDispersion: 0,1790 timeDispersion: 0.0011791 );1792 1793 sig = sig * ((1-(rms/2))*0.2);1794 1795 sig = Limiter.ar( sig, 0.95, 0.1 );1796 1797 sig = sig * env;1798 1799 sig = HPF.ar( sig, freq: 80 );1800 1801 1802 1803 sig = PanAz.ar(1804 numChans: ~numOfSpeakers,

1805 in: sig,1806 pos: Line.kr( Rand( 0.0, 2.0 ), Rand( 0.0, 2.0 ), dur ),1807 width: Line.kr( Rand( 1.0, 3.0 ), Rand( 1.0, 3.0 ), dur ),1808 orientation: 0.21809 );1810 1811 1812 1813 Out.ar( out, sig );1814 1815 }).add;1816 1817 1818 ~sysOut = SynthDef(\output, {1819 arg out = 0, mainIn = 0, limiterLevel = 0.95;1820 var sig, outSig;1821 1822 sig = In.ar( mainIn, ~systemOutputBus.numChannels );1823 outSig = Limiter.ar( sig, limiterLevel, dur: 0.1 );1824 1825 // NYU 6th floor Immersive Lab playback1826 // Out.ar( 2, HPF.ar( sig[0], freq: 100 ) );1827 // Out.ar( 3, HPF.ar( sig[1], freq: 100 ) );1828 // Out.ar( 4, HPF.ar( sig[2], freq: 100 ) );1829 // Out.ar( 5, HPF.ar( sig[3], freq: 100 ) );1830 // Out.ar( 6, HPF.ar( sig[4], freq: 100 ) );1831 // Out.ar( 7, HPF.ar( sig[5], freq: 100 ) );1832 // // sub split1833 // Out.ar( 8, Limiter.ar(1834 // in: LPF.ar( HPF.ar( Mix.new( sig )*0.08, freq: 20 ), freq: 100 ),1835 // level: 0.951836 // )1837 // );1838 1839 Out.ar( 2, HPF.ar( sig[0], freq: 30 ) );1840 Out.ar( 3, HPF.ar( sig[1], freq: 30 ) );1841 Out.ar( 4, HPF.ar( sig[2], freq: 30 ) );1842 Out.ar( 5, HPF.ar( sig[3], freq: 30 ) );1843 Out.ar( 6, HPF.ar( sig[4], freq: 30 ) );1844 Out.ar( 7, HPF.ar( sig[5], freq: 30 ) );1845 1846 // headphone mix via main out on MOTU1847 Out.ar( 0, Limiter.ar(1848 in: Mix.new( [sig[0], sig[2], sig[4] ]),1849 level: 0.951850 )1851 );1852 Out.ar( 1, Limiter.ar(1853 in: Mix.new( [sig[1], sig[3], sig[5] ]),1854 level: 0.951855 )1856 );1857 1858 // // traditional stereo playback1859 // Out.ar( 0, outSig[0] );1860 // Out.ar( 1, outSig[1] );1861 // if( outSig.size > 2, {1862 // Out.ar( 2, outSig[2] );1863 // Out.ar( 3, outSig[3] );1864 // });1865 1866 // for recording1867 // Out.ar( 12, sig[0] );1868 // Out.ar( 13, sig[1] );1869 // if( sig.size > 2, {

1870 // Out.ar( 14, sig[2] );1871 // Out.ar( 15, sig[3] );1872 // });1873 1874 1875 1876 1877 }).play( args: [\mainIn, ~systemOutputBus], addAction: \addToTail);1878 1879 1880 1881 1882 1883 1884 1885 1886 // display MFCC and slope values in real-time1887 // SynthDef(\valuesViz,{1888 // var mfccSlider, mfccSlider2, slopeSlider, fluxSlider, myWindSize, updateTask;1889 //1890 // // create a window1891 // ~mfccWin = Window.new( ~numCoeff + "values (MFCC, slope)",1892 // Rect(10, 600, 500, 840));1893 //1894 // // create a multi slider to display values1895 // mfccSlider = MultiSliderView.new(~mfccWin,1896 // Rect(2, 2,1897 // 400,1898 // 4001899 // ));1900 // // create a multi slider to display values1901 // mfccSlider2 = MultiSliderView.new(~mfccWin,1902 // Rect(2, 402,1903 // 400,1904 // 4001905 // ));1906 //1907 // slopeSlider = MultiSliderView(~mfccWin,1908 // Rect(410, 2, 30, 400));1909 //1910 // fluxSlider = MultiSliderView(~mfccWin,1911 // Rect(450, 2, 40, 400));1912 //1913 // // initialize values1914 // mfccSlider.value = (Array.fill(~numCoeff , 0.0));1915 // mfccSlider.size_(13);1916 // set window properties1917 // mfccSlider.elasticMode = 1;1918 // mfccSlider.valueThumbSize_(5.0);1919 // mfccSlider.indexThumbSize_(20.0);1920 // mfccSlider.gap_(0);1921 // mfccSlider2.value = (Array.fill(~numCoeff , 0.0));1922 // mfccSlider2.size_(13);1923 // set window properties1924 // mfccSlider2.elasticMode = 1;1925 // mfccSlider2.valueThumbSize_(5.0);1926 // mfccSlider2.indexThumbSize_(20.0);1927 // mfccSlider2.gap_(0);1928 // slopeSlider.elasticMode = 1;1929 // slopeSlider.valueThumbSize_(5.0);1930 // slopeSlider.indexThumbSize_(30.0);1931 // slopeSlider.gap_(0);1932 // slopeSlider.value = [0.0];1933 // slopeSlider.size_(1);1934 // fluxSlider.elasticMode = 1;

1935 // fluxSlider.valueThumbSize_(5.0);1936 // fluxSlider.indexThumbSize_(20.0);1937 // fluxSlider.gap_(0);1938 // fluxSlider.value = (Array.fill(2, 0.0));1939 // fluxSlider.size_(2);1940 // slopeSlider.drawLines = true;1941 //1942 // // push it to the front1943 // ~mfccWin.front;1944 //1945 // // this updates the visulization1946 // updateTask = {1947 // inf.do{1948 // ~mfccBus.getn(1949 // ~numCoeff, {1950 // arg val;1951 // { mfccSlider.value_(val * 0.8) }.defer1952 // });1953 // ~smoothedMFCC.getn(1954 // ~numCoeff, {1955 // arg val;1956 // { mfccSlider2.value_(val * 0.8) }.defer1957 // });1958 // ~slopeBus.get({1959 // arg val;1960 // slopeSlider.value = [val/4000 + 0.5];1961 // });1962 // ~fluxBus.getn(1963 // 2, {1964 // arg val;1965 // { fluxSlider.value_(val / 5) }.defer1966 // });1967 // 1.wait; // 10 frames per second1968 // };1969 // }.fork();1970 // });1971 1972 // plot properties1973 (1974 GUI.skin.plot.gridLinePattern = FloatArray[1, 0];1975 GUI.skin.plot.fontColor = Color.white;1976 GUI.skin.plot.gridColorX = Color.yellow(0.5);1977 GUI.skin.plot.gridColorY = Color.yellow(0.5);1978 GUI.skin.plot.background = Color.black;1979 GUI.skin.plot.plotColor = (10..0).normalize(0.1, 1).collect { |i| Color.rand(i) };1980 GUI.skin.plot.gridLineSmoothing = true;1981 GUI.skin.plot.gridFont = Font( "Source Code Pro", 14 );1982 );1983 1984 CmdPeriod.doOnce({1985 // ~mfccWin.close;1986 // clear the audio buffers out1987 // ~audioArray.do({|val, id| if(~audioArray[id].notNil, {~audioArray[id].free;

~audioArray[id]=nil;}); });1988 // ~slopeArray.do({|val, id| if(~slopeArray[id].notNil, {~slopeArray[id].free;

~slopeArray[id]=nil;}); });1989 ~audioWindow1 = nil;1990 ~pitchPlot = nil;1991 ~slopeWindow1 = nil;1992 // ~kbuf_mfcc.loadToFloatArray(1993 // action: {|array|1994 // "mfcc classifier values".postln;1995 // array.postln;1996 // array.do({|val| val.postln;})1997 // }

1998 // );1999 "Time running was: ".post; ~time.postln;2000 });2001 2002 2003 )2004 2005 2006 2007 2008 2009 2010 /******************************************************************************/2011 // Things for development2012 ~classyCountGoal = 2;