making high performance embedded instruments with bela...
Post on 23-Mar-2018
215 Views
Preview:
TRANSCRIPT
MakingHigh-PerformanceEmbeddedInstrumentswithBelaandPureData
GiulioMoro1,AstridBin2,RobertH.Jack3,ChristianHeinrichs4,AndrewP.McPherson5
CentreforDigitalMusic,QueenMaryUniversityofLondon,UK
1g.moro@qmul.ac.uk,2a.bin@qmul.ac.uk,3r.h.jack@qmul.ac.uk,4c.heinrichs@qmul.ac.uk,5a.mcpherson@qmul.ac.uk
Abstract.Belaisanembeddedplatformforultra-lowlatencyaudioandsensorprocessing.WepresentherethehardwareandsoftwarefeaturesofBelawithparticularfocusonitsintegrationwithPureData.SensorinputsonBelaaresampledataudiorate,whichopenstothepossibilityofdoingsignalprocessingusingPureData’saudio-rateobjects.
Keywords:embeddedaudio,sonicinteractiondesign,sensors,lowlatency,musicalinstrumentdesign.
IntroductionTheincreasingpowerandavailabilityofmicrocontrollersandsingle-boardcomputershasgivenrisetomanynewplatformsforcreatingmusicalinstrumentsandplatformsforinteractiveaudio.Choosingasuitableplatformcanbeachallenge,involvingtradeoffsbetweencomputingpower,hardwareconnectivity,easeofprogrammingandprice.
Manycurrentapproachestodesigningsoundingobjectscombinetwoormoredevicestogether,forinstanceanArduinowhichhandlesanaloganddigitalsensorinputcommunicatingviaUSB-serialwithaacomputerrunningtheaudioprocessing.Usingaself-containedembeddedplatforminthecreationofDMIsandinteractiveaudiosystemshasseveraladvantagesoversuchasetup.
• ReliabilityUsingasingledeviceislesspronetocommunicationerrorsanditiseasiertoprovideabackupsolutionforasimplersystem.
• PerformanceWithacompositesetupastheonedescribedabove,theserialconnectionisslowandthethroughputislimited.Assuch,thelatency,samplingrateandjitteroftheacquireddataareallaffectednegatively,whichmayinturnaffecttheexpressivenessoftheperformance.MIDIdevicestypicallyperformbetterthanserialones,whilewirelesslinksmaybeaffectedbypacketlossorchannelcongestion(McPherson,Jack,andMoro2016).
• ReproducibilityItiseasierforotherpeopletorecreateadeviceifitdoesnotrelyonmultiplepiecesofsoftwareandhardwaredevicesandspecificrevisionsofeachofthem.
• SustainabilitySimilarly,thedevelopersthemselveswillfinditeasiertomaintainanddevelopasystemthatdoesnothavemultipledependencies,alsototheadvantageofmakingsoftwareversioncontroleasier.
RecentEmbeddedPlatformsforDigitalMusicalInstrumentCreationArduinoandsimilarboardsareanaccessiblewayofprovidinglow-levelconnectivitytoanaloganddigitalsensors,butthelow-poweredAVRmicrocontrollerdoesnotallowaudioon-boardaudioprocessing.Thex-OSCboardprovidesanaloganddigitalI/Osoverawirelesslink(MadgwickandMitchell2013).
Twoaudio-oriented,self-containedplatformsbasedona168MHzCortexM4microcontrollerhitthemarketinthepastfewyears:theOwl1programmabledigitaleffect(Webster,LeNost,andKlang2014),whichsurfacedin2013,andAxoloti2,which
1http://hoxtonowl.com/2http://www.axoloti.com/
cameoutearly2015.TheformercanbeprogrammedthroughaC++APIorcanrunPureDatapatchesusingtheHeavyAudioToolsfromEnzienAudio,3whilethelatterprovidesacustomgraphicalpatcherwhichincludesDSPmodulesandcanbeexpandedwithC++.
RaspberryPiisarguablythemostpopularsingle-board-computerintheworldanditslatestrevision3featuresaquad-core1.2GHz64bitCPU.TheCCRMASatellitedistribution4(BerdahlandJu2011)wasdevelopedtoprovideanefficientaudio-orientedenvironmentfortheRaspberryPi.
CoalaisanaudioprocessingplatformbasedontheBeagleBoneBlack5whichwaspresentedin(Piéchaud2014).ThesoftwareandhardwarearchitectureofCoalaweredevelopedforthespecifictaskofmodalcontrol6,whichrequiresaverytightfeedbackloop.Theplatformisthereforeoptimizedforfastsample-by-sampleprocessinginordertominimizeround-triplatency.
Bela:anembeddedplatformforaudioandsensorprocessingBela7(formerlyknownasBeagleRT)isacombinedhardwareandsoftwareenvironmentthatconsistsofaBeagleBoneBlackwithanexpansion“cape”(McPhersonandZappi2015a).ItwasoriginallydevelopedfortheD-BoxHackableDigitalInstrument(ZappiandMcPherson2014)whichrequiredmultiplelow-latencyhybridanalog-digitalfeedbackloops(McPhersonandZappi2015b).Belacombinestheconnectivityofamicrocontrollerwiththeprocessingcapabilityofasingle-boardcomputer.ThecapeprovidesstereoaudioI/Oincluding1Wspeakeramplifiers,8channelseachof16-bitanalogI/O,and16digitalGPIOpins.Belaisopen-sourcehardwareandsoftware.Sourcecodeanddesignmaterialsarepubliclyavailable.8
TheBelasoftwareusesaDebianLinuxdistributionwiththeXenomai9real-timekernelextensions.TheProgrammableRealtimeUnit(PRU),a200MHzmicrocontrolleronthesamechipastheBeagleBoneBlackCPU,transfersaudioandsensordatadirectlytothehardware,bypassingthekerneldrivers.Theuser’sBelacodethereforerunsatthehighestpriorityofanytaskontheboard,includingtheLinuxkernelitself.Thisallowsaudioblocksizesaslowas2samples,resultinginround-tripaudiolatencyof1ms(orevendownto100usifusingtheanaloginputsandoutputsratherthantheaudioconverters)(McPherson,Jack,andMoro2016).
OnBela,everyanaloganddigitalchannelisautomaticallysampledataudiorates,synchronouslywiththeaudioclock.ThehighsamplingrateoftheanaloganddigitalchannelsareuniquetoBelaandtheirjitter-freealignmentwiththeaudiomakesitidealforinteractive,intuitive,responsiveaudioapplications.
ComparedtoAxolotiandOwl,Belahasmoreprocessingpower,whilestillprovidinghardreal-timeperformances,withtheaddedconvenienceofafullLinuxOSandwhilebeingminimallyaffectedbysystemload.Itismoregeneral-purposethanCoalawhichaddressesthespecificfieldofreal-timecontrol,thoughCoalaiscapableofevenlowerlatenciesthanBela.OnaRaspberryPirunningCCRMASatellite,despitethehighprocessingpoweravailableonboard,audiodependsonthestandardLinuxaudiodrivers,sothatlow-latencyprocessingisdifficultbecauseofthepresenceofotherprocessesontheboard,whichmaycauseunderrunsatsmallaudioblocksizesevenwhentheCPUloadislowonaverage.TheBeagleBoneBlackCPUislesspowerfuloverallbuttheXenomaiextensionsusedintheBelasoftwareallowreliableandconsistentperformancewithsub-millisecondlatency.Additionally,thenumberofI/OsavailableinBelaisgreaterthanthoseoncommonlyavailableRaspberryPihats.
3http://enzienaudio.com/4https://ccrma.stanford.edu/~eberdahl/Satellite/5http://beagleboard.org/black6http://instrum.ircam.fr/smartinstruments/7http://bela.io8http://bela.io/code/9http://xenomai.org/
Bela,providingalargenumberofI/Osforaudioandsensors,poweroutputforloudspeakers,andprovidingenoughprocessingpowertosatisfymostneeds,entirelyfulfillstherequirementsofaself-containeddevice,whichcanbeembeddedinastand-aloneDigitalMusicalInstrumentorsoundingobject.
PureDataonBelaPureData10(Pd)isapopularopensourcegraphicalprogramminglanguagewidelyusedbymusiciansandsounddesignersalike,whichallowsforquickprototypingofsoundandsensormappings.PdpatchesareusuallyrunwithinPditself,orusingthesharedlibrarylibpd11.ThemessagingarchitectureandtheaudioengineofPdwasnotdesignedtobefastandcomputationallyefficientwhichcanleadperformancepenaltiesonplatformswithlimitedcomputationalpower.
�� Figure1:TheBelacape Figure2:UsingaforcesensitiveresistorwithBela
HeavyAudioToolsTheHeavyAudioToolsfromEnzienAudiousePdasafront-endtogenerateoptimisedCcode.ByanalyzingthegraphofconnectionsbetweenobjectsinthePdcode,Heavyiscapableofproducinghigh-performancevectorizedCcodewhichcanoutperformlibpd,makingitparticularlywellsuitedforembeddeddevicesand,moregenerally,hardwarewithlimitedcomputationalpower.Heavyisaproprietary,cloud-basedserviceandthegeneratedcodeislicensedundertheMITnon-commerciallicense.
TheCcodeproducedbyHeavyiswell-suitedtobeintegratedinaXenomaienvironment,asmemoryisallocatedonthestack,thusavoidingsystemcallsduringexecution.AnautomatedscripttakescareofuploadingthePdpatchtoHeavy’sserver,collectthegeneratedCcodeandcompileitontheBelaboard.TheentireprocessgenerallytakeslessthanoneminuteandmostofthetimeisspentcompilingtheCcodeontheBeagleBoneBlack.
libpdMinimalmodificationswererequiredtoportlibpdforBela,theseincludedallowingblocksizesassmallas8samplesperblockandremovingsocketanddiskI/Ofromtheaudiothread.Additionally,thecallstothepthreadfunctionswerewrappedintoXenomaifunctions.TheresultingsharedlibrarycanbelinkedtoaBelaprogramandlibpd_process_float()istheninvokedfromwithinBela’saudiocallback.
DeployingaPdpatchusinglibpdisvirtuallyinstantaneousasitdoesnotrequirecompiling.AssoonasthepatchissavedontheBeagleBone’sfilesystem,theBelaprogramcanberestartedanditwillloadtheupdatedpatch.AnaddedadvantageofusinglibpdisthatitiseasiertoportPdexternalswhentheirsourcecodeisavailable.ThesameprecautionslistedaboveshouldbetakenfornewexternalsinordertomakesurethatnewobjectsdonotintroduceXenomaimodeswitchesintheaudiothread12.
10http://puredata.info11http://libpd.cc/12https://xenomai.org/2014/08/porting-a-linux-application-to-xenomai-dual-kernel/
PerformancecomparisonRunninganexamplepatchcontainingagenerativeaudiocomposition,Heavycodecompiledwiththeclang13compileruses26%oftheCPU.ThesameHeavycode,compiledwithgcc14,occupies43%oftheCPUcycles.Runningthepatchusinglibpduses53%oftheCPU.
Traditionally,thehighest-performanceplatformshavealsoplacedthemosttechnicaldemandsontheprogrammer.Formanyyears,customDSPboardsofferedthebestbalanceofhardreal-timeperformanceandhighprocessingpower,buttheyweregenerallyprogrammedinlow-levellanguagesusingcustomdevelopmentenvironments.High-levelmusicprogramminglanguagesoftencomewithsignificantprocessingoverhead.RunningPureDataonBela,especiallythroughtheHeavyAudioTools,providesaconvenientgraphicalenvironmentwithminimalsacrificeinperformancecomparedtoprogramminginC++.
��(a)Smoothing(b)Re-centering
(c)Differentiating (d)Thresholding
Figure3:UsingPureDataobjectstoprocesssensordata
SensorprocessinginPureDataManyinteractivesystemstakeapproacheswheresensorsaresampledatlowandnon-constantratesandthemostrecentframeofsensordataisusedtomodulateaparticularsonicparameter.Butinactualfact,themeaningofsensordataisoftendeeper,initsbehaviourovertimeoritsfrequencycontent.Ahighsamplingrateyieldsaveryhighbandwidthofinteractionwhichcapturessubtledetailsthatmightbelostatlowersamplerates.Thoughallthesametechniquescouldbeimplementedatcontrolrate,audio-ratesensordatacanhelpreorientthedesigner’sthinkingtobecomemoreawareofthesepossibilities.WhenusingBelawithPd,thisallowstoconvenientlyprocesssensorsignalsusingaudio-rateobjects.Someexamplesinclude:
• SmoothingSomesensorsareinherentlynoisy,forinstanceapotentiometermaygeneratehigh-frequencynoisewhenitisactuated,oraninfra-redopticalsensormaybesubjecttotransientperturbationsfromotheremittingsources.Thenoiseinthesensorreadingsmayleakintotheaudiosignal,dependingonthesignalflow.Aneasyapproachtoremovehigh-frequencynoiseistoapplyalow-passfilterwithanappropriatecut-offfrequency,asinFigure3a.
13http://clang.llvm.org/14https://gcc.gnu.org/
• Re-centeringReadingsfromaccelerometersandothersensorshaveinherentDC-offsetswhichmaybeundesirableforcertainapplications.Aquickwayofremovingthemwhichdoesnotrequirecalibrationisusinganhigh-passfilterwithanappropriatecut-offfrequency,asinFigure3b.
• DifferentiatingSomesound-generatorparametersarebettercontrolledusingthevelocityofasensorreading,ratherthanwiththerawreading.Ahigh-passfilterwithacut-offfrequencyof0,properlyrescaledcanusedforthispurpose,asinFigure3c.
• ThresholdingAmorecomplicatedexampleinFigure3dshowshowtocombinefull-waverectification,smoothing,DCshiftandconstraintothresholdasignal.
ConclusionThereareseveraltradeoffsinvolvedindifferentdigitalmusicalinstrumentdesigntools:processingpower,latency,connectivity,sensorbandwidth,easeofprogrammingandaccessibility.Withanyoftheprogrammingenvironments,BelabringstogethertheconnectivityandCPUpowerofanembeddedLinuxcomputerwiththelowlatencyandprecisesynchronisationofamicrocontrollerandbringsahigh-bandwidthdimensiontosensorprocessing.UsingeithertheHeavyorthelibpdenvironments,Belaisalsosuitableforrapidprototypingusingthewidely-usedPureDatagraphicalprogramminglanguage,withfullaccesstobothaudioandsensors.
Acknowledgements.ThisworkissupportedbygrantsEP/K032046/1,EP/K009559/1andEP/L019981/1fromtheUKEngineeringandPhysicalSciencesResearchCouncilandtheQueenMaryCentreforPublicEngagement.
ReferencesBerdahl,Edgar,andWendyJu.2011.“SatelliteCCRMA:AMusicalInteractionandSoundSynthesisPlatform.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Oslo,Norway,173–178.
Madgwick,Sebastian,andThomasJMitchell.2013.“x-OSC:AversatilewirelessI/Odeviceforcreative/musicapplications.”SMCSoundandMusicComputingConference.Stockholm,Sweden:KTHRoyalInstituteofTechnology.
McPherson,A.P.,andV.Zappi.2015a.“Anenvironmentforsubmillisecond-latencyaudioandsensorprocessingonBeagleBoneBlack.”AudioEngineeringSocietyConvention138.AudioEngineeringSociety.
McPherson,Andrew,andVictorZappi.2015b.“ExposingtheScaffoldingofDigitalInstrumentswithHardware-SoftwareFeedbackLoops.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.BatonRouge,Louisiana,USA:LouisianaStateUniversity,162–167.
McPherson,AndrewP.,RobertH.Jack,andGiulioMoro.2016.“Action-SoundLatency:AreOurToolsFastEnough?”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Brisbane,Australis.
Piéchaud,Robert.2014.“ALightweightC++Real-TimeActiveControlFramework.”16thRealTimeLinuxWorkshop,October12to13,2014attheCCDCongressCenterDusseldorfcollocatedwithLinuxConEuropeinDusseldorf,Germany.
Webster,Thomas,GuillaumeLeNost,andMartinKlang.2014.“TheOWLprogrammablestageeffectspedal:Revisingtheconceptoftheon-stagecomputerforlivemusicperformance.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.London,UnitedKingdom:Goldsmiths,UniversityofLondon,621–624.
Zappi,V.,andA.McPherson.2014.“DesignandUseofaHackableDigitalInstrument.”ProceedingsoftheInternationalConferenceonLiveInterfaces.Lisbon,Portugal,208–219.
top related