hacking the body 2.0: flutter/stutterusers.sussex.ac.uk/~thm21/icli_proceedings/2016/... · hacking...
TRANSCRIPT
HackingtheBody2.0:Flutter/StutterKateSicchio1,CamilleBaker2,TaraBaothMooney3,andRebeccaStewart4
1ParsonsTheNewSchoolforDesignNewYork,NewYork,[email protected]
2UniversityfortheCreativeArtsEpsom,Surrey,UK
Wolverhampton,[email protected]
4QueenMaryUniversityofLondonLondon,UK
Abstract.Flutter/Stutterisanimprovisationaldancepiece,partoftheHackingtheBody2.0project,thatusesnetworkedsoftcircuitsensorstotriggersoundandhapticactuatorsintheformofasmallmotorthatticklestheperformers.Dancersembodytheflutterofthemotorandrespondwiththeirownmovementthatreflectsthisfeeling.Thisresearchexploresusingtheconceptofhackingdatatorepurposeandre-imaginebiofeedbackfromthebody.Itinvestigatesunderstandingsofstatesofthebodyandhackingthemtomakenewartworkssuchasperformanceandcostumes.Throughperformanceweaimtocommunicatetothepublicnewwaystoengagewiththeirbodiesandtechnologywithintimacyandsensationembeddedinwearables.
Keywords:Wearabletechande-textiles,Performance,Sustainablegarments,Ethicaldatacollection.
IntroductionTheHackingtheBody2.0projectexploresusingtheconceptofhackingdatatorepurposeandre-imaginebiofeedbackfromthebody.Itinvestigatesunderstandingsofstatesofthebodyandhackingthemtomakenewartworkssuchasperformanceandcostumes.Throughperformanceweaimtocommunicatetothepublicnewwaystoengagewiththeirbodiesandtechnologywithintimacyandsensationembeddedinwearables.
Flutter/StutterisonepiecewithinthelargerHackingtheBody2.0researchproject.Assuch,dancersembodytheflutterofthemotorandrespondwiththeirownmovementthatreflectsthisfeeling.Thesensorsandactuators,alongwiththegarmentstheyareembeddedwithin,arebespokedesignsbyBeckyStewartandTaraBaothMooneythatinteract,influenceandinterruptthedanceandhackthebody.
ContextThecurrenttechnologyfervouroverwearabletechnologythatcollectsusers’intimatebodydata,underthepretenseofmedicalorfitnessmonitoring,highlightsthatitistimethatcriticalquestionswereraised.Theethicsofcorporateownershipofbodydataforconsumeristagendasisrarelydiscussedbeyondthefineprintonthesedevices.Moreawarenessandeducationontheseissues,wouldpotentiallyallowmoreaccess,ownership,andcreativityintheuseofone'sownbodydata,andwaystoexpresspersonalidentitythroughthisdata.
Thisprojectquestionshowbodydatamaybeabletodemonstratewhoweare,throughmovement,throughourphysiology.Howdoesaccesstopersonaldataenabletheperformertoshowtheiridentity,ratherthanwhatissubscribedbythecorporationmakingthesensingdevice?Howcanweexploretheseissueswhileenablingpeopleaccesstotheirowndata,especiallyinperformancecontexts,inordertointeractwithit?
ThenewiterationofthecollaborativeprojectHackingtheBody2.0bringsperformerstogethertoattempttoaddresstheethicalissuesaroundidentityanddataownershipwhenusingwearabletechinperformance.Theprojectdevelopsmethodstouseandhackcommercialwearabledevices,aswellasmakinghandmadee-textilessensingdevicesforperformance.Assuch,weaimtoengageperformerstoaccesstheirownphysiologicaldataforpersonaluse,butalsotocreateuniqueandinteractiveperformances.
PerformanceInvestigationsThelatestiterationofthecollaborativeprojectHackingtheBody2.0,bymediaartistCamilleBakerandmediaartist/choreographerKateSicchio,attemptedtoaddresstheethicalissuesaroundidentityanddataownershipwhenperformersusewearabletechinperformance.Theprojecthasusedvariousmethodstouseandhackcommercialwearabledevices,aswellasmakinghandmadee-textilessensingdevicesforperformanceaswasdoneforFlutter/Stutter.Assuch,weaimtoengageperformerstoaccesstheirowndataforpersonaluse,butalsotocreateuniqueandinteractiveperformances.Thiscollaborationhasbeenevolvingsince2011,andtheapproachisinfluencedbypreviousprojectsinasimilarvein,suchasthatofTheclaSchiphorstandSusanKozel’swhisper(s)project,aswellasBaker’sandSicchio’sindividualPhDresearchprojects,nottomentionthehaptic,biofeedbackandwearablemusicperformanceworks,suchasthosesuchasLoke,DonnarummaandTanaka,andmanyothersnowworkinginthisfield.YetthispieceanditspartnerperformancepieceFeelMe(alsopartofHackingtheBody2.0,butnotdiscussedhere),stakesoutasingularnewterraininitsexplorationofwaystoaddressethicalissuesofdatacollection,useofthetechnologiestorepresentpersonalidentitiesofdancers,developnon-verbalcommunicationinteractionmethodsandincorporatingalivecodingofthedancersbythechoreographerintothework.
Thegoalisto:enabledancerstointeractorrespondtowornsensorsandactuators;toinstigatenewmovement‘dialogue’orinteractionbetweenperformers;andtoexploretheiridentities.Byusingthetechnologiesdevelopedforgatheringpersonaldata,butcircumventingcorporatedatacollection,wefacilitatedirectcommunicationbetweeneachbody/dancertocreateaconversation.Atthesametime,thechoreographercanalsointervenedirectlywithandparticipateinthedialoguebetweendancers’bodiesandtheirmovementresponses,bytriggeringthechimesdirectlyforthecomputercontrollingtheinteraction.Inthisway,theperformersreclaimthedatasensingandcollectionbyusingthetechnologyasanothertooltothemhelpdevisemovementandco-createorchoreographperformanceworks.Thisworkputsintonewlight,theethicalissuesofcorporateownershipbyputtingtheownershipbackintothehandsoftheuser.Thisinturnmaybeconsideredacriticalactofmakingandconfrontationoftheissuesofsurveillanceanddatacontrol.
InFebruary2016,wewereabletotaketheprojecttotheperformancestage,withperformancesinLondonandSheffield.Flutter/Stutterwasperformedonceateachlocationinfrontofanaudience.Thechoreographicsideofthepiecefocusesontwotechnologicalaspects.Thesecanbeusedasmethodsforbothdevelopingmovementaswellasastructureforimprovisation.Firstly,thedancerswererespondingtoactuatorsontheircostumesandtoeachotherthroughmovement.Bymovinginreactiontotheseimpulsesontheirbody,therhythm,timingsanddynamicswereaffectedinafeedbackloop.Amoresubtleapproachtomovementwastakenbythedancers,toreflectthesensationofbeingtickledontheneckbyaribbononamotortriggeredbytheotherdancer.Therelationshipbetweendataandsubsequentreactivemovementsbytheperformersisthattouchontheshoulderpiecesandthesubsequentactuationresponsesaremeanttoinitiatemovementbythedancerreceivingthevibrationortickle,astheresponseinthe“callandresponse”non-verbaldialogue.Eachdanceralsohastotakeintoconsiderationhowtoinitiatethetriggeroftheactuationontheotherdancer,aspartofthe“conversation”betweenthem.
Thesecondwaythetechnologyisrepurposedintothechoreography,wasthroughthestructureoftheinteractiondesign.Thesystemwasdevelopedtoadaptandprovideseveralopportunitiesforuserinteractionsbythedancers.Thescoreforthedanceimprovisationwasthenstructuredaroundtimeandavarietyofthesedancerinteractions.Eachoneledtoadistinctmomentinthepieceandallowedtheoverallcompositiontobuildovertime.Theaudiencewasabletounderstandtheinteractionsnotonlythroughtheexpressivitybythedancersandthewayinwhichtheyhada‘callandresponse’behaviourwhenactivatingthedevices,butalsobythesoundsthegarmentsalsotriggered,tocreateaimprovisedsoundtrackofvariouslypitchedreverberantchimes.Theoperatingmodesofthesensorsand
actuatormaynotbeclearlydiscernedbytheaudiencethroughthesepitchedchimes,astheintentionofthepieceisnottomakeimprovisedmusic,buttoallowtheaudiencetounderstandthattheinteractionandtheactuationisliveandcreatedbythehaptics.Wechosetorestrictthesystemtouseasinglesensorandactuatoratthispointintheresearchduetofundingandtimelimitations,butintendtointroducemoresensor/actuatorsastheprojectprogresses,inordertoincreasetheperformancepotentialinfutureiterations.
WearablePerformanceSystemImplementationThesensorsandactuatorswerebuiltintogarmentswornononeshoulderofeachdancerandincorporatedintoacompletecostumewornduringperformance.Thenetworkconnectingthesensorsandactuatorsutilisedtheinfrastructuretools,emergingasthepreferredprotocolfor‘InternetofThings’applications,whilethevisualdesignofthegarmentsactivelydistanceditselffromatechnologicalaesthetic.
Figure1.ImagesfromperformanceofthepieceinFebruary2016inLondon,UK
GarmentDesignThegarments,whichhavebeencreatedforthisresearchproject,exploretheideaofmakinganddesigningforinteractiondesignthroughhacking.Allgarmentscanactasaninterfacebetweenindividualsandtheirimmediateenvironment.Thisappliestobothgarmentsforperformanceandgarmentsforeverydaywear.
Liketheacquisitionofclothingforoneoffevents,sotoogarmentswhicharecreatedforaspecificperformanceoreventcanberenderedobsoleteaftertheperformancehastakenplace.
Thiscollaborationpromptedquestionsaroundthecontinuedvalueofperformativegarmentsinaworldwhichincreasinglydevaluesobjects.Thechallengesforthegarmentdesignerinthisprojectweretocreateamodulargarmentwhichmightcontainwithinthemthepotentialtobecomesomethingelseaftertheperformanceshavetakenplace.Asecondarychallengewastoworkwiththeinteractiondesigner,RebeccaStewart,tocreateacollectionofgarmentswhichdidnotvisuallybetraythetechnologyembeddedwithinthem.
FortheFlutter/Stutterpiece,pre-existingcottont-shirtswereusedwhichwerealreadyownedbythedesigner,alongwithsomecastofflong-sleevedcottont-shirtsfoundinthriftstores,andwereconsideredasrawmaterialforanewmanifestationofthematerial.Theproductdevelopmenttookplaceovera4-monthperiod,wherethedesignerintermittentlyworkedwiththeinteractiondesignerandtheneachwithdrewtoworkalone.Thepiecesneededtobeflexibleenoughtomakesmallchangesifnecessary,andalsoneededtohaveasolidstructure.Thus,themodularelementsweresituatedabovethestructure.Theseelementsincludedthefabricsensorandembellishments,whichactedasthetiestosecurethegarmentsontothebody.
Asbothartistanddesignertheaimwas,throughadeepengagementwiththematerials,tocreatesomething,throughhacking,whichwasvisuallyorganicratherthantechnological,somethingwhichmovedwiththemovementofthosewearingthem,andfinallysomethingwhichwasinherentlymodularandre-purposable,andwhichcould
potentiallymanifestassomethingelseinthefuture.Itwasthroughthedestructionoftheoriginalgarmentsthatareconstructionofnewgarmentstookplace.
ElectronicsDesignThewearablecomputingsystemconsistsofacollectionofcapacitivetouchsensorswornononeshoulderofeachdancerandamotorwitharibbonattachedplacedneartheneck.Thesensorsandactuatoroneachdancerarecontrolledbyawifi-enabledmicrocontroller1.EachofthecapacitivetouchsensorelectrodesareasingleconductivethreadsewntotheundersideofafabricpleatasseeninFigure2.Theelectrodesareconnectedtoacapacitivesensingchip2whichhandlesthecalibrationandmeasurementofthesignals.
Figure2.ImagefromarehearsalinSheffield,UKofthedancersinteractingwiththetextiletouchsensorsembeddedinthepinkpleatedfabric.
Theactuatorisasmallmotorwithanintegratedplanetarygearbox.Themotor’scircuitisconstructedfromconductivefabricmountedonapatchofblackleatherseeninFigure3.Afabricribbonisconnectedtothemotorsothatitrotatesandbrushesagainstthewearer’sneck,causingaticklingsensation.Forthisreason,theactuatorisreferredtoasa‘ticklemotor’.
Themicrocontrollersconnectedtothesensorsandactuatorspublishmessagestothenetworkwheneverasensorisactivatedbyatouch-byeitherthewearerortheotherdancer.Themicrocontrollersalsosubscribetoafeedtobenotifiedwhentheirmotorshouldbeturnedontorotatetheribbon.ThissystemofpassingaroundmessagesisimplementedusingtheMessageQueueingTelemetryTransport(MQTT)protocol.Aprogramcalledabroker3isrunonaserverconnectedtothesamelocalwifinetworkasthemicrocontrollers.Aseparateprogramthatactsasanotherclientofthebroker,listenstotheincomingsensormessagesandredirectsthemtothecorrespondingactuatorfeed.Thisprogramalsoactsasaportalforthechoreographertointerceptthesemessagesanddirectlysendherownmessagestotheactuators.
Soundsofreverberantchimeswithvariouspitchesaretriggeredwhenmessagesarereceived.Thesoundofthechimesistheentiretyoftheaudioaccompanyingtheperformance.Thesoundstriggeredaremappedtocorrespondinglinesofconductivestitchingintheshoulderpleats,givingthedancerssomecontroloverthesoundsmadeandthereforeanotherformofphysicalinputintheinteractiontoenablethecallandresponsedimension.
1ESP8266HUZZAHhttps://www.adafruit.com/products/24712MPR121BreakoutBoardhttps://www.sparkfun.com/products/96953Mosquittohttp://mosquitto.org/
Figure3.Leather-mountedtextilecircuitoftheticklemotorwornbyeachdancertoprovidehapticfeedback
ThisarrangementofsensorsreportingtoabrokerandactuatorsrespondingtothesamebrokerusingMQTTisanincreasinglystandardarchitecturefornetworkeddevicescommonlyreferredtoasthe‘InternetofThings’(Al-Fuqahaetat2015).However,insteadofmonitoringairqualityacrossacityorthecontentsofarefrigerator,ournetworkismonitoringtheactionsofdancerswithinaperformanceandprovidinganinvisibly-connectednetworkwithphysically-realisedendpoints.
DiscussionResearchersinrecentyearshavebeenexploringwearabletechnologiesfromthemobilehealthdimension,butasSusanElizabethRyanhasnotedinherrecentbookGarmentsofParadise(2014:8),fewareexploringthefullpotentialofwearabletechnologyinperformance,letalonetheotherrelatedissuesofidentityandbodydataownershipinperformance.Shewrites:
Wearablesinthecontextofperformancepresentopportunitiesforexploringourrelationshipswithourbodiesandhowwemovethem…[orhow]communicationsinterfaces,andothersoftandsensorytechnologiesallowustoexperienceortranscendourbodies,andhowtheconceptoftheatricalperformancecanbeexpandedinvirtualspace(2014:8)
Thiscollaborationaddressestheissues,challengesandproblemsofdevelopingmethodsofmakingandusinghandmadewearablesensingandactuationdevicestoaccessphysiologicaldataandcreateuniqueinteractiveperformances.Assuch,weseethisaswaytodrawinnewcommunities,especiallywithinperformance,intothedevelopment,evolutionof,andconversationaroundwearabletechnologyandetextilesproduction,thesustainablefashionissues,datacollectionethics,andinparticularhowtheseskin-basedtechnologiesmightenhanceperformancecreation,whilemakingthemplayfulandchallenging.
Futurepracticalexplorationsinclude:organisingmoreperformancesintheUKandtheUS,continuingtodevelopdifferentapproachestousingwearabletechandetextilesinperformancecontexts,aswellasmakingmorerobustcustomwearabletechgarments,embeddedwithbothspecialistsensorsandactuators,thatenabletheperformerstointervenewitheachother’sexpressionusingtheirbodydata.Theultimategoalisthatperformersengagewiththeirownandother’sbodycodetocreatenewformsof‘livedataperformance’,wheretheperformerisinitiatingtheinteractionusingthewearabledevicestoaidtheirinteraction.
Theoveralllongtermdirectionofthisprojectistorefinemethodsofworkingwithperformers,toenablethemtocontrolhowtheyusethephysiologicaldatafromtheirbodyorthedatafromanotherperformertointeractandmove.Thiswillbedevelopedthroughfurtheriterationsofthedevicesandgarmentsworn,toenablethemtointeractandrespondtoeachotherandcreateanewmovement‘dialogue’.Inthisway,theperformersreclaimthedatacollectionbyusingthetechnologyasanothertooltothemhelpdevisemovementandco-createorchoreographperformanceworks,andeventuallywithchoreographer'srolechangingmoretothatofperformanceexperiencedesignerorartisticdirector,whiletheaudience'srolewillchangetopartperformer/partaudience/partchoreographer.Thiscircumventsandhopestoshinenewlightontheethicalissuesofwearabletechnologiesbyputtingthecontrolbackintothehandsoftheusers,inthiscase,theperformersandartisticdirector.Thisinturnismaybeconsideredacriticalactofmakingandconfrontationoftheissuesofsurveillanceanddatacontrol.
Acknowledgements.ThisworkwassupportedbyArtsCouncilEnglandGrantfortheArts.
ReferencesAl-Fuqaha,Ala,MohsenGuizani,MehdiMohammadi,MohammedAledhari,andMoussaAyyash.2015.Internetofthings:Asurveyonenablingtechnologies,protocols,andapplications.IEEECommunicationsSurveys&Tutorials17(4):2347-76.
Kozel,Susan(2008)Closer:Performance,Technologies,Phenomenology.CambridgeandLondon:MITPress.
Manning,Erin(2009)Relationscapes:Movement,Art,Philosophy.CambridgeandLondon:MITPress.
Ryan,SusanElizabeth(2014)GarmentsofParadise,CambridgeandLondon:MITPress,8.
Schiphorst,Thecla(2007),TheVarietiesofUserExperience:BridgingEmbodiedMethodologiesfromSomaticsandPerformancetoHumanComputerInteraction,PhDDissertation,SchoolofComputing,Plymouth,UnitedKingdom:CAiiAStarattheUniversityofPlymouth.