tangible musical interfaces - lmu medieninformatik...++ reactable - tangible modular synthesizer the...
TRANSCRIPT
![Page 1: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/1.jpg)
Tangible Musical Interfaces
![Page 2: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/2.jpg)
Gegenständliche musikalische Interaktion
![Page 3: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/3.jpg)
++ Martin Kaltenbrunner
Interface Culture LabUniversity of Art and Design, Linz, Austria
http://interface.ufg.ac.at/
Reactable Systems, Barcelona, Spainhttp://www.reactable.com/
![Page 4: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/4.jpg)
musical instruments 35.000 years ago ...
![Page 5: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/5.jpg)
… until the 19th century
![Page 6: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/6.jpg)
the end of the acoustic possibilities ...
![Page 7: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/7.jpg)
The pioneers of electronic music
![Page 8: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/8.jpg)
from analog sound synthesis
![Page 9: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/9.jpg)
to digital sound synthesis
![Page 10: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/10.jpg)
an office tool as musical instrument?
![Page 11: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/11.jpg)
but how can we control all these parameters?
![Page 12: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/12.jpg)
various types of musical controllers
![Page 13: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/13.jpg)
decoupling control & sound generation
![Page 14: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/14.jpg)
++ remote control?
![Page 15: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/15.jpg)
++ graphical user interfaces ...
... separate control ...
... from representation
![Page 16: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/16.jpg)
++ a typical GUI user
from Igoe & Sullivan: Physical Computing
![Page 17: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/17.jpg)
a musical instrument ...
... unifies control and feedback
![Page 18: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/18.jpg)
Tangible User Interfaces
... unify control & representationwithin tangible physical artifacts
![Page 19: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/19.jpg)
Embodiment
physical objects represent digitalinformation and processes
![Page 20: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/20.jpg)
![Page 21: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/21.jpg)
++ reactable - tangible modular synthesizer
![Page 22: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/22.jpg)
The instrument should be perceived as such, and not just only as a controller device.
All sound synthesis and control dimensions should be available simultaneously at any time through direct manipulation of tangible instrument components.
The instrument should provide visual and haptic feedback in addition to its sonic core functionality.
The instrument should be intuitive and learnable, but also provide sufficient musical depth. No sound toy!
Support the collaboration of two or more musicians.
Goals for Tangible Instrument Design
![Page 23: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/23.jpg)
++ fundamental inspirations
![Page 24: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/24.jpg)
++ dynamic patching paradigm
![Page 25: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/25.jpg)
++ shape: generic object classes
![Page 26: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/26.jpg)
++ development history
![Page 27: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/27.jpg)
++ Reactable Experience
![Page 28: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/28.jpg)
++ Reactable Live!
![Page 29: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/29.jpg)
++ Reactable Mobile
![Page 30: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/30.jpg)
++ amoeba symbols
![Page 31: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/31.jpg)
++ region adjacency graph
![Page 32: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/32.jpg)
++ marker & finger tracking
![Page 33: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/33.jpg)
++ marker, finger & object tracking
![Page 34: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/34.jpg)
++ object, cursor & blob abstraction
![Page 35: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/35.jpg)
![Page 36: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/36.jpg)
++ TUIO components
+ original TUIO 1.0 Objects: /tuio/2Dobj
describes arbitrary physical objects, which are usually tracked with the help of visual symbols (fiducial markers), RFID tags or similar methods
tokens are not defined by their physical appearance but by their IDencodes position and rotation angle.
Cursors: /tuio/2Dcur
describes surface pointers such as finger touches or dedicated devices
multiple pointers are only distinguished by their position
+ extended TUIO 1.1Blobs: /tuio/2Dblb
describes the bounds of untagged physical objects
encodes position, and oriented bounding box (angle, width, height)
can be used to additionally describe the approximate object geometry
Source: allows the multiplexing of multiple TUIO trackers
![Page 37: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/37.jpg)
++ TUIO framework architecture
![Page 38: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/38.jpg)
++ tangible interface component abstraction
![Page 39: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/39.jpg)
++ FTIR – multitouch becomes popular
![Page 40: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/40.jpg)
++ CCV – community implementation
![Page 41: Tangible Musical Interfaces - LMU Medieninformatik...++ reactable - tangible modular synthesizer The instrument should be perceived as such, and not just only as a controller device](https://reader034.vdocuments.us/reader034/viewer/2022043003/5f816451e0c8414f8f37090f/html5/thumbnails/41.jpg)
++ TUIO 2.0 - tangible abstraction framework
+ revised component definitionTokens (objects), Pointers (cursors), Geometries (blobs)
+ additional componentsSymbols: allow the encoding of extended symbol content
Controls: for the association of additional control dimensions
Associations: allows description of physical connections & relations
Geometries: Contour, Skeleton, Area ... (incremental detail)
+ extended attributese.g. pointers include dedicated pointer/user ID, pressure attribute, ...
tokens allow the use of different symbol types
+ timing infrastructurefor improved gesture recognition capabilities