auditory display in mir
TRANSCRIPT
![Page 1: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/1.jpg)
stop looking for music and start listening to it:
auditory display in music information retrieval interfaces
Becky [email protected]
Centre for Digital MusicSchool of Electronic Engineering and Computer ScienceQueen Mary, University of London
![Page 2: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/2.jpg)
In this talk we will ...
![Page 3: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/3.jpg)
• Review how search and browse for information
In this talk we will ...
![Page 4: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/4.jpg)
• Review how search and browse for information
• Look at current commercially-available interfaces
In this talk we will ...
![Page 5: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/5.jpg)
• Review how search and browse for information
• Look at current commercially-available interfaces
• Discuss why listening should be integrated
In this talk we will ...
![Page 6: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/6.jpg)
• Review how search and browse for information
• Look at current commercially-available interfaces
• Discuss why listening should be integrated
• Look at solutions presented by academia
In this talk we will ...
![Page 7: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/7.jpg)
• Review how search and browse for information
• Look at current commercially-available interfaces
• Discuss why listening should be integrated
• Look at solutions presented by academia
• Review recent research from C4DM
In this talk we will ...
![Page 8: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/8.jpg)
• Review how search and browse for information
• Look at current commercially-available interfaces
• Discuss why listening should be integrated
• Look at solutions presented by academia
• Review recent research from C4DM
• Wrap up and conclude
In this talk we will ...
![Page 9: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/9.jpg)
how do we find information?
![Page 10: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/10.jpg)
let’s start with something easy...
![Page 11: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/11.jpg)
![Page 12: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/12.jpg)
Familiar interface
Summarizes information
Users seldom scroll down, almost never go to next page
![Page 13: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/13.jpg)
how about better browsing?
![Page 14: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/14.jpg)
![Page 15: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/15.jpg)
Easy to traverse information
Relationships between items can be inferred
Encourages browsing
![Page 16: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/16.jpg)
what about something other than text?
![Page 17: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/17.jpg)
![Page 18: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/18.jpg)
Users seldom go on to next page of results
Broad overview, but can zoom in on specific result
All other information beyond image is suppressed, but recallable
![Page 19: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/19.jpg)
what about time-based media?
![Page 20: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/20.jpg)
![Page 21: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/21.jpg)
Less helpful than the image search results
Difficult to navigate results
Have to go to web page to view any portion of the video
Music or audio results only is not an option
![Page 22: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/22.jpg)
so what about music interfaces? how do we find music?
![Page 23: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/23.jpg)
![Page 24: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/24.jpg)
commercial interfaces use a combination of text fields and seed songs/artists
![Page 25: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/25.jpg)
commercial interfaces use a combination of text fields and seed songs/artists
academic interfaces like maps
![Page 26: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/26.jpg)
commercial interfaces use a combination of text fields and seed songs/artists
for searches results are lists of text perhaps enhanced with images, general knowledge and hyperlinks
academic interfaces like maps
![Page 27: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/27.jpg)
commercial interfaces use a combination of text fields and seed songs/artists
for searches results are lists of text perhaps enhanced with images, general knowledge and hyperlinks
songs are played back one at a time and only if explicitly requested by user
academic interfaces like maps
![Page 28: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/28.jpg)
Also a recent increase in network interaction paradigms.
Enter artist name
Fast As You Can
Fiona Apple
Maze Radio
00:22
Fiona Apple has beenvisited 376 times.
Share Path Have an account? Sign In
history
Powered by The Echo Nest. Music powered by Rdio More info at Music Machinery Check out the Labyrinth of Genre
Laura Marling
Joan as Police Woman
Mystery Jets
Jeremy Warmsley
Emmy the Great
Basia Bulat
Regina Spektor
Nellie McKay
Kimya Dawson
Fiona Apple
Imogen Heap
Rilo Kiley
Tori Amos
Alanis Morissette
Ani DiFranco
Aimee Mann
Liz Phair
Sara Bareilles
Sarah McLachlan
![Page 29: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/29.jpg)
why should audio be integrated?
![Page 30: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/30.jpg)
Bjork / Björk
• textual metadata can be malformed or wrong
• an empty text field is less than inspiring
• text can be a barrier to discovery
• previous knowledge is needed
• difficult to move into tail, will stay in the head
Celma and Cano From hits to niches? or how popular artists can bias music recommendation and discovery. In Proc. of 2nd Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition (ACM KDD), Las Vegas, Nevada, USA, August 2008.
![Page 31: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/31.jpg)
listening makes a difference
• users make different judgements about playlists when metadata is missing
L. Barrington, R. Oda, and G. Lanckriet. Smarter than Genius: human evaluation of music recommender systems. In Proc. of ISMIR’09: 10th Int.Society for Music Information Retrieval Conf., pages 357–362, Kobe, Japan, October 2009.
![Page 32: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/32.jpg)
listening is faster
• when search results are compiled into a single audio stream instead of a list of results, users find what they are looking for quicker
S. Ali and P. Aarabi. A cyclic interface for the presentation of multiple music files. IEEE Trans. on Multimedia, 10(5):780–793, August 2008.
• listeners can find music without a GUI faster than with an iPod, and be just as happy with their selection
Andreja Andric, Pierre-Louis Xech, and Andrea Fantasia, “Music mood wheel: Improving browsing experience on digital content through an audio interface,”in Proc. of 2nd Int. Conf. on Automated Production of Cross Media Content for Multi-Channel Distribution (AXMEDIS’06), 2006.
![Page 33: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/33.jpg)
listening is effective
• users can understand and navigate a collection of music as effectively without a GUI as with one
• they are slower, but don’t make significantly more mistakes
S. Pauws, D. Bouwhuis, and B. Eggen. Programming and enjoying music with your eyes closed. In CHI ’00: Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, pages 376–383. ACM, 2000. doi: 10.1145/332040.332460.
![Page 34: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/34.jpg)
how can interfaces use more listening?
![Page 35: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/35.jpg)
not by being VoiceOver
![Page 36: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/36.jpg)
not by being VoiceOver
![Page 37: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/37.jpg)
maps
![Page 38: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/38.jpg)
mused
• passive listening
G. Coleman. Mused: navigating the personal sample library. In Proc. of ICMC: Int. Computer Music Conf., Copenhagen, Denmark, August 2007.
• youtubehttp://www.youtube.com/watch?v=DuuESpj558Y&feature=related
![Page 39: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/39.jpg)
mused
• passive listening
G. Coleman. Mused: navigating the personal sample library. In Proc. of ICMC: Int. Computer Music Conf., Copenhagen, Denmark, August 2007.
• youtubehttp://www.youtube.com/watch?v=DuuESpj558Y&feature=related
![Page 40: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/40.jpg)
sonic browser
• hugely influential interface
• introduced aurally exploring a map of sounds
• direct sonification
M. Fernström and E. Brazil. Sonic browsing: an auditory tool for multimedia asset management. In Proc. of ICAD ’01: Internation Conf. on Auditory Display, pages
132–135, Espoo, Finland, August 2001. M. Fernström and C. McNamara. After direct manipulation - direct sonification. In Proc. of ICAD ’98: Int. Conf. on Auditory Display, 1998.
![Page 41: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/41.jpg)
soundtorch
• 3D version of sonic browser
S. Heise, M. Hlatky, and J. Loviscach. SoundTorch: Quick browsing in large audio collections. In Proc. of AES 125th Conv., San Francisco, CA, October 2008.
S. Heise, M. Hlatky, and J. Loviscach. Aurally and visually enhanced audio search with SoundTorch. In CHI ’09: Proc. of the 27th int. conf.e extended abstracts on Human factors in computing systems, pages 3241–3246, Boston, MA, USA, April 2009. doi: 10.1145/1520340.1520465.
• youtube http://www.youtube.com/watch?v=eiwj7Td7Pec
![Page 42: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/42.jpg)
soundtorch
• 3D version of sonic browser
S. Heise, M. Hlatky, and J. Loviscach. SoundTorch: Quick browsing in large audio collections. In Proc. of AES 125th Conv., San Francisco, CA, October 2008.
S. Heise, M. Hlatky, and J. Loviscach. Aurally and visually enhanced audio search with SoundTorch. In CHI ’09: Proc. of the 27th int. conf.e extended abstracts on Human factors in computing systems, pages 3241–3246, Boston, MA, USA, April 2009. doi: 10.1145/1520340.1520465.
• youtube http://www.youtube.com/watch?v=eiwj7Td7Pec
![Page 43: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/43.jpg)
neptune
• based on Islands of Music
P. Knees, M. Schedl, T. Pohle, and G. Widmer. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In MULTIMEDIA ’06: Proc. of the 14th annual ACM int.l conf. on Multimedia, pages 17–24, Santa Barbara, CA, USA, 2006. doi: 10.1145/1180639.1180652.
![Page 44: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/44.jpg)
neptune
• based on Islands of Music
P. Knees, M. Schedl, T. Pohle, and G. Widmer. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In MULTIMEDIA ’06: Proc. of the 14th annual ACM int.l conf. on Multimedia, pages 17–24, Santa Barbara, CA, USA, 2006. doi: 10.1145/1180639.1180652.
![Page 45: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/45.jpg)
sonixplorer
• extension of neptune
• landscape can be marked up by user
• introduced focus
• youtube http://www.youtube.com/watch?v=mIfWg2Eex74
D. Lübbers. Sonixplorer: Combining visualization and auralization for content-based exploration of music collections. In Proc. of ISMIR’05: 6th Int. Society for Music Information Retrieval Conf., pages 590–593, London, UK, 2005.
D. Lübbers and M. Jarke. Adaptive multimodal exploration of music collections. In Proc. of ISMIR’09: 10th Int. Society for Music Information Retrieval Conf., pages 195–200, Kyoto, Japan, 2009.
![Page 46: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/46.jpg)
sonixplorer
• extension of neptune
• landscape can be marked up by user
• introduced focus
• youtube http://www.youtube.com/watch?v=mIfWg2Eex74
D. Lübbers. Sonixplorer: Combining visualization and auralization for content-based exploration of music collections. In Proc. of ISMIR’05: 6th Int. Society for Music Information Retrieval Conf., pages 590–593, London, UK, 2005.
D. Lübbers and M. Jarke. Adaptive multimodal exploration of music collections. In Proc. of ISMIR’09: 10th Int. Society for Music Information Retrieval Conf., pages 195–200, Kyoto, Japan, 2009.
![Page 47: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/47.jpg)
what’s the problem?
![Page 48: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/48.jpg)
what’s the problem?
• too much information thrown at the user
![Page 49: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/49.jpg)
what’s the problem?
• too much information thrown at the user
• does not translate well to mobile devices
• rendering spatial audio
• reliance on screens
![Page 50: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/50.jpg)
my research
![Page 51: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/51.jpg)
map paradigm without any visuals
![Page 52: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/52.jpg)
![Page 53: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/53.jpg)
![Page 54: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/54.jpg)
![Page 55: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/55.jpg)
evaluation
• user study with 12 users
• most liked the idea
• but the implementation needed improvement
• confusion as to how to navigate through the space
• some people adverse to concurrent playback
![Page 56: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/56.jpg)
add visuals and improve physical controller, but keep dependence on audio
![Page 57: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/57.jpg)
cyclic playback
• inspired by
S. Ali and P. Aarabi. A cyclic interface for the presentation of multiple music files. IEEE Trans. on Multimedia, 10(5):780–793, August 2008.
• hear everything within 20 seconds
• user can control concurrent playback
![Page 58: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/58.jpg)
![Page 59: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/59.jpg)
![Page 60: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/60.jpg)
![Page 61: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/61.jpg)
evaluation
• no formal evaluation, but demonstrated to a variety of individuals and small groups (approximately 40 people)
• improved interaction with physical controller
• perhaps too many controls, much steeper learning curve
• much room for improvement
![Page 62: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/62.jpg)
art installation
![Page 63: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/63.jpg)
Michela Magas
![Page 64: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/64.jpg)
public installation
• shown in Information Aesthetics at SIGGRAPH 2009
• approximately 1000 passed through the exhibit
• children, students, artists, designers, technologists
• quick to bring smiles - it was fun, people even brought back friends to experience it
• easy to learn how to use
![Page 65: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/65.jpg)
conclusions drawn from research
![Page 66: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/66.jpg)
conclusions drawn from research
• context is key when shaping interaction
• users will approach an interface with previous knowledge, need to build on and incorporate that knowledge
![Page 67: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/67.jpg)
conclusions drawn from research
• context is key when shaping interaction
• users will approach an interface with previous knowledge, need to build on and incorporate that knowledge
• audio can’t be subtle
• can’t rely on complex information to be universally implied through only audio
![Page 68: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/68.jpg)
conclusions drawn from research
• context is key when shaping interaction
• users will approach an interface with previous knowledge, need to build on and incorporate that knowledge
• audio can’t be subtle
• can’t rely on complex information to be universally implied through only audio
• can (and should) be fun
![Page 69: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/69.jpg)
conclusions drawn from research
• context is key when shaping interaction
• users will approach an interface with previous knowledge, need to build on and incorporate that knowledge
• audio can’t be subtle
• can’t rely on complex information to be universally implied through only audio
• can (and should) be fun
• maps aren’t great, there must be something better
![Page 70: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/70.jpg)
why haven’t these ideas caught on?
![Page 71: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/71.jpg)
why haven’t these ideas caught on?
• solutions use non-scalable algorithms that are impractical for commercial applications (a problem not limited to only interfaces within MIR)
• music is increasingly in the cloud, looking at entire collections at once is not useful
• portability across devices
• many of them just don’t work that well
• most have very simple acoustics models
• too much information thrown at user, or information is not organized in an accessible way
![Page 72: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/72.jpg)
flickr:matsber
flickr:jlcwalker
![Page 73: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/73.jpg)
what am I doing at nyu?
![Page 74: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/74.jpg)
concentrating on how a small collection of songs can be best presented to a user
![Page 75: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/75.jpg)
concentrating on how a small collection of songs can be best presented to a user
i.e. how can the results of a search or browse query be better presented?
![Page 76: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/76.jpg)
![Page 77: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/77.jpg)
![Page 78: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/78.jpg)
![Page 79: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/79.jpg)
![Page 80: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/80.jpg)
Experimental Design - Aims of Experiment
To determine the best interface parameters for music search and browsing tasks.
![Page 81: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/81.jpg)
Experimental Design - Independent Variables
Number of Songs: 1 to 5 songs play concurrently
![Page 82: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/82.jpg)
Experimental Design - Independent Variables
Number of Songs: 1 to 5 songs play concurrently
Musical and Signal Content of Songs: Similar or dissimilar.
![Page 83: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/83.jpg)
Experimental Design - Independent Variables
Number of Songs: 1 to 5 songs play concurrently
Musical and Signal Content of Songs: Similar or dissimilar.
Visualization: Whether interactive graphics representing each song are presented
![Page 84: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/84.jpg)
Experimental Design - Dependent Variables
![Page 85: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/85.jpg)
Experimental Design - Dependent Variables
Search
• A song is played and the participant needs to find that song in the collection.
• No metadata is displayed.
• The task is timed.
![Page 86: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/86.jpg)
Experimental Design - Dependent Variables
Search
• A song is played and the participant needs to find that song in the collection.
• No metadata is displayed.
• The task is timed.
Browse
• A situation is described and the participant is asked to find a song that fits the situation.
• The task is timed.
![Page 87: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/87.jpg)
Experimental Design - Participant Experience
![Page 88: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/88.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
![Page 89: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/89.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
![Page 90: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/90.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
3. For about 45 minutes, the participant completes a series of search and browsing tasks.
![Page 91: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/91.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
3. For about 45 minutes, the participant completes a series of search and browsing tasks.
4. The participant completes a short questionnaire about their experience so far.
![Page 92: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/92.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
3. For about 45 minutes, the participant completes a series of search and browsing tasks.
4. The participant completes a short questionnaire about their experience so far.
5. 15 minute break away from the computer and headphones.
![Page 93: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/93.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
3. For about 45 minutes, the participant completes a series of search and browsing tasks.
4. The participant completes a short questionnaire about their experience so far.
5. 15 minute break away from the computer and headphones.
6. The participant completes a second 45 minute session of search and browsing tasks.
![Page 94: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/94.jpg)
Experimental Design - Participant Experience
1. Participant uses simplified version of interface with only 1 song to choose an HRTF set.
2. A video explains how to use the interface and the participant has approximately 5 minutes to practice a search task and a browsing task.
3. For about 45 minutes, the participant completes a series of search and browsing tasks.
4. The participant completes a short questionnaire about their experience so far.
5. 15 minute break away from the computer and headphones.
6. The participant completes a second 45 minute session of search and browsing tasks.
7. The participant completes a final questionnaire.
![Page 95: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/95.jpg)
to conclude
![Page 96: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/96.jpg)
search engines are tuned for the type of information being sought
![Page 97: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/97.jpg)
search engines are tuned for the type of information being sought
but they break down when presenting time-based media
![Page 98: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/98.jpg)
search engines are tuned for the type of information being sought
but they break down when presenting time-based media
in our case, music
![Page 99: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/99.jpg)
direct manipulation to direct sonification
![Page 100: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/100.jpg)
direct manipulation to direct sonification
listen to the music first, then get more information if so desired
![Page 101: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/101.jpg)
direct manipulation to direct sonification
listen to the music first, then get more information if so desired
this is done by using auditory displays
![Page 102: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/102.jpg)
a lot of focus on map-based paradigms, but it is time to move on
![Page 103: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/103.jpg)
a lot of focus on map-based paradigms, but it is time to move on
concurrent presentation of audio is a good idea
![Page 104: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/104.jpg)
a lot of focus on map-based paradigms, but it is time to move on
concurrent presentation of audio is a good idea
but spatialization should not be used to represent complex relationships
![Page 105: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/105.jpg)
a lot of focus on map-based paradigms, but it is time to move on
concurrent presentation of audio is a good idea
but spatialization should not be used to represent complex relationships
music is complex
![Page 106: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/106.jpg)
incorporating listening improves music search and discovery
![Page 107: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/107.jpg)
incorporating listening improves music search and discovery
so it should continue
![Page 108: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/108.jpg)
incorporating listening improves music search and discovery
so it should continue
the work I am doing during my visit at nyu will measure whether this presented interface can assist people in performing search and browse tasks more efficiently
![Page 109: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/109.jpg)
however, what I believe to be the most difficult problem still remains to be addressed:
the cold start problem
future work needs to concentrate on how you initiate a search or browsing task
![Page 110: Auditory Display in MIR](https://reader034.vdocuments.us/reader034/viewer/2022042715/558e2c041a28ab37048b4739/html5/thumbnails/110.jpg)
thank you
these slides can be found at http://www.slideshare.net/beckystewart/presentations