“it's always incumbent on a researcher to make the effort to reach out to the community to...
Post on 26-Dec-2015
217 Views
Preview:
TRANSCRIPT
““It's always incumbent on a researcher to make the effort to It's always incumbent on a researcher to make the effort to reach out to the community to engage with the population reach out to the community to engage with the population
they're serving.”they're serving.”
3
Outline
• Introduction• Local Perception Module• Experiments in the past• New experiments• Conclusion
4
Introduction
• Blind people40 millions in 2020 (90% after the age of 1 year)4 millions in OCDE countries
• Long term goal : sensing the environment with soundsAction is key to perception
• A number of pixels are directional sound sources
6
See ColOr Seeing Colours with an Orchestra
head-centric
stereovision
distance
video
saxophone
piano
colour encoding2D sound spatialization
headphones
25 points
8
Colour encoding
• Colour system: Hue-Saturation-Luminance
• Hue: musical instrument• Saturation: 4 instrument
notes• Luminance: 2 instruments
with different notes
10
Saturation and luminance
• 4 levels of saturation
• 8 levels of luminance
Human voiceDouble bass
Low High
11
Prototypes
– Stereoscopic or Kinect camera– Sonification of a row containing 25 points– CIPIC Kemar HRTFs (2D spatialization)– HSL colour coding– Two depth coding schemes– Lower peripheral resolution
[15 12 9 7 5 3 3 2 2 1 1 1 1 1 1 1 2 2 3 3 5 7 9 12 15]
12
Depth coding (first mode)
For all sonified pixels (25) :• 90 ms for undetermined depth (D);• 160 ms for 0 <D<1• 207 ms for 1<D<2• 254 ms for 2<D<3• 300 ms for D > 3
13
Depth coding (second mode)
• The strategy here is to sonify the dominant colour of the row with only 1 sound source (providing average depth)
• The centroid of the first red area is laterally spatialized
[R R R R R B B J B B B R R G G G J J B B B G G G G] [R R R R R ]
• The volume V after 3 meters starts to decrease:F(V) = V * exp(-k*D)
16
Left picture : the 6 experiment participants interpreted the major colours as the sky the sea and the sun; clouds were more difficult to infer (2 subjects); instead of ducks, all the subjects found an island (with yellow sand) or a ship.
Right picture : 4 participants correctly recognised the tree with the sky and the grass; one subject qualified the tree as a strange dark object and finally the last individual imagined a nuclear explosion !
19
Participant Time (mn) Success rate (pairs)P1 12 5P2 24 5P3 7 5P4 6 3P5 16 5Average 13.0 (7.3) 4.6 (0.9)
Participant Time (mn) Success rate (pairs)P6 5 5P7 10 5P8 4 5P9 4 5P10 18 5Average 8.2 (6.0) 5.0 (0.0)
20
Following a red path
• In this experiment the goal was to follow a coloured path in an outdoor environment.
21
Participant Path Length (m) Speed Average (m/h)P1 M+C = 88 723P2 M = 84 710P3 M+B = 110 485P4 M+A = 93 656P5 M = 84 484P6 M+A+C = 97 600P7 M+A+C = 97 451P8 M = 84 869P9 M = 84 1326P10 M = 84 1096Average 90.5(8.7) 740.0(284.6)
19/04/23 25
Current Prototype with Kinect
Alerting System
Object Recognition Text reading
Local Perception Global Perception
35
• Sonification of two very important vision features (colour and depth) made it possible to achieve a number of tasks related to mobility and recognition.
• We will pursue to define new challenging experiments requiring the activation of several See ColOr modules.
• We hope that new small 3D RGB cameras will be in the market soon (like the new Kinect Capri) !
top related