mandyam srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › l8... ·...
TRANSCRIPT
![Page 1: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/1.jpg)
Queensland Brain Institute and Institute of Electrical and Electronic
Engineering University of Queensland
and ARC Centre of Excellence in Vision Science
Mandyam Srinivasan
Vision and navigation in bees and birds and applications to robotics
![Page 2: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/2.jpg)
Sam Baker, Daniel Bland, Natalie Bland, Nikolai Liebsch, Richard Moore, Gavin Taylor, Saul Thurrowgood, Dean Soccol
![Page 3: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/3.jpg)
Peculiarities of insect vision
Small interocular separation
Therefore, stereo vision is difficult
Insects rely heavily on image motion cues to infer object distance, perceive the world in 3-D and navigate in it
![Page 4: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/4.jpg)
Dr. Miriam Lehrer University of Zürich
![Page 5: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/5.jpg)
Kirchner & Srinivasan Naturwissenschaften (1988)
Srinivasan, Lehrer, Kirchner & Zhang Vis. Neurosci. (1991)
Bees negotiate narrow gaps by balancing the image velocities in the two eyes
![Page 6: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/6.jpg)
Centering response in budgerigars P. Bhagavatula, C. Claudianos, M. Ibbotson, M.V. Srinivasan Current Biology (2011)
N.S.
![Page 7: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/7.jpg)
Visual control of flight speed - bees
~300 deg/sec
Baird, Srinivasan, Zhang and Cowley, J Exp Biol (2005)
![Page 8: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/8.jpg)
Speed of flight is regulated by holding the global image velocity constant
Control of flight speed
Srinivasan, Zhang, Lehrer & Collett J. Exp. Biol. (1996)
![Page 9: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/9.jpg)
0
0.1
0.2
0.4
0.3
0.5
Battling headwind
Barron and Srinivasan, J. Exp. Biol. (2006)
0 40 80 120 160
0.9 m/s 1.6 m/s 2.8 m/s 3.8 m/s
Ground speed m/s
Volts
Headwind speed
Headwind
![Page 10: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/10.jpg)
Landing
How does a bee perform a smooth, grazing landing ?
![Page 11: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/11.jpg)
h
(height above surface)
(Forward speed)
(Descent speed) V d
V f !=Tan-1"
V f
V d
!"(Descent angle)
Landing parameters
![Page 12: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/12.jpg)
Horizontal flight speed versus height
Srinivasan, Zhang, Chahl, Barth & Venkatesh, Biol. Cybern (2000)
![Page 13: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/13.jpg)
![Page 14: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/14.jpg)
Landing on a vertical surface
![Page 15: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/15.jpg)
With Emily Baird, Norbert Boeddeker, Michael Ibbotson
![Page 16: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/16.jpg)
0 100 200 300 400 500 600 700-300
-200
-100
0
100
200
300
400
Time (t), msec
Posi
tion
(Z),
mm
S61
Z=-273.48+519.7*exp(-0.0028*t)
Stat 6 arms
11 flights
Distance decreases exponentially as a function of time, suggesting that approach speed is controlled by holding rate of image expansion constant
Honeybee landing
Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)
![Page 17: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/17.jpg)
0 200 400 600 800 1000 1200-250
-200
-150
-100
-50
0
50
100
150
200
250Expanding 1 rpsContracting 1rpsStatic
Posi
tion
(Z),
mm
Time (t), msec
Test of hypothesis: Expanding spiral reduces approach speed; contracting spiral increases it
Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)
![Page 18: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/18.jpg)
200 deg/sec
Constant approach speed
Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)
![Page 19: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/19.jpg)
The waggle dance
Honeybee odometry
Wag
gle
dura
tion
(sec
)
Q: How does a bee work out how far she has flown?
Karl von Frisch
![Page 20: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/20.jpg)
![Page 21: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/21.jpg)
Round dance
(feeder distance < 50m)
Waggle dance
(feeder distance > 50m)
![Page 22: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/22.jpg)
Honeybee navigation: How does a bee estimate how far it has flown?
![Page 23: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/23.jpg)
6 m
Feeder
6 m
Feeder
Hive
Hive
Feeder Hive
6 m
Dance signal
~ 0 m
~ 200 m
Round dance
Waggle dance
Round dance ~ 0 m
Srinivasan, Zhang, Altwein & Tautz, Science (2000) Esch, Zhang, Srinivasan & Tautz Nature (2001)
![Page 24: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/24.jpg)
1 msec of waggle corresponds to 18o of image movement in the eye during flight
18o
Visual calibration of the honeybee’s odometer and dance
Srinivasan, Zhang, Altwein & Tautz, Science (2000)
![Page 25: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/25.jpg)
Scouts returning from 8 m tunnel signal a distance of 140 m
Recruits search at 140 m
8 m
How do the recruits respond to the dancing tunnel bees?
Esch, Zhang, Srinivasan & Tautz Nature (2001)
![Page 26: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/26.jpg)
Image courtesy Jeff Wilson and Judith Reinhard
Honeybee perception and ‘cognition’
![Page 27: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/27.jpg)
Maze learning
![Page 28: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/28.jpg)
![Page 29: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/29.jpg)
Maze navigation: following a colour mark
![Page 30: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/30.jpg)
![Page 31: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/31.jpg)
![Page 32: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/32.jpg)
Music in the wind
Bev Doolittle
![Page 33: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/33.jpg)
Rewarded Unrewarded 49.0% 51.0%
63.1% 36.9%
61.2% 38.8%
(n=1099, P > 0.50
(n=141, P < 0.005)
(n=509, P < 0.001) 62.6% 37.4%
(n=238, P < 0.001)
“Top-down” processing in honeybees
Zhang & Srinivasan Nature (1994)
![Page 34: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/34.jpg)
Associative recall
![Page 35: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/35.jpg)
2m
Rose-scented feeder
Hive
21 individually marked bees
Lemon-scented feeder
Training paradigm
Can scent trigger memories of locations?
Reinhard, Srinivasan and Zhang, Nature (2004)
![Page 36: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/36.jpg)
2m
Former “rose” feeder
Former “lemon” feeder
rose scent
4
41 40
5
Hive
21 individually marked bees
lemon scent
Test
Can scent trigger memories of locations?
Reinhard, Srinivasan and Zhang, Nature (2004)
![Page 37: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec](https://reader034.vdocuments.us/reader034/viewer/2022042311/5ed9ea32bf9eb936e238ea79/html5/thumbnails/37.jpg)