Jürgen ‘Juxi’ Leitner
Reactive Reaching and Grasping on a Humanoid Robot
Dalle Molle Institute for AI (IDSIA)
#ICINCO 2014
humanoidour iCub
!
perceptionvisual
thanks to G. Metta and IIT for this picture
!
objectsdetecting
Harding et al., 2013Leitner et al., ICDL 2012, ARS 2012, BICA 2012, CEC 2013
cartesian genetic programming
+ min dilate avg INP INP INP
[Leitner et al, 2012a/b, Harding et al., 2013]
learningapproach
detection
! icImage GreenTeaBoxDetector::runFilter() { icImage node0 = InputImages[6]; icImage node1 = InputImages[1]; icImage node2 = node0.absdiff(node1); icImage node5 = node2.SmoothBilateral(11); icImage node12 = InputImages[0]; icImage node16 = node12.Sqrt(); icImage node33 = node16.erode(6); icImage node34 = node33.log(); icImage node36 = node34.min(node5); icImage node49 = node36.Normalize(); ! //cleanup ... icImage out = node49.threshold(230.7218f); return out; }
detect
detect
detection
approachcgp
handsdetecting
approachsupervised learning
BUT
segmentationfeature
saliencymap
collaboration FIAS
presegmentation
approachcombined
!
transferringspatial perception
setuplearning
trainingset
9DOF
iCubbounding box
6 per eye Carte
sian
Coor
dinate
s
.
.
.
spatial perception neural network
. . .
9DO
F iC
ubbo
undi
ng b
ox
6 pe
r eye
Cart
esian
Coor
dina
tes
!fu
lly c
onne
cted
!fu
lly c
onne
cted
. . .
MoBeEframework Frank et al., ICINCO, 2012.
MoBeEFrank et al., ICINCO, 2012.
Frank et al., ICINCO, 2012.
generationmotionStollenga et al, 2013
Shak
ey 2
013
Win
ner
MoBeEv2[Frank et al., 2011,2012, 2013]
hand/armop-space forcing
CSWorld
CSHand
CSR/CSL
[Leitner et al, in prep]
-
coordinationhand-eye
model
http://robotics.idsia.ch/
manipulation for improved perception
manipulation actions
extracting information
improveddetection
detection
! icImage* BlueCupFilter::runFilter() { icImage* node43 = InputImages[4]; icImage* node49 = node43->LocalAvg(15); ! icImage* out = node49->threshold(81.532f); return out; }
detection
! icImage* BlueCupFilter::runFilter() { icImage* node0 = InputImages[4].Exp(); icImage* node5 = InputImages[0]; icImage* node16 = node0->Gabor(-8,14,1,13); icImage* node17 = InputImages[4]->LocalAvg(6); icImage* node18 = node16->Laplace(5); icImage* node19 = node5->Sobel(13,9); icImage* node24 = node17->Erode(5); icImage* node28 = node19->Min(node18); icImage* node29 = node28->Min(node24); icImage* node41 = node29->LocalAvg(7); icImage* node49 = node41->LocalMax(7); ! icImage* out = node49->threshold(68.032f); return out; }
resulting
operationtele
teleoperation
!
workfuture
improved eye-hand coordination
different object representations
online, continuous learning
vSLAM aerial robotics & for world model
for listeningthanks
[email protected] http://Juxi.net/projects
http://dilbert.com/strips/comic/2013-10-24/