![Page 1: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/1.jpg)
iCuba shared platform for research in robotics & AI
Woods Hole, MA
Giorgio Metta & the iCub teamIstituto Italiano di TecnologiaVia Morego, 30 - Genoa, Italy
1
![Page 2: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/2.jpg)
we have a dream
© Source Unknown. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
2
![Page 3: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/3.jpg)
the iCub
price: 250K€30 iCubdistributed since 2008about 3-4 iCub’s/year
© RobotCub Consortium. License GPL v2.0. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
3
![Page 4: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/4.jpg)
why is the iCub special?• hands: we started the design from the hands
– 5 fingers, 9 degrees of freedom, 19 joints
• sensors: human-like, e.g. no lasers– cameras, microphones, gyros, encoders, force, tactile…
• electronics: flexibility for research– custom electronics, small, programmable (DSPs)
• reproducible platform: community designed− reproducible & maintainable yet evolvable platform − large software repository (~2M lines of code)
© RobotCub Consortium. License GPL v2.0. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
4
![Page 5: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/5.jpg)
why humanoids?• scientific reasons
– e.g. elephants don’t play chess
• natural human-robot interaction
• challenging mechatronics
• fun!
.Image removed due to copyright restrictions
Please see the video.
.Image removed due to copyright restrictions
Please see the video.
Image removed due to copyright restrictions.Please see the video.
.Image removed due to copyright restrictions
Please see the video.
5
![Page 6: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/6.jpg)
why open source?• repeatable experiments
• benchmarking
• quality
this resonates with industry-grade R&D in robotics
.Image removed due to copyright restrictions
Please see the video.
.Image removed due to copyright restrictions
Please see the video.
.Image removed due to copyright restrictions
Please see the video.
6
![Page 7: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/7.jpg)
Yet Another Robot Platform• YARP is an open-source (LGPL)
middleware for humanoid robotics
• history
– an MIT / Univ. of Genoa collaboration
– born on Kismet, grew on COG, under QNX
– with a major overhaul, now used by the iCub project
• C++ source code (some 400K lines)
• IPC & hardware interface
• portable across OSs and developmentplatforms
2000-2001
2001-2002
2003
2004-Today
© Bryan Adams. All rights reserved. Thiscontent is excluded from our CreativeCommons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
Courtesy of Laboratory forIntegrated Advanced Robotics.Used with permission.
© RobotCub Consortium. License GPL v2.0.This content is excluded from our CreativeCommons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
Image of Tibi Robot removeddue to copyright restrictions.
Image is in the public domain.
7
![Page 8: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/8.jpg)
exploit diversity: portability• operating system portability:
– Adaptive Communication Environment, C++ OS wrapper: e.g. threads, semaphores, sockets
• development environment portability:– CMake
• language portability:– via Swig: Java (Matlab), Perl, Python, C#
C/C++library
C/C++library
C/C++library
C/C++library
Projectdescription
(.txt)
LINUX:Makefiles,Kdevelopfiles, ...
WINDOWS:MSVC files,Borland files,
...
OSX:Makefiles,
Xcode files, ...
8
![Page 9: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/9.jpg)
wiki & manual
SVN & GIT
part lists
drawings
© RobotCub Consortium. License GPL v2.0. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
9
![Page 10: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/10.jpg)
iCub sensors
© RobotCub Consortium. License GPL v2.0. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
10
![Page 11: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/11.jpg)
the robot skin
crot
apac
i
electrodes: etched on a flexible PCBparameters: shape, folding, etc.
soft material: e.g. siliconeparameters: dielectric constant, mechanical stiffness, etc.
ground plane: e.g. conductive fabricparameters: mechanical properties, impedance, etc.
© Source Unknown. All rights reserved.
This content is excluded from our Creative
Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
11
© Source Unknown. All rights reserved.
This content is excluded from our Creative
Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
© Source Unknown. All rights reserved.
This content is excluded from our Creative
Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
© Source Unknown. All rights reserved.
This content is excluded from our Creative
Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
![Page 12: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/12.jpg)
© RobotCub Consortium. License GPL v2.0. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
12
![Page 13: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/13.jpg)
Fabric, conductive+protective
Inner support for PCB
PCB
fingernailCDC chip
Plastic interface transforming to curved shape
12 sensors14.5 x 13 mm
13
© RobotCub Consortium. License GPL v2.0. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
![Page 14: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/14.jpg)
learning dynamics
Six axis F/T sensor
Inertial sensor
• learning body dynamics– compute external forces– implement compliant control
• so far we did it starting from e.g. the CAD models– but we’d like to avoid it
Courtesy of World Scientific Publishing Company. Used with permission.
14
![Page 15: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/15.jpg)
incremental experiments
15
![Page 16: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/16.jpg)
temperature compensation
16
![Page 17: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/17.jpg)
dataset
Human“Teacher”
MotionDetection
Verbal (weak) Supervision
sprinkler
© Journal of Machine Learning Research. All rights reserved. This content is excluded from our
Creative Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
17
![Page 18: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/18.jpg)
methodsLocal Coding-Pooling Coding Pooling
e.g. SIFT, HOG,SURF
e.g. VQ, SC,LLC
e.g. SpatialPyramid
ImageConvolution, NL, Pooling
Mainstream Object Recognition Pipelines:
e.g. SVM, RLS
ClassifierImage
Convolution, NL, Pooling
Convolutional Neural Networks:
Representation
e.g. SVM, RLS
ClassifierRepresentation
© Source Unknown. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
18
![Page 19: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/19.jpg)
exploiting continuity in time
sprinkler sprinkler sponge sprinkler sprinkler sprinkler
0.5 sec
19
© Source Unknown. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
![Page 20: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/20.jpg)
incremental learning
• Present: test on current day• Causal: test on current and
past days• Future: test on future days
(current not included)• Independent: train & test on
current day only
Cumulative learning on the 4 days of acquisition. Tested on:
daya
ccu
racy (
avg
. o
ve
r cla
sse
s)
1 2 3 4
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
day
accu
racy
(avg
. ove
r cla
sses
)
OF presentOF causalOF futureOF Independent
20
![Page 21: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/21.jpg)
3D vision for grasping
Input Segmentation Disparity
© Source Unknown. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
21
![Page 22: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/22.jpg)
force reconstruction
22
© Source Unknown. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
![Page 23: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/23.jpg)
Photo of researchers removed due to copyright restrictions. Please see the video.
23
![Page 24: Woods Hole, MA Giorgio Metta& the iCub team · 2020. 8. 12. · iCub a shared platform for research in robotics & AI Woods Hole, MA Giorgio Metta& the iCub team IstitutoItalianodi](https://reader036.vdocuments.us/reader036/viewer/2022062415/604618e564e5f323066b8bb6/html5/thumbnails/24.jpg)
MIT OpenCourseWarehttps://ocw.mit.edu
Resource: Brains, Minds and Machines Summer CourseTomaso Poggio and Gabriel Kreiman
The following may not correspond to a p articular course on MIT Openprovided by the author as an individual learning resource.
CourseWare, but has been
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.