when robots open their eyes

57
ETSETB (Barcelona) / ETSEIAAT (Terrassa) Universitat Politècnica de Catalunya (UPC) Xavier Giró i Nieto, “When robots open their eyes...”. UPC Outreach Programme (February 2016) When robots open their eyes... Xavier Giró-i-Nieto @DocXavi Versió 2016/2

Upload: xavier-giro

Post on 13-Apr-2017

359 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

When robots open their eyes

Xavier Giroacute-i-Nieto

DocXavi

Versioacute 20162

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)2

Introduction

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)3

Densely linked slides

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)4

Acknowledgements

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Why studying engineering

5Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 2: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)2

Introduction

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)3

Densely linked slides

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)4

Acknowledgements

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Why studying engineering

5Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 3: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)3

Densely linked slides

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)4

Acknowledgements

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Why studying engineering

5Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 4: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)4

Acknowledgements

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Why studying engineering

5Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 5: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Why studying engineering

5Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 6: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)6

Source University degrees with higher employment rate in Span (El Paiacutes 22122015)

Why studying engineering

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 7: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)7

Source 25 Best jobs in America (Glassdoor)

IT degrees (telecommunication and computer science) are the closest ones to the best job Data scientist

Why studying IT

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 8: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where are the robots

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 9: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Li Fei-Fei ldquoHow wersquore teaching computers to understand picturesrdquo TEDTalks 2014

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 10: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 11: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Source Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 12: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Font Andrej Karpathy ldquoWhat I learned from competing against a computer on ImageNetrdquo (2014)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 13: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

LeCun Y Bottou L Bengio Y amp Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE 86(11) 2278-2324

Machine learning(Deep learning)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 14: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Krizhevsky Alex Ilya Sutskever and Geoffrey E Hinton Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems pp 1097-1105 2012

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 15: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object detection

Girshick Ross Jeff Donahue Trevor Darrell and Jitendra Malik Region-based convolutional networks for accurate object detection and segmentation Pattern Analysis and Machine Intelligence IEEE Transactions on 38 no 1 (2016) 142-158

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 16: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object segmentation

Source Pascal Visual Object Challenge

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 17: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Face detection and recognition

Farfade Sachin Sudhakar Mohammad Saberian and Li-Jia Li Multi-view Face Detection Using Deep Convolutional Neural Networks ICMR (2015)

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 18: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Activity Recognition

Tran Du Lubomir Bourdev Rob Fergus Lorenzo Torresani and Manohar Paluri Learning spatiotemporal features with 3D convolutional networks In Proceedings of the IEEE International Conference on Computer Vision pp 4489-4497 2015

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 19: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Learn more with Nat amp Lo 20 Google Project

Where are the robots

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 20: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Only open their eyes

Nexi del MIT Media Lab (Foto Spencer Lowel)

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 21: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Big data

Internet of things - IoT

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 22: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Personal data

Big data

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 23: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Atlas de Boston Dynamics

Robust manipulation and motion

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 24: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Games (reinforcement learning)

(Google) DeepMind

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 25: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Autonomous Driving

Google Self-driving car

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 26: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual arts

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 27: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Google Research ldquoGoing deeper into neural networksrdquo - DeepDream (2015)

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 28: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

httpturingdeepartio

Visual arts

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 29: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Music composition

Manuel Araoz ldquoTraining a Recurrent Neural Network to Compose Musicrdquo (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 30: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Poetry

Ross Goodwin Neuralsnap (2016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 31: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoScriptsrdquo ()

Darknet

JONHe leaned close and onions barefoot from his shoulder I am not a purple girl he said as he stood over him The sight of you sell your father with you a little choiceI say to swear up his sea or a boy of stone and heart down Lord Tywin said I love your word or her to me

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 32: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Public Health

Announcement of Google DeepMind Health (24022016)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 33: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nacho Hernandez ldquoWhy artificial intelligence will democratize healthcarerdquo (TEDx Talk 2014)

Public health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 34: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nancy Lublin ldquoThe heartbreaking text that inspired a crisis helplinerdquo (TED Talk 2015)

Mental health

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 35: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

ldquoGooglersquos chairman (Eric Schmidth) thinks artificial intelligence will let scientists solve some of the worldrsquos hard problems like population growth climate change human development and educationrdquo (Bloomberg Business 11012016)

[+info MIT Technology Review]

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 36: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Nomeacutes poden obrir els ulls Jeremy Howard ldquoThe wonderful and terrifying implications of computers that can learnrdquo TEDTalks 2014

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 37: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Elon Musk (Tesla) one of OpenAI promoters

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 38: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Neil Lawrence OpenAI wonrsquot benefit humanity without open data sharing (The Guardian 14122015)

Phd Comics Who owns your data (Hint it is not you)

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 39: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)39

Xavier Sala-i-Martin (Columbia University) ldquoLes conclusions del Fograverum de Davosrdquo

(TV3 03022016) - in Catalan

Carles Boix (Princeton University) ldquoLa quarta revolucioacute industrialrdquo

(Diari Ara 08022016) - in Catalan

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 40: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Psychological support and counseling

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 41: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)41

Affective computing

Rana el Kalioubi ldquoThis app know how you feel from the look on your facerdquo TEDTalks 2015

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 42: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)42

Nexi Project from MIT Media Lab (Photos Spencer Lowel)

[video]

Only open their eyes Affective computing

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 43: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)43

When robots open their eyes

is because they have learned to see

Only open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 44: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)44

Where to learn from

In school you are also (mainly) learning to learn

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 45: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)45

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 46: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)46

Antonio Torralba Professor at MIT

Cristian CantonMicrosoft Research

Joan BrunaProfessor at Berkeley

These experts (and MANY MORE who are not in this page)

Oriol Vinyals Google DeepMind

studied at

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 47: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)47

Source La UPC als llistat de ragravenquins internacionals

Evolution of the position of the UPC in the

QS World University Rankings and the QS WUR by Faculty

Ranking 2015 edition 2014 edition

Catalonia Technical

university

in Spain

Spain Europe World Catalonia Technical

university

in Spain

Spain Europe World

QS World

University

Ranking

general

classification

4 1 7 134 299 4 1 7 152 337

QS WUR by

Faculty

Engineering

and

Technology

1 1 1 22 82 1 1 1 17 60

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 48: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 49: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

More details on the website by Professor Jordi Torres

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 50: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 51: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Object retrieval from videos

V - T Nguyen -Dinh-Le D Salvador A -Zhu C Nguyen D - L Tran M - T Duc T Ngo Duong D Anh Satoh S ichi and Giroacute-i-Nieto X ldquoNII-HITACHI-UIT at TRECVID 2015 Instance Searchrdquo in TRECVID 2015 Workshop Gaithersburg MD USA 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 52: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

EEG-based object segmentation

E Mohedano Healy G McGuinness K Giroacute-i-Nieto X OConnor N and Smeaton A F ldquoObject segmentation in images using EEG signalsrdquo in ACM Multimedia Orlando Florida (USA) 2014

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 53: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Visual saliency prediction

J Pan and Giroacute-i-Nieto X ldquoEnd-to-end Convolutional Network for Saliency Predictionrdquo in Large-scale Scene Understanding Challenge (LSUN) at CVPR Workshops Boston MA (USA) 2015 [Slides]

LSUN Challenge

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 54: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)54

A Salvador Zeppelzauer M Manchon-Vizuete D Calafell-Oroacutes A and Giroacute-i-Nieto X ldquoCultural Event Recognition with Visual ConvNets and Temporal Modelsrdquo in CVPR ChaLearn Looking at People Workshop 2015 2015 [slides]

ChaLearn WorshopImage classification

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 55: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Sentiment analysis

V Campos Salvador A Jou B and Giroacute-i-Nieto X ldquoDiving Deep into Sentiment Understanding Fine-tuned CNNs for Visual Sentiment Predictionrdquo in 1st International Workshop on Affect and Sentiment in Multimedia Brisbane Australia 2015

Where to learn from

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 56: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Conclusions

56

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi

Page 57: When robots open their eyes

ETSE

TB (B

arce

lona

) E

TSEI

AA

T (T

erra

ssa)

Uni

vers

itat P

olitegrave

cnic

a de

Cat

alun

ya (U

PC)

Xavier Giroacute i Nieto ldquoWhen robots open their eyesrdquo UPC Outreach Programme (February 2016)

Thanks a lot

57

Slides available at

httpsimatgeupceduwebpeoplexavier-giro

DocXavi

ProfessorXavi