cognitive learning and the multimodal memory game: toward human-level … · 2015. 11. 24. · tlk...

Cognitive Learning and the Multimodal Memory Game: Cognitive Learning and the Multimodal Memory Game: Toward HumanToward Human--Level Machine LearningLevel Machine LearningToward HumanToward Human Level Machine LearningLevel Machine Learning

2008 IEEE World Congress on Computational Intelligence (WCCI 2008)2008 IEEE World Congress on Computational Intelligence (WCCI 2008)2008 IEEE World Congress on Computational Intelligence (WCCI 2008)2008 IEEE World Congress on Computational Intelligence (WCCI 2008)Cognitive Architectures: Towards HumanCognitive Architectures: Towards Human--Level Intelligence SessionLevel Intelligence Session

June 5, 2008, Hong KongJune 5, 2008, Hong Kong

Byoung-Tak Zhang

Biointelligence LaboratorySchool of Computer Science and Engineering

Cognitive Science, Brain Science, and Bioinformatics ProgramsSeoul National UniversitySeoul National University

Seoul 151-744, Korea

[email protected]@http://bi.snu.ac.kr/

T lk O tliT lk O tliTalk OutlineTalk OutlineHuman-level machine learning is a prerequisite to achieving human-level machine intelligence.♦ Differences of behaviors in humans and machines

What principles are underlying the cognitive learning and memoryin humans?♦ A proposal for three principles♦ A proposal for three principles

What tasks are challenging enough to study human-level machine learning?♦ A proposal for the multimodal memory game (MMG)♦ A proposal for the multimodal memory game (MMG)

Some illustrative results♦ Linguistic memory♦ Language vision translation♦ Language-vision translation

Future directions♦ Toward human-level machine intelligence

© 2008, SNU Biointelligence Lab, http://bi.snu.ac.kr/

2

Cognitive LearningCognitive Learning

H d M hiH d M hiHumans and MachinesHumans and Machines

Humans are ♦ creative

Humans are ♦ imprecise, ♦ creative,

♦ compliant, ♦ attentive to change

p ,♦ sloppy, ♦ distractable, ♦ attentive to change,

♦ resourceful, and ♦multipurpose

,♦ emotional, and ♦ illogical♦multipurpose

To achieve human level intelligence these

♦ illogical

To achieve human-level intelligence theseproperties should be taken into account.


4

T d HT d H L l I lli L l I lli Toward HumanToward Human--Level Intelligence Level Intelligence

Human intelligence develops situated in a multimodal environment [Gibbs, 2005].The human mind makes use ofThe human mind makes use of multiple representations and problem-solving strategies [Fuster, 2003]. The brain consists of functionalThe brain consists of functional modules which are localized in subcortical areas but work togetheron the whole-brain scale [Grillner et al., 2006].al., 2006]. Humans can integrate the multiple tasks into a coherent solution [Jones, 2004]. Humans are versatile and come upHumans are versatile and come up with many new ideas and solutions to a given problem [Minsky, 2006].


5

Learning and Memory as a Substrate for Learning and Memory as a Substrate for Learning and Memory as a Substrate for Learning and Memory as a Substrate for IntelligenceIntelligence

It is our memory that enables us to value everything elsewe possess Lacking memory we would have no abilityIt is our memory that enables us to value everything elsewe possess Lacking memory we would have no abilitywe possess. Lacking memory, we would have no abilityto be concerned about our hearts, achievements, lovedones, and incomes. Our brain has an amazing capacity to

we possess. Lacking memory, we would have no abilityto be concerned about our hearts, achievements, lovedones, and incomes. Our brain has an amazing capacity too es, a d co es. Ou b a as a a a g capac ty tointegrate the combined effects of our past experiencestogether with our present experiences in creating our

o es, a d co es. Ou b a as a a a g capac ty tointegrate the combined effects of our past experiencestogether with our present experiences in creating ourthought and actions. This is all possible by the memoryand the memories are formed by the learning process.thought and actions. This is all possible by the memoryand the memories are formed by the learning process.

McGaugh, J. L. Memory & Emotion: The Making of Lasting Memories, 2003.


6

P i i l f L i E l Id P i i l f L i E l Id Principles of Learning: Early Ideas Principles of Learning: Early Ideas

Aristotle: Three Laws of Association [Crowder, 1976]♦ Similarity♦ C t t♦ Contrast♦ Contiguity

James Mill (1773-1836): Strength Criteria ofJames Mill (1773 1836): Strength Criteria of Association♦ Permanence♦ C t i t♦ Certainty♦ Spontaneity♦ “Mental Compounding”p g

John Stuart Mill (1806-1873)♦ “Mental Chemistry”


7

P i i l f L i M d CP i i l f L i M d CPrinciples of Learning: Modern ConceptsPrinciples of Learning: Modern Concepts

Types of learning: Accretion, tuning, restructuring (e.g., R lh & NRumelhart & Norman, 1976)Encoding specificity

i i l (T l i 1970’ )principle (Tulving, 1970’s)Cellular and molecular basis of learning and

(K d l t lmemory (Kandel et al., 1990’s) Conceptual blend and chemical scramble (e gchemical scramble (e.g., Feldman, 2006)


8

M th d f M hi L iMethods of Machine Learning

Symbolic Learning♦ Version Space Learning

Probabilistic Learning♦ Bayesian Networks

H l h l M hi♦ Case-Based LearningNeural Learning♦ M ltil P t

♦ Helmholtz Machines♦ Latent Variable Models♦ Generative Topographic ♦ Multilayer Perceptrons

♦ Self-Organizing Maps ♦ Support Vector Machines

p g pMapping

Other Machine Learning Methodspp

Evolutionary Learning♦ Evolution Strategies

Methods♦ Decision Trees♦ Reinforcement Learning

♦ Evolutionary Programming♦ Genetic Algorithms♦ Genetic Programming

♦ Boosting Algorithms♦ Kernel Methods♦ Independent Component♦ Genetic Programming ♦ Independent Component

Analysis

Three Fundamental Principles of Cognitive Three Fundamental Principles of Cognitive Three Fundamental Principles of Cognitive Three Fundamental Principles of Cognitive Learning: Our ProposalLearning: Our Proposal

Continuity. Learning is a continuous, lifelong process. “The experiences of each immediately past moment are memories that merge with currentmemories that merge with current momentary experiences to create the impression of seamless continuity in our lives” [McGaugh, 2003]Glocality. “Perception is dependent on context” and it is important to maintain both global and local, i.e. glocal representations [Peterson andglocal, representations [Peterson and Rhodes, 2003]Compositionality. “The brain activates existing metaphorical g pstructures to form a conceptual blend, consisting of all the metaphors linked together” [Feldman, 2006]


10

Research Platform for Cognitive Research Platform for Cognitive Research Platform for Cognitive Research Platform for Cognitive LearningLearning

Toward HumanToward Human--Level Machine Learning: Level Machine Learning: Multimodal Memory Game (MMG)Multimodal Memory Game (MMG)

But, I'm getting married tomorrowBut, I'm getting married tomorrowBut, I'm getting married tomorrowBut, I'm getting married tomorrowBut, I'm getting married tomorrowBut, I'm getting married tomorrowBut, I m getting married tomorrowWell, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.

But, I m getting married tomorrowWell, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.But, I'm getting married tomorrowW ll b IBut, I'm getting married tomorrowW ll b IBut, I'm getting married tomorrowW ll b IBut, I'm getting married tomorrowW ll b I

But, I m getting married tomorrowWell, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.



But, I m getting married tomorrowWell, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.g g g pAre you thinking about me?But if you are, call me tonight.

g g g pAre you thinking about me?But if you are, call me tonight.

Well, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.Are you thinking about me?








Are you thinking about me?But if you are, call me tonight.Are you thinking about me?But if you are, call me tonight.Are you thinking about me?But if you are, call me tonight.Are you thinking about me?But if you are, call me tonight.

Image Sound Text

Text HintHint Image


Image-to-Text Generator Text-to-Image GeneratorMachine Learner

T G i G (f I )T G i G (f I )Text Generation Game (from Image)Text Generation Game (from Image)

Image SoundSound Text

L iLearningby ViewingT

I2T T2IGameManager

Text HintT


13

I G i G (f T )I G i G (f T )Image Generation Game (from Text)Image Generation Game (from Text)

TextImage SoundSound

L iLearningby ViewingI

I2T T2IGameManager

Hint ImageI


14

Some Experimental ResultsSome Experimental Results

Th E iTh E iThree ExperimentsThree Experiments

Sentence Generation♦ Learn: a linguistic recall memory from a sentence corpus♦ Gi ti l t t♦ Given: a partial or corrupt sentence♦ Generate: a complete sentence

Image-to-Text Translationg♦ Learn: an image-text joint model from an image-text pair corpus♦ Given: an image (scene)♦ G t t t (di l f th )♦ Generate: a text (dialogue of the scene)

Text-to-Image Translation♦ Learn: an image-text joint model from an image-text pair corpusg j g p p♦ Given: a text (dialogue)♦ Generate: an image (scene of the dialogue)


16

E i 1 L i Li i i ME i 1 L i Li i i MExperiment 1: Learning Linguistic MemoryExperiment 1: Learning Linguistic Memory

Dataset: scripts from dramas♦ Friends♦ House♦ 24♦ Grey Anatomy♦ Grey Anatomy ♦ Gilmore Girls ♦ Sex and the City

Training data: 289,468 sentences Test data: 700 sentences with bl kblanksVocabulary size: 34,219 words


17

S t C l ti R ltS t C l ti R ltSentence Completion ResultsSentence Completion Results

? gonna ? upstairs ? ? a shower I'm gonna go upstairs and take a shower

We ? ? a lot ? giftsWe don't have a lot of gifts

d d? have ? visit the ? roomI have to visit the ladies' room

? ? don't need your ?if I don't need your help

? still ? believe ? did thisI still can't believe you did this

? ? a dream about ? In ?I had a dream about you in Copenhagen

? i i if ? ll h b ? ?

? ? ? decisionto make a decision

I still can't believe you did this

What ? ? ? hereWhat are you doing here

? ? fi ? f di l h l

I had a dream about you in Copenhagen

? appreciate it if ? call her by ? ? I appreciate it if you call her by the way

I'm standing ? the ? ? ? cafeteria I' t di i th f th f t i

Would you ? to meet ? ? Tuesday ? W ld i t t i T d d

? you ? first ? of medical schoolAre you go first day of medical school

Why ? you ? come ? down ? Why are you go come on down here

? think ? I ? met ? somewhere beforeI think but I am met him somewhere before

I'm standing in the one of the cafeteriaWould you nice to meet you in Tuesday and


18

Experiments 2 & 3: Crossmodal Experiments 2 & 3: Crossmodal Experiments 2 & 3: Crossmodal Experiments 2 & 3: Crossmodal TranslationTranslation

D t t d di

The order (k) of hyperedge♦ Text: Order 2~4

Dataset: scenes and corresponding scripts from two dramas♦ Friends

♦ Image: Order 10~340The method of creating hyperedges from training data

♦ Prison Break

Training data: 2,808 scenes and scripts

g♦ Text: Sequential sampling from a

randomly selected position♦ Image: Random sampling in 4,800 p

Scene (image) size: 80 x 60 = 4800 binary pixelsV b l i 2 579 d

g p gpixel positions

Number of samples from an image-text pair

Vocabulary size: 2,579 words

Where am I giving birth

p♦ From 150 to 300

Where am I giving birth

I know it's been really hard for you

So when you guys get in there


19

So when you guys get in there

ImageImage--toto--Text Translation ResultsText Translation Results

Matching &

ImageImage toto Text Translation ResultsText Translation Results

AnswerQuery

I don't know

gCompletion

I don't know what happeneddon't know whatknow what happened

There's a kitty in my guitar case

There's aa kitty in

case…in my guitar case

Maybe there's something I can do to make sure I get pregnant

Maybe there's somethingthere's something I

…pregnantI get pregnant


TextText--toto--Image Translation ResultsImage Translation Results

Matching &

TextText toto Image Translation ResultsImage Translation Results

Query Matching &Completion Answer

I don't know what happened

Take a look at this

There's a kitty in my guitar case

Maybe there's something I can d t k I t tdo to make sure I get pregnant


Hypernetwork Architecture for Hypernetwork Architecture for Hypernetwork Architecture for Hypernetwork Architecture for Cognitive LearningCognitive Learningg gg g

x1=1

x2=0

x3=0

x4=1

x5=0

x6=0

x7 =0

x8=0

x9=0

x10=1

x11=0

x12=1

x13=0

x14=0

x15=0

y= 1

x1=0

x2=1

x3=1

x4=0

x5=0

x6=0

x7 =0

x8=0

x9=1

x10=0

x11=0

x12=0

x13=0

x14=1

x15=0

y= 0

1

2

Learningx1=0

x2=0

x3=1

x4=0

x5=0

x6=1

x7 =0

x8=1

x9=0

x10=0

x11=0

x12=0

x13=1

x14=0

x15=0

y=1

4 Data Items3

x1=0

x2=0

x3=0

x4=0

x5=0

x6=0

x7 =0

x8=1

x9=0

x10=0

x11=1

x12=0

x13=0

x14=0

x15=1

y=14

x1x2 x15

x4 x10 y=1x1Round 1Round 2Round 3

x3 x144 10 y1

x4 x12 y=1x1

x10 x12 y=1x4

1

x12

x4 x13x3 x9 y=0x2

x3 x14 y=0x22x12

x5x9 x14 y=0x3

x6 x8 y=1x3

x6 x11x6 x13 y=1x3

x8 x13 y=1x6

3


23x8 x9

x7x10x11 x15 y=0x84

k f D A l lHypernetwork of DNA Molecules

[Zhang, DNA-2006]

Hypernetwork as Chemical Associative Hypernetwork as Chemical Associative Hypernetwork as Chemical Associative Hypernetwork as Chemical Associative MemoryMemory

WSXH ),,( asdefined isrk hypernetwoThe

= ∑∑ −−−= )()()()3()()()2()( ...61

21);(

rkhypernetwo theofenergy The

iiiiiiiiii

nnnnnn xxxwxxwWE x

K

iii

i SkXSSSxxxX I

)()3()2(

,...,,

|| , , )( 21

=⊆=

=

∑

∑∑

−=

321321321

212121

)()(

,,,

)];(exp[1)|(

ondistributiy probabilit The62

)(iiiiiiiiii

nn

iiiii

WEWP xx β

Nn

K

D

WWWW

1)(

)()3()2(

}{ :set Training

),...,,(

x=

=

∑∑ ⎥⎦

⎤⎢⎣

⎡++=

321321321

212121

,,

)()()()3(

,

)()()2( ...61

21exp

)Z(1

)];(exp[)Z(

)|(

iiiiiiiiiiiii

nnn

ii

nn xxxwxxwW

WEW

WP xx β

1}{

∑ ∑ ⎥⎦

⎤⎢⎣

⎡=

⎦⎣

= 2121...21

32121

2 ,...,,

)()()()(

,,,

,...)(

1exp)Z(

1

)(

kkiiikiii

K

k iii

nnnk

iiiii

xxxwkcW

∑ ⎥⎦

⎤⎢⎣

⎡∑ ∑==)( 21

21...212 ,...,,

)()()()( ...)(

1exp)Z(

isfunction partition thewhere

m kkiiikiii

K

k iii

mmmk xxxwkc

Wx

[Zhang, DNA-2006]


25For more details: Zhang, B.-T., IEEE Computational IntelligenceMagazine, August 2008 (in press)

I t T t (R ll R t )I t T t (R ll R t )Image to Text (Recall Rate)Image to Text (Recall Rate)

0 9

1

Recall Rate

0.7

0.8

0.9

0 4

0.5

0.6

Rat

e

0.2

0.3

0.4

Perfect Recall

0

0.1

10 40 70 100 130 160 190 220 250 280 310 340

Tolerant Recall


26Image Order

T t t I (R ll R t )T t t I (R ll R t )Text to Image (Recall Rate)Text to Image (Recall Rate)

1

Recall Rate

0.7

0.8

0.9

0.4

0.5

0.6

Rat

e

0.1

0.2

0.3

0

0.1

2 3 4 5 6

Text Order


27

Text Order

Toward HumanToward Human--Level IntelligenceLevel Intelligence

F Mi d M l l d B kF Mi d M l l d B kFrom Mind to Molecules and BackFrom Mind to Molecules and BackMind

BrainBrain

Cell

Molecule∞ memory

Molecule1011 cells

>103 molecules


29

P di f C t ti l I t lliP di f C t ti l I t lliParadigms for Computational IntelligenceParadigms for Computational Intelligence

Symbolism Connectionism Dynamicism Hyperinter-actionism

Metaphor symbolt

neuralt dynamical system biomolecular

tp system system y y systemMechanism logical electrical mechanical chemical

Description syntactic functional behavioral relational

Representation localist distributed continuous collective

Organization structural connectionist differential combinatorial

Adaptation substitution tuning rate change self-assembly

Processing sequential parallel dynamical massively parallel

Structure procedure network equation hypergraphStructure procedure network equation hypergraph

Mathematics logic, formallanguage

linear algebra,statistics geometry, calculus graph theory,

probabilistic logicS /ti f l ti l t l ti t l


30

Space/time formal spatial temporal spatiotemporal

[Zhang, IEEE Comp. Intel. Mag., August 2008]

S d C l iS d C l iSummary and ConclusionSummary and ConclusionWe argue that understanding and implementing the principles of cognitive learning and memory is a prerequisite to achieving human-level intelligence.gSuggested three principles as the most fundamental to cognitive learning. ♦ Continuity, glocality, compositionality

Proposed the multimodal memory game (MMG) as a research platform forProposed the multimodal memory game (MMG) as a research platform for studying the architectures and algorithms for cognitive learning.Presented the hypernetwork model as a cognitive architecture for learning i MMG i tin an MMG environment.Showed some experimental results to illustrate the usefulness of the platform.♦ Linguistic recall memory or sentence completion♦ Language-vision crossmodal translation tasks

Future work can extend the experimental setups in various dimensions, such


31

p p ,as corpus size, kinds of modality, and learning strategies.

D i ti f th L i R lDerivation of the Learning Rule

⎪⎫⎪⎧ ⎤⎡

∂∂ x )|}({ln

...21

1)(

)( WPw

siii

N K

Nns

∑ ∑ ∑

∑ ∑ ∑

⎪⎬⎫⎪

⎨⎧ ∂

⎥⎤

⎢⎡∂

⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

−⎥⎦

⎤⎢⎣

⎡

∂∂

== =

)(l1

)(ln...)(

1exp 21

21...21...21

)()()()(

1 2 ,...,,

)()()()()(

WZ

WZxxxwkcw

kkiiikiii

siii

N Knnnk

N

n

K

k iii

nnnks

{ }∑

∑ ∑ ∑

−=

⎪⎭

⎪⎬

⎪⎩

⎪⎨ ∂

∂−⎥

⎦⎢⎣∂

∂=

= =

x......

)(ln...)(

1exp

2121

...212121...21

...21

)|(

)()()(

1)(

2 ,...,,

)()()()()(

xxxxxx

WZw

xxxwkcw

ssiii

siiikkiiikiii

siii

N

WPiiinnn

ns

k iii

nnnks

{ }{ }−=

=

x

x

where

...... 2121

21

)|(

1)|(

xxxxxxNss

s

WPiiiDataiii

nWP

[ ]

[ ]∑

∑

=

==

x )|(

...1...2121

1

)()()(

WPxxxxxx

xxxN

xxxsiiis

N

n

nnn

Dataiii

[ ]∑=x

xx )|(......

2121 )|(WPxxxxxx

siiis WPiii

C l i f O Mi i W d (1/3)C l i f O Mi i W d (1/3)Completion of One Missing Word (1/3)Completion of One Missing Word (1/3)

700

800Sentences with One Missing Words Completion

500

600

200

300

400

0

100

200Order 2Order 3Order 4

40K 80K 120K 160K 200K 240K 280K 290K

The number of completion increases while the number of training


34sentences become larger.

C l i f T Mi i W d (2/3)C l i f T Mi i W d (2/3)Completion of Two Missing Words (2/3)Completion of Two Missing Words (2/3)

700

800Sentences with Two Missing Words Completion

500

600

200

300

400

0

100


40K 80K 120K 160K 200K 240K 280K 290K

The number of completions increases until the number of missing


35words equals the order – 1. (ex) Orders 3 and 4

C l i f Th Mi i W d (3/3)C l i f Th Mi i W d (3/3)Completion of Three Missing Words (3/3)Completion of Three Missing Words (3/3)

700

800Sentences with Three Missing Words Completion

500

600

200

300

400

0

100


40K 80K 120K 160K 200K 240K 280K 290K

The number of completions rapidly decreases if the number of


36missing words becomes larger than the order. (ex) Orders 2 and 3

Multimodal Memory Game as a Platform Multimodal Memory Game as a Platform Multimodal Memory Game as a Platform Multimodal Memory Game as a Platform for Cognitive Machine Learningfor Cognitive Machine Learning

Image SoundSound Text

LearningI2T Learningby Viewing T2I


37

Ill t ti R ltIll t ti R ltIllustrative ResultsIllustrative Results

Query Completion Classification

who are youwho are you

? are youwho ? you

what are youwho are you

FriendsFriends

who are ? who are you Friends

you need to wear it

? need to wear it you ? to wear ityou need ? wear it

I need to wear ityou want to wear ityou need to wear it

242424you need ? wear it

you need to ? ityou need to wear ?

you need to wear ityou need to do ityou need to wear a

24House24


38

I t T t T l tiI t T t T l tiI

Image to Text TranslationImage to Text TranslationImage Text

- Where am I giving birth- You guys really don't know

thiL i anything- So when you guys get in there

- I know it's been really hardf

Learningby Viewing

for you- …

Question:Where ? I giving ?

User Answer: TextUser


39Where am I giving birth Corpus

T t t I T l tiT t t I T l tiI

Text to Image TranslationText to Image Translation

Text- Where am I giving birth- You guys really don't know

thiL i

Image

anything- So when you guys get in there

- I know it's been really hardf

Learningby Viewing

for you- …

Q tiQuestion:You've been there

UserImage Corpus

Answer:

UserImage Corpus


40

cognitive learning and the multimodal memory game: toward human-level … · 2015. 11. 24. · tlk...

Documents