all-inclusive [?] generative tasks for aialumni.media.mit.edu/~dustin/mmp-task.pdfall-inclusive [?]...
TRANSCRIPT
![Page 1: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/1.jpg)
Dustin [email protected]
2010
draft: version 3
All-Inclusive [?] Generative Tasks for
AI
Thursday, February 18, 2010
![Page 2: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/2.jpg)
This is a proposal for unifying areas of AI in order to stitch them together to make progress in new directions.
The particular solution outlined within is intended to create new research opportunities, clarify problem formulations, but certainly not to dictate the direction of the entire field or usurp AI’s diversity of goals.
Benchmarks pose a risk, particularly in the hands of research funders, of prematurely optimizing a field. AI has been stunted by bad benchmarks in the past.
This proposal is incomplete; a starting point that needs development and I am distributing it to encourage critique from researchers in all areas of AI.
2
An Explanation of Authorʼs Intentions
Thursday, February 18, 2010
![Page 3: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/3.jpg)
- Turing Test [Turing 1950]
- Chess [Shannon 1950]
- Compression of English Wikipedia Text [Hutter 2006]
3
Previous Suggestions for Canonical AI Tasks
Thursday, February 18, 2010
![Page 4: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/4.jpg)
- Turing Test [Turing 1950]
- Chess [Shannon 1950]
- Compression of English Wikipedia Text [Hutter 2006]
4
Previous Suggestions for Canonical AI Tasks
Thursday, February 18, 2010
![Page 5: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/5.jpg)
- Too difficult? Sufficient but not necessary condition for intelligence.
- Indivisible. Does not give insight into sub-problems and is too hard for researchers to get started on (it’s easier to imagine short-cut solutions than all of the intermediate steps necessary to actually solve the problem)
- Irreproducible: Tests cannot be easily replicated
5
Turing Test
User2: Hello.Judge: Hi, What’s your name?User2: C:\
User1: Hi.Judge: Hi, What’s your name?User1: John.
Thursday, February 18, 2010
![Page 6: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/6.jpg)
6
Previous Suggestions for Canonical AI Tasks
- Turing Test [Turing 1950]
- Chess [Shannon 1950]
- Compression of English Wikipedia Text [Hutter 2006]
Thursday, February 18, 2010
![Page 7: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/7.jpg)
- Claude Shannon had the spirit of the task in mind -- chess’ enormous state-space, he thought, would prohibit the direct engineering of chess playing machines and would encourage research on pattern recognition, learning and other general qualities of problem solving.
- We must assume AI solutions will “overfit” the task specification
- The solution is in software engineering, we must expect straightforward solutions that do not generalize to other classes of problems,
... unless the task itself is an unlimited class of problems!
7
Chess: “Already Implemented”
Thursday, February 18, 2010
![Page 8: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/8.jpg)
8
Previous Suggestions for Canonical AI Tasks
- Turing Test [Turing 1950]
- Chess [Shannon 1950]
- Compression of English Wikipedia Text [Hutter 2006]
Thursday, February 18, 2010
![Page 9: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/9.jpg)
- Hutter prize, 50000€, for compressing human knowledge (wikipedia text).
- Reductionist design: Human knowledge reduces to [the ability to generate redundancies in]Wikipedia articles. Wikipedia articles reduce to a binary string that can be compressed - Similar in spirit to algorithmic information theory; intelligence as a program generator?
- Turing Tar Pit: Theoretician : All Flash animations can be written in Assembly language.
Software Engineer: So what? 9
The Hutter Prize
intelligence 1101000101111001100111101010100111100110011110101010
Thursday, February 18, 2010
![Page 10: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/10.jpg)
10
Qualities of a good task
- Easily understandable: i.e. a researcher studying language acquisition can have gross understanding of how his model affects planning and learning.
- Tests only relevant components of intelligence, abstracting away “irrelevant” parts.
- Continual/Scalable benchmarks that extend from easy to difficult.
- Modular whenever possible, recognizing the merits of specialization (and the limitations of humans).
Thursday, February 18, 2010
![Page 11: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/11.jpg)
11
Qualities of a good task
- Easily understandable: i.e. a researcher studying language acquisition can have gross understanding of how his model affects planning and learning.
- Tests only relevant components of intelligence, abstracting away “irrelevant” parts.
- Continual/Scalable benchmarks that extend from easy to difficult.
- Modular whenever possible, recognizing the merits of specialization (and the limitations of humans).
Thursday, February 18, 2010
![Page 12: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/12.jpg)
12
Easily UnderstandableThe task: An agent is plopped in a strange new world, needs to learn about the environment, learn a language, and use the language. Finally, he is tested on his ability to solve problems as explained to him by others in their common language.
Sound familiar?
Thursday, February 18, 2010
![Page 13: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/13.jpg)
13
Easily Understandable -> VisualThe task: An agent is plopped in a strange new world, needs to learn about the environment, learn a language, and use the language. Finally, he is tested on his ability to solve problems as explained to him by others in their common language.
1. Download and edit initialization file,
2. Program generates world to specs
3. AI learns and is evaluated in this world
have a visual representation of agent’s problem solving!
Thursday, February 18, 2010
![Page 14: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/14.jpg)
14
Qualities of a good task
- Easily understandable: i.e. a researcher studying language acquisition can have gross understanding of how his model affects planning and learning.
- Tests only relevant components of intelligence, abstracting away “irrelevant” parts.
- Continual/Scalable benchmarks that extend from easy to difficult.
- Modular whenever possible, recognizing the merits of specialization (and the limitations of humans).
Thursday, February 18, 2010
![Page 15: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/15.jpg)
- Adaptive: can learn to solve many problems and goals, which may not be fixed in advance.
- Resourceful: can solve problems in many ways
- Reflective: reasons about itself and is pragmatically rational (will reason about itself, to spend time on problems proportionate to expected utility)
- Communicational: can communicate with humans in natural languages.
- “Useful”: helps us to live and better understand Earth (intelligence of a human form, not just any intelligence).
15
Relevant components of intelligence
Thursday, February 18, 2010
![Page 16: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/16.jpg)
- Adaptive: can learn to solve many problems and pursue goals, which may not be fixed in advance. Test the agent on its learning (generate new problems and constrain agent’s life span)
- Resourceful: can solve problems in many ways. Complex environment favors ingenuity in problem solving; evaluation penalizes stubbornness.
- Reflective: reasons about itself and is pragmatically rational (will reason about itself, to spend time on problems proportionate to expected utility) Give agent many goals, evaluate overall achievement
- Communicational: can communicate with humans in natural languages. Evaluate ability to learn and communicate knowledge by how well the agent/listener can use the knowledge to solve problems.
- “Useful”: helps us to live and better understand our world. Generate problems that are relevant to human-level tasks. Use human languages.
16
Relevant components of intelligence
Thursday, February 18, 2010
![Page 17: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/17.jpg)
17
Generating a planning spaceInitialization file
- objects: 3 # generates three unnamed objects with random
# len, width, height, mass [?]
Thursday, February 18, 2010
![Page 18: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/18.jpg)
18
Generating a planning spaceInitialization file
- objects: 3 # generates three unnamed objects with random
# len, width, height, mass [?]
Simulator produces random objects in world:
“o1” “o2” “o3”
Thursday, February 18, 2010
![Page 19: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/19.jpg)
19
Generating a planning spaceInitialization file
- objects: {“bacon”, {name:“pig”,color:pink}, “box”}
Simulator produces random objects in world:
“bacon” “pig” “box”
Thursday, February 18, 2010
![Page 20: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/20.jpg)
20
Generating a planning spaceInitialization file
- objects: {“bacon”, “pig”, “box”} # with default names
- actions: {left,right,forward,back,pickup,drop}
Predefined set of actions.
Could be richer predicate representations. move(direction={l,r,f,b})
Thursday, February 18, 2010
![Page 21: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/21.jpg)
21
Generating a planning spaceInitialization file
- objects: {“bacon”, “pig”, “box”} # with default names
- actions: {left,right,forward,back,pickup,drop}
- obj-obj-relations: {on,in,holding}
Thursday, February 18, 2010
![Page 22: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/22.jpg)
22
Generating a planning spaceInitialization file
- objects: {“bacon”, “pig”, “box”} # with default names
- actions: {left,right,forward,back,pickup,drop}
- obj-obj-relations: {on,in,holding}
- goal_states: 3
Simulator produces random goals from object/relations and values
(these are hidden and must be learned)
G1: in(pig,box) = 2
G2: on(bacon,pig) = 5
G3: \All in(pig,box) = 100
Thursday, February 18, 2010
![Page 23: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/23.jpg)
23
Generating a planning spaceInitialization file
- objects: {“bacon”, “pig”, “box”} # with default names
- actions: {left,right,forward,back,pickup,drop}
- obj-obj-relations: {on,in,holding}
- goal_states: 3
- event_relations: {during,before,after}
Thursday, February 18, 2010
![Page 24: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/24.jpg)
24
Generating a planning spaceInitialization file
- objects: {“bacon”, “pig”, “box”} # with default names
- actions: {left,right,forward,back,pickup,drop}
- obj-obj-relations: {on,in,holding}
- goal_states: 3
- event_relations: {during,before,after}
- events: 2
Simulator produces random hidden external conditions
and events:
E1: on(bacon,pig) -> green(pig)
E2: in(bacon,pig) -> in(Pig,box)
Thursday, February 18, 2010
![Page 25: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/25.jpg)
25
Phase 1: Exploration
Agent builds model of own actions, differentiating the world in order to achieve goals and avoid anti- goals.
e.g.,
? ? ? ? ? ? ? ?
Start States (S1)
Actions (A)
Start States (S1)
Actions (A)
End States (S2)
? ? ? ? ? ? ? ?
End States (S2)
23431134
23431134
23431134
23431134
23431134
23431134
23431134
23431134
Thursday, February 18, 2010
![Page 26: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/26.jpg)
26
Phase 2: Language LearningImplicit goal #1: the agent is rewarded when it utters the correct label when presented the with the corresponding example. Envision a Rosetta Stone kind of interface, iterating through examples and counter-examples of:
- basic properties: map between regions in geometric perceptual space and color/size/etc words (adjectives)
- object labels: map between collections of properties and nouns
- object relation labels: map between (objects re-presented in visual space?) and prepositions denoting spatial configurations.
- action labels: map between actions (possibly with relations to objects, etc) and verbs
- event labels: map between states or transitions between states and verbs.
- event relation labels: map between event states; temporal event relations (durative, punctual, composition, sequence)
Language labels are structured (taxonomic) but doesn’t matter yet.Thursday, February 18, 2010
![Page 27: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/27.jpg)
27
Phase 2: Language LearningImplicit goal #1: the agent is rewarded when it utters the correct label when presented the with the corresponding example. Envision a Rosetta Stone kind of interface, iterating through examples and counter-examples of:
- basic properties: map between regions in geometric perceptual space and color/size/etc words (adjectives)
- object labels: map between collections of properties and nouns
- object relation labels: map between (objects re-presented in visual space?) and prepositions denoting spatial configurations.
- action labels: map between actions (possibly with relations to objects, etc) and verbs
- event labels: map between states or transitions between states and verbs.
- event relation labels: map between event states; temporal event relations (durative, punctual, composition, sequence)
Thursday, February 18, 2010
![Page 28: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/28.jpg)
28
Phase 2: Language Learning
- basic properties: map between regions in geometric perceptual space and color/size/etc words (adjectives)
Hue
Saturation
Brightness
Kinds of English Adjectives [Wikipedia]1. article or pronouns used as adjectives2. intensifier3. quality4. size5. age6. color7. participle8. proper adjective9. noun used as adjectives10. headnoun
“green”
Thursday, February 18, 2010
![Page 29: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/29.jpg)
29
Another agent (blank slate?) is plopped into your world, but doesn’t get to explore. You have to tell ‘em what you’ve learned through exploration [implicit goal #2].
- Using only language, tell another agent:
- what to do / not do,
- how to do it / not do it
Phase 3: Communication
Thursday, February 18, 2010
![Page 30: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/30.jpg)
30
- Story Telling: Tell ‘em what you’ve learned.
- Using only language, tell another agent
- what to do / not do,
- how to do it / not do it
- Story Listening: Learn what to do!
- Listen to another agent describe how to solve problems in his foreign world!
Phase 3: Communication
Thursday, February 18, 2010
![Page 31: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/31.jpg)
31
Overall Evaluation
Agent 1
Agent 2
Agent 3
Agent 4Agent 5
5 Researchers program 5 AI agents.
Thursday, February 18, 2010
![Page 32: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/32.jpg)
32
Overall Evaluation
- goals: 3- events: 20
Step 1: 5 researchers share a configuration file -- agreeing on the set of labels (words), objects, relations, to evaluate their agents with.
Researchers and their AI agents.
Shared environment initialization file.
Agent 1
Agent 2
Agent 3
Agent 4
Agent 5
Thursday, February 18, 2010
![Page 33: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/33.jpg)
33
Overall Evaluation
- goals: 3- events: 20
Step 1: 5 researchers share a configuration file -- agreeing on the set of labels (words), objects, relations, to evaluate their agents with.
Researchers and their AI agents.
Shared environment initialization file.
Agent 1
Agent 2
Agent 3
Agent 4
Agent 5
Step 2. Each simulator randomly generates hidden goals and events in each environment.
Environment 1
Environment 2
Environment 3
Environment 4
Environment 5
Generated environments
Thursday, February 18, 2010
![Page 34: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/34.jpg)
34
Overall EvaluationStep 3. Each agent explores the environment and then learns the common language to map to states in their environment.
Agent 1
Agent 2
Agent 3
Agent 4
Agent 5
Environment 1
Environment 2
Environment 3
Environment 4
Environment 5
Learning Agents Generated Environments
Step 4: ...
Thursday, February 18, 2010
![Page 35: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/35.jpg)
35
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
Thursday, February 18, 2010
![Page 36: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/36.jpg)
36
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
Bo’s agent 1 explains
goals in env 1 and how
to solve them in
Thursday, February 18, 2010
![Page 37: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/37.jpg)
37
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
All agents try to
follow Agent 1’s stories
to solve problems
in Env 1 without
exploring
Thursday, February 18, 2010
![Page 38: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/38.jpg)
38
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
Dustin’s agent 2 explains
which goals are in env 2
and how to solve them
Thursday, February 18, 2010
![Page 39: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/39.jpg)
39
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
All agents try to
follow Agent 2’s stories
to solve problems
in Env 2
Thursday, February 18, 2010
![Page 40: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/40.jpg)
40
Overall Evaluation
Agent 1 Agent 2
Agent 3
Agent 4
Agent 5
Env 1 Env 2
Env 5
Env 4
Env 3
... and so on ...
Thursday, February 18, 2010
![Page 41: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/41.jpg)
Result
Explaining Listening
Agent 1 good ! impressive!
Agent 2 amazing ! unprecedented!
Agent 3 fantastic ! super!
Agent 4 splendid ! wonderful!
Agent 5 significant ! terrific!
- A report card
- Explaining: representing how well the Agent i performed relative to the other agent’s performance in Environment i.
- Listening: representing how well the agent was able to solve problems in the other environments.
Thursday, February 18, 2010
![Page 42: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/42.jpg)
Result
Explaining Listening
Agent 1 good ! impressive!
Agent 2 amazing ! unprecedented!
Agent 3 fantastic ! super!
Agent 4 splendid ! wonderful!
Agent 5 significant ! terrific!
- A report card
- Explaining: representing how well the Agent i performed relative to the other agent’s performance in Environment i.
- Listening: representing how well the agent was able to solve problems in the other environments.
Thursday, February 18, 2010
![Page 43: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/43.jpg)
- Easily understandable: i.e. a researcher studying language acquisition can have gross understanding of how his model affects planning and learning.
- Tests only relevant components of intelligence, abstracting away “irrelevant” parts.
- Continual/Scalable benchmarks that extend from easy to difficult.
- Modular whenever possible, recognizing the merits of specialization (and the limitations of humans).
43
Qualities of a good task
Thursday, February 18, 2010
![Page 44: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/44.jpg)
44
Continuously Scaling
This simple task framework can improve as researchers build better agents.
- Objects in environment become richer, with textures, parts, mass, shapes, ...
- Agents get more detailed sensory input, and motor controls (actions they can accomplish)
- Researchers hand code common goal structures as benchmarks:
- Other agents in environment, leading to model social phenomenon.
- Extends to the full set of prepositions (spatial and temporal relations) and increases vocabulary.
- Lexicon becomes structured, hierarchical labels (“thing”/”moveable”/”dog”)
Thursday, February 18, 2010
![Page 45: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/45.jpg)
45
Qualities of a good task
- Easily understandable: i.e. a researcher studying language acquisition can have gross understanding of how his model affects planning and learning.
- Tests only relevant components of intelligence, abstracting away “irrelevant” parts.
- Continual/Scalable benchmarks that extend from easy to difficult.
- Modular whenever possible, recognizing the merits of specialization (and the limitations of humans).
Thursday, February 18, 2010
![Page 46: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/46.jpg)
- The components of an agent are modular, and can be shared between researchers (hopefully).
46
An agent
Agent 1
Exploration and Model Learning
Meta-reasoning
Language Generation
Language Comprehension
Language LearningPlanning
Thursday, February 18, 2010
![Page 47: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/47.jpg)
47
Extra Slides(about more specific ideas)
Thursday, February 18, 2010
![Page 48: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/48.jpg)
48
Many ways to learnAn agent is plopped in a strange new world, needs to learn about the environment, learn a language, and use the language. Finally, he is tested on his ability to solve problems as explained to him by others in their common language.
Many ways to learn:
- By observing the effects of your actions (blind search, experimentation,...)
- By observing others (their direct actions, inferring implicit intentions)
- By being told (by someone you trust)
- By reasoning (finding and using patterns in knowledge)
Thursday, February 18, 2010
![Page 49: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/49.jpg)
49
Many ways to learnAn agent is plopped in a strange new world, needs to learn about the environment, learn a language, and use the language. Finally, he is tested on his ability to solve problems as explained to him by others in their common language.
Many ways to learn:
- By observing the effects of your actions (blind search, experimentation,...)
- By observing others (their direct actions, inferring implicit intentions)
- By being told (by someone you trust)
- By reasoning (finding and using patterns in knowledge)
Thursday, February 18, 2010
![Page 50: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/50.jpg)
50
Many ways to learn: By Being Told
The simplest (and most general) model of action/world mapping: a tensor mapping state1,action to state2.
Start States (S1)
Actions (A)
extensional view of f(s1,a) -> s2
End States (S2)
Markov assumption, all states 2s only depend on previous state 1.
MDP: learn action/state conditional probabilities, find optimal action for any state (a policy),
POMDP, the probabilistic generalization of MDPs, introduces observed states, which can be different than actual states.
Thursday, February 18, 2010
![Page 51: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/51.jpg)
51
Many ways to learn: By Being Told
Consider some of the things you know about (poaching an egg, installing RAM, how to placate a baby, copying a DVD) with a quadratic transition space, how can you pick what to do next?
- state spaces are full of rich, reusable structure.
- when you are planning, you can cache/ignore large branches of your decision space.
- Imprimer: puts constraints into planning space (desired or avoided states)
Thursday, February 18, 2010
![Page 52: All-Inclusive [?] Generative Tasks for AIalumni.media.mit.edu/~dustin/mmp-task.pdfAll-Inclusive [?] Generative Tasks for AI Thursday, ... has been stunted by bad benchmarks in the](https://reader034.vdocuments.us/reader034/viewer/2022051723/5ab97e1d7f8b9aa6018df50e/html5/thumbnails/52.jpg)
52
Language LearningAn agent is plopped in a strange new world, needs to learn about the environment, learn a language, and use the language. Finally, he is tested on his ability to solve problems in explained to him by others in their common language.
Language: A reflective level operation
- “situated” in the context of problem solving (semantics of language is now the mapping to the agent’s acquired planning knowledge)
- Adjectives: map to sub-spaces of geometric spaces, for example perceptual topologies.
- Nouns: map to combinations of properties (the regions denoted by adjectives)
- Verbs: map to categories of events (composite relations between objects and properties) and orderings between events.
Thursday, February 18, 2010