1 update on learning by observation learning from positive examples only tolga konik university of...

1

Update on Learning By Observation

Learning from Positive Examples Only

Tolga KonikUniversity of Michigan

2

GOAL Generate AI agents by observing expert

task execution Engineering Goal

Reduce the cost of agent development Reduce the expertise required to develop

agent development.

AI Goal Agents that improve themselves observing

experts

3

Learning Framework

Episodic Database

Behavior trace

rules

Annotations

Agent Architecture

Agent Program

BackgroundKnowledge

examples

Expert

Annotated Behavior trace

Behavior Recorder

Environmental Interface

Training Set Generator

Concept Learner

(ILP)

Knowledge Generator

Environment

external

Internal

4

Learning with Redux

Episodic Database

Behavior trace

rules

Annotations

Agent Architecture

Agent Program

BackgroundKnowledge

examples

Expert

Annotated Behavior trace

Behavior Recorder

Environmental Interface

Training Set Generator

Concept Learner

(ILP)

Knowledge Generator

Environment

external

Internal

Redux

5

Current Experiments

Episodic Database

Behavior trace

rules

Annotations

Agent Architecture

Agent Program

BackgroundKnowledge

examples

Expert

Annotated Behavior trace

Behavior Recorder

Environmental Interface

Training Set Generator

Concept Learner

(ILP)

Knowledge Generator

Environment

external

Internal

Expert Soar Agent

7

Experiments in Haunt 2 Domain

8

d1 d2 d3 d4

Move-to example

move-to-via-nodemove-to-connected-node

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

r3

move-to-area

9

move-to-via-node(Node)

move-to-area(Area)

An Example in Haunt Domain

r1

r2 r3

r4d1

d2d3 d4

d5 d6

move-to-connected-node(Node)

10

r1

r2 r3

r4d1

d2d3 d4

d5 d6

move-to-via-node(Node)

move-to-area(Area)

move-to-connected-node(Node)

An Example in Haunt Domain

11

r1

r3

d1

Correct selection condition for move-to-via-node

move-to-via-node(Node)

move-to-area(Area)

move-to-connected-node(Node)

An Example in Haunt Domain

13

Termination(A)

A

positivenegative

Example GenerationOperator Concepts

14

Selection(A)

A B

positive negative

Example GenerationOperator Concepts

15

A Positive Example: selection(Sit20, move-to-via-node(d1) )

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

Learning Examples

16

General to Special Search with positive and negative examples

17

General to Special Search with positive and negative examples

18

General to Special Search with positive and negative examples

19

General to Special Search with positive and negative examples

20

General to Special Search with positive and negative examples

21

move-to-via-node

Selection(move-to-via-node)

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

move-to-connected-node

Problem in Choosing Parameters

22

move-to-via-node

Positive Negative

Selection(move-to-via-node)

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

move-to-connected-node

Problem in Choosing Parameters

24

General to Specific Learning with Positive Examples Only

Positive

25

General to Specific Learning with Positive Examples Only

d1

Positive

26

A Positive Example of move-to-via-node:

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

Learning Examples

27

Random Examples of move-to-via-node

r1

r2 r3

r4d1

d2d3 d4

d5 d6 i4

i3

d5b d6b

For each positive example, use the same situation with parameters selected in other situations

Learning Examples

28

Nuggets

Move-to operators are learned in Haunt domain ~ 3 mins of trace ~ 35000 situations ~ 10 min to prepare examples ~20 min for learning.

29

Coals

Missing Components It is still research not a tool