concept formation in a design optimization tool wei peng and john s. gero 7, july, 2006

Key Centre of Design Computing and Cognition – University of Sydney

Concept Formation in a Design Optimization Tool

Wei Peng and John S. Gero7, July, 2006


Outlines

Design Optimization Concept formation Concept formation from a situated lens A situated agent-based design optimization tool The agent’s experience and concept formation

engine Prototype system Testing results and future direction


Design Optimization

Three major tasks Interactive process Design knowledge

requirement Application scenario –

how the agent learn to recognize design optimization problem

FormulateReformulate

OBJFConstraints

Design Knowledge

Needs

SelectApply

Optimizers

SolutionEvaluationTrade-Off


Recognition of appropriate optimization model is fundamental to design decision problems Can be expressed into semantic relationships between design elements For example

Focus on learning and adapting the knowledge of recognizing an optimization problem

if all the variables are of continuous typeand all the constraints are linearand the objective function in linearthen conclude that the model is linear programmingand execute linear programming algorithm

Design Optimization Knowledge


Concept Formation (CF)

Concept learning – given a set of examples of some concept/class/category, determine if a given example is an instance of concept

Concept formation – incremental unsupervised acquisition of categories and their intentional descriptions

Concept in designing – a consequence of the situatedness of designing


Situated Agent

Effector

Concept Formation

ExperienceSensor

Designer

Interactions in Designing

Concept – Coupled Interactions in Designing

Virtual Knowledge Flows between two Worlds


Concept Formation through a Situated Lens

Situatedness – notion of conceptual situations that are based on the observers’ experiences and inseparable from interactions (Dewey, 1902)

The concept formation process – the way agent orders its experience in time (Clancey,1999) as conceptual coordination Concept formation framework – in a situated agent (Gero and Fujii, 2000)


Situated Concept Formation

Perceptual Categorization 2

Perceptual Categorization 1

C1

C2what I’m-doing-nowC

time t’

time t

Concept as higher order categorization of a sequence

S1

S2

C1

S3

C2

E1

E2

S4

C3

time t’

time t

time t’’

C: PerceptualCategories

S: Sensory Data

E: Previous Conceptual Coordination

Experience

S1

S2

C1

S3

C2

E1

E2

S4

C3

time t’

time t

time t’’

C: PerceptualCategories

S: Sensory Data

E: Previous Conceptual Coordination

Experience

Situated concept formation


A Constructive Memory Model

SITUATION

EXPERIENCE MEMORIES

The original experiences

Constructed memories

New memories as new interpretations of theexperience

Incorporation of pertinent situation

Experiential response

Adding the constructed memory to the experience

Constructed memory becomes part of the situation


A Situated Agent I

A situated agent contains sensors, effectors, experience and a concept formation engine

A concept formation engine consists of a perceptor, a cue_Maker, a conceptor, a hypothesizer, a validator and related processes

Sense data takes the form of a sequence of actions and their initial descriptions

S (t) {…… “click on objective function text field”, key stroke of “x”, “(”, “1”, “)”, “+”, “x”, “(, “2”, “)”…}

Percepts are intermediate data structures of environment states with multimodal information. It can be described as (Objective Function Object, Objective_Function, “x(1)+x(2)”)


A Situated Agent II

Proto-concepts are initial or intermediate concept structures Tree or rule structures Hypotheses depict the agent’s explanations about failures in

correctly predicting a situation Backward chaining rules Validation allows concepts and hypotheses to be evaluated in

interactions Concepts are grounded proto-concepts or hypotheses Invariants about the agent’s experience


Concept Formation I

Recast Concept Formation in A Constructive Memory Model

EXPERIENCE

HYPOTHESISERh

SENSASOR

ENVIRONMENTe1

PERCEPTOR

CUE_MAKER

CONCEPTOR

s1

p1

Cu

C

e2 e3

s2

p2

Cu

(e1,e2)(e1) s3

p3

(e1,e2,e3)

e Events performed

Memory reactivation

Memory activation Memory construction in reflective learningTime

D

I

MEMORIES

D Deductive learner

I Inductive learnerSensory datas

Perceptsp

Memory cueCu

Concepts learned frominductionC

Hypotheses learned fromdeductionh

2

1 4

3

5

6

64

7

8

n

nn


Concept Formation II

Recast Concept Formation in A Constructive Memory Model

SENSASOR

ENVIRONMENTe1

PERCEPTOR

CUE_MAKER

CONCEPTOR

e2 e3 …

EFFECTOR

VALIDATOR

c

+-

vAffecting

Pull

The end of a design process

Constructive learning

Pull process

Affecting process

Grounding via weight adaptation

v

+ Positive validation

- Negative validation

EXPERIENCE MEMORIES

I

e Events performed ConceptsC

I Inductive learner

Validation functionTime


Learning Scenario I

Agent (22)

AE, DEAE

[1|A,B,C,D,E]……

Agent (t)

Conceptual Experience

Expectation

Hypotheses

Validator

Experience(instance and

property nodes)

A-E Environmentstates

Sensing

Affecting

√ Valid

x Not valid

None

N New label

O Old label

Agent (0)

A B

Agent (1) Agent (4)[1|A,B,C,D,E]

E

Time

End of instance

Agent (20)

AE, DE

[1|A,B,C,D,E]……

After a number ofdesign instances

Environment

Learning Initial Experience

A B

√

F

Environment

Learning a new concept A⌐DF

Agent (21)

AE, DEAE

[1|A,B,C,D,E]……Agent (23)

AE, DEAE

[1|A,B,C,D,E]……

C

√

⌐D

Agent (24)

AE, DEAE

[1|A,B,C,D,E]……

xN

Agent (25)

A^DE, A^⌐DF

[1|A,B,C,D,E][2|A,B,C, ⌐D,F]

……

Conceptual Labeling

Labelling process


System Architecture

Situated Agent-based Design Optimization Tool

Situated Agent

Interface Agent

Matlab(Optimisation

Toolbox)

Wrapper(ToolWrapper

class) M-scriptingAgent

User

CallbackAgent

Sensor

Effector

Concept Formation

Experience


Learning Scenario II

Agent (0)[1|A1]

Environment

Environment

OBJF_Type = Quadratic (A1)

var_Type = Continuous (B1)

Optimizer = Quad-Programming (E1)

Agent (1)[1|A1,B1]

Agent (4)

Quad-Programming

[1|A1,B1,C1,…,E1]

After 7design

instances

Agent (20)

[1|A1,B1,C1,…,E1][2|A2, B2, C2 ,…, E2]

……

C1

1. OBJF_Type (Quadratic)Quad-Programming

2. OBJF_Type (Nonlinear)Nonlin-Programming

3. OBJF_Type (Linear)Lin-Programming

C1

Agent (21)

C1Quad-Programming

[1|A1,B1,C1,…,E1]

OBJF_Type =Quadratic (A8)

Agent (22)

C1Quad-Programming

[1|A1,B1,C1,…,E1]

var_Type = Continuous (B8)

√

Agent (2)[1|A1,B1,C1]

Provide_Hessian = true (C1)

Agent (23)

C1Quad-Programming

[1|A1,B1,C1,…,E1]

X

Provide_Hessian = false (C8)

Agent (25)

[1|A1,B1,C1,…,E1][2|A2, B2, C2 ,…, E2]

……

C2

N

Agent (24)

C1Quad-Programming

[1|A1,B1,C1,…,E1]

Optimizer = Nonlin-Programming (E8)

ConceptualLabeling

C2 (New Concept)

1. Provide_Hessian (false) and Optimal_Achieved(NA) Nonlin-Programming2. Provide_Hessian (false) and Optimal_Achieved(Global-min) Lin-Programming3. Provide_Hessian (true) Quad-Programming (3.0)


The Agent’s Experience

Instance Node(activated)

2

Activation

InhibitionPropertyCohort

InstanceCohort

Property Node(activated)

1

Var_Type

OBJF_Type

OBJF

Optimiser

Cons_Type

Provide_Hessian

o1 o2

c1

c2ft2

ft1

Property Node(inhibited)

Instance Node(inhibited)

h2

h1

f1 f2

v2

v1


The Experiential Response


Grounding Experience I

(a) (b)


Grounding Experience II

Percepts at Runtime

Initial Experience

(a) (b)


Prototype System

Conceptual Knowledge

WrapperInitial ExperienceGrounded Experience

Grounded Experience

Activation Diagram

A

B

C

cues

Activation

Explanation-basedHypotheses

Constructive Learning

Grounding by Weight Adaptation

Activating Existing Experience

Backward-chaining Hypothesizing

Inductive Learning

= Linear = Nonlinear = Quadratic

A conceptual labelis obtained bytraversing from theroot node to a leaf node

Root Node

Leaf nodes represent design decisions for selecting optimizers

OptimizerLin-Programming

(4.0)

OptimizerNonlin-Programming

(2.0)

OptimizerQuad-Programming

(5.0/1.0)

OBJF_Type


Using similar design tasks – linear programming

4.04.55.05.56.06.57.07.58.08.59.0

1 2 3 4 5 6 7 8 9 10

Testing Epoch

Res

pons

e V

alue

10

15

20

25

30

35

40

45

50

Res

pons

e Ti

me Response

Value (Ra)

Time toEquilibrium(Te)

Test I


Using novel design optimization scenarios {L, Q, Q, L, NL, Q, NL, L, L, NL, Q, Q, L, L, L} Initial experience – a quadratic experience Behaviour charts and characteristics Performance (prediction rate) for a static, reactive and situated system:

testtheinspredictionofnumbersTotalspredictioncorrectofNumberPs

Test II


Stage II Stage III

e1 e2 e3 e 4 e5 e6 e7 e8

9e1 e2 e3 e 4

10 11e1 e2 e3 e 4 e5 e6

12e1 e2 e3 e 4 e5 e6 e1 e2 e3 e 4

13e1 e2 e3 e 4

14e1 e2 e3

15

Sensation

Perception

Reaction

Validation

Grounding

Reflection I

Conception

Hypothesizing

Reflection II

Reflexion

C-Learning

Tasks and Events

Beh

avio

urs

PerceptionConceptionValidationHypothesizingReaction

Reflection I – reflection via reactivated experienceReflection II – reflection via reactivated experience on hypothesis

Constructive learning (C-Learning)Grounding via weight adaptationReflexion

Learning stage III

SensationLearning stage I

Learning stage IITime

Agent Behaviour

Stage I

1e1 e2 e3 e 4

2 3 4 5 6 7e1 e2 e3 e 4 e5 e6 e1 e2 e3 e 4 e1 e2 e3 e1 e2 e3 e 4 e5 e6 e1 e2 e3 e 4 e1 e2 e3 e 4 e1 e2 e3 e 4

8

Sensation

Perception

Reaction

Validation

Grounding

Reflection I

Conception

Hypothesizing

Reflection II

Reflexion

C-LearningStage II

Tasks and Events

Beh

avio

urs

PerceptionConceptionValidationHypothesizingReaction

Reflection I – reflection via reactivated experienceReflection II – reflection via reactivated experience on hypothesis

Constructive learning (C-Learning)Grounding via weight adaptationReflexion

Learning stage III

SensationLearning stage I

Learning stage IITime

Agent Behaviour

Behaviour Charts


0123456789

10111213141516

Stage I Stage II Stage III

0%

10%

20%

30%

Se Pe Cn Va Hy GD C-L Ra Re1 Re2 Rx

Se Pe Cn Va Hy GD C-L Ra Re1 Re2 Rx

(a)

(b)

Per

cent

age

of b

ehav

iour

sin

the

thre

e st

ages

Num

bero

f beh

avio

urs

in th

e th

ree

stag

es

Processes

Processes

Behaviour Characteristics


Static System

0.00.20.40.60.81.0

1 3 5 7 9 11 13 15Task(a)

Per

form

ance

Reactive System

0.00.20.40.60.81.0

1 3 5 7 9 11 13 15Task(b)

Per

form

ance

Situated System

0.00.20.40.60.81.0

1 3 5 7 9 11 13 15Task

(c)

Per

form

ance

Prediction Rates


Summary and Future Work

Concept formation in a situated agent New concept (new knowledge structure) Interaction plays a role in shaping structures and

behaviours Co-evolution relation between structures and

behaviours Future direction 1: maintaining user models in

design interactions Future direction 2: learning from enriched contexts in

design optimisation


The End Thanks!

concept formation in a design optimization tool wei peng and john s. gero 7, july, 2006

Documents