impasses and substates - web.eecs.umich.edusoar/ijcai16/tutorial-2016-substates.pdf · 33 substates...
TRANSCRIPT
![Page 1: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/1.jpg)
1
Impasses and Substates[20 min]
Soar TutorialJuly 2016
![Page 2: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/2.jpg)
22
Impasses
What if insufficient or conflicting knowledge for selecting an operator? Called an impasse in Soar
• when no operator is proposed.– [state no-change]
• when multiple operators are proposed by insufficient preferences to select between them– [tie]
• when an operator is selected, but it can’t be applied by a single rule– [operator no-change]
![Page 3: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/3.jpg)
33
Substates• Substate is created if there is an impasse• Substate has structures that define impasse
– ^superstate– ^impasse – no-change, tie, conflict, … – ^item – tied or conflicted operators
– …
• Substate is context for deliberate reasoning and accessing additional knowledge sources– Long-term memories– External environment– Internal reasoning
• Select and apply operators in substate– Access superstate information through – (<s> ^superstate <ss>)
^superstate^item
^impasse
![Page 4: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/4.jpg)
44
Substates
• Substates are removed when the impasse is resolved– A new operator is selected.
• Results are structures created in substatebut linked to the superstate
• Can have recursive stack of impases/substates
^superstate^item
^impasse
![Page 5: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/5.jpg)
5
Overview One-step Look-ahead Using Selection Problem Space
(on A Table)(on B Table)(on C A)
A BC
move(C,B)
move(B,C)
move(C,Table)
Evaluate-operator(move(C,Table))
(on A Table)(on B Table)(on C A)
move(C,Table)
Tie Impasse
(on A Table)(on B Table)(on C Table)
Evaluation=1
Evaluate-operator(move(C,B)) Evaluate-operator(move(B,C))
copy
Evaluation=0 Evaluation=0Evaluation=1
move(C,Table)>move(C,B)move(C,Table)>move(B,C)move(B,C)=move(C,B)
A B C
ABC
Goal
![Page 6: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/6.jpg)
66
Operator Implementation• Add two numbers by counting up and down.
Add 6 5
65
74
83
92
101
110+
-+-
+-
+-
+-
![Page 7: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/7.jpg)
77
Problem Solving in Substatesuperstate
substate
6add-pair
superstate
adden1
adden25
add-pair
add
operator
add
adden2
superstate
add
operator
6add-pair
adden1
5
add-pair
add count-up-count-down
4
7
adden2
superstate
add
operator
7add-pair
adden1
4
add-pair
add count-up-count-down
3
8
![Page 8: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/8.jpg)
88
Results• No explicit marking of results.
• Structures created by rules that match structures in the substate, but create structures linked to a superstate.
• Side effect of working memory structure.
8
![Page 9: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/9.jpg)
9
Result Examplessuperstate
substate
superstate
substate
new result
superstate
substate new results
superstatepost-impasse
![Page 10: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/10.jpg)
1010
Problem Solving in Substatesuperstate
substate
6add-pair
superstate
adden1
adden25
add-pair
add
operator
add count-up-count-down
6
adden25
adden1
6add-pair
adden1
adden25
add-pair
add
operator
add
6
adden2 5
adden1
4
7
count-up-count-down
6add-pair
adden1
adden25
add-pair
add
operator
add
10
adden2 1
adden1
0
11
11sum
![Page 11: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/11.jpg)
1111
Operator Implementation• Add operator to move in a cardinal direction:
(<o> ^name move ^direction << north south east west >> )
![Page 12: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/12.jpg)
1212
Move Operator
• Need only a proposal.• Apply will be by operators in substate.
If name eater andin direction <dir> there is a non-wall
thenpropose move in direction <dir>
Cardinal directions don’t “blink” during rotate, only during forward
![Page 13: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/13.jpg)
1313
Propose move
If name eater andin direction <dir> there is a non-wall
thenpropose move in direction <dir>
sp {eater*propose*move(state <s> ^name eater
^io.input-link <input>)(<input> ^{ << west east north south >> <dir> } <> wall)
-->(<s> ^operator <op> + =)(<op> ^name move
^direction <dir>)}
![Page 14: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/14.jpg)
1414
Substate Structure(s1 ^superstate nil
^type state...)
(s9 ^attribute operator^choices none^impasse no-change^superstate s1^type state^smem …...)
No ^io structure in substate
![Page 15: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/15.jpg)
1515
Rotate in Substate
# If not facing direction of move (in superstate), propose rotate.# When rotate, will retract this proposal (orientation blinks)# Move operator will not retract (directions don’t blink on rotate)
sp {move*propose*rotate(state <s> ^superstate <ss>)(<ss> ^operator <o>
^io.input-link.orientation <dir>)(<o> ^name move
^direction <> <dir>)-->
(<s> ^operator <op> + =)(<op> ^name rotate)}
sp {apply*rotate(state <s> ^operator.name rotate
^superstate.io.output-link <out>)-->
(<out> ^rotate <r>)}
![Page 16: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/16.jpg)
1616
Forward in Substate
# If facing direction of move (in superstate), propose forward.
sp {move*propose*forward(state <s> ^superstate <ss>)(<ss> ^operator <o>
^io.input-link.orientation <dir>)(<o> ^name move
^direction <dir>)-->
(<s> ^operator <op> + =)(<op> ^name rotate)}
sp {apply*forward(state <s> ^operator.name forward
^superstate.io.output-link <out>)-->
(<out> ^forward <r>)}
![Page 17: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/17.jpg)
1717
Adding Selection Knowledge
Select operators based on value of food they will consume.1. Maintain in working memory information on what
each color is worth.2. Propose operators with value they will get3. Use that values to select operators.
![Page 18: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/18.jpg)
1818
Values in working memory
sp {hierarchical*elaborate*map-object*reward(state <s> ^name eater)
-->(<s> ^color-values <r>)(<r> ^wall -10
^empty 0^red 5^purple 10^green 15^blue 20)}
![Page 19: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/19.jpg)
1919
Numeric Indifferent Rulesp {eater*propose*move
(state <s> ^name eater^io.input-link <input>)
(<input> ^{ << west east north south >> <dir> } { <> wall <color>)
-->(<s> ^operator <o> + =)(<o> ^name move
^direction <dir>^color <color>)}
sp {eater*select*move*operator*indifferent(state <s> ^operator <o> +
^color-values.<color> <value>) (<o> ^name move
^color <color>)-->
(<s> ^operator <o> = <value>)}
![Page 20: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/20.jpg)
2020
Comparison Rulesp {eater*select*move*operator
(state <s> ^name eater^operator <o1> +^operator { <> <o1> <o2> } +^color-values <cv>)
(<cv> ^<color1> <value1>^<color2> < <value1>)
(<o1> ^color <color1>)(<o2> ^color <color2>)-->(<s> ^operator <o1> > <o2>)}
![Page 21: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/21.jpg)
2121
Hierarchical RL• Use RL for move operators
sp {eater*propose*move(state <s> ^name eater
^io.input-link <input>)(<input> ^{ << west east north south >> <dir>}
{ <> wall <color>}) -->
(<s> ^operator <o> +)(<o> ^name move
^direction <dir>^color <color>)}
gp {eater*select*move(state <s> ^name eater
^operator <o> +)(<o> ^name move
^color [ purple red blue green empty ] )-->
(<s> ^operator <o> = 0)}
![Page 22: Impasses and Substates - web.eecs.umich.edusoar/ijcai16/Tutorial-2016-substates.pdf · 33 Substates • Substate is created if there is an impasse • Substate has structures that](https://reader034.vdocuments.us/reader034/viewer/2022050209/5f5ba759ad5a4c39103c9c55/html5/thumbnails/22.jpg)
2222
Hierarchical RL• Use RL for move operatorssp {RL*elaborate*state # same as in normal RL
(state <s> ^name eater^reward-link <rl>^io.input-link.score-diff <d>)
-->(<rl> ^reward.value <d>)}