integrating high- and low-level expectations in deliberative agents michele piunti -...
TRANSCRIPT
![Page 1: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/1.jpg)
Integrating high- and low-level Expectations in Deliberative Agents
Michele Piunti - [email protected] Institute of Cognitive Sciences and Technologies – ISTC, C.n.r.
João Gonçalves - [email protected] Superior Técnico - IST, Lisbon
![Page 2: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/2.jpg)
1. Towards an integrated architecture
1. Expectations, Emotions, Anticipation
2. High and Low level Expectations
3. From deliberative to anticipatory agents
2. Design
1. Mental States
2. Subjective Expected Utilities
• ISTC: Beliefs and Goal
• IST: Emotivectors
3. Experimental comparision and discussion
4. Future works
Outline
![Page 3: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/3.jpg)
Expectations, Emotions, Anticipation
Agent anticipation in partially observable environments can rely in :
• The ability to adjust quickly to changes (making quick decisions with limited information and bounded resources)
• Catching world dynamics and regularities
• Building representations of future states (Expectations)
• Affective competences (Emotions) via Behavioral and Mental changes:
– Long term : intention reconsideration, attention, resource allocation– Long term: appraisal, belief revision and learning
![Page 4: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/4.jpg)
Explicit Vs. Implicit anticipatory representation:
• Expectations-enabled agents may or not make use of explicit representations of the future world state (and/or of the agent internal state).
• Agent architecture may compute, or not, explicit representation of these states
Anticipatory representations and Computational Models
Cognitive anticipatory agents can be endowed with expectations following different design approaches:
• Statistical learning, prediction mechanisms and component;
• Cognitive (and model driven) architectures:
• top down: Architecture for goal driven, affective and anticipatory agents
• bottom up: distribuited, drives, schema driven design (AKIRA), on line expectations beginning from perception and sensor motor control
![Page 5: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/5.jpg)
Top down approach: from deliberative to anticipatory agents
• Deliberative (Goal directed) agents evaluate and chose between alternative courses of actions and their respective outcomes.
• Anticipatory competences require dealing with uncertainty and bounded knowledge about the future.
We do not introduce Expectations as a new primitive of the architecture but in terms of agent epistemic states (Beliefs) and motivational states (Goals).
![Page 6: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/6.jpg)
Scenario
![Page 7: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/7.jpg)
From deliberative to anticipatory agents
We introduce Expectations at many levels :
1. Weak, low level Expectations as moods and Mental States clustering attitudes in Reasoning.
2. Case based reasoning (means-end reasoning and action-selection processes)
3. Expectation ‘driven’ Deliberation:• Subjective Expected Utility (as a function of the agent’s beliefs and desires
(Bratmann88).• IST: Emotivectors (Matinho 2005)
• Surprise: due to (and signal of) experienced mismatch between ’what is expected’ and ’what is perceived’ (at a given level of representation)
• Expectations are ’prerequisites’ for surprise.• Affective States elicited by Surprise can be described in fuctional terms
beginning from expectations
![Page 8: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/8.jpg)
Clustering Mental States in Reasoning
From the series of local observations of unexpected events stored in a short term memory, an agent controller periodically defines the mental state to adopt through a transition function.
Expectations here have a weak (low level) representation(e.g. negative expectation of risk, threats)
![Page 9: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/9.jpg)
Mental States: fuctional role
• Cautiousness elicits mental and behavioral changes on Short term: alert, to become more vigilant, to look ahead, to check better while and before moving (prudence against threats); Long term augmenting the control or doing the, action in another less risky way, using alternatives in repertoires.
• Excitement: increasing the explorative activity for searching for the ’good’ events.
• Lack of surprises produces a special mood: boredom.
• The persistence of boredom can bring to curiosity, whose outcome is to shift from exploitation to exploration attitudes.
![Page 10: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/10.jpg)
Pay OffsIntention reconsideration is a costly process
Space-Time payoffsResources allocation
To be cautious is advantegeous only in highlythreatful environment (see: energy-time)
Balance of resources is ‘environment dependent’
![Page 11: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/11.jpg)
Subjective Expected UtilityForaging Task: drives and motivations to explore Location of Interest (LOI)are balanced:
DriveToAloi = self_confidence.rule_LOI_a * reward_a.getAvg();DriveToBloi = self_confidence.rule_LOI_b * reward_b.getAvg();DriveToCloi = self_confidence.rule_LOI_b * reward_b.getAvg();
1. Subjective Expected Utility: multiply subjective prevision to find valuables close to LOI and expected reward value (based on a k-history lenght items stored in a working memory).
2. Fully represented in domain of probability.
1. Meta-level planning: ε-Greedy strategies to select ‘best expected’ area (i.e. best SEU) to look for valuables at a.
We integrated 2 mechanismsK-lenght history buffers ◄► emotivector predictors
![Page 12: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/12.jpg)
Analysis of the ISTC Architecture
• Emotivectors model expectations– Prediction– Desired Value– Evaluation
• Expectations identified in 3 levels– Based on beliefs about the world– Associated with mental/emotional states – Associated with goal achievement
![Page 13: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/13.jpg)
Expectations in 3 levels
–Based on beliefs about the world
– Associated with mental/emotional states
– Associated with goal achievement
![Page 14: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/14.jpg)
Modeling Beliefs with the Emotivector – Predictor
• Modeling Food Score– Scenario enhancement
![Page 15: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/15.jpg)
Modeling Beliefs with the Emotivector – Affective Evaluation
• This feeling towards a certain kind of food reflects if its getting better or worst
![Page 16: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/16.jpg)
Action Selection
• Previously the Agent used Subjective Expected Utility (SEU)– Just the predicted energy reward and
probability of success
• Now it as a affective bias on the SEU for a specific kind of food– Affective Subjective Expected Utility
![Page 17: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/17.jpg)
Preliminary Results
• Seasons
• No Seasons
![Page 18: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/18.jpg)
Expectations in 3 levels
– Based on beliefs about the world
– Associated with mental/emotional states
– Associated with goal achievement
![Page 19: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/19.jpg)
Possible Integrations
• Associated with mental/emotional states – Emotivector monitors “rate/number” of positive events
• Anticipates mental state change
– In part done by the weight decay mechanism
• Associated with goal achievement– Very context dependent
![Page 20: Integrating high- and low-level Expectations in Deliberative Agents Michele Piunti - michele.piunti@istc.cnr.itmichele.piunti@istc.cnr.it Institute of](https://reader035.vdocuments.us/reader035/viewer/2022070410/56649ebc5503460f94bc5103/html5/thumbnails/20.jpg)
References
• [Castelfranchi et al., 2006a] C. Castelfranchi, R. Falcone, and M. Piunti. Agents with anticipatory behaviors: To be cautious in a risky environment. In Proc. of European Conf. on Artificial Intelligence, Trento, Italy., 2006.
• [Castelfranchi et al., 2006b] C. Castelfranchi, R. Falcone, and M. Piunti. Developing anticipatory and affective competences in MAS. In Proceedings of InternationalWorkshop on Multi-agent Systems and Simulation (ISC 2006), Palermo, Italy., 2006.
• [Castelfranchi, 2005] C. Castelfranchi. Mind as an anticipatory device: For a theory of expectations. pages 258–276, 2005.
• [C.Martino and Paiva, 2005] C.Martino and A. Paiva. Synthetic emotivectors. In In proc. of Social Intelligence and Interaction in Animals, Robots and Agents AISB, 2005.
• [C.Martino and Paiva, 2006] C.Martino and A. Paiva. Using anticipation to create believable behaviour. In In proc. Of the AAAI 2006 conference, 2006.
• [Falcone et al., 2007] R. Falcone, M. Piunti, and C. Castelfranchi. Surprise as shortcut for anticipation: clustering mental states in reasoning. In In Proc. of IJCAI-07 (to appear), Hyberadad, India., 2007.