metacognition for effective deliberation in artificial agents darsana josyula 18 november 2011 bowie...
TRANSCRIPT
Metacognition for Effective Deliberation in Artificial Agents
Darsana Josyula18 November 2011
Bowie State University
Artificial Agents
Deliberation
– Process by which agents plan the tasks to perform in order to accomplish current goals
Action
– Process by which agents perform each task in the plan
Deliberation Time
Static Environments
– Deadlines Dynamic Environments
– Change in Environment
– Preconditions not in effect
– Plans not effective
Approaches to Managing Deliberation Time
Hybrid architectures with deliberative, reactive and action selection components
– Gerhard Lakemeyer (Golog for Robotic Soccer)
– Deliberation time decided by the action selection component
Anytime algorithms (estimates the efficiency of a solution as a factor of algorithm run time)
– Nikos Vlassis
– Deliberation time decided by the process that invokes the anytime algorithm
Approaches to Deliberation Time Management
Metacognition
– Awareness of one's own thoughts and the factors that influence one's thinking
– Based on estimates of allowable deliberation and action times
– Monitoring deliberation and action
– Making adjustments to deliberation and action processes
Metacognition in Deliberation and Action
Metacognition
Deliberation Action
Factors that influence Metacognition
Goals Emotions Resource Constraints Plans Performance Optimization Influence of other Agents
Goals
Set of goals to be achieved Choosing relevant goals
Type of goals
– Mandatory versus Desirable (Needs versus Wants)
– Desires versus Intentions (BDI agents)
– Desires -> Intentions -> Actions
– Maintenance versus Achievement
– Conflicting goals Source of goals
Progress towards goals
Metacognitive Monitoring and Control of Goal Processing
Marking the type of goals
Setting priorities for goals
Maintaining Expectations on progress towards goal
Metacognitive Monitoring and Control of Goal Processing
Expectation failures
– Possible anomalies to be evaluated
– Create new goals to deal with expectation failures
– NAG cycle
– MCL
– Indications, Failures and Resonses
Emotions
Animals that exhibit more emotional behavior tend to be better suited for survival
D. Keltner, “Darwin’s touch: Survival of the kindest,” Psychology Today, February 11 2009.
Emotion has been shown to alter the cognitive process in human beings allowing for different responses to the same problem based on different emotional states.
L. Berkowitz, Causes and Consequences of Feelings, Cambridge University Press, 2000.
Emotions - Russel's Circumplex model of Positive and Negative Affect
Affective states organized in a circular structure in a 2D plane
Emotions arise from cognitive interpretations of neural sensations that are the product of two independent neurophysiological systems
These neurophysiological systems correspond to the pleasure axis and activation axis in the circumplex model
Emotions - Russel's Circumplex model of Positive and Negative Affect
The circumplex model of affects is consistent with findings in cognitive neuroscience, neuroimaging, and developmental studies of affects.
The circumplex model has been used to study the development of affective disorders as well as the genetic and cognitive underpinnings of affective processing within the central nervous system.
Emotions – Pleasure Axis Transitions
Represent the agent’s feelings about its own performance
Correspond to the number of expectation violations that occur
When no expectation violation occurs, the agent is pleased with its performance and hence moves its state to the right on the pleasure axis
When expectation violations occur, the system is frustrated by its inability to quell the violations and hence moves to the left on the pleasure axis
The intensity of the expectation violation (the difference between the observed value and the expected value) decides how far to the left the system moves with respect to its current emotional state
Emotions - Activation Axis Transitions
The activation axis transitions are based on the observations of the system and represent the system’s feeling of stress
As the number of observables that the system has to deal with increases, the system becomes more stressed and hence its emotional state moves upward in the activation axis
As the number of observables decrease, the system can relax and hence its emotional state moves downward in the activation axis
Metacognitive Monitoring and Control of Emotions
Monitoring number of observables Maintaining Expectations on number of observables
Monitoring number of expectation violations
Metacognitive Monitoring and Control of Emotions
In the MCL model, failure nodes activate a set of possible responses and instantiates the highest utility response that corresponds to the type of failure.
Which action is deemed to be the best is a learned metric that could be altered for different emotional states; for instance, when stressed the best action may simply be the quickest, when relaxed the best action may be the slowest.
Plans/Actions to Perform
How a goal is achieved Is a plan to achieve the goal known ?
If a plan is unknown, agent has to create a plan
Are the pros and cons of the plans known or unknown? Which plan is better?
Success rate Costs Resource Usage
Conflicting plans
Metacognitive Monitoring and Control of Plans
Monitoring the success rate of plans adopted Maintaining expectations on rate of success for plans adopted
Monitoring the actual costs for adopted plans and resource usage Maintaining plan cost expectations
Monitoring the resource usage of adopted plans Maintaining resource usage expectations
Monitoring Contradictions Active Logic
Resource Constraints
Time Constraints Deadlines
Resource Constraints Minimize resource usage
Metacognitive Monitoring and Control of Resource Constraints
Monitoring passage of Time Maintaining expectations on time requirements
Active Logic
Monitoring usage of Resources Minimize resource usage
Influence of Other Agents
Competitive versus Cooperative
Cooperative agent changing to competitive or vice versa
Metacognitive Monitoring and Control of Influence of other Agents
Maintaining Expectations on the influence of other agents
Performance Optimization
May adversely influence resource constraints
Very important in competitive settings, but may be important in single agent settings as well
Examples
Get better in achieving a goal Win against an opponent Collect more rewards even at the expense of
spending more resources
Metacognitive Monitoring and Control of Performance Optimization
Monitoring Performance Metrics
Maintaining Expectations of Performance Metrics
Metacognition – Vedantic Underpinnings
Goals
– (Kāma – desire / goal to be achieved) Emotions
– (Krōdha – anger / emotional state) Resource Constraints
– (Lōbha – greed / miserliness) Plans
– (Mōha – delusion / Not seeing pros and cons) Performance Optimization
– (Mada – pride / vanity) Influence of Other Agents
– (Mātsarya – competitiveness)
Conclusion
Monitoring Expectations is the key?