c463 / b551 artificial intelligence

C463 / B551C463 / B551Artificial IntelligenceArtificial Intelligence

Dana VrajitoruDana Vrajitoru

Intelligent AgentsIntelligent Agents

Artificial Intelligence – D. VrajitoruArtificial Intelligence – D. Vrajitoru

Intelligent AgentIntelligent Agent

AgentAgent: entity in a program or environment : entity in a program or environment capable of generating action.capable of generating action.

An agent uses An agent uses perceptionperception of the environment to of the environment to make decisions about actions to take.make decisions about actions to take.

The perception capability is usually called a The perception capability is usually called a sensorsensor..

The actions can depend on the most recent The actions can depend on the most recent perception or on the entire history (percept perception or on the entire history (percept sequence).sequence).


Agent FunctionAgent Function

The The agent functionagent function is a mathematical is a mathematical function that maps a sequence of function that maps a sequence of perceptions into action.perceptions into action.The function is implemented as the The function is implemented as the agent agent programprogram..The part of the agent taking an action is The part of the agent taking an action is called an called an actuatoractuator..environment environment sensors sensors agent function agent function actuators actuators environment environment


Sensors

Percept (Observations)

Actuator

Action

Environment

Environment

EnvironmentEnvironment

Agent

Agent Function


Rational AgentRational Agent

A A rational agentrational agent is one that can take the is one that can take the right decision in every situation.right decision in every situation.

Performance measurePerformance measure: a set of criteria/test : a set of criteria/test bed for the success of the agent's bed for the success of the agent's behavior.behavior.

The performance measures should be The performance measures should be based on the desired effect of the agent based on the desired effect of the agent on the environment.on the environment.


RationalityRationality

The agent's rational behavior depends on:The agent's rational behavior depends on: the performance measure that defines the performance measure that defines

successsuccess the agent's knowledge of the environmentthe agent's knowledge of the environment the action that it is capable of performingthe action that it is capable of performing the current sequence of perceptions.the current sequence of perceptions.

DefinitionDefinition: for every possible percept sequence, : for every possible percept sequence, the agent is expected to take an action that will the agent is expected to take an action that will maximize its performance measure.maximize its performance measure.


Agent AutonomyAgent Autonomy

An agent is An agent is omniscientomniscient if it knows the actual if it knows the actual outcome of its actions. Not possible in practice.outcome of its actions. Not possible in practice.An environment can sometimes be completely An environment can sometimes be completely known in advance.known in advance.ExplorationExploration: sometimes an agent must perform : sometimes an agent must perform an action to gather information (to increase an action to gather information (to increase perception).perception).AutonomyAutonomy: the capacity to compensate for : the capacity to compensate for partial or incorrect prior knowledge (usually by partial or incorrect prior knowledge (usually by learning).learning).



Task environment Task environment – the problem that the – the problem that the agent is a solution to.agent is a solution to.Properties:Properties:ObservableObservable - fully or partially - fully or partiallyA fully observable environment needs less A fully observable environment needs less representation.representation.Deterministic or stochasticDeterministic or stochasticStrategicStrategic –deterministic except for the –deterministic except for the actions of other agents.actions of other agents.



Episodic or sequentialEpisodic or sequentialSequentialSequential – future actions depend on the – future actions depend on the previous ones. previous ones. EpisodicEpisodic – individual unrelated tasks for the – individual unrelated tasks for the agent to solve.agent to solve.

Static – dynamicStatic – dynamic

Discrete – continuous Discrete – continuous

Single agent – multi agentSingle agent – multi agentMultiple agents can be competitive or Multiple agents can be competitive or cooperative.cooperative.


More Definitions of AgentsMore Definitions of Agents

""An agent is a An agent is a persistent softwarepersistent software entity dedicated to a entity dedicated to a specific purposespecific purpose.. " (Smith, Cypher, and Spohrer 94 ) " (Smith, Cypher, and Spohrer 94 )"Intelligent agents are software entities that carry out "Intelligent agents are software entities that carry out some set of operations some set of operations on behalf of a useron behalf of a user or another or another program with some degree of independence or program with some degree of independence or autonomy, and in so doing, employ some knowledge or autonomy, and in so doing, employ some knowledge or representation of the user's goals or desires."representation of the user's goals or desires." (IBM) (IBM)"Intelligent agents continuously perform three functions: "Intelligent agents continuously perform three functions: perceptionperception of dynamic conditions in the environment; of dynamic conditions in the environment; actionaction to affect conditions in the environment; and to affect conditions in the environment; and reasoningreasoning to interpret perceptions, solve problems, draw to interpret perceptions, solve problems, draw inferences, and determine actions.inferences, and determine actions. "(Hayes-Roth 94) "(Hayes-Roth 94)


Agent vs. ProgramAgent vs. Program

SizeSize – an agent is usually smaller than a – an agent is usually smaller than a program.program.PurposePurpose – an agent has a specific purpose – an agent has a specific purpose while programs are multi-functional.while programs are multi-functional.PersistencePersistence – an agent's life span is not – an agent's life span is not entirely dependent on a user launching entirely dependent on a user launching and quitting it.and quitting it.AutonomyAutonomy – an agent doesn't need the – an agent doesn't need the user's input to function.user's input to function.


Simple AgentsSimple Agents

Table-driven agentsTable-driven agents: the : the function consists in a function consists in a lookup table of actions to lookup table of actions to be taken for every be taken for every possible state of the possible state of the environment.environment.If the environment has If the environment has nn variables, each with variables, each with tt possible states, then the possible states, then the table size is table size is ttnn..Only works for a small Only works for a small number of possible states number of possible states for the environment.for the environment.

Simple reflex agentsSimple reflex agents: : deciding on the action to deciding on the action to take based only on the take based only on the current perception and current perception and not on the history of not on the history of perceptions.perceptions.Based on the Based on the condition-condition-actionaction rule: rule:(if (condition) action)(if (condition) action)Works if the environment Works if the environment is fully observableis fully observable


(defun (defun table_agenttable_agent (percept) (percept) (let ((action t))(let ((action t)) (push percept percepts)(push percept percepts) (setq action (setq action (lookup percepts table))(lookup percepts table)) action))action))

(defun (defun reflex_agentreflex_agent (percept) (percept) (let ((rule t) (state t) (action t))(let ((rule t) (state t) (action t)) (setq state (interpret percept))(setq state (interpret percept)) (setq rule (match state))(setq rule (match state)) (setq action (decision rule))(setq action (decision rule)) action))action))


percepts = []percepts = []table = {}table = {}def def table_agenttable_agent (percept): (percept): action = Trueaction = True percepts.append(percept)percepts.append(percept) action = lookup(percepts, table)action = lookup(percepts, table) return actionreturn action

def def reflex_agentreflex_agent (percept): (percept): state = interpret(percept)state = interpret(percept) rule = match(state)rule = match(state) action = decision(rule)action = decision(rule) return actionreturn action


Model-Based Reflex AgentsModel-Based Reflex Agents

If the world is not fully observable, the If the world is not fully observable, the agent must remember observations about agent must remember observations about the parts of the environment it cannot the parts of the environment it cannot currently observe.currently observe.This usually requires an internal This usually requires an internal representation of the world (or internal representation of the world (or internal state).state).Since this representation is a model of the Since this representation is a model of the world, we call this model-based agent.world, we call this model-based agent.


(setq state t) ; the world model(setq state t) ; the world model

(setq action nil) ; latest action(setq action nil) ; latest action

(defun (defun model_reflex_agentmodel_reflex_agent (percept) (percept)

(let ((rule t))(let ((rule t))

(setq state (setq state

(update_state (update_state

state action percept))state action percept))

(setq rule (match state))(setq rule (match state))

(setq action (decision rule))(setq action (decision rule))

action))action))


state = True # the world modelstate = True # the world model

action = False # latest actionaction = False # latest action

def def model_reflex_agentmodel_reflex_agent (percept) (percept)

state = update_state(state,state = update_state(state,

action, action,

percept)percept)

rule = match(state)rule = match(state)

action = decision(rule)action = decision(rule)

return actionreturn action


Goal-Driven AgentsGoal-Driven Agents

The agent has a The agent has a purposepurpose and the action to be and the action to be taken depends on the current state and on what taken depends on the current state and on what it tries to accomplish (the goal).it tries to accomplish (the goal).

In some cases the goal is easy to achieve. In In some cases the goal is easy to achieve. In others it involves others it involves planningplanning, sifting through a , sifting through a searchsearch space for possible solutions, developing space for possible solutions, developing a a strategystrategy..

Utility-based agentsUtility-based agents: the agent is aware of a : the agent is aware of a utility function that estimates how close the utility function that estimates how close the current state is to the agent's goal.current state is to the agent's goal.


Learning AgentsLearning Agents

Agents capable of acquiring new Agents capable of acquiring new competence through observations and competence through observations and actions.actions.Components: Components: learning element (modifies the performance learning element (modifies the performance

element)element) performance element (selects actions)performance element (selects actions) feedback element (critic)feedback element (critic) exploration element (problem generator).exploration element (problem generator).


Other Types of AgentsOther Types of Agents

Temporarily continuousTemporarily continuous – a continuously running – a continuously running process,process,CommunicativeCommunicative agent – exchanging information agent – exchanging information with other agents to complete its task.with other agents to complete its task.MobileMobile agent – capable of moving from one agent – capable of moving from one machine to another one (or from one machine to another one (or from one environment to another).environment to another).FlexibleFlexible agent – whose actions are not scripted. agent – whose actions are not scripted.CharacterCharacter – an agent with conversation skills, – an agent with conversation skills, personality, and even emotional state.personality, and even emotional state.


Agent ClassificationAgent Classification


Agent ExampleAgent Example

A file manager agent.A file manager agent.Sensors: commands like ls, du, pwd.Sensors: commands like ls, du, pwd.Actuators: commands like tar, gzip, cd, rm, Actuators: commands like tar, gzip, cd, rm, cp, etc.cp, etc.Purpose: compress and archive files that Purpose: compress and archive files that have not been used in a while.have not been used in a while.Environment: fully observable (but partially Environment: fully observable (but partially observed), deterministic (strategic), observed), deterministic (strategic), episodic, dynamic, discrete. episodic, dynamic, discrete.

c463 / b551 artificial intelligence

Documents

agent program

environment capable

observable environment

agents behavior

environmentthe action

successthe agents knowledge

possible percept sequence

mathematical function