artificial intelligence – lecture 7130.243.105.49/~mbl/ai/lectures.2010/ai-7.pdf · measure given...
Post on 07-Jun-2020
1 Views
Preview:
TRANSCRIPT
Artificial Intelligence – Lecture 7
2
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Lecture plan
• AI in general (ch. 1)
• Search based AI (ch. 4)
• search, games, planning, optimization
• Agents (ch. 8)
• applied AI techniques in robots, software agents, ...
• Knowledge representation (ch. 2)
• semantic networks, frames, logic, resolution
• Expert systems (ch. 3)
• forward/backward chaining, uncertainty, Bayesian networks
• Natural language processing (ch. 5)
• Machine learning (ch. 7)
• version spaces, decision trees, classification, neural networks
3
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
What is an agent
• An agent interacts with its environments
• Perceive through sensors
• Human agent: eyes, ears, nose etc.
• Robotic agent: cameras, infrared range finder etc.
• Soft agent: receiving keystrokes, network packages etc.
• Act through actuators
• Human agent: hands, legs, mouse etc.
• Robotic agent: arms, wheels, motors etc.
• Soft agent: display, sending network packages etc.
• A rational agent is
• One that does the right thing
• Or one that acts so as to achieve best expected outcome
4
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agents
• Agent can be described as a mapping from percept sequences to actions. • Specifying which action to take for each possible
percept sequence provides the ideal agent.• cf. game playing, optimal strategies
• Table in memory mapping percept sequences to actions• Size of table too large
• To difficult to build that table
• Lack of generality
5
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Rational agents
• Select action expected to maximize the performance measure given evidence provided by percept sequence to date and any prior knowledge. • Performance measure defined the success criteria
• Percept sequence required to not just react to local percepts
• Prior knowledge of environment
6
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Rational agents
• Select action expected to maximize the performance measure given evidence provided by percept sequence to date and any prior knowledge. • Rational thinking may still fail due to lack of information
• Selecting the right performance measure and collecting information part of the problem
• Example: build a vacuum cleaning robot and give the criteria that all visible dirt should be gone when I get home
• Scenario 1: it doesn't see any dirt so it does nothing
• Scenario 2: it hides the dirt
• Success criteria: collect as much dirt as possible
• Scenario 3: It goes outdoors and vaccums up a big pile of dirt
7
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Rational agents
• What is rational?
• The performance measure
• The percept sequence
• What an agent knows about the environment• take all previous percepts into account. Model the environment
• The actions that the agent can perform• consider also future effects of these actions.
An ideal rational agent should do whatever action is expected to maximize it's performance measure, on the basis of the evidence provided by the percept sequence and whatever builtin knowledge the agent has.
Not forgetting to do actions in order to obtain useful information!
8
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Rationality versus perfection
• How and when should we evaluate the performance measure?
• Is it better to take an immediate action giving a small increase in performance, or a an action that will/might give a larger increase in performance later?
• Example: should I withdraw my money from the bank and have fun, or wait until next year when I have more interest? Or 100 years after that? ...
• Amortized costs and utilities
• The utility of getting a reward in the distant future is less (but not zero!) than compared to getting it right now. How much less?
• Autonomy
• Rationality requires exploration, learning and autonomy
9
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agent environment
• PEAS when designing rational agents• Performance
• Environment
• Actuators
• Sensors
10
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agent environment
• Properties of different environments• Fully observable vs. partially observable
• Deterministic vs. stochastic
• Episodic vs. sequential
• Static vs. dynamic
• Discrete vs. continous
• Single agent vs. multiagent
11
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Fully observable vs. partially observable
• Fully observable
• Agent can access the complete state of the environment at all times
• Agent can detect all aspects that are relevant to the choice of action
• Partially observable
• Eg. vacuum cleaner with only local dirt sensor, doesn't know the situation at the other square(s)
• Eg. taxi driver, cannot see where all the other drivers are or their destinations.
12
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Deterministic vs. stochastic
• Deterministic
• Next state of the environment is completely determined by the current state and the agents current action
• This is suitable for eg. the types of statespace search, planning etc. that we have looked at so far
• Eg. vacuum world is deterministic (if we clean a square, it will be clean).
• Stochastic
• Next state in environment affected by current state and unobservable factors (eg. other agents, random outcomes of actions)
• It is stochastic if eg. the suction mechanism sometimes requires multiple tries to remove the dirt.
13
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Episodic vs. sequential
• Episodic• Agents experience is divided into episodes
• Each episode consists of agent perceiving and performing a single action
• Next episode does not depend on the actions in this episode.
• Example:• Classification task: spotting and removing defective
parts on an assembly line are episodic.
• Chessplaying and taxidriving are sequential
14
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Static vs. dynamic environments
• Dynamic
• The environment can change while the agent is deliberating
• Agent is continuously asked what to do next• Thinking means “do nothing”
• Examples
• Taxi driving is dynamic. Other cars and ourselves keep moving while thinking and planning the route
• Playing chess is static
• Semidynamic
• Environment doesn't change, but agents performance score does.
• Eg. chess playing with a clock
15
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Discrete vs. continuous
• Environment states, actions and/or perceptions are discrete (symbols, integers, ...) or continuous (real values). • Eg. chess playing is discrete Taxi driving is continuous
(position is a real value, continuous control signals, continuous percepts).
• The real physical world usually is continuous..
• Common properties of continuous environments• Large amounts of data
• Often sensor noise
16
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Single agent vs. multi agents
• Single agent• eg. vacuum cleaner example
• Multi agent• multiple agents exists in environment
• Cooperative• eg. Multiple vacuum cleaners are cooperative, taxi driving is
partially cooperative (avoid collisions).
• Communication may be required
• Competitive• Eg. chess playing
17
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agent environment
• Examples
• The hardest case• partially observable, stochastic, sequential, dynamic,
continuous, multi agent
18
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Structure of agents
• How does the inside of agents work?• Especially, in addition to their behaviours?
• How can they model the environment
• How to interact with the environment
• ...
19
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agents and environments
• Assumption: agents can perceive their own actions
20
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Agents and environments
• Percept (P)
• Agents perceptual input at any given time
• Percept sequence (P*)
• Complete history of anything it has ever perceived
• Agent function
• A mapping from any given percept sequence to actions
• Agent function is implemented by an agent program
• Agent program
• Runs on the physical agent, to implement f
21
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
What is an agent?
• Agent = Architecture + Program• Hardware and environment (or simulation)
• Inputs/outputs from the program to the hardware
• What do we need to know to design an agent• Possible percepts and actions
• Primitive control signals, or high level actions
• Goals or performance measure
• Model of the environment• Static vs. dynamic environment
• Dealing with uncertainty
22
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Different kinds of agents
• Informal classification of agent programs• Tabledriven agents
• Doesn't work very well
• Simplebased reflex agents• Always does the same thing in the same situation
• Modelbased reflex agents• Always does the same thing when the whole environment and
state is the same
• Goalbased agents• Deliberation (eg. search, planning...) to reach the goal
• Utilitybased agents• Maximize the performance (utility) function
23
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Tabledriven agents
• Agents select actions based on entire percept sequence• Requires a huge table. Compare with search trees for
games. Grows exponentially over length of percept sequences.
• Problems• Memory/space requirement
• Non generic
• Time for constructing the table
• This approach generally does not work!
24
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Simplereflex based agents
• Select actions based on the current percept, ignoring rest of percept history• Memory less. Respond directly to pecercepts
• Can get stuck in local optima
25
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Simple reflex agents
• Problems• Work properly if environment is fully observable
• Limited range of application
• Randomized vs deterministic simple reflex agent• Eg. vacuum cleaning is deprived of location sensor
• Compare roomba vs. hauzen vcre70
26
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Example
• Robot navigating towards goal in continous env.
• Inputs: Goal position, My position, Obstacles in front
Act(Goal, Mypos, Obst):
Angle = atan2(goal , mypos)
If len(goal – mypos) < 0.1: return none
If Obst: return 'turnright
Else if Angle > 0.1: return 'turnright
Else if angle < 0.1: return 'turnleft
Else return 'justforward
27
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Model based reflex agents
• Agent maintain internal state to track aspects of the world that are not evident in current state
• Parts of percept history kept to reflect some unobserved aspects of current state
• Updating internal state information require knowledge:• which perceptual information is significant
• how the world evolves independently
• how our actions affect the world
28
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Goal based agents
• The actiondecision process involve some sort of goal information describing situations that are desirable• Combine goal information with possible actions
proposed by internal state to choose actions leading to goal. Gives flexibility in what the agent can do.
• Search and planning based techniques
29
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Utility based agents
• Crisp goal provides a crude distinction between “happy” and “unhappy” states
• Utility: maximize the agents expected happiness
• eg. choose the quicker, safer more reliable route rather than justs one that works.
• Utility function
• Map a state (or sequence of states) onto a real number denoting degree of “happiness”
• Explicit utility functions provide appropriate tradeoff or uncertainties to be reached of several goals
• Deals with conflicting goals (eg. speed vs. safety)
• Likelihood of success between different plans
30
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Utility based agents
31
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
SenseModelDeliberateAct
• Sensing the environment• Interpret the percepts, handle noise, uncertainty...
• Model the environment• Handle the partial observability of environment
• Deliberation• Eg. planning, search, heuristics, ...
• Act• Implement the
actions as control signalsEnvironmentEnvironment
P
M
C
D
32
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
SenseModelDeliberateAct
• Sensing the environment• Ex: Filter the sonar readings
• Model the environment• Ex: building a map of the room
• Deliberation• Ex: plan a path that brings robot to dirty or unknown
part of room
• Act• Ex: Use lowlevel
actions to implement path following EnvironmentEnvironment
P
M
C
D
33
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Learning agents
• Learning allows the agent to operate in unknown environments and become more competent than it's initial knowledge allows
• Learning algorithms
• Learning agent consists of:
• Learning element: making improvements
• Performance element: selecting external actions
• Critic: determine how performance element should be modified according to learning standard
• Supervised vs. unsupervised learning
• Problem generator: suggest actions that lead to new and informative experiences if agent is willing to explore a little.
34
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Learning agents
35
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Learning agents
• Example: taxi driver makes quick left turn over multiple lanes
• critic observes the shocking language from other drivers
• learning element formulates a rule that this was a bad choice of action
• performance element makes sure to never do it again
• Problem generator may suggest certain actions needing to be investigated in different situations.
• Eg. testing the brakes when driving on a road under new circumstances (eg. wet, icy ...)
• Critic may also observe behaviors of others, to learn from them.
36
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Simulating agents
• Use simulations of robots/agents during development / testing• Cheaper development.
• No risk for dangerous misstakes / costs
• Simplified problem, “perfect” sensors, actions
37
Agents and
Architectures
Sem. Net.
Wordnet
Frames
Cyc/CycL
Do we really want intelligent devices?
top related