learning: introduction and overview chapter 18-21 also: ai 2 nd ed, rich and knight, chapter 17...

Learning: Introduction and Overview

Chapter 18-21

also: AI 2nd ed, Rich and Knight, chapter 17

COMP151April 13, 2007

What is Learning?

• The act, process, or experience of gaining knowledge or skill.

• Psychology. Behavioral modification especially through experience or conditioning.

American Heritage Dictionary

What is Learning?• Learning denotes “changes in the system that

are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more efficiently and more effectively the next time”.

- Herbert A. Simon, 83

• Agents can improve their performance through diligent study of their own experiences.

- Russell & Norvig

What is Learning?

• Working definition for our discussions:Learning is self-modification that results in beneficial changes to behavior.

• Can machines learn?

• If you program a computer, has it learned anything?

• If you add data to a database, is it learning?

Methods of Learning• Rote learning:

– simple storage of computed information

• Learning by taking advice:– similar to rote learning,

but advice may need to be operationalized

• Learning from problem-solving experience:– remembering effective structures and methods

• Learning from examples: – usually involves a teacher who helps to classify things

Why Should Machines Learn?

• Learning is essential for unknown environs– Everything in the environment cannot be

anticipated– designer lacks omniscience

• Learning is an alternative to explicit design– expose the agent to reality rather than

trying to tell it about reality– lazy designer

Learning Agents

Learning Element• Design of a learning element is affected by

– Which components of the performance element are to be learned

– What feedback is available to learn these components

– What representation is used for the components

Performance Element Components

• Mapping from conditions to actions:– Instructor says “brake”, agents learns

condition-action rule for stopping

• Means to infer properties of world from percepts– Agent studies pictures of buses to learn how

to recognize a bus

• Info about how world evolves and results of possible agent actions– Brake on a wet road and observe result

Performance Element Components

• Info about desirability of world states– Agent receives no tip from a passenger of a

rough ride and learns value of steadiness

• Info about desirability of actions

• Goals that describe classes of states that maximize agent’s utility

Types of Feedback• Supervised learning:

– Agent is given correct answers for each example– Agent is learning a function from examples of its

inputs and outputs

• Unsupervised learning: – Agent must infer correct answers– Completely unsupervised learning is impractical,

since agent has no context

• Reinforcement learning: – Agent is given occasional rewards for correct– Typically involves subproblem of learning

“how the world works”

Representation

• The representation of learned knowledge affects the learning method/algorithm– Prop. Logic inductive learning (chap 18)– FOL inductive logic (chap 19)– Mathematical statistical learning (chap 20)

Rote Learning: Example

• Caching or memoization:– Storage of computed results to avoid

recomputation later– Improves performance– Doesn’t improve effectiveness

Supervised Learning• Example: Inductive Learning

Simplest form: learn a function from examples

f is the target function

An example is a pair (x, f(x))

Problem: find a hypothesis hsuch that h ≈ f, given a training set of examples

–•

Inductive Learning• Construct/adjust h to agree with f on training set

(h is consistent if it agrees with f on all examples)• example, curve fitting:

•

•

Inductive Learning

Inductive Learning

Inductive Learning

Inductive Learning

• Ockham’s razor: prefer the simplest hypothesis consistent with data

• Overfitting: finding overly complex functions to account for noise or irrelevant data

Inductive Learning

Learning Decision Trees (LDT)

• Decision trees branch on values of a set of input attributes, leading to answers at the leaves.

• Construction (and optimization) of the tree is a learning problem.– Classification learning: learning a discrete

function– Regression: learning a continuous function

DT Example• Problem: Wait for a table at a restaurant?• Attributes:

Alternate: is there an alternative restaurant nearby?Bar: is there a comfortable bar area to wait in?Fri/Sat: is today Friday or Saturday?Hungry: are we hungry?Patrons: number of people in the restaurant (None, Some, Full)Price: price range ($, $$, $$$)Raining: is it raining outside?Reservation: have we made a reservation?Type: kind of restaurant (French, Italian, Thai, Burger)WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60)

DT Example

• This is a Boolean classification problem

• Example decisions:

•

DT Example

Training and Test Sets• The agent must construct (learn) the DT

from attribute decisions pairs– This is the training set– Restaurant example: training set supplied by

human to mimic own behavior

• Assessing Performance:– Available example data is divided (randomly)

among the training set and the test set.– Test set is then used to predict future

performance

Knowledge in Learning• How can an agent make use of what it already

knows, when learning new things?

Hypothesis Descriptions ╞ Classifications

Hypothesis is unknown: solve constraint from some hypothesis space

• Adding background knowledge

Background Hypothesis Descriptions ╞ Classifications

Cumulative Learning• Cumulative learning:

agent uses and adds to its stock of knowledge

knowledge-based inductive learning

hypothesis

priorknowledge

observations

Statistical Learning

• Learning as uncertain reasoning from observations

• Bayesian learning: computes probabilities for hypotheses and use that to make predictions

• Neural networks: adjusts neuron activation functions based on example input/output pairs

Neural Networks

input bits

output

bits

percepts answers

operational mode

input bits

output

bits

perceptscorrectanswers

training mode

net adjusts its internal parameters by comparing its outputs to the correct answers