© daniel s. weld 1 take-home handed out friday 12/12 2:30pm due monday 12/15 2:30pm points worth 2...

© Daniel S. Weld 1

Take-Home • Handed out

Friday 12/12 2:30pm• Due

Monday 12/15 2:30pm• Points

Worth 2 problem sets

© Daniel S. Weld 2

Tournament• Wednesday

3 rounds each (unless requested or finals)• Before each battle, teams describe

project 6 min per group Both people to talk Focus on specific approach, surprises, lessons 2 slides max (print on transparencies)

© Daniel S. Weld 3

Report• 8 page limit; 12 pt font; 1” margins• Model on conference paper

Include abstract, conclusions, but no introduction Online and offline aspect of your agent Describe your use of search (if you did) Section on lessons learned / experiments Short section explaining who did what

• Due 12/18 1pm• Points

Worth 40-60% of project score

© Daniel S. Weld 4

Outline• Review of topics• Hot applications

Internet “Search” Ubiquitous computation Crosswords

© Daniel S. Weld 5

573 Topics

Agency

Problem Spaces

Search Knowledge Representati

on

Planning

Perception NLP Multi-

agentRobotic

s

Uncertainty MDPs Supervised

Learning

Reinforcement

Learning

© Daniel S. Weld 6

State Space Search•Input:

Set of states Operators [and costs] Start state Goal state test

•Output: Path Start End May require shortest path

© Daniel S. Weld 7

•GUESSING (“Tree Search”) Guess how to extend a partial solution to a

problem. Generates a tree of (partial) solutions. The leafs of the tree are either “failures” or

represent complete solutions•SIMPLIFYING (“Inference”)

Infer new, stronger constraints by combining one or more constraints (without any “guessing”)Example: X+2Y = 3 X+Y =1 therefore Y = 2

•WANDERING (“Markov chain”) Perform a (biased) random walk through the

space of (partial or total) solutions

© Daniel S. Weld 8

Some Methods• Guessing – State Space Search

1. BFS, DFS2. Iterative deepening, limited discrep3. Bidirectional4. Best-first search, beam5. A*, IDA*, SMA*6. Game tree7. Davis-Putnam (logic)

• Simplification – Constraint Propagation1. Forward Checking2. Path Consistency 3. Resolution

• Wandering – Randomized Search1. Hillclimbing2. Simulated annealing3. Walksat4. Monte-Carlo Methods

Constraint Satisfaction+

© Daniel S. Weld 9

Admissable Heuristics• f(x) = g(x) + h(x)• g: cost so far• h: underestimate of remaining costs

Where do heuristics come from?

© Daniel S. Weld 10

Relaxed Problems• Derive admissible heuristic from exact cost of a solution to a

relaxed version of problem For transportation planning, relax requirement that car has to stay

on road Euclidean dist For blocks world, distance = # move operations heuristic = number

of misplaced blocks What is relaxed problem?

# out of place = 2, true distance to goal = 3• Cost of optimal soln to relaxed problem

cost of optimal soln for real problem


CSP Analysis: Nodes ExploredBT=BM

BJ=BMJ=BMJ2

CBJ=BM-CBJ

FC-CBJ

FC

More

Fewer=BM-CBJ2


Semantics• Syntax: a description of the legal

arrangements of symbols (Def “sentences”)

• Semantics: what the arrangement of symbols means in the world

Sentences

FactsFacts

Sentences

Representation

World

Semantics

Semantics

Inference


Propositional. Logic vs. First Order

Ontology

Syntax

Semantics

Inference Algorithm

Complexity

Objects, Properties, Relations

Atomic sentencesConnectives

Variables & quantificationSentences have structure: termsfather-of(mother-of(X)))

UnificationForward, Backward chaining Prolog, theorem proving

DPLL, GSATFast in practice

Semi-decidableNP-Complete

Facts (P, Q)

Interpretations (Much more complicated)Truth Tables


Computational Cliff• Description logics• Knowledge representation systems


Machine Learning Overview• Inductive Learning

Defn, need for bias, …• One method: Decision Tree Induction

Hill climbing thru space of DTs Missing attributes Multivariate attributes

• Overfitting• Ensembles• Naïve Bayes Classifier• Co-learning


Learning as search thru hypothesis space

Yes

Outlook Temp

Humid Wind

On which attribute should we split?When stop growing tree?


Ensembles of Classifiers• Assume

Errors are independent Majority vote

• Probability that majority is wrong…

• If individual area is 0.3• Area under curve for 11 wrong is 0.026• Order of magnitude improvement!

Ensemble of 21

classifiers

Prob 0.2

0.1

Number of classifiers in error

= area under binomial distribution


PS 3 Feedback• Independence in Ensembles

Classifier AClassifier BClassifier C

Example 1

Example 2

Example 3

X X

Ensemble

X X

XX X XX

66% error66% error66% error100% error


Planning• The planning problem

Simplifying assumptions• Searching world states

Forward chaining (heuristics) Regression

• Compilation to SAT, CSP, ILP, BDD

• Graphplan Expansion (mutex) Solution extraction (relation to CSPs)


Simplifying AssumptionsEnvironment

Percepts Actions

What action next?

Static vs.

Dynamic

Fully Observable vs.

Partially Observable

Deterministic vs.

Stochastic

Instantaneous vs.

Durative

Full vs. Partial satisfaction

Perfectvs.

Noisy


How Represent Actions?• Simplifying assumptions

Atomic time Agent is omniscient (no sensing necessary). Agent is sole cause of change Actions have deterministic effects

• STRIPS representation World = set of true propositions Actions:

• Precondition: (conjunction of literals)• Effects (conjunction of literals)

aa

anorth11 north12

W0 W2W1


STRIPS Planning actions

(:operator walk :parameters (?X ?Y) :precondition (at ?X)

:effect (and (at ?y) (not (at ?x))))

(:operator walk :parameters (?X ?Y) :precondition (and (at ?X)

(neq ?Y ?Z)) :effect (and (at ?y)

(not (at ?x))))

Frame problemQualification problemRamification problem


Markov Decision ProcessesS = set of states set (|S| = n)A = set of actions (|A| = m)Pr = transition function Pr(s,a,s’)

represented by set of m n x n stochastic matriceseach defines a distribution over SxS

R(s) = bounded, real-valued reward functionrepresented by an n-vector


Value Iteration (Bellman 1957)Markov property allows dynamic programming

Value iteration Policy iteration

)'(' )',,Pr(max)()( 1 ss VsassRsV kk

a

ssRsV ),()(0

)'(' )',,Pr(maxarg),(* 1 ss Vsasks k

a

Vk is optimal k-stage-to-go value function

Bellman backup


Dimensions of Abstraction

A B C

A B C

A B C

A B C

A B C

A B C

A B C

A B C

A

A B C

A B

A B C

A

B

C=

5.3

5.3

5.3

5.3

2.9

2.9 9.3

9.3

5.3

5.2

5.5

5.3

2.9

2.79.3

9.0

Uniform

Nonuniform

Exact

Approximate

Adaptive

Fixed


Partial observability• Belief states

POMDP MDP


Reinforcement Learning• Adaptive dynamic programming

Learns a utility function on states• Temporal-difference learning

Don’t update value at every state• Exploration functions

Balance exploration / exploitation• Function approximation

Compress a large state space into a small one

Linear function approximation, neural nets, …


PROVERB


PROVERB• Weaknesses


PROVERB• Future Work


Grid Filling and CSPs


CSPs and IRDomain from ranked candidate list?Tortellini topping:TRATORIA, COUSCOUS, SEMOLINA, PARMESAN,RIGATONI, PLATEFUL, FORDLTDS, SCOTTIES,ASPIRINS, MACARONI, FROSTING, RYEBREAD,STREUSEL, LASAGNAS, GRIFTERS, BAKERIES,…MARINARA, REDMEATS, VESUVIUS, …Standard recall/precision tradeoff.


Probabilities to the Rescue?Annotate domain with the bias.


Solution ProbabilityProportional to the product of the

probability of the individual choices.

Can pick sol’n with maximum probability.Maximizes prob. of whole puzzle correct. Won’t maximize number of words correct.


Trivial Pursuit™Race around board, answer questions.Categories: Geography, Entertainment,

History, Literature, Science, Sports


WigwamQA via AQUA (Abney et al. 00)

• back off: word match in order helps score.• “When was Amelia Earhart's last flight?”

• 1937, 1897 (birth), 1997 (reenactment)• Named entities only, 100G of web pages

Move selection via MDP (Littman 00)• Estimate category accuracy.• Minimize expected turns to finish.


Mulder• Question Answering System

User asks Natural Language question: “Who killed Lincoln?”

Mulder answers: “John Wilkes Booth”• KB = Web/Search Engines• Domain-independent• Fully automated


User

Architecture

?QueryFormulation

??

?

QuestionParsing

SearchEngine

AnswerExtraction

AnswerSelection

Final Answers

QuestionClassification


Experimental Methodology• Idea: In order to answer n questions,

how much user effort has to be exerted• Implementation:

A question is answered if • the answer phrases are found in the result

pages returned by the service, or• they are found in the web pages pointed to by

the results. Bias in favor of Mulder’s opponents


Experimental Methodology• User Effort = Word Distance

# of words read before answers are encountered

• Google/AskJeeves query with the original question


Comparison Results

% Questions Answered

70

60

50

40

30

20

10

0

User Effort (1000 Word Distance)

0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Mulder

Google

AskJeeves


Know It All• Research project started June 2003• Large scale information extraction

Domain-independent extraction PMI-IR Completeness of web


Domain-independent extraction• Cities such as X, Y, Z

• Movies such as X, Y, Z

Proper nouns


TOEFL SynonymsUsed in college applications.fish

(a) scale(b) angle(c) swim(d) dredge

Turney: PMI-IR


Comprehensive CoverageSearch on: boston seattle paris chicago london


Ubiquitous Computing


Mode Prediction


Placelab• Location-aware computing

Adapting

UIs to

Device (&

User)

Characteristics

© daniel s. weld 1 take-home handed out friday 12/12 2:30pm due monday 12/15 2:30pm points worth 2...

Documents