© daniel s. weld 1 midterm search space bayes net learning / cross validation lessons for the final...
DESCRIPTION
© Daniel S. Weld 3 Structures Two astronomers in Chile & Alaska measure (M1, M2) the number (N) of stars. Normally there is a chance of error, e, of under or over counting by 1 star. But sometimes, with probability f, a telescope can be out of focus (events F1 and F2) in which case the affected scientist will undercount by 3 stars F1F1 F2F2 M1M1 M2M2 N F1F1 F2F2 M1M1 M2M2 N F1F1 F2F2 M1M1 M2M2 NTRANSCRIPT
© Daniel S. Weld 1
Midterm• Search Space• Bayes Net• Learning / Cross Validation
• Lessons for the Final I’ll hit these areas again I’ll include one midterm problem verbatim Likely others with small changes
© Daniel S. Weld 2
Bayes Nets• Since a high-stakes murder case revolves on a
DNA sample found near the murder scene, the sample is sent to two different labs.
• Each lab uses it’s own machine to determine if there is a match (events M1 and M2).
• The accuracy of each machine depends on whether it is calibrated (C1 and C2).
• The judge hires you to analyze the situation, specifically to advise on the guilt, G, of the suspect.
© Daniel S. Weld 3
Structures
• Two astronomers in Chile & Alaska measure (M1, M2) the number (N) of stars.
• Normally there is a chance of error, e, of under or over counting by 1 star.
• But sometimes, with probability f, a telescope can be out of focus (events F1 and F2) in which case the affected scientist will undercount by 3 stars
F1 F2
M1 M2
N
F1 F2
M1 M2
N
F1 F2
M1 M2
N
© Daniel S. Weld 4
Cross validation
• Partition examples into k disjoint equiv classes• Now create k training sets
Each set is union of all equiv classes except one So each set has (k-1)/k of the original training data
Train Te
st
© Daniel S. Weld 5
Cross Validation
• Partition examples into k disjoint equiv classes• Now create k training sets
Each set is union of all equiv classes except one So each set has (k-1)/k of the original training data
Test
© Daniel S. Weld 6
Cross Validation
• Partition examples into k disjoint equiv classes• Now create k training sets
Each set is union of all equiv classes except one So each set has (k-1)/k of the original training data
Test
© Daniel S. Weld 7
Search• You are given 12 seemingly identical coins and
a two-pan scale. • One of the coins is lighter than or heavier than
the others. • Using the scale, you must identify the bogus
coin and determine if it is lighter than or heavier than the others;
• You are allowed to do at most 3 measurements.
Describe this challenge as a search problem
© Daniel S. Weld 8
Search
• States Partial “plans” to test the coins
• Operators Plan modification operators (e.g. adding a
measurement)
© Daniel S. Weld 9
Need Two Viewpoints• Belief State of Agent
Initially 24 possible world states: Coin A … L is bogus x {heavy, light}
• Plan of Agent And / or tree
measureA vs B …AB vs CD …ABCD vs EFGH
< >=
Al Bl Cl Dl Eh Fh Gh Hh Ah Bh Ch Dh El Fl Gl Hl
Il Ih Jl Jh Kl Kh Ll Lh
© Daniel S. Weld 10
Operators
measure ABCD vs EFGH
< >=
Al Bl C
l Dl E
h Fh G
h Hh
measureA vs B …AB vs CD …ABE vs CDF
< >=
© Daniel S. Weld 11
Administrivia• Reading for today’s class: ch 5
• Reading for next Tues: ch 22
• Problem Set Out soon Programming?
© Daniel S. Weld 12
Activity Recognition Natural
Language
573 Core Topics
Agency
Problem Spaces
Search
Knowledge Representation
ReinforcementLearning
Inference
PlanningClassical,
MDP, POMDP SupervisedLearning
Logic-Based Probabilistic
Constraint Satisfaction
CSE 573University of Washington
© Daniel S. Weld 14
OutlineProblem spaces Search
BlindInformedLocal Heuristics & Pattern DBs forConstraint satisfaction
Definition• Factoring state spaces
Backtracking policies Variable-ordering heuristics Preprocessing algorithms
© Daniel S. Weld 15
Constraint Satisfaction• Kind of search in which
States are factored into sets of variables Search = assigning values to these variables Structure of space is encoded with constraints
• Backtracking-style algorithms work E.g. DFS for SAT (i.e. DPLL)
• But other techniques add speed Propagation Variable ordering Preprocessing
© Daniel S. Weld 16
Chinese Food as Search? • States?
• Operators?
• Start state?
• Goal states?
• Partially specified meals
• Add, remove, change dishes
• Null meal
• Meal meeting certain conditions (rating?)
© Daniel S. Weld 17
Factoring States• Rather than state = meal• Model state’s (independent) parts, e.g.
Suppose every meal for n people Has n dishes plus soup Soup = Meal 1 = Meal 2 = … Meal n =
• Or… physical state = X coordinate = Y coordinate =
© Daniel S. Weld 18
Chinese Constraint Network
Soup
Total Cost< $30
ChickenDish
Vegetable
RiceSeafood
Pork Dish
Appetizer
Must beHot&Sour
No Peanuts
No Peanuts
NotChow Mein
Not BothSpicy
© Daniel S. Weld 19
CSPs in the Real World
• Scheduling space shuttle repair• Airport gate assignments• Transportation Planning• Supply-chain management• Computer configuration• Diagnosis• UI optimization• Etc...
20
[Gajos & Weld, IUI-04][Gajos et al., CHI-08]
Adapting to Devices
© Daniel S. Weld 21
Supple Problem Formulation
+
Hierarchy of State Vars +Methods
Screen Size,Available Widgets &Interaction Modes
Func.Interface
Spec.DeviceModel
User Trace
CustomInterface
Rendering+
Model of an IndividualUser’s Behavior(or that of a Group)
{<root, -, -> <LeftLight:Power, off, on> <Vent, 1, 3> <Projector:Input, video, computer> … }
Approaches:
•Templates•Expert System•Optimization
Functional Spec.
• Typed DAG…Not Tree Interior Nodes Container Types Leaves Simple Types
+ DeviceModel
User Trace
CustomInterface
Rendering+
Func.Interface
Spec.
© Daniel S. Weld 23
Add supple
© Daniel S. Weld 24
Binary Constraint Network• Set of n variables: x1 … xn• Value domains for each variable: D1 … Dn• Set of binary constraints (also “relations”)
Rij Di Dj Specifies which value pairs (xi, xj) are consistent
• V for each country• Each domain = 4
colors• Rij enforces
© Daniel S. Weld 25
Binary Constraint NetworkPartial assignment of values = tuple of pairs
{...(x, a)…} means variable x gets value a...Tuple=consistent if all constraints satisfiedTuple=full solution if consistent + has all vars
Tuple {(xi, ai) … (xj, aj)} = consistent w/ a set of vars {xm … xn}
iff am … an such that {(xi, ai)…(xj, aj), (xm, am)…(xn, an)} } =
consistent
© Daniel S. Weld 26
N Queens• As a CSP?
Variables? Domain? Constraints?
© Daniel S. Weld 27
N Queens• Variables = board columns• Domain values = rows• Rij = {(ai, aj) : (ai aj) (|i-j| |ai-aj|)
e.g. R12 = {(1,3), (1,4), (2,4), (3,1), (4,1), (4,2)}
Q
Q
Q
• {(x1, 2), (x2, 4), (x3, 1)} consistent with (x4)?• Shorthand: “{2, 4, 1} consistent with x4”• {(x1, 2), (x2, 4), (x3, 1)} consistent with (x4) ?• Shorthand: “{2, 4, 1} consistent with x4”
© Daniel S. Weld 28
Cryptarithmetic SEND+ MORE------ MONEY
• State Space Set of states Operators [and costs] Start state Goal states
• Variables?• Domains (variable values)?• Constraints?
© Daniel S. Weld 29
Classroom Scheduling• Variables?
• Domains (possible values for variables)?
• Constraints?
© Daniel S. Weld 30
CSP as a search problem?• What are states?
(nodes in graph)• What are the operators?
(arcs between nodes)• Initial state?• Goal test?
Q
Q
Q
© Daniel S. Weld 31
Chronological Backtracking (BT) (e.g., depth first
search)
Q
Q
Q
Q
Q
Q
Q
Q
Q
Q
Q
1
2
34
5
6
Consistency check performed in the order in which vars were instantiatedIf c-check fails, try next value of current varIf no more values, backtrack to most recent var
© Daniel S. Weld 32
Backjumping (BJ)• Similar to BT, but
more efficient when no consistent instantiation can be found for the current var
• Instead of backtracking to most recent var… BJ reverts to deepest var which was c-checked
against the current var
BJ Discovers (2, 5, 3, 6) inconsistent with x6
No sense trying other values of x5
Q
Q
Q
Q
Q
© Daniel S. Weld 33
5
Conflict-Directed Backjumping (CBJ)• More sophisticated backjumping behavior
• Each variable has conflict set CS Set of vars that failed c-checks w/ current val Update this set on every failed c-check
• When no more values to try for xi Backtrack to deepest var, xd, in CS(xi) And update CS(xd):=CS(xd)CS(xi)-{xd}
CBJ Discovers(2, 5, 3) inconsistent with {x5, x6 }
Q
Q
Q
Q
Q
1 13
23
3 3
2123456
x1 x2 x3 x4 x5 x6
CS(x5)1,2,3
CS(x6)1,2,3,5
Too complex to explain
(Animation may be incorrect)
© Daniel S. Weld 34
BT
{vs. BJ vs. CBJ
Consistent node
Inconsistent node
© Daniel S. Weld 35
Forward Checking (FC)• Perform Consistency Check Forward• Whenever a var is assigned a value
Prune inconsistent values from As-yet unvisited variables Backtrack if domain of any var ever collapses
Q
Q
Q
Q
Q
FC only visits consistent nodes but not all such nodes skips (2, 5, 3, 4) which CBJ visitsBut FC can’t detect that (2, 5, 3) inconsistent with {x5, x6 }
© Daniel S. Weld 36
Number of Nodes ExploredBT=BM
BJ=BMJ=BMJ2
CBJ=BM-CBJ
FC-CBJ
FC
More
Fewer=BM-CBJ2
© Daniel S. Weld 37
Number of Consistency Checks
BMJ2
BT
BJ
BMJ
BM-CBJ
CBJFC-CBJ
BM
BM-CBJ2
FC
More
Fewer
© Daniel S. Weld 38
Crosswords