![Page 1: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/1.jpg)
Lecture 1 is based on David Heckerman’sTutorial slides.
(Microsoft Research)
Bayesian Networks
Lecture 1:Basics and Knowledge-
Based Construction
RequirementsRequirements: 50% home works; 50% Exam or a project: 50% home works; 50% Exam or a project
![Page 2: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/2.jpg)
What I hope you will get out of
this course... What are Bayesian networks? Why do we use them? How do we build them by hand? How do we build them from data? What are some applications? What is their relationship to other models? What are the properties of conditional
independence that make these models appropriate?
Usage in genetic linkage analysis
![Page 3: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/3.jpg)
Applications of hand-built Bayes Nets
Answer Wizard 95, Office Assistant 97,2000 Troubleshooters in Windows 98 Lymph node pathology Trauma care NASA mission control
Some Applications of learned Bayes Nets
Clustering users on the web (MSNBC) Classifying Text (spam filtering)
![Page 4: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/4.jpg)
Some factors that support intelligence
Knowledge representation Reasoning Learning / adapting
![Page 5: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/5.jpg)
Artificial Intelligence
![Page 6: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/6.jpg)
Artificial Intelligence is better than
none !
![Page 7: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/7.jpg)
Artificial Intelligence is better than
ours !
![Page 8: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/8.jpg)
Outline for today
Basics Knowledge-based construction Probabilistic inference Applications of hand-built BNs at Microsoft
![Page 9: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/9.jpg)
Bayesian Networks: History
1920s: Wright -- analysis of crop failure 1950s: I.J. Good -- causality Early 1980s: Howard and Matheson, Pearl Other names:
directed acyclic graphical (DAG) models belief networks causal networks probabilistic networks influence diagrams knowledge maps
![Page 10: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/10.jpg)
Bayesian Network
Fuel
FuelFuelGaugeGauge
StartStart
BatteryBattery
EngineEngineTurns OverTurns Over
p(b)
p(t|b)
p(g|f,b)
p(s|f,t)
p(f)
Directed Acyclic Graph, annotated with prob distributions
![Page 11: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/11.jpg)
BN structure: Definition
Missing arcs encode independencies such that
n
iiin xpxxp
11 )|(),,( pa
Fuel
FuelFuelGaugeGauge
StartStart
BatteryBattery
EngineEngineTurns OverTurns Over
p(b)
p(t|b)
p(g|f,b)
p(s|f,t)
p(f)
),|()|(),|()()(
),,,,(
tfspbtpbfgpfpbp
stgfbp
![Page 12: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/12.jpg)
Independencies in a Bayes net
(*))|(),,(1
1
n
iiin xpxxp pa
n
iiin xxxpxxp
1111 ),|(),,(
iii XXX Pa|),,( 11
Many other independencies are entailed by (*): can beread from the graph using d-separation (Pearl)
Example:
![Page 13: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/13.jpg)
Explaining Away and Induced Dependencies
Fuel
Start
TurnOver
|FT
)|( SFT
"explaining away"
"induced dependencies"
![Page 14: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/14.jpg)
Local distributions
Table:Table:p(S=y|T=n,F=e) = 0.0p(S=y|T=n,F=n) = 0.0p(S=y|T=y,F=e) = 0.0p(S=y|T=y,F=n) = 0.99
Fuel(empty, not)
Start
(yes, no)
TurnOver
(yes, no)
T F
S
![Page 15: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/15.jpg)
Local distributions
Tree:Tree:
Fuel(empty, not)
Start
(yes, no)
TurnOver
(yes, no)
T F
STurnOver
Fuel
noyes
empty notempty
p(start)=0
p(start)=0 p(start)=0.99
![Page 16: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/16.jpg)
Lots of possibilities for a local distribution...
y = discrete node: any probabilistic classifier Decision tree Neural net
y= continuous node: any probabilistic regression model Linear regression with Gaussian noise Neural net
)( 1 n,...,xy|xp
node parents
![Page 17: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/17.jpg)
Naïve Bayes Classifier
Class
Input 1 Input 2 Input n...
discrete
![Page 18: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/18.jpg)
Hidden Markov Model
H1
X1
H2
X2
H3
X3
H4
X4
H5
X5
......
discrete, hidden
observations
![Page 19: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/19.jpg)
Feed-Forward Neural Network
X1 X1 X1
Y1 Y2 Y3
hidden layer
inputs
outputs (binary)
sigmoid
sigmoid
![Page 20: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/20.jpg)
Outline
Basics Knowledge-based construction Probabilistic inference Decision making Applications of hand-built BNs at Microsoft
![Page 21: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/21.jpg)
Building a Bayes net by hand(ok, now we're starting to be
Bayesian) Define variables Assess the structure Assess the local probability distributions
![Page 22: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/22.jpg)
What is a variable?
Collectively exhaustive, mutually exclusive values
Error Occured
No Error
![Page 23: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/23.jpg)
Clarity Test: Is the variable knowable in principle
Is it raining? {Where, when, how many inches?} Is it hot? {T 100F , T < 100F}
Is user’s personality dominant or submissive? {numerical result of standardized personality test}
![Page 24: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/24.jpg)
Assessing structure(one approach)
Choose an ordering for the variables For each variable, identify parents Pai such
that
p x x x p xi i i i( | , ) ( | )1 1 pa
i
iii
iin xpxxxpxxp )|(),|(),( 111 pa
![Page 25: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/25.jpg)
Example
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
![Page 26: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/26.jpg)
Example
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
p(f)
![Page 27: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/27.jpg)
Example
p(b|f)=p(b)
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
p(f)
![Page 28: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/28.jpg)
Example
p(b|f)=p(b)p(t|b,f)=p(t|b)
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
p(f)
![Page 29: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/29.jpg)
Example
p(b|f)=p(b)p(t|b,f)=p(t|b)
p(g|f,b,t)=p(g|f,b)
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
p(f)
![Page 30: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/30.jpg)
Example
p(b|f)=p(b)p(t|b,f)=p(t|b)
p(g|f,b,t)=p(g|f,b)
p(s|f,b,t,g)=p(s|f,t)
p(f,b,t,g,s) = p(f) p(b) p(t|b) p(g|f,b) p(s|f,t)
Fuel GaugeGauge StartStartBatteryBattery TurnOverTurnOver
p(f)
![Page 31: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/31.jpg)
Why is this the wrong way?Variable order can be critical
BatteryBatteryTurnOverTurnOverStartStart FuelFuelGauge
![Page 32: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/32.jpg)
A better way:Use causal knowledge
Fuel
GaugeGauge
StartStart
BatteryBattery
TurnOverTurnOver
![Page 33: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/33.jpg)
Conditional Independence Simplifies Probabilistic
Inference
tfb
tb
sgtbfp
sgtbfp
gsp
gsfpgsfp
,,
,
),,,,(
),,,,(
),(
),,(),|(
f b ttfb
tfspbfgpbtpbpfpsgtbfp ),|(),|()|()()(),,,,(,,
Fuel GaugeGaugeBatteryBattery TurnOverTurnOver StartStart
f b t
tfspbtpbfgpbpfp ),|()|(),|()()(
![Page 34: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/34.jpg)
Online Troubleshooters
![Page 35: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/35.jpg)
Define Problem
![Page 36: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/36.jpg)
Gather Information
![Page 37: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/37.jpg)
Get Recommendations
![Page 38: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/38.jpg)
(see Breese & Heckerman, 1996)
Portion of BN for print troubleshooting
![Page 39: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/39.jpg)
Office Assistant 97
![Page 40: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/40.jpg)
Lumière Project
User’s GoalsUser’s Goals
User’s NeedsUser’s Needs
User ActivityUser Activity
(see Horvitz, Breese, Heckerman, Hovel & Rommelse 1998)
![Page 41: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/41.jpg)
Studies with Human Subjects
“Wizard of OZ” experiments at MS Usability Labs
Expert AdvisorExpert Advisor Inexperienced userInexperienced user
User Actions
Typed Advice
![Page 42: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/42.jpg)
.
Activities with Relevance to User’s Needs
Several classes of evidenceSeveral classes of evidence
SearchSearch: e.g., menu surfing: e.g., menu surfing
IntrospectionIntrospection: e.g., sudden pause, slowing of command : e.g., sudden pause, slowing of command streamstream
Focus of attentionFocus of attention: e.g, selected objects: e.g, selected objects
Undesired effectsUndesired effects: e.g., command/undo, dialogue opened : e.g., command/undo, dialogue opened and cancelledand cancelled
Inefficient command sequencesInefficient command sequences
Goal-specific sequences of actionsGoal-specific sequences of actions
![Page 43: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/43.jpg)
Summary so far
Bayes nets are useful because... They encode independence explicitly
more parsimonious models efficient inference
They encode independence graphically Easier explanation Easier encoding
They sometimes correspond to causal models Easier explanation Easier encoding Modularity leads to easier maintenance
![Page 44: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/44.jpg)
Teenage Bayes
MICRONEWS 97:Microsoft Researchers Exchange Brainpower with Eighth-grader
Teenager Designs Award-Winning Science Project
.. For her science project, which she called "Dr. Sigmund Microchip," Tovar wanted to create a computer program to diagnose the probability of certain personality types. With only answers from a few questions, the program was able to accurately diagnose the correct personality type 90 percent of the time.
![Page 45: Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction](https://reader031.vdocuments.us/reader031/viewer/2022032805/56813194550346895d980533/html5/thumbnails/45.jpg)
Artificial Intelligence is a promising fieldalways was, always will be.