Instructor : Elham Gholami

Bayesian Belief Network

Today’s Topics

•••

Introduction

Bayesian

BayesianNetwork

NetworkExamples

Evaluation and Learning

Bayesian Belief Network

Today’s Topics

•••

Introduction

Bayesian

BayesianNetwork

NetworkExamples

Evaluation and Learning

Bayesian Belief Network

Introduction

• Graphs are an intuitive way of representing andvisualizing the relationships among many variables.

Probabilistic graphical models provide a tool to deal with two problems: uncertainty and complexity. Hence, they provide a compact representation ofjoint probability distributions using a combination ofgraph theory and probability theory.

The graph structure specifies statistical dependencies among the variables and the local probabilistic models specify how these variables are combined.

•

•

•

Bayesian Belief Network

Introduction

ProbabilityTheory

Probabilisticgraphicalmodels

GraphTheory

Bayesian Belief Network

Introduction

• Two main kinds of graphical models. Nodescorrespond to random variables. Edges represent thestatistical dependencies between the variables.

Bayesian Belief Network

Bayesian Networks

• Bayesian networks (BN) are probabilistic graphicalmodels that are based on directed acyclic graphs.

There are two components of a BN model: M= {G, Θ. {

•

– Each node in the graph G represents a random variableand edges represent conditional independencerelationships.

The set Θ of parameters specifies the probability distributions associated with each variable.CPT (Conditional Probability Table)

–

Bayesian Belief Network

Bayesian Networks

• Edges represent “causation” so no directed cyclesallowed.

are

• Markov property: Each node is conditionallyindependent of its ancestors given its parents.

Bayesian Belief Network

A portion of a belief network, consistsing of a node X, having variable values (x1, x2, ...), its parents (A and B), and its children (C and D)

Assumptions

Bayesian Belief Network

x and z are marginally independent, but when y is given, they are conditionally dependent. This is called explaining away.

When y is given, x and z are conditionally independent. Think of y as the common cause of the two independent effects x and z.

When y is given, x and z are conditionally independent. Think of x as the past, y as the present, and z as the future.

Assumptions

• The independence assumption is of particularimportance to the Bayesian inference.

Bayesian Belief Network

•All of the root nodes are independent of each other. For example, for the two root nodes X1 and X2, it satisfies P(X1X2)=P(X1)P(X2).•If two nodes have common immediate parent nodes and there is no direct arc between these two nodes, they are conditionally independent of each other giventhe states of their immediate parent nodes. For instance, X1 is parent of X3 and X4. Given X1, X3 and X4 are conditionally independent of each other, i.e. P(X3|X1X4)=P(X3|X1).•For any non-root node, it is conditionally independent of its non-immediate parent nodes given the states of all of its immediate parent nodes. For instance, when the immediate parent node X4 is given, X5 is independent of X1, X2, X3, P(X5|X1X2X3X4)=P(X5|X4).

Bayes RuleProduct Rule:

P A B P A B P B

P A B P B A P A

|

|

Equating Sides: P B A

P A B P BP A

|( | ) ( )

( )

P Class evidenceP evidence Class P Class

P evidence|

( | ) ( )( )

i.e.

All classification methods can be seen as estimates of Bayes’ Rule, with different techniques to estimate P(evidence|Class).

Bayesian Networks

•

•

Once we know the joint probability distribution encoded inthe network, we can answer all possible inference questions about the variables using marginalization.

Bayesian Belief Network

Bayesian Networks

Bayesian Belief Network

P(a, b, c, d, e) =P(a)P(b)P(c|b)P(d|a,c)P(e|d)

Bayesian Networks

Bayesian Belief Network

P(a, b, c, d) = P(a)P(b|a)P(c|b)P(d|c)

Bayesian Networks

Bayesian Belief Network

P(e, f, g, h) =P(e)P(f|e)P(g|e)P(h|f, g)

Today’s Topics

•••

Introduction

Bayesian Network ExamplesBayesian Network Evaluation and Learning

Bayesian Belief Network

Example 1

••

You have a new burglar alarm installed at home.

It is fairly reliable at detecting burglary, but also sometimes responds to minor earthquakes.You have two neighbors, Ali and Veli, who promised to call you at work when they hear the alarm.Ali always calls when he hears the alarm, but sometimes confuses telephone ringing with the alarm and calls too.Veli likes loud music and sometimes misses the alarm.

•

•

•• Given the evidence of who has or has

would like to estimate the probabilitynot called, weof a burglary.

Bayesian Belief Network

Example 1

Bayesian Belief Network

The Bayesian network for the burglar alarm example. Burglary (B) and earthquake (E) directly affect the probability of the alarm (A) going off, but whether or not Ali calls (AC) or Veli calls (VC) depends only on the alarm.(Russell and Norvig, Artificial Intelligence: A Modern Approach, 1995)

Example 1

• What is the probability that the alarm has soundedbut neither a burglary nor an earthquake hasoccurred, and both Ali and Veli call?

capital letters represent variables havingtrue, and ¬ represents negation

the value

Bayesian Belief Network

•

Example 1

• What is the probability that there is a burglary giventhat Ali calls?

What about if Veli also calls right after Ali hangs up?

Bayesian Belief Network

Example 2

Bayesian Belief Network

Another Bayesian network example.The event that the grass being wet (W = true) has two possiblecauses: either the water sprinkler was on (S = true) or it rained (R = true).

(Russell and Norvig, Artificial Intelligence: A Modern Approach, 1995)

a

Example 2

• Suppose we observe the fact that the grass is wet.There are two possible causes for this: either itrained,likely?

or the sprinkler was on. Which one is more

We see that it is more likely that the grass is wetbecause it rained.

Bayesi n Belief Network

Applications of Bayesian Networks

• Example applications include:––––––––

Machine learning

StatisticsComputer visionNatural language ProcessingSpeech recognitionError-control codesBioinformatics Medical

Weather forecastingdiagnosis

Bayesian Belief Network

Applications of Bayesian Networks

• Example systems include:–––

PATHFINDER medical diagnosis system at Stanford

Microsoft Office assistant and troubleshootersSpace shuttle monitoring system at NASA Mission ControlCenter in Houston

Bayesian Belief Network Dr. Shuang LIANG, SSE, TongJi

Data Mining Community Top Resource ( KD nuggest )for Analytics, Data Mining, and Data Science Software, Companies, Data, Jobs, Education, News, and morehttp://www.kdnuggets.com/software/bayesian.html

Bayesian Network ToolsFree:•BAYDA 1.0 •Bayesian belief network software (Win95/98/NT/2000,) from J. Cheng, including BN PowerConstructor: An efficient system for learning BN structures and parameters from data; BN PowerPredictor: A data mining program for data modeling/classification/prediction. •Bayesian Network tools in Java (BNJ): an open-source suite of Java tools for probabilistic learning and reasoning (Kansas State University KDD Lab) • Chordalysis, a log-linear analysis method for big data, which exploits recent discoveries in graph theory by representing complex models as compositions of triangular structures (aka chordal graphs )FDEP, induces functional dependencies from a given input relation(GNU C)•GeNIe, decision modeling environment implementing influence diagrams and Bayesian networks (Windows). Has over 2000 users. •JavaBayes •jBNC, a Java toolkit for training, testing, and applying Bayesian Network Classifiers. •JNCC2, Naive Credal Classifier 2 (in Java), an extension of Naive Bayes towards imprecise probabilities; it is designed to return robust classification, even on small and/or incomplete data sets. •MSBN: Microsoft Belief Network Tools, tools for creation, assessment and evaluation of Bayesian belief networks. Free for non-commercial research users. •PNL: Open Source Probabilistic Networks Library, a tool for working with graphical models, supporting directed and undirected models, discrete and continuous variables, various inference and learning algorithms. •Pulcinella, tool for Propagating Uncertainty through Local Computations based on the Shenoy and Shafer framework. (Common Lisp)

Bayesian Network Tools…commercial :

•AgenaRisk, visual tool, combining Bayesian networks and statistical simulation (Free one month evaluation). •Analytica, influence diagram-based, visual environment for creating and analyzing probabilistic models (Win/Mac). •AT-Sigma Data Chopper, for analysis of databases and finding causal relationships. •BayesiaLab, complete set of Bayesian network tools, including supervised and unsupervised learning, and analysis toolbox. •Bayes Server, advanced Bayesian network library and user interface. Supports classification, regression, segmentation, time series prediction, anomaly detection and more. Free trial and walkthroughs available. •Bayesware Discoverer 1.0, an automated modeling tool able to extract a Bayesian network from data by searching for the most probable model •BNet, includes BNet.Builder for rapidly creating Belief Networks, entering information, and getting results and BNet.EngineKit for incorporating Belief Network Technology in your applications. •DXpress, Windows based tool for building and compiling Bayes Networks. •Flint, combines bayesian networks, certainty factors and fuzzy logic within a logic programming rules-based environment. •HUGIN, full suite of Bayesian Network reasoning tools •Netica, bayesian network tools (Win 95/NT), demo available. •PrecisionTree, an add-in for Microsoft Excel for building decision trees and influence diagrams directly in the spreadsheet

Today’s Topics

•••

Introduction

Bayesian Network ExamplesBayesian Network Evaluation and Learning

Bayesian Belief Network

Two Fundamental Problems for BNs

• Evaluation (inference) problem: Given the model andthe values of the observed variables, estimate thevalues of the hidden nodes.

Learning problem: Given training data and priorinformation (e.g., expert knowledge, causalrelationships), estimate the network structure, or theparameters of the probability distributions, or both.

Bayesian Belief Network

Bayesian Network Evaluation Problem

• If we observe the “leaves” and try to infer the valuesof the hidden causes, this is called diagnosis, orbottom-up reasoning.

If we observe the “roots” and try to predict the effects, this is called prediction, or top-down reasoning.Exact inference is an NP-hard problem because the number of terms in the summations (integrals) for discrete (continuous) variables grows exponentially with increasing number of variables.

•

•

Bayesian Belief Network

Bayesian Network Evaluation Problem

• Some restricted classes of networks, namely the singlyconnected networks where there is no more than onepath between any two nodes, can be efficiently solved intime linear in the number of nodes.

There are also clustering algorithms that convert multiply connected networks to single connected ones.

•

• However, approximate inference methods such as–––

sampling (Monte Carlo) methods

variational methods loopy belief propagation

have to be used for most of the cases.

Bayesian Belief Network

Bayesian Network Learning Problem

• The simplest situation is the one where the networkstructure is completely known (either specified byexpert or designed using causal relationshipsbetween the variables).

an

• Other situations with increasing complexity are:–––

known structure but unobserved variables,

unknown

unknownstructure

structurewith

withobserved variables,

unobserved variables.

Bayesian Belief Network

Pattern Recognition, Fall 2012

Naive Bayesian Network

• When the dependencies among the features areunknown, we generally proceed with the simplestassumption that the features are conditionallyindependent given the class.

• This corresponds to the naive Bayesian networkgives the class-conditional probabilities

Naïve Bayesian network structure. It looks like a

that

• verysimple model but it often works quite well in practice.

Bayesian Belief Network

33

بررسی کاربرد شبکه بیزی در شبکه 1. - خالصه یک مقاله های اجتماعی

مرتبط

تکلیف