markov random fields - inaoeesucar/clases-mgp/notes/c6-mrf.pdf · physical analogy a mrf can be...

Introduction

MarkovNetworks

RegularMarkovRandomFields

GibbsRandomFields

Inference

ParameterEstimation

ApplicationsImage smoothing

Improving imageannotation

References

Markov Random Fields

Probabilistic Graphical Models

L. Enrique Sucar, INAOE

(INAOE) 1 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Outline

1 Introduction

2 Markov Networks

3 Regular Markov Random Fields

4 Gibbs Random Fields

5 Inference

6 Parameter Estimation

7 ApplicationsImage smoothingImproving image annotation

8 References

(INAOE) 2 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

• Certain processes, such as a ferromagnetic materialunder a magnetic field, or an image, can be modeled asa series of states in a chain or a regular grid

• Each state can take different values and is influencedprobabilistically by the states of its neighbors

• These models are known as Markov random fields(MRFs)

(INAOE) 3 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

Ising Model

• In an Ising model, there are a series of randomvariables in a line; each random variable represents adipole that could be in two possible states, up (+) ordown (-)

• The state of each dipole depends on an external fieldand the state of its neighbor dipoles in the line

• A configuration of a MRF is a particular assignment ofvalues to each variable in the model

• A MRF is represented as an undirected graphical model

(INAOE) 4 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

Example

(INAOE) 5 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

Properties and Central Problem

• An important property of a MRF is that the state of avariable is independent of all other variables in themodel given its neighbors in the graph

• The central problem in a MRF is to find theconfiguration of maximum probability

• The probability of a configuration depends on thecombination of an external influence (e.g., a magneticfield in the Ising model) and the internal influence of itsneighbors

(INAOE) 6 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

Physical Analogy

• A MRF can be thought of as a series of rings in poles,where each ring represents a random variable, and theheight of a ring in a pole corresponds to its state

• Each ring is attached to its neighbors with a spring, thiscorresponds to the internal influences; and it is alsoattached to the base of its pole with another spring,representing the external influence

• The relation between the springs’ constants defines therelative weight between the internal and externalinfluences

• If the rings are left loose, they will stabilize to aconfiguration of minimum energy, that corresponds tothe configuration with maximum probability

(INAOE) 7 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Introduction

Physical Analogy

(INAOE) 8 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Random Fields

• A random field (RF) is a collection of S randomvariables, F = F1, . . .Fs, indexed by sites

• Random variables can be discrete or continuous• In a discrete RF, a random variable can take a value fi

from a set of m possible values or labels L = {l1, l2, ...lm}

(INAOE) 9 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Markov Random Field

• A Markov random field or Markov network (MN) is arandom field that satisfies the locality property: avariable Fi is independent of all other variables in thefield given its neighbors:

P(Fi | Fc) = P(Fi | Nei(Fi)) (1)

• Graphically, a Markov network (MN) is an undirectedgraphical model which consists of a set of randomvariables, V, and a set of undirected edges, E

(INAOE) 10 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Independence Relations

• A subset of variables A is independent of the subset ofvariables C given B, if the variables in B separate A andC in the graph

(INAOE) 11 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Joint Probability Distribution

• The joint probability of a MN can be expressed as theproduct of local functions on subsets of variables

• These subsets should include, at least, all the cliques inthe network

• For the example:P(q1,q2,q3,q4,q5) =(1/k)P(q1,q4,q5)P(q1,q2,q5)P(q2,q3,q5)

(INAOE) 12 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Definition

• A Markov network is a set of random variables,X = X1,X2, ...,Xn that are indexed by V , such thatG = (V ,E) is an undirected graph, that satisfies theMarkov property

• A variable Xi is independent of all other variables givenits neighbors, Nei(Xi):

P(Xi | X1, ...Xi−1,Xi+1, ...,Xn) = P(Xi | Nei(Xi)) (2)

• The neighbors of a variable are all the variables that aredirectly connected to it

(INAOE) 13 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Markov Networks

Factorization

• Under certain conditions (if the probability distribution isstrictly positive), the the joint probability distribution of aMRF can be factorized over the cliques of the graph:

P(X) = (1/k)∏

C∈Cliques(G)

φC(XC) (3)

• A MRF can be categorized as regular or irregular.When the random variables are in a lattice it isconsidered regular; for instance, they could representthe pixels in an image; if not, they are irregular.

(INAOE) 14 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Regular Markov Random Fields

Neighboring system

• A neighboring system for a regular MRF F is defined as:

V = {Nei(Fi) | ∀i ∈ Fi} (4)

• V satisfies the following properties:1 A site in the field is not a neighbor to itself.2 The neighborhood relations are symmetric, that is, if

Fj ∈ Nei(Fi) then Fi ∈ Nei(Fj).

(INAOE) 15 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Regular Grid

• For a regular grid, a neighborhood of order i is definedas:

Neii(Fi) = {Fj ∈ F | dist(Fi ,Fj) ≤ r} (5)

• The radius, r , is defined for each order. For example,r = 1 for order one, each interior site has 4 neighbors;r =√

2 for order two, each interior site has 8 neighbors;r = 2 for order three, each interior site has 12 neighbors

(INAOE) 16 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

1st Order Regular Grids

(INAOE) 17 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

2nd Order Regular Grids

(INAOE) 18 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Parameters

• The parameters of a regular MRF are specified by a setof local functions

• These functions correspond to joint probabilitydistributions of subsets of completely connectedvariables in the graph

• In the case of a first order MRF, there are subsets of 2variables; in the case of a second order MRF, there aresubsets of 2, 3 and 4 variables

• In general:P(F) = (1/k)

f (Xi) (6)

• We can think of these local functions as constraints thatwill favor certain configurations

(INAOE) 19 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Gibbs Random Fields

• The joint probability of a MRF can be expressed in amore convenient way given its equivalence with a GibbsRandom Field (GRM), according to theHammersley–Clifford theorem:

P(F) = (1/z)exp(−U) (7)

• U is known as the energy, given its analogy withphysical energy. So maximizing P(F) is equivalent tominimizing U

• The energy function can also be written in terms of localfunctions:

UF =∑

Ui(Xi) (8)

(INAOE) 20 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Gibbs Random Fields

Energy Function

• Considering a regular MRF of order n, the energyfunction can be expressed in terms of functions ofsubsets of completely connected variables of differentsizes, 1,2,3, ...:

UF =∑

U1(Fi) +∑i,j

U2(Fi ,Fj) +∑i,j,k

U3(Fi ,Fj , fk ) + ...

(9)• Given the Gibbs equivalence, the problem of finding the

configuration of maximum probability for a MRF istransformed to finding the configuration of minimumenergy

(INAOE) 21 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Gibbs Random Fields

MRF specification

• In summary, to specify a MRF we must define:• A set of random variables, F, and their possible values,

L.• The dependency structure, or in the case of a regular

MRF a neighborhood scheme.• The potentials for each subset of completely connected

nodes (at least the cliques).

(INAOE) 22 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Inference

Most Probable Configuration

• The most common application of MRFs consists infinding the most probable configuration; that is, thevalue for each variable that maximizes the jointprobability - minimizing the energy function

• The set of all possible configurations of a MRF isusually very large, as it increases exponentially with thenumber of variables in F.

• Thus, it is impossible to calculate the energy (potential)for every configuration, except in the case of very smallfields

(INAOE) 23 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Inference

Stochastic Search

• Usually posed as a stochastic search problem• Starting from an initial, random assignment of each

variable in the MRF, this configuration is improved vialocal operations, until a configuration of minimumenergy is obtained

• After initializing all the variables with a random value,each variable is changed to an alternative value and itsnew energy is estimated

• If the new energy is lower than the previous one, thevalue is changed; otherwise, the value may also changewith a certain probability

(INAOE) 24 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Inference

Stochastic Search Algorithm

FOR i = 1 TO SF (i) = lk (Initialization)

FOR i = 1 TO NFOR j = 1 TO S

t = lk+1 (An alternative value for variable F(i))IF U(t) < U(F (i))

F (i) = t (Change value of F(i) if theenergy is lower)

ELSEIF random(U(t)− U(F (i))) < T

F (i) = t (With certain probabilitychange F(i) if the energy is higher)

(INAOE) 25 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Inference

Variants

• The way in which the optimal configuration is defined,for which there are two main alternatives: MAP andMPM:

• Maximum A posteriori Probability or MAP, the optimumconfiguration is taken as the configuration at the end ofthe iterative process

• Maximum Posterior Marginals or MPM, the mostfrequent value for each variable in all the iterations istaken as the optimum configuration

(INAOE) 26 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Inference

Optimization process

Iterative Conditional Modes (ICM): it alwaysselects the configuration of minimum energy.Metropolis: with a fixed probability, P, it selectsa configuration with a higher energy.Simulated annealing (SA): with a variableprobability, P(T ), it selects a configuration withhigher energy; where T is a parameter knownas temperature. The probability of selecting avalue with higher energy is determined basedon the following expression: P(T ) = e−δU/T

(INAOE) 27 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Parameter Estimation

Definition of a MRF

• The structure of the model –in the case of a regularMRF the neighborhood system.

• The form of the local probability distribution functions–for each complete set in the graph.

• The parameters of the local functions.

(INAOE) 28 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Estimation with labeled data

• We know the structure and functional form, and we onlyneed to estimate the parameters

• The set of parameters, θ, of a MRF, F , are estimatedfrom data, f , assuming no noise

• Given f , the maximum likelihood (ML) estimatormaximizes the probability of the data given theparameters, P(f | θ); thus the optimum parameters are:

θ∗ = ArgMaxθP(f | θ) (10)

(INAOE) 29 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Bayesian approach

• When the prior distribution of the parameters, P(θ), isknown, we can apply a Bayesian approach andmaximize the posterior density obtaining the MAPestimator:

θ∗ = ArgMaxθP(θ | f ) (11)

• Where:P(θ | f ) ∼ P(θ)P(f | θ) (12)

(INAOE) 30 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Approximation• The main difficulty in the ML estimation for a MRF is

that it requires the evaluation of the normalizingpartition function Z

• One possible approximation is based on the conditionalprobabilities of each variable in the field, fi , given itsneighbors, Ni : P(fi | fNi ), and assuming that these areindependent - pseudo-likelihood

• Then the energy function can be written as:

U(f ) =∑

Ui(fi , fNi ) (13)

Assuming a first order regular MRF, only single andpairs of nodes are considered, so:

Ui(fi ,Ni) = V1(fi) +∑

V2(fi , fj) (14)

(INAOE) 31 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Pseudo-Likelihood

• The pseudo-likelihood (PL) is defined as the simpleproduct of the conditional likelihoods:

PL(f ) =∏

P(fi | fNi ) =∏

exp−Ui(fi , fNi )∑fi exp−Ui(fi , fNi )

• Using the PL approximation, and given a particularstructure and form of the local functions, we canestimate the parameters of a MRF model based on data

(INAOE) 32 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Histogram Technique

• Assuming a discrete MRF and given several realizations(examples), the parameters can be estimated usinghistogram techniques

• Assume there are N distinct sets of instances of size kin the dataset, and that a particular configuration (fi , fNi )occurs H times, then an estimate of the probability ofthis configuration is P(fi , fNi ) = H/N

(INAOE) 33 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Applications Image smoothing

Image smoothing

• Digital images are usually corrupted by high frequencynoise

• For reducing the noise a smoothing process can beapplied to the image

• We can define a MRF associated to a digital image, inwhich each pixel corresponds to a random variable

• Considering a first order MRF, each interior variable isconnected to its 4 neighbors

• Additionally, each variable is also connected to anobservation variable that has the value of thecorresponding pixel in the image

(INAOE) 34 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

MRF for image smoothing

(INAOE) 35 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Restrictions

• A property of natural images is that, in general, theyhave certain continuity, that is, neighboring pixels willtend to have similar values

• Restrictions: (i) neighboring pixels to have similarvalues, by punishing (higher energy) configurations inwhich neighbors have different values, (ii) have a valuesimilar to the one in the original image; so we alsopunish configurations in which the variables havedifferent values to their corresponding observations

• The solution will be a compromise between these twotypes of restrictions

(INAOE) 36 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Potential functions

• The energy function can be expressed as the sum oftwo types of potentials: one associated to pairs ofneighbors, Uc(fi , fj); and the other for each variable andits corresponding observation, Uo(fi ,gi)

UF =∑i,j

Uc(Fi ,Fj) + λ∑

Uo(Fi ,Gi) (16)

• Where λ is a parameter which controls which aspect isgiven more importance, the observations (λ > 1) or theneighbors (λ < 1); and Gi is the observation variableassociated to Fi

(INAOE) 37 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Potentials

• A reasonable function is the quadratic difference.• The neighbors potential is:

Uc(fi , fj) = (fi − fj)2 (17)

• The observation potential is:

Uo(fi ,gi) = (fi − gi)2 (18)

• Using these potentials and applying the stochasticoptimization algorithm, a smoothed image is obtainedas the final configuration of F

(INAOE) 38 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Image smoothing

(INAOE) 39 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Applications Improving image annotation

Automatic Image Annotation

• Automatic image annotation is the task of automaticallyassigning annotations or labels to images or segmentsof images, based on their local features

• When labeling a segmented image, we can incorporateadditional information to improve the annotation of eachregion of the image

• The labels of each region of an image are usually notindependent; for instance in an image of animals in thejungle, we will expect to find a sky region above theanimal, and trees or plants below or near the animal

• Spatial relations between the different regions in theimage can help to improve the annotation

(INAOE) 40 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

MRFs for improving annotations

• Using a MRF we can combine the information providedby the visual features for each region (externalpotential) and the information from the spatial relationswith other regions in the image (internal potential)

• By combining both aspects in the potential function, andapplying the optimization process, we can obtain aconfiguration of labels that best describe the image

(INAOE) 41 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Procedure

The procedure is basically the following:1 An image is automatically segmented (using

Normalized cuts).2 The obtained segments are assigned a list of labels and

their corresponding probabilities based on their visualfeatures using a classifier.

3 Concurrently, the spatial relations among the sameregions are computed.

4 The MRF is applied, combining the original labels andthe spatial relations, resulting in a new labeling for theregions by applying simulated annealing.

5 Adjacent regions with the same label are joined.

(INAOE) 42 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Procedure - block diagram

(INAOE) 43 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Energy function

• The energy function to be minimized combines theinformation provided by the classifiers (labels’probabilities) with the spatial relations (relations’probabilities)

• Spatial relations are divided in three groups: topologicalrelations, horizontal relations and vertical relations -contains four terms, one for each type of spatial relationand one for the initial labels:

Up(f ) = α1VT (f ) + α2VH(f ) + α3VV (f ) + λ∑

Vo(f ) (19)

(INAOE) 44 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Parameters

• These potentials can be estimated from a set of labeledtraining images

• The potential for a certain type of spatial relationbetween two regions of classes A and B is inverselyproportional to the probability (frequency) of thatrelation occurring in the training set

(INAOE) 45 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Application

• By applying this approach, a significant improvementcan be obtained over the initial labeling of an image

• In some cases, by using the information provided by thisnew set of labels, we can also improve the initial imagesegmentation as illustrated

(INAOE) 46 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Example - improving segmentation

(INAOE) 47 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Sucar, L. E, Probabilistic Graphical Models, Springer 2015 –Chapter 6

(INAOE) 48 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Additional Reading (1)

Binder, K: Ising Model. Hazewinkel, Michiel,Encyclopedia of Mathematics, Springer-Verlag (2001).

Hammersley, J. M., Clifford, P.: MarkovFields on Finite Graphs and Lattices. Unpublished Paper.http://www.statslab.cam.ac.uk/ grg/books/hammfest/hamm-cliff.pdf (1971). Accessed 14 December2014.

Hernandez-Gracidas, C., Sucar, L.E., Montes, M.:Improving Image Retrieval by Using Spatial Relations.Journal of Multimedia Tools and Applications, vol. 62,pp. 479–505 (2013).

(INAOE) 49 / 50

Introduction

MarkovNetworks

GibbsRandomFields

Inference

ParameterEstimation

References

Additional Reading (2)

Kindermann, R., Snell, J.L.: Markov Random Fields andTheir Applications. American Mathematical Society, vol.1 (1980).

Lafferty, J., McCallum, A., Pereira, F.: ConditionalRandom Fields: Probabilistic Models for Segmentingand Labeling Sequence Data. In: InternationalConference on Machine Learning (2001).

Li, S. Z.: Markov Random Field Modeling in ImageAnalysis. Springer-Verlag, London (2009).

Sutton, C., McCallum, A.: An Introduction to ConditionalRandom Fields for Relational Learning. In: Geetor, L.,Taskar, B. (eds.) Introduction to Statistical RelationalLearning. MIT Press. (2006).

(INAOE) 50 / 50

markov random fields - inaoeesucar/clases-mgp/notes/c6-mrf.pdf · physical analogy a mrf can be...

Documents

markov random fieldsmarkov random fields umamahesh srinivas...

markov random fields - university of...

markov random fields - university of pittsburgh

markov random fields & conditional random fields

introduction to markov random...

smoothing 3d meshes using markov random...

markov random fields and gibbs distributions

markov random fields, markov cocycles and the 3 … ·...

dense photometric stereo a markov random field …

image as markov random field and applications

markov random fields - department of computer science

markov random field (mrf)

markov chain random walk

markov random fields goal

random walks and markov chains

probabilistic models for images markov random fields

markov random fields (mrf)

markov random fields ( mrf)

maximum entropy markov models - conditional random fields

markov random field: a brief introduction