review: hidden markov models efficient dynamic programming algorithms exist for –finding pr(s)...

Review: Hidden Markov Models

• Efficient dynamic programming algorithms exist for– Finding Pr(S)– The highest probability path P that maximizes

Pr(S,P) (Viterbi)• Training the model

– State seq known: MLE + smoothing– Otherwise: Baum-Welch algorithm

S2

S4

S1

0.9

0.5

0.50.8

0.2

0.1

S3

A

C

0.6

0.4

A

C

0.3

0.7A

C

0.5

0.5

A

C

0.9

0.1

HMM for Segmentation

• Simplest Model: One state per entity type

HMM Learning

• Manally pick HMM’s graph (eg simple model, fully connected)

• Learn transition probabilities: Pr(si|sj)

• Learn emission probabilities: Pr(w|si)

Learning model parameters• When training data defines unique path through HMM

– Transition probabilities• Probability of transitioning from state i to state j =

number of transitions from i to j total transitions from state i

– Emission probabilities• Probability of emitting symbol k from state i =

number of times k generated from i number of transition from I

What is a “symbol” ???

Cohen => “Cohen”, “cohen”, “Xxxxx”, “Xx”, … ?

4601 => “4601”, “9999”, “9+”, “number”, … ?

000.. . . .999

3 -d ig i ts

00000 .. . .99999

5 -d ig i ts

0 ..99 0000 ..9999 000000 ..

O th e rs

N u m b e rs

A .. ..z

C h a rs

a a ..

M u lt i -le tte r

W o rds

. , / - + ? #

D e lim ite rs

A ll

Datamold: choose best abstraction level using holdout set

What is a symbol?

Ideally we would like to use many, arbitrary, overlapping features of words.

St -1

St

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

We can extend the HMM model so that each state generates multiple “features” – but they should be independent.

Borthwick et al solution

We could use YFCL: an SVM, logistic regression, a decision tree, …. We’ll be talking about logistic regression.

St -1

St

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

Instead of an HMM, classify each token. Don’t learn transition probabilities, instead constrain them at test time.

Stupid HMM tricks

startPr(red)

Pr(green) Pr(green|green) = 1

Pr(red|red) = 1

Stupid HMM tricks

startPr(red)

Pr(green)Pr(green|green) = 1

Pr(red|red) = 1

Pr(y|x) = Pr(x|y) * Pr(y) / Pr(x)

argmax{y} Pr(y|x) = argmax{y} Pr(x|y) * Pr(y)= argmax{y} Pr(y) * Pr(x1|y)*Pr(x2|y)*...*Pr(xm|y)

Pr(“I voted for Ralph Nader”|ggggg) = Pr(g)*Pr(I|g)*Pr(voted|g)*Pr(for|g)*Pr(Ralph|g)*Pr(Nader|g)

From NB to Maxent

Zy

yw

docf

yjkydocf

k

i

kj

/)Pr(

)|Pr(

ncombinatiok j,th -i )(

0]:?1 class of doc of position at appears [word ),(,

xjw

ywyZ

xy

k

jk

in wordis where

)|Pr()Pr(1

)|Pr( i

yxfi ),(0

From NB to Maxent

xjw

ywyZ

xy

k

jk

in wordis where

)|Pr()Pr(1

)|Pr( i

yxfi ),(0

From NB to Maxent

xjw

ywyZ

xy

k

jk

in wordis where

)|Pr()Pr(1

)|Pr( i

yxfi ),(0

i

i yxfxyP ),()|(log 0 Or:

Idea: keep the same functional form as naïve Bayes, but pick the parameters to optimize performance on training data.

One possible definition of performance is conditional log likelihood of the data:

t

tt xyP )|(log

MaxEnt Comments– Implementation:

• All methods are iterative• For NLP like problems with many features, modern gradient-like or

Newton-like methods work well• Thursday I’ll derive the gradient for CRFs

– Smoothing: • Typically maxent will overfit data if there are many infrequent features. • Old-school solutions: discard low-count features; early stopping with

holdout set; …• Modern solutions: penalize large parameter values with a prior centered

on zero to limit size of alphas (ie, optimize log likelihood - sum alpha); other regularization techniques

What is a symbol?

Ideally we would like to use many, arbitrary, overlapping features of words.

St -1

St

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

Borthwick et al idea

St -1

St

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

Idea: replace generative model in HMM with a maxent model, where state depends on observations

...)|Pr( tt xs

Another idea….

St -1

St

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

Idea: replace generative model in HMM with a maxent model, where state depends on observations and previous state

...),|Pr( ,1 ttt sxs

MaxEnt taggers and MEMMs

St -1 S

t

Ot

St+1

Ot +1

Ot -1

identity of wordends in “-ski”is capitalizedis part of a noun phraseis in a list of city namesis under node X in WordNetis in bold fontis indentedis in hyperlink anchor…

…

…part of

noun phrase

is “Wisniewski”

ends in “-ski”

Idea: replace generative model in HMM with a maxent model, where state depends on observations and previous state history

......),|Pr( ,2,1 tttt ssxs

Learning does not change – you’ve just added a few additional features that are the previous labels.

Classification is trickier – we don’t know the previous-label features at test time – so we will need to search for the best sequence of labels (like for an HMM).

Partial history of the idea

• Sliding window classifiers– Sejnowski’s NETTalk, mid 1980’s

• Recurrent neural networks and other “recurrent” sliding-window classifiers– Late 1980’s and 1990’s

• Ratnaparkhi’s thesis– Mid-late 1990’s

• Frietag, McCallum & Pereira ICML 2000– Formalize notion of MEMM

• OpenNLP– Based largely on MaxEnt taggers, Apache Open Source

Ratnaparkhi’s MXPOST

• Sequential learning problem: predict POS tags of words.

• Uses MaxEnt model described above.

• Rich feature set.• To smooth, discard features

occurring < 10 times.

MXPOST

MXPOST: learning & inference

GISFeature

selection

23

Using the HMM to segment

• Find highest probability path through the HMM.• Viterbi: quadratic dynamic programming

algorithm

House

ot

Road

City

Pin

15213 Butler Highway Greenville 21578

House

Road

City

Pinot

House

Road

City

Pin

15213 Butler ... 21578

Inference for MENE (Borthwick et al system)

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

Goal: best legal path through lattice (i.e., path that runs through the most black ink. Like Viterbi but cost of possible transitions are ignored.)

Inference for MXPOST

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

),|Pr(

)...,,|Pr(

)...,,|Pr()|Pr(

1

1,

1,1

iii

iikii

iii

yxy

yyxy

yyxyxy

(Approx view): find best path, weights are now on arcs from state to state.

window of k tagsk=1

Inference for MXPOST

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

More accurately: find total flow to each node, weights are now on arcs from state to state.

'

11 )',|Pr()'()(y

tttt yYxyYyy

Inference for MXPOST

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

),|Pr(

)...,,|Pr(

)...,,|Pr()|Pr(

1,2

1,

1,1

iiii

iikii

iii

yyxy

yyxy

yyxyxy

Find best path? tree? Weights are on hyperedges

Inference for MxPOST

I

O

iI

iO

When will prof Cohen post the notes …

oI

oO

iI

iO

oI

oO

iI

iO

oI

oO

iI

iO

oI

oO

iI

iO

oI

oO

iI

iO

oI

oO

… …

Beam search is alternative to Viterbi: at each stage, find all children, score them, and discard all but the top n states

MXPost results

• State of art accuracy (for 1996)

• Same approach used successfully for several other sequential classification steps of a stochastic parser (also state of art).

• Same (or similar) approaches used for NER by Borthwick, Malouf, Manning, and others.

MEMMs

• Basic difference from ME tagging:– ME tagging: previous state is feature of MaxEnt

classifier– MEMM: build a separate MaxEnt classifier for each

state.• Can build any HMM architecture you want; eg parallel nested

HMM’s, etc.• Data is fragmented: examples where previous tag is “proper

noun” give no information about learning tags when previous tag is “noun”

– Mostly a difference in viewpoint– MEMM does allow possibility of “hidden” states and

Baum-Welsh like training

MEMM task: FAQ parsing

MEMM features

MEMMs

Looking forward

• HMMS– Easy to train generative model– Features for a state must be independent (-)

• MaxEnt tagger/MEMM– Multiple cascaded classifiers– Features can be arbitrary (+)– Have we given anything up?

37

HMM inference

House

ot

Road

City

Pin

• Total probability of transitions out of a state must sum to 1

• But …they can all lead to “unlikely” states

• So…. a state can be a (probable) “dead end” in the lattice

House

Road

City

Pinot

House

Road

City

Pin

15213 Butler ... 21578

Inference for MXPOST

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

More accurately: find total flow to

each node, weights are now on arcs

from state to state.

'

11 )',|Pr()'()(y

tttt yYxyYyy

Flow out of each node is always fixed:

y

tt yYxyYy 1)',|Pr(,' 1

Label Bias Problem (Lafferty, McCallum, Pereira ICML 2001)

• Consider this MEMM, and enough training data to perfectly model it:

Pr(0123|rob) = Pr(1|0,r)/Z1 * Pr(2|1,o)/Z2 * Pr(3|2,b)/Z3= 0.5 * 1 * 1

Pr(0453|rib) = Pr(4|0,r)/Z1’ * Pr(5|4,i)/Z2’ * Pr(3|5,b)/Z3’= 0.5 * 1 *1

Pr(0123|rib)=1Pr(0453|rob)=1

Another max-flow scheme

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

More accurately: find total flow to

each node, weights are now on arcs

from state to state.

'

11 )',|Pr()'()(y

tttt yYxyYyy

Flow out of a node is always fixed:

y

tt yYxyYy 1)',|Pr(,' 1

Another max-flow scheme: MRFs

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

Goal is to learn how to weight edges in the graph:• weight(yi,yi+1) = 2*[(yi=B or I) and isCap(xi)] + 1*[(yi=B and

isFirstName(xi)] - 5*[(yi+1≠B and isLower(xi) and isUpper(xi+1)]

Another max-flow scheme: MRFs

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

B

I

O

When will prof Cohen post the notes …

Find total flow to each node, weights are now on edges from state to state. Goal is to learn how to weight edges

in the graph, given features from the examples.

Another view of label bias [Sha & Pereira]

So what’s the alternative?