learning bit by bit hidden markov models. weighted fsa weather the is outside 1.0.7.3
Post on 20-Dec-2015
212 views
TRANSCRIPT
![Page 1: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/1.jpg)
Learning Bit by Bit
Hidden Markov Models
![Page 2: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/2.jpg)
Weighted FSA
weatherweatherTheThe isis
outsideoutside
1.0
.7
.3
![Page 3: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/3.jpg)
Markov Chain
• Computing probability of an observed sequence of events
![Page 4: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/4.jpg)
Markov Chain
weatherweather
TheThe
isis
outsideoutside
.7
.3
Observation = “The weather outside”
windwind
.5
.5
.1
.9
![Page 5: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/5.jpg)
Parts of Speech
• Grammatical constructs like noun, verb
![Page 6: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/6.jpg)
POS examples• N noun chair, bandwidth, pacing• V verb study, debate, munch• ADJ adjective purple, tall, ridiculous• ADV adverb unfortunately, slowly• P preposition of, by, to• PRO pronoun I, me, mine• DET determiner the, a, that, those
![Page 7: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/7.jpg)
Parts of Speech-uses
• Speech recognition• Speech synthesis• Data mining• Translation
![Page 8: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/8.jpg)
POS Tagging
• Words often have more than one POS: back– The back door = JJ– On my back = NN– Win the voters back = RB– Promised to back the bill = VB
• The POS tagging problem is to determine the POS tag for a particular instance of a word.
![Page 9: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/9.jpg)
POS Tagging
• Sentence = sequence of observations• Ie. “Secretariat is expected to race tomorrow”
![Page 10: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/10.jpg)
Disambiguating “race”
![Page 11: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/11.jpg)
Hidden Markov Model
• Observed• Hidden
![Page 12: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/12.jpg)
Hidden Markov Model
• 2 kinds of probabilities:– Tag transitions – Word likelihoods
![Page 13: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/13.jpg)
Hidden Markov Model
• Tag transition prob = P( tag | previous tag)– ie. P(VB | TO)
![Page 14: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/14.jpg)
Hidden Markov Model
• Word likelihood probability = P(word | tag)– ie. P(“race” | VB)
![Page 15: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/15.jpg)
• Actual probabilities:– P (NN | TO) = .00047– P (VB | TO) = .83
![Page 16: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/16.jpg)
• Actual probabilities:– P (NR| VB) = .0027– P (NR| NN) = .0012
![Page 17: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/17.jpg)
• Actual probabilities:– P (race | NN) = .00057– P (race | VB) = .00012
![Page 18: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/18.jpg)
![Page 19: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/19.jpg)
Hidden Markov Model
• Probability “to race tomorrow” =“TO VB NR”• P(VB|TO) * P(NR|VB) * P(race|VB)• .83 * .0027 * .00012 = 0.00000026892
![Page 20: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/20.jpg)
Hidden Markov Model
• Probability “to race tomorrow” =“TO NN NR”• P(NN|TO) * P(NR|NN) * P(race|NN)• .00047* .0012* .00057 = 0.00000000032148
![Page 21: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/21.jpg)
Hidden Markov Model
• Probability “to race tomorrow” =“TO NN NR” = 0.00000000032148• Probability “to race tomorrow” =“TO VB NR”
= 0.00000026892
![Page 22: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/22.jpg)
Bayesian Inference
• Correct answer = max (P (hypothesis | observed))
![Page 23: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/23.jpg)
Bayesian Inference
• Prior probability = likelihood of the hypothesis
![Page 24: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/24.jpg)
Bayesian Inference
• Likelihood = probability that the evidence matches the hypothesis
![Page 25: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/25.jpg)
Bayesian Inference
• Bayesian vs. Frequentists• Subjectivity
![Page 26: Learning Bit by Bit Hidden Markov Models. Weighted FSA weather The is outside 1.0.7.3](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d425503460f94a1d36a/html5/thumbnails/26.jpg)
Examples