lecture 6: hidden variables and expectation-maximization

24

Machine Learning for Language Technology Lecture 6: Hidden Variables and Expecta6on Maximiza6on (EM) Marina San6ni Department of Linguis6cs and Philology Uppsala University, Uppsala, Sweden Autumn 2014 Acknowledgement: Thanks to Prof. Joakim Nivre for course design and materials

Upload: marina-santini

Post on 05-Dec-2014

132 views

Category:

Education

1 download

Report

Download

Embed Size (px):

DESCRIPTION

Maximum Likelihood Estimation, Hidden and Latent Variables, Expectation-Maximization, EM for Naive Bayes

TRANSCRIPT

Page 1: Lecture 6: Hidden Variables and Expectation-Maximization

Machine Learning for Language Technology Lecture 6: Hidden Variables and Expecta6on Maximiza6on (EM)

Marina San6ni Department of Linguis6cs and Philology Uppsala University, Uppsala, Sweden

Autumn 2014

Acknowledgement: Thanks to Prof. Joakim Nivre for course design and materials

Page 2: Lecture 6: Hidden Variables and Expectation-Maximization

Repe66on: Baysian Approach to Classifica6on

Page 3: Lecture 6: Hidden Variables and Expectation-Maximization

MLE

Page 4: Lecture 6: Hidden Variables and Expectation-Maximization

Hidden and Latent Variables

Page 5: Lecture 6: Hidden Variables and Expectation-Maximization

Expecta6on-‐Maximaza6on (1)

Page 6: Lecture 6: Hidden Variables and Expectation-Maximization

Expecta6on-‐Maximaza6on (2)

Page 7: Lecture 6: Hidden Variables and Expectation-Maximization

Expecta6on-‐Maximiza6on (3)

Page 8: Lecture 6: Hidden Variables and Expectation-Maximization

Expecta6on-‐Maximiza6on (4)

Page 9: Lecture 6: Hidden Variables and Expectation-Maximization

Naive Bayes Revised

•  Equa6ons are ok

Page 10: Lecture 6: Hidden Variables and Expectation-Maximization

EM for Naive Bayes

Page 11: Lecture 6: Hidden Variables and Expectation-Maximization

A Simple Example

Page 12: Lecture 6: Hidden Variables and Expectation-Maximization

Supervised Learning

Page 13: Lecture 6: Hidden Variables and Expectation-Maximization

Unsupervised Learning

Page 14: Lecture 6: Hidden Variables and Expectation-Maximization

First Guess

Page 15: Lecture 6: Hidden Variables and Expectation-Maximization

First E-‐Step

Page 16: Lecture 6: Hidden Variables and Expectation-Maximization

First M-‐Step

Page 17: Lecture 6: Hidden Variables and Expectation-Maximization

Second E-‐Step

Page 18: Lecture 6: Hidden Variables and Expectation-Maximization

Second M-‐Step

Page 19: Lecture 6: Hidden Variables and Expectation-Maximization

Third E-‐Step

Page 20: Lecture 6: Hidden Variables and Expectation-Maximization

Third M-‐Step

Page 21: Lecture 6: Hidden Variables and Expectation-Maximization

Fourth E-‐Step

Page 22: Lecture 6: Hidden Variables and Expectation-Maximization

Fourth M-‐Step -‐ Convergence

Page 23: Lecture 6: Hidden Variables and Expectation-Maximization

Uses of EM

Page 24: Lecture 6: Hidden Variables and Expectation-Maximization

The end

The Expectation-maximization Algorithm

CSC 2515 Lecture 9: Expectation-Maximization

LECTURE 10: EXPECTATION MAXIMIZATION (EM)

Gaussian Mixture Models and Expectation Maximization

Gaussian Mixture Models and Expectation Maximization

Expectation & Maximization

Expectation-Maximization (EM) Algorithm

GENERAL CONVERGENT EXPECTATION MAXIMIZATION … · Expectation maximization (EM), Richardson-Lucy, simultaneous alge- braic reconstruction technique (SART), image reconstruction,

Expectation Maximization

K-Means, Expectation Maximization and Segmentationluthuli.cs.uiuc.edu/~daf/courses/cs543computervision/week 6/emseg.pdf · ... Expectation Maximization and Segmentation ... Color

Expectation Maximization (Intuition) Expectation ... Maximization (Intuition) Expectation Maximization (Maths) 1 Stefanos Zafeiriou Adv. Statistical Machine Learning (course 495) •

EM Demystified: An Expectation-Maximization Tutorial

Clustering and The Expectation-Maximization Algorithm ...mpetrik/teaching/intro_ml_17_files/class10.pdf · Clustering and The Expectation-Maximization Algorithm Unsupervised Learning

Expectation-Maximization · Expectation Maximization {Convenientalgorithm forcertain Maximum Likelihood problems. { Viablealternativeto Newton or Conjugate Gradient algorithms. {

Maximum Likelihood Estimation & Expectation Maximization

Gaussian Mixture Models and Expectation-Maximization Algorithm

The Expectation Maximization (EM) Algorithm

CS B 553: A LGORITHMS FOR O PTIMIZATION AND L EARNING Parameter Learning with Hidden Variables & Expectation Maximization

Unified Expectation Maximization

Wavelet based multiresolution expectation maximization ...raheja/publications/wmrem_cmig.pdf · Wavelet based multiresolution expectation maximization image reconstruction algorithm

Expectation Maximization and Gaussian Mixture Models

MODIFIED EXPECTATION MAXIMIZATION ALGORITHM FOR ...morse.uml.edu/~vkrishnan/index/homepage/pdpfiles/My Papers/ Patiwat.pdf · MODIFIED EXPECTATION MAXIMIZATION ALGORITHM FOR TRANSMISSION

Expectation Maximization (EM) Mixture of Gaussians

Lecture 18 Expectation Maximization

Expectation Maximization Machine Learning. Last Time Expectation Maximization Gaussian Mixture Models

LECTURE 11: EXPECTATION MAXIMIZATION (EM)

Expectation Maximization Algorithm

The expectation - maximization algorithm - IEEE …home.ustc.edu.cn/~xiaosong/ppt/EM_tutorial.pdfTitle The expectation - maximization algorithm - IEEE Signal Processing Magazi ne Author

Expectation Maximization Method

Expectation Maximization A “Gentle” Introduction

Progressive Expectation–Maximization for Hierarchical Volumetric Photon Mappingwjarosz/publications/jakob11... · 2013. 7. 7. · Progressive Expectation–Maximization for Hierarchical

Expectation-Maximization for Learning Determinantal Point ...jgillenw.com/nips2014.pdf · Expectation-Maximization for Learning Determinantal Point Processes Jennifer Gillenwater

Expectation-Maximization Algorithm for Clustering ... · Expectation Maximization Tutorial by Avi Kak Expectation-Maximization Algorithm for Clustering Multidimensional Numerical

07 Machine Learning - Expectation Maximization

Expectation Maximization - Dr Gunnar BLOHMcompneurosci.com/wiki/images/e/ef/EM_algorithm.pdf · coin. Solution: Expectation-maximization . Coin Flip With hidden variables •What