advanced signal processing 2, se 1 patrick gampp graz, 04/29/08 hmm - basics

Advanced Signal Processing 2, SE

1

Patrick Gampp Graz, 04/29/08 HMM - Basics

HMM - Basics


2


Content

• Hidden Markov Model (HMM)• The Three Basic Problems for HMMs

– Problem 1 Solution: Forward/ Backward Algorithm– Problem 2 Solution: Viterbi Algorithm– Problem 3 Solution: Baum- Welch Algorithm

• An Overview: HMM in Speech Synthesis System

Content

HMM

Three Basic

Problems

Speech System

Overview


3


HMM

URN 2

URN 1 URN 3

P(red) = 0.8

P(green) = 0.1

P(blue) = 0.1

P(red) = 0.2

P(green) = 0.2

P(blue) = 0.6

P(red) = 0.5

P(green) = 0.4

P(blue) = 0.1

Content

HMM

Three Basic

Problems

Speech System

Overview


4


Elements of an HMM

• N, number of states S = {S1,S2,S3, … , SN}• M, number of observation symbols

V = {v1,v2,v3, … , vM}• State transition probability distribution: A = {aij}• Observation symbol probability distribution in state j:

B = bj(k)• Initial state distribution: π = {πi}• T, number of observations in the sequence

O = O1 O2 O3… OT

HMM completely characterized by:

λ = (A, B, π)

Content

HMM

Three Basic

Problems

Speech System

Overview


5


Why HMM?

• No one-to-one mapping: speech – word symbol

• Different symbols – same sound

• Large variation in speech– Speaker variability– Mood– Environment

• No explicit symbol boundary detection

Speech waveform is NOT a concatenation of static patterns

Content

HMM

Three Basic

Problems

Speech System

Overview


6


The Three Basic Problems: Problem 1

Content

HMM

Three Basic

Problems

Speech System

Overview

Solution: Forward - Algorithm


7


Forward - Algorithm

Forward variable:

1) Initialization:

2) Induction:

3) Termination:

Content

HMM

Three Basic

Problems

Speech System

Overview


8



Content

HMM

Three Basic

Problems

Speech System

Overview

Solution: Viterbi - Algorithm


9


Viterbi- Algorithm (1)

• Highest probability along a single path:

1) Initialization

2) Recursion

Content

HMM

Three Basic

Problems

Speech System

Overview


10


Viterbi- Algorithm (2)

3) Termination

4) Path Backtracking

Content

HMM

Three Basic

Problems

Speech System

Overview


11



Content

HMM

Three Basic

Problems

Speech System

Overview

Solution: Baum – Welch Algorithm

(finds local maximum only)


12


Baum – Welch - Algorithm(1)

• Define:

• Forward/backward variable: Content

HMM

Three Basic

Problems

Speech System

Overview


13


Baum- Welch- Algorithm(2)

• Define:

• Relation:

Content

HMM

Three Basic

Problems

Speech System

Overview


14


Baum- Welch- Algorithm(3)

• Reestimation formulas (use iteratively to local maximum!)

• Baum‘s auxiliary function:

Derive reestimation formulas directly

Content

HMM

Three Basic

Problems

Speech System

Overview


15


HMM - Based Speech Synthesis System

Content

HMM

Three Basic

Problems

Speech System

Overview


16


References

[1] „A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition“. Lawrence R. Rabiner (1989)

[2] „An HMM-Based Speech Synthesis System Applied to English“.

Keiichi Tokuda et al.

[3] Talk About HMM-Based Speech Synthesis. Keiichi Tokuda (2006)

[4] HTK Book. Cambridge University Engineering Department (2006)


17



18


Markov- Chain(1)

• Transition probability:

• Markov- property:

• Initial state probability:

Content

Markov-Chain

HMM

Three Basic

Problems

Speech System

Overview


19


Markov- Chain: An Example

Content

Markov-Chain

HMM

Three Basic

Problems

Speech System

Overview


20


The Backward Variable

Backward variable:

1) Initialization:

2) Induction:

Content

HMM

Three Basic

Problems

Speech System

Overview

advanced signal processing 2, se 1 patrick gampp graz, 04/29/08 hmm - basics

Documents