Temporal model

Time and uncertainty

In the static world, random variables had fixed value. There are many cases , when environment

changes in time .Consider a diabetic patient .evidence variables are insulin doses, food intake,

blood sugar measurement, which change over time , and the past record is used to make decision

about the next meal and insulin dose. Time related tasks are dynamic problem, like sequence of

spoken words, medical diagnosis, etc.

The world changes; we need to track and predict it

Probabilistic reasoning for dynamic world.

Repairing car-diagnosis Vs treating diabetic patient

States and Observations

Process of change is viewed as series of snapshots, each describing the state of the world

at a particular time

Each time slice involves a set or random variables indexed by t:

1. the set of unobservable state variables Xt

2. the set of observable evidence variable Et

The observation at time t is Et = et for some set of values et

The notation Xa:b denotes the set of variables from Xa to Xb

Markov processes (Markov chains)

Example: The umbrella world

You are security guard at some secret underground installation. You want to know

whether it is raining today, but your access to the outside world occurs each morning when

you see agent coming in with, or without umbrella.

For each day t, the set Et contain a single evidence variable (Umbrella), i.e. Et={ Ut }

For each day t, the set Xt contain a single state variable (Umbrella), i.e. Xt= { Rt }

The interval between times slices also depends on the problem-in general we will

assume a fixed, finite time interval (day, hour...)

Notation j:k will be used to denote the sequence of integers from time j to time k

(inclusive)

We will assume that the state sequence starts at t=0.

In the umbrella world the sequence of sate variable will be R0, R1, and R2...

We will assume that the evidence sequence starts at t=1.

In the umbrella world the sequence of evidence variables will be U1, U2, U3,

Stationary process and the Markov Assumption

Stationary process: Is a process of change that is governed by laws that do not themselves

change over time

Markov Process: Is that the current state depend on only a finite history of previous states.

First-order Markov Process: In which the current state depends only on the previous state and

not on any earlier sates.

P(Xt|X0:t-1)=P(Xt|Xt-1)

Here P(Xt|Xt-1) is the conditional distribution

Second-order Markov Process: The evidence variables at time t depend only on the current

state

P(Et|X0:t,E0:t-1)=P(Et|Xt)

The conditional distribution P(Et|Xt) is called the sensor model (for sometimes observation

model). Because it describes how the “sensors”-that is, the evidence variables ,are affected by

the actual state of the world.

Example for first-order Markov process and second-order Markov process is as follows

Example (2): Bayesian Network structure and conditional distribution describing the umbrella

world. The transition model is P(Raint/Rain t-1) and the sensor model is P(umbrella t/Rain t)

What is P(X0)?

Given:

Transition model: P(Xt|Xt-1)

Sensor model: P(Et|Xt)

Prior probability: P(X0)

Then we can specify complete joint distribution:

At time t, the joint is completely determined:

P(X0,X1,…Xt,E1,…,Et) =P(X0) • ∏i t P(Xi|Xi-1)P(Ei|Xi)

In the umbrella world, the rain causes the umbrella to appear.

In this example, the probability of rain is assumed to depend only on whether it rained the

previous day.

The first-order Markov assumption says that the state variables contain all the

information needed to characterize the probability distribution for the next slice.

Inferencing in Temporal model

Filtering: P (Xt|e1: t). This is the belief state – input to the decision process of a rational

agent. Also, as a artifact of the calculation scheme, we get the probability

needed for speech recognition if we are interested.

computing current belief state, given all evidence to date

What is the probability that it is raining today, given all the umbrella observations

up through today?

Prediction: P(Xt+k/e1:t) for k > 0. Evaluation of possible action sequences; like filtering without

the evidence

computing probability of some future state

What is the probability that it will rain the day after tomorrow, given all

the umbrella observations up through today?

Smoothing: P(Xk|e1:t) for 0 ≤ k < t. Better estimate of paststates – Essential for learning

computing probability of past state (hindsight)

What is the probability that it rained yesterday, given all the umbrella

observations through today?

Most likely explanation: arg maxx1:t P (x1:t|e1:t). Speech recognition, decoding with a noisy

channel

arg maxx1,..xtP(x1,…,xt|e1,…,et)

given sequence of observation, find sequence of states that is most likely

to have generated those observations.

Filtering

Let us begin with filtering,We will show that this can be done in a simple , online fashion give the result of filtering upto time t, one can easily compute the result for t+1 from the new evidence et+1,that is

P(Xt+1/e1.t+1)=f(et+1,P(xt/e1:t)

for some function f.This process is called recursive estimation.

On day 2, the umberlla appears so U2=true, The predicition from t=1,t=2 is

Prediction

Smoothing

Smoothing is the process of computing the distribution over past states given evidence up to present.

The forward message f1:k can be computed by filtering forward 1 to k.

Most likely Sequence

Possible state sequence for Rain can be viewed as paths through a graph of the possible

sates at each time step.

Operation of the viterbi algorithm for the umbrella observation sequence [true, true, false,

false, and true].

The task of finding the most likely path through this graph, where the likelihood of any

path is the product of the transition probabilities of the given observation at each time.

The most likely path to the state Rain5 =true consists of the most likely path to some state

at time 4 followed by the transition to Rain5=true; and the state at time 4 that will become

part of the path to Rain5=true whichever maximizes the likelihood of the path

There is a recursive relationship between most likely path to each state X t+1 and most

likely paths to each state Xt.

The summation over Xt is replaced by the maximization over xt