UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g1 o

f 17

IN35

7:W

IEN

ER

FIL

TE

RIN

GC

ou

rse

bo

ok:

Ch

ap.7

.2St

atis

tica

lDig

ital

Sign

alP

roce

ssin

gan

dm

od

elin

g,M

.Hay

es19

96.

Dav

idG

esb

ert

Sign

alan

dIm

age

Pro

cess

ing

Gro

up

(DSB

)h

ttp

://w

ww

.ifi.u

io.n

o/~

gesb

ert

Mar

ch20

03

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g2 o

f 17

Ou

tlin

e•

Pu

rpo

seo

fWie

ner

filt

erin

g

•E

xam

ple

1:te

mp

ora

lWie

ner

filt

erin

g

•O

pti

mu

mlin

ear

filt

erin

g

•G

eom

etri

cali

nte

rpre

tati

on

•Fi

nd

ing

the

solu

tio

n:T

he

Wie

ner

-Ho

pfe

qu

atio

ns

•A

pp

licat

ion

1:Li

nea

rp

red

icti

on

•A

pp

licat

ion

2:N

ois

eca

nce

ling

•A

pp

licat

ion

3:M

ult

i-an

ten

nas

div

ersi

tyco

mb

inin

g

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g3 o

f 17

Pu

rpo

seo

fWie

ner

filt

erin

gG

oal

:“To

esti

mat

ean

un

know

nra

nd

om

pro

cess

(RP

)fr

om

ase

tof

ob

serv

edR

Ps,

wit

hw

hic

hit

isco

rrel

ated

.”

{d(n

)}d

esir

edra

nd

om

pro

cess

(un

ob

serv

ed)

{x0(n

)}o

bse

rved

ran

do

mp

roce

ss{x

2(n

)}o

bse

rved

ran

do

mp

roce

ss. . .

{xp−

1(n

)}o

bse

rved

ran

do

mp

roce

ss

d(n)

d(n)

^

e(n)

x (

n)x

(n)

x (

n)

2 p−10

filte

rW

erro

r si

gnal

desi

red

sign

al

estim

ated

sig

nal

p ob

serv

atio

ns

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g4 o

f 17

Pu

rpo

seo

fWie

ner

filt

erin

g(I

I)Tw

oso

luti

on

sto

fin

dth

eo

per

ato

r(fi

lter

)W

:

•(1

)o

ne

has

atr

ain

ing

sign

alfo

rd

(n)

and

on

ead

just

sW

tom

inim

ize

the

pow

ero

fe(n

).

•(2

)on

eu

ses

know

led

geo

fco

rrel

atio

nb

etw

een{x

i(n

)}an

dd

(n).

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g5 o

f 17

Exa

mp

le1:

Lin

ear

tem

po

ralfi

lter

ing

⇒W

isa

linea

rF

IRfi

lter

.

⇒{x

i(n

)},i

=0,..p−

1ar

eth

esa

mp

les

ofa

wid

ese

nse

stat

ion

ary

(WSS

)ra

nd

om

sign

al.

xi(n

)=x

(n−i),i

=0..,p−

1

W(z

)=wo

+w

1z−

1+w

1z−

2+..

+wp−

1z−p+

1

. . .d̂

(n)

=w

0x

(n)

+w

1x

(n−

1)+w

2x

(n−

2)+...+

wp−

1x

(n−p

+1)

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g6 o

f 17

Vect

or

Form

ula

tio

n

W=

[w0,w

1,..,wp−

1]T

X(n

)=

[x0(n

),x

1(n

),..,x

p−1(n

)]T

d̂(n

)=W

TX

(n)

wh

ereT

isth

etr

ansp

ose

op

erat

or.

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g7 o

f 17

Op

tim

um

lin

ear

filt

erin

gP

rob

lem

:On

ew

ish

esto

fin

dth

elin

ear

filt

erW

that

min

imiz

esth

eer

ror

bet

wee

nd

(n)

andd̂

(n).

e(n

)=d

(n)−d̂

(n)

J(W

)=E|e

(n)|2

wh

ereE

()is

the

exp

ecta

tio

n.

Fin

dW

osu

chth

atJ

(Wo)

ism

inim

um

.Wo

isth

eo

pti

mu

mli

nea

rfi

lter

inth

eW

ien

erse

nse

.

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g8 o

f 17

Geo

met

rica

lIn

terp

reta

tio

nVe

cto

rsp

ace

rep

rese

nta

tio

n

•W

SSra

nd

om

pro

cess

es({e(n

)},{d

(n)},{x

0(n

)},..,{x

p−1(n

)}ca

nb

evi

ewed

asel

emen

tsin

ave

cto

rsp

ace.

•T

he

scal

arp

rod

uct

bet

wee

ntw

oR

Ps{u

(n)}

and{v

(n)}

isd

efin

edb

yth

eco

rrel

atio

n:<

u,v>

=E

(u(n

)v(n

)∗).

•O

rth

ogo

nal

ity

=d

eco

rrel

atio

n

p=2

x (

n)0x (

n)1d(

n)

e(n)

d(n)

^

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g9 o

f 17

Fin

din

gth

eso

luti

on

Th

eso

luti

onW

ois

give

nb

yth

eW

ien

er-H

op

feq

uat

ion

s.

RxW

o=r dx

wh

ere

(1)

Rx

=E

(X(n

)∗X

(n)T

)(2

)r dx

=E

(d(n

)X(n

)∗)

(3)

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g10

of 1

7

Der

ivat

ion

oft

he

Wie

ner

-Ho

pfe

qu

atio

nD

irec

tmet

ho

du

sin

go

pti

miz

atio

nth

eory

:Th

eex

trem

ao

fth

eco

stfu

nct

ionJ

(W)

are

fou

nd

fro

m(s

eeb

oo

kb

yH

ayes

p.49

):

∂J

∂w∗ 0|W

=Wo

∂J

∂w∗ 1|W

=Wo

. . .∂J

∂w∗ p−

1|W

=Wo

=

0 0 . . . 0

(4

)

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g11

of 1

7

Der

ivat

ion

usi

ng

geo

met

ry{d̂

(n)}

isth

eo

pti

mal

esti

mat

eif

f

•{d̂

(n)}

isth

ep

roje

ctio

no

f{d

(n)}

on

toth

esp

ace

span

ned

by

{x0(n

)},..,{x

p−1(n

)},o

req

uiv

alen

tly:

•{e

(n)}

iso

rth

ogo

nal

toth

esp

ace

span

ned

by{x

0(n

)},..,{x

p−1(n

)}.

<e(n

),x

0(n

)>

=E

(e(n

)x0(n

)∗)

=0

<e(n

),x

1(n

)>

=E

(e(n

)x1(n

)∗)

=0

. . .<e(n

),xp−

1(n

)>

=E

(e(n

)xp−

1(n

)∗)

=0

(5)

p=2

x (

n)0x (

n)1d(

n)

e(n)

d(n)

^

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g12

of 1

7

Exp

ress

ion

for

the

min

imu

mer

ror

Th

em

inim

um

erro

r,o

bta

ined

wit

hW

o,i

sJ

(Wo)

:

J(W

o)

=E|d

(n)|2

+W

H oRxW

o−W

H or dx−rH dxW

o

J(W

o)

=E|d

(n)|2−rH dxW

oW

o

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g13

of 1

7

Ap

pli

cati

on

1:L

inea

rp

red

icti

on

Hay

es’b

oo

k,p.

342

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g14

of 1

7

Ap

pli

cati

on

2:N

ois

eca

nce

lin

gH

ayes

’bo

ok,

p.34

9

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g15

of 1

7

Ap

pli

cati

on

3:M

ult

i-an

ten

na

div

ersi

tyco

mb

inin

g

Weig

htsph

ases

estim

ation

*s

y

1 2

h h1 2w w

=h =h

1 2*

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g16

of 1

7

An

ten

na

div

ersi

tyco

mb

inin

g(I

I)R

ecei

ved

sign

alm

od

el:

X(n

)=Hs(n

)+V

(n)

(6)

wh

ereX

(N)

colle

cts

the

sign

als

rece

ived

onp

ante

nn

as.s

(n)

isth

etr

ansm

itte

dsy

mb

ols

equ

ence

(e.g

.s(n

)=

+/−

1).H

isth

ech

ann

elve

cto

ro

fsiz

ep.V

(n)

isth

ew

hit

en

ois

eve

cto

r.

Pro

ble

m:F

indW

osu

chth

at

E|W

T oX

(n)−s(n

)|2m

inim

um

UN

IVE

RS

ITY

OF

OS

LO

DE

PA

RT

ME

NT

OF

INFO

RM

AT

ICS

D. G

esbe

rt: I

N35

7 S

tati

stic

al S

ign

al P

roce

ssin

g17

of 1

7

An

ten

na

div

ersi

tyco

mb

inin

g(I

II)

An

swer

(Wie

ner

-Ho

pf)

:

RxW

o=r dx

(H∗ H

T+σ

2 vI

)Wo

=H∗

Wo

=(H∗ H

T+σ

2 vI

)−1H∗

wh

ereσ

2 vis

the

vari

ance

ofn

ois

esa

mp

les.

NO

TE

:Th

isre

ceiv

eris

equ

ival

ent

toth

em

inim

um

mea

nsq

uar

eer

ror

(MM

SE)

rece

iver

.