estimation: parts of chapters 10-12 - uic...
TRANSCRIPT
![Page 1: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/1.jpg)
Estimation: parts of Chapters 10-12
Bayesian Estimation
Natasha Devroye
http://www.ece.uic.edu/~devroye
Spring 2010
Summary
Bayesian philosophy
![Page 2: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/2.jpg)
Bayesian risk
Bayesian risk
![Page 3: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/3.jpg)
Quadratic error cost
Absolute error cost
![Page 4: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/4.jpg)
Hit-or-miss cost (MAP)
MMSE estimators
![Page 5: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/5.jpg)
MMSE estimators
Example of MMSE
![Page 6: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/6.jpg)
Bayesian estimation on Gaussian priors/noise
This is crucial to know!
Bayesian estimation on Gaussian priors/noise
This is crucial to know!
![Page 7: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/7.jpg)
Bayesian linear model
Bayesian linear model
![Page 8: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/8.jpg)
Examples: MMSE in Gaussian noise
Examples: MMSE in Gaussian noise
![Page 9: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/9.jpg)
Examples: MMSE in Gaussian noise
Properties of MMSE
![Page 10: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/10.jpg)
Maximum a Posteriori (MAP) estimation
Example
![Page 11: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/11.jpg)
MAP properties
Linear MMSE (LMMSE)
![Page 12: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/12.jpg)
LMMSE definition
Geometric LMMSE
![Page 13: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/13.jpg)
Geometric LMMSE
LMMSE example
![Page 14: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/14.jpg)
LMMSE properties
LMMSE properties
![Page 15: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/15.jpg)
Sequential LMMSE
Algebraic approach Geometric approach
Sequential LMMSE: algebraic
![Page 16: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/16.jpg)
Sequential LMMSE: geometric approach
3
!"#$%&"'()**+,"-.$%
/)0“Hit-or-Miss”
Cost Function
! "! "1!1!2/
1!!
|ˆ :Cov. Err.
ˆ :Estimate
!
!
#
#
//34“Squared” Cost Function
(Nonlinear Estimate)
56.$+(
2,%6(
78'-6&,'%
9//347,+-$ Linear Estimate
:',;': E{$},E{1}, CJointly Gaussian 1 and $
(Yields Linear Estimate)
! " ! "% &
1!11!1!!!
11!1
2222/
1122!!
1ˆ
1
:Cov. Err.
ˆ :Estimate
'
'
'#
'(# !! ! " ! "% &
1!11!1!!!
11!1
2222/
1122!!
1ˆ
1
:Cov. Err.
ˆ :
'
'
'#
'(# !!Estimate3"<$=
Bayesian Linear Model
(Yields Linear Estimate)
! " % & % &
% & !!!!!
!!!
>22>>2>22/
>?12>>2>2!!
1
ˆ
1
:Cov. Err.
ˆ :Estimate
'
'
('#
'((#
"
##
"
##!
![Page 17: Estimation: parts of Chapters 10-12 - UIC Engineeringdevroye/courses/ECE531/lectures/b1c10.pdfBayesian estimation on Gaussian priors/noise This is crucial to know! Bayesian estimation](https://reader033.vdocuments.us/reader033/viewer/2022041703/5e42e7e3cb750e1d5f037206/html5/thumbnails/17.jpg)
See http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522_files/EECE%20522%20Notes_28%20Ch_12B.pdf