doing bayesian data analysis, chapter 5
DESCRIPTION
03/AUG/2013 @Matsuo lab, the University of Tokyo.TRANSCRIPT
www.***.com
Part2: Chapter 5 Inferring a Binominal Proportion via Exact Mathematical Analysis�
Haru Negami
03/08/2013�
summary�
! binomial proportion " the likelihood function
! the Bernoulli likelihood function " the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+z) " comparison ---- best model <-p(D|M)
Binomial Proportion�
! dlcW�Gp�W[W ! binomial or dichotomous ! ��
" 50��m 8nKTW.#X&,�6
ahbbC�
p(θ|q) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
Likelihood function�
! Example : coin flipping " � y = {0,1} ex) (0:1, 1:2)
" p(y=1|θ)=f(θ)=θA;mθX�8[0,1]S7/n " p(y=0|θ) = 1-θ
" the Bernoulli distribution " p(y|θ) = θy(1-θ)(1-y)
! �MBθX��CyX��SD^C ! *'��SD^C (ΣyAp(y|θ) = 1)�
Likelihood function�
! Bernoulli likelihood function " p(y|θ) = θy(1-θ)(1-y)
" y`��Bθ`��T3^C ! ��9�XθG�O^KTV7/W�`T^C ! gkela��XyKTV:�)U�`�^C
" ��9�X*'��SXUEJTV$�O^C ! p(y|θ) = θy(1-θ)(1-y)(y=i)`θS+�C
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done�
belief (to make a model)�
! ��o " p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ) " p(θ)Tp(y|θ)p(θ)G�N��S!I^C
! �N�`(ER50��`�\OJTG�"^C
! ��p " denominator Σyp(y|θ) G4-SH^C
" J_]`%POp(θ)` a conjugate prior for p(y|θ) TEFC
belief (to make a model)�
! p(θ)A= θa(1-θ)b Xgkela��9�WYTQW���*'��9�
>
p(y|θ)×p(θ)A= θy(1-θ)(1-y) × θa(1-θ)b
= θy+a(1-θ)(1-y+b)
! JW�S1L_^*'��9�`beta distributionT�ZC�
belief (to make a model)�
! beta distribution " a, bW2QWfjilcGD^ (a,b > 0) " p(θ|a,b) = beta(θ|a,b) = θ(a-1)(1-θ)(b-1)/B(a,b) ! B(a,b)Xbeta functionT�ZC
" beta distribution W normalizer ! B(a,b) = ∫01dθAθ(a-1)(1-θ)(b-1)
belief (to make a model)�
! Beta Distribution�
b�
a�
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done�
belief in detail (prior)�
! beta distribution : beta(θ|a,b)
" two parameters ! mean :
! standard deviation :
belief in detail (prior)�
! beta distribution : beta(θ|a,b) " guess the values of a and b
! from (observed) data " ex) a=b=1, a=b=4, etc…
! m = a/(a+b), n=(a+b)TO^T a = mn, b = (1-m)n
! from mean and standard deviation
<a@1 & b@1 G�EGB a<1 &/or b<1W� [
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done� done�
belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)
! posterior distribution : beta(θ|z+a, N-z+b)
belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)
! posterior distribution : beta(θ|z+a, N-z+b)
one of the beauties of mathematical approach to
Bayesian inference!�
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
0� z/N� a/(a+b)�
1-α� α�
α = N/(N+a+b)TO^C�
Discussion(?) Three inferential goals�
! from chapter 4 " estimating the binominal proportion " predicting Data " comparing models
estimation�
! uncertainty of the prior distribution " From the posterior distribution
! HDI : the highest density interval (chapter 3)
HDI L : broad R : narrow
prior dist. L : more uncertain�
estimation�
! reasonable credibility of a value concerned " From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5 credible?
ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible
estimation�
! reasonable credibility of a value concerned " From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5 credible?
ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible
includes many extra assumptions!�
prediction�
! p(y) = ∫dθp(y|θ)p(θ) <-posterior�
prediction�
! p(y) = ∫dθp(y|θ)p(θ) <-posterior�
0� z/N� a/(a+b)�
1-α� α�
α = N/(N+a+b)TO^C�
prediction (example 1)�
! 1st beta(θ|1,1) mean 1/2 (= p(y)) observation : head (N=1,z=1)
! 2nd beta(θ|2,1) mean 2/3 (= p(y)) observation : head (N=1,z=1)
! 3rd beta(θ|3,1) mean 3/4 (= p(y))
prediction (example 2)�
! 1st beta(θ|100,100) : 1/2 (= p(y)) observation : head (N=1,z=1)
observation : head (N=1,z=1)
! 3rd beta(θ|102,100) : 102/202(?50%)
comparison�
to compare the models,
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
comparison�
! Calculation of evidence " p(D|M) = p(z,N)�
comparison�
uniform strongly peaked� uniform strongly peaked�
N = 14, z = 11� N = 14, z = 7�
p(D|M)=0.000183>p(D|M)=6.86×10-5� p(D|M)=1.94×10-5<p(D|M)=5.9×10-5�
comparison�
! both are important " the prior distribution " the likelihood function
" in detail, see chapter 4
The best model (so far) is not a good model.
summary�
! binomial proportion " the likelihood function
! the Bernoulli likelihood function " the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+b) " comparison ---- best model <-p(D|M)