an introduction to bayesian statistics
DESCRIPTION
test presentationTRANSCRIPT
An Introduction to
Bayesian Statistics
Paul HerendeenApril 2013
1960 1970 1980 1990 2000 20100
2000
4000
6000
8000WOS "Bayesian"Citations by Year
The rise of Bayesian statistics
So…What are Bayesian Statistics?
So…What are Bayesian Statistics?
1) A fundamentally different approach to probability
So…What are Bayesian Statistics?
1) A fundamentally different approach to probability
2) An associated set of mathematical tools
Frequentists vs. BayesianRound 1
Parameters fixed
Data varies
Data fixed
Parameters Vary
Frequentists vs. BayesianRound 1
Probability
Likelihood
Frequentists vs. BayesianRound 1
Confidence Interval
Credible Interval
Conditional Probability in 2 minutes
Conditional Probability in 2 minutes
All possible outcomes
Conditional Probability in 2 minutes
red blue
𝑃 (𝑅 ,𝐵 )=?
Conditional Probability in 2 minutes
red blue
𝑃 (𝑅 ,𝐵 )=¿𝑃 (𝑅 ) 𝑃 (𝐵 )
Conditional Probability in 2 minutes
red blue
𝑃 (𝐵|𝑅 )=¿𝑃 (𝐵 ,𝑅 )𝑃 (𝑅)
Conditional Probability in 2 minutes
red blue
𝑃 (𝑅|𝐵 )= 𝑃 (𝐵|𝑅 )𝑃 (𝑅 )𝑃 (𝐵)
Conditional Probability in 2 minutes
red blue
𝑃 (𝑅|𝐵 )= 𝑃 (𝐵|𝑅 )𝑃 (𝑅 )𝑃 (𝐵)
Bayes’ Theorem
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 ) 𝑃 (𝜃 )
∫𝑃 (𝐷|𝜃 ) 𝑃 (𝜃)
Bayes’ Theorem
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 ) 𝑃 (𝜃 )𝑃 (𝐷)
Prior
Bayes’ Theorem
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 ) 𝑃 (𝜃 )𝑃 (𝐷)
Likelihood
Prior
Bayes’ Theorem
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 ) 𝑃 (𝜃 )𝑃 (𝐷)
Likelihood
Prior
Evidence
Bayes’ Theorem
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 ) 𝑃 (𝜃 )𝑃 (𝐷)
Likelihood
Prior
Posterior
Evidence
Frequentist vs. BayesianRound 2
“The Strength of the Prior”
𝑃 (𝜃|𝐷 )= 𝑃 (𝜃 )𝑃 (𝐷|𝜃 )𝑃 (𝐷)
Sparse Data
𝑃 (𝜃|𝐷 )= 𝑃 (𝜃 )𝑃 (𝐷|𝜃 )𝑃 (𝐷)
Abundant Data
𝑃 (𝜃|𝐷 )= 𝑃 (𝜃 )𝑃 (𝐷|𝜃 )𝑃 (𝐷)
Uniform Prior
Where do Priors Come From?
So…What are Bayesian Statistics?
1) A fundamentally different approach to probability
2) An associated set of mathematical tools
𝑃 (𝜃|𝐷 )= 𝑃 (𝐷|𝜃 )𝑃 (𝜃 )
∫𝑃 (𝐷|𝜃 ) 𝑃 (𝐷)
How do you actually do this?
So how do you actually do this?
1.Analytical methods
So how do you actually do this?
1.Analytical methods
2.Grid approximation
So how do you actually do this?
1.Analytical methods
2.Grid approximation
3.Markov Chain Monte Carlo
MCMC• Algorithm for exploring parameter
space
MCMC• Algorithm for exploring parameter
space1.Pick a starting point
MCMC• Algorithm for exploring parameter
space1.Pick a starting point2.Propose a move
MCMC• Algorithm for exploring parameter
space1.Pick a starting point2.Propose a move3.Accept or decline move based on
probability
MCMC• Algorithm for exploring parameter
space1.Pick a starting point2.Propose a move3.Accept or decline move based on
probability• Time spent at each point
approximates parameter distribution
MCMC• Algorithm for exploring parameter space
1.Pick a starting point2.Propose a move3.Accept or decline move based on
probability• Time spent at each point approximates
parameter distribution• E.g. Metropolis-Hastings, Gibbs
sampling
MCMC2D example
MCMC2D example
So what does all this get us?
Bayesian methods really shine in complex (hierarchical) models…
For example,
IndividualFecundity
Group Effect
Population Effect
Foraging success
Environment
or…
Individual Fecundity
Group Effect
Population Effect
Environment
Many benefits to this approach
• Simultaneously estimate parameters• …as well as parameter relationships• “Borrow” strength across studies• Model comparison
So, is it a Bayesian Revolution?
Bayesian stats can do most things
frequentist,
Bayesian stats can do most things
frequentist, but…• Many simple models don’t gain
much• Better do something ‘boring’
well than something exciting poorly
Bayesian stats can do most things
frequentist, but…• Many simple models don’t gain
much• Better do something ‘boring’
well than something exciting poorly
• Don’t be this guy
DO use Bayesian methods if
• You have a complex model with many interacting parameters
• You have ‘messy’ data• You don’t want to make
assumptions about distributions
In Conclusion• Bayesian methods are powerful
tools for ecological research • Like most things statistical, they
are no substitute for thinking• They are here to stay, and you
should at least be familiar with them
Great, I want to learn more!
JAGS(Just Another Gibbs Sampler)