bayesian vs frequentist - indico · srivastava, rahul (ific, valencia) taken from xkcd. different...
TRANSCRIPT
BayesianvsFrequentist
Xia,Ziqing (PurpleMountainObservatory)Duan,Kaikai (PurpleMontainObservatory)Centelles Chuliá,Salvador(Ific,valencia)
Srivastava,Rahul(Ific,Valencia)
Taken from xkcd
DifferentStatisticalQuestions
TheFrequentist likelihoodandtheBayesianposteriorasktwodifferentstatisticalquestionsofthedata:
Regionsofhighqualityoffit
Giventhepriorandthedata,extractthepdfoftheparameters.
Thenotionofprobability- Frequentist
• If thenumberoftrialsapproachesinfinity, therelativefrequencywillconverge exactly tothetrueprobability.
• Repeatabilityofanexperimentisthekeyconcept.
The number of trials where the event X occurred
The total number of trials
• TheMaximumLikelihoodEstimator:
Thenotionofprobability- Bayesian
Posterior
Marginal likelihood
PriorLikelihood function
It is a direct consequence of the Bayes theorem :
• Bayestheoremrelatestheposteriorprobability(whatweknowabouttheparameterafterseeingthedata)tothelikelihood (derivedfroma statisticalmodel fortheobserveddata) andtheprior (whatweknewabouttheparameterbeforewesawthedata).
• A generalruletoupdateourknowledgefromtheprior tothe posterior.
A SimpleProblem:PhotonCounts• Imaginethatweobservethelightcomingfromasinglestar.We assumethatthestar’struefluxisconstantwithtime.
• Giventhe measurementsanderrors,whatisthebestestimateofthetrueflux?
• we usePythontogeneratesometoydata ( Poissondistribution)
Frequentist ApproachtoSimplePhotonCounts• Theprobabilitydistributionofthemeasurement:
• constructthe likelihoodfunctionbycomputingtheproductoftheprobabilitiesforeachdatapoint:
• Thebestestimate:
BayesianApproachtoSimplePhotonCounts• Theposteriorprobability:
• Themodelprior: a standardchoiceistotakeauniformprior.
• TheBayesianprobabilityismaximizedatpreciselythesamevalueasthefrequentist result! InthecaseofaGaussian
likelihoodanduniformprior,theposteriorpdfandtheprofilelikelihoodareidenticalandthusthe
questionofwhichtochoosedoesnotarise.
BayesianApproachtoSimplePhotonCounts
• Theposteriorprobability:
TheBayesianBilliardGame
• AliceandBobcan’tseethebilliardtable.• Carolrollsaballdownthetable,andmarkswhereitlands.Oncethismarkisinplace,Carolbeginsrollingnewballsdownthetable.
• Iftheballlandstotheleftofthemark,Alicegetsapoint;ifitlandstotherightofthemark,Bobgetsapoint.
• Thefirstpersontoreach sixpointswinsthegame.• NowsaythatAliceisleadingwith5pointsandBobhas3points.WhatcanbesaidaboutthechancesofBobtowinthegame?
Frequentist (naive) ApproachtoTheBilliardGame• FiveballsoutofeightfellonAlice'ssideofthemarker• Maximumlikelihoodestimateof pthatanygivenrolllandsinAlice'sfavor.
• Assumingthismaximumlikelihoodprobability,wecancomputetheprobabilitythatBobwillwin,whichisgivenby:
Frequentist ProbabilityofBobWinning:0.05
BayesianApproachtoTheBilliardGame
• Theposteriorprobability:
PosteriorProbabilityofBobWinning:0.09?!
UniformPrior
MonteCarloApproachtoTheBilliardGame
• UseaMonteCarlosimulationtodeterminethecorrectanswer.
The correct ProbabilityofBobWinning:0.09!Bayesianwin!!!
Conclusions
• The Bayesian vs Frequentist debate has a long history and the battleis still going on
• People oftenmake statements claiming one to be better than other• In our opinion, it is not accurate to say that one is inherently superiorto other
• Both approaches can give poor results if not done carefully• However, frompractical point of view, Bayesian seems to bettersuited to handle “dirty” data
• Situation similar to classical numerical integration techniques vsMonte Carlo integration techniques
Open questions
• How to knowwhen to use one approach or the other?• What defines a good prior?• Is there another approach (the third one) can resolve this case?
References:[1] RobertoTrotta: BayesianMethodsinCosmology. arXiv: 1701.01467[2]JakeVanderPlas: Frequentism andBayesianism:APython-drivenPrimer.
arXiv: 1411.5018
Thankyou!