mcmc software options - biostatisticsbrad/7440/slide_hatfield_mcmc_software.pdfmcmc software options...

MCMC Software Options

I BUGS and BUGS-esque: WinBUGS, OpenBUGS, and JAGS

I R Calling Functions for BUGS: BRugs, R2WinBUGS, rbugs,rjags, R2jags, and runjags

I R-based: mcmc

I Python-based: PyMC

All of these can produce so-called ”coda” files (i.e., raw MCMCchain output) that may be processed with coda package, amongothers. Saving model output in this format is the most portable.

Toy Model to Illustrate Differences

We will fit the following toy model as an example in each of thesoftware packages described.

yi ∼ N(β0 + β1xi , σ2y )

β0 ∼ N(µ1, σ21)

β1 ∼ N(µ2, σ22)

where the hyperparameters (σ2y , σ21, σ

22, µ1, µ2)′ are fixed at

(2, 10, 10, 5, 1)′, i = 1, . . . , 100, and the xi are randomly generatedfrom N(0, 1).

Generating the Data

For purposes of the R code snippets below, assume we have runthe following:

N <- 100

set.seed(221983)

x <- rnorm(N,0,1)

beta <- c(4,2)

y <- rnorm(N,beta[1]+beta[2]*x,sqrt(2))

bprec <- c(1/10,1/10)

mu <- c(5,1)

prec <- 1/2

WinBUGS

Dave Lunn & Nicky Best, Imperial College Londonhttp://www.mrc-bsu.cam.ac.uk/bugs/winbugs/

I Stand-alone GUI software, written in Component Pascal

I Runs natively in Windows and in Linux/Mac via WINE

I Includes spatial models (GeoBUGS)

I “Point-and-click” interface, but can call from R

WinBUGS model code

model{

for (i in 1:N){

y[i] ~ dnorm(meen[i],prec)

meen[i] <- beta[1] + beta[2]*x[i]

}

beta[1] ~ dnorm(mu[1],bprec[1])

beta[2] ~ dnorm(mu[2],bprec[2])

}

# Data

list(list(y=c(-4.79537E-01, 4.91775E+00, ...),

x=c(-1.21613E+00, 6.44501E-01, 3.61216E-01, ...),

mu=c(5,1),prec=.5,bprec=c(.1,.1),N=100)

# Inits

list(beta=c(0,0))

list(beta=c(1,1))

WinBUGS

OpenBUGS

Andrew Thomas, University of Helsinkihttp://www.openbugs.info/w/

I Open-source successor to WinBUGS, still Component Pascal

I Runs in Windows, Mac (via WINE), and Linux (no GUI)

I Minor differences vs WinBUGS model specification

I No spatial models

OpenBUGS model code

model{

for (i in 1:N){

y[i] ~ dnorm(meen[i],prec)

meen[i] <- beta[1] + beta[2]*x[i]

}

beta[1] ~ dnorm(mu[1],bprec[1])

beta[2] ~ dnorm(mu[2],bprec[2])

}

(Note: identical to WinBUGS for this model.)

OpenBUGS

JAGS

Martyn Plummer, International Agency for Research on Cancerhttp://calvin.iarc.fr/~martyn/software/jags/

I Open-source C++ software with BUGS-type modelspecification

I Runs natively on many OSs (Win, Mac, various Linux flavors)

I Distinguishes between censoring and truncation (unlikeWinBUGS)

JAGS

(Model code is the same; data and inits format slightly different.)

y <- c(3.06582, 1.26428, 0.54347, ...)

x <- c(-0.03570, -1.27062, -0.99170, ...)

mu <- c(5, 1)

prec <- 0.5

bprec <- c(0.1, 0.1)

N <- 100

JAGS

model in "JAGS_model.txt"

data in "JAGS_data.txt"

compile, nchains(2)

parameters in "JAGS_inits.txt"

initialize

monitor beta

update 10000

coda *

JAGS

rjags

library(rjags)

mymodel <- jags.model(file="JAGS_model.txt",

data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),

inits=list(list(beta=c(0,0)),list(beta=c(1,1))),n.chains=2)

mysamples <- jags.samples(mymodel,

variable.names=’beta’,n.iter=10000)

summary(mysamples$beta,quantile,

prob=c(.025,.5,.975))

rjags

R2WinBUGS

Andrew Gelman, Columbiahttp://www.stat.columbia.edu/~gelman/bugsR/

I Calls WinBUGS or OpenBUGS from within R

I (Using OpenBUGS option requires BRugs)

I Includes ability to run WinBUGS via WINE

R2WinBUGS

fit <- bugs(

data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),

inits=list(list(beta=c(0,0)),list(beta=c(1,1))),

parameters.to.save=’beta’,model.file="WB_model.txt",

n.chains=2,n.iter=10000,n.burnin=0,program="WinBUGS")

plot(fit)

R2WinBUGS

WBDev

Dave Lunn, Cambridgehhttp://www.winbugs-development.org.uk/

I Allows users to extend WinBUGS using new functions anddistributions

I Works within BlackBox Component Builder 1.5 (Windows)http://www.oberon.ch/blackbox.html

WBDev

MODULE WBDevMyNorm;

...

PROCEDURE LogFullLikelihood

(node: WBDevUnivariate.Node; OUT value: REAL);

VAR

x, meen, prec: REAL;

BEGIN

x := node.value;

meen := node.arguments[location][0].Value();

prec := node.arguments[inverseScale][0].Value();

value := 0.5 * (Math.Ln(prec) -

prec * Math.IntPower(x - meen, 2) - log2Pi);

value := value - Math.Ln(1 -

WBDevSpecfunc.Phi(-Math.Sqrt(prec) * meen));

END LogFullLikelihood;

...

WBDev

model{

for (i in 1:N){

y[i] ~ MyNorm(meen[i],prec)

meen[i] <- beta[1] + beta[2]*x[i]

}

beta[1] ~ dnorm(mu[1],bprec[1])

beta[2] ~ dnorm(mu[2],bprec[2])

}

WBDev

BRugs

http://www.openbugs.info/w/UserContributedCode

http://www.biostat.umn.edu/~brad/software/BRugs/

I (Silently) calls OpenBUGS from within R

I No longer distributed on CRAN, bugg on Windows 7 (?)

I Includes handy data formatting function bugs.data

BRugs

myfit <- BRugsFit(

modelFile="WB_model.txt",

data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),

inits=list(list(beta=c(0,0)),list(beta=c(1,1))),

nIter=10000,nBurnin=0,

numChains=2,parametersToSave=’beta’)

myfit$Stats

samplesHistory(’beta’)

samplesDensity(’beta’)

BRugs

rbugs

Jun Yan & Marco Prates, UConn

I Calls either Win/OpenBUGS from R

I Possibly better for Mac/Linux using WINE

rbugs

library(rbugs)

rbugsfit <- rbugs(

data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),

inits=list(list(beta=c(0,0)),list(beta=c(1,1))),

paramSet=’beta’, model=’OB_model.txt’,

n.chains=2,n.iter=10000,n.burnin=0,

bugs="C:/Program Files (x86)/OpenBUGS/OpenBUGS321/OpenBUGS.exe",

OpenBugs=T,bugsWorkingDir=getwd())

rbugs

mcmc

Charlie Geyer, University of Minnesotahttp://www.stat.umn.edu/geyer/mcmc/

I Simple random-walk Metropolis algorithm

I User supplies a function that evaluates the log unnormalizedtarget density

I A few control parameters

mcmc

library(mcmc)

myll <- function(params,data){

meen <- params[1] + params[2]*data$x

ll <- sum(dnorm(data$y,meen,data$sigy,log=T))+

sum(dnorm(params,data$mu,data$sigb,log=T))

return(ll)

}

metropfit <- metrop(myll,initial=c(0,0),

nbatch=10000,blen=1,scale=.2,

data=list(x=x,y=y,mu=mu,sigy=sqrt(1/prec),

sigb=sqrt(1/bprec)))

metropfit$accept

mcmc

PyMC

Anand Patil (Univ Oxford), David Huard (McGill), and ChrisFonnesbeck (Vanderbilt)http://code.google.com/p/pymc/

I Wrapper Python code that calls updating routines written inC and Fortran

I Open-source, extensible, fast