The Monte Carlo methodNathan Baker(baker@biochem.wustl.edu)BME 540

What is Monte Carlo?A method to evaluate integralsPopular in all areas of science, economics, sociology, etc.Involves:Random sampling of domainEvaluation of functionAcceptance/rejection of points

Why Monte Carlo?The domain of integration is very complex:Find the integral of a complicated solvent-solute energy function outside the boundary of a protein.Find the volume of an object from a tomographic data set.The integrals are very high-dimensional:A partition function or free energy for a proteinAnalyzing metabolic networksPredicting trends in the stock market

Basic Monte Carlo AlgorithmSuppose we want to approximate

in a high-dimensional spaceFor i = 1 to nPick a point xi at randomAccept or reject the point based on criterionIf accepted, then add f(xi) to total sumError estimates are free by calculating sums of squaresError typically decays as

Example: the area of a circleSample points randomly from square surrounding circle of radius 510,000 sample pointsAcceptance criterion: inside circleActual area: 78.54Calculated area: 78.66

Example: more complicated shapes

Example: multiple dimensionsWhat is the average of a variable for a N-dimensional probability distribution?Two approaches:QuadratureDiscretize each dimension into a set of n pointsPossibly use adaptivity to guide discretizationFor a reasonably smooth function, error decreases as n-N/2Monte CarloSample m points from the spacePossibly weight sampling based on reference functionError decreases as m-1/2

Problems: sampling tails of distributionsWe want toIntegrate a sharply-peaked functionUse Monte Carlo with uniformly-distributed random numbersWhat happens?Very few points contribute to the integral (~9%)Poor computational efficiency/convergenceCan we ignore the tails?NO!Solution: use a different distribution

Improved sampling: change of variablesOne way to improve sampling is to change variables:New distribution is flatterUniform variates more usefulAdvantages:SimplicityVery useful for generating distributions of non-uniform variates (coming up)DisadvantagesMost useful for invertible functions

Change of variables: methodGiven an integral

Transform variables

Choose variables to give (nearly) flat distribution

Integrate

Change of variables application: exponentialGiven an integral

Transform variables

Choose variables to give (nearly) flat distribution

Integrate

Change of variables application: exponentialBefore transformation:0.5% of points in domain contribute to integralSlow convergence

Change of variables example: exponentialBefore transformation:0.5% of points in domain contribute to integralSlow convergenceAfter transformation:All of the points in domain contributeRapid (exact) convergenceIn practice:Speed-up best when inversion is nearly exact

Importance samplingFunctions arent always invertibleIs there another way to improve sampling of important regions of the functions?Find flat distributionsBias samplingFind a function that is almost proportional to the one of interest:

Rewrite your integral as a weighted integral:

Importance sampling example: a lumpy Gaussian Our original integrand is

This is close to

Therefore:Sample random numbers from (well talk about how to do this later):

Evaluate the following integrand over the random number distribution:

Importance sampling example: a lumpy Gaussian Convergence is pretty good (actual value 1.41377)

Evolution of Monte Carlo methods so farUniform points and original integrandbut this had very poor efficiency.Uniform points and transformed integrandbut this only worked for certain integrands.Non-uniform points and scaled integrandbut this is very cumbersome for complicated integrandsNow, we try Markov chain approaches

Markov chainsPropertiesA sequence of randomly-chosen statesThe probability of transitions between states is independent of historyThe entire chain represents a stationary probability distributionExamplesRandom number generatorsBrownian motionHidden Markov Models(Perfectly) encrypted data

Detailed balanceWhat sorts of Markov chains reach stationary distributions?This is the same question as posed for the Liouville equationEquilibrium processes have stationary distributionsWhat does it mean to be at equilibrium?ReversibilityDetailed balance relates stationary probabilities to transitions:

Detailed balance: exampleMarkov chain with a Gaussian stationary probability distributionDetailed balance satisfied

Detailed balance: exampleMarkov chain with a Gaussian stationary probability distribution, EXCEPT:Steps to the right are 1% more favorableDetailed balance violatedWhen both states are unfavorable, a 1% bias makes a big difference!

Markov chain Monte CarloAssembling the entire distribution for MC is usually hard:Complicated energy landscapesHigh-dimensional systemsExtraordinarily difficult normalizationSolution:Build up distribution from Markov chainChoose local transition probabilities which generate distribution of interest (i.e., ensure detailed balance)Each random variable is chosen based on the previous variable in the chainWalk along the Markov chain until convergence reachedResult: Normalization not required, calculations are local

Application: microcanonical ensembleConsider the particle in a N-D box.Monte Carlo algorithm for microcanonical ensemble:Sample Determine the number of states with energy ERelate to the microcanonical partition function by differentiation

Application: microcanonical ensemble9-dimensional particle in a box

Markov chain Monte Carlo: flavorsMolecular simulation: MetropolisBayesian inferences: GibbsHidden Markov Models: Viterbietc.

Application: stochastic transitionsThis is a very simplistic version of kinetic Monte CarloEach Monte Carlo step corresponds to a time intervalThe probability of moving between states in that time interval is related to the rate constantsSimulate to give mean first passage times, transient populations, etc.

Metropolis Monte CarloStart with the detailed balance condition

Derive an acceptance ratio condition

Choose a particular acceptance ratio

Application: canonical ensembleOur un-normalized probabilities look like Boltzmann factors:

Our acceptance ratio is therefore:

Algorithm: NVT Metropolis Monte CarloMetropolis MC in a harmonic potential

Advantages of Metropolis MC simulationsDoes not require forcesRapidly-changing energy functionsNo differentiation requiredAmenable to complex move sets:TorsionsRotamersTautomersEtc.

Monte Carlo machineryBoundary conditionsFinitePeriodicInteractionsCompleteTruncatedPeriodicHow do we choose moves?

Monte Carlo movesTrial movesRigid body translationRigid body rotationInternal conformational changes (soft vs. stiff modes)Titration/electronic statesQuestions:How big a move should we take?Move one particle or many?

Monte Carlo movesHow big a move should we take?Smaller moves: better acceptance rate, slower samplingBigger moves: faster sampling, poorer acceptance rateAmortize mean squared displacement with respect to CPU timeRules of thumb: percent acceptance rateMove one particle or many?Possible to achieve more efficient sampling with correct multi-particle movesOne-particle moves must choose particles at random

Random variatesDont develop your own uniform variate methodLook for long period, lack of bias, etc.These are usually fulfilled in random(), drand48(), etc.I like Mersenne twisterUse uniform variates for starting point of other distributionsCan use all the methods weve described:Standard sampling (rejection method)TransformationsMetropolis (watch out for correlations)

Random variates: transformationsUseful for functions with easy inversesSample uniform variable and transform via inverse to give particular variatesTougher distributions require rejection with respect to easy distribution

Problem: poor convergenceError in Monte Carlo decreases as N-1/2What if you want to improve error by 1%?This is due to the filling behavior of uniformly-distributed random pointsAffects nearly all areas of MC simulationsConvergence can also be difficult to detect:Observe decrease in variance of dataEmploy error estimatesSub-sample data

Quasi-random sequencesCompare to the pseudo-random methods weve discussed so far.Can we find a sub-random sequence with low internal correlations that fills space better?Solution: a maximally-avoiding set of pointsNumber-theoretic methods:Base transformationsPrimesPolynomialsConvergence can reach N-1 (Sobol sequence)Caveats:How many points (resolution)?What about sharply-varying functions?

Hammersley quasi-random sequence

Application: the bootstrap methodUseful for estimating error fit parametersResampling of original dataExamination of resulting fit parametersAlgorithm:Generate fit parameters for the original dataFor (number of resamples desired)Resample the data with replacementGenerate new fit parametersSave deviation with respect to original fit parametersAnalyze deviation statistics

SummaryA method for samplingIntegralsConfiguration spaceError spaceAny distribution that can be represented by transitions can be sampledSampling can be accelerated using various tricks