How about that Bayes: Bayesian techniques and the simple pendulum

Matthew Heffernan1, ∗

1Department of Physics, McGill University, 3600 University Street, Montreal, QC, H3A 2T8, Canada

Physics increasingly relies on Bayesian techniques for rigorous data analysis and model-to-data comparison. This paper describes how these methods can be implemented to answerquestions of relevance to teaching laboratories. It demonstrates the Bayesian approach tostatistical modeling and model selection in a rigorous step-by-step workflow. The exampleof the simple pendulum allows for a demonstration with the precision commonly seen instudent measurements in the introductory laboratory. This can be used to provide realis-tic quantitative guidance for model preference between the small angle approximation andmore complicated formula. This extends the current simple pendulum literature’s focus be-yond comparing individual idealized assessments of different approximations and places suchdiscussions on a data-driven foundation while providing actionable guidance for teachinglaboratory design.

I. INTRODUCTION

Introductory physics courses commonly teachthat the period of a simple pendulum displaced bya small angle may be found by applying the approx-imation sin θ ≈ θ, which applies when θ is small.However, the period of a pendulum also depends onthe initial angular displacement. Attempts to in-crease accuracy in approximations of the pendulumperiod have so far focused on more closely approxi-mating the full non-analytic integral expression forthe period so students in undergraduate laboratoriescan investigate the dependence of the period on ini-tial angular displacement [1]. Recently, model com-parison and complexity has been applied to the pen-dulum problem or the teaching laboratory [2].

Guidance for the use of the small angle ap-proximation in simple pendula has not previouslybeen rigorously quantified using realistic uncertain-ties measured in teaching laboratories. Often, theonly suggestion is that after ∼ 15◦, the differencebetween sin θ and θ in radians exceeds 1% and thesmall angle approximation should no longer be used[3]. This does not take into account the varietyof measurement uncertainties detailed in introduc-tory physics laboratory materials or if more elabo-rate formulae are demanded by the data. This studydemonstrates that students with both an exact andapproximate analytical formula for the period of apendulum could successfully establish a quantitativepreference between the two at moderate displace-ments using Bayesian tools. This can also be ex-tended to the case of choosing between various more

∗ heffernan@physics.mcgill.ca

complex approximations. This study remedies a gapin the literature by quantitatively motivating morecomplex formula in teaching laboratories via realis-tic measurements and will provide guidance to in-structors and their students in restricting the initialangular displacement or employing a more complexformula (c.f. [4] for a recent review).

This paper will demonstrate how to estab-lish these quantitative criteria by introducing anddemonstrating the techniques of Bayesian data anal-ysis. Using Bayesian model selection, a quantitativepreference between the small angle approximationand a numerical calculation of the exact integralequation is determined. First, an introduction toBayesian statistics is provided. The exact expres-sion for the period of a simple pendulum is derivedas well as the small angle approximation to estab-lish the Physics background. It is then establishedvia prior predictive distributions that it is possibleto differentiate the two formula in the presence of re-alistic measurement uncertainties. This is followedby Bayesian inference to fit the models to data; firstto pseudodata generated by each individual modelto determine that the statistical power of the anal-ysis is sufficient and then to measurements of theperiod of a simple pendulum taken with appara-tus available in the undergraduate teaching labora-tory. The posterior distribution is then analyzed andposterior predictive checks are made and are con-sistent with expectations. Finally, it is determinedat what initial angular displacement weak, moder-ate, and strong preference for the exact calculationare found given realistic, but precise, measurementsin the absence of nonconservative forces. This pref-erence can be used to inform recommendations of

arX

iv:2

104.

0862

1v2

[ph

ysic

s.ed

-ph]

25

Aug

202

1

2

maximum initial angular displacement in laboratorymaterials and textbooks and as a general introduc-tion to Bayesian techniques for implementation inthe undergraduate physics curriculum.

II. BAYESIAN STATISTICS

Bayesian data analysis is an increasingly-commonly used tool in physics research, e.g.gravitational-wave astronomy [5] and recently inultra-relativistic heavy ion collisions [6], amongmany others [7]. The techniques and process aresimilarly ripe for application to pedagogical ques-tions.

In Bayesian statistics, probability is interpretedas the degree of belief in a proposition given theavailable data and prior knowledge. Put anotherway, Bayesian statistics quantifies the plausibilityor betting odds of a proposition being true. TheBayesian approach is well-suited to applications withlimited data where other common-sense knowledgemay exist – a description that fits the introductorylaboratory well.

The central theorem of Bayesian statistics isBayes’ theorem,

p(H|d, I) =p(d|H, I) p(H|I)

p(d|I). (1)

Throughout this work, p(·) denotes probability andthe vertical bar in p(·|·) denotes conditionality, i.e.p(A|B,C) should be read as the probability of Agiven B and C. A colloquial description of Bayes’theorem is that it formalizes the idealized processof learning: a prior belief is compared with data.Based on how well it matches the observation, theprior belief is determined to be relatively more orless likely. This updated understanding forms theposterior belief. More formally, p(H|I) is called the“prior” and represents the belief in the hypothesis Hprior to comparison with measurements. p(d|H, I)is the “likelihood” and is the likelihood for the datad to be true given the hypothesis H. p(d|I) is the“Bayes evidence” and is typically treated as a nor-malization constant such that

∫p(H|d, I)dH = 1. It

quantifies how likely one believes the data to be givenother information I, which can include a given modelin which a particular choice of parameters forms ahypothesis H. I also denotes any other additionalinformation such as reasonable expectations that are

not informed by d, such as expectations from first-principles theory. p(H|d, I) is called the “posterior”and quantifies the belief in any hypothesis H poste-rior to comparison with measurement data d.

Bayes’ theorem may be used to solve problemsby inversion, asking “how likely is the data given aset of model parameters” (the likelihood) in order toanswer the primary question of interest, “how likelyis the set of model parameters given the data” (theposterior). Consequently, a hypothesis for which theobserved data is unlikely is itself an unlikely hypoth-esis. These concepts will be demonstrated using thefamiliar case of the simple pendulum.

In practice, the hypothesis H often representsa particular choice or distribution of parameter(s).Special cases, called conjugate priors, exist wherethe posterior can be found in closed form. In mostapplications, these special cases are not ideal for theproblem at hand. When this is the case, Bayes’ The-orem is numerically evaluated using computationaltools such as Markov Chain Monte Carlo, from whichthe posterior may be constructed. This yields an es-timate of the parameters of the model as well asany covariance structure, in turn allowing for theprecise quantification of the degree of belief. An in-troduction to Bayesian inference, including contrastswith Frequentist statistics, can be found in [8–10]. Aguide to rigorous Bayesian modeling may be foundin [10–12].

In this study, Bayes’ Theorem and related tech-niques will be used to determine when, given realis-tic data, it is possible to detect weak, moderate, andstrong preference for the exact expression for the pe-riod of a pendulum relative to the expression for theperiod of a pendulum in the small angle approxima-tion . To do so, realistically-generated pseudodataand principled prior expectations will be used in or-der to evaluate Bayes’ Theorem (Eq. 1) in a rigorousmodeling workflow. This study employs the use ofthe Gaussian likelihood function, usually representedin its logarithmic form, which assumes that uncer-tainties are normally distributed around a centralvalue.

ln p(d|H, I) = −1

2

N∑i=1

[ln(2πσ2) +

(di − YModel)2

σ2

],

(2)where YModel is the model calculation of the periodand σ2 = σ2data + σ2model. In this study, only oneparameter – the gravitational acceleration g – willbe varied and a particular value of g constitutes a

3

hypothesis H given the choice of model or otherassumptions I. Generalizing to higher dimensionsis beyond the scope of this work, but is well docu-mented in standard introductory textbooks [9, 10].

III. THE PERIOD OF A SIMPLEPENDULUM

For completeness, the small angle approximationand the exact expression for the formula of a simplependulum are derived. An ideal simple pendulumhas a bob of mass m suspended from a frictionlessattachment point by a massless rod of length L andmoves in a single angular dimension θ. In the ab-sence of nonconservative forces such as friction anddrag, the equation of motion is

θ = −g sin(θ)

L(3)

where g is the gravitational acceleration. In thisstudy, a fixed value L = 0.807 ± 0.0005m was mea-sured on a standard laboratory apparatus availablefor use in introductory physics laboratory courseson a meter-stick with milimeter gradations. In linewith common advice given in introductory labora-tories, the quoted uncertainty corresponds to halfof the smallest increment of the measuring device.The uncertainty on initial angular displacement cor-responds to the use of digital protractor with 0.01degree degradations; all angular model inputs thushave uncertainty ±0.005◦.For small displacements,sin θ ≈ θ and Eq. 3 can be identified as a simple har-monic oscillator. The period of a simple harmonicoscillator is T = 2π

√mk and it is trivial to identify

k = mgL . Consequently, in this regime,

T0 = 2π

√L

g(4)

and the uncertainty on the angular measurementdoes not enter into the model calculation. An ex-act expression for the period of a simple pendulum isreadily derived using the conservation of energy. Theexpression is simplified by assuming the pendulum tobe initially stationary and at its initial angular dis-placement θ0. The zero of potential energy is chosento be when θ(t) = 0. The energy conservation equa-tion is mgL (1− cos θ0) = 1

2mL2θ2 +mgL(1−cos θ).

It is then straightforward to solve for θ and integrateθ from 0 to θ0, corresponding to one-quarter of the

period. This yields T = 2√

2√

Lg

∫ θ00

1√cos θ−cos θ0

dθ.

This integral is improper when θ = θ0, but a simplesubstitution cos θ = 1−2 sin2(θ/2) may be combined

with a change of variables sinφ = sin(θ/2)sin(θ0/2)

to yield

T = 4

√L

g

∫ π/2

0

1√1− sin2(θ0/2) sin2(φ)

dφ (5)

and may be evaluated numerically using tools suchas SciPy [1, 13, 14]. Uncertainty on the initial angu-lar displacement θ0 is propagated with the assump-tion that this uncertainty is normally distributed anduncorrelated to the uncertainty in g. The small an-gle approximation (Eq. 4) and the exact expression(Eq. 5) are the models considered in this work.

A. Generating realistic measurements

In order to determine when the small angle ap-proximation becomes insufficient using realistic mea-surements, realistic choices must be made. Usingreal measurements in order to determine model pref-erence thresholds demands a sufficiently large num-ber of measurements as to be impractical, so someof the data in this study is generated using Eq. 5and assigning realistic uncertainties. Generated dataused in model validation is termed “pseudodata”,real measurements are performed for the final infer-ence, and generated data is used to calculate modelpreference thresholds. Pseudodata is generated formodel validation by making a set of simple choicesinformed by experience in undergraduate teachinglaboratories. In these laboratories, standard adviceabout the uncertainty is that the uncertainty of ameasurement is equal to half of the smallest incre-ment of the measurement device. A common mea-suring device in teaching laboratories includes me-ter sticks with millimetre gradations, correspond-ing to an uncertainty of ±0.0005 m. In teachinglaboratories, more precise timing mechanisms suchas photodiodes may also be available and the useof smartphone accelerometers in teaching laborato-ries has become more common [15]. The time stepin these measurements is significantly more precisethan the precision of a stopwatch and is not impactedby human reaction times. Common measurementswith smartphone accelerometers in recent courses atMcGill University had uncertainty of approximately±0.02 s [15] while timing uncertainties with the pho-todiode timer used in measurement are smaller still

4

at approximately ±0.005 s. Finally, for timing un-certainties corresponding to the use of a stopwatchin model selection is taken to be

√2 (0.250) s, corre-

sponding to a reaction time of 0.250 s [16] to startand stop the timer added in quadrature. Advicegiven to students is to time up to 10 periods of apendulum in order to reduce the relative error onthe measurement, a process replicated in this work.This study assumes that measurements are normallydistributed with standard deviation correspondingto the uncertainty.

Different values of the “true” gravitational accel-eration and timing precision may be specified whengenerating pseudodata; it is then possible to specifythe initial angular displacement and produce pseu-dodata that reflects the precision of data in introduc-tory teaching laboratories. It is then demonstratedthat the exact formula (Eq. 5) describes real datawell, strongly supporting the use of the exact for-mula to generate data when taking measurements isimpractical. It is possible to compare model predic-tions from the two methods of calculating the periodof a simple pendulum to this realistically-generateddata and establish ideal thresholds. A simple calcu-lation is sufficient to show that the models becomeincreasingly differentiable as θ0 increases, but thiswork will examine when this differentiation trans-lates into quantitative model preference. Generateddata is used for this study in order to calculate anidealized limit of model preference without system-atic biases from unaccounted-for physics.

IV. INFERENCE IN A RIGOROUSWORKFLOW

Reliable and robust data analysis requires a rig-orous workflow. Following such a step-by-step pro-cess acts as a safeguard that ensures the validity ofthe analysis as well as a rich understanding of themodels. In both research and pedagogical settings,clearly-defined steps are able to guide analysis, allowfor careful implementation and refinement, and en-sure that the final results are reliable. An additionalbenefit of a rigorous workflow for teaching is that itallows for the analysis to be broken into digestible,concrete steps that may be implemented in teachingmaterials.

The workflow presented here progresses from ex-plicitly defining prior knowledge all the way to ver-ifying that the final inference reproduces expecta-

tions. Each step is described, motivated, and im-plemented in the models already defined before pro-gressing to the next step. At the end of the work-flow, when both models have been validated andcalibrated to data, Bayesian model selection is in-troduced to establish rigorous criteria for when thedata itself demands one model over the other.

A. Step 1: Defining the prior state ofknowledge

The first step in Bayesian data analysis, once themodels have been defined, is to clearly motivate thechoice of prior. Ideally, the choice of prior reflectsconcrete understanding of the parameter and is suf-ficiently general as to constrain the problem withoutpre-determining or biasing the result [17]. For ex-ample, a prior that precludes the true result by con-struction can never recover the true result. The goalof the prior is to be constraining, but allow for thebulk of the constraint on belief to be determined bythe systematic comparison of the model to the datavia the likelihood.

As the initial angular displacement and periodof a simple pendulum are easily measured and thelength of the pendulum has been fixed, the only freemodel parameter in both the equations using thesmall angle approximation and the exact expressionfor the period of a pendulum is the gravitational ac-celeration g. It is reasonable to assert a priori thatfor the period to be rigorously defined, g cannot bezero. As gravity is an attractive force and the pe-riod is positive, g must be positive definite. It is alsopossible to ascribe some degree of physical intuitionto the problem. Gravitational acceleration on thesurface of a body scales with the mass of the bodyand gravitational acceleration on the Moon is ap-proximately 1.625 m/s2 [18] and the surface gravityof Jupiter is 24.79 m/s2 [19]. Proceeding with con-fidence that the mass of the Earth is likely betweenthat of the Moon and Jupiter, a weakly-informativeprior can be constructed that allows for some prob-ability of surprise. A common choice for a weakly-informative prior on positive definite quantities is theinverse gamma distribution. The inverse gamma dis-tribution has support on the interval (0,∞), match-ing the criteria we set and is easily tuned so thatonly 1% of the probability falls below 1.625 m/s2

and above 24.79 m/s2. No positive real value hasbeen ruled out, although some values have been de-

5

termined to be unlikely. This prior expectation forg is shown explicitly in Fig. 1. Note that the loca-tion of the mode of the distribution is not particu-larly relevant – by constructing a weakly-informativeprior, the goal is that the information contained inthe data will far outweigh the information in theprior. Other choices of prior that has similar sup-port are possible, but the inverse gamma distribu-tion has the nice feature of smoothly approaching 0on the bounds of its support. This matches physicalintuition. For example, if the prior set was the uni-form distribution U(1.625, 24.79), this would likelynot change the results of the inference, but it wouldrequire justifying that 24.8 m/s2 was infinitely lesslikely than 24.79m/s2 while all the values in betweenare equally likely.

FIG. 1. (Color online) The proposed prior probabilitydensity function (PDF) of the gravitational accelerationg measured using a simple pendulum used in a teachingphysics laboratory. This PDF is p(H|I) in Eq. 1 whereeach value of g is a hypothesis H.

B. Step 2: Prior predictive checks

Prior predictive checking is an important step ingaining an understanding of model behavior [20]. Inprior predictive checks, the computational model isevaluated with draws from the prior distribution.This is then used to assess the underlying modeland determine the presence of non-trivial featuresor predictions that are at odds with expectations.In this simple case, a prior defined with support[0,∞) would suggest that a finite value for the pe-riod should be expected when the gravitational ac-celeration is 0. However, in this case, the period is

ill-defined and so the expectation is at odds with ex-pectations. In more complex models, a non-trivialfeature could be an interaction between two compo-nents of a multi-component model that yields un-physical results and should be excluded by a morecarefully constructed prior. In this study, few ofthese features exist, but the step is still important inorder to demonstrate a rigorous modeling procedure.Predictions for the period with g sampled accordingto the prior distribution are shown in Fig. 2. Theseare compatible with the prior knowledge defined inSec. IV A. It is expected that measured periods ofthe pendulum are between approximately 1 and 5seconds, while longer and shorter periods and corre-sponding values of g are still theoretically possible,but disfavored. The absolute difference between thetwo models is small relative to the impact of thegravitational acceleration, meaning that the resultis highly sensitive to the parameter g.

FIG. 2. (Color online) 100 predictions for the period withg sampled from the prior and initial angular displacementθ0 sampled from a uniform distribution U(0, π/4).

C. Step 3: Model validation

Model validation, also known as closure testing orempirical coverage, is a means of assessing whetherthe model has sufficient power to constrain the pa-rameters given data. This is established by generat-ing pseudodata with each model and using the samemodel that generated the data to see to what degreethe parameters are constrained by the data and ifthe input value may be recovered when everything inthe data is contained in the model. This is an impor-tant step as it represents the best-case scenario: the

6

model is known to contain all the physics in the data.While this may not be a large concern in this simpleexample, it is a critical feature in more intensive nu-merical applications with sophisticated observables,e.g. gravitational wave, cosmology, or heavy ion col-lision data analysis.

Validation pseudodata is generated with gravita-tional acceleration g = 9.80665 m/s2 correspondingto the NIST reference value [21] and a timing un-certainty corresponding to the use of a photodiodetimer over 10 periods, comparable to the data usedin the full inference. Initial displacements are chosenwith arbitrary spacing, θ0 = [0.05, 0.2, 0.35, 0.4, π/5]radians, and calculations of the corresponding peri-ods are shown in Fig. 3. The models are then both

FIG. 3. (Color online) Pseudodata used for model vali-dation.

calibrated to their respective validation pseudodata(Fig. 3) and the posterior distributions are shown inFig. 4, where it can be clearly seen that both modelsare able to achieve closure when the data is perfectlyconsistent with the model. The small angle approx-imation model’s posterior is slightly more peakedthan that of the exact model because of the un-certainty from the angular measurement (the smallangle approximation model has no angular depen-dence). For each parameter value considered, thelikelihood (Eq. 2) is calculated and the value of theposterior can be computed. While this is simplein a small number of dimensions, this is impracti-cal for high-dimensional applications and comput-ing the posterior directly becomes computationallyinefficient. To overcome this, the posterior is evalu-ated numerically using Markov Chain Monte Carlo(MCMC). MCMC algorithms are efficient samplingtechniques in which “walkers” walk through the pa-

rameter space according to an algorithm. Each steptaken by a walker is a sample and, when converged,the distribution of samples corresponds to a specifiedtarget distribution – in this case, the posterior spec-ified in Eq. 1. The longer the walkers walk, the moresamples are drawn and the distribution of those sam-ples more closely resembles the distribution of inter-est. A chain of these samples can then be analyzed asdraws from the posterior. This study uses a numer-ical implementation of MCMC that uses a paralleltempering algorithm, ptemcee, whose features allowfor straightforward calculation of the Bayes evidence[22–24]. It is important to consider the MCMC be-

9.795 9.800 9.805 9.810 9.815 9.820g (m/s2)

gExact=9.8183+0.00340.0034gSmall Angle=9.7151+0.0014

0.0014

Small Angle Model PercentilesExact Model PercentilesTrue value

Exact ModelSmall Angle Approximation

FIG. 4. (Color online) Posterior distributions for g inmodel validation using the exact formula for the periodand the small angle approximation. The true value isshown in red and vertical dashed lines denote the 16th,50th, and 84th percentile of the samples in increasing g.When the models contain the full information present inthe data to which they are compared, both are able torecover the true value.

havior to ensure that the chains did in fact convergeto their target distributions, i.e. that samples indeedapproximate draws from the distribution of interest.This is most simply analyzed by considering auto-correlation, which quantifies how correlated samplesare within a certain “lag”.1 True independent sam-ples of the posterior should be uncorrelated, while

1 Autocorrelation is calculated by taking the inner productof the unshifted chain of samples with a chain shifted bythe lag, normalized by the product of the magnitudes ofthe shifted and unshifted chains. As a result, the 0 lagautocorrelation is 1 by construction.

7

correlated samples suggest that the walkers are nottaking random steps, but are walking in a particulardirection and do not represent draws from the distri-bution of interest. The autocorrelation for the exactand small angle approximation MCMC chains areshown in Fig. 5, which show that after even a sin-gle step, the chain is uncorrelated and is samplingthe target distribution successfully, uninfluenced bythe previous step. The final check that must be per-

0 5 10 15 20 25lag (steps)

0.0

0.2

0.4

0.6

0.8

1.0

Auto

corre

latio

n

Exact modelSmall angle approximation

FIG. 5. (Color online) Autocorrelation of MCMC chainsfor model validation. The rapid drop and subsequentsmall-scale fluctuations around 0 are indicative of a decor-related chain, meaning that the chain is sampling fromthe target distribution (the posterior).

formed before confidently progressing to the final in-ference is to consider the posterior predictive distri-bution, Fig. 6. Predictions can be made from theposterior distribution to form the posterior predic-tive distribution, which helps determine if the modeltruly behaves as anticipated by considering the dis-tribution of results. If something has gone wrongin the inference or the MCMC, then the predictionsmade using the posterior distribution may not re-produce the data or may reveal underlying struc-ture which is contrary to expectations but useful forimproving the model. For example, the posteriorpredictive distributions may reproduce some of thedata, but not other data, indicating that there ismissing physics that is causing tension in the model.As expected from Fig. 4, where the models are cali-brated to data generated by themselves, both mod-els are able to successfully reproduce pseudodata.This is encouraging, as it is known by constructionthat there is no feature of the data that the respec-tive models cannot account for. The distribution ofmodel predictions are shown with violin plots, which

show distributional information analagously to box-and-whisker plots. In violin plots, the width of theblobs corresponds to the probability density of thedata and is able to show more information than abox-and-whisker plot, allowing for more intuitive in-terpretation of the data.

FIG. 6. (Color online) Violin plot of the prior and poste-rior predictive distributions for g from model validation.The posterior distributions are those shown in Fig. 4.

D. Step 4: Inference with data

After all the previous steps are completed, it isappropriate to systematically compare the models toreal data. All the previous steps have indicated thatthe models should have discriminating power andshould be capable of describing the data and haveset expectations for the constraint.

Measurements were made with a pendulum oflength L = 0.807±0.0005 m and a cylindrical bob ofmass 10± 0.05g with times recorded using a photo-transistor connected to a Pasco 850 universal inter-face Model UI-5000 for data acquisition. Measure-ments of initial angular displacement θ0 were madeby a digital protractor and the bob was releasedfrom the same location with the guide of a standto ensure uniformity of angle. 5 measurements of10 periods were made at each angular displacement,consistent with the typical number of measurementsperformed by students in introductory teaching lab-oratories [15].

The posterior is evaluated numerically usingMCMC, as performed and validated in the previ-ous section. The posterior distribution for the grav-itational acceleration g is shown in Fig. 7; the ex-

8

act model is able to infer the gravitational acceler-ation with a relatively precise 68% credible region– 9.818 ± 0.003 m/s2 – while the small angle ap-proximation infers a result with higher precision –9.715 ± 0.001 m/s2 – but sacrifices accuracy with aremarkable ∼ 0.1 m/s2 bias from the standard value9.80665 m/s2 [21]. While the reference value is notwithin the 68% credible region of the exact model,the bias is greatly reduced and predictions with theposterior are investigated further in the next sub-section and match the data well. The source of thedifference in precision between the formula is read-ily interpretable: as the exact formula requires thepropagation of uncertainty from the initial angulardisplacement, the posterior has a wider 68% credibleinterval than that of the small angle approximationwhich does not account for the angular displacementat all.

9.72 9.74 9.76 9.78 9.80 9.82g (m/s2)

gExact=9.818+0.0030.003gSmallAngle=9.715+0.001

0.001

Small Angle PercentilesExact Model Percentiles9.80665 m/s2

Exact ModelSmall Angle Approximation

FIG. 7. (Color online) Independently normalized poste-rior distributions for g using the exact formula for theperiod and the small angle approximation. The standardvalue [21] is shown in red and vertical dashed lines denotethe 16th, 50th, and 84th percentile of the samples in in-creasing g. This corresponds to the central value and the68% credible interval.

E. Step 5: Posterior predictive checks

Now that the model has been calibrated to data,it is important to examine the posterior predictivedistributions. This can also elucidate features ofthe predictions and provide insight as to why onemodel may fail to recover the expected result while

the other succeeds. These predictions are made bytaking samples from the posterior and using themto make predictions for the period using the modelin question. The distribution of these predictionsforms the posterior predictive distribution (Fig. 8).The source of the bias in the small angle approxi-mation model can be clearly seen in the posteriorpredictive distribution. The predictions from theexact model posterior are constrained around themeasurements and are broadly consistent with themeasured uncertainty, while the small angle modelis biased by undershooting at large angular displace-ment and overshooting small displacements. As a re-sult, the posterior predictive distributions make theincompleteness of the small angle approximation im-mediately apparent and interpretable. This clearly

FIG. 8. (Color online) Violin plot of the posterior predic-tive distributions for g. The posterior distributions arethose shown in Fig. 7.

establishes that it is possible to detect the bias incalculations of gravitational acceleration introducedby using the small angle approximation of the periodof a pendulum while including large initial angulardisplacements and realistic measurement uncertain-ties. Through the use of a rigorous modeling work-flow, it is straightforward to determine and interpretthe source of this bias through visualization of theposterior and posterior predictive distributions. Itis also clear that the exact formula for the period ofa simple pendulum contains the majority of relevantphysics necessary to describe real data.

9

F. Model selection

Now that it has been established that a differenceis detectable with a finite number of measurementsin θ0, it is time to turn to the tools of Bayesianmodel selection to quantify when a preference for theexact expression for the period is weak, moderate,and strong. Bayesian model selection is evaluatedvia the Bayes factor. The Bayes factor is the ratio ofBayes evidences (the denominator of Eq. 1) betweentwo models,

B01 =p(d|M0)

p(d|M1)(6)

where models Mi are subscripted 0 and 1. TheBayes factor B01 is used to quantify model prefer-ence. Hereafter, the exact formula for the periodand the small angle approximation are subscripted 0and 1, respectively. A Bayes factor greater than onerepresents an increase of support for M0 relative toM1 and directly corresponds to the odds of M0:M1.

In addition to explicit quantification of the pref-erence between two models, the Bayes factor allowsfor determination of a preferred model when the re-sult is not immediately obvious (c.f. [6]). In thesimple pendulum, a single observable – the period– is considered. However, in more sophisticated ex-periments, there may be multiple observables andthe comparison becomes more complex. While toolssuch as the chi-squared per degree of freedom ex-ist, these often make assumptions about the shapeof the posterior distribution, e.g. the chi-squaredtest implicitly assumes normally-distributed data;the Bayes factor makes no additional distributionalassumptions beyond those explicitly chosen by thepriors and the likelihood. Another advantage of theBayes factor is that it penalizes the incorporation ofadditional parameters. A maxim attributed to Johnvon Neumann is that “with four parameters I canfit an elephant, and with five I can make him wig-gle his trunk” [25]. It is therefore prudent to followthe guidance of William of Ockham’s Law of Parsi-mony: it is futile to do with more things that whichcan be done with fewer [26]. Accordingly, a modelwith many parameters should be disfavored in com-parison with an equally-successful model with fewerparameters. The Bayes factor is an implementationof Occam’s Razor and requires the demand for ad-ditional complexity to come from the data itself. Afurther discussion of Bayesian model selection maybe found in [8].

Empirical scales are used to determine when thereis weak, moderate, and strong evidence for M0 vs.M1. The Bayes factor is easily interpretable as itgives the direct odds ratio of one model to the other.In this work, the Jeffreys’ Scale is used (Table I),although in principle one can construct an appro-priate scale for the application at hand. Data is

|lnB01| Odds Probability Strength of evidence< 1.0 ≤ 3 : 1 < 0.750 Inconclusive1.0 ∼ 3 : 1 0.750 Weak evidence2.5 ∼ 12 : 1 0.923 Moderate evidence5.0 ∼ 150 : 1 0.993 Strong evidence

TABLE I. The Jeffreys’ Scale, reproduced from [8].

generated for the model selection study as follows.Taking sufficient accurate measurements by hand isunreasonable and it has already been demonstratedthat the exact expression of the period describes themotion of a pendulum well. Therefore, it is rea-sonable to generate data using the model, given aset of simple assumptions. It is unreasonable to as-sume that students would be able to make measure-ments at increments finer than 1◦ with a one-meter-long pendulum or at more than 12 increments ininitial angular displacement θ0. Typically, studentsmake fewer measurements in teaching laboratoriesand this study represents an attempt to find an ide-alized, maximally-restrictive constraint on initial an-gular displacement θ0.

12 proxy data points are generated if the max-imum θ0 is larger than 12◦. If the maximum θ0 isless than 12◦, data is generated in 1◦ increments.The Bayes factor is then calculated and it is possibleto interpolate between calculations with their uncer-tainty to determine at what Max[θ0] the evidence forthe exact model becomes weak, moderate, or strong.These data points are generated with the uncertain-ties commensurate with photodiode, accelerometer,and stopwatch timing as discussed in Sec. III A. In-cluding these results demonstrates the importance ofaccounting for uncertainty when offering data-drivenguidance and will allow for appropriate implementa-tion of these results in the classroom.

The ln Bayes factor is shown in Fig. 9. As antici-pated, as the maximum initial angular displacementθ0 increases, there is progressively stronger prefer-ence for the exact expression. The constraint thendiminishes as the number of measurements decrease.Preference for the exact calculation at maximum ini-tial angular displacement θ0 is given in Table II. It

10

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8Max[ 0] (rad)

10 1

100

101

102

103

104ln

B01

Model Selection in a Simple PendulumStrong PreferenceModerate PreferenceWeak Preference

Bayes Factor: StopwatchBayes Factor: AccelerometerBayes Factor: Photodiode

FIG. 9. (Color online) lnB01 demonstrating preferencefor the exact expression for the period of the pendulumover the small angle approximation. Positive values de-note preference for the exact model. Horizontal lines de-note thresholds for weak, moderate, and strong prefer-ence.

may be stated confidently that the small angle ap-proximation is valid below these model preferencethresholds.

Strength of evidence |lnB01| θ0 (rad) θ0 (deg)Photodiode timingWeak 1.0 0.140± 0.003 8.0+0.1

−0.2

Moderate 2.5 0.155± 0.002 8.89+0.09−0.1

Strong 5.0 0.171± 0.001 9.81± 0.06Accelerometer timingWeak 1.0 0.195± 0.007 11.2± 0.4Moderate 2.5 0.234± 0.004 13.4+0.2

−0.3

Strong 5.0 0.275± 0.003 15.8± 0.2Stopwatch timingWeak 1.0 0.62+0.04

−0.03 35± 2Moderate 2.5 0.78± 0.02 44.6± 0.9

TABLE II. Model preference thresholds in which the ex-act formula is preferred over the small angle approxima-tion. Subscript 0 denotes the exact formula and subscript1 denotes the small angle approximation. Strong prefer-ence was not found for stopwatch timing when maximumangular displacement was limited to π/4 rad.

Timing Precision Recommended Restriction(rad) (deg)

Photodiode (± ∼ 0.005 s) ∼ 0.17 ∼ 10Accelerometer (± ∼ 0.02 s) ∼ 0.26 ∼ 15

Stopwatch (± ∼√

2(0.250) s) π/4 45

TABLE III. Guidance for restricting initial angular dis-placement for simple pendula when using the small angu-lar approximation.

V. CONCLUSIONS

This work demonstrates how to set rigorous data-driven constraints on the use of the small angle ap-proximation in introductory physics experiments us-ing Bayesian inference as well as how to approachstatistical modeling through a workflow. This is thefirst instance in the literature of modern computa-tional and statistical techniques being applied to thisproblem and demonstrates how the modeling proce-dure followed in research applications may be ap-plied to pedagogical settings.

The heuristic guidance of constraining simplependula to 15◦ is overly restrictive when timingwith typical precision found in stop watches, butis reasonable when using a smartphone accelerom-eter and even overly-permissive when using precisetiming methods such as optical timing with photo-diodes. The data does not exhibit strong prefer-ence for the exact expression with photodiode tim-ing until 0.171 ± 0.001 radians (9.81 ± 0.06◦) andwith accelerometer timing until 0.275±0.003 radians(15.8±0.2◦), while stopwatch timing is insufficientlyprecise to establish strong preference between thetwo models at maximum displacements up to π/4radians (45◦). This suggests that current guidanceof restriction to ∼ 15◦ is appropriate for timing ofprecision ∼ ±0.02 s, but displacements with moreprecise timing (∼ ±0.005 s) should be restricted to∼ 10◦ while displacements with less precise timingcommensurate with stopwatches and human reactiontimes should not be restricted below ∼ 45◦.

This analysis represents the current state-of-the-art guidance for laboratory manuals and textbooksas it has used realistic best-case uncertainties andadvanced statistical techniques to determine whenthe breakdown of the approximation is actually de-tectable. Future works should continue with thepresent guidance, but modify their justification tobe data-driven rather than being motivated by a 1%discrepancy in the small angle approximation. Thiswould both be more directly applicable to instructorsand would more faithfully communicate to studentshow scientific guidance is derived. Additionally, thisdemonstration sets out a workflow by which studentscould derive their own guidance in this and otherlaboratory applications while gaining experience inrigorous statistical and scientific modeling.

11

VI. ACKNOWLEDGEMENTS

This work was supported in part by the Natu-ral Sciences and Engineering Research Council ofCanada. Thanks are due to C. Gale, N. Provatas,M. Singh, M. Frick, N. Fortier, and R. Yazdi forcritical reading of the manuscript and to D. Everettfor both a critical reading of the manuscript and de-

tailed discussions on the implementation of Bayesiantechniques and their interpretation. R. Turner, P.Movafegh, and S. Biunno provided materials and ex-pertise for obtaining measurements. R. Furnstahl’sresources on Bayesian inference were instrumental inthe development of this work. C. Gale, S. Jeon, andK. Ragan provided guidance, useful discussions, andsupport to this study.

[1] F. M. S. Lima and P. Arun, “An accurate formulafor the period of a simple pendulum oscillating be-yond the small angle regime,” American Journal ofPhysics 74, 892–895 (2006), https://doi.org/10.1119/1.2215616.

[2] Brian S. Blais, “Model comparison in the intro-ductory physics laboratory,” The Physics Teacher58, 209–213 (2020), https://doi.org/10.1119/1.5145420.

[3] Douglas C Giancoli, Physics: principles with appli-cations (Boston: Pearson, 2016).

[4] Peter F Hinrichsen, “Review of approximate equa-tions for the pendulum period,” European Journalof Physics 42, 015005 (2020).

[5] Rory J E Smith, Gregory Ashton, Avi Va-jpeyi, and Colm Talbot, “Massively parallelBayesian inference for transient gravitational-wave astronomy,” Monthly Notices of the RoyalAstronomical Society 498, 4492–4502 (2020),https://academic.oup.com/mnras/article-pdf/

498/3/4492/33798799/staa2483.pdf.[6] D. Everett et al. (JETSCAPE), “Multi-system

Bayesian constraints on the transport coefficients ofQCD matter,” (2020), arXiv:2011.01430 [hep-ph].

[7] Udo von Toussaint, “Bayesian inference in physics,”Rev. Mod. Phys. 83, 943–999 (2011).

[8] Roberto Trotta, “Bayes in the sky: Bayesian infer-ence and model selection in cosmology,” Contempo-rary Physics 49, 71–104 (2008), https://doi.org/10.1080/00107510802066753.

[9] Devinderjit Sivia and John Skilling, Data analysis:a Bayesian tutorial (OUP Oxford, 2006).

[10] Andrew Gelman, John B Carlin, Hal S Stern,David B Dunson, Aki Vehtari, and Donald B Rubin,Bayesian data analysis (CRC press, 2013).

[11] Andrew Gelman, Aki Vehtari, Daniel Simpson,Charles C. Margossian, Bob Carpenter, YulingYao, Lauren Kennedy, Jonah Gabry, Paul-ChristianBurkner, and Martin Modrak, “Bayesian workflow,”(2020), arXiv:2011.01808 [stat.ME].

[12] M. Betancourt, “Falling (in love with princi-pled modeling),” (2019), betanalpha.github.io/assets/case_studies/falling.html.

[13] Stephen D. Schery, “Design of an inexpensive pen-dulum for study of large-angle motion,” AmericanJournal of Physics 44, 666–670 (1976), https://

doi.org/10.1119/1.10352.[14] Pauli Virtanen et al., “Scipy 1.0: Fundamental algo-

rithms for scientific computing in python,” NatureMethods (2020).

[15] Private communication with instructor, “McGillUniversity Physics 101 lab manual,” (2020).

[16] Robert T. Wilkinson and Sue Allison, “Age and Sim-ple Reaction Time: Decade Differences for 5,325Subjects,” Journal of Gerontology 44, P29–P35(1989).

[17] Stan Development team, “Prior choice recommen-dations,” (2020), https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations.

[18] C. Hirt and W.E. Featherstone, “A 1.5km-resolutiongravity field model of the Moon,” Earth and Plane-tary Science Letters 329-330, 22–30 (2012).

[19] NASA, “By the numbers: Jupiter,” (2021),https://solarsystem.nasa.gov/planets/

jupiter/by-the-numbers/.[20] George EP Box, “Sampling and Bayes’ inference in

scientific modelling and robustness,” Journal of theRoyal Statistical Society: Series A (General) 143,383–404 (1980).

[21] Ambler Thompson and Barry N. Taylor, “Guidefor the use of the international system of units(SI),” NIST Special Publication 811 (2008), https://physics.nist.gov/cuu/pdf/sp811.pdf.

[22] W. D. Vousden, W. M. Farr, and I. Man-del, “Dynamic temperature selection for par-allel tempering in Markov chain Monte Carlosimulations,” Monthly Notices of the RoyalAstronomical Society 455, 1919–1937 (2015),https://academic.oup.com/mnras/article-pdf/

455/2/1919/18514064/stv2422.pdf.[23] https://github.com/willvousden/ptemcee.[24] Daniel Foreman-Mackey, David W. Hogg, Dustin

Lang, and Jonathan Goodman, “emcee: TheMCMC hammer,” Publications of the AstronomicalSociety of the Pacific 125, 306–312 (2013).

[25] Freeman Dyson, “A meeting with Enrico Fermi,”Nature 427, 297–297 (2004).

12

[26] W. M. Thorburn, “The myth of Occam’srazor,” Mind XXVII, 345–353 (1918),

https://academic.oup.com/mind/article-pdf/

XXVII/3/345/9877777/345.pdf.