physics 736 experimental methods in nuclear-, …neutrino.physics.wisc.edu/teaching/phys736/... ·...

Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013

Physics 736

Experimental Methods in Nuclear-, Particle-, and Astrophysics

- Statistics and Error Analysis -

Karsten [email protected]


Statistics & Error AnalysisTopics

• introduction to statistics and error analysis• probability distributions• treatment of experimental data• measurement process and errors• maximum likelihood• parameter estimation• method of least squares • Bayesian and frequentist approach• hypothesis testing and significance• confidence intervals and limits



• introduction to statistics and error analysis• treatment of experimental data• probability distributions• maximum likelihood• parameter estimation• method of least squares • Bayesian approach• hypothesis and significance testing • intervals and limits



• introduction to statistics and error analysis• probability distributions• treatment of experimental data• measurement process and errors• maximum likelihood• parameter estimation• method of least squares • Bayesian and frequentist approach• hypothesis testing and significance• confidence intervals and limits

Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009

Probability Distributions - Revisited


Statistical Distributions

ννννν

binomial

Poisson

Gaussian

chisquare distribution



Gaussian

binomial

Poisson

chisquare distribution

P (r) =N !

r!(N ! r)!pr(1 ! p)N!r

P (r) =µre!µ

r!

P (x) =1

!!

(2")e!

(x!µ)2

2!2A

B

C

P (u)du =(u/2)(u/2)!1e!u/2

2!(!/2)duD

what are distributions A-D?


Probability and Statistics

binomial distribution

r r



binomial distributionfor large N




for large N



Poisson distribution



Gaussian distribution




σ



Chisquare distribution

ννννν

χ2

family of distributions

different χ2 distribution for each value of the degrees of freedom



p ! 0.05

p ! 0.05


Mean, Median, Mode

mode=most frequent value in data set


Measurement Process and Errors


Probability and StatisticsMeasurement is a random process


Probability and StatisticsMeasurement is a random process described by a

probability distribution




σ=instrumental precision


Data and Error BarsSystematic and Random Errors


Probability and StatisticsWhat about errors?


1

Figure 2: A historical perspective of values of a few particle properties tabulated in this Review as a function of date ofpublication of the Review. A full error bar indicates the quoted error; a thick-lined portion indicates the same but withoutthe “scale factor.”


Data with Error Bars



• What fraction of data points (+error bars) do you not expect to fall on the line?



For ±1σ, 1/3 of data should be outside fit


Sampling and Parameter Estimation



some terminology

sample • data • set of N measurements

population• observable space • underlying parent distribution

estimate• best value

variance of estimate• error on best value




how confident are we in our measurement?

best estimate

standard deviation



best value minimizes variance between estimate and true value

• method of estimation– 1) determine best estimate– 2) determine uncertainty on best estimate


Examples

mean and error from series of measurements

mental Dat.

(4.s2)

;ense that.51). TheEquationat unlikernent be-s it quire

(4.53)

(4.s4)

r fromen oc-lantityof the'moremeth-imum

valuelevia-tY oi.

f.55)

cor-

,56)

t in

rl5 Examples of Applications 97

1,5 Examples of Applications

15.1 Mean and Error from a Series of Messurements

f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:

17.62l7.61t7.61

17.6217.6217.615

| 7.6t 511.625t7.61

17.62l7.6217.@5

17.6117.6217.61

Uhat is the best estimate for the length of this object?

Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then

I = 17.61533

rhile (4.52) gives the standard deviation

d: 5.855 x l0-3 .

This can now be used to calculate the standard error of the mean (4.50),

o(r) = A/V15:0.0015 .The best value for the length of the object is thus

x = 17.616x.0.002 .

Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!

f5.2 Combining Data with Different Errors

Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:

2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us

2.203 t 0.004 ps2.198t0.(X)2 ps

2.202+0.003 ps2.1!X5t0.0020 us

What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;

however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value

t = 2.1%96

with an error


Examples

combining data with different errors

mental Dat.

(4.s2)

;ense that.51). TheEquationat unlikernent be-s it quire

(4.53)

(4.s4)

r fromen oc-lantityof the'moremeth-imum

valuelevia-tY oi.

f.55)

cor-

,56)

t in

rl5 Examples of Applications 97

1,5 Examples of Applications

15.1 Mean and Error from a Series of Messurements

f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:

17.62l7.61t7.61

17.6217.6217.615

| 7.6t 511.625t7.61

17.62l7.6217.@5

17.6117.6217.61

Uhat is the best estimate for the length of this object?

Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then

I = 17.61533

rhile (4.52) gives the standard deviation

d: 5.855 x l0-3 .

This can now be used to calculate the standard error of the mean (4.50),

o(r) = A/V15:0.0015 .The best value for the length of the object is thus

x = 17.616x.0.002 .

Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!

f5.2 Combining Data with Different Errors

Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:

2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us

2.203 t 0.004 ps2.198t0.(X)2 ps

2.202+0.003 ps2.1!X5t0.0020 us

What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;

however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value

t = 2.1%96

with an error

muon lifetime measurements


Examples

count rates and errors

98

o(r) = 0.00061.

Note that this value isThe best value for the

*.{tr

4. Stat ist ics and the Trca

smaller than the error on anv of the inclifetime is thus

t = 2.1970+ 0.0006 ps .

4.5.3 Determination of Count Rales and Their Errors

Example 4.3 Consider the following series of measuremenrs ofrom a detector viewing a 22Na source,

2201 2145 )r) ' ,

What is the decay rate and its uncertainty?

Since radioactive decay is described by a Poisson distributicfor this distribution to find

fr=i=2205.6 and

o(ir) =

The count rate is thus

Count Rate = (2206x.21) counts/min.

It is interesting to see what would happen if instead of cotriods we had counted the total 5 minutes without stopping.served a total of 11028 counts. This constitutes a sample of 4for 5 minutes is thus 11 208 and the error on this, o = l.counts per minute, we divide by 5 (see the next section) toidentical to what was found before. Note that the error takcncount rate in 5 minutes. A common error to be avoided is tominute and then take the square root of this number.

4.5.4 Null Experiments. Setting Confidence Limis \l 'her

Many experiments in physics test the validity of certain theosearching for the presence of specific reactions or decal's Isuch measurements, an observation is made for a cenain anif one or more events are observed. the theoretical lau isevents are observed. the converse cannot bc said to bc truc

23m21ffi

lE=---- ' - =215


Examples

error propagation (e.g. polarization measurements)

! =R ! L

R + L


Measurements and LimitsConfidence Levels




how confident are we in our measurement?

best estimate

standard deviation



For ±1σ, 1/3 of data should be outside fit


One-parameter Confidence Level


Multiparameter Confidence Levels

hold some parameters fixed, vary only subset

allow all parameters to vary


rate orflux or# of events

x confidenceinterval (CL=68.3%)

confidenceinterval (CL=99%)

Confidence Intervals: Measurements and Limits


Issue of Coverage

• Correct coverage

• Confidence intervals overcover (i.e. are too conservative)

• Reduced power to reject wrong hypotheses

• Confidence intervals undercover

• Measurement pretends to be more accurate than it actually is

Proper coverage can be tested by Monte Carlo simulations

physics 736 experimental methods in nuclear-, …neutrino.physics.wisc.edu/teaching/phys736/... ·...

Documents