physics 736 experimental methods in nuclear-, particle...

Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2010

Physics 736

Experimental Methods in Nuclear-, Particle-, and Astrophysics

Lecture 12

Karsten [email protected]


course websitehttp://neutrino.physics.wisc.edu/teaching/PHYS736/

Course Schedule and Reading

todayʼs homework comes in 2 parts:

first part = reading + slides, due on Monday, March 15, 2010second part = some exercises, due next Wednesday, March 17, 2010

http://neutrino.physics.wisc.edu/teaching/PHYS736/

http://neutrino.physics.wisc.edu/teaching/PHYS736/


Statistics & Error AnalysisTopics

• introduction to statistics and error analysis• probability distributions• treatment of experimental data• measurement process and errors• maximum likelihood• parameter estimation• method of least squares • Bayesian and frequentist approach• hypothesis testing and significance• confidence intervals and limits

Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009

Statistical Distributions



which distributions are continuous?

which are discrete?

binomial

Poisson

Gaussian

chisquare distribution



ννννν

binomial

Poisson

Gaussian




P (r) =N !

r!(N ! r)!pr(1 ! p)N!r

P (r) =µre!µ

r!

P (x) =1

!!

(2")e!

(x!µ)2

2!2A

B

C

P (u)du =(u/2)(u/2)!1e!u/2

2!(!/2)duD

what are distributions A-D?



Gaussian

binomial

Poisson


P (r) =N !

r!(N ! r)!pr(1 ! p)N!r

P (r) =µre!µ

r!

P (x) =1

!!

(2")e!

(x!µ)2

2!2A

B

C

P (u)du =(u/2)(u/2)!1e!u/2

2!(!/2)duD

what are distributions A-D?



p ! 0.05

p ! 0.05


Probability and StatisticsMeasurement is a random process described by a

probability distribution




σ=instrumental precision


Probability and StatisticsWhat about errors?


1

Figure 2: A historical perspective of values of a few particle properties tabulated in this Review as a function of date ofpublication of the Review. A full error bar indicates the quoted error; a thick-lined portion indicates the same but withoutthe “scale factor.”


Sampling and Parameter Estimation



some terminology

sample • data • set of N measurements

population• observable space • underlying parent distribution

estimate• best value

variance of estimate• error on best value



best value minimizes variance between estimate and true value

• method of estimation– 1) determine best estimate– 2) determine uncertainty on best estimate


Examples

mean and error from series of measurements

mental Dat.

(4.s2)

;ense that.51). TheEquationat unlikernent be-s it quire

(4.53)

(4.s4)

r fromen oc-lantityof the'moremeth-imum

valuelevia-tY oi.

f.55)

cor-

,56)

t in

rl5 Examples of Applications 97

1,5 Examples of Applications

15.1 Mean and Error from a Series of Messurements

f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:

17.62l7.61t7.61

17.6217.6217.615

| 7.6t 511.625t7.61

17.62l7.6217.@5

17.6117.6217.61

Uhat is the best estimate for the length of this object?

Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then

I = 17.61533

rhile (4.52) gives the standard deviation

d: 5.855 x l0-3 .

This can now be used to calculate the standard error of the mean (4.50),

o(r) = A/V15:0.0015 .The best value for the length of the object is thus

x = 17.616x.0.002 .

Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!

f5.2 Combining Data with Different Errors

Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:

2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us

2.203 t 0.004 ps2.198t0.(X)2 ps

2.202+0.003 ps2.1!X5t0.0020 us

What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;

however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value

t = 2.1%96

with an error


Examples

combining data with different errors

mental Dat.

(4.s2)

;ense that.51). TheEquationat unlikernent be-s it quire

(4.53)

(4.s4)

r fromen oc-lantityof the'moremeth-imum

valuelevia-tY oi.

f.55)

cor-

,56)

t in

rl5 Examples of Applications 97

1,5 Examples of Applications

15.1 Mean and Error from a Series of Messurements

f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:

17.62l7.61t7.61

17.6217.6217.615

| 7.6t 511.625t7.61

17.62l7.6217.@5

17.6117.6217.61

Uhat is the best estimate for the length of this object?

Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then

I = 17.61533

rhile (4.52) gives the standard deviation

d: 5.855 x l0-3 .

This can now be used to calculate the standard error of the mean (4.50),

o(r) = A/V15:0.0015 .The best value for the length of the object is thus

x = 17.616x.0.002 .

Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!

f5.2 Combining Data with Different Errors

Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:

2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us

2.203 t 0.004 ps2.198t0.(X)2 ps

2.202+0.003 ps2.1!X5t0.0020 us

What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;

however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value

t = 2.1%96

with an error

muon lifetime measurements


Examples

count rates and errors

98

o(r) = 0.00061.

Note that this value isThe best value for the

*.{tr

4. Stat ist ics and the Trca

smaller than the error on anv of the inclifetime is thus

t = 2.1970+ 0.0006 ps .

4.5.3 Determination of Count Rales and Their Errors

Example 4.3 Consider the following series of measuremenrs ofrom a detector viewing a 22Na source,

2201 2145 )r) ' ,

What is the decay rate and its uncertainty?

Since radioactive decay is described by a Poisson distributicfor this distribution to find

fr=i=2205.6 and

o(ir) =

The count rate is thus

Count Rate = (2206x.21) counts/min.

It is interesting to see what would happen if instead of cotriods we had counted the total 5 minutes without stopping.served a total of 11028 counts. This constitutes a sample of 4for 5 minutes is thus 11 208 and the error on this, o = l.counts per minute, we divide by 5 (see the next section) toidentical to what was found before. Note that the error takcncount rate in 5 minutes. A common error to be avoided is tominute and then take the square root of this number.

4.5.4 Null Experiments. Setting Confidence Limis \l 'her

Many experiments in physics test the validity of certain theosearching for the presence of specific reactions or decal's Isuch measurements, an observation is made for a cenain anif one or more events are observed. the theoretical lau isevents are observed. the converse cannot bc said to bc truc

23m21ffi

lE=---- ' - =215


Examples

error propagation (e.g. polarization measurements)

! =R ! L

R + L


Mean, Median, Mode


Method of Least SquaresQuality of Fit


Data with Error Bars

For ±1σ,

1/3 of data should be outside fit


Accuracy and Precision

a) precise but inaccurate datab) accurate but imprecise data



2-4 keV

Time (day)

Res

idua

ls (c

pd/k

g/ke

V)

DAMA/LIBRA ! 250 kg (0.87 ton"yr)

2-5 keV

Time (day)

Res

idua

ls (c

pd/k

g/ke

V)


2-6 keV

Time (day)

Res

idua

ls (c

pd/k

g/ke

V)


Figure 1: Experimental model-independent residual rate of the single-hit scintillationevents, measured by DAMA/LIBRA,1,2,3,4,5,6 in the (2 – 4), (2 – 5) and (2 – 6)keV energy intervals as a function of the time. The zero of the time scale is January1st of the first year of data taking of the former DAMA/NaI experiment [15]. Theexperimental points present the errors as vertical bars and the associated time binwidth as horizontal bars. The superimposed curves are the cosinusoidal functionsbehaviors A cos!(t ! t0) with a period T = 2!

" = 1 yr, with a phase t0 = 152.5 day(June 2nd) and with modulation amplitudes, A, equal to the central values obtainedby best fit over the whole data including also the exposure previously collected bythe former DAMA/NaI experiment: cumulative exposure is 1.17 ton " yr (see alsoref. [15] and refs. therein). The dashed vertical lines correspond to the maximumexpected for the DM signal (June 2nd), while the dotted vertical lines correspond tothe minimum. See text.

5




how confident are we in our measurement?

best estimate

standard deviation


Probability and Statistics

Gaussian distribution


One-parameter Confidence Level


Multiparameter Confidence Levels

hold some parameters fixed, vary only subset

allow all parameters to vary


rate orflux or# of events

x confidenceinterval (CL=68.3%)

confidenceinterval (CL=99%)

Confidence Intervals: Measurements and Limits


Issue of Coverage

• Correct coverage

• Confidence intervals overcover (i.e. are too conservative)

• Reduced power to reject wrong hypotheses

• Confidence intervals undercover

• Measurement pretends to be more accurate than it actually is

Proper coverage can be tested by Monte Carlo simulations


Flip-Flopping

• Flip-flopping between measurements and upper limits with different confidence levels spoils the coverage of the stated confidence intervals

• Easy to show with a toy Monte Carlo

„We will state a measurement with a 1σ error (i.e. CL=68.3%) if the measurement result is above mσ, and an 99% CL upper limit otherwise.“

The flip-flopping attitude (example):


Classical Confidence Intervals


Bayesian Interval


Example


Feldman & Cousins Approach

• Provides confidence intervals that change smoothly from upper limits to measurements

• „User“ just needs to decide for a confidence level

• Flip-flopping problem is solved

• Uses Neyman‘s construction and a Likelihood Ratio to decide what values are included into confidence intervals


• (Frequentist) Definition of the confidence interval for the measurement of a quantity x:• If the experiment were repeated and in each attempt a

confidence interval is calculated, then a fraction α of the confidence intervals will contain the true value of x (called µ). A fraction 1-α of the confidence intervals will not contain µ .

• Note: Experiments must not be identical

physics 736 experimental methods in nuclear-, particle...

Documents