statistical methods used for higgs boson searches
TRANSCRIPT
![Page 1: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/1.jpg)
KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association
INSTITUTE OF EXPERIMENTAL PARTICLE PHYSICS (IEKP) – PHYSICS FACULTY
www.kit.edu
Statistical Methods used for Higgs Boson Searches
Roger Wolf03. June 2014
![Page 2: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/2.jpg)
Institute of Experimental Particle Physics (IEKP)2
Recap from Last Time (Simulation of Processes)
● From “paper & pen” statements to high precision predictions on observable quantities (at the LHC):
● Discussed in lectures 1-3.
![Page 3: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/3.jpg)
Institute of Experimental Particle Physics (IEKP)3
Recap from Last Time (Data Analysis)
● Observable → real measurement:
![Page 4: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/4.jpg)
Institute of Experimental Particle Physics (IEKP)4
Recap from Last Time (Data Analysis)
● Observable → real measurement:
Data preparation techniques:
● Calibration of energy response.
● Alignment of track detectors.
● Reconstruction of traces in the detector units.
● Reconstruction & selection efficiency (“Tag & probe”, “MC Embedding”)
● How well are background processes understood?
![Page 5: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/5.jpg)
Institute of Experimental Particle Physics (IEKP)5
of Today
● Observable → real measurement:
Data preparation techniques:
● Calibration of energy response.
● Alignment of track detectors.
● Reconstruction of traces in the detector units.
● Reconstruction & selection efficiency (“Tag & probe”, “MC Embedding”)
● How well are background processes understood?
How to establish a new (small) signal on top
of a “reasonably” well known background?
![Page 6: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/6.jpg)
Institute of Experimental Particle Physics (IEKP)6
Quiz of the Day
● What is the relation between the Binomial, Gaussian & Poisson distribution?
● What is the relation between a minimal fit and a Maximum Likelihood fit?
● How exactly do I calculate a 95% CL limit and how does it relate to classical hypothesis tests?
![Page 7: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/7.jpg)
Institute of Experimental Particle Physics (IEKP)7
Quiz of the Day
● What is the relation between the Binomial, Gaussian & Poisson distribution?
● What is the relation between a minimal fit and a Maximum Likelihood fit?
● How exactly do I calculate a 95% CL limit and how does it relate to classical hypothesis tests? Can you interpret this plot?
![Page 8: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/8.jpg)
Institute of Experimental Particle Physics (IEKP)8
Quiz of the Day
● What is the relation between the Binomial, Gaussian & Poisson distribution?
● What does a “ evidence” or a “ discovery” mean?
● What is the relation between a minimal fit and a Maximum Likelihood fit?
● How exactly do I calculate a 95% CL limit and how does it relate to classical hypothesis tests? Can you interpret this plot?
![Page 9: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/9.jpg)
Institute of Experimental Particle Physics (IEKP)9
Schedule for Today
Probability distributions & Likelihood functions.
Parameter estimates (=fits).
Limits, p-values, significances.
1
2
3
![Page 10: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/10.jpg)
Institute of Experimental Particle Physics (IEKP)10
Schedule for Today
Probability distributions & Likelihood functions.
Parameter estimates (=fits).
Limits, p-values, significances.
1
2
3Walk through statistical methods that will appear in the next lectures:● You will see all these methods acting in
real life during the next lectures.
● To learn about the interiors of these methods check KIT lectures of Modern Data Analysis Techniques.
![Page 11: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/11.jpg)
Institute of Experimental Particle Physics (IEKP)11
Statistics ↔ Particle Physics
Theory:● QM wave functions are interpreted
as probability density functions.
● The Matrix Element, ,gives the probability to find final state f for given initial state i.
● Each of the statistical processes pdf → ME → hadronization → energy loss in material → digitization are statistically independent.
● Event by event simulation using Monte Carlo integration methods.
![Page 12: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/12.jpg)
Institute of Experimental Particle Physics (IEKP)12
Statistics ↔ Particle Physics
Theory: Experiment:● QM wave functions are interpreted
as probability density functions.
● All measurements we do are derived from rate measurements.
● We record millions of trillions of particle collisions.
● Each of these collisions is independent from all the others.
● The Matrix Element, ,gives the probability to find final state f for given initial state i.
● Each of the statistical processes pdf → ME → hadronization → energy loss in material → digitization are statistically independent.
● Event by event simulation using Monte Carlo integration methods.
![Page 13: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/13.jpg)
Institute of Experimental Particle Physics (IEKP)13
Statistics ↔ Particle Physics
● Particle physics experiments are a perfect application for statistical methods.
Theory: Experiment:● QM wave functions are interpreted
as probability density functions.
● All measurements we do are derived from rate measurements.
● We record millions of trillions of particle collisions.
● Each of these collisions is independent from all the others.
● The Matrix Element, ,gives the probability to find final state f for given initial state i.
● Each of the statistical processes pdf → ME → hadronization → energy loss in material → digitization are statistically independent.
● Event by event simulation using Monte Carlo integration methods.
![Page 14: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/14.jpg)
Institute of Experimental Particle Physics (IEKP)14
Probability Distributions & Likelihood Functions
![Page 15: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/15.jpg)
Institute of Experimental Particle Physics (IEKP)15
Characterization of Probability Distributions
● Expectation Value:
● Variance:
● Covariance:
● Correlation coefficient:
![Page 16: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/16.jpg)
Institute of Experimental Particle Physics (IEKP)16
Probability Distributions
(Binomial distribution)
Expectation: Variance:
![Page 17: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/17.jpg)
Institute of Experimental Particle Physics (IEKP)17
Probability Distributions
Central limit theorem of de Moivre & Laplace.
(Binomial distribution)
(Gaussian distribution)
Expectation: Variance:
![Page 18: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/18.jpg)
Institute of Experimental Particle Physics (IEKP)18
Probability Distributions
Central limit theorem of de Moivre & Laplace.
(Binomial distribution)
(Gaussian distribution)
(Poisson distribution)
Will be shown on next slide.
Expectation: Variance:
![Page 19: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/19.jpg)
Institute of Experimental Particle Physics (IEKP)19
Probability Distributions
Central limit theorem of de Moivre & Laplace.
(Binomial distribution)
(Gaussian distribution)
(Poisson distribution)
Will be shown on next slide.
Expectation: Variance:
motivation for uncertainty.
![Page 20: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/20.jpg)
Institute of Experimental Particle Physics (IEKP)20
Binomial ↔ Poisson Distribution
![Page 21: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/21.jpg)
Institute of Experimental Particle Physics (IEKP)21
Uncertainties on Counting Experiments
counting experiment
uncertainty
![Page 22: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/22.jpg)
Institute of Experimental Particle Physics (IEKP)22
Uncertainties on Counting Experiments
Binned Histogram
counting experiment
uncertainty
Number of events in depends on and on probability .
underlying
![Page 23: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/23.jpg)
Institute of Experimental Particle Physics (IEKP)23
Relations between Probability Distributions
Binomial
Gaussian
Poisson
Look for something that is very rare very often.
Random variable variable made up of a sum of many single measurements.
Central Limit Theorem:
![Page 24: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/24.jpg)
Institute of Experimental Particle Physics (IEKP)24
Relations between Probability Distributions
Binomial
Gaussian
Poisson
Log-normal
Look for something that is very rare very often.
Random variable variable made up of a sum of many single measurements.
Random variable variable made up of a product of many single measurements.
exp
Central Limit Theorem:
![Page 25: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/25.jpg)
Institute of Experimental Particle Physics (IEKP)25
Relations between Probability Distributions
Binomial
Gaussian
Poisson
Log-normal Distribution
Look for something that is very rare very often.
Random variable variable made up of a sum of many single measurements.
Random variable variable made up of a product of many single measurements.
logexp
What does the parameter k correspond to in the distributions?
Central Limit Theorem:
![Page 26: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/26.jpg)
Institute of Experimental Particle Physics (IEKP)26
Relations between Probability Distributions
Binomial
Gaussian
Poisson
Log-normal Distribution
Look for something that is very rare very often.
Random variable variable made up of a sum of many single measurements.
Random variable variable made up of a product of many single measurements.
logexp
k=ndof=dim of Gaussian (for more details wait till slides 32ff).
What does the parameter k correspond to in the distributions?
Central Limit Theorem:
![Page 27: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/27.jpg)
Institute of Experimental Particle Physics (IEKP)27
Likelihood Functions
● Problem: truth is not known!
● Deduce “truth” from measurements (usually in terms of models).
● Likeliness of a model to be true quantified by likelihood function .
model parameters.
measured number of events (e.g. in bins i).
![Page 28: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/28.jpg)
Institute of Experimental Particle Physics (IEKP)28
Likelihood Functions
● Problem: truth is not known!
● Deduce “truth” from measurements (usually in terms of models).
● Likeliness of a model to be true quantified by likelihood function .
● Example:signal on top of known background in a bin-ned histogram:
Product of pdfs for each bin (Poisson).
background signal
model parameters.
measured number of events (e.g. in bins i).
![Page 29: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/29.jpg)
Institute of Experimental Particle Physics (IEKP)29
Parameter Estimates
![Page 30: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/30.jpg)
Institute of Experimental Particle Physics (IEKP)30
Parameter Estimates
● Problem: find most probable parameter(s) of a given model.
● Usually minimization of negative ln likelihood function (NLL):● ln is a monotonic function and very often numerically easier to handle.● e.g. products of probability distributions turn into sums.
● e.g. if probability distributions are Gaussians NLL turns into minimization:
![Page 31: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/31.jpg)
Institute of Experimental Particle Physics (IEKP)31
Parameter Estimates
● Problem: find most probable parameter(s) of a given model.
● Usually minimization of negative ln likelihood function (NLL):● ln is a monotonic function and very often numerically easier to handle.● e.g. products of probability distributions turn into sums.
● e.g. if probability distributions are Gaussians NLL turns into minimization:
Clear to everybody?
![Page 32: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/32.jpg)
Institute of Experimental Particle Physics (IEKP)32
Parameter Estimates
● Problem: find most probable parameter(s) of a given model.
● Usually minimization of negative ln likelihood function (NLL):● ln is a monotonic function and very often numerically easier to handle.● e.g. products of probability distributions turn into sums.
● e.g. if probability distributions are Gaussians NLL turns into minimization:
Clear to everybody?
Number of 'i determines dimension of the Gaussian distribution.
![Page 33: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/33.jpg)
Institute of Experimental Particle Physics (IEKP)33
Parameter Estimates
● Problem: find most probable parameter(s) of a given model.
● Usually minimization of negative ln likelihood function (NLL):● ln is a monotonic function and very often numerically easier to handle.● e.g. products of probability distributions turn into sums.
● e.g. if probability distributions are Gaussians NLL turns into minimization:
● The minimization usually performed:
● analytically (like in an optimization exercise in school).
● numerically (usually the more general solution).
● by scan of the NLL (for sure the most robust method).
Clear to everybody?
Number of 'i determines dimension of the Gaussian distribution.
![Page 34: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/34.jpg)
Institute of Experimental Particle Physics (IEKP)34
Parameter(s) of Interest (POI)
● Each case/problem defines its own parameter(s) of interest (POI's):
● POI could be the mass .
● Example:signal on top of known background in a bin-ned histogram:
Product of pdfs for each bin (Poisson).
background signal
![Page 35: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/35.jpg)
Institute of Experimental Particle Physics (IEKP)35
Parameter(s) of Interest (POI)
● Each case/problem defines its own parameter(s) of interest (POI's):
● POI could be the mass .
● Example:signal on top of known background in a bin-ned histogram:
Product of pdfs for each bin (Poisson).
● In our case POI usually is the signal strength for a fixed value for .
background signal
![Page 36: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/36.jpg)
Institute of Experimental Particle Physics (IEKP)36
Systematic Uncertainties
● Systematic uncertainties are usually incorporated as nuisance parameters:
● Example:signal on top of known background in a bin-ned histogram:
Product of pdfs for each bin (Poisson).
● Example: assume background normalization is not absolutely known, but with an uncertainty :
background signal
uncertainty
expected value
possible values in single measurements
![Page 37: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/37.jpg)
Institute of Experimental Particle Physics (IEKP)37
Hypothesis Tests
![Page 38: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/38.jpg)
Institute of Experimental Particle Physics (IEKP)38
Hypothesis Separation
● Start with two alternative hypotheses & .
● Define a test statistic that can distinguish these two hypotheses.
● The test statistic with the best separation power is the likelihood ratio (LR):
● can be calculated for the observation (obs), for the expectation for and for the expectation for :
pdf from toys based on (usually sig).
pdf from toys based on (usually BG).
toys
obs
● Observed is a single value (outcome of measurement).
● Expectation is a mean value with uncertainties based on toy measurements.
![Page 39: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/39.jpg)
Institute of Experimental Particle Physics (IEKP)39
Hypothesis Separation
● Define a test statistic that can distinguish these two hypotheses.
● The test statistic with the best separation power is the likelihood ratio (LR).
● can be calculated for the observation (obs), for the expectation for and for the expectation for :
pdf from toys based on (usually sig).
pdf from toys based on (usually BG).
toys
obs
● Observed is a single value (outcome of measurement).
● Expectation is a mean value with uncertainties based on toy measurements.
Sorry! No price...
Signal on topof background!
● Start with two alternative hypotheses & .
![Page 40: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/40.jpg)
Institute of Experimental Particle Physics (IEKP)40
Test Statistics (LEP)
nuisance parameters integrated out (by throwing toys → MC method) before evaluation of (→marginalization).
● Start with two alternative hypotheses & .
● Define a test statistic that can distinguish these two hypotheses.
● The test statistic with the best separation power is the likelihood ratio (LR):
![Page 41: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/41.jpg)
Institute of Experimental Particle Physics (IEKP)41
Test Statistics (Tevatron)
nominator maximized for given before marginalization. Denominator for . Better estimates on nuisance parameters. Reduces uncertainties on nuisance parameters.
● Start with two alternative hypotheses & .
● Define a test statistic that can distinguish these two hypotheses.
● The test statistic with the best separation power is the likelihood ratio (LR):
![Page 42: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/42.jpg)
Institute of Experimental Particle Physics (IEKP)42
Test Statistics (LHC)
nominator maximized for given before marginalization. For the denominator a global maximum is searched for at . In addition allows use of asymptotic formulas (→ no need for toys).
● Start with two alternative hypotheses & .
● Define a test statistic that can distinguish these two hypotheses.
● The test statistic with the best separation power is the likelihood ratio (LR):
![Page 43: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/43.jpg)
Institute of Experimental Particle Physics (IEKP)43
Classical Hypothesis Testing
● Classical hypothesis test interested in probability to observe given that or is true:
● We are usually interested in “upper limits”, which corresp. to “lower bounds” (→ how often
signal ≤ observed deviation?).
toys
upper bound lower bounddefines defines
![Page 44: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/44.jpg)
Institute of Experimental Particle Physics (IEKP)44
95% CL Upper Limits
● Our pdf's usually depend on another parameter, which is the actual POI ( in SM, in MSSM case).
● Traditionally we set 95% CL upper limits on this POI.
toys
● pdf's move apart from each other.
● The more separate the pdf's are the more & are distinguishable.
● Find for which:
for this in 95% of all toys .
interested in & blue pdf from below.
![Page 45: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/45.jpg)
Institute of Experimental Particle Physics (IEKP)45
95% CL Upper Limits
● Our pdf's usually depend on another parameter, which is the actual POI ( in SM, in MSSM case).
● Traditionally we set 95% CL upper limits on this POI.
toys
● pdf's move apart from each other.
● The more separate the pdf's are the more & are distinguishable.
● Find for which:
for this in 95% of all toys .
● is the value at which in case that is the true hypothesis the chance that is 95%.
● Still there is a chance of 5% that .
95% CL Upper Limit:
interested in & blue pdf from below.
![Page 46: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/46.jpg)
Institute of Experimental Particle Physics (IEKP)46
95% CL Upper Limits
● Our pdf's usually depend on another parameter, which is the actual POI ( in SM, in MSSM case).
● Traditionally we set 95% CL upper limits on this POI.
toys
interested in integration of blue pdf.
● pdf's move apart from each other.
● The more separate the pdf's are the more & are distinguishable.
● Find for which:
for this in 95% of all toys .
● is the value at which in case that is the true hypothesis the chance that is 95%.
● Still there is a chance of 5% that .
95% CL Upper Limit:
● Assume our POI is : does the 90% CL upper limit on correspond to a higher or a lower value ?
![Page 47: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/47.jpg)
Institute of Experimental Particle Physics (IEKP)47
95% CL Upper Limits
● Our pdf's usually depend on another parameter, which is the actual POI ( in SM, in MSSM case).
● Traditionally we set 95% CL upper limits on this POI.
toys
interested in integration of blue pdf.
● pdf's move apart from each other.
● The more separate the pdf's are the more & are distinguishable.
● Find for which:
for this in 95% of all toys .
● is the value at which in case that is the true hypothesis the chance that is 95%.
● Still there is a chance of 5% that .
95% CL Upper Limit:
● Assume our POI is : does the 90% CL upper limit on correspond to a higher or a lower value ? It's lower!
1%probability of to be “more background like” than .
10%
![Page 48: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/48.jpg)
Institute of Experimental Particle Physics (IEKP)48
CLs Limits
● In particle physics we set more conservative limits than this, following the CLs method:
toys
● Find for which:
● Assume to be signal+background and to be background only hypothesis.
interested in integration of magenta pdf & blue pdf from below.
![Page 49: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/49.jpg)
Institute of Experimental Particle Physics (IEKP)49
CLs Limits
● In particle physics we set more conservative limits than this, following the CLs method:
toys
● Find for which:
● If & are clearly distinguishable .
● Assume to be signal+background and to be background only hypothesis.
interested in integration of magenta pdf & blue pdf from below.
![Page 50: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/50.jpg)
Institute of Experimental Particle Physics (IEKP)50
CLs Limits
● In particle physics we set more conservative limits than this, following the CLs method:
toys
● Find for which:
● If & are clearly distinguishable .
● If they cannot be distinguished .
● Assume to be signal+background and to be background only hypothesis.
interested in integration of magenta pdf & blue pdf from below.
![Page 51: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/51.jpg)
Institute of Experimental Particle Physics (IEKP)51
CLs Limits (more schematic)to
ys
PO
Iinterested in integration of magenta pdf & blue pdf from below.
● Assume to be signal+background and to be background only hypothesis.
● In particle physics we set more conservative limits than this, following the CLs method:
![Page 52: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/52.jpg)
Institute of Experimental Particle Physics (IEKP)52
Expected Limit (canonical approach)
● To obtain the expected limit mimic calculation of observed, but base it on toy experiments.
● Make use of the fact that the pdf's do not depend on toys (i.e. schematic plot on the left does not change).
PO
I
● Throw number of toys under the BG only hypothesis ( ) determine distribution of 95% CL limits on POI.
POI
toys
0.02
5
0.16
0
0.50
0
0.84
0
0.97
5
● Obtain quantiles for expected limit from this distribution.
![Page 53: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/53.jpg)
Institute of Experimental Particle Physics (IEKP)53
And if the signal shows up...
![Page 54: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/54.jpg)
Institute of Experimental Particle Physics (IEKP)54
p-Value
● How do we know whether what we see is not just a background fluctuation?
● The p-value is the probability to observe values of larger than under the assumption that the background only hypothesis is the true hypothesis.
● Think of...
… the limit as a way to falsify the signal plus background hypothesis ( ).
… the p-value as a way to falsify the background only hypothesis ( ).
![Page 55: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/55.jpg)
Institute of Experimental Particle Physics (IEKP)55
Significance
● If the measurement is normal distributed is distributed according to a distribution.
● The probability can then be interpreted as a Gaussian confidence interval.
p-values:
![Page 56: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/56.jpg)
Institute of Experimental Particle Physics (IEKP)56
Significance (in practice)
● If the measurement is normal distributed is distributed according to a distribution.
● The probability can then be interpreted as a Gaussian confidence interval.
● Usual approximation in practice is to estimate significances by:
![Page 57: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/57.jpg)
Institute of Experimental Particle Physics (IEKP)57
Significance (in practice)
● If the measurement is normal distributed is distributed according to a distribution.
● The probability can then be interpreted as a Gaussian confidence interval.
● Usual approximation in practice is to estimate significances by:
expected signal events
![Page 58: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/58.jpg)
Institute of Experimental Particle Physics (IEKP)58
Significance (in practice)
● If the measurement is normal distributed is distributed according to a distribution.
● The probability can then be interpreted as a Gaussian confidence interval.
● Usual approximation in practice is to estimate significances by:
Poisson uncertainty on expected background events.
expected signal events
![Page 59: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/59.jpg)
Institute of Experimental Particle Physics (IEKP)59
Significance (in practice)
● If the measurement is normal distributed is distributed according to a distribution.
● The probability can then be interpreted as a Gaussian confidence interval.
● Usual approximation in practice is to estimate significances by:
Poisson uncertainty on expected background events.
expected signal events
![Page 60: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/60.jpg)
Institute of Experimental Particle Physics (IEKP)60
Concluding Remarks
● Reviewed all statistical tools necessary to search for the Higgs signal (→ as a small signal above a known background):
● In particle physics we call an observation with an evidence.
● We call an observation with a discovery.
● Probability distributions, likelihood functions, limits, p-values, ...
● Limits are a usual way to 'exclude' the signal hypothesis ( ).
● p-values are a usual way to 'exclude' the background hypothesis ( ).
● Under the assumption that the test statistic is distributed p-values can be translated into Gaussian confidence intervals .
![Page 61: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/61.jpg)
Institute of Experimental Particle Physics (IEKP)61
Concluding Remarks
● Reviewed all statistical tools necessary to search for the Higgs signal (→ as a small signal above a known background):
● In particle physics we call an observation with an evidence.
● We call an observation with a discovery.
● Probability distributions, likelihood functions, limits, p-values, ...
● Limits are a usual way to 'exclude' the signal hypothesis ( ).
● p-values are a usual way to 'exclude' the background hypothesis ( ).
● Under the assumption that the test statistic is distributed p-values can be translated into Gaussian confidence intervals .
● Once a measurement is established the search is over! Measurements of properties are new and different world!
![Page 62: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/62.jpg)
Institute of Experimental Particle Physics (IEKP)62
Sneak Preview for Next Week
● Review indirect estimates of the Higgs mass and searches for the Higgs boson that have been made before 2012:
● Estimates of and from high precision measurements at the Z-pole mass at LEP.
● Direct searches for the Higgs boson at LEP.
● Direct searches for the Higgs boson at the Tevatron.
● For the remaining lectures we then will turn towards the discovery of the Higgs boson at the LHC.
During the next lectures we will see 1:1 life examples of all methods that have been presented here.
![Page 63: Statistical Methods used for Higgs Boson Searches](https://reader030.vdocuments.us/reader030/viewer/2022012020/616889c3d394e9041f705d4e/html5/thumbnails/63.jpg)
Institute of Experimental Particle Physics (IEKP)63
Backup & Homework Solutions