e/pi separation in caldet and nue identification in mdc

e/pi separation in CalDet and nue identification in MDC

T.J. Yang

Stanford University

e/pi separation

Thanks to great help from Patricia Vahle and helpful discussion with Adam Para

What do an electron and a pion look like in CalDet? P=1GeV/c

e

pi


e

pi

Information from CalDet data

EM shower is more compact than hadronic shower. And most of the EM showers have the same pattern regardless of energy: 4-5 planes long and 1-2 strip(s) wide.EM shower develops much faster than hadronic shower.Pions usually achieve the maximum energy lost later than electrons.

Fractions of energy deposited in 1,2,3,4,5,6 consecutive plane(s) with the highest ph

Fractions of energy deposited in 2,4,6,8,10,12 counters with the highest ph

Fraction of energy in a narrow road

Ratio of energy deposited in the first (second) plane to the second (third) plane

Shower_max

Position (with respect to the first plane) of the plane with the highest energy lost in the event.

Summary of discriminating variables (reference:NuMI-L-284)

Longitudinal features:Track lengthFractions of energy deposited in 1,2,3,4,5,6 consecutive plane(s) with the highest p.h.Ratio of energy deposited in plane 0 to plane 1Position of the maximum energy lost

Lateral features:Fraction of energy deposited in a narrow road

Others:Fractions of energy deposited in 2,4,6,8,10,12 strips with the highest p.h.

Run numbers used for CalDet e/pi separation analysis

P(GeV/c) Run Numbers

1.0 40622,40624,40626

2.0 40715,40717,40722,40724

3.0 40924,40495,50497,50456

4.0 50531,50553

5.0 50533,50535,50555,50557,50559

Results P(GeV/c) e pi/muon

1.0

before cuts 10613 59098

after cuts 2052 58

eff. & rej. 0.193 9.8e-4

2.0


after cuts 781 20

eff. & rej. 0.105 1.1e-4

3.0


after cuts 3376 0

eff. & rej. 0.121 0

Results (continued)

P(1GeV/c) e pi/muon

4.0


after cuts 2820 17

eff. & rej. 0.177 3.6e-4

5.0


after cuts 6977 76

eff. & rej. 0.224 5.9e-4

Hadron rejection ~1e-4 – 1e-3 for e~10-20%

nue identification

Files used for nue analysis

beam files: f24100001_0000.sntp.R1.9.root

-f24100020_0000.sntp.R1.9.root (all)

nue files: f24110001_0000.sntp.R1.9.root – f24110009_0000.sntp.R1.9.root (all)

nutau files: f24130001_0000.sntp.R1.9.root – f24130020_0000.sntp.R1.9.root (half)

Statistics and parameters

numu->nue: ~26,000 NC: ~33,000 CC: ~66,000

|Ue3|2 = 0.01

m2 = 2.5e-3eV2

assume a 2.5-yr run

pot/yr = 3.7e20

Strategy

Making cuts on the variables from SR: total p.e. , track range, shower range and total number of strips

Calculating the same variables as I used in the CalDet e/pi separation analysis

Using neural network and boosted decision trees to get the optimal results

Total pe (ph.pe)

Track range (trk.ds)

Shower range (shw.plane.n)

Total number of strips (nstrip)

Cuts on SR variables

200<ph.pe<700

trk.ds<0.9

5<shw.plane.n<14

14<nstrip<56

signal NC CC bnue Nutau FOM

all 28.6 1294 3371 68.4 33.9 0.41

survived 16.9 292 133 8.3 9.5 0.80

Calculating variablesFractions of energy deposited in 1,2,3,4,5,6 consecutive plane(s) with the highest phFractions of energy which are deposited in 2,4,6,8,10,12 counters with the highest phFraction of energy in a narrow road

Problems:Distributions don’t look quite different between signal and backgroundsAfter the preliminary cuts, the distributions look almost the same

Fraction of energy deposited in 3 consecutive planes

Fraction of energy deposited in 3 counters with the highest ph

Fraction of energy in a narrow road

The use of neural network

Input variables: fract_1_cons, fract_2_cons, fract_3_cons, fract_4_cons, fract_5_cons, fract_6_cons, fract_1_count, fract_2_count, fract_3_count, fract_4_count, fract_6_count, fract_8_count, fract_ct, fract_ct_2, rms1u, rms1v

Structure: 17:14:1

Tried two different packages: Jetnet and TMultiLayerPerceptron(MLP) in ROOT

Jetnet (training)


8.8 58.4 19.4 4.2 3.0 0.95

Jetnet (testing)


8.9 59.9 20.9 5.8 3.4 0.94

MLP (training)


7.7 47.7 15.3 3.4 2.7 0.94

MLP (testing)


7.8 45.5 14.5 5.0 2.8 0.95

Boosted Decision Trees

Jake Klamka

University of Toronto

Boosted Decision TreesIn the past several years there has been a “revolution in the field of machine learning inspired by the extension of decision trees by boosting.” (J. Friedman -- PHYSTAT2003)BOOSTING: A procedure that combines the outputs of many “weak” classifiers to produce a powerful “committee”.Multiple decision trees are created using weighted versions of the training dataset. Final event classification is based on a linear combination of individual decision trees.

Rough Outline of a Boosting Algorithm:

1. Weight all events in the data sample equally.

2. ** LOOP **:a) Train new decision tree with current event weig

hts.

b) Re-weight events, giving higher weight to events that were misclassified in (a).

3. Take the linear combination of all decision trees with most accurate decision trees given more weight.

AdaBoost.M1 (Freund and Schapire 1996)

Software: See5 (C5)

Commercial decision tree software created by Ross Quinlan.New version allows for boosting. Previous version known as C4.5.Pros: 10 days free trial, easy to install and setup, very fast classification.Cons: Exact boosting algorithm used by See5 is unknown (proprietary), not easily customizable.

Results with decision trees(no boosting)

no boosting, cost 1:1

no boosting, cost 1:2


Training 9.3 57.1 21.2 3.9 2.7 1.00

Testing 9.0 67.7 25.1 5.6 3.3 0.89


Training 4.7 13.6 3.9 2.0 0.8 1.04

Testing 4.5 15.1 5.0 2.9 1.6 0.89

Results with boosted decision trees

boosting: 3 trials, cost 1:1

boosting: 3 trials, cost 1:2


Training 9.0 56.4 19.3 3.8 2.7 1.00

Testing 9.0 64.3 22.5 5.3 3.2 0.92


Training 4.3 10.8 2.7 1.9 0.9 1.06

Testing 4.3 13.6 3.2 3.0 1.2 0.94

BOOSTING in NEUTRINO PHYSICS:The MiniBooNE collaboration is using boosted decision trees and has announced 20% to 80% improvements over their best results with neural networks. Their analysis is described in a recent preprint and their code is available online. (physics/0408124)

FUTURE PROSPECTS:Customized boosted decision tree code (use MiniBooNE code as prototype?)Try different boosting algorithms.Boosting algorithm can be applied to any classification technique, not just decision trees… “Boosted Neural Networks” may offer even better results.

More information and links:http://home.fnal.gov/~jklamka/

Summary


all 28.6 1294 3371 68.4 33.9 0.41

pre-sel 16.9 292 133 8.3 9.5 0.80

Jetnet 8.9 59.9 20.9 5.8 3.4 0.94

MLP 7.8 45.5 14.5 5.0 2.8 0.95

BDT 4.3 13.6 3.2 3.0 1.2 0.94

714 8.5 27.2 3.9 5.6 3.0 1.35

Ely 9.5 68.4 9.2 10.3 4.4 0.99

Conclusion

For the CalDet e/pi separation study, we got the hadron rejection ~1e-4 – 1e-3 for e~10-20%

For the nue identification in MDC, we got FOM ~0.94

We will try to understand several issues and modify the variables to improve the FOM.

e/pi separation in caldet and nue identification in mdc

Documents