michigan reu final presentations, august 10, 2006matt jachowski 1 multivariate analysis, tmva, and...

Michigan REU Final Presentations, August 10, 2006 Matt Jachowski 1

Multivariate Analysis, TMVA, and Artificial Neural Networks

Matt Jachowski

[email protected]

http://tmva.sourceforge.net/


Multivariate Analysis

• Techniques dedicated to analysis of data with multiple variables

• Active field – many recently developed techniques rely on computational ability of modern computers


Multivariate Analysis and HEP

• Goal is to classify events as signal or background

• Single event defined by several variables (energy, transverse momentum, etc.)

• Use all the variables to classify the event

• Multivariate analysis!



• Rectangular cuts optimization common



• Likelihood Estimator analysis also common

• Use of more complicated methods (Neural Networks, Boosted Decision Trees) not so common (though growing) – why?– Difficult to implement– Physicists are skeptical of new methods


Toolkit for Multivariate Analysis (TMVA)

• ROOT-integrated software package with several MVA techniques

• Automatic training, testing, and evaluation of MVA methods

• Guidelines and documentation to describe methods for users – this isn’t a black box!


Toolkit for Multivariate Analysis (TMVA)

• Easy to configure methods

• Easy to “plug-in” HEP data

• Easy to compare different MVA methods


TMVA in Action


TMVA and Me

• TMVA started in October 2005– Still young– Very active group of developers

• My involvement– Decorrelation for Cuts Method (mini project)– New Artificial Neural Network implementation

(main project)


Decorrelated Cuts Method

• Some MVA methods suffer if data has linear correlations– i.e. Likelihood Estimator, Cuts

• Linear correlations can be easily transformed away

• I implemented this for the Cuts Method


Decorrelated Cuts Method

• Find the square root of the covariance matrix (C=C’C’)

• Decorrelate the data

• Apply cuts to decorrelated data

CSSD T TSDSC'

xC'x'


Artificial Neural Networks (ANNs)

• Robust non-linear MVA technique


Training an ANN

• Challenge is training the network

• Like human brain, network learns from seeing data over and over again

• Technical details:Ask me if you’re really interested


MLP

• MLP (Multi-Layer Perceptron) – my ANN implementation for TMVA

• MLP is TMVA’s main ANN

• MLP serves as base for any future ANN developments in TMVA


MLP – Information & Statistics

• Implemented in C++

• Object-Oriented

• 4,000+ lines of code

• 16 classes


Acknowledgements

• Joerg Stelzer

• Andreas Hoecker

• CERN

• University of Michigan

• Ford

• NSF


Questions?

(I have lots of technical slides in reserve that I would be glad to talk about)


Synapses and Neurons

.

.

.

w0j

w1j

wnj

y0

y1

yn

vj yj

v0

v1

vn

),...,,,...,( 00 njjnj wwyyfv

)( jj vy


Synapses and Neurons

jvjje

vy 1

1)(

i

n

iijnjjnj ywwwyyfv

0

00 ),...,,,...,(

yj

vj


Universal Approximation Theorem

Every continuous function that maps intervals of real numbers to some output interval of real numbers can be approximated arbitrarily closely by a multi-layer perceptron with just one hidden layer (with non-linear activation functions).

j

jj bwf )()( xvx j

inputsweights between input and hidden layer

biasnon-linear activation function

weights between hidden and output layer

output


Training an MLPTraining Event:

Network:

x0

x1

x2

x3

y

dxxxxf ),,,( 3210

yxxxxg ),,,( 3210

ydeerror


Training an MLPAdjust weights to minimize error (or an estimator that is some function of the error)

}_{

2 )(2

1)(

neuronsoutputjj nenε

)()()( nyndne jjj

N

navg n

N 1

)(1


Back-Propagation AlgorithmMake correction in direction of steepest descent

)()()1( nwnwnw ijijij

)(

)()(

nw

nnw

ij

ijij

Corrections made to output layer first, propagated backwards

michigan reu final presentations, august 10, 2006matt jachowski 1 multivariate analysis, tmva, and...

Documents