1. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 1 Multivariate Analysis, TMVA, and Artificial Neural Networks Matt Jachowski jachowski@stanford.edu http://tmva.sourceforge.net/

2. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 2 Multivariate Analysis Techniques dedicated to analysis of data with multiple variables Active field – many recently developed techniques rely on computational ability of modern computers

3. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 3 Multivariate Analysis and HEP Goal is to classify events as signal or background Single event defined by several variables (energy, transverse momentum, etc.) Use all the variables to classify the event Multivariate analysis!

4. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 4 Multivariate Analysis and HEP Rectangular cuts optimization common

5. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 5 Multivariate Analysis and HEP Likelihood Estimator analysis also common Use of more complicated methods (Neural Networks, Boosted Decision Trees) not so common (though growing) – why? –Difficult to implement –Physicists are skeptical of new methods

6. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 6 Toolkit for Multivariate Analysis (TMVA) ROOT-integrated software package with several MVA techniques Automatic training, testing, and evaluation of MVA methods Guidelines and documentation to describe methods for users – this isn’t a black box!

7. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 7 Toolkit for Multivariate Analysis (TMVA) Easy to configure methods Easy to “plug-in” HEP data Easy to compare different MVA methods

8. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 8 TMVA in Action

9. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 9 TMVA and Me TMVA started in October 2005 –Still young –Very active group of developers My involvement –Decorrelation for Cuts Method (mini project) –New Artificial Neural Network implementation (main project)

10. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 10 Decorrelated Cuts Method Some MVA methods suffer if data has linear correlations –i.e. Likelihood Estimator, Cuts Linear correlations can be easily transformed away I implemented this for the Cuts Method

11. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 11 Decorrelated Cuts Method Find the square root of the covariance matrix ( C=C’C’) Decorrelate the data Apply cuts to decorrelated data

12. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 12 Artificial Neural Networks (ANNs) Robust non-linear MVA technique

13. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 13

14. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 14 Training an ANN Challenge is training the network Like human brain, network learns from seeing data over and over again Technical details: Ask me if you’re really interested

15. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 15 MLP MLP (Multi-Layer Perceptron) – my ANN implementation for TMVA MLP is TMVA’s main ANN MLP serves as base for any future ANN developments in TMVA

16. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 16 MLP – Information & Statistics Implemented in C++ Object-Oriented 4,000+ lines of code 16 classes

17. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 17 Acknowledgements Joerg Stelzer Andreas Hoecker CERN University of Michigan Ford NSF

18. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 18 Questions? (I have lots of technical slides in reserve that I would be glad to talk about)

19. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 19

20. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 20 Synapses and Neurons...... w 0j w 1j w nj y0y0 y1y1 ynyn vjvj yjyj v0v0 v1v1 vnvn

21. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 21 Synapses and Neurons yjyj vjvj

22. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 22 Universal Approximation Theorem Every continuous function that maps intervals of real numbers to some output interval of real numbers can be approximated arbitrarily closely by a multi-layer perceptron with just one hidden layer (with non-linear activation functions). inputs weights between input and hidden layer bias non-linear activation function weights between hidden and output layer output

23. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 23 Training an MLP Training Event: Network: x0x0 x1x1 x2x2 x3x3 y

24. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 24 Training an MLP Adjust weights to minimize error (or an estimator that is some function of the error)

25. Michigan REU Final Presentations, August 10, 2006Matt Jachowski 25 Back-Propagation Algorithm Make correction in direction of steepest descent Corrections made to output layer first, propagated backwards