Gaudi Framework Tutorial, Interfacing Gaudi with Python

8-2 Gaudi Framework Tutorial, 2004 Objectives After completing this lesson, you should be able to: Understand how Gaudi is interfaced to Python Convince you that Python is a very good programming language for Rapid Prototyping Write simple Python scripts to access LHCb data After completing this lesson, you should be able to: Understand how Gaudi is interfaced to Python Convince you that Python is a very good programming language for Rapid Prototyping Write simple Python scripts to access LHCb data

8-3 Gaudi Framework Tutorial, 2004 Python Language Python is an Open Source programming language created by Guido van Rossum. While it has been available since 1990, it has recently seen a dramatic increase in popularity in a wide variety of domains Python is simple, elegant, powerful and reliable Features: Variables & Arithmetic expressions String manipulations Conditionals (if/else statements),Loops (for/while) Functions, Lists, Dictionaries, Classes (Objects), Exceptions, Modules Python is an Open Source programming language created by Guido van Rossum. While it has been available since 1990, it has recently seen a dramatic increase in popularity in a wide variety of domains Python is simple, elegant, powerful and reliable Features: Variables & Arithmetic expressions String manipulations Conditionals (if/else statements),Loops (for/while) Functions, Lists, Dictionaries, Classes (Objects), Exceptions, Modules

8-4 Gaudi Framework Tutorial, 2004 Why Python? Free, Open-Source, Maintainable Cross-Platform, Portable General-Purpose, High-level, Object-Oriented Interpreted However it is quite fast (byte code idea from Java) Dynamically typed, Introspective Not need to declare any variable Simple syntax, Robust Emphasis by the author to minimize typing Variety of Shells Powerful built-in types and modules  Ideal for Scripting and Prototyping Free, Open-Source, Maintainable Cross-Platform, Portable General-Purpose, High-level, Object-Oriented Interpreted However it is quite fast (byte code idea from Java) Dynamically typed, Introspective Not need to declare any variable Simple syntax, Robust Emphasis by the author to minimize typing Variety of Shells Powerful built-in types and modules  Ideal for Scripting and Prototyping

8-5 Gaudi Framework Tutorial, 2004 Python as a “Glue” Extension Modules Very easy to interface to C++ classes (C-API)Collaborative Extension modules can be plugged and used together Extension Modules Very easy to interface to C++ classes (C-API)Collaborative Extension modules can be plugged and used together GUI Python math shell gaudipython DatabaseEDG API GUI Very rich set of Python standard modules Several GUI toolkits XML Very rich set specialized generic modules Gaudi Framework PyROOT Root Classes PVSS JPE Java Classes LHC modules Gateways to other frameworks

8-6 Gaudi Framework Tutorial, 2004 Python Binding Techniques Python provides a very complete C-API By hand bindings are possible but even for medium-sized projects, it becomes cumbersome and unmaintainable. Developing Python bindings Several existing options: SWIG, Boost.Python, SIP PyLCGDict: Python binding to the LCG Dictionary It allows the user to interact with any C++ class for which the LCG Dictionary has been generated With this module, there is no need to generate specialized Python bindings or wrapper code to interact with C++ classes Python provides a very complete C-API By hand bindings are possible but even for medium-sized projects, it becomes cumbersome and unmaintainable. Developing Python bindings Several existing options: SWIG, Boost.Python, SIP PyLCGDict: Python binding to the LCG Dictionary It allows the user to interact with any C++ class for which the LCG Dictionary has been generated With this module, there is no need to generate specialized Python bindings or wrapper code to interact with C++ classes

8-7 Gaudi Framework Tutorial, 2004 PyLCGDict: Mode d’emploi From class definitions (.h files) a “dictionary” library is produced Description of the class “stub” functions to class methods Absolutely non-intrusive The PyLCGDict module does the adaptation between Python objects and C++ objects in a generic way It works for any dictionary From class definitions (.h files) a “dictionary” library is produced Description of the class “stub” functions to class methods Absolutely non-intrusive The PyLCGDict module does the adaptation between Python objects and C++ objects in a generic way It works for any dictionary MyClass.h MyDict.so select.xml lcgdict Python interpreter load MyClass.so MyScript.py PyLCGDict

8-8 Gaudi Framework Tutorial, 2004 PyLCGDict: Supported Features Conversion of C++ and Python primitive types C++ classes Mapped to Python classes and loaded on demand. Templated classes supported. C++ namespaces Mapped to python scopes. The "::" separator is replaced by the python "." separator Class methods Static and non static class methods are supported. Default arguments. Method arguments are passed by value or by reference The return values are converted into Python types and new Python classes are created if required. Dynamic type returned if possible. Method overloading works by dispatching sequentially to the available methods with the same name until a match with the provided arguments is successful. Conversion of C++ and Python primitive types C++ classes Mapped to Python classes and loaded on demand. Templated classes supported. C++ namespaces Mapped to python scopes. The "::" separator is replaced by the python "." separator Class methods Static and non static class methods are supported. Default arguments. Method arguments are passed by value or by reference The return values are converted into Python types and new Python classes are created if required. Dynamic type returned if possible. Method overloading works by dispatching sequentially to the available methods with the same name until a match with the provided arguments is successful.

8-9 Gaudi Framework Tutorial, 2004 PyLCGDict: Supported Features Class data members Public data members are accessible as Python properties Emulation of Python containers Container C++ classes (std::vector, std::list, std::map like) are given the behavior of the Python collections to be use in iterations and slicing operations. Operator overloading Standard C++ operators are mapped to the corresponding Python overloading operators Class data members Public data members are accessible as Python properties Emulation of Python containers Container C++ classes (std::vector, std::list, std::map like) are given the behavior of the Python collections to be use in iterations and slicing operations. Operator overloading Standard C++ operators are mapped to the corresponding Python overloading operators

8-10 Gaudi Framework Tutorial, 2004 PyLCGDict: Example (1) #include namespace Example { class MyClass { public: MyClass() : m_value(0) {} MyClass(const MyClass& m ) : m_value(m.m_value) {} ~MyClass() {} int doSomething(const std::string& something ) { std::cout << “I am doing something with “ << something << std::endl; return something.size(); } int value() { return m_value; } void setValue(int v) { m_value = v; } private: int m_value; public: float fprop; std::string sprop; }; } MyClass.h

8-11 Gaudi Framework Tutorial, 2004 PyLCGDict: Example (2) > python Python (#1, Feb , 12:11:31) [GCC 3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import PyLCGDict >>> PyLCGDict.loadDict('MyClassDict') >>> m = Example.MyClass() >>> m.doSomething('care') I am doing something with care 4 >>> m.setValue(99) >>> print m.value() 99 >>> n = Example.MyClass(m) # copy constructor >>> print n.value() 99 >>> n.fprop = 1.2 >>> n.sprop = 'String property' >>> print n.sprop, n.fprop String property 1.2

8-12 Gaudi Framework Tutorial, 2004 Another Example: ROOT >>> from ROOT import gRandom, TCanvas, TH1F >>> c1 = TCanvas('c1','Example',200,10,700,500) >>> hpx = TH1F('hpx','px',100,-4,4) >>> for i in xrange(25000):... px = gRandom.Gaus()... hpx.Fill(px)... >>> hpx.Draw() >>> c1.Update() The ROOT.py loads the ROOTDict.so and initializes the ROOT system

8-13 Gaudi Framework Tutorial, 2004 GaudiPython Enabling the interaction of Gaudi components from Python interpreted Configuration, Interactivity, etc. Starting from Gaudi v14r1, GaudiPython has been re-implemented using PyLCGDict Generated dictionaries for most common Gaudi “Interfaces” and “Base classes” (~80 classes) Not need to generate dictionaries for all classes (in particular the implementations) Enabling the interaction of Gaudi components from Python interpreted Configuration, Interactivity, etc. Starting from Gaudi v14r1, GaudiPython has been re-implemented using PyLCGDict Generated dictionaries for most common Gaudi “Interfaces” and “Base classes” (~80 classes) Not need to generate dictionaries for all classes (in particular the implementations)

8-14 Gaudi Framework Tutorial, 2004 GaudiPython The end-user module “gaudimodule.py” hides some of the technicalities and adds some handy functionality Very easy to extern/modify/adapt since is written in Python Basically backward compatible with previous version The end-user module “gaudimodule.py” hides some of the technicalities and adds some handy functionality Very easy to extern/modify/adapt since is written in Python Basically backward compatible with previous version

8-15 Gaudi Framework Tutorial, 2004 Using GaudiPython Add in the requirements file: use GaudiPython v*

8-16 Gaudi Framework Tutorial, 2004 Starting Methods Starting a Gaudi application from Python Shell >>> import gaudimodule >>> gaudi = gaudimodule.AppMgr() >>> gaudi.run(100)... >>> gaudi.exit() Starting a Python interpreted from a Gaudi application ApplicationMgr.DLLs += { "GaudiPython“ }; ApplicationMgr.Runable = "PythonScriptingSvc"; PythonScriptingSvc.StartupScript = "../options/AnalysisTest.py"; Optional initial script

8-17 Gaudi Framework Tutorial, 2004 Configuration Instead of using “JobOptions” files we could use Python to completely configure Gaudi applications ATLAS is going in this direction “Properties” are mapped to native Python types (int, float, str, list, etc) Instead of using “JobOptions” files we could use Python to completely configure Gaudi applications ATLAS is going in this direction “Properties” are mapped to native Python types (int, float, str, list, etc) >>> gaudi = gaudimodule.AppMgr() >>> gaudi.EvtMax = 100 >>> gaudi.TopAlg = [‘Alg1’, ‘Alg2’] >>> alg1 = gaudi.algorithm(‘Alg1’) >>> alg1.EnergyCut = 10.7 >>> msgsvc = gaudi.service('MessageSvc') >>> msgsvc.OutputLevel = 3...

8-18 Gaudi Framework Tutorial, 2004 Interacting with the Event Since we have the complete set of Dictionaries we can interact with any Event class References to Event objects are obtained by asking the Event data service ( AppMgr.evtSvc() ) There are currently some small limitations (i.e. handling directly KeyedContainer, etc.) Since we have the complete set of Dictionaries we can interact with any Event class References to Event objects are obtained by asking the Event data service ( AppMgr.evtSvc() ) There are currently some small limitations (i.e. handling directly KeyedContainer, etc.) >>> evt = gaudi.evtSvc() >>> header = evt['Header'] >>> print ‘Run number = ‘, header.runNum() >>> mcparts = evt['MC/Particles'] >>> for p in mcparts.containedObjects() :... print p.momentum().px()

8-19 Gaudi Framework Tutorial, 2004 Interacting with Histograms It is possible to book and fill histograms using the Histogram data service Same interface as in C++ The complete AIDA Histogram interfaces are available to Python It is possible to book and fill histograms using the Histogram data service Same interface as in C++ The complete AIDA Histogram interfaces are available to Python >>> import random >>> his = gaudi.histsvc() >>> h1 = his.book('h1',‘Histogram title', 40, 0, 1000) >>> for v in range(1000) :... h.fill(random.uniform(0,1000)) >>> print h1.mean()

8-20 Gaudi Framework Tutorial, 2004 Complete Example: Read a DST file import gaudimodule gaudi = gaudimodule.AppMgr(outputlevel=5, joboptions='$EVENTSYSROOT/options/PoolDicts.opts') sel = gaudi.evtsel() sel.open(['PFN:rfio:/castor/cern.ch/lhcb/DC04/ _ _10.dst']) evt = gaudi.evtsvc() his = gaudi.histsvc() h1 = his.book('h1','# of MCParticles', 40, 0, 5000) class DumpAlg(gaudimodule.PyAlgorithm): def execute(self): evh = evt['Header'] mcps = evt['MC/Particles'] print 'event # = ',evh.evtNum() h1.fill(mcps.size()) gaudi.addAlgorithm(DumpAlg()) gaudi.run(10) print h1, h1.contents() Ex/PythonExample

8-21 Gaudi Framework Tutorial, 2004 Complete Example: Read a DST file EventSelector SUCCESS Reading Event record 0. Record number within stream 0: 0 event # = 1 event # = 2 event # = 3 event # = 4 event # = 5 event # = 6 event # = 7 event # = 8 event # = 9 event # = 10 Histogram 1D "# of MCParticles" 40 bins [ , ] [5, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

8-22 Gaudi Framework Tutorial, 2004 Building on GaudiPython It is fairly easy to extern the functionality provided by GaudiPython There are already several “products” which are build on top of GaudiPython Bender Physics Analysis on PythonPanoramix Detector and Event Visualization It is fairly easy to extern the functionality provided by GaudiPython There are already several “products” which are build on top of GaudiPython Bender Physics Analysis on PythonPanoramix Detector and Event Visualization

8-23 Gaudi Framework Tutorial, 2004 Summary This is an attempt to convince you that Python is a very good programming language for Rapid Prototyping Efficient way to test your Physics Analysis ideas Interfacing C++ classes to Python is fairly easy Several well established technologies exists (Boost.Python, etc.) PyLCGDict is another way of doing it PyLCGDict exploits the LCG dictionaries that are already available due to object persistency (POOL)GaudiPython Is the package that enables the interaction of Gaudi from Python Latest version of GaudiPython is based on PyLCGDict Examples of the available functionality has been shown This is an attempt to convince you that Python is a very good programming language for Rapid Prototyping Efficient way to test your Physics Analysis ideas Interfacing C++ classes to Python is fairly easy Several well established technologies exists (Boost.Python, etc.) PyLCGDict is another way of doing it PyLCGDict exploits the LCG dictionaries that are already available due to object persistency (POOL)GaudiPython Is the package that enables the interaction of Gaudi from Python Latest version of GaudiPython is based on PyLCGDict Examples of the available functionality has been shown