1. Reconstruction Configuration with Python Chris Jones University of Cambridge

2. Chris JonesPage 2Reconstruction Python Tutorial Introduction Guide to python Configuration in Brunel / Reconstruction Based on latest Brunel version Some changes in up coming v34r3 release Assumes some basic familiarity with Configurables See Marco’s slides from last S/W http://indico.cern.ch/conferenceDisplay.py?confId=44134

3. Chris JonesPage 3Reconstruction Python Tutorial Basic Philosophy Use the Higher Level Configuration Methodology Write configurable objects that inherit from ConfigurableUser Actually use a derived class LHCbConfigurableUser. More on this later. Utilise full power of python and object orientation. Instrument each Configurable with easy to understand options E.g. Switch between DC06 and 2008 simulations Use a top-down hierarchy of Configurables Brunel delegates overall reconstruction configuration to RecSys RecSys delegates sub-detector Configuration to TrackSys etc.

4. Chris JonesPage 4Reconstruction Python Tutorial LHCbConfigurableUser Derives from ConfigurableUser in Gaudi Used as a base class configurable for 'LHCb' Configurables Similar idea to GaudiAlg C++ base classes Adds some additional functionality not in Gaudi LHCb specific so not appropriate Testing ground for things that could move to Gaudi eventually Some things already have Defined in the Kernel/LHCbKernel package Usual python/ /Configuration.py convention

5. Chris JonesPage 5Reconstruction Python Tutorial LHCbConfigurableUser Now part of Gaudi itself Methods to propagate options from one configurable to another E.g. Brunel passes information to RecSys (Functionality already being merged with core Configurable) ## @brief Set the given property in another configurable object # @param other The other configurable to set the property for # @param name The property name def propagateProperty(name,other): ## @brief Set the given property in another configurable object # @param other The other configurable to set the property for # @param name The property name def propagateProperty(name,other): “initialize()” method. Called once during configurable construction Default method implemented than does nothing Users can re-implement in derived configurables to do whatever they wish

6. Chris JonesPage 6Reconstruction Python Tutorial How are options Handled ? A python dictionary called “__slots__” E.g. The Brunel Configurable (truncated here)‏ # Brunel Steering options __slots__ = { "EvtMax": -1 # Maximum number of events to process,"SkipEvents": 0 # events to skip,"PrintFreq": 1 # The frequency at which to print event numbers,"DataType" : "2008" # Data type, can be ['DC06','2008'],"WithMC": False # set to True to use MC truth,"Simulation": False # set to True to use SimCond,"RecL0Only": False # set to True to reconstruct only L0-yes events,"InputType": "MDF" # or "DIGI" or "ETC" or "RDST" or "DST","OutputType": "DST" # or "RDST" or "NONE","InitSequence": ["Reproc", "Brunel", "Calo"],"MoniSequence": ["CALO","RICH","MUON","VELO","Track","ST"] } Methods provided to set and get these ‘properties’ Brunel().setProp(“InputType”,”DIGI”) InputType = Brunel().getProp( "InputType" )

7. Chris JonesPage 7Reconstruction Python Tutorial Configurables Hierarchy Each Configurable has only those options it needs to have E.g. Brunel has options related to the general configuration of the application as a whole Database Input/Output data files … and some options it needs to dispatch to other used Configurables General configuration of the various sequences Event initialisation, which sub-dets to run etc. Delegates the configuration of the reconstruction to RecSys configurable (RecSysConf()) Some options are passed from Brunel to RecSysConf() self.propagateProperties(RecSysConf(),["SpecialData","RecoSequence"])

8. Chris JonesPage 8Reconstruction Python Tutorial Brunel Configurable – Full Options List Removed in v34r3

9. Chris JonesPage 9Reconstruction Python Tutorial Using the Brunel Configurable Selection of python files in options sub-directory Configured for various common scenarios, e.g. DC06 data : Brunel-DC06.py 2008 Conditions : Brunel-2008.py 2008 Conditions with MC: Brunel-2008-MC.py Selection of python files containing data DC06-Files.py, 2008-Files.py, 2008-TED-Data.py Run the usual way passing both to gaudirun.py, e.g. > gaudirun.py Brunel-2008-MC.py 2008-Files.py > gaudirun.py Brunel-2008.py 2008-TED-Data.py (new) AppConfig Package Set of standard options for production jobs (All applications, not just Brunel) Book-keeping will refer to these options

10. Chris JonesPage 10Reconstruction Python Tutorial Using the Brunel Configurable Minimal Brunel.py file You must instantiate the Brunel() configurable, even if you only use the default settings. DC06 2008(NoMC) from Configurables import Brunel Brunel() from Configurations import Brunel Brunel().DataType = "DC06" Brunel().InputType = "DIGI" Brunel().WithMC = True from Configurations import Brunel Brunel().SpecialData = ["fieldOff"] # All 2008 real data was taken with Magnet OFF

11. Chris JonesPage 11Reconstruction Python Tutorial Using the Brunel Configurable Settings special data options E.g. From Brunel-Cosmics.py #-------------------------------------------------------------------------------- # Set the special data options #-------------------------------------------------------------------------------- Brunel().SpecialData = ["fieldOff","cosmics"] #-------------------------------------------------------------------------------- # Tweak the tracking options for cosmics #-------------------------------------------------------------------------------- Brunel().ExpertTracking += [ "noDrifttimes" ] TrackSys().TrackPatRecAlgorithms = [ "PatSeed" ]

12. Chris JonesPage 12Reconstruction Python Tutorial RecSysConf Part of the RecSys package Knows details of how to configure each sub-detector All options set by Brunel Delegates the sub-detector configuration to other Configurables (or options files). E.g. ## Steering options __slots__ = { "RecoSequence" : [] # The Sub-detector sequencing. Default is all known,"SpecialData" : [] # Various special data processing options. See KnownSpecialData,"ExpertHistos": False # set to True to write out expert histos,"Context": "Offline” # The context within which to run the reco sequences } recoSeq = self.getProp("RecoSequence") if "RICH" in recoSeq: richConf = RichRecSysConf() self.propagateProperties(richConf,["SpecialData","Context"]) richConf.RecoSequencer = GaudiSequencer("RecoRICHSeq")

13. Chris JonesPage 13Reconstruction Python Tutorial RecSysConf recoSeq = self.getProp("RecoSequence”) # Tracking DoTracking = False for seq in self.DefaultTrackingSubdets: if seq in recoSeq: DoTracking = True if DoTracking: trackConf = TrackSys() self.propagateProperties(trackConf,["ExpertHistos","SpecialData"]) # MUON if "MUON" in recoSeq: GaudiSequencer("RecoMUONSeq").Members += [ "MuonRec", "MuonID" ] importOptions("$MUONIDROOT/options/MuonID.py”) More examples :-

14. Chris JonesPage 14Reconstruction Python Tutorial Tracking Configurable : TrackSys() Expose more specialised Tracking options # Steering options __slots__ = { "ExpertHistos": False # set to True to write out expert histos,"SpecialData" : [] # Various special data processing options.,"ExpertTracking": [] # list of expert Tracking options, see KnownExpertTracking,"TrackPatRecAlgorithms": [] # List of track pattern recognition algorithms to run,"TrackExtraInfoAlgorithms": [] # List of track 'extra info' algorithms to run,"WithMC": False # set to True to use MC truth } ## Possible expert options KnownExpertTracking = ["noDrifttimes", "simplifiedGeometry", "kalmanSmoother"] ## Default track pattern recognition algorithms to run DefaultPatRecAlgorithms = ["Velo","Forward","TsaSeed","Match","Downstream","VeloTT"] ## Default track 'extra info' algorithms to run DefaultExtraInfoAlgorithms = ["CloneFlagging","TrackLikelihood","GhostProbability"]

15. Chris JonesPage 15Reconstruction Python Tutorial TrackSys() To set the special tracking options not exposed by Brunel() Import directly the TrackSys() configurable in your options Exactly what. Not set inoptions are available in TrackSys() is completely up to the author to decide stone. You have the full power of python from Configurables import TrackSys # Change the tracking algorithms to be run TrackSys().TrackPatRecAlgorithms = [ “Velo”,”Forward” ] # Change the “Extra Info” tracking algorithms TrackSys().TrackExtraInfoAlgorithms = [ “CloneFlagging” ]

16. Chris JonesPage 16Reconstruction Python Tutorial RichRecSysConf() Similar concept to TrackSys() ## Steering options __slots__ = { "UseCaloMomentumTracks" : False # Use Tracks cloned from originals with P updated using the CALO,"Context": "Offline" # The context within which to run,"Radiators": None # The radiators to use,"Particles": None # The particle species to consider. Default is (el,mu,pi,ka,pr),"ConfigureTools": True # Configure the general RICH reconstruction tools,"ConfigureAlgs": True # Configure the reconstruction algorithms,"PreloadRawEvent": False # Preload the RawEvent prior to the RICH algorithms,"PreloadTracks": False # Preload the input tracks prior to the RICH algorithms,"InitPixels": True # Run an initialisation algorithm to create the pixels,"InitTracks": True # Run an initialisation algorithm to create the tracks,"InitPhotons": True # Run an initialisation algorithm to create the photons,"TracklessRings": False # Run the trackless ring finding algorithms,"CheckProcStatus": True # Check the status of the ProcStatus object,"PidConfig": "FullGlobal" # The PID algorithm configuration,"MakeSummaryObjects": False # Make the reconstruction summary TES data objects,"HpdRandomBackgroundProb" : -1.0 # If positive, will add additional random background to the data at the given pixel rate,"SpecialData" : [] # Various special data processing options. See KnownSpecialData in RecSys for all options,"RecoSequencer" : None # The sequencer to add the RICH reconstruction algorithms to }

17. Chris JonesPage 17Reconstruction Python Tutorial Using Configurables Calling __apply_configuration__ (applyConf()) Before Gaudi v20r3, users had to call this method by hand With Gaudi v20r3, this should not be done. Framework does this for you By and large makes little difference to users One side effect though no longer possible to redefine things afterwards. Use options provided by Configurables If not possible, improve Configurables Clearly some use cases will be difficult. Configurables are still evolving as authors learn how to use them so please report any problems. Configurables Framework provides a solution to run some python code after *all* Configurables have been used def doMyChanges(): GaudiSequencer("Hlt2Selections").Members = [] appendPostConfigAction(doMyChanges)

18. Chris JonesPage 18Reconstruction Python Tutorial Conclusions Any Questions ?