1. SEAL Developers http://iguana.cern.chMarch 3, 2003 Lassi A. Tuura, Northeastern University CMS Components Plug-in Manager Lassi A. Tuura Northeastern University, Boston

2. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 2 CMS Frameworks: Structure Federationwizards Detector/EventDisplay Data Browser Analysis job wizards Generic analysis Tools ORCA FAMOS Objytools GRID OSCAR COBRA Distributed Data Store & Computing Infrastructure CMStools Consistent User Interface Coherent basic tools and mechanisms

3. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 3 ODBMS GEANT 3 / 4 CLHEP PAW Replacement C++ Standard Library + Extension Toolkits C++ Standard Library + Extension Toolkits Frameworks: Layering Calibration Objects Calibration Objects Generic Application Framework Physics modules Grid-Uploadable BasicServices Adapters and Extensions Configuration Objects Configuration Objects Event Objects Event Objects (Grid-aware) Data-Products SpecificFrameworks Event Filter Reconstruction Algorithms Physics Analysis Data Monitoring

4. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 4Modules v What’s a module? r An (extension) component to a framework (often dynamically loaded) r Framework knows only that a module extends a particular API (usually abstract interfaces), not how and what it needs to do so r Examples abound: Netscape plug-ins, Apache modules, Python and PERL modules, Illustrator and PhotoShop extensions, … r Used in many ways in CMS projects: package builders, detector units, hit/digi readers and writers, and many algorithms are essentially extensions of APIs provided by COBRA, and “subframeworks” in ORCA v Why modules? r Eases management—framework doesn’t need to know lots of details about the modules, but can simply load them and gain additional capabilities r Helps to give symbolic names to capabilities and allow framework to reason about them: “here’s all you will need to read data product X” r Allows better services to external parties (Python, GRID, IGUANA, …) without having to put that knowledge into many applications

5. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 5 Plug-ins in CMS: Background v Much of CMS software has “done” plug-ins for a while r IGUANA: a few years – The origin of the model described here – Apart from a tiny kernel, virtually everything else is in plug-ins – Plug-ins and dynamic linking are 100% orthogonal r COBRA: probably longer than I have been in CMS – Essentially the same model as IGUANA, technically not so refined but functionally much the same – Planned to use IGUANA system but then SEAL came along r We have a few years of experience with this model and like it v Basic principles r Framework outside r Simplicity r Lightweight

6. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 6

7. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 7 Implementation Features v Maintains a list of directories with plug-in module definitions v Plug-in module = shared library with well-defined entry points r Self-describing through a query entry point r But really orthogonal to dynamic loading – Some or all plug-ins can be built into the program v Automatically detects new, changed or removed modules r For new or changed ones loads it, queries its capabilities, and unloads it r Maintains a cache of module properties r Updates the cache if the directory is writable v When requesting information from the database, uses the cache instead of loading all the modules all the time v Two packages: generic base plus specific implementation r The model described here is the specific one, the generic one has no model

8. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 8 Technical Basics v Any C++ type can be created through plug-in mechanism v Any module/package can define a new plug-in category r Also other plug-in modules (the only way we use it, our kernel is empty) r Different plug-in categories in same or different projects coexist peacefully v Separate dynamic loading and factory interfaces r Clients see only a (powerful) factory r Modules see a relatively simple factory provider interface r Physical decomposition choices have no impact on interfaces r Dynamic loading is optional, changing packaging does not affect code v Automated caching r Program can reason about run-time environment without loading it all r Optional and transparent, failure to cache only shows up as (a big) slowdown v No external meta data: every plug-in module is self-contained r Build system automatically provides a list of available plug-in modules

9. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 9 Implementation Structure v Plug-in manager r (Green means user-visible class) r Practical layers: concrete object factories – IgBrowserDB, IgSiteDB, … r Concrete implementation – IgPluginDB, IgPluginDirectory, IgPlugin, IgPluginDef – IgPluginDBViewBase, IgPluginDBView, IgPluginDBItem r Generic caching and module-loading base – IgPluginCache, IgPluginBase – Knows how to manage plug-in modules (shared libraries) – Maintains a plug-in information cache per directory – The cache is an automatically updated text file (provided it is writable)

10. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 10 Plug-In Cache Object Factory Object Factory Implementation Structure… v Each module has three entry points: attach, detach, and query r Query just caches information: the module capabilities are recorded in the cache and the in-memory database that merges all caches r Attach and detach happen on module load/unload: module installs factories for various named plug-in objects it can create v Application state management and service components r Object location service: how you find all those plug-ins you’ve created r Won’t get into this in this presentation, but related and already involved Plug-in Database Plug-In Cache Module Object Factory Attached Unattached

11. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 11 System API Base Manager Module Providers Module Providers Factory Clients Factory Clients Cache View Item Structure

12. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 12 Classes #1 v Basic machinery r IgPluginCache: Maintains raw cache data per directory – Automatic update mechanism – Means to access per-module raw cache data (= “descriptors”) r IgPluginBase: Base class for a loadable module – Ability to load, unload, query, check if attached, check entry points, … – A derived class must implement actual behaviour, no meaning to given to the above, just services that typical plug-in managers will find useful r SharedLibrary (from classlib): Dynamic loading support

13. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 13 Classes #2 v Syntactic sugar: our plug-in manager model r IgPlugin, IgPluginDef: Module wrapper – Module providers derive IgPluginDef – Unit for managing the cached data r IgPluginDirectory: A directory of our modules – Derives IgPluginCache and instantiates IgPlugin – Mechanism to write to and reconstruct from cache r IgPluginDB: Collection of plug-in data – Collection of IgPluginDirectories – Collection of plug-in views (categories) r IgPluginDBViewBase, IgPluginDBView – An actual view of the plug-in database r IgPluginDBItem: Cached item – Interacts with the IgPluginDBView through IgPlugin and IgPluginDB – Also reconstructed from the cache (managed by IgPluginDB and view)

14. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 14 Classes #3 v Practical layer r IgBrowserDB, IgBrowserInfo – Browser factory, details about what is cached about browsers (= name) – Note! This is the only level at which the actual object type, factory argument lists, constructor arguments etc. are defined and used! r IgSiteDB, IgSiteInfo – etc. r …

15. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 15 Module Provider Interface class MyPluginDef : public IgPluginDef { public: virtual voidquery (void); virtual voidattach (void); }; DEFINE_IGUANA_PLUGIN (MyPluginDef); void MyPluginDef::query (void) { IgBrowserDB::Def ::declare (this); } void MyPluginDef::attach (void) { IgBrowserDB::Def ::installFactory (this); }

16. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 16 Module Provider Interface // COBRA way of doing the same thing static PKBuilder me;

17. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 17 Factory Client Interface void SomeFoo (void) { IgState*state = …; IgSite*site = …; IgBrowser*browser = IgBrowserDB::get () ->instantiate (“MyFoo”, state, site); }

18. March 3, 2003 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 18 Category Definition class IgBrowserInfo : public IgPluginDBItem { public: typedef IgBrowser Object; struct Factory { virtual ~Factory (void); virtual Object *create (IgState *state, IgSite *parent) = 0; }; template struct AutoFactory : Factory { virtual Object *create (IgState *state, IgSite *parent) { return new T (state, parent); } }; // Some methods (ctor, create(), attach(), factory()) } class IgBrowserDB : public IgPluginDBView { public: virtual IgBrowser *instantiate (const std::string &name, IgState *state, IgSite *parent); };