1. SEAL Project Overview Lorenzo Moneta/ CERN-EP on behalf of the SEAL team ACAT03 IX International Workshop on Advanced Computing and Analysis Techniques in Physics Research 1-5 December, 2003 KEK

2. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN2 Outline  Introduction –Scope and motivation  SEAL Work Packages –Description –Current status  Future plans and summary

3. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN3 Project motivation  What is SEAL ? –project in LCG Application Area –started last year after BluePrint RTAG recommendations –common project on core tools and services »SEAL: Shared Environment for Applications at LHC  SEAL aims to –Provide the software infrastructure, basic frameworks, libraries and tools that are common among the LHC experiments –Select, integrate, develop and support foundation and utility class libraries –Develop a coherent set of basic framework services to facilitate the integration of LCG and non - LCG software  Do not re-invent the wheel –Try to use as much as possible of existing software –Develop/adapt/generalize to achieve necessary level of coherency

4. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN4 Project scope  Foundation Class Libraries –Basic types (STL, Boost,…), utility libraries, system isolation libraries, domain specific foundation libraries  Mathematical Library –Basic mathematical functions, linear algebra, fitting and minimization libraries  Basic Framework Services –Component model, reflection, plug-in management, incident (event) management, distributed computing, grid services, scripting Basic Framework Foundation Libraries Simulation Framework Reconstruction Framework Visualization Framework Applications... Optional Libraries Other Frameworks Software Structure

5. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN5 Project Work Packages FoundationFoundation and Utility Libraries and Plug-in Manager MathLibsMath Libraries Support and Coordination DictionaryLCG Object Dictionary FrameworkComponent Model and Basic Framework services ScriptingScripting Services GridGrid Services DocumentationEducation and Documentation Not yet active

6. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN6 Foundation and Utility Libraries  Inventory of existing C++ utilities ( http://seal.cern.ch/components.html) ( http://seal.cern.ch/components.html) –Recommended classes useful for LCG projects and experiments –Large number from Boost (open source project) »Support for Boost is provided (in contact with developers)  Develop in SEAL what is missing in STL and Boost  Have a set of low-level, fairly independent class libraries –  200 classes originated mainly from classlib (CMS) –Basic abstraction and foundation utilities classes –Timing utilities –Stream-oriented I/O, compression and decompression  High quality, efficient, well tested and documented library –Recommend best practice to LHC developers

7. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN7 Plug-in Manager  Plug-in Management –Service in charge of managing, querying, [un]loading plug-ins  Low level plug-in management (no framework constraints) –Ability to create “module” libraries containing “plug-ins” –Ability to instantiate concrete implementations knowing the plug-in category and the concrete type –Caching information for fast loading »what modules contain »what kind of plug-ins exist and which categories  Imported from CMS IGuana –Basic concept: advanced object factory »Interfaces for object instantiation and plug-in provider –Dynamic loading completely orthogonal — and optional!

8. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN8 Mathematical Libraries  GSL (Gnu Scientific Library) evaluation –Extensive comparison with NagC –Validation and test suite  Re-implementation of MINUIT in C++ –Prototype available with Migrad and Minos –Undergoing numerical and performance tests »Comparison tests with Fortran version (compatibility within errors)  Linear Algebra –Prototype wrapper around GSL matrices. –Comparison of various linear algebra packages (CLHEP, GSL, [u]CBLAS)  Participation in CLHEP project  Provide to experiments math and statistics libraries to be used in analysis, reconstruction and simulation. –support, consultancy, etc..

9. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN9 Object Dictionary  Adding “introspection/reflection” capabilities to C++  Use for persistency and interactivity (e.g. Python)  Modern languages provide reflection inherently –In C++ reflection is very limited (RTTI)  The basic C++ concepts shall be supported –e.g. inheritance, methods, data members, accessibility, templates  Seal provides : –Reflection Builder and Reflection (API interface) packages –Dictionary generator to produce dict. source from C++ files »Uses gcc-xml + Python script –Dictionary for some standard packages (STL, CLHEP)  Used by POOL (persistency)

10. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN10  gcc-xml to parse header-files (extension to gcc )  XML file generated with dictionary information  Python script to generate dictionary C++ code –use an additional XML file to customize class selection  Compiled and loaded at run-time to create dictionary –Provide meta information about the object to the user Dictionary Generator.xml.h lcgdict.h gccxml.xml filter + exten gendict par ser _dict.cpp..Dict.so make selection file (Python script) #include files Reflection Libraries Clients User

11. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN11 Framework Services  Motivation: –Allow LCG developers and users to write portable and modular code –Provide abstractions for modelling software systems  Component model definition –Basic Components »Set of “base classes” and interfaces needed to build more complex components »Provide standard functionality  instantiation, configuration, lifetime management, etc  Users (framework developers) should model their services using these proposed base classes  SEAL provide in addition a set of basic services implemented following the component model

12. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN12 SEAL Component Model  Hierarchy of bases classes –Components living in a Context –Services are Components providing their local Context  User classes inherit from Component or Services –Plug-in functionality for free Context Application C1 C2 Component Svc2 C1C6C1C7 Svc1 C1C3C1C4C1C5 Svc1 Service C1C7

13. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN13 Basic Common Framework Services  Application Service –Defines the top level Context –Defines initial set of Components to be loaded in the application  Message Service –Message composition, filtering and reporting  Configuration Service –Management of Component properties and loading configurations –Multiple backends foreseen: »Gaudi style options,.INI style, CMS style, XML,...  Other services will come later: –Object Whiteboard (Transient Object repository) –Dictionary Service (Loading Dictionaries in the application)

14. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN14 Scripting Services  Scripting needed for Interactivity, Configuration, Rapid prototyping, Integration (Component bus), etc.  Main scripting language : –Python  Define guidelines for developing Python bindings : –Various tools available (Boost.Python, SWIG) –Recommend solutions: »Boost.Python »LCG Dictionary GUI Python math shell PyGaudi Database EDG API GUI XML Gaudi Frame work PyROOT Root Classes PVSS JPE Java Classes

15. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN15 Scripting Packages  PyLCGDict –Python extension module to the LCG dictionary –Provide access to C++ libraries from Python, “for free” –Automatically generates Python proxies for C++ objects using dictionary information »Everyone can create binding effortlessly  PyROOT –Python bindings for ROOT classes »access to ROOT functionality from Python –Binding using the ROOT dictionary (CINT) –Missing gateway ROOT -> Python

16. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN16 Education/Documentation  Produce documentation –Reference documentation (generated with doxygen) –HowTo pages for the released elements »Plugin Manager, Dictionary generation, PyROOT, PyLCGDict, etc. –Release notes  Training material (tutorials) –Help developers incorporating SEAL into the LCG projects and the experiment frameworks  Python courses –Hands-on introduction course to Python programming –Very positive feedback received

17. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN17 Software Process and QA  Design –Team design sessions –Prototypes using Pythons  Code Review –No formal code reviews –Coding done very often in pairs (XP style) –More than one developer knowledgeable for each package  Testing –More than 200 SEAL unit tests based on CppUnit –All are driven and run automatically by QmTest  Bug reporting and tracking –Savannah Portal ( http://savannah.cern.ch/projects/seal) ( http://savannah.cern.ch/projects/seal)

18. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN18 Future plans (1)  Foundation libraries (Plug-in Manager) –Follow Boost developments –Little development planned in SEAL (hash-maps)  MathLibs: –Produce inventory of mathematical functions –Evolve Minuit C++ into a final product –Linear Algebra studies –Generate dictionary library for Python and CINT  Dictionary: –Implementation of new reflection model »Overcome some existing limitations (typedef,…) –Converge to a common dictionary with CINT ?

19. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN19 Future plans (2)  Framework: –Develop Whiteboard (object repository) and Dictionary service (loading of dictionary libraries on-demand)  Scripting : –PYLCGDict: »Migrate much of functional core from C++ to Python »Support more natural Python features (eg iterator protocol) –PyROOT: »Unify core with PyLCGDict »Develop gateway ROOT -> Python  Grid : –New LCG project (ARDA) is starting –Expect need to interface to some of the ARDA services  Documentation: –Produce tutorials for foundation classes, Boost, etc..

20. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN20 Summary  SEAL has delivered basic foundation, utility libraries and object dictionary –The main “client” have been POOL (persistency project) and PI »Plug-in manager is also used in CMS –Very positive feedback received for plug-in manager –Currently being integrated into experiments’ frameworks of LHCB (Gaudi) and ATLAS (ATHENA)  The first version of the Component Model and Framework services is available –Start integrating in POOL and the LHC experiments –Wait for feedback before developing more services  Scripting based on Python –Boost.Python and PyLCGDict recommended for Python bindings –PyLCGDict is used to provide bindings for classes of Physics Interfaces (PI) project

21. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN21 http://seal.cern.ch/ Acknowledgments: Seal team members: R. Chytracek, J. Generowicz, W. Lavrijsen, M. Marino, P. Mato, S. Roiser, L. Tuura, M. Winkler Conclusions

22. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN22 Additional Slides

23. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN23 LCG Application Domain Coverage

24. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN24 Module Library PluginManager PluginFactoryBase PluginFactory T PluginInfo XXXFactoryXXXInfo ModuleModuleDef ModuleBase MyModuleDef * * PluginCache Client * Plug-in Manager Design  Originated from CMS Iguana

25. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN25 PropertyListItem MethodField ItemBuilder MethodBuilderFieldBuilderClassBuilder NamespaceBuilder Reflection ReflectionBuilder ClassNamespace Reflection System  Reflection Builder: the loading interface  Reflection : the user API

26. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN26 LCG Object Dictionary: Usage ROOT I/O LCG Dictionary CINT Dict Streamer.h in out ROOTCINT CINT generated code Dict generating code.adl.xml ADL/GOD Other Clients: (python, GUI, etc.) LCG to CINTDict gateway (2) (1) Population Conversion Reflection GCC-XML

27. ACAT03, 1-5 Dec. 2003SEAL Project OverviewL. Moneta / CERN27 Seal Component Model  Hierarchy of bases classes  Each Component lives in a Context  Contexts form their own hierarchy  Support for component look-up  A Service provides its own local Context –Components of a Service live in the scope defined by its own Context  User classes inherit from Component or Service –Plug-in functionality for free Component Service MyComponentMyService Context * * 1 Component Factory Component Loader 1