1. News from EP SFT John Harvey FOCUS Meeting – October 3 rd 2003

2. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Outline Overview of EP SFT Group Activities LCG Application Area Project News  SPI  SEAL  Simulation ROOT CERNLIB Reviews

3. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Fluka 1.0 Overview of EP SFT Group Activities LCG Applications Area Projects 1.7 Simulation 3.8 PI 2.1 SEAL 3.2 POOL 0.7 SPI 4.5 ROOT 4.7 User - provider Geant 4 9.2 Atlas 2.3 CMS 1.5 LHCb 1.4 ALICE 1.2 HEP wide Projects Experiments CERNLIB 0.3

4. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s SPI (Software Process & Infrastructure) – LCG Policy Use same set of tools standards and procedures - these together with the software are centrally installed maintained and supported Testing and documentation are integral parts of the software development process Commonly used open source or commercial software will be adopted in preference to 'do it yourself' solutions No distinction between the communication and/or problem reporting tools and fora used internally by developers and those used by end users and the community; they will be the same Web based information and tools will be given a high priority Exploring synergy with other projects, in particular EGEE middleware

5. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Savannah : Project Portal and Bug Tracker Bug Tracker A central installation so that each project has an available bug tracker system Centrally managed but customizable for different needs Easier to post bug if projects use similar systems No work of maintenance and upgrade on the projects Web based Project Portal The development front-end of every project With all features needed, configured and integrated  News and Faq  Task management  Mailing lists access  Bug tracking access  Download area  Access to the CVS repository We have customized “Savannah” from GNU http://savannah.cern.ch Now reintegrated in open source

6. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s http://savannah.cern.ch

7. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s

8. Software Testing Procedures, tools and methodologies used within the LCG Application Area projects SPI provides frameworks for Software TestingSoftware Testing  CppUnit and PyUnit, for unit-tests (C++ and Python)  Oval, can cover from unit to validation tests  QMtest, a GUI interface to organize and run all tests All tools are available via the SPI Software Service “HowTo” documents are available in the LCG Workbook  to help both users and developers Software Testing simple templates  Test plan, test cases, etc

9. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Service : External Software Library Provide support for external software libraries used in the LCG project All software needed by one LCG project shall be installed in common areas Maintained centrally with support and help to the user Make external documentation available Maintain contact with the authors, whenever possible Prepare distribution and pre-installation

10. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s http://spi.cern.ch/extsoft

11. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Many more services available Code documentation  Doxygen, Lxr Build and release tool  Scram Memory Leaks  Valgrind Coding guidelines  RuleChecker Standard directory structures  LCG policies Automatic Builds  Nicos Software distribution LCG Developers workbook Software documentation, templates, etc See SPI web http://spi.cern.ch/http://spi.cern.ch/

12. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s SEAL: Foundation and Utility Libraries Aims  To provide the C++ foundation libraries and tools that are common among the LHC experiments by selecting and integrating existing ones or by developing new ones. Current activities  Support for Boost library Open source utility library (parts of which are destined to be included in the forthcoming C++ standard) Installation, validation, user consultancy, …  Participation to CLHEP project Some HEP foundation classes already exist in CLHEP Provide help (infrastructure, manpower) in its maintenance  Development of new packages Develop SEAL utility and operating system isolation libraries complementary to STL, Boost and CLHEP Mainly adopted from CMS classlib

13. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s SEAL: Math Libraries Aims  Provide experiments with math and statistics libraries to be used in analysis, reconstruction, simulation.  Math consultancy – need to maintain in-house expertise Current Activities  Support for GSL (Gnu Scientific Library) Evaluation by Indian collaborators (Indore) – comparison with NAGC No defects found - functions in use in LHC experiments supported Installation, validation, user consultancy, communication with GSL developers, extensions of the library  Re-implementation of MINUIT in C++ Prototype is available (Migrad and Minos implemented) The numerical results of the two prototypes compared to the Fortran version. Compatible within the errors.  Other studies Comparison of various linear algebra packages …

14. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s SEAL : Scripting with Python Aims  Provide the scripting needs for Interactivity, Job Configuration, Rapid prototyping, Integration, etc.  Bring Python and C++ (CINT) at the same level of functionality. Support interoperability between them. The user can choose. Current Activities  Guidelines for developing Python bindings for C++ classes Evaluated existing options Recommended Boost.Python and SWIG (see report)see report  Development of bindings for standard packages PyROOT, PyLCGDict, PyCLHEP,…  Python courses given by EP SFT group member Part of the CERN Technical Training program

15. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s SEAL - Frameworks Develop components that can be combined to provide frameworks for data processing and interactive applications  Standard messaging, object whiteboards, plug-in managers, etc Components used by other LCG projects (e.g. POOL) but also by experiments (e.g. ATLAS, LHCb,..)

16. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s LCG Simulation Project Framework WP Geant4 WP FLUKA integration WP Physics Validation WP Shower Param WP Generator Services WP Subprojects Work packages Geant4 Project FLUKA Project Experiment Validation MC4LHC

17. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Simulation Physics validation subproject very active Physics validation subproject  Working well with both Geant4 and FLUKA  e.g. simple benchmark comparisons with thin targets, tracking detectors Generator Services (GENSER) subproject very active Generator Services  Centrally organized code repository for generators and common generator tools  Beta release of GENSER library made according to schedule on September 30 th  Contains those given top priority by LHC experiments HERWIG, HIJING, ISAJET and PYTHIA  Others to follow

18. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Simulation Geant4 working well and closely with the experiments  See Presentations at the 8th Geant4 Collaboration Workshop - TRIUMF, Vancouver, 2-6 September, 2003 for up to date statusPresentations  Technical Forum created (Developers/Users) for user input to work planning – setting priorities etc next to be held at CERN - Tuesday, October 7th, 15:00-17:30 (tbc)  As of 2 weeks ago more than one million physics events have been simulated with OSCAR/Geant4 by the CMS production team FLUKA actively working with physics validation  Subproject involves FLUKA integration with the generic framework  CERN-INFN agreement on FLUKA established Documentation – Fluka physics publication, User Guide Training – 1 st User Course Oct 13-17, Frascati1 st User Course  expect to repeat at CERN Make Fluka source code available CERN provides 1 PDSA - timescale to complete tasks ~12 months

19. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s ROOT User/provider relationship is working very well POOL is getting what it needs from the ROOT team, and (with experiment integrators) is feeding back to ROOT Clear that full access to ROOT analysis capability needed  Both directly or via interfaces: PyROOT, Aida_ROOT 6 developers at CERN  LCG funded - Ilka Antcheva, Gerri Ganis, Valeri Onuchin  CERN funded - R.Brun, O.Couet, F.Rademakers ROOT workshop at SLAC: 25 Feb 04, hosted by Babar

20. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s ROOT Release 3.05/07 July 2003 (R.Brun) New Users Guide Many tiny improvements - consolidation. Platforms - Unix/Linux(es)/Windows(VC++,gcc)/MacOSX Interfaces to - MySQL, Oracle, Postgres, SapDB IO implementations Dcache(Desy/FNAL), CHIRP (Condor) New package TFumili (up to 5 times faster than TMinuit) New Geometry package

21. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Development version 3.10 IO performance improvements (> 10%) Better support for STL collections (IO & queries) First demonstration of PROOF in a realistic case PROOF with Alien (ARDA) TCastorFile: support more efficient than old RFIO GUI: Many improvements and new widgets New NeuralNet package: TMultiLayerPerceptron Many new tutorials. Test suite improvements New roottalk Forum (see Root web page)

22. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s CERNLIB (Ian Mclaren) 2002 – ‘final’ CERNLIB release 2003 – three releases following urgent requests (mainly) from external users Platform O/S C Fortran Linux/IntelRH 7.3gcc 3.2g77 3.2 Linux/Intel RH 7.3 icc 7.0 ifc 7.0 Linux/Intel RH 10 gcc 3.3 g77 3.3 Some (very few) bug fixes made The source files for latest release completely up to date

23. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Integration and Experiment Support Message from Manpower Review  The LHC experiments are finding their schedules and their manpower to be very stretched by LCG software integration “Release early and often” only works when complemented by “Integrate early and often” Prompt and successful integration in experiments is essential to project success We are helping now, but we must help even more  by exploring a tighter association between project and experiments at the developer level  by assigning presently ‘unaligned’ developers with a particular experiment, to enhance liaison and integration support

24. SFT S o F T w a r e D e v e l o p m e n t f o r E x p e r i m e n t s Apps Area internal technical review Oct 20-22 Examine design, implementation, performance and experiment experience Particular focus: SEAL and POOL software and its integration and validation by the experiments Will be public Basic intent: a helpful exercise to validate and improve our software and direction Secondarily: useful preparation for an LHCC Comprehensive Review of the LCG project in November