1. VI/ Rome Nov 23 Preparing to Change the Baseline for CMS Persistency Vincenzo Innocente Workshop CMS-Italia Computing & Software Roma, Nov 23 2001

2. VI/ Rome Nov 23 Slide 2 Baselining Architecture/Framework/Toolkits Current schedule is to Baseline CMS “Offline” Software in time for the Physics TDR. Major activity in next 12/18 month will be to define and prototype the initial production software for LHC operation è Review Architecture è Choose products è Prototype and implement middleware è Implement framework and toolkits A primary goal is to ensure that the architecture will support and take profit of the evolution of IT technology at negligible cost for CMS physics software è Some components harder to change: p Programming language p Data Management layer Object Store plays a central role in CMS computing model: è Persistency cannot be regarded as just other basic computing service è We must ensure to be able to access data for the whole lifetime of the experiment (even longer)

3. VI/ Rome Nov 23 Slide 3 File Distributed Data Store Data Browser Analysis job wizards Simulation Reconstruction PersistencyServices NetworkServices Coherent Analysis Environment Visualization BatchServices VisualizationTools AnalysisTools Software Development

4. VI/ Rome Nov 23 Slide 4 CMS Data Analysis Model Detector Control Online Monitoring Environmental data store Request part of event Simulation store Data Quality Calibrations Group Analysis User Analysis on demand Request part of event Request part of event Store rec-Obj and calibrations Quasi-online Reconstruction Request part of event Store rec-Obj Persistent Object Store Manager Database Management System Event Filter Object Formatter PhysicsPaper

5. VI/ Rome Nov 23 Slide 5 Analysis & Reconstruction Framework ODBMS Geant3/4 CLHEP Paw Replacement C++ standard library Extension toolkit Reconstruction Algorithms Data Monitoring Event Filter Physics Analysis Calibration Objects Event Objects Configuration Objects Generic Application Framework Physics modules Utility Toolkit Specific Framework adapters and extensions

6. VI/ Rome Nov 23 Slide 6 HEP Data Event DataSet DataSetMeta-Data Event Electrons Electrons Tracker Alignment Tracks Tracks Ecal calibration Ecal calibration User Tag (N-tuple) l Event-Collection Meta-Data l Environmental data è Detector and Accelerator status è Calibrations, Alignments (luminosity, selection criteria, …) l … l Event Data, User Data Navigation is essential for an effective physics analysis Complexity requires coherent access mechanisms Event Collection CollectionMeta-Data

7. VI/ Rome Nov 23 Slide 7 DataBase Management System DBMS Server Distributed, Hierarchical, File Storage System Application (Distributed) DBMS Client Application Representation Persistent Data Representation Database internal Representation Database Storage (Server+Files) Tertiary Storage (Tapes) NETWORK

8. VI/ Rome Nov 23 Slide 8 Objectivity l Objectivity è Currently about 30TB in Objectivity DB’s è Experience with writing into DB with up to 300 CPU’s in parallel è Little experience to date with large numbers of parallel readers è We have confidence that we could make an Objectivity based solution work l Commercial Considerations è Object databases have not taken off as forecast è Objectivity is the only major vendor of an ODBMS è CMS, envisages the possibility to continue using the product for maybe 1 year if the company should disappear (such that all support disappeared) l Baseline Software è Milestone at end of 2002 to go into Physics and CCS TDR Process è Make changes to baseline before PTDR, not during or after (If we can avoid it)

9. VI/ Rome Nov 23 Slide 9 Oracle Assessment l Latest Oracle version (9i) implements the key features of an Object Relational Database è Ability to store instances of user defined classes as objects è C++ and java interface as well as SQL l Resilient server architectures, interesting new developments on cluster architectures l Interesting development plans for total storage system management l But, currently heavy size (time?) overheads to store our sort of data l Laborious, multi-step, process for specifying to the DB our complex objects l CMS has submitted (to IT/DB) a list of approximately 50 areas of concern with the intention that we can rapidly determine if there are any “show-stoppers” before investing major effort è One full time CMS CERN-Fellow) is working with the IT/DB Oracle team to build CMS expertise on the possible ways to store our sort of data. p We can now store and retreive ORCA SimHits in Oracle è IT/DB plans to report back in January on the “Show-stoppers” l Probably not a solution we can adopt in next year or more. è Interesting R&D, but probably not somewhere CMS can afford to spend its limited manpower

10. VI/ Rome Nov 23 Slide 10 Data Access Problems l All Productions (at CERN) and Analysis (everywhere) have encountered debilitating data access problems è Disk failures. Lack of reliable hardware puts production and analysis in conflict è Castor+RFIO immaturity è Network limitations è Late delivery of hardware l At CERN Many problems, some of which we associate with an inadequate manpower situation è Net effect of these leads to job failure rates in the >15% range. p Wrong LSF parameters, daemons killing servers, signals trapped by LSF wrappers etc, complex systems with non-understood interconnections l These problems are features of the large amount of data we have now, small amount of disk, long CPU times. è Same problems for Objectivity or non-Objectivity p But the tools we have in Objectivity to relocate files, optimize their serving, compensate for hardware failure have been very useful and give guidance to how our systems should be setup in the future Data Access has had serious impact at CERN, FNAL, DESY this year. Cheapest service is not working anywhere for our sort of challenges Data Access has had serious impact at CERN, FNAL, DESY this year. Cheapest service is not working anywhere for our sort of challenges

11. VI/ Rome Nov 23 Slide 11 CMS-ROOTIO Workshop (Oct 10/11) 1) ROOT team has built a powerful system for object streaming and object description in the IO files. We want both these types of feature l Users have built, and now ROOT is implementing, inter-file references (OIDS!) è See http://www.AmbySoft.com/mappingObjects.pdf for example, strong emphasis on having control of your “oids”http://www.AmbySoft.com/mappingObjects.pdf p Do not allow a proprietary layer to do this ! p Much easier to change persistency later 2) Need a true DB layer at least for the OID mapping, collections etc 3) We also need the functionalities as currently supplied by the Objectivity AMS/RRP etc to delegate “files” and “file responsibilities” to a very low level of the system è GRID should concentrate the mind on this issue as the whole concept of physical and logical files requires addressing l With these three layers one could build a sustainable persistency solution that could satisfy most reasonable use-cases for LHC

12. VI/ Rome Nov 23 Slide 12 ROOTIO/CMS Mismatches ? l CMS has some important technical issues to resolve with the ROOT team è Degree of intrusiveness p Resolvable. è Technical c++ issues p Global-state, threads, exception handling.. è Use of external software p Rather than subsumption of it zlib example è Definitions of modularity have to be agreed p Long term scalability and maintenance is at issue here è Development model p Management, oversight, SW packaging, priorities, "ownership", and so on è Legacy support p May be an issue by 2005,2010 l These issues should be tractable

13. VI/ Rome Nov 23 Slide 13 Key issues Object Model vs Data Model è Schema, dictionary (how is generated, where and how is stored) è streamers, converters (generated, user-provided) Object identification (OID, URL,…) è How and object will be identified in the CMS “universal” storage system? p From a transient application p By internal navigation in the storage itself è How replication and re-clustering (reorganization of the physical data storage) will be supported? Storage System Administration è OS vs “DBA” Management and future developments è Product ownership è Architecture, Modularity and Packaging è Collaboration with other experiments è (undesirable) Backward compatibility

14. VI/ Rome Nov 23 Slide 14 Plan Approved in Joint Technical Board l Maintain all required support for DAQTDR è But wherever possible avoid new developments l Now to Xmas: è Estimate program of work to reach a new baseline for 2003 p ROOTIO based object storage for event and possibly other (eg calibration objects?) p Oracle or similar DB and OID-mapping layer p Low-level file handling tools for data management è Try to establish common projects with LHC community p All of these will be common requirements for LHC experiments l New Year: Evaluate progress è Estimate impact on post DAQ-TDR milestones l Establishing a working group now è Vital to ensure this is a team effort, both within CMS and in LHC p Maximize intellectual contributions, avoid jamboree è Concentrated timescale, aim for participation from CMS(C,P&T)+ROOT+IT/DB p Get main players involved together from the start

15. VI/ Rome Nov 23 Slide 15 A HEPIO Standard? l The physical and logical format of the object store should (?) be an LHC (or even HEP?) specification. è ROOTIO format may be a concrete area we can start on now p Document and specify ROOTIO physical and logical format p Identify missing concepts and iterate with ROOT team, For example the OID issues, long and short references Intention is to keep the ROOTIO solutions, possibly with some extensions p Establish this format and an agreed mechanism for changing it We must know that we can write/read/interpret this data and schema “forever” l With this standard agreed, è We establish an important component of ROOT as a guaranteed layer è We start to collaborate in a concrete technical way è ROOT itself of course interfaces to this format now, so is in a strong position è But, one can imagine building now, or in the future, other products p ROOT++, and/or something quite different l Need to identify, now, someone in CMS willing to do much of this work in collaboration with ROOT team

16. VI/ Rome Nov 23 Slide 16 Why now? l The LHC community is ready to find common solutions è Prototyping is over, need real solutions within finite resources l We need to effectively use external intellectual effort è Alleviate our manpower problems l There are many technical issues to resolve that we have specific requirements on. è Act now and get most of what we want, or wait and get what we get l We have the most advanced experience in LHC on real production issues è Put that at the disposal of the other groups l We have to go into the Physics TDR with a baseline that we expect to last è And that will probably take a years work for a small team

17. VI/ Rome Nov 23 Slide 17 Current Activities Root Prototype è Prototype using as much as possible from Root without modifying current architecture è SimEvent (Tracks, Vertices, Hits) è Crossing Building (including 10 34 pileup) è Transient digitization l Technical (informal) forum (LHC experiments, Root, IT/DB) è Confront current architectures (and implementations) è Discover possible common components è Discuss “missing” functionalities in ROOT è Discuss alternatives l SC2 TAG è CMS will actively promote a common project on a HEP solution for a Data(Base) Management System