M.Frank LHCb/CERN - In behalf of the LHCb GAUDI team Data Persistency Solution for LHCb ã Motivation ã Data access ã Generic model ã Experience & Conclusions

M.Frank LHCb/CERNGAUDI Motivation ã Physics software should be independent of the underlying data storage technology ã Data of different nature has to be accessed ä Event data, detector data, statistical data, … ä The access patterns differ ä The data set size varies from several Mbytes to 1 Pbyte ä Legacy data was written in ZEBRA format ã It is unclear how these data will be stored ä Locking into one technology may be a disadvantage

M.Frank LHCb/CERNGAUDI Strategy ã Transient data representation is separated from the persistent data representation ä Each representation can be optimized separately ä Transient representation can be used to convert to any other representation ã Minimize coupling between algorithms and the transient data ä Algorithms see only transient data ä Transient data items are not intelligent ä Algorithms post and retrieve transient data from a “black-board”, the data store

M.Frank LHCb/CERNGAUDI How are Data Accessed? Event Data Service Transient Event Store Detec. Data Service Transient Detector Store Histogram Service Transient Histogram Store Algorithm

M.Frank LHCb/CERNGAUDI Functionality of Data Stores ã Manage data objects like a librarian ä Clients store objects ä Other clients pick up objects when needed ä Retrieve object collections ã Manage ownership ä Does cleanup ã Manage objects of similar lifetime

M.Frank LHCb/CERNGAUDI Structure of the Data Store ã Tree - similar to file system ã Tree node ä has data members (payload) ä contains other node objects (directory structure) ã Identification by logical addresses: ”/Event/Mc/MCParticles” ã Browse capability

M.Frank LHCb/CERNGAUDI Symbolic links C D /node2 /node2/D /node2/C Data Members Data Object Layout of the Data Object /node1 /node1/B /node1/A Data Members Node objects /node1/A/A2 /node1/A/A1 Data Object

M.Frank LHCb/CERNGAUDI Store Dynamics: Loading (5) Register Data Service Algorithm (1) retrieveObject(…) Try to access an object data (2) Search in Store Data Store Will be unsuccessful, requested object is not present Converter (4) Request creation (3) Request load Persistency Service Conversion Service Request dispatcher Objy, SICB, ROOT,..

M.Frank LHCb/CERNGAUDI Store Dynamics: Storing Converter Conversion Service (2) Create persistent representations Persistency Service Output Stream Criteria (1) Collect references Data Store

M.Frank LHCb/CERNGAUDI Generic Persistent Model Objects & pointersObjects, object IDs, collections & DBs Transient Persistent C++ pointer >> object ID

M.Frank LHCb/CERNGAUDI Database Technologies ã Identify commonalties and differences Necessary knowledge when reading/writing ã RDBMS: More or less traditional ã Objy is different: How to match ?

M.Frank LHCb/CERNGAUDI Generic Model: Assumptions Data Store ConversionSvc Disk Storage database Database Container Objects Class ID, Object path Storage type DB name Cont.name Item ID

M.Frank LHCb/CERNGAUDI Converter Technology XID (2)(3) (4) (1) Link ID Link Info DB/Cont.name, Lookup table Generic Model: References

M.Frank LHCb/CERNGAUDI Generic Model: Extended Object ID l Storage Type l Class ID l Storage Type dependent part OR l OID l Link ID l Record ID Objectivity ZEBRA ROOT RDBMS

M.Frank LHCb/CERNGAUDI Experience & Conclusions ã It is possible to write physics data without knowledge of the underlying store technology ã Our approach can adopt any technology based on database files, collections and objects within collections ä ZEBRA, ROOT, Objy and RDBMS ä We are able to choose technologies according to needs ã Overhead of transient-persistent separation looks manageable

M.Frank LHCb/CERNGAUDI Object Evolution ã Handling of dictionary discrepancies between the application and the database ã “Generic” handling? ä default values ? ã Architectural problem ä Handled inside “Converters” ä Better chance to supply reasonable values Database Dictionary Application Broker Dictionary