1. An Overview of the BaBar Conditions Database
Igor A. Gaponenko
Lawrence Berkeley National Laboratory
( )

2. Introduction…
The Conditions/DB has been developed in the context of the BABAR experiment at the Stanford Linear Accelerator Center.
Basic requirements:
The database is meant to store:
detector alignments;
calibrations constants;
Other time-dependent records, under which the experimental events are taken.
A possibility to have multiple versions of conditions data;
The data are recorded every 30 minutes (in average).
The technology:
Is coherent with the general trends in the BABAR software:
Provides an API for applications in C++
Uses Objectivity/DB as the underlying storage technology
An overview of the design, the current status and gained experience are presented in this talk.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

3. The Functional Definition
An abstraction - the Conditions Database is a collection of so called conditions functions:
2 access keys:
validity time
revision
Provides: condition objects
The definition of validity time:
Is implemented as 64-bit unsigned integer (only 32 uppermost bits are used)
The granularity: 1 second
Covers a period from logical minus infinity (Jan 1, 1900 UTC) to logical plus infinity (+232 seconds)
The secondary key is revision
Allows to choose among different versions of the same condition
Is implemented as 32-bit unsigned integer
Each condition function has its own set of revisions
At the highest level of abstraction the Conditions database may be viewed as a collection of so-called conditions functions providing the values (represented as persistent objects) for the conditions parameterized by the validity time, which serves as the primary access key. A 64-bit unsigned integer is used to specify the validity time, although access granularity is determined by the uppermost 32-bits, corresponding to a resolution of one second.
Each conditions function covers the whole history of the corresponding condition ranging from a logical minus infinity (starting from January 1, 1900, UTC) through a logical plus infinity, thus covering the period of 232 seconds. Internally this history is split onto a sequence of intervals, each covering the period where the value of the condition is constant.
The possibility of multiple versions of conditions for the same validity time is also available in the Conditions database. This requires the use of a secondary key for the conditions function specified above. This vertical dimension is called the revision.
Any modifications to a conditions function are also recorded in a special persistent journal, which is specific to each function. This provides for better control over modifications of the functions as well as providing for the possibility of rolling back accidental modifications when required.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

4. The Structural Components
Users’ code
Management Utilities
Proxies
Transient cache
Data store & fetch API
Management API
This slide represents the Conditions/DB as a layer in the BaBar software.
The following components are shown:
The specially structured persistent store (based on the Objectivity/DB) as a data store;
The core code of the Conditions database software implemented as C++ and DDL classes grouped into three software packages;
A layer of proxy classes between the application and core interfaces of the database;
A number of management tools (utilities).
Generally there are three different types of applications:
Applications loading data into the database. These work directly through the core API.
Applications fetching conditions from the database during event reconstruction or detector calibration. This code typically accesses data through proxies;
Special management utilities managing the contents of the database. A special management API is provide for these.
Proxies are responsible for a number of ancillary tasks, including transaction management, conversion of conditions objects from a persistent to transient representation, and dealing with schema compatibility issues. The core packages provide just a base class for proxies, leaving the specific implementations of them up to the developers of final (detector-specific) proxies.
In order to improve the overall performance of the database two levels of caching have been implemented in the Conditions database software. The first level, built into the core classes, caches the most recently accessed persistent handles pointing to the database object, container object and a condition object itself. This cache is used by applications fetching the data from the database. Another level of caching deals with the transient representations of conditions objects inside proxies. All of this effectively reduces the number of the “real” I/O operations for the most frequent data access patterns - when the condition values are accessed sequentially at the validity time.
Transient API
Persistent cache
Persistent Store (Objectivity/DB)
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

5. The Database API
The main functions provided through the Conditions database API are:
A two-layered namespace for the condition functions:
A detector is used as a namespace for conditions;
A detector corresponds to a group of the BaBar Authorization System.
The “store” interface allows storing either:
a single value for a condition for a specified validity interval;
or a vector of conditions.
The “fetch” interface is meant to retrieve stored data (values of condition functions) from the persistent store.
It’s also possible to iterate in both dimensions in the history of a specific condition.
The revision (vertical dimension in the condition function) interface. This interface supports the creation and management of revisions.
An extensive management API is intended to manage the contents of the database:
Deletion, renaming, copying and verifying of condition functions;
Browsing the contents of database;
Etc.
This slide classifies the functionality provided through the Conditions/DB API.
The functionality falls into five major groups.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

6. Two-layered Namespace
<valueA> = FA ( <validity time>, <revision> )
<valueB> = FB ( <validity time>, <revision> )
<valueC> = FC ( <validity time>, <revision> )
. . .
<valueZ> = FZ ( <validity time>, <revision> ) A B C
EMC T V
All the conditions functions have symbolic names in the database. According to the adopted convention these names must be the names of the corresponding persistent (DDL) classes.
The functions are grouped into detectors, forming two layered name space, made from detectors and function names. Each detector group has independent set of names.
Another important role of detector names is that they also serve to the data protection purposes. We use the BaBar Authorization System to authorize access to the Conditions/DB. In the current implementation each detector corresponds to a group of users in the Authorization System, thus providing all registered members of this group with equal rights to modify the contents of any conditions functions inside the detector.
Conditions functions
DCH Y Z
Detectors
ETC
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

7. A Layout of the Persistent Store
Link database
Index database(s)
Object database(s) A A B C B
A sample layout of persistent objects for a particular detector is show on this slide.
Internally, in the persistent store of an Objectivity federation, the conditions data are spread between three types of databases: link, index and object.
The link databases serve as catalogues (namespaces) for the conditions functions. There is one link database per detector group. Each named container in these databases corresponds to a single condition function. This container has a symbolic link to a persistent container in the index database.
The index databases maintain the time and revision history or shape of the condition functions, including all bookkeeping information for each function. As in case of the link databases, there is one-to-one correspondence between a condition function and a persistent container. There may be more then one such database per detector, each corresponding to a different origin (a federation where the corresponding condition function was created and is managed).
The third type of databases is used for storage of the condition objects themselves, providing the specific conditions information. The objects are referenced through the index databases.
Such a multi-layered architecture provides more efficient clustering of the Conditions data, and subsequently faster access to those data. This separation for example makes possible browsing the history of a specific condition (the shape of the condition function) without loading huge amount of data from the object databases when it’s not needed. C
A namespace for conditions.
Each container has a symbolic link.
Each container keeps a history of a particular condition.
“Real” condition objects are stored in here.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

8. The File System of the Database
...conditions/
DOMAIN
emc/ ...
drc/
DETECTOR
con_drc_Link
con_drc_Index_core
con_drc_Index_opr-core
drc/
A sample layout of database files is show on this slide.
There is a single root catalogue “conditions” which keeps a catalogue of known detectors.
Each detector catalogue has a “link” database, and at least one “index” database, whose name ends with the “origin” of the database.
The “origin” indicates a federation where the enclosed condition functions have been created and are maintained.
The tree remaining databases on the bottom of the slides represents so called “object” databases, the ones which keep the conditions objects.
drc /
con_drc_drc002000
con_drc_drc002001
con_drc_drc002002
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

9. A plain Condition Function
Validity time
“Plain” history of a calibration object:
FIRST
LAST
-Infinity
+Infinity
Characteristics:
In this trivial case the condition function is parameterized by a single parameter – validity time.
The history of a condition object is divided onto intervals: [begin, end).
Each interval covers a period in the validity time where the value of the condition is constant.
The intervals are connected into a linked list.
The internal logical structure of a persistent interval container, containing the shape of a simple condition function is shown on this slide.
This particular function does not have any (vertical) versions for the condition. Therefore its (function) value depends on the validity time only.
The history of the condition object in a horizontal dimension is divided onto “intervals”. Each interval covers a period of validity time where the value of the condition function is constant.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

10. A versioned Condition Function
“baseline” intervals are shown in blue.
“Versioned” history of a calibration object: V4 V2 V3 V1 V1 V2
FIRST V0 V0
LAST
-Infinity
+Infinity
Characteristics:
The intervals above the baseline level are called versions.
Each interval may have many version within its validity time limits.
The versions are organized into trees.
Each tree grows from a baseline interval.
Here is more complicated structure of a condition function. This function allows so called “versions” for conditions for the same validity time.
The versions grow in vertical dimension within validity limits of their parent interval.
The versions are organized into trees, growing from the corresponding baseline interval. The nodes of trees are connected by bi-directional links.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

11. A Concept of Revisions Revisions: Characteristics:
FIRST V0 V0 V0
LAST
-Infinity
+Infinity
revision 3
revision 2
Revisions:
revision 1
“baseline”
A concept of the so called revision is presented on this slide.
The definition:
The revision is defined as a collection of intervals;
Each revision, except the baseline one, should have a base revision;
In some sense a revision may be viewed as a patch to their base revision (this is a recursive definition);
Each revision with their direct and indirect base revisions provides a complete timeline for the corresponding condition.
Each revision has a number (32-bit unsigned integer).
The revision is used as secondary keys to the condition functions.
Characteristics:
A revision is a second (vertical) key of the condition function.
A revision is made of intervals. Each revision, except a baseline one, has a base revision.
Each revision with their direct and indirect base revisions provides a complete timeline for the condition.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

12. The Persistent Schema (simplified)
BdbPString
Symbolic links
Link databases
BdbRegistry
1..n 1
BdbRevision
0..n
0..1
BdbIntervalR
Condition functions
Index databases
A simplified diagram of persistent classes is show. Some of the classes (in particular those ones, providing the bookkeeping) are omitted at the diagram.
The objects of these classes are spread between tree persistent database: link, index and object.
Object databases
BdbObject
Condition objects
Condition A
Condition B
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

13. The Modification History
[n ] store( Obj3, t2, +oo )
[n+1] store( Obj4, t3, +oo )
[n+2] store( Obj5, t4, +oo )
[n+3]
+oo
-oo t1 t2
Obj2 t3 t4
Obj3
Obj4
Obj5 t0
Obj1
State I:
State II:
Operations
State III:
Obj6
[n+4] store( Obj6, t3, t4 )
[n+5]
Another important design solution which has been taken was to implement a kind of persistent bookkeeping system for condition functions. In the Conditions/DB API this is called the “history”.
The history data structures provides independent bookkeeping records for every condition function. All activities modifying the shape of conditions functions are recorded in the history. These records may be used for various purposes, including:
A possibility to analysis a cause of the wrong contents of the database;
A possibility to roll-back the shape of the condition function to one of its previous states.
This slides illustrates how the contents of a particular interval container evolves between three sequential states as new data are recorded into the database.
The first state (State I) represents the initial contents of the container. This contents changes to the next state (State II) after three operations shown in brackets. The last state is a product of an operation applied to the previous state.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

14. Current Status and Plans
The status:
The Conditions database has been in production use since May 1999;
The total amount of data stored in the Conditions database has reached 2 GB;
There are about 250 different types of conditions grouped into 10 detectors;
The database software has gone through a number of performance improvements and extensions in its functionality.
Further development:
More (and better) management tools are needed;
Working on a Java-based GUI browser to simplify management and visualization of the conditions data;
Considering a CORBA-based implementation of the Database:
To decouple clients from a particular federation;
To make management easier.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

15. Problems, Experience, etc…
Performance/space issue:
Having multiple (more than a few 10-s) of vertical revisions for a particular condition function does seem to create performance problems:
“vertical” optimization is needed
A very detailed (highly granulated) history of a particular condition may cause a persistent space problems:
Splitting of the history between more than one persistent container
A problem of composite objects (see the next slide):
Essential part of (250) conditions classes represent so called composite objects:
These objects are hard to manage: copy, move, delete
Objectivity/DB does not provide a good solution for this
Troubles with incremental data distribution of these data around Collaboration
Our solutions (are still under development):
Deep copy interfaces in a base class BdbObject;
Deep deletion interface in the base class.
This two slides of the talk are mainly intended for the follow up discussion.
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000

16. Problem of “composite objects”
ClassA
Step I
ClassA
Step II
The main problem is that the Conditions/DB API deals with the handles of the root objects of the composite trees. The complete structure of these objects is difficult to manage.
The branches have other type then
ClassA. They are not being copied
Igor A. Gaponenko: An Overview of the BaBar Conditions Database
February 10, 2000