ATLAS DAQ Configuration Databases CHEP 2001September 3 - 7, 2001 Beijing, P.R.China I.Alexandrov 1, A.Amorim 2, E.Badescu 3, D.Burckhart-Chromek 4, M.Caprini 3, M.Dobson 4, R.Hart 5, R.Jones 4, A.Kazarov 4,8, S.Kolos 4,8, V.Kotov 1, D.Liko 4, L.Lucio 2, L.Mapelli 4, M.Mineev 1, L.Moneta 6, M.Nassiakou 4, L.Pedro 2, A.Ribeiro 2, Yu.Ryabov 7, D.Schweiger 4, I.Soloviev 4,8 1) Joint Institute for Nuclear Research, Dubna, Russia 2) University of Lisboa (FCUL), Lisbon, Portugal 3) Institute of Atomic Physics, Bucharest, Romania 4) European Organization for Nuclear Research (CERN), Geneva, Switzerland 5) National Institute for Nuclear Physics and High Energy Physics (NIKHEF), Amsterdam, Netherlands 6) Physics Section, University of Geneva, Geneva, Switzerland 7) Petersburg Nuclear Physics Institute (PNPI), Gatchina, St. Petersburg, Russia 8) On leave from 7. The Trigger & Data Acquisition System for the future LHC ATLAS experiment needs several databases to store large number of parameters describing architecture, hardware and software components, running modes and the system status. Due to large number of subcomponents a unique approach to databases is considered essential. In accordance with the User Requirements these databases must:  store multiple versions of data and support schema evolution;  provide an application programming interface (API) and graphical user interface (GUI) to store, retrieve and navigate the databases contents;  be available on front-end computers running real-time operating systems and do not degrade the performance of the applications running on them. Introduction Choice of Persistent Object Managers During the Prototype –1 Project the ATLAS DAQ group evaluated various commercial and shareware data persistence systems (relational and object databases, object managers) but no single system satisfied all the documented user requirements. As a consequence, a two-tier architecture was adopted:  a light-weight in-memory persistent object manager is used to support the real-time requirements;  an object database management system was chosen as a back-up solution and for long-term data management. The possibility to exchange data between such managers was tested with the Objectivity/DB and the OKS (see below), but only the OKS implementation is currently used. For the run-time object manager a package called OKS (Object Kernel Support) has been developed on top of Standard C++ Library. The OKS system:  is based on an object model close to object models supported by commercial persistent object management systems;  stores database schema and data files in XML format (which also can be understood by non- OKS applications);  allows different schema and data files to be merged into a single database in-memory;  provides a C++ API, read-only remote CORBA API, Motif-based GUIs to design schema and manipulate OKS objects;  has several utilities to verify and compare database files. Interfaces The C++ API includes direct access of database contents via OKS API and also via C++ data access libraries (DAL). The DALs are used to hide the details of the low-level persistent object manager from programmers and details of database schema (i.e. map schema to “application view”). A programmer needs to write less code using a DAL then using the native API. The DALs are:  “federation loader” loads database files describing partition and provides consistency checks;  “data-flow DAL” maps database schema to data-flow group application view;  “software DAL” maps database schema to online group view of partition’s software;  “generated C++ binding” maps database to C++ objects accessible via get/set…() methods. The CORBA API is used for remote read-only access of configuration databases. It allows to access the databases from any computer connected via Internet from applications written in different languages (e.g. it is used to access the databases from the Java based Integrated DAQ GUI). The CORBA API is implemented on top of ILU (freeware CORBA implementation) and consists of a C++ server using native OKS API and an IDL interface that allows clients to be written in different languages and access databases world-wide. There are several GUI applications providing interfaces to edit the databases. To design the configuration databases schema it is possible to use the native OKS Schema Editor or StP Object Model Editor (for the latter there is a script to convert StP classes to OKS classes). To edit the configuration databases data it is possible to use the OKS Data Editor or dedicated end-user ConfDB GUI application, shown below: The database objects can be presented in the form of icons or tables where any attribute or any link with other objects can be edited. Similar objects are grouped in dedicated windows (e.g. software, hardware, partition). The knowledge about a mapping between configuration database classes and where and how they are presented by the editor are stored in an external XML configuration file. Going hand-in- hand with extendable schema, a user may add his own classes and change accordingly the configuration file to add or redefine graphical windows and their contents (i.e. description of graphical objects including icons, visible attributes and relationships, titles). C++ Application Programming Interface Remote Access Graphical User Interfaces Database Files Structure The DAQ configuration database files are organized into a tree-like structure that represents the actual hierarchy of the DAQ system. For a particular run the database files are grouped into a single federated database (objects may have inter-files references). Each subdirectory belongs to a subgroup and defines it’s specific classes and objects. The typical information provided by each subgroup is:  schema classes designed by the subgroup (normally inherited from the core classes provided by the online group);  software repository containing the description of subgroup’s binaries (such information is updated in case of software release changes);  hardware description describes the subgroup’s detector structure, crates, modules, computers, links and any other physical objects (such information is updated in case of physical changes like inserting of a new module);  available partitions (a partition describes what software, hardware and other partitions are used for a given run). To improve maintainability a partition describing a large configuration consists of nested partitions describing smaller parts of the system. A leaf of the partition tree describes some small part of the system like a single crate. Also there is a naming convention describing what identities should be given to objects created by different groups to avoid clashes during assembly of the federated database and consistency rules defining expected contents of database files. Such techniques permit the creation of partitions containing more than 1,000 applications running on a hundred computers used during recent online performance and scalability tests, which corresponds to the size of expected final configuration of ATLAS experiment in Top-level partition with 1111 controllers 10 detector partitions Top-level controller Detector partition with 111 controllers 10 base partitions Detector controller Base partition with 11 controllers 10 crate controllers Detector subsystem controller Implementation Details The configuration databases are available on platforms supported by DAQ online:  Red Hat Linux 7.1 (kernel 2.4.2) with g  Red Hat Linux 6.1 (kernel ) with egcs  Sun Solaris 2.6 with egcs  Sun Solaris 2.7 with SunPro CC-5.2  Lynx OS with g gnupro-98r2 The OKS and configuration databases DALs are written in C++ using the C++ Standard Library and store data files in human-readable XML files. The GUI applications require Motif 1.2 or higher (freeware implementation of Motif named LessTif can also be used). No commercial software is required to build or use the databases. Both the OKS and the configuration databases are available as a part of the ATLAS Online Software which can be downloaded from Several tests and benchmarks were performed on configuration databases to evaluate the performance for different configurations, to test compatibility of OKS with commercial persistent object managers and to prove databases reliability (007 benchmark). These tests have showed expected and acceptable performance up to the largest tested configurations, reliability of implementation and compatibility of the OKS object model with commercial persistent object management systems. Two fragments of the configuration databases performance test results from a minimal up to the expected final ATLAS DAQ configuration (see grey area below corresponding to 50, ,000 objects) are presented below: The tests were performed on the following hardware:  sun - Sun Ultra-Enterprise 3000 (167 MHz, 2 CPUs, 512 MB RAM) running Solaris 2.6  pc – Pentium III based PC (800 MHz, 128 MB RAM) running Red Hat Linux 7.1  ppc – PowerPC 600 on VME 8062EA (200 MHz, 16 MB RAM) running Lynx OS Load databaseDump database contents Plans for Future Work The chosen architecture covers all requirements for the DAQ configuration databases (high performance, availability for real-time operating systems running on VME, GUI tools, data and schema versioning) and provides a way to exchange configuration data with other applications. The plans for future work include:  closer integration with all Trigger/DAQ sub-systems;  additional DALs required by the sub-systems (e.g. the hardware DAL);  partition templates and partition comparison tools;  making partition descriptions available via Internet browsers. Created by Igor Soloviev, CERN, 2001 Further info: Base partition is stored in 1 file. Detector partition is composed of 11 files. Top-level partition is composed of 111 files. C++GUIXMLCORBA Open Source C++GUIXMLCORBA