1. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 1 D0 Luminosity Database: Checklist for Production Elizabeth Gallas Fermilab Computing Division / D0 Computing and Analysis Group for the Luminosity Database effort within the The DØ Luminosity Group DØ Data Quality Meeting November 1, 2004

2. ElizabethGallas -- D0 Luminosity Db 2 Luminosity Database Schema Sorry, I have to Put the picture here. It is not here yet.

3. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 3 The Luminosity Database contains information from many sources: Luminosity Monitor Detector Trigger Framework Level 3 Fermilab Tevatron Accelerator Offline Data Processing (SAM) QUALITY flags –“Group” assigned data quality STATUS flags –“Consistency” determined quality Correction factors by –LBN –LBN range

4. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 4 LDB Project Components / Status Database Schema (table definitions) –Passed DB Schema review Oct 7, 2004 –Status: Instantiated on d0ofdev1 in Oct 2004 Loading DATA –Status: on d0ofdev1 with 6  14 weeks of data from Online (exposure by stream, trigger, tick…) –took 2 days. Crashed the machine only once (log files) Offline –Quality tables (Dan) –File processing tables (Marco) lmAccess –Calculating luminosity from the database for data samples by trigger using CORBA based database server –Status: Functional. Working on performance. Operations Reports –Dynamic web reports from the database using python By Run, Store, and time period (Daily, Weekly…) –Status: Functional. Working on performance. Group assigned QUALITY –Status: All data loaded, all interfaces converted

5. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 5 LumDB: Going to production. Summary  It is a big system with many components that all must be ready to declare a ‘production’ mode. We have made a lot of progress on all tasks !!! (next slides – look for checks)  Many items must be resolved before we reach production mode !!! (next slides – look for arrows)

6. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 6 LumDB: Going to production. Database readiness Defining the Tables, Views Database Schema – ready for production Added critical disk space in October 2004  Size is comparable to SAM Loading existing Run 2 data Automated loading scripts are working  A few STATUS flags remain to be identified  But we need to ‘manage’ online disk space (data is loaded using flat files created from flat files)  We consider triggers and constraints critical DB Support must stage Oracle archive logs to prevent system crash while 3 years of data being loaded We can’t load the data too quickly  We consider backup capability to be critical Trying to find a solution between our groups (DB Support, D0 Database Group, Luminosity Group) Possible solution: ‘partitioning’ read-only data Loading Run 2 data in real time  Programs need to be modified to load data  If we can load it soon after data taking, the information can be used for online diagnostics

7. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 7 LumDB: Going to production. Interface readiness lmAccess Functional using the database/server  In the process of being certified for production Operations Reports (web based) Functional  In the process of being certified for production  Implement caching of commonly requested reports (?) Data Quality Component First component to be ready for production (thanks Dan!): Ready to load data (as soon as online data is loaded) Interfaces are Functional No performance issues

8. November 1, 2004 ElizabethGallas -- D0 Luminosity Db 8 Luminosity DB Summary All components of the application are close to being production ready … but … Estimate>20 days to load 3 years of data –not including the time needed to make the load files ! (multiply this time by 2 or 3) only THEN can we fully address: –performance and –final certification of components during which time both systems will be available, except for the Data Quality component where: –Once we switch, there is no going back !!!