Presentation is loading. Please wait.

Presentation is loading. Please wait.

Status and plans for bookkeeping system and production tools

Similar presentations


Presentation on theme: "Status and plans for bookkeeping system and production tools"— Presentation transcript:

1 Status and plans for bookkeeping system and production tools
Eric van Herwijnen Thursday, 19 september 2002

2 Contents Status remote centres & datachallenge plans
DataGrid status and plans Data management and production tools Job submission Bookkeeping Configuration Data production and monitoring Roadmap and conclusions

3 Status remote centres Center No. of CPU’s (1 GHz) Production tools
CERN ~ 400 new Lyon 60 + Liverpool ~ 120 Imperial College ~ 100 DataGrid ~ 20 RAL ~ 300 old Bologna ~ 200 Nikhef Bristol Edinburgh Cambridge ~ 15 Oxford ~ 10 Moscow ~ 40 Rio Total ~ 1000 (outside CERN)

4 Physics Data Challenge plans
MC production = Physics Data Challenge Available capacity seems to match requirements Planning: Preproduction: mid Dec 2002 – mid Jan 2003 Production: Feb – May 2003

5 DataGrid status and plans
Installation operational Long job problem fixed Long file transfer problem (~ 1 Gb) New production tools being installed Test: Run 500 event MC generation Store on SE Recover logs and histograms to CERN Run reconstruction. Output to SE. Recover log files and histos. Write recon output to mass store (Castor) Read Castor data with an analysis job outside Grid

6 Data Management

7 Job submission New tools designed and written by A. Tsaregorodtsev
Site dependencies concentrated in 3 scripts Simple installation procedure: AFS independent Standard directory structure No (or very little) java More flexible updating of bookkeeping database Remote centers should migrate to new system

8

9 Job submission, future Data Production Job.opts DB Bookkeeping DB
Production done Modify Create job(s) script Prod.Mgr Configuration DB Build new configuration Selection of Defaults Information Flow

10 Job submission, future Workflow to be implemented (template scripts for each step, parameters added via web page) Prototype exists (M. Frank)

11 Bookkeeping

12 Bookkeeping Most components ready (S. Ponce, F. Loverre)
Structure of database independent of tools, easy to add new datatypes, identify productions, replicas of datasets Database (Oracle) filled by a Java server via XML files API implemented in Java and Python Migration to new database transparent for user Final tests under way before production

13 Configuration Prototype of configuration database exists
Need to integrate with job submission tools (5 line python script?) Prototype of GUI to view the database developed by G. Klamke (summer student) Need to create tool to add configurations to database, and integrate with Ganga

14

15 Data Production and Monitoring

16 Data production and monitoring
Current PVSS system needs to be brought beyond the prototype stage Cosmetic changes, cleanup of dead entries, speed Adapt to new job submission tools Alarms should really be alarms, add corrective action Migrate to new version of PVSS Web interface Production database + interface need to be created Data quality check tools need to be reviewed and created Ongoing activity

17 Roadmap and conclusions
Installation of new job submission tools: oct-nov 2002 New bookkeeping db: dec 2002 Integration of job configuration db with job submission tools: feb 2003 Integration of job configuration db with Ganga: summer 2003 Creation of data production db, integration with monitoring system: summer-fall 2003


Download ppt "Status and plans for bookkeeping system and production tools"

Similar presentations


Ads by Google