5/12/06T.Kurca - D0 Meeting FNAL1 p20 Reprocessing Introduction Computing Resources Architecture Operational Model Technical Issues Operational Issues.

Slides:

Advertisements

Similar presentations

GridPP July 2003Stefan StonjekSlide 1 SAM middleware components Stefan Stonjek University of Oxford 7 th GridPP Meeting 02 nd July 2003 Oxford.

Advertisements

4/2/2002HEP Globus Testing Request - Jae Yu x Participating in Globus Test-bed Activity for DØGrid UTA HEP group is playing a leading role in establishing.

6/2/2015 Michael Diesburg HCP Distributed Computing at the Tevatron D0 Computing and Event Model Michael Diesburg, Fermilab For the D0 Collaboration.

Sep Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.

Enabling Grids for E-sciencE Medical image processing web portal : Requirements analysis. An almost end user point of view … H. Benoit-Cattin,

The SAM-Grid Fabric Services Gabriele Garzoglio (for the SAM-Grid team) Computing Division Fermilab.

Hall D Online Data Acquisition CEBAF provides us with a tremendous scientific opportunity for understanding one of the fundamental forces of nature. 75.

High Energy Physics At OSCER A User Perspective OU Supercomputing Symposium 2003 Joel Snow, Langston U.

PCGRID ‘08 Workshop, Miami, FL April 18, 2008 Preston Smith Implementing an Industrial-Strength Academic Cyberinfrastructure at Purdue University.

Alexandre A. P. Suaide VI DOSAR workshop, São Paulo, 2005 STAR grid activities and São Paulo experience.

ISG We build general capability Introduction to Olympus Shawn T. Brown, PhD ISG MISSION 2.0 Lead Director of Public Health Applications Pittsburgh Supercomputing.

The SAMGrid Data Handling System Outline:  What Is SAMGrid?  Use Cases for SAMGrid in Run II Experiments  Current Operational Load  Stress Testing.

Remote Production and Regional Analysis Centers Iain Bertram 24 May 2002 Draft 1 Lancaster University.

Grid Job and Information Management (JIM) for D0 and CDF Gabriele Garzoglio for the JIM Team.

CDF data production models 1 Data production models for the CDF experiment S. Hou for the CDF data production team.

November 7, 2001Dutch Datagrid SARA 1 DØ Monte Carlo Challenge A HEP Application.

Building a distributed software environment for CDF within the ESLEA framework V. Bartsch, M. Lancaster University College London.

BINP/GCF Status Report BINP LCG Site Registration Oct 2009

D0 SAM – status and needs Plagarized from: D0 Experiment SAM Project Fermilab Computing Division.

MySQL and GRID Gabriele Carcassi STAR Collaboration 6 May Proposal.

Deploying and Operating the SAM-Grid: lesson learned Gabriele Garzoglio for the SAM-Grid Team Sep 28, 2004.

3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.

CHEP 2003Stefan Stonjek1 Physics with SAM-Grid Stefan Stonjek University of Oxford CHEP th March 2003 San Diego.

CHEP'07 September D0 data reprocessing on OSG Authors Andrew Baranovski (Fermilab) for B. Abbot, M. Diesburg, G. Garzoglio, T. Kurca, P. Mhashilkar.

Distribution After Release Tool Natalia Ratnikova.

8th November 2002Tim Adye1 BaBar Grid Tim Adye Particle Physics Department Rutherford Appleton Laboratory PP Grid Team Coseners House 8 th November 2002.

1 st December 2003 JIM for CDF 1 JIM and SAMGrid for CDF Mòrag Burgon-Lyon University of Glasgow.

Sep 21, 20101/14 LSST Simulations on OSG Sep 21, 2010 Gabriele Garzoglio for the OSG Task Force on LSST Computing Division, Fermilab Overview OSG Engagement.

SAMGrid as a Stakeholder of FermiGrid Valeria Bartsch Computing Division Fermilab.

GridPP18 Glasgow Mar 07 DØ – SAMGrid Where’ve we come from, and where are we going? Evolution of a ‘long’ established plan Gavin Davies Imperial College.

DØ Computing Model & Monte Carlo & Data Reprocessing Gavin Davies Imperial College London DOSAR Workshop, Sao Paulo, September 2005.

F. Fassi, S. Cabrera, R. Vives, S. González de la Hoz, Á. Fernández, J. Sánchez, L. March, J. Salt, A. Lamas IFIC-CSIC-UV, Valencia, Spain Third EELA conference,

26SEP03 2 nd SAR Workshop Oklahoma University Dick Greenwood Louisiana Tech University LaTech IAC Site Report.

São Paulo Regional Analysis Center SPRACE Status Report 22/Aug/2006 SPRACE Status Report 22/Aug/2006.

D0RACE: Testbed Session Lee Lueking D0 Remote Analysis Workshop February 12, 2002.

Dzero MC production on LCG How to live in two worlds (SAM and LCG)

June 10, D0 Use of OSG D0 relies on OSG for a significant throughput of Monte Carlo simulation jobs, will use it if there is another reprocessing.

09/02 ID099-1 September 9, 2002Grid Technology Panel Patrick Dreher Technical Panel Discussion: Progress in Developing a Web Services Data Analysis Grid.

4 March 2004GridPP 9th Collaboration Meeting SAMGrid:JIM and CDF Development CDF Accepts the Need for the Grid –Requirements How to Meet the Need –Status.

The SAM-Grid / LCG Interoperability Test Bed Gabriele Garzoglio ( ) Speaker: Pierre Girard (

Data reprocessing for DZero on the SAM-Grid Gabriele Garzoglio for the SAM-Grid Team Fermilab, Computing Division.

Stefano Belforte INFN Trieste 1 CMS Simulation at Tier2 June 12, 2006 Simulation (Monte Carlo) Production for CMS Stefano Belforte WLCG-Tier2 workshop.

February 28, 2003Eric Hjort PDSF Status and Overview Eric Hjort, LBNL STAR Collaboration Meeting February 28, 2003.

Jan 10, 2007, Clermont-Ferrand Tibor Kurca, Tutorial Grille1 DØ Computing Introduction - Fermilab & Tevatron & DØ Experiment DØ Computing Model 1. data.

High Energy FermiLab Two physics detectors (5 stories tall each) to understand smallest scale of matter Each experiment has ~500 people doing.

Integration of the ATLAS Tag Database with Data Management and Analysis Components Caitriana Nicholson University of Glasgow 3 rd September 2007 CHEP,

May Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.

UTA MC Production Farm & Grid Computing Activities Jae Yu UT Arlington DØRACE Workshop Feb. 12, 2002 UTA DØMC Farm MCFARM Job control and packaging software.

Worldwide Data Processing with SAMGrid. As experiments refine their understanding of raw data, a point is reached where it becomes desirable to reanalyze.

ISG We build general capability Introduction to Olympus Shawn T. Brown, PhD ISG MISSION 2.0 Lead Director of Public Health Applications Pittsburgh Supercomputing.

Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.

December 07, 2006Parag Mhashilkar, Fermilab1 Samgrid – OSG Interoperability Parag Mhashilkar, Fermi National Accelerator Laboratory.

April 25, 2006Parag Mhashilkar, Fermilab1 Resource Selection in OSG & SAM-On-The-Fly Parag Mhashilkar Fermi National Accelerator Laboratory Condor Week.

Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,

UTA Site Report Jae Yu UTA Site Report 7 th DOSAR Workshop Louisiana State University Apr. 2 – 3, 2009 Jae Yu Univ. of Texas, Arlington.

July 26, 2007Parag Mhashilkar, Fermilab1 DZero On OSG: Site And Application Validation Parag Mhashilkar, Fermi National Accelerator Laboratory.

Jianming Qian, UM/DØ Software & Computing Where we are now Where we want to go Overview Director’s Review, June 5, 2002.

The SAM-Grid / LCG interoperability system: a bridge between two Grids Gabriele Garzoglio, Andrew Baranovski, Parag Mhashilkar Anoop Rajendra*, Sudhamsh.

Apr. 25, 2002Why DØRAC? DØRAC FTFM, Jae Yu 1 What do we want DØ Regional Analysis Centers (DØRAC) do? Why do we need a DØRAC? What do we want a DØRAC do?

CERN LCG1 to LCG2 Transition Markus Schulz LCG Workshop March 2004.

DØ Grid Computing Gavin Davies, Frédéric Villeneuve-Séguier Imperial College London On behalf of the DØ Collaboration and the SAMGrid team The 2007 Europhysics.

Real Time Fake Analysis at PIC

LCG 3D Distributed Deployment of Databases

Monte Carlo Production and Reprocessing at DZero

Production Resources & Issues p20.09 MC-data Regeneration

Simulation use cases for T2 in ALICE

SA1 ROC Meeting Bologna, October 2004

Patrick Dreher Research Scientist & Associate Director

Tibor Kurca, Tutorial CPPM

DØ MC and Data Processing on the Grid

Presentation transcript:

5/12/06T.Kurca - D0 Meeting FNAL1 p20 Reprocessing Introduction Computing Resources Architecture Operational Model Technical Issues Operational Issues - Status Tibor Kurča IPN Lyon

5/12/06T.Kurca - D0 Meeting FNAL2 Introduction Goal: reprocess ~500 M RunIIb events (83 TB) with newly calibrated detector & improved reconstruction software by end of March ‘07 Where: SAMGrid/OSG & native SAMGrid (CCIN2P3) Issues: SAMGrid –OSG new environment - missing experience - fast problems identification  solution ? - importance of organization, preparation  efficiency ?

5/12/06T.Kurca - D0 Meeting FNAL3 Computing Resources Needs: 2000 CPUs, 4 TB disk cache, 1Gb links Available : #CPU (not guaranted) Disk Cache Oklahoma U TB Indiana U TB Nebraska U 250 CMS –FNAL 250 Fermilab 4 TB NERSC 250 SPRACE 250 Purdue ? Florida ? CCIN2P3 500 (non-OSG, SAMGrid)

5/12/06T.Kurca - D0 Meeting FNAL4 Basic Architecture SAM-Grid OSG SAM-Grid / OSG Forwarding Node SAM-Grid VO-Specific Services Flow of Job Submission Offers services to … Main issues to track down: Accessibility of the services Usability of the resources Scalability New SAM

5/12/06T.Kurca - D0 Meeting FNAL5 Current Configuration FW SAM- Grid C S FW C S Network Boundaries Forwarding Node LCG Cluster SAM Stager VO-Service (SAM) Job Flow Offers Service - new SAM FNAL C IU C C SPRACE UNL NERSC stg CMS C OU stg C

5/12/06T.Kurca - D0 Meeting FNAL6 Operational Model Production & Merging: - production – reconstruction at each site  unmerged TMBs at FNAL - merging preferentially at FNAL Organization - define submitter teams à 2 person - assign datasets to each team - define primary resp. secondary OSG clusters for each team where they should submit - submission from central UI installed on d0mino Operation Problems Solution - multilevel expertise - identify problem : SAMGrid or OSG SAMGrid: contact d0_reprocessing - official OSG way : open ticket at GOC-Indiana - contact directly local administrators (if 1st way not working)

5/12/06T.Kurca - D0 Meeting FNAL7 Multilevel Expertise Submitters/shifters: to submit, check logfiles  report problems Mandy ROMINSKY, Sohrab HOSSAIN University of Oklahoma Joseph STEELE Louisiana Tech University Yanwen LIU University of Science and Technology of China Dag GILLBERG, Zhiyi LIU Simon Fraser University, Canada Experienced people: first aid Daniel, Joel, Mike, Tibor …. & others SAMGrid experts: problem solving, intervention Andrew, Gabriele, Parag OSG experts /local administrators: OSG related issues …we have established contacts ; … (subject should mention dzero reprocessing)

5/12/06T.Kurca - D0 Meeting FNAL8 Technical Issues Tests done: OSG clusters & CCIN2P3 tested deployment of storage queues done  working To be done !: - central UI installation on d0mino node jim_client, d0repro tools …. Most urgent, UI for all submitters Issue: - binary input RTE-file size ~600 MB !!! …. To be shipped with each job … 2x raw data file size!!!  ???? To reduce it very desirable !!!!!!

5/12/06T.Kurca - D0 Meeting FNAL9 Operational Issues - Status Relevant information at Grid certificates: most of the submitters have their DOEGrids certificates user account at NERSC ….. Procedure started Test runs  submitters training  hands on experience - jobs submission using d0repro tools - where to look for logs  to be done ! …this Thursday? d0mino UI ?  large scale test -scalability issues? … early next week Production start - early January ?