LCG 3D StatusDirk Duellmann1 LCG 3D Throughput Tests Scheduled for May - extended until end of June –Use the production database clusters at tier 1 and.

Slides:

Advertisements

Similar presentations

Replication solutions for Oracle database 11g Zbigniew Baranowski.

Advertisements

CERN - IT Department CH-1211 Genève 23 Switzerland t LCG Persistency Framework CORAL, POOL, COOL – Status and Outlook A. Valassi, R. Basset,

RLS Production Services Maria Girone PPARC-LCG, CERN LCG-POOL and IT-DB Physics Services 10 th GridPP Meeting, CERN, 3 rd June What is the RLS -

CERN - IT Department CH-1211 Genève 23 Switzerland t Relational Databases for the LHC Computing Grid The LCG Distributed Database Deployment.

Outline Network related issues and thinking for FAX Cost among sites, who has problems Analytics of FAX meta data, what are the problems  The main object.

CERN IT Department CH-1211 Genève 23 Switzerland t Streams new features in 11g Zbigniew Baranowski.

Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.

Database monitoring and service validation Dirk Duellmann CERN IT/PSS and 3D

Database Services for Physics at CERN with Oracle 10g RAC HEPiX - April 4th 2006, Rome Luca Canali, CERN.

CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.

LCG Database Workshop Summary and Proposal for the First Distributed Production Phase Dirk Duellmann, CERN IT (For the LCG 3D Project :

SC4 Workshop Outline (Strong overlap with POW!) 1.Get data rates at all Tier1s up to MoU Values Recent re-run shows the way! (More on next slides…) 2.Re-deploy.

RLS Tier-1 Deployment James Casey, PPARC-LCG Fellow, CERN 10 th GridPP Meeting, CERN, 3 rd June 2004.

CERN - IT Department CH-1211 Genève 23 Switzerland t Tier0 database extensions and multi-core/64 bit studies Maria Girone, CERN IT-PSS LCG.

ATLAS Metrics for CCRC’08 Database Milestones WLCG CCRC'08 Post-Mortem Workshop CERN, Geneva, Switzerland June 12-13, 2008 Alexandre Vaniachine.

Introduction: Distributed POOL File Access Elizabeth Gallas - Oxford – September 16, 2009 Offline Database Meeting.

Databases Technologies and Distribution Techniques Dirk Duellmann, CERN HEPiX, Rome, April 4th 2006.

Daniela Anzellotti Alessandro De Salvo Barbara Martelli Lorenzo Rinaldi.

Workshop Summary (my impressions at least) Dirk Duellmann, CERN IT LCG Database Deployment & Persistency Workshop.

LCG 3D Project Status and Production Plans Dirk Duellmann, CERN IT On behalf of the LCG 3D project CHEP 2006, 15th February, Mumbai.

Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:

Database Administrator RAL Proposed Workshop Goals Dirk Duellmann, CERN.

CERN Physics Database Services and Plans Maria Girone, CERN-IT

CSCS Status Peter Kunszt Manager Swiss Grid Initiative CHIPP, 21 April, 2006.

LFC Replication Tests LCG 3D Workshop Barbara Martelli.

1 Database mini workshop: reconstressing athena RECONSTRESSing: stress testing COOL reading of athena reconstruction clients Database mini workshop, CERN.

CERN - IT Department CH-1211 Genève 23 Switzerland t COOL Conditions Database for the LHC Experiments Development and Deployment Status Andrea.

CERN-IT Oracle Database Physics Services Maria Girone, IT-DB 13 December 2004.

Database Readiness Workshop Summary Dirk Duellmann, CERN IT For the LCG 3D project SC4 / pilot WLCG Service Workshop.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Implementation and performance analysis of.

CERN Database Services for the LHC Computing Grid Maria Girone, CERN.

Site Validation Session Report Co-Chairs: Piotr Nyczyk, CERN IT/GD Leigh Grundhoefer, IU / OSG Notes from Judy Novak WLCG-OSG-EGEE Workshop CERN, June.

Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 24 January 2005.

3D Testing and Monitoring Lee Lueking LCG 3D Meeting Sept. 15, 2005.

3D Project Status Dirk Duellmann, CERN IT For the LCG 3D project Meeting with LHCC Referees, March 21st 06.

Plans for Service Challenge 3 Ian Bird LHCC Referees Meeting 27 th June 2005.

LCG 3D Project Update (given to LCG MB this Monday) Dirk Duellmann CERN IT/PSS and 3D

CMS Computing Model summary UKI Monthly Operations Meeting Olivier van der Aa.

Maria Girone CERN - IT Tier0 plans and security and backup policy proposals Maria Girone, CERN IT-PSS.

FroNtier Stress Tests at Tier-0 Status report Luis Ramos LCG3D Workshop – September 13, 2006.

Site Services and Policies Summary Dirk Düllmann, CERN IT More details at

Status of tests in the LCG 3D database testbed Eva Dafonte Pérez LCG Database Deployment and Persistency Workshop.

ATLAS FroNTier cache consistency stress testing David Front Weizmann Institute 1September 2009 ATLASFroNTier chache consistency stress testing.

CERN IT Department CH-1211 Geneva 23 Switzerland t Distributed Database Operations Workshop CERN, 17th November 2010 Dawid Wójcik Streams.

PSM, Database requirements for POOL (File catalog performance requirements) Maria Girone, IT-DB Strongly based on input from experiments: subject.

TAG and iELSSI Progress Elisabeth Vinek, CERN & University of Vienna on behalf of the TAG developers group.

Database Project Milestones (+ few status slides) Dirk Duellmann, CERN IT-PSS (

LHCC Referees Meeting – 28 June LCG-2 Data Management Planning Ian Bird LHCC Referees Meeting 28 th June 2004.

D. Duellmann, IT-DB POOL Status1 POOL Persistency Framework - Status after a first year of development Dirk Düllmann, IT-DB.

Replicazione e QoS nella gestione di database grid-oriented Barbara Martelli INFN - CNAF.

A quick summary and some ideas for the 2005 work plan Dirk Düllmann, CERN IT More details at

2007 Workshop on INFN ComputingRimini, 7 May 2007 Conditions Database Service implications at Tier1 and Tier2 sites Andrea Valassi (CERN IT-PSS)

CERN - IT Department CH-1211 Genève 23 Switzerland t Service Level & Responsibilities Dirk Düllmann LCG 3D Database Workshop September,

Database Requirements Updates from LHC Experiments WLCG Grid Deployment Board Meeting CERN, Geneva, Switzerland February 7, 2007 Alexandre Vaniachine (Argonne)

Dario Barberis: ATLAS DB S&C Week – 3 December Oracle/Frontier and CondDB Consolidation Dario Barberis Genoa University/INFN.

Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 6 April 2005.

INFN Tier1/Tier2 Cloud WorkshopCNAF, 22 November 2006 Conditions Database Services How to implement the local replicas at Tier1 and Tier2 sites Andrea.

Database Readiness Workshop Summary Dirk Duellmann, CERN IT For the LCG 3D project GDB meeting, March 8th 06.

Dirk Duellmann CERN IT/PSS and 3D

Database Replication and Monitoring

LCG 3D Project Update for the meeting with the LHCC referees

LCG 3D Distributed Deployment of Databases

3D Application Tests Application test proposals

Elizabeth Gallas - Oxford ADC Weekly September 13, 2011

Database Readiness Workshop Intro & Goals

Conditions Data access using FroNTier Squid cache Server

Workshop Summary Dirk Duellmann.

3D Project Status Report

Case studies – Atlas and PVSS Oracle archiver

Oracle Streams Performance

Presentation transcript:

LCG 3D StatusDirk Duellmann1 LCG 3D Throughput Tests Scheduled for May - extended until end of June –Use the production database clusters at tier 1 and obtain a first estimate for the replication throughput which can be achieved with the setup –Input to experiment models for calibration / tag data flow Tests started on time, but progress was slower then planned –Main reasons Server setup problem (often db storage) at sites Firewall configuration DBA coverage during vacation period (several hires in the queue) Throughput optimization need Oracle experts to be involved

LCG 3D Status 14 June 2006 Eva Danfonte PerezStreams Throughput Tests2 Preliminary Results Condition-like workload  small (10 column) rows  10, 20 and 50 MB user data  Factor ~5 for DB log data Conclusions so far  Apply is the bottleneck  Parallelism helps!  Throttling (flow control) works  Queues size at destinations need appropriate size Regular contact with Patricia McElroy - Oracle Principal Product Manager Distributed Systems CERN to CERN CERN to CNAF

LCG 3D Status 14 June 2006 Eva Danfonte PerezStreams Throughput Tests3 Replication Throughput Online-Offline (LAN) Tier 0 - Tier 1 (WAN) ~ MB/min achieved (little optimisation so far) Strong dependency on row size and commit frequency WAN reaches ~50% of LAN throughput

LCG 3D Status 14 June 2006 Eva Danfonte PerezStreams Throughput Tests4 Throughput vs Row Size Replication rate of 100 MB/s reached for 2kB rows

LCG 3D Status 14 June 2006 Eva Danfonte PerezStreams Throughput Tests5 LCR Rate/Volume vs Row Size LCR rate is almost constant (around 2k/sec) LCRs number in relation to the row size

LCG 3D StatusDirk Duellmann6 Frontier/SQUID Tests Stress tests with many clients at CERN to validate FroNTier and SQUID production setup at T0 –Obtained cached and un-cached client access rates –Maximum # of connections per server box –Validated DNS failover CMS Frontier tests at FNAL –Focus on connection retry and failover and CORAL integration –CMS July release may pickup LCG AA s/w release which is currently prepared Setting up an additional node for ATLAS FroNTier tests with COOL

LCG 3D Status 14 June 2006 Dirk DuellmannStreams Throughput Tests7 FroNTier Stress Tests