Large-Scale Distributed Computing in the Netherlands a status overview.

Slides:



Advertisements
Similar presentations
SURFnets connections to StarLight Erik-Jan Bos Manager Network Services SURFnet STAR TAP SM INET 2001 Annual Meeting Stockholm, Sweden - June 5th, 2001.
Advertisements

CERN STAR TAP June 2001 Status of the EU DataGrid Project Fabrizio Gagliardi CERN EU-DataGrid Project Leader June 2001
University of Illinois at Chicago The Future of STAR TAP: Enabling e-Science Research Thomas A. DeFanti Principal Investigator, STAR TAP Director, Electronic.
Why is optical networking interesting?
1 IEEAF APAN Opportunities APAN 2002 August 26, 2002 Shanghai, China Dr. Donald R. Riley Chair, IEEAF Vice President and CIO University of Maryland, College.
The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.
An open source approach for grids Bob Jones CERN EU DataGrid Project Deputy Project Leader EU EGEE Designated Technical Director
Science Park Amsterdam, October A World of Science in a City of Inspiration Johan A.A. Vos Director Science Park Amsterdam (WTCW N.V.)
S.L.LloydATSE e-Science Visit April 2004Slide 1 GridPP – A UK Computing Grid for Particle Physics GridPP 19 UK Universities, CCLRC (RAL & Daresbury) and.
S.L.LloydGrid Steering Committee 8 March 2002 Slide 1 Status of GridPP Overview Financial Summary Recruitment Status EU DataGrid UK Grid Status GridPP.
Tony Doyle - University of Glasgow GridPP EDG - UK Contributions Architecture Testbed-1 Network Monitoring Certificates & Security Storage Element R-GMA.
The First 16 Years of the Distributed ASCI Supercomputer Henri Bal Vrije Universiteit Amsterdam COMMIT/
Grid Computing Test beds in Europe and the Netherlands David Groep, NIKHEF
Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences DAS-1 DAS-2 DAS-3.
Opening Workshop DAS-2 (Distributed ASCI Supercomputer 2) Project vrije Universiteit.
David Groep / Virtual Lab overview1 Virtual Laboratory Overview A collaborative analysis environment for applied experimental science Distributed.
Nikhef Jamboree 2008 BiG Grid Update Jan Just Keijser.
Large-Scale Distributed Computing in the Netherlands an overview David Groep, NIKHEF,
Grid Jeff Templon PDP Group, NIKHEF NIKHEF Jamboree 22 december 2005 Throbbing jobsGoogled Grid.
The DutchGrid Platform Collaboration of projects from –Computer Science, HEP and service providers Participating and supported projects –Virtual Laboratory.
The Distributed ASCI Supercomputer (DAS) project Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences.
February 12, 2002 DØRACE 1 Grid activities in NIKHEF Willem van Leeuwen.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Henri Bal Vrije Universiteit Amsterdam vrije Universiteit.
Optical networking research in Amsterdam Paola Grosso UvA - AIR group.
Large-Scale Distributed Computing in the Netherlands an overview David Groep, NIKHEF.
Virtual Laboratory for e-Science (VL-e) Henri Bal Department of Computer Science Vrije Universiteit Amsterdam vrije Universiteit.
The Distributed ASCI Supercomputer (DAS) project Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences.
4 december, The Distributed ASCI Supercomputer The third generation Dick Epema (TUD) (with many slides from Henri Bal) Parallel and Distributed.
GridPP Steve Lloyd, Chair of the GridPP Collaboration Board.
Why is optical networking interesting? Cees de Laat
Grids at NIKHEF Grid NIKHEF David Groep NIKHEF PDP The (data) problem to solve beyond meta-computing: the Grid realizing.
NIKHEF Test Bed Status David Groep
Using Grid Computing David Groep, NIKHEF
DataGrid is a project funded by the European Union VisualJob Demonstation EDG 1.4.x 2003 The EU DataGrid How the use of distributed resources can help.
Astro-WISE & Grid Fokke Dijkstra – Donald Smits Centre for Information Technology Andrey Belikov – OmegaCEN, Kapteyn institute University of Groningen.
Dutch Tier Hardware Farm size –now: 150 dual nodes + scavenging 200 nodes –buildup to ~1500 up-to-date nodes in 2007 Network –now: 2 Gbit/s internatl.
Grid Computing Status Report Jeff Templon PDP Group, NIKHEF NIKHEF Scientific Advisory Committee 20 May 2005.
Using the VL-E Proof of Concept Environment Connecting Users to the e-Science Infrastructure David Groep, NIKHEF.
The II SAS Testbed Site Jan Astalos - Institute of Informatics Slovak Academy of Sciences.
Using Grid Computing at NIKHEF David Groep, NIKHEF
The DutchGrid Platform – An Overview – 1 DutchGrid today and tomorrow David Groep, NIKHEF The DutchGrid Platform Large-scale Distributed Computing.
HEPix April 2006 NIKHEF site report What’s new at NIKHEF’s infrastructure and Ramping up the LCG tier-1 Wim Heubers / NIKHEF (+SARA)
The Scaling and Validation Programme PoC David Groep & vle-pfour-team VL-e Workshop NIKHEF SARA LogicaCMG IBM.
Authors: Ronnie Julio Cole David
Key prototype applications Grid Computing Grid computing is increasingly perceived as the main enabling technology for facilitating multi-institutional.
NIKHEF Test Bed Status David Groep
Scaling and Validation Programme David Groep & vle-pfour-team VL-e SP Meeting NIKHEF SARA LogicaCMG IBM.
NIKHEF CT/ Status NIKHEF (NL). NIKHEFDataGrid/Oxford/July DutchGrid Participation of High-energy Physics Earth observation Computer.
119 May 2003HEPiX/HEPNT National Institute for Nuclear Physics and High Energy Physics Coordinates all (experimental) subatomic physics research in The.
DataGrid is a project funded by the European Commission under contract IST rd EU Review – 19-20/02/2004 The EU DataGrid Project Three years.
Physics Data Processing at NIKHEF Jeff Templon WAR 7 May 2004.
EU DataGrid Tutorial NIKHEF, Amsterdam, December 12, 13
Research organization technology David Groep, October 2007.
Grid Computing Jeff Templon Programme: Group composition (current): 2 staff, 10 technicians, 1 PhD. Publications: 2 theses (PD Eng.) 16 publications.
SCARIe: using StarPlane and DAS-3 Paola Grosso Damien Marchel Cees de Laat SNE group - UvA.
DutchGrid KNMI KUN Delft Leiden VU ASTRON WCW Utrecht Telin Amsterdam Many organizations in the Netherlands are very active in Grid usage and development,
J. Templon Nikhef Amsterdam Physics Data Processing Group Large Scale Computing Jeff Templon Nikhef Jamboree, Utrecht, 10 december 2012.
(BiG) Grid: from research to e-Science David Groep, NIKHEF Graphics: Real Time Monitor, Gidon Moont, Imperial College London, see
Grid Computing: dealing with GB/s dataflows David Groep, NIKHEF Graphics: Real Time Monitor, Gidon Moont, Imperial College London, see
Grid Computing at NIKHEF Shipping High-Energy Physics data, be it simulated or measured, required strong national and trans-Atlantic.
Virtual Laboratory Amsterdam L.O. (Bob) Hertzberger Computer Architecture and Parallel Systems Group Department of Computer Science Universiteit van Amsterdam.
Bob Jones EGEE Technical Director
A Dutch LHC Tier-1 Facility
Ian Bird GDB Meeting CERN 9 September 2003
UK GridPP Tier-1/A Centre at CLRC
CERN-USA connectivity update DataTAG project
Grid activities in NIKHEF
Optical Networking Activities in NetherLight
Presentation transcript:

Large-Scale Distributed Computing in the Netherlands a status overview

What is DutchGrid? collaboration of 12 organisations established in 2000 functions as a platform, getting manpower & hardware from partner orgs major contributors –WTCW Virtual Laboratory Project ( ICES/KIS - II ) –ASCI (5 combined Dutch CS Faculties) –Foundation NCF (NWO) –NIKHEF

DutchGrid Sites KNMI Nijmegen Delft Leiden Amsterdam VU WCW Utrecht Telin NIKHEF SARA UvA AMOLF CWI DutchGrid site map David Groep, NIKHEF ASTRON, JIVE WCW Amsterdam

Resources ASCI DAS-2 (VU, UvA, Leiden, TUDelft, Utrecht) –200 dual P-III 1GHz CPUs –homogeneous clusters, 5 locations NIKHEF DataGrid clusters –75 dual P-III ~ 1GHz –1Gb/s IPv4 + 1Gb/s IPv6 NCF Grid national computer facilities foundation from NWO –66 node dual AMD - K7 Fabric Research Cluster ( NIKHEF ) –32 node duals “production quality” cluster ( SARA )* –10Gb/s optical “lambda” test bed † –… BioASP – various smaller O (10 node) clusters

Resources (cont.) SARA – National HPC Centre Processing –SGI 1024 processor MPP Mass storage –StorageTek NearLine tape robot –currently: 200 TByte –will grow “indefinitely” (with funds from NCF) User expertise centre In Total Operations/deployment effort ~ 8 FTE available Research effort O (40 FTE) over 12 organisations

CR1-1CR1-2 CR2-1CR2-2BR2 AR5 BR1 AMS 10G POS 2.5G POS BR1 CHI 622M PO Prot. Tyco/IEEAF AMSTERDAM 2x2.5Gb 622Mb L2 VPN 2xGbE 622Mb “rest” Global Xing 622Mb to “rest” NY CR1-1CR1-2 CR2-1CR2-2BR2 AR5 BR1 AMS 10G POS 2.5G POS BR1 CHI 622M PO Prot. Tyco/IEEAF AMSTERDAM 622Mb L2 VPN NY CR1-1CR1-2 CR2-1CR2-2BR2 AR5 BR1 AMS 10G POS 2.5G POS 622M PO Prot. Tyco/IEEAF 622Mb L2 VPN NY Networks IGRID 6509 BR4 SP 01 OC48c SR Star Light 6509 ONS ONS Star Light T640 ABILEN E NY ARGONNE IL SingAREN TANet2 … SingAREN TANet2 … Star Light RTR STAR TAP GbE CERN RTR ONS 10G POS Tyco 10G POS Level3 2xGbE 10GbE 2xGbE 2.5Gb lambda OC48c lambda 7xGbE GbE GbE’s CHICAGO, IL CERN,CH 18xGbE UvA NIKHEF “NCSA” “ANL”

DutchGrid Services Authentication Services (Certification Authority) –EDG CACG “proper” CA –Student and educational CA –EDG “Tutorial” CA Regular coordination and info meetings Installation support –Have been distributing EDG 1.2 till now User support

Initiatives inside NL NCFGrid strengthening Grid Infrastructure in NL 4 partners ( NIKHEF, ASCI, SARA, ASTRON ) 600 k EUR funding hardware (3 clusters, 10Gb/s networking) and people (1.5 FTE over 3 years) approved August 2002 fabric research + production-quality grids new apps: bio informatics, astronomy First cluster assembled and being tested…

Initiatives inside NL Virtual Laboratory for e-Science Dutch “knowledge infrastructure” project 14 partners 6 application domains extensive middleware research programme test bed HPCSec AAA Networks Int Visual. Inf. Mngt Intg Exp Envs UIs D.I.Sc.FoodInfBioInf Large Equip Bio diversity Personal Care Test Bed Scaling & Validation in project planning stage with estimated O (25 M EUR ) funding

Some Apps WTCW Virtual Laboratory visual analysis environment for (small-scale) experimental science –Chemo-physical surface analysis (PIXE,FT-IR,SIMS) –BioASP – correlating “omics” databases –Virtual Surgery (by-pass surgery, artery defects) DØ – Monte-Carlo and Analysis EcoGrid correlating bio-diversity information Radio Astronomy and VLBI (JIVE) LOFAR, AVO

Research Areas in NL Generic AAA and `security’ UvA, NIKHEF Adaptive programming environments Free University Amsterdam Memory-processor co-allocation TU Delft High-bandwidth networking SURFnet, UvA, SARA, NIKHEF Federated Information & Content Management UvA Interactive visualisation CWI, UvA Cluster, Storage Management & Monitoring NIKHEF, SARA

Summary DutchGrid Platform started informally in 2000 De-facto centre for Grid computing in NL Tasks supported by partners: CA, web, … Mix of people from all layers of the ISO stack networks, AAA, programming envs, semantic DB, apps Plenty of opportunities for collaboration!