Presentation is loading. Please wait.

Presentation is loading. Please wait.

T.Strizh (LIT, JINR) 1 Грид-технологии в науке, образовании и бизнесе Кореньков В.В. (ОИЯИ, Дубна) Дубна,, 13.10.09 Дубна, ИНТРУС-2009, 13.10.09.

Similar presentations


Presentation on theme: "T.Strizh (LIT, JINR) 1 Грид-технологии в науке, образовании и бизнесе Кореньков В.В. (ОИЯИ, Дубна) Дубна,, 13.10.09 Дубна, ИНТРУС-2009, 13.10.09."— Presentation transcript:

1 T.Strizh (LIT, JINR) 1 Грид-технологии в науке, образовании и бизнесе Кореньков В.В. (ОИЯИ, Дубна) Дубна,, 13.10.09 Дубна, ИНТРУС-2009, 13.10.09

2 e-Infrastructures: reference model

3 e-Infrastructures: network layer

4 e-Infrastructures: grid layer

5 e-Infrastructures: data layer Standards OGF

6 T.Strizh (LIT, JINR) 6 GÉANT2 Pan-European Backbone Dark Fiber Core Among 19 Countries:  Austria  Belgium  Croatia  Czech Republic  Denmark  Finland  France  Germany  Hungary  Ireland  Italy  Netherlands  Norway  Slovakia  Slovenia  Spain  Sweden  Switzerland  United Kingdom 34 NRENs, ~30M Users; 50k km Leased Lines 12k km Dark Fiber; Point to Point Services GN3 Next Gen. Network Projected Start Q2 2009

7 T.Strizh (LIT, JINR) 7 GEANT2 International Connectivity New Projects in 2008-9  ALICE2 in and to Latin America  CAREN for Central Asia  FEAST: Feasibility Study for Sub-Saharan Africa Inter-Regional Connections  GEANT2/ESNet over US LHCNet for US Tier2/ EU Tier1 Connections  4 10G Links to US R&E Nets  1 GbE Connection to the Ubuntunet Alliance (Africa); also TENET in South Africa  ORIENT: CERNET + CSTNET (China) to Europe at 2.5G  TEIN2: 9 Asian Countries + Australia to Europe; TEIN3 Approved 2008-2011  EUMEDConnect: 11 Medi- terranean + N. Africa  ALICE: 12 Latin American countries to GEANT2 at 622 Mbps  SuperSINET (Japan) at New York: 10G  India: 100 Mbps

8 T.Strizh (LIT, JINR) 8 A Global Partnership of R&E Networks and Advanced R&D Projects Supporting the LHC Program

9 T.Strizh (LIT, JINR) 9 CERN - - European Organization for Nuclear Research - European Organization for Nuclear Research Research/ Discovery Training Technology Collaborating

10 T.Strizh (LIT, JINR) 10

11 T.Strizh (LIT, JINR) 11 Tier 0 at CERN: Acquisition, First pass reconstruction, Storage & Distribution Ian.Bird@cern.ch 1.25 GB/sec (ions)

12 T.Strizh (LIT, JINR) Tier 0 – Tier 1 – Tier 2 Ian.Bird@cern.ch 12 Tier-0 (CERN): Data recording Initial data reconstruction Data distribution Tier-1 (11 centres): Permanent storage Re-processing Analysis Tier-2 (>200 centres): Simulation End-user analysis

13 T.Strizh (LIT, JINR) 13 Global Community

14 T.Strizh (LIT, JINR) 14 Grids in LHC experiments Almost all Monte Carlo and data processing today is done via Grid Almost all Monte Carlo and data processing today is done via Grid There are 20+ Grid flavors out there There are 20+ Grid flavors out there Almost all are tailored for a specific application and/or specific hardwareAlmost all are tailored for a specific application and/or specific hardware LHC experiments make use of only 3 Grid flavors: LHC experiments make use of only 3 Grid flavors: gLitegLite ARCARC OSGOSG All experiments develop own higher-level Grid middleware layers All experiments develop own higher-level Grid middleware layers ALICE – AliEnALICE – AliEn ATLAS – PANDA, GANGA, DDMATLAS – PANDA, GANGA, DDM LHCb – DIRAC, GANGALHCb – DIRAC, GANGA CMS – ProdAgent, CRAB, PhEDExCMS – ProdAgent, CRAB, PhEDEx

15 T.Strizh (LIT, JINR) 15 Bioinformatics and Grid Many large clusters utilized for Many large clusters utilized for ServicesServices Sequence similarity (BLAST queues) Sequence similarity (BLAST queues) ResearchResearch Molecular modeling (folding, docking) Molecular modeling (folding, docking) Training of novel predictors Training of novel predictors Jobs are typically short (3 minutes) Jobs are typically short (3 minutes) But plenty (all against all → 10 12 ) But plenty (all against all → 10 12 ) Considerable preparation for single job (couple of gigabytes of data to transfer) Considerable preparation for single job (couple of gigabytes of data to transfer)

16 T.Strizh (LIT, JINR) 16 Biomedical applications Biomedicine is also a pilot application area Biomedicine is also a pilot application area More than 20 applications deployed and being ported More than 20 applications deployed and being ported Three sub domains Three sub domains Medical image processingMedical image processing BiomedicineBiomedicine Drug discoveryDrug discovery Use Grid as platform for collaboration Use Grid as platform for collaboration (don’t need same massive processing power or storage as HEP)

17 T.Strizh (LIT, JINR) Saturday, May 02, 2015Saturday, May 02, 2015Saturday, May 02, 2015Saturday, May 02, 2015 EGEE Applications 17 Applications Example: WISDOM Grid-enabled drug discovery process for neglected diseases Grid-enabled drug discovery process for neglected diseases In silico dockingIn silico docking compute probability that potential drugs dock with target protein compute probability that potential drugs dock with target protein To speed up and reduce cost to develop new drugsTo speed up and reduce cost to develop new drugs WISDOM (World-wide In Silico Docking On Malaria) ‏ : WISDOM (World-wide In Silico Docking On Malaria) ‏ : Three large-scale deployments with more than 6 centuries of computationsThree large-scale deployments with more than 6 centuries of computations achieved in 190 days 3,5TB of data produced3,5TB of data produced Up to 5000 computers in 50 countriesUp to 5000 computers in 50 countries Some promising in-vitro tests, with relevant biological results.Some promising in-vitro tests, with relevant biological results.

18 T.Strizh (LIT, JINR) 18 Radio astronomy needs Grid, too E normous datasets, massive computing, innovative instrumentation E normous datasets, massive computing, innovative instrumentation Dozens of new surveys launched recentlyDozens of new surveys launched recently Many (10 – 100) terabytes per surveyMany (10 – 100) terabytes per survey High data ratesHigh data rates 10 – 100 researchers per survey10 – 100 researchers per survey International collaborations (almost always)International collaborations (almost always) Data is non-proprietary (usually)Data is non-proprietary (usually)

19 T.Strizh (LIT, JINR) 19 Computational Chemistry GEMS (Grid Enabled Molecular Simulator) application GEMS (Grid Enabled Molecular Simulator) application Calculation and fitting of electronic energies of atomic and molecular aggregates (using high level ab initio methods)Calculation and fitting of electronic energies of atomic and molecular aggregates (using high level ab initio methods) The use of statistical kinetics and dynamics to study chemical processesThe use of statistical kinetics and dynamics to study chemical processes Virtual Monitors Virtual Monitors Angular distributionsAngular distributions Vibrational distributionsVibrational distributions Rotational distributionsRotational distributions Many body systemsMany body systems End-User applications End-User applications NanotubesNanotubes Life sciencesLife sciences Statistical ThermodynamicsStatistical Thermodynamics Molecular Virtual RealityMolecular Virtual Reality Angular distribution Rotational distribution Vibrational distribution Many body system Angular distribution

20 T.Strizh (LIT, JINR) 20Fusion Large Nuclear Fusion installations Large Nuclear Fusion installations E.g. International Thermonuclear Experimental Reactor (ITER)E.g. International Thermonuclear Experimental Reactor (ITER) Distributed data storage and handling neededDistributed data storage and handling needed Computing power needed forComputing power needed for Making decisions in real time Making decisions in real time Solving kinetic transport  particle orbits Solving kinetic transport  particle orbits Stellarator optimization  magnetic field to contain the plasma Stellarator optimization  magnetic field to contain the plasma

21 T.Strizh (LIT, JINR) 21 Earth Science Applications Community Community Many small groups that aggregate for projects (and separate afterwards)Many small groups that aggregate for projects (and separate afterwards) The Earth The Earth Complex systemComplex system Independent domains with interfacesIndependent domains with interfaces Solid Earth – Ocean – Atmosphere Solid Earth – Ocean – Atmosphere Physics, chemistry and/or biologyPhysics, chemistry and/or biology Applications Applications Earth observation by satelliteEarth observation by satellite SeismologySeismology HydrologyHydrology ClimateClimate GeosciencesGeosciences PollutionPollution Meteorology, Space WeatherMeteorology, Space Weather Mars AtmosphereMars Atmosphere Database CollectionDatabase Collection

22 T.Strizh (LIT, JINR) 22 Some history 1999 – Monarc Project 1999 – Monarc Project Early discussions on how to organise distributed computing for LHCEarly discussions on how to organise distributed computing for LHC 2001-2003 - EU DataGrid project 2001-2003 - EU DataGrid project middleware & testbed for an operational gridmiddleware & testbed for an operational grid 2002-2005 – LHC Computing Grid – LCG 2002-2005 – LHC Computing Grid – LCG deploying the results of DataGrid to provide adeploying the results of DataGrid to provide a production facility for LHC experiments 2004-2006 – EU EGEE project phase 1 2004-2006 – EU EGEE project phase 1 starts from the LCG gridstarts from the LCG grid shared production infrastructureshared production infrastructure expanding to other communities and sciencesexpanding to other communities and sciences 2006-2008 – EU EGEE-II 2006-2008 – EU EGEE-II Building on phase 1Building on phase 1 Expanding applications and communities …Expanding applications and communities … 2008-2010 – EU EGEE-III 2008-2010 – EU EGEE-III CERN

23 T.Strizh (LIT, JINR) 23 WLCG depends on two major science grid infrastructures EGEE - Enabling Grids for E-Science OSG - US Open Science Grid EGEE & OSG

24 T.Strizh (LIT, JINR) 24 EGEE (Enabling Grids for E-sciencE) The aim of the project is to create a global Pan-European computing infrastructure of a Grid type. - Integrate regional Grid efforts - Represent leading grid activities in Europe 10 Federations, 27 Countries, 70 Organizations

25 T.Strizh (LIT, JINR) 25 350 sites 55 countries 150,000 CPUs 26 PetaBytes (Disk) 40 PetaBytes (Tape) >15,000 users >300 Vos 12 mln jobs/month Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences …

26 T.Strizh (LIT, JINR) Collaborating e-Infrastructures Potential for linking ~80 countries by 2008

27 T.Strizh (LIT, JINR) 27 The EGEE project - Bob Jones - EGEE'08 - 22 September 2008 27 European e-Infrastructure Need to prepare permanent, common Grid infrastructure Ensure the long-term sustainability of the European e-infrastructure independent of short project funding cycles Coordinate the integration and interaction between National Grid Infrastructures (NGIs) Operate the European level of the production Grid infrastructure for a wide range of scientific disciplines to link NGIs

28 T.Strizh (LIT, JINR) 28 Russian Data Intensive Grid infrastructure (RDIG) RDIG Resource Centres: – ITEP – JINR-LCG2 – Kharkov-KIPT – RRC-KI – RU-Moscow-KIAM – RU-Phys-SPbSU – RU-Protvino-IHEP – RU-SPbSU – Ru-Troitsk-INR – ru-IMPB-LCG2 – ru-Moscow-FIAN – ru-Moscow-GCRAS – ru-Moscow-MEPHI – ru-PNPI-LCG2 – ru-Moscow-SINP The Russian consortium RDIG (Russian Data Intensive Grid), was set up in September 2003 as a national federation in the EGEE project. Now the RDIG infrastructure comprises 15 Resource Centers with > 7000 kSI2K CPU and > 1850 TB of disc storage.

29 T.Strizh (LIT, JINR) The main directions in development and maintenance of RDIG e-infrastructure are as the following: - support of basic grid-services; - support of basic grid-services; - Support of Regional Operations Center (ROC); - Support of Regional Operations Center (ROC); - Support of Resource Centers (RC) in Russia; - Support of Resource Centers (RC) in Russia; - RDIG Certification Authority; - RDIG Certification Authority; - RDIG Monitoring and Accounting; - RDIG Monitoring and Accounting; - participation in integration, testing, certification of grid-software; - participation in integration, testing, certification of grid-software; - support of Users, Virtual Organization (VO) and application; - support of Users, Virtual Organization (VO) and application; - User & Administrator training and education; - User & Administrator training and education; - Dissemination, outreach and Communication grid activities. - Dissemination, outreach and Communication grid activities. Development and maintenance of RDIG e-infrastructure

30 T.Strizh (LIT, JINR) 30 JINR - Moscow telecommunication channel Collaboration with RSCC, RIPN, MSK-IX, JET Infosystems, Nortel

31 T.Strizh (LIT, JINR) 31 JINR Central Information and Computing Complex In 2008, total CICC performance was 1400 kSI2K, the disk storage capacity 100 TB At present, the CICC performance equals 2400 kSI2K and the disk storage capacity 500 TB 200820092010 CPU(kSI2K)125017502500 Disk(TB)4008001200 Tapes(TB)-100200 PLAN

32 T.Strizh (LIT, JINR) 32 JINR CICC structural scheme

33 T.Strizh (LIT, JINR) 33 CICC comprises: 65 servers 4 interactive nodes 960 computing nodes, Xeon 5150, 8GB RAM (GEthernet) (160 computing nodes, Xeon X5450, 16GB RAM, InfiniBand). Site name: JINR-LCG2 Internal CICC network – 1GEthernet (10 GE) Operating system -Scientific Linux CERN release 5.2.; Middleware version GLITE-3.1 File Systems – AFS (the Andrew File System) for user Software and home directories is a world-wide distributed file system. AFS permits to share easily files in an heterogeneous distributed environment (UNIXes, NT) with a unique authentication scheme (Kerberos). dCache- for data (500 TB). User registration system – Kerberos 5 ( AFS use Kerberos 5 for authentication ) JINR WLCG infrastructure

34 T.Strizh (LIT, JINR) 34 The protocol between CERN, Russia and JINR on participation in LCG Project was approved in 2003. MoU on Worldwide LHC Computing Grid (WLCG) signed by JINR in October, 2007 The tasks of the JINR in the WLCG: WLCG-infrastructure support and development at JINR;WLCG-infrastructure support and development at JINR; participation in WLCG middleware testing/evaluation, participation in WLCG middleware testing/evaluation, participation in Service/Data Challenges, participation in Service/Data Challenges, grid monitoring and accounting tools development; grid monitoring and accounting tools development; FTS-monitoring and testing; FTS-monitoring and testing; participation in ARDA activities in coordination with experiments; participation in ARDA activities in coordination with experiments; JINR LCG portal support and development; JINR LCG portal support and development; HEP applications; HEP applications; MCDB development; MCDB development; support of JINR Member States in the WLCG activities; support of JINR Member States in the WLCG activities; User & Administrator training and education. User & Administrator training and education. Worldwide LHC Computing Grid Project (WLCG)

35 T.Strizh (LIT, JINR) 35 Portal www.egee-rdig.ru www.egee-rdig.ruwww.egee-rdig.ru

36 T.Strizh (LIT, JINR) 36 RDIG monitoring&accounting http://rocmon.jinr.ru:8080 Monitored values CPUs - total /working / down/ free / busy Jobs - running / waiting Storage space - used / available Network - Available bandwidth Accounting values Number of submitted jobs Used CPU time Totally sum in seconds Normalized (with WNs productivity) Average time per job Waiting time Totally sum in seconds Average ratio waiting/used CPU time per job Physical memory Average per job JINR CICC Monitoring – allows to keep an eye on parameters of Grid sites' operation in real time Accounting - resources utilization on Grid sites by virtual organizations and single users

37 T.Strizh (LIT, JINR) 37 File Transfere System Monitoring and Testing

38 T.Strizh (LIT, JINR) Integration with Google Erath NEC09, Varna, Julia Andreeva (CERN, IT/GS) 38 Experiment specific monitoring systems provide input data Dashboard agents publish this information in the KML format Strong contribution to the development of Sergey Mitsyn (JINR) Application will be shown during the LHC demo at the EGEE conference in Barcelona

39 T.Strizh (LIT, JINR) 39 Production Normalised CPU time per EGEE Region (June-August 2009)

40 T.Strizh (LIT, JINR) 40 Russia and JINR Normalized CPU time per SITE (January 2009 - September 2009 )

41 T.Strizh (LIT, JINR) Production Normalised CPU time per EGEE site for VO LHC (June – September 2009) 41 GRID-site CPU time Num CPU ------------------------------------------------------- FZK-LCG 8,095,787 8620 CERN-PROD 4,552,891 6812 INFN-T1 4,334,940 2862 GRIF 4,089,269 3454 JINR 3,957,790 960 CYFRONET-LCG 3,948,857 2384 PIC 3,921,569 1337 UKI-GLASGOW 3,860,298 1912 RAL-LCG2 3,793,504 2532 UKI-LT2-IC-HEP 3,752,747 960 IN2P3-CC 3,630,425 4544

42 T.Strizh (LIT, JINR) 42 Frames for Grid cooperation of JINR Worldwide LHC Computing Grid (WLCG); Enabling Grids for E-sciencE (EGEE); RDIG Development (Project of FASI) CERN-RFBR project “Grid Monitoring from VO perspective” BMBF grant “Development of the Grid-infrastructure and tools to provide joint investigations performed with participation of JINR and German research centers” “Development of Grid segment for the LHC experiments” was supported in frames of JINR-South Africa cooperation agreement; NATO project "DREAMS-ASIA“ (Development of gRid EnAbling technology in Medicine&Science for Central ASIA); JINR - FZU AS Czech Republic Project “The GRID infrastructure for the physics experiments” JINR - FZU AS Czech Republic Project “The GRID infrastructure for the physics experiments” NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-infrastructures for distributed CMS (CERN) data processing during the first two years of the Large Hadron Collider operation” NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-infrastructures for distributed CMS (CERN) data processing during the first two years of the Large Hadron Collider operation” Project “Elaboration of distributed computing JINR-Armenia grid-infrastructure for carrying out mutual scientific investigations“ Project “Elaboration of distributed computing JINR-Armenia grid-infrastructure for carrying out mutual scientific investigations“ JINR-Romania cooperation Hulubei-Meshcheryakov programme Project "SKIF-GRID" (Program of Belarussian-Russian Union State). Project GridNNN (National Nanotechnological Net)

43 T.Strizh (LIT, JINR) 43 Development of the JINR Grid-environment – 2010-2016 Network level: links between Moscow and Dubna on the basis of state-of-the-art technologies DWDM and 10Gb Ethernet. JINR Local area network : JINR High-speed backbone construction – 10Gbps Resource level: requirements of the LHC experiments stimulate the development of a global Grid-infrastructure, together with the resource centers of all the cooperating organizations. First of all, this is of primary concern for such large research centers as the JINR. To reach effective processing and analysis of the experimental data, further increase in the JINR CICC performance and disk space is needed.

44 T.Strizh (LIT, JINR) 44 3-rd International Conference "Distributed Computing and Grid-technologies in Science and Education“ 30 June – 4 July, 2008 126 reports, 211 participants http://grid2008.jinr.ru

45 T.Strizh (LIT, JINR) 45 gLite training infrastructure schema

46 T.Strizh (LIT, JINR) Информационный Информационный Повышение скорости и надежности доступа к информацииПовышение скорости и надежности доступа к информации Вычислительный Вычислительный Увеличение доступных вычислительных мощностей, их эффективное использованиеУвеличение доступных вычислительных мощностей, их эффективное использование Ускорение работы бизнес-приложенийУскорение работы бизнес-приложений Корпоративный Корпоративный Снижение затрат при сохранении результатаСнижение затрат при сохранении результата Более эффективное использование существующей инфраструктурыБолее эффективное использование существующей инфраструктуры Существенное увеличение готовых к использованию вычислительных ресурсовСущественное увеличение готовых к использованию вычислительных ресурсов Повышение доступности данных и компьютерных ресурсов, возможность их быстрой мобилизацииПовышение доступности данных и компьютерных ресурсов, возможность их быстрой мобилизации Какие преимущества могут дать ГРИД- технологии для бизнеса

47 T.Strizh (LIT, JINR) BEinGRID an special FP6 project

48 T.Strizh (LIT, JINR) BEinGRID Approach Technical cross activities Trust & Security VO Management Service & Data Mgt Architecture & Interop...... Business cross activities Legal Issues Market Study Business Modelling...... Sector 1Sector 2 Sector N BE1 BE2BE3... BE18 BE19 BE25 Repository Phase 1Phase 2

49 T.Strizh (LIT, JINR) Cloud computing e.g. Amazon EC2 and S3, or Sun Grid Compute Utility e.g. Salesforce.com e.g. Force.com

50 T.Strizh (LIT, JINR) 50 EGEE workload in 2007 CPU: 114 Million hours Data: 25Pb stored 11Pb transferred Estimated cost if performed with Amazon’s EC2 and S3: $58,688,679 http://gridview.cern.ch/GRIDVIEW/same_index.phphttp://gridview.cern.ch/GRIDVIEW/same_index.php http://calculator.s3.amazonaws.com/calc5.html? 17/05/08http://calculator.s3.amazonaws.com/calc5.html? Erwin Laure – gLite and Business – EGEE’08 BT – 22 Sept 2008

51 T.Strizh (LIT, JINR) The blind men and the elephant in the room Cyberinfrastructure Grids Shared Infrastructure/ Shared Services SaaS SOA Virtualization Web 2.0 Automation


Download ppt "T.Strizh (LIT, JINR) 1 Грид-технологии в науке, образовании и бизнесе Кореньков В.В. (ОИЯИ, Дубна) Дубна,, 13.10.09 Дубна, ИНТРУС-2009, 13.10.09."

Similar presentations


Ads by Google