1. Technische Universität München Universitätsbibliothek E-Research in Germany Joint Seminar November 2009 Dr. Reiner Kallenborn Technische Universität München Library

2. Technische Universität München Universitätsbibliothek … the large scale science that will increasingly be carried out through distributed global collaborations enabled by the Internet UK National e-Science Centre E-Science / E-Research is … November 2009 Dr. Reiner Kallenborn Technische Universität München Library big computational research team research networked research dynamic interdisciplinary heterogeneous requiring a different set of working tools and methods

3. Technische Universität München Universitätsbibliothek … in former times: E-Science / E-Research is … November 2009 Dr. Reiner Kallenborn Technische Universität München Library … now grid-infrastructures offer: direct access to resources themselves, i.e. CPU power Storage Scientific instruments Applications web access to “static” information Data Sensors Middleware Services

4. Technische Universität München Universitätsbibliothek Grid-Infrastructures have many advantages November 2009 Dr. Reiner Kallenborn Technische Universität München Library Better use of resources Practically infinite storage and CPU capacity Flexibility, scalability Automatic adaptation of complex computation processes Dynamic and concerted operation of networked resources Higher quality results through grid supported development User oriented cost reduction

5. Technische Universität München Universitätsbibliothek … what’s up in Germany? E-Science / E-Research … November 2009 Dr. Reiner Kallenborn Technische Universität München Library

6. Technische Universität München Universitätsbibliothek Solitary activities o UNICORE projects (Uniform Interface to Computing Resources; expired) o DFN development programme (expired) o Sporadic consortia and projects of individual persons or institutes o LHC-Grid at Tier1-Centre Karlsruhe E-Science in Germany up to 2004 November 2009 Dr. Reiner Kallenborn Technische Universität München Library

7. Technische Universität München Universitätsbibliothek Grid requirements do not stop at national boundaries German scientists are part of international communities German scientists depend on access to international resources (computers, data, experiments) No national initiative comparable to UK, USA, NL No national effort for a grid infrastructure There is no grid without standards There is no use for each scientific community reinventing all grid aspects Germany can’t afford to throw away opportunities offered by grid computing November 2009 Dr. Reiner Kallenborn Technische Universität München Library E-Science in Germany up to 2004  several workshop initiated by the Research Centre Karlsruhe and the Helmholtz Association of German Research Centres initiating D-Grid

8. Technische Universität München Universitätsbibliothek  several workshop initiated by the Research Centre Karlsruhe and the Helmholtz Association of German Research Centres initiating D-Grid November 2009 Dr. Reiner Kallenborn Technische Universität München Library E-Science in Germany from 2004 on: D-Grid D-Grid Strategy Paper and Policy Statement –discussing status and consequences of the grid technology on scientific research and recommending a long-term strategic grid research and development initiative –resulting in an announcement of the German e-Science Initiative by the German Ministry for Research and Education (BMBF) in March 2004, together with a call for proposals in the areas of Grid Computing, e-Learning, and Knowledge Management –Demanding (material) committment of scientific communities  June 15th, 2004: Initiation of a BMBF D-Grid support programme allocating 300 million Euro within 5 years

9. Technische Universität München Universitätsbibliothek Net-based coordinated operation and assignment of computer and storage resources Constitution of a German wide IT infrastructure for complex applications of research and industry Covering a multitude of applications Development of community-specific grid services and working methods Integration of a broad spectrum of resources and IT services  Hardware: computer and storage systems as well as networking facilities  Software: proprietary, freely available or commercial  Data: internal und external databases  Personnel: counselling, training and support D-Grid: Objectives

10. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library D-Grid: Layered Framework Physics Medicine Finances Services / Resources Operation / Middleware User Community Support, Security …

11. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library D-Grid: Organization

12. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library  Homogenization and qualification of the existing Grid-infrastructure by prototypical applications  Generic middleware development (toolkit for e-science)  IT services for scientists, designed and developed by the ‘early adopters’ of the computer science and scientific computing communities  This global services infrastructure is being tested and used by so-called Community Grids in the areas of highenergy physics, astrophysics, alternative energy, medicine and life sciences, earth sciences (e.g. climate), engineering sciences, and scientific libraries  Target groups: Science and scientific computing communities D-Grid 1 (2005 – 2008)

13. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library D-Grid 2 (2008 – 2010)  stronger focus on grids with industry participation, with more than 40 companies involved  national Grid-Infrastructure on production level  establishing of e-science applications and services in Construction industry Finance Aerospace Automotive Enterprise information Resource planning systems Geographical data General IT services

14. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library Gap Projects Service Projects Commercial Applications Academic Applications Integration Project Project Categories in D-Grid

15. Technische Universität München Universitätsbibliothek Special Projects Management of Virtual Organisations Monitoring of Resources and Services Short-lived Certificates Net Attacks Recognition Interoperability of Scheduling Systems New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library

16. Technische Universität München Universitätsbibliothek Service Projects Management Information Systems Backup und Recovery Sustainability of Grid Infrastructures Support for Service Level Agreements Grid-based Knowledge Structures New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library

17. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library

18. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library

19. Technische Universität München Universitätsbibliothek Logistics Analyses and Use of Multimedia Data Molecular Simulation Optimisation via Numerical Simulation Analyses of Medical Signal and Image Data Plasma Technology Support of Dynamic Value Added Networks New Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library

20. Technische Universität München Universitätsbibliothek 2004200720102009201120082012 Special Projects 20062005 Integration Project Service Projects Academic Applications Commercial Applications D-Grid Ltd. Working GroupsPre-ProjectsPhase 1 Phase 2 Phase 3 D-Grid History November 2009 Dr. Reiner Kallenborn Technische Universität München Library

21. Technische Universität München Universitätsbibliothek Sustainability Forthcoming Endeavour: November 2009 Dr. Reiner Kallenborn Technische Universität München Library

22. Technische Universität München Universitätsbibliothek Integration Project und Gap Projects Procedures of resource integration Procedures for generic grid management of resources Universally valid security models and operational concepts Business model DGSE (D-Grid Support Establishment) Service Layer Projects Offer of community-overlapping services Compatibility with standards of the German Computer Science Association Use of proprietary business models Community Projects Offers for users within the community Recipients of DGSE capacities and of service providers D-Grid 3 (2009 - ): Structural Concepts of the D-Grid Initiative November 2009 Dr. Reiner Kallenborn Technische Universität München Library

23. Technische Universität München Universitätsbibliothek Direct contracts between user and provider In case of only few bilateral contacts Contracts between community and provider Delegation of trust to the users within a community Framework agreements between communities and Gauss Alliance Gauss Alliance = Community of German Computer Centres Framework agreements between community representatives and providers Example: Licences for D-Grid Special regulations for investment and special grants D-Grid Ltd. (founded 2008) - Coordination, Use of Resources November 2009 Dr. Reiner Kallenborn Technische Universität München Library

24. Technische Universität München Universitätsbibliothek Bremerhaven Köln Offenbach Dortmund Core group Freiburg Heidelberg Tübingen Mannheim Ulm Konstanz Refional group (bwGRiD) User group (C3-Grid) Further resource provider Marburg Paderborn Aachen Bonn Berlin Darmstadt Siegen Würzburg Göttingen Kassel Leipzig Wuppertal Berlin Stuttgart Dresden München Hannover Jülich Karlsruhe Hamburg Marburg Paderborn Aachen Bonn Berlin Darmstadt Siegen Würzburg Göttingen Kassel Leipzig Wuppertal Oldenburg D-Grid Resources

25. Technische Universität München Universitätsbibliothek Development of an Organisational Structure Vertretung Representatives? Vertretung Providers of IT Services and Resources Academic Domains of Application D-Grid Ltd. Industrial Applications Organisational Matters Legal Matters Gauss Alliance WissGrid November 2009 Dr. Reiner Kallenborn Technische Universität München Library Publicity Matters International Cooperation

26. Technische Universität München Universitätsbibliothek 25th NORDUnet Conference, Copenhagen Dieter Kranzlmüller26 290 Sites 55 Countries 144.000 Cores 25 PetaBytes Disk >17.000 Users >200 VOs >330.000 Jobs/Day Archaeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences … Status July 2009: http://project.eu-egee.org/index.php?id=417

27. Technische Universität München Universitätsbibliothek 42 European NGIs + Asia, US, Latin America + OGF-Europe + PRACE + … EGI, European Grid Initiative

28. Technische Universität München Universitätsbibliothek … how do libraries fit in? E-Science / E-Research … November 2009 Dr. Reiner Kallenborn Technische Universität München Library

29. Technische Universität München Universitätsbibliothek The Evolving World of e-Science: Impact and Implications for Science and Technology Libraries Purdue University (West Lafayette, IN) and Chicago, IL, United States June 20-24, 2010 http://blogs.lib.purdue.edu/iatul2010/ 31st Annual IATUL Conference November 2009 Dr. Reiner Kallenborn Technische Universität München Library

30. Technische Universität München Universitätsbibliothek November 2009 Dr. Reiner Kallenborn Technische Universität München Library Join IATUL! Thank you!

31. Technische Universität München Universitätsbibliothek http://www.d-grid.de/ H.-G. Hegering: D-Grid Initiative – Statusbericht http://www.rv-nrw.de/content/koop/workshops/20040601.dgrid/Hegering_DGrid_Statusbericht_ZKI_Mar04.pdf W. Gentzsch: 2007D-Grid and the Integration of New Communities http://www.ogf.org/OGF20/materials/783/OGF20-D-grid.pdf F. Schlie-Roosen et al.: German Steps to e-Science http://www.pd.infn.it/einfrastructures/eirgSlides/German_View_escience.pdf U. Lang: D-Grid – Hype and reality http://www.dini.de/veranstaltungen/jahrestagungen/2009/ H. Neuroth, M. Kerzel, W. Gentzsch http://webdoc.sub.gwdg.de/univerlag/2007/D-Grid_en.pdf U. Schwiegelshohn http://www.d-grid-ggmbh.de/downloads/BroschuereAHM2009.pdf References November 2009 Dr. Reiner Kallenborn Technische Universität München Library

32. Technische Universität München Universitätsbibliothek Special Projects Management of Virtual Organisations Monitoring of Resources and Services Short-lived Certificates Net Attacks Recognition Interoperability of Scheduling Systems New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library IVOM: Integration of Shibboleth into the D-Grid infrastructure and the enhancement of the corresponding VO-Management

33. Technische Universität München Universitätsbibliothek Special Projects Management of Virtual Organisations Monitoring of Resources and Services Short-lived Certificates Net Attacks Recognition Interoperability of Scheduling Systems New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library D-MON: Realization of a Grid wide monitoring architecture across several underlying, heterogeneous systems taking in consideration multiple resource providers and virtual organisations

34. Technische Universität München Universitätsbibliothek Special Projects Management of Virtual Organisations Monitoring of Resources and Services Short-lived Certificates Net Attacks Recognition Interoperability of Scheduling Systems November 2009 Dr. Reiner Kallenborn Technische Universität München Library GIDS: Grid Intrusion Detection Systems

35. Technische Universität München Universitätsbibliothek Special Projects Management of Virtual Organisations Monitoring of Resources and Services Short-lived Certificates Net Attacks Recognition Interoperability of Scheduling Systems November 2009 Dr. Reiner Kallenborn Technische Universität München Library DGSI: conception and development of an interoperability layer for scheduling in service Grids, allowing the users of a community to distribute the workload among resources within the management domain of another community

36. Technische Universität München Universitätsbibliothek Service Projects Management Information Systems Backup und Recovery Sustainability of Grid Infrastructures Support for Service Level Agreements Grid-based Knowledge Structures New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library BisGRID: Enabling Grid technologies to be used for the integration of decentralised business information systems. This will be achieved by developing and providing organisational and technical extensions based on the current state of the art in Grid technologies, EAI, and SOA

37. Technische Universität München Universitätsbibliothek Service Projects Management Information Systems Backup und Recovery Sustainability of Grid Infrastructures Support for Service Level Agreements Grid-based Knowledge Structures New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library F&L Grid: Establishing a generic service-grid for research and education. The pilot sevice-offering will be Backup- and Archive services, later expanded to Information Lifecycle Management

38. Technische Universität München Universitätsbibliothek Service Projects Management Information Systems Backup und Recovery Sustainability of Grid Infrastructures Support for Service Level Agreements Grid-based Knowledge Structures New Projects November 2009 Dr. Reiner Kallenborn Technische Universität München Library SuGI: Dissemination of the knowledge of grid technology and enhancement of its use, especially the organization of training courses and the multimedia processing of further D-Grid courses for Web based publishing.

39. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library AstroGrid-D Joining together astronomical science institutions into a unified nationwide research infrastructure of collaborative distributed environments using innovative grid technology Creation of a grid based infrastructure for astronomical and astrophysical research, based on integration of their computational facilities, improving the efficiency and usability of their hardware resources Integration of distributed astronomical data archives and, in an extended time frame, of instruments and experiments into a common research infrastructure Support of other research institutes to enable them to share their resources, data and application software within the AstroGrid infrastructure, especially to enable small university workgroups to join the collaboration Strengthening the ties of the German astronomy community to the huge developments within the international activities of the astronomy community

40. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library C3 Grid Connection of distributed climate data archives Uniform access and central metadata catalogue Work packages User Interface (API/GUI) Grid - Information Service Connection of local database-systems / Metadata Backend-Interface Dataproviding and Preprocessing Grid-Data Management Coordidated Grid-Scheduling Implementation and Integration of the C3- Grid Sustainable Connection with Earth System Science Project Management and Co-operation with the Integration Project

41. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library HEP Grid work packages Scalable storage Job monitoring Distributed data analysis and use of grid resources

42. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library MediGRID will demonstrate the feasibility and usefulness of grid services in medicine and life sciences by applying these services in biomedical research with multidimensional data and by correlating genotypic and phenotypic data. At the same time generic recommendations for the technical and logical infrastructure are compiled and submitted to the D-GRiD integration project. The MediGRID project has a modular setup: The four methodological modules (middleware, ontology, resource fusion and eScience) plan to incrementally develop and provide a Grid infrastructure while taking into account the need of the biomedical users. The user communities are represented in three research modules for biomedical informatics, image processing and clinical research. The gained experiences are spread to other communities and flow into the further advancement of the project to create synergies and convergence

43. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library inGRID, Innovative Grid Developments for Engineering Applications Support of virtual prototyping by means of internet-based collaborative methods mapping of engineering workflows on a grid environment integration of specific technical data sources building of portals for selected engineering scenarios. Development of a knowledge-based system for the optimization of engineering workflows and support of decision-making processes Development of co-operation, business, safety and confidence models for scientific engineering applications

44. Technische Universität München Universitätsbibliothek Academic Applications Astronomy Climate Research High Energy Physics Medicine Engineering Humanities November 2009 Dr. Reiner Kallenborn Technische Universität München Library Text Grid, a Virtual Research Environment for the Humantities creating a community grid for the collaborative editing, annotation, analysis, and publication of specialist text resources establishes a truly interdisciplinary platform and a virtual workbench for research focussed on (annotated) text as a data type

45. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library AeroGrid: Creating a permanent and practice- oriented grid infrastructure to facilitate co- operation between large-scale research institutions, industry and university research in the field of aeronautics

46. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library FinGrid: design and development of three Grid-based prototypes Prototype I is a pricing and billing component for Grid-based services – in cooperation with Deutsche Bank and IBM Prototype II is a Grid-based customer portfolio performance measurement and management tool – in cooperation with DataSynapse and Dresdner Bank Prototype III is an asset-backed security factory based on Grid architecture – in cooperation with FinanzIT and PA Consulting

47. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library Biz2Grid aims at migrating industrial business applications to existing grid- middleware systems by developing a price based scheduler to the D-Grid community: provide foundations for an effective application of Grid technologies in enterprises design and implementation of the billing and pricing component business-models have to be developed that are adequate for an application in the Grid distribute and parallelize real world applications and conceptualize and implement economic business models at the same time

48. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library BauVoGrid: a Grid-based Platform for the Virtual Organisation in Construction controlled representation of responsibility / authorisation structures fast configuration and management of both global VO processes and local company-specific processes in accordance with the set up responsibility VO structure fast, flexible and secure access to information from different sources (documents, drawings, photographic material etc.) both from headquarters and from the construction site ad-hoc changes in the process flow, using semi-automatic process simulation mobile capturing of processes and process/product data on the site, thereby providing a basis for faster and more efficient decision making with participation of all site workers

49. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library GDI Grid: Efficient mining and processing of spatial data for the simulation of noise dispersion and disaster management

50. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library PartnerGrid develops a collaboration platform for the co- operation of companies on the basis of D-Grid. Typical scenarios of different branches: Collaboration of the customer and the foundry by the designing of cast parts Collaboration in the metal- processing industry at the planning of forming processes

51. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library WISENT - a Knowledge Network in Energy Meteorology IT-optimised cooperation in research and development of organisations in the area of energy meteorology Development of electronic services for analysis and visualization of large datasets in atmopheric research and meteorology Development of new collaborative methods for the optimization in the use of renewable energy resources like solar and wind energy

52. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library ProGrid - Grid technology for collaborative product development - goals utilize Grid technology for collaborative product development and demonstrate grid advantages by means of selected use cases Process optimization Optimization in fluid mechanics Multidisciplinary structural optimization Connection between product data management software and computer aided engineering systems Collaboration platform

53. Technische Universität München Universitätsbibliothek Aviation Industry Finances Automobile Industry Building and Construction Industry Geographical Information Systems Generic Engineering Applications Meteorology for Renewable Power Industry Cooperative Product Development Medical Engineering and Services Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library S@M - Services@MediGRID: Service Grid for biomedical network research lowering the entry threshold for the utilisation of the grid computing infrastructure in the life sciences on the basis of the results of the initial project MediGRID

54. Technische Universität München Universitätsbibliothek Logistics Analyses and Use of Multimedia Data Molecular Simulation Optimisation via Numerical Simulation Analyses of Medical Signal and Image Data Plasma Technology Support of Dynamic Value Added Networks New Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library MoSGrid makes the D-Grid infrastructure available for high- performance computing in the area of molecular simulations, including the annotation of metadata results and its provision for data mining and knowledge generation

55. Technische Universität München Universitätsbibliothek Logistics Analyses and Use of Multimedia Data Molecular Simulation Optimisation via Numerical Simulation Analyses of Medical Signal and Image Data Plasma Technology Support of Dynamic Value Added Networks New Commercial Applications November 2009 Dr. Reiner Kallenborn Technische Universität München Library ValueGrids opens the D-Grid-infrastructure for a fundamentally new community, the users and providers of software-based services A concept is to be provided allowing for seamless integration of interconnected providers into business processes as well as allows for flexible management of value networks based on service level agreements.