Data Grids: Globus vs SRB. Maturity SRB  Older code base  Widely accepted across multiple communities  Core components are tightly integrated Globus.

Slides:



Advertisements
Similar presentations
Data Management Expert Panel. RLS Globus-EDG Replica Location Service u Joint Design in the form of the Giggle architecture u Reference Implementation.
Advertisements

Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.
GridFTP: File Transfer Protocol in Grid Computing Networks
Connect. Communicate. Collaborate Click to edit Master title style MODULE 1: perfSONAR TECHNICAL OVERVIEW.
Lightweight Preservation Environment Gary Jackson.
1 CENTER FOR PARALLEL COMPUTERS An Introduction to Globus Toolkit® 3 -Developing Interoperable Grid services.
Globus Toolkit 4 hands-on Gergely Sipos, Gábor Kecskeméti MTA SZTAKI
Intelligent Grid Solutions 1 / 18 Convergence of Grid and Web technologies Alexander Wöhrer und Peter Brezany Institute for Software.
The Globus Toolkit Gary Jackson. Introduction The Globus Toolkit is a product of the Globus Alliance ( It is middleware for developing.
Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.
Software Frameworks for Acquisition and Control European PhD – 2009 Horácio Fernandes.
Dspace – Digital Repository Dawn Petherick, University Web Services Team Manager Information Services, University of Birmingham MIDESS Dissemination.
Apache Axis: A Set of Java Tools for SOAP Web Services.
4b.1 Grid Computing Software Components of Globus 4.0 ITCS 4010 Grid Computing, 2005, UNC-Charlotte, B. Wilkinson, slides 4b.
UMIACS PAWN, LPE, and GRASP data grids Mike Smorul.
Virtual Observatory Single Sign-on U.S. National Virtual Observatory National Center for Supercomputing Applications Ray Plante, Bill Baker.
Data Grid: GRASP Mike Smorul. Grid Retrieval and Search Platform Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the.
Archival Prototypes and Lessons Learned Mike Smorul UMIACS.
Globus Computing Infrustructure Software Globus Toolkit 11-2.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Globus 4 Guy Warner NeSC Training.
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
TeraGrid Information Services John-Paul “JP” Navarro TeraGrid Grid Infrastructure Group “GIG” Area Co-Director for Software Integration and Information.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
NeSC Grid Apps Workshop Exposing Legacy Applications as OGSI Components using pyGlobus Keith R. Jackson Distributed Systems Department Lawrence Berkeley.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
 Chapter 6 Architecture 1. What is Architecture?  Overall Structure of system  First Stage in Design process 2.
NeSC Apps Workshop July 20 th, 2002 Customizable command line tools for Grids Ian Kelley + Gabrielle Allen Max Planck Institute for Gravitational Physics.
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
ESP workshop, Sept 2003 the Earth System Grid data portal presented by Luca Cinquini (NCAR/SCD/VETS) Acknowledgments: ESG.
Grid Resource Allocation and Management (GRAM) Execution management Execution management –Deployment, scheduling and monitoring Community Scheduler Framework.
1 All-Hands Meeting 2-4 th Sept 2003 e-Science Centre The Data Portal Glen Drinkwater.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,
Shannon Hastings Multiscale Computing Laboratory Department of Biomedical Informatics.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Code Applications Tamas Kiss Centre for Parallel.
Holding slide prior to starting show. A Portlet Interface for Computational Electromagnetics on the Grid Maria Lin and David Walker Cardiff University.
Jian Gui WANG New Implementation of Agriculture Models APAN19---Jan New Implementations of Agriculture Models Using Mediate Architecture.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
All Hands Meeting 2005 BIRN Portal Architecture: Security Jana Nguyen
Grid Services I - Concepts
LEGS: A WSRF Service to Estimate Latency between Arbitrary Hosts on the Internet R.Vijayprasanth 1, R. Kavithaa 2,3 and Raj Kettimuthu 2,3 1 Coimbatore.
GRIDS Center Middleware Overview Sandra Redman Information Technology and Systems Center and Information Technology Research Center National Space Science.
PoC Induction 19-April VBrowser (VL-e Toolkit) The single point of access to the grid  Medical use case: functional MRI (fMRI)  VBrowser design  VBrowser.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Applications.
Experiment Management System CSE 423 Aaron Kloc Jordan Harstad Robert Sorensen Robert Trevino Nicolas Tjioe Status Report Presentation Industry Mentor:
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
GCRC Meeting 2004 BIRN Coordinating Center Software Development Vicky Rowley.
Cole David Ronnie Julio. Introduction Globus is A community of users and developers who collaborate on the use and development of open source software,
Registries, ebXML and Web Services in short. Registry A mechanism for allowing users to announce, or discover, the availability and state of a resource:
Distributed Data for Science Workflows Data Architecture Progress Report December 2008.
Interactive Data Analysis on the “Grid” Tech-X/SLAC/PPDG:CS-11 Balamurali Ananthan David Alexander
On Using BPEL Extensibility to Implement OGSI and WSRF Grid Workflows Aleksander Slomiski Presented by Onyeka Ezenwoye CIS Advanced Topics in Software.
Data Manipulation with Globus Toolkit Ivan Ivanovski TU München,
Globus: A Report. Introduction What is Globus? Need for Globus. Goal of Globus Approach used by Globus: –Develop High level tools and basic technologies.
USGS GRID Exploratory Status Review Stuart Doescher Mike Neiers USGS/EDC May
Partnerships in Innovation: Serving a Networked Nation Grid Technologies: Foundations for Preservation Environments Portals for managing user interactions.
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
Data Infrastructure in the TeraGrid Chris Jordan Campus Champions Presentation May 6, 2009.
OGSA-DAI.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
Amy Krause EPCC OGSA-DAI An Overview OGSA-DAI on OMII 2.0 OMII The Open Middleware Infrastructure Institute NeSC,
Grid Services for Digital Archive Tao-Sheng Chen Academia Sinica Computing Centre
Architecture Review 10/11/2004
Introduction to Data Management in EGI
Study course: “Computing clusters, grids and clouds” Andrey Y. Shevel
Viet Tran Institute of Informatics Slovakia
Presentation transcript:

Data Grids: Globus vs SRB

Maturity SRB  Older code base  Widely accepted across multiple communities  Core components are tightly integrated Globus  Data transfer components stable  Control, web service protocols are in flux  Optional value-added components have varying levels of stability

Interoperability SRB  Proprietary data and control protocol  Limited documentation available  Many language bindings Globus  Standards based protocols GGF Grid-ftp OGSI, WSRF  Common underlying communications protocol  Components are modular and can be mixed together in often arbitrary ways

Ease of use SRB  GUI and command line clients available  All developed clients must use provided tools  Simplified central administration Globus  Custom clients need to be designed.  Multiple entry points, so clients need not be aware of complete system  Each component has separate administration module

Cost SRB  Free software  Easy setup for simple installations  High cost to extend core functionality  Ease of developing clients through multiple tools  Ease of user access through supplied tools Globus  Free software  Complex setup of multiple disjoint components  Easier to extend core functionality using standard protocols  Clients may be complex due to multiple components  User access is up to grid developer Custom portals, registry services

Target Audiences SRB  Data access, preservation, management groups  Groups requiring ease of multiple datasets across administrative and technological boundaries  Limited to no internal data transformation requirements Globus  Computing and service based needs  Data components designed to feed into other services and usually not directly accessible to end users  Data exposed as a service

Support SRB  Single point of contact for support  Mailing lists, bug tracking, online manuals   Occasional tutorials at SDSC Globus  Multiple support groups depending on number of components used  From the Globus Alliance and the Globus community  Manuals, Mailing lists, online tutorials  Numerous seminars and tutorials around the world   Commercial support forthcoming (IBM, HP)

Component Comparison: Security SRB  Clear text passwords, GSI authentication  Central authorization  Complete separation between underlying operating system and SRB Globus  GSI authentication  Authorization depends on local sites and individual components Commonly just map GSI entities to local system users

Component Comparison: Data Access SRB  Srbmasters provide srb specific data moving protocol  MCAT tracks all available data holdings on srbmasters Tight coupling, all available data MUST be registered in MCAT Globus  Gridftp provides extended ftp services (striping, GSI authentication, etc)  Data locating handled through RLS Not tightly coupled to gridftp

Component Comparison: Data Discovery SRB  MCAT stores metadata system and descriptive centrally  Can be queried using SQL- like syntax  Database pass-throughs can be registered and directly queried with limited output transformation Globus  MCS can track limited descriptive metadata  OGSI-DAI can provide 3 rd party access to existing data sources  Not tightly coupled to underlying data on ftp services  Sepeaation between system and descriptive

Component Comparison: Processing SRB  Limited remote execution  Must be registered in MCAT and application installed on srbmaster  Cannot execute arbitrary code Globus  Easy to design add on services as web service  Can tie into existing compute resources  In hpc manner, many components can execute arbitrary jobs

Lessons Learned SRB  The SRB can easily handle textual metadata.  Extended metadata support requires extensive code modification  SRB needs to be treated as an end to end data grid and not as individual components Globus  Globus is flexible, but also complicated  Some Globus components are fragile (MCS, RFT) while others are very solid and reliable (GSI interfaces, GridFTP)  Globus is evolving and improving: the implementation was made much better with subsequent toolkit releases