GCRC Meeting 2004 BIRN Coordinating Center Software Development Vicky Rowley

 Incorporate software from any organization, source or binary, via CVS or data grid  Track version control at both entire system level and sub-component level  Allow continual improvement of the BIRN software infrastructure in terms of performance, robustness and interoperability  Support demonstration of latest development efforts without disruption to development effort or production operation BIRN Development Environment Objectives

BIRN Development Infrastructure  Shared CVS Repository facilitates collaborative development of core system tools and utilities and accomplishes version tracking  Separately instantiated development, staging and production environments provide robust, yet flexible areas for developing, testing, and deploying BIRN system software

Simple Cluster Frontend General Compute Networked Storage General Compute User Workstation

System Deployment  BIRN specific extensions to Rocks grid management software, also under CVS, means automated, repeatable deployment of any version of the BIRN system  Rocks, with BIRN extensions, includes automated deployment mechanism for Middleware (Security, Computational, Data) Data mediation/integration Application codes Portal and other Workflows

BIRN Grid Web/Portal App/Mediator DB Server Frontend/GPOP Nettools Gen Compute Networked Storage Frontend/GPOP Nettools Gen Compute Networked Storage Frontend/GPOP Nettools Gen Compute Networked Storage Frontend/GPOP Nettools Gen Compute Networked Storage …...

System Testing  Provides flexibility necessary to support testing efforts Hardware can be reinstalled and reconfigured quickly and easily Supports short-term specialized testing (network issues, database upgrades, etc.) Supports “rolling baseline”, or instantiation of everyone’s latest and greatest, until software release testing begins

BIRN System Software Releases  Standardizing software delivery process provides semi-annual software releases April & October Developing internal processes for alpha, beta and production releases Improving documentation and tutorials for all components  BIRN 2.0 due in April Software from testbeds and open source from the community, as well as software for data grid, security, grid operations, etc., is included as part of the integrated BIRN software release

GCRC Meeting 2004 BIRN Coordinating Center Data Grid Vicky Rowley

Local workstation Simple FTP FTP Server User: Anonymous Passwd: None Path: /pub/dir1/dir/2...

FTP Collaboration Local workstation FTP Server User: vickyr Passwd: ***** Path /pub/vickyr User: rowleyv Passwd: ***** Path /pub/rowleyv User: vrowley Passwd: ***** Path /pub/vrowley User: vmrowley Passwd: ***** Path /pub/vmrowley User: vicky Passwd: ***** Path /pub/vicky

Data Sharing Issues  Authentication – who is requesting my data?  Authorization – are they allowed to access it?  Auditing – who has used my data for what?  Organization – where is it?  Network transmission – how long will it take?  Scalability – How many times do I need to repeat this?

FTP vs. data grid  Account on each system, or anonymous access  Need knowledge of each systems directory structure  Read/write access  Individual/group/all.  Directories represent physical storage  Doesn’t scale well  One account recognized on all systems  Individual directory structures abstracted away  Read/write/annotate/audit  Individual, group, domain, public  Folders represent collections of related data  Better scalability

BIRN Data Grid SRB Server MCAT

BIRN Portal

BIRN Data Grid  Client/Server with data catalog  GSI enabled  Access control beyond Windows/Unix Read, write, annotate, audit, etc. Individual and/or group access  Easy to use interfaces (programmable, full-featured interfaces, too)  Functionality to ease network loading (e.g. proxy commands and bulk transfers)  Scalable (with improved scalability on it’s way)

Monitoring Grid Status 0.7 TB 5.2 TB 0 TB 1.6 TB 0.8 TB 3.2 TB 0.8 TB 2.4 TB 0.8 TB 2.4 TB 1.6 TB 0.8 TB 5.0 TB 0.78 TB 0.08 TB

Performance and Trouble Shooting

3.6 Million Files

5.6 Terabytes Used

Logical Resources

Stability BIRN Grid is up and available approximately 98% Example of typical site uptime: