Vicky Rowley Solution Architect BIRN Coordinating Center - University of California San Diego E-x-t-e-n-d-i-n-g Rocks: The Creation and Management of Grid Systems for Biomedical Research OSGC Conference - May 14, 2008

BIRN is Data Storage (SRB) and Processing, but… UNM UMN UI Duke UCSD UCI BWH MGH Yale UCLA Stanford = Support existing sites = Establish new sites = Replicate for new community Cluster

Rocks Standard vs. Rocks for BIRN  Cluster building focus  Data processing focus  Lots of big clusters  Collaboration focus  Data storage/sharing focus  A few relatively small clusters  Data Grid was needed before clusters processing was needed

MGH Segmentation De-identification And upload JHU Shape Analysis of Segmented Structures BIRN Data Grid BWH Visualization Scientific Goal: classify patient status from morphometric results Large Scale Distributed Computing N=45 Data Donor Site (WashU)‏ So what does BIRN _do_? Large Deformation Diffeomorphic Metric Mapping using the TeraGrid Preliminary Study: 46 hippocampus data sets 30,000 CPU hours, 4 TB data

Shape-derived metrics can be used to detect class-specific information 6 semantic dementia subjects 18 Alzheimer subjects21 control subjects SASHA: Shape Analysis Pipeline Results

The BIRN Collaboratory Today Enabling collaborative research at 28 research institutions comprised of 37 research groups.

How does Rocks make it do that?  Installs operating system software  Turns individual servers into a “Grid” Portals & web servers Data grid for access & management Compute clusters Database servers  Distributes, installs and updates 3rd party, domain- specific scientific software packages  Updates system software

What would be better?  Add/Improve security & performance monitoring  Detect and capture configuration changes  Track versions  Ideally, reduce, reuse, recycle…

Desired System Qualities  Agile Fast response with updates Self-help for developers  Repeatable Tracking of versions Tracking of deployments  Modular/Flexible Handles unique site requirements Handles unique project requirements  Customizable  Scalable Highly automated Supports addition of several sites per year, plus additional projects over 5 years Basic System Software  Operating System  Security… Server Definition Software  Apache/Tomcat  Globus… Application Software  Gridsphere  HID  Mediator  Scientific Applications… BIRN/Rocks Software Stack Custom BIRN Server

A BIRN Grid Portal/WebBIRN Rack GPOP GComp Nettools NAS MCAT DB Server Registry DB UMLS HID DB Mediator GAMA Server MyProxy Globus CAS HID What’s involved in a single grid? CVS, SVN & SRB Repos Testbeds Rocks Central & YUM Rolls: * RHEL4 * area51 * base * birn * birnafs * birncondor * birnportal * birnsrb * CentOS * condor * cvsserver * freesurfer * gama-naregi 1.0 * gama-naregi 4.1 * ganglia * grid * gridsphere * hardwareutils * hid * hpc * java * kernel * mediator * nagios * oracle * postgres * sciapps * sge * srb34 * tomcat * updates-CentOS * webserver 14 Rocks Rolls (-2 for OS)‏ 17 Custom Rolls

Software Development & Integration Rocks/YUM Server Testbeds - update local CVS/SVN - update tarballs - update RPMs - new config/install BIRN-CC - large source into SRB - updates RPMs - Makefile - version.mk - *.spec.in - updates XML (rare)‏

Software Deployment CVS Development Area -Integrate software for many diverse sources -Version control at system and sub-component levels -Rolling baseline -Integration and Functional Testing Staging Area -Verify interoperation of latest code -Support demonstration of latest development efforts without disruption to production -Functional system/Beta Testing Production Area - Stable - Reliable - Facilitates research Rocks Development Server Rocks Staging Server Rocks Production Server SRB

What we love?  Repeatability All the web servers are the same All the database servers are the same  Flexibility Mix & Match rolls  Level of automation Experienced person can “kick” a server in 5 minutes IPs, hostnames, software configuration done  Open Source  Result: Not one grid - Many! Not one project - Many!

What drives us nuts?  Turn around time for updates  Steep learning curve  RPM building not standard  Build time large  Software developers are not co- located with integrators  Reinstalling to get updates is not an option  Lack of advanced roll development training

More info?  See the project website:  vrowley_at_ucsd_dot_edu