Alain Roy Computer Sciences Department University of Wisconsin-Madison Condor & Middleware: NMI & VDT

Condor Team Experience › Condor works on many platforms › Condor build & packaging experience › Condor deployed in many environments

Condor: platforms › Condor Today (6.6)  Linux Intel (3 versions)  Linux Alpha  Windows  Mac OS X  Solaris (3 versions)  HPUX  IRIX  Digital Unix  Tru64  AIX › This isn’t new (Condor 6.0, 1999)  Linux Intel  Solaris (2 versions)  HPUX  IRIX (3 versions)  Digital Unix › We have experience with many platforms

Condor: builds & tests › We believe in nightly builds & tests  Running since early 2000  Runs on every platform  Developers immediately know when something breaks

Condor: build pool (subset)

Condor: Many environments › Maintain large pool in CS department  Three platforms  Many diverse applications › Have assisted with many pools world- wide:  You’ve seen many of them today  Commercial  Research  Large & Small

Question › Can we apply this expertise to building, packaging, and distributing Grid software? › YES  NSF Middleware Initiative (NMI)  Virtual Data Toolkit (VDT)

What is NMI? › NMI is a software infrastructure for middleware, especially grid › Many groups contribute software to NMI › Building, packaging, and testing is based here in Madison, with Condor Team members

NMI Builds Sources (CVS) Patching GPT src bundles Build & Test Condor pool (30+ computers) … Build Test Binaries Web Builds distributed with Condor 18 distinct platforms

Builds benefit from Condor › Fault tolerance  Computer fails—build just runs later › Can use DAGMan for ordered builds › New platform? Just add a computer

Condor & NMI › NMI infrastructure builds lots of software  Capable of doing nightly builds and tests  Condor is one of these pieces of software  Helps Condor with nightly builds and regular tests › We use Condor to build Condor (via NMI) We eat our own dog food

NMI GRIDS Status › NMI Release 4: December 2003  Globus  Condor-G  Network Weather Service  KX509  GSI OpenSSH  MyProxy  MPICH-G2  Grid Packaging Tools  Gridconfig  Gridsolve  PyGlobus  UberFTP › NMI Release 5: coming soon › Six platforms:  RedHat 7.2, 8.0, 9.0  Solaris 8  IA64 RedHa  IA64 SuSE Linux

What is the VDT? › A collection of software  NMI software + extras tailored for specific collaborations  Common Grid middleware (Condor, Globus, and more…)  Virtual data software  Utilities › An easy installation & configuration mechanism  Goal: Push a button, everything you need to be a consumer or provider of Grid resources just works  Two methods: Pacman: installs and configures it all RPM: installs some of the software, no configuration › A support infrastructure  Coordinate bug fixing  Help desk  Understand community needs and wishes

What’s in the VDT? › Condor Group  Condor/Condor-G  DAGMan  Fault Tolerant Shell  ClassAds › Globus Alliance  Job submission (GRAM)  Information service (MDS)  Data transfer (GridFTP)  Replica Location (RLS) › EDG & LCG  Make Gridmap  Certificate Revocation List Updater  Glue Schema/Info prov. › ISI & UC  Chimera & Pegasus › NCSA  MyProxy  GSI OpenSSH › LBL  PyGlobus  Netlogger › Caltech  MonaLisa › VDT  VDT System Profiler  Configuration software › Others  KX509 (U. Mich.)

VDT Builds Sources (CVS) Patching GPT src bundles NMI Build & Test Condor pool (30+ computers) … Build Test Package VDT Build Contributors (VDS, etc.) Build Pacman cache RPMs Binaries Test Will use NMI processes soon

Recent Success: Grid2003 › VDT was deployed on 27 sites  VDT/Condor Team members contributed expertise and support › VDT provided access to CPUs › Condor-G used by most users  Close collaboration promoted understanding

Many VDT Customers › GriPhyN collaborators  US-CMS  US-Atlas  LIGO  SDSS › Particle Physics Data Grid › European Data Grid Project › Enabling Grids for e-Science in Europe Project › LHC Computing Grid Project › Alliance Grid Testbed (AGT)

Questions?