Campus High Throughput Computing (HTC) Infrastructures (aka Campus Grids) Dan Fraser OSG Production Coordinator Campus Grids Lead

Key Campus Grid Players University of Nebraska, Lincoln Derek Weitzel (Ace Developer) Derek Weitzel (Ace Developer) Brian Bockelman (Technical Advisor) Brian Bockelman (Technical Advisor) David Swanson (Director, HCC) David Swanson (Director, HCC) Holland Computing Center Holland Computing Center OSG Sites Coordinator Marco Mambelli (Testing, Support & Integrated Documentation) Marco Mambelli (Testing, Support & Integrated Documentation)

An Outline Modes of HPC Computing Models of Campus Sharing Making sense of the OSG The OSG Campus Grid Model Advantages/trade-offs of the model Next steps

The two familiar HPC Modes Capability Computing A few jobs parallelized over the whole system A few jobs parallelized over the whole system Depends on the parallel s/w on each system Depends on the parallel s/w on each system App portability is a problem Adds complexity to grid computing High Throughput Computing (HTC) Run ensembles of single core jobs Run ensembles of single core jobs Does not require an expensive backplane Does not require an expensive backplane The OSG (and Campus Grids) are based on the HTC model The OSG (and Campus Grids) are based on the HTC model

HTPC – an important third mode Ensembles of small- way parallel jobs (10’s – 1000’s) Use whatever parallel s/w you want Use whatever parallel s/w you want (It ships with the job) Also supported on the OSG HTPC expands the user base for HTC

Making sense of the OSG OSG = Technology + Process + Sociology Virtual Organizations VOs are Multidisciplinary Research Groups VOs are Multidisciplinary Research Groups VOs often contribute resources VOs often contribute resources 70+ sites Resources contributed to the OSG Resources contributed to the OSG OSG Delivers: >1M CPU hours every day >1M CPU hours every day ~1 Pbyte of data transferred every day ~1 Pbyte of data transferred every day

A Picture of the OSG

Campus Grids National & Global Cyber- Infrastructures Community Grids Federated Autonomous Cyberinfrastructures

Existing models of Campus Grids Condominium model All single users buy into a single large cluster All single users buy into a single large cluster Economical and effective, but some loss of autonomy Economical and effective, but some loss of autonomy The Fermi-grid model A mini OSG on Campus A mini OSG on Campus Users submit jobs with a grid credential Users submit jobs with a grid credential All resources sit behind a Globus gatekeeper All resources sit behind a Globus gatekeeper Works at National Labs (e.g. FNAL) Works at National Labs (e.g. FNAL)

Existing models of Campus Grids cont. Condor Farm Model All clusters run Condor All clusters run Condor Use the Condor flocking model so that users can easily submit jobs to multiple resources Use the Condor flocking model so that users can easily submit jobs to multiple resources Can connect multiple universities (DiaGrid) Can connect multiple universities (DiaGrid) What about sites that don’t use Condor? That’s where this technology comes in That’s where this technology comes in Nebraska is the prototype Nebraska is the prototype

Users login to their local cluster and submit jobs Campus OSG Cloud PBS LSF Condor Local Cluster Campus User Login

Users login to a Submit Host and transparently use other resources Campus OSG Cloud PBS LSF Submit Host (Condor) Campus Factory Condor Campus User Login Local Cluster

Even resources outside of the campus can be available Campus OSG Cloud PBS LSF Local Cluster Submit Host (Condor) Campus Factory Condor Local User Credential External Campus

Every resource “trusts” all jobs from the submit host (or they cut off access to your account) Campus OSG Cloud PBS LSF Submit Host (Condor) Campus Factory Condor Local User Credential External Campus Local Cluster Campus User Login

The same submit model can include access to OSG resources Campus OSG Cloud PBS LSF Submit Host (Condor) Campus Factory Condor Local User Credential + Grid Cert + Glide-in VO Front End External Campus Local Cluster

The OSG Campus Grid Model Campus users login to the submit host No other credentials are required to use campus resources No other credentials are required to use campus resources Campus Factory is an integrated package Not just an architecture recipe Not just an architecture recipe Can use an existing OSG submit host An opportunity for Tier-3 users to use additional local resources An opportunity for Tier-3 users to use additional local resources A single submission model can utilize heterogeneous clusters running different batch schedulers Even extra-campus resources Even extra-campus resources Researchers can install the system (non-root install) Users can access the the entire OSG without changing their submit model (with a grid cert) Can be linked to an excellent accounting system

Some trade-off’s Users have access to multiple resources Users must learn the Condor submit syntax Users must learn the Condor submit syntax All jobs run as the owner of the Campus Factory Accounting can keep track of job submitters Accounting can keep track of job submitters The model works for High Throughput Computing Jobs HTPC (small way parallel jobs) is an option HTPC (small way parallel jobs) is an option But not really for large scale parallel jobs But not really for large scale parallel jobs What’s missing? Coherent Campus data management strategy Coherent Campus data management strategy May need more than one model for data

Some Parting Thoughts The value of a grid is in its user community Find the “hungry” users first Find the “hungry” users first Meet their needs Meet their needs The future belongs to those who collaborate The World is Flat (Friedman) The World is Flat (Friedman) Research “silos” will eventually miss the boat Research “silos” will eventually miss the boat

Let’s talk further…