Scaling NVO Services to the Teragrid Roy Williams Conrad Steenberg Matthew Graham Joe Jacob Ray Plante

NESSSI NVO Extensible Secure Scalable Service Infrastructure Services are science-oriented Services are made by trusted developers from the science community Web forms OR command line (Python API) Built-in security (X.509 certificates) Very large jobs can be run Easy to get a certificate No complex install needed by client Different levels of certificate get different service Is installed on Teragrid Services can be part of a workflow

Desired Characteristics of NVO Services Service oriented architecture Services should be easily and quickly deployable and usable on workstations or supercomputers Services deployed, managed, and upgraded by their developers Service developers/deployers are trusted users Service developer acts as a broker between computing customer and computer center Service users authenticated with graduated security Easy to start, but great power is possible Asynchrony for compute intensive jobs Jobs submitted to batch queue Unique sessionID may be used to monitor job & return results From clicking to scripting Services may be accessed by clicking on a web page or with scripted client codes Authentication for web clicking comes from a certificate store Scripted access requires a certificate (strong or weak) straight from the client Services as workflow components A service user may be another service (a computer, not a human!)

A Graduated Security Model Web form - anonymous access, small jobs Some science.... Get NVO weak certificate - access logged, but identity not verified More science.... Full TeraGrid account - browser access Big-iron computing.... Scripted access Power user Portal-Based

Traditional Grid Security client Show us your Certificate! I will do exactly what you want.

Graduated Security client May I have your Request and your Certificate?

This is a US drivers licence. In the US it proves identity strongly. It is like a strong certificate. This is a loyalty card where I buy food. (You can put a false address on the application.) It is like a weak certificate. This is a $50 gift card at a bookstore. It does not prove my identity in any way. It is like an anonymous certificate. Certificates The Virtual Observatory as a Virtual Organization

service implementation web forms python API graduated security certificates certificate chains root certificates proxy certificates proxy certificate chains 2nd level proxy chains xforms secure https redirection teragrid security police caltech security police NCSA security police chown directory ownership NFS root-squashing PBS stdout permissions pubcookie

A proxy is a copy of a certificate with a 24-hour expiry date It is safer than sending the full certificate. A proxy can come from a certificate store released by username/password A proxy can be built with a local tool eg nesssi_proxy_init or globus_proxy_init Proxy Certificates

Web Portal client certificate repository nesssi web portal nesssi node web form SOAP http queue fetch proxy select user account sandbox storage open http certificate policies

Commandline Portal clientnesssi node Teragrid cluster certificate policies queue select user account sandbox storage Secure SOAP certificate open http build proxy

Exercise: Running a Nesssi Service see

The NVO Certificate Authority The NVO now has a certificate authority

Getting an NVO login

The Web Portal

Getting a proxy certificate % cd $NVOSS_HOME % source bin/setup.csh [snip] All set up for the 2006 NVO Summer School. % cd nesssi % java NesssiInit YourUsername YourPassword /tmp/x509up_u501 % ls -l /tmp/x* -rw roy wheel 2231 Sep 1 12:40 /tmp/x509up_u501 web portal command line is this your UID?

SessionID and Sandbox Identify which job we are talking about 32 character hex string eg cb28d0753a7fec9a485981f741d425ec Used to monitor a running job sessionID = nesssiServer.cutout.init() msg = server.cutout.monitor(sessionID) Used to form URL where results appear, eg /clarens/shell/cb/cb28d0753a7fec9a485981f741d425ec/cutouts/index.html If you lose the sessionID, you lose your job

Cutout ux daf5ef52facc68cb03db4b1fdc815 clarens/shell/ 77/774daf5ef52facc68cb03db4b1fdc815 clarens/shell/ 77/774daf5ef52facc68cb03db4b1fdc815 /cutouts/index.html 149.envoy.cacr.calte roy batch C8845cb :00 R -- Monitoring a Nesssi job service name running as this user session ID sandbox URL results URL queue status (R = running)

Example: SleepyAdd nesssiServer=nesssi.client(' # nesssiServer=nesssi.client(' sessionID = nesssiServer.sleepyadd.init() print "Your session ID is", sessionID # Run: sleep 30 seconds then add 52 and 344 nesssiServer.sleepyadd.run(sessionID, "-time 30 -n 52 -m 344") web portal command line

Monitoring the Run Key n is 52 Key m is 344 Key time is 30 Sleeping for 30 seconds Waking up... Sum of 52 and 344 is 396 Sleepyadd ux a3a167a383111c0cbd b8659aa dtf-mgmt1.sds ux dque Ca3a :00 Q --

Mosaic Service nesssiServer=nesssi.client(' mosaic_loc = "-ra dec rawidth 0.5 -decwidth 0.5 -filt f -bgcorr 0" session = nesssiServer.dpossMosaic.mosaic(mosaic_loc) print "Your session ID is %s." % session msg = dbsvr.dpossMosaic.monitor(session) print msg

nesssiServer. dpossMosaic.mosaic ( -ra dec rawidth 0.5 -decwidth 0.5 -filt f -bgcorr 0)

Coadd Service nesssiServer=nesssi.client(' # Initialize the service sessionID = nesssiServer.hyperatlas.init() print "Session id is ", sessionID # Arguments for service, the coaddition to do args = "-bandpass z1 -ra dec rawidth 1.0 -decwidth 1.0"

-bandpass z1 -ra dec rawidth 1.0 -decwidth 1.0

Cutout Service nesssiServer=nesssi.client(' sessionID = nesssiServer.cutout.init() print "Session id is ", sessionID # Upload locations file remoteinputfile = "/shell/%2s/%s/inputfile.xml" % (sessionID[0:2], sessionID) nesssiServer.upload_file(inputfile, remoteinputfile) # Arguments for service, surveys to use and cutout size args = "-surveys PQ:gr,PQ:gi,PQ:z1,PQ:z2,SDSS:r,SDSS:i,SDSS:z,2MASS:k,2MASS:h " args += "-size 64" # Run service nesssiServer.cutout.run(sessionID, args)

Cutout Monitoring

cutouts from Palomar-Quest, SDSS, 2MASS of sources from Veron quasar catalog