Presentation is loading. Please wait.

Presentation is loading. Please wait.

Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National.

Similar presentations


Presentation on theme: "Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National."— Presentation transcript:

1 Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National Laboratory & University of Chicago)

2 2 Overview Swift Highlights Swift Performance on Beagle Swift Components Examples Summary of Swift benefits on Beagle

3 3 Swift Highlights Swift is a parallel scripting system for distributed computing – For loosely-coupled applications: application and utility programs linked by exchanging files – Can use local and/or distributed resources: PADS, Beagle, TeraGrid, OSG, clouds, etc. Swift is easy to write – Simple high-level C-like functional language – Small Swift scripts can do large-scale work Swift is easy to run – Contains all services for running workflow: a java application, run from a commandline – Preinstalled as Beagle module : module load swift

4 4 Submit host (Laptop, Linux server, …) Workflow status and logs ????? Compute nodes f1 f2 f3 a1 a2 Data server f1 f2 f3 Provenance log script App a1 App a2 site list app list File transport Clouds Swift supports clusters, grids, and supercomputers Download, untar, and run Architecture

5 5 Swift Performance on Beagle Decision Support System for Agrotechnology Transfer (DSSAT) Application: Soil Analysis DSSAT Numbers : 120,000 App Calls, dataset Beagle Plots: Peak load of cores, completed in < 3hours

6 6 Swift Components sites.xml tc.data config rawdata = sim(settings); stats = analysis(rawdata); … compile select resources allocate resourceswrite script  execute

7 7 Swift Examples (1) Simple Script A simple SwiftScript type file; app (file output) cat(file input) { } file data ; file out ; out = cat(data); Commandline $ swift –sites.file sites.xml –tc.file tc.data catsn.swift

8 8 Swift Examples (2) Script with mappers and loop A SwiftScript with mappers and loop type file; app (file output) cat(file input) { } file out[] ; file data ; foreach j in { out[j] = cat(data); }

9 9 More Swift Examples and Demos /lustre/beagle/ketan/beagle-swift-training- examples – sites.xml, tc, cf – 01-app.swift : A simple file copy application – 02-function.swift : functions in Swift – 03-wrapper.swift : Calling wrapper script passing Swift parameters – 04-arrays.swift : Arrays in Swift – 05-regexp.swift : Regular expression mappers

10 10 Swift Benefits on Beagle Hassle-free SuperComputing – No need to write qsub scripts and aprun command Easy scaling – sites.xml parameters Easy app wrapping – tc.data Parallel Computation – futures, foreach loops

11 Thanks! Swift Team: Michael Wilde (PI), Daniel Katz (PI), Mihael Hategan, Justin Wozniak, David Kelly, and many others Swift Students: Alberto Chavez, Papia Rizwan, Jon Monette, Tim Armstrong, Glen Hocky Questions? (or later: or in person, we are located on fifth floor (5.E.10) at Argonne-TCS Swift Homepage:


Download ppt "Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National."

Similar presentations


Ads by Google