Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hung-Sheng Tsao, Ph.D UNC/BASS 2008 SGE overview.

Similar presentations


Presentation on theme: "Hung-Sheng Tsao, Ph.D UNC/BASS 2008 SGE overview."— Presentation transcript:

1 Hung-Sheng Tsao, Ph.D UNC/BASS 2008 SGE overview

2 Sun Proprietary/Confidential: Internal Use Only 2 Sun Grid Engine Scope and Overview Architecture Application Integration Enterprise-scale Features

3 Sun Proprietary/Confidential: Internal Use Only 3 SGE Scope and Overview This section talks about how SGE works, and where it fits into the overall Grid picture At the end, you should be able to  Describe the different levels of Grid deployment and capabilities needed for each  Talk about how N1GE plays a key role in a Grid infrastructure

4 Sun Proprietary/Confidential: Internal Use Only 4 Enterprise Grids Multiple user communities Single organization Global Grids Multiple user communities Multiple organizations Department Grids Single user community Single organization Levels of Grid Computing

5 Sun Proprietary/Confidential: Internal Use Only 5 Grid Enabling Technologies Department Grid Infrastructure Global Grid Infrastructure Enterprise Grid Infrastructure Service Discovery Authentication/ Authorization Data Management Policy Management Resource Management System Management Data Access Across all levels of deployment Industry Standards and partner technologies OGSA, WSRF Globus Toolkit, Avaki Sun Grid Engine Solaris Containers N1 Grid Service Provisioning System Sun Control Station Sun QFS/SamFS Solaris CacheFS Sun Grid Engine Solaris Containers N1 Grid Service Provisioning System Sun Control Station Sun QFS/SamFS Solaris CacheFS Sun technologies

6 Sun Proprietary/Confidential: Internal Use Only 6 Sun Grid Engine: Sun's Key Workload Manager Software to manage jobs & resources on distributed systems Well-established software  Initially developed in 1994, acquired by Sun in 2000  Core technology powers over 8000+ Grids worldwide Sun Grid Engine 6.1 : Sun Grid Engine 6.1 :  Enterprise-scale components on top of open-source core technology, licensed per-CPU  Fully supported by Sun on: Solaris, Linux, Mac OS X, HP-UX, AIX, Irix,Windows

7 Sun Proprietary/Confidential: Internal Use Only 7 Examples of SGE Grids Cycle-stealing: use resources when unoccupied Compute Farms: racks of dedicated nodes "desktop grid" datacenter sharing High-throughput "Beowulf" HPC Clusters: clumps of large SMP systems Massively-parallel jobs, e.g., weather, astro Or any combination of these

8 Sun Proprietary/Confidential: Internal Use Only Dynamic Resource Management Jobs Results Dispatch Distributed Resource Manager

9 Sun Proprietary/Confidential: Internal Use Only System characteristics System status Resources Job policies Resources Resource Matching JOB User User policies Groups Roles Departments Projects SelectionScheduling

10 Sun Proprietary/Confidential: Internal Use Only qmaster DRMAA Sun Grid Engine Components QMaster ExecutionDaemon ExecutionDaemon ExecutionDaemon ExecutionDaemon Scheduler ARCo qsub qrsh qlogin qmon qtcsh App Shadow Master

11 Sun Proprietary/Confidential: Internal Use Only 6.1 Supported Platforms

12 Sun Proprietary/Confidential: Internal Use Only 12 Sun Grid Engine Overview # # BLAST # blastall -p blastn -i /nfs/data Grid Engine Selection of Jobs Job-based policies for resource ROI, SLA, QoS, etc User-based policies for sharing, ranking, by departments, projects, user groups, etc Selection of Resources System characteristics: CPU, memory, OS, patches, etc. Status of systems: avail. mem, load, free disk space, etc. Status of other resources: licenses, shared storage, other software, etc. Selection of Jobs Job-based policies for resource ROI, SLA, QoS, etc User-based policies for sharing, ranking, by departments, projects, user groups, etc Selection of Resources System characteristics: CPU, memory, OS, patches, etc. Status of systems: avail. mem, load, free disk space, etc. Status of other resources: licenses, shared storage, other software, etc. Resource Selection

13 Sun Proprietary/Confidential: Internal Use Only 13 Sun Grid Engine Overview # # BLAST # blastall -p blastn -i /nfs/data Grid Engine Control of jobs Customizable action methods for Suspend, Resume, Kill, Migrate, Restart Manual or automated via triggers Control of resources Regulate load on systems based upon resource value thresholds Control access to systems via permissions, time/date, jobtype Control of jobs Customizable action methods for Suspend, Resume, Kill, Migrate, Restart Manual or automated via triggers Control of resources Regulate load on systems based upon resource value thresholds Control access to systems via permissions, time/date, jobtype Resource Control

14 Sun Proprietary/Confidential: Internal Use Only 14 Sun Grid Engine Overview # # BLAST # blastall -p blastn -i /nfs/data Grid Engine Accounting of jobs Current resource consumption always monitored Total detailed consumption recorded at end of job Includes record of user, department, project, etc, Accounting of resources Utilization of all resources recorded (e.g., CPU, memory, license, etc)‏ Usage accounting kept for users, departments, projects, etc. Accounting of jobs Current resource consumption always monitored Total detailed consumption recorded at end of job Includes record of user, department, project, etc, Accounting of resources Utilization of all resources recorded (e.g., CPU, memory, license, etc)‏ Usage accounting kept for users, departments, projects, etc. Resource Accounting

15 Sun Proprietary/Confidential: Internal Use Only 15 Summary Grid Computing is set of capabilities which work together to provide a software infrastructure Sun Grid Engine is the product which provides the core capabilities of Resource selection, control and accounting

16 Sun Proprietary/Confidential: Internal Use Only 16 Architecture This section covers the architecture of the SGE software, including features targeted for enterprise customers At the end, you should be able to  describe how SGE handles jobs and resources  explain how SGE can be used to address a large company's needs

17 Sun Proprietary/Confidential: Internal Use Only 17 Grid Engine Architecture Submit Host Admin Host Master Host Sche dd Qma ster Exec Host exec d Access TierCompute TierManagement Tier SGE daemons TCP/IP “host” = role

18 Installation File server $SGE_ROOT/ \_ $SGE_CELL/ \_ (other directories)‏ Master host Exec host Submit host Admin host NFS mount (default, not required)‏

19 Sun Proprietary/Confidential: Internal Use Only 19 Resources Per Host, e.g., load_avg mem_free OS/patch-level Global, e.g., floating licenses shared storage Job resource request: job A needs 1 license and 1GB Action thresholds: suspend jobs if load_avg > 1.5 Resource preference: send jobs to hosts with least load; out of those, choose hosts with most free memory Resources used for THE HEART OF GRID ENGINE MANAGEMENT Built-in and custom resources Static resources: strings, numbers, Boolean Countable resources: e.g., licenses, MB of memory/disk Measured resources: value provided through Load Sensor

20 cluster Load Sensors: tracking arbitrary resources compute host compute host compute host compute host compute host compute host Host-specific & global information. Examples: "amount of interactive idle time" (host-specific)‏ "number of licenses of a particular SW in use" (global)‏ Custom script/program queried periodically on one or more hosts Information used for, e.g., resource requests ("run job only if license available")‏ load/suspend thresholds ("suspend jobs when idle time is less than 10 minutes")‏

21 Sun Proprietary/Confidential: Internal Use Only 21 Parallel and Checkpointing Environments Environment: a set of hosts and associated procedures that are used to support parallel or checkpointing applications applications must inherently support parallel/checkpointing execution H2 H3 H1H4 H5 H6 H7

22 Sun Proprietary/Confidential: Internal Use Only 22 Data Access Exec host s App binaries Job data CONFIGURED INDEPENDENTLY NFS sharing File staging Data Grid

23 Sun Proprietary/Confidential: Internal Use Only 23 Application Integration This section covers the different ways to integrate applications with N1GE At the end, you should be able to decide how to run an application with N1GE based upon its usage patterns

24 Sun Proprietary/Confidential: Internal Use Only 24 Unmodified Binaries job script any script, e.g., #!/bin/bash #!/bin/perl can do pre-work, e.g., preprocessing may stage in/out binary/datasets, if desired invokes application with any options Master host spooled onto master host at submission Exec host sent to exec host at execution  use job script, or  exec binary directly % qsub -l memfree=1GB, arch=linux, jobtype=analysis myjob.sh arg1 arg2 applications run without modification:

25 Sun Proprietary/Confidential: Internal Use Only 25 Application Integration Methods queue/host prolog job script END queue/host epilog terminate method resume method suspend method parallel start parallel stop queue/host epilog migration command clean command requeu e job checkpoint command run at specifi ed interval s START starter method General methods Parallel methods Checkpointing methods

26 Sun Proprietary/Confidential: Internal Use Only 26 Interactive Applications submit host % qrsh mycadtool & exec host mycadtool \ DISPLAY on submit host master client command server daemon session active interactive job request request dispatched Allows demanding interactive jobs to be off-loaded to more powerful systems

27 Sun Proprietary/Confidential: Internal Use Only Distributed Resource Management Application API Standard from the Open Grid Forum > Submit, monitor, control jobs > Language & platform agnostic ISVs > “Grid-enable” their applications > Avoid DRM/Grid system lock-in In-house developers > Integrate Grid tasks into workflow, orchestration, online apps, etc.

28 Sun Proprietary/Confidential: Internal Use Only DRMAA Specification http://www.drmaa.org/ Working group goals > Easy to use > Universally implementable Sun Grid Engine Bindings > C binding – supported > Java binding – supported > Perl binding – not supported > Python binding – not supported > Ruby binding – not supported

29 Sun Proprietary/Confidential: Internal Use Only DRMAA Command-line Parity Core Libraries Job API (JAPI) ‏ qsubDRMAA To the qmaster

30 Sun Proprietary/Confidential: Internal Use Only 30 Summary Existing applications can be run in an N1GE grid in a number of ways without being modified The DRMAA standard is a promising way for ISVs and end users to write applications which take advantage of a Grid infrastructure

31 Sun Proprietary/Confidential: Internal Use Only 31 Enterprise-scale Features

32 Sun Proprietary/Confidential: Internal Use Only 32 SGE 6 Scheduler Next-generation Functionality Sophisticated Policy system  Align resource allocation with business priorities  Ability to implement SLA, QoS guarantees  Ensure greatest utilization for your assets, both hardware and software

33 Sun Proprietary/Confidential: Internal Use Only 33 1) Ticket-based Policies Resource allocation aligned to business priorities policy basis includes: department, project, user groups, cumulative utilization, category priority powerful, flexible, tunable, easy to configure

34 Sun Proprietary/Confidential: Internal Use Only 34 Ticket policy: Share Tree All jobs Project 3 50% Group B: 25% Construct tree according to hierarchy of groups leaf must be project or user user leaves can hang off project leaf Biology 25% Chemistr y 75% user 1 user 2 user 3 nod e Project 4: 50% Project 2: 50% Project 1: 50% project leaf user leaf user 4 user 5

35 Sun Proprietary/Confidential: Internal Use Only 35 Unifying Ticket Policies Sum of all ticket contributions Share Tree policy average entitlements over time Share Tree policy average entitlements over time Functional policy fixed entitlements Functional policy fixed entitlements Override policy fine-tuning, temporary changes Override policy fine-tuning, temporary changes tickets Number of tickets affects priority of job

36 Sun Proprietary/Confidential: Internal Use Only 36 2) Urgency Policies Resource-based urgency: gives priority to  expensive asset (SW license, $$$ server)‏  demanding job (multi-CPU, lots of memory)‏ Deadline-based urgency  Enables SLA-type policies Wait-time urgency  Enables quality-of-service guarantees

37 Sun Proprietary/Confidential: Internal Use Only 37 3) POSIX priority Sub-policy Priority value assigned to jobs with a simple number between -1023 and 1024 Used to implement site-specific custom priorities (e.g., external co- scheduler)‏ Also enables admin override of any other policies

38 Sun Proprietary/Confidential: Internal Use Only 38 Combining Policies Final priority of jobs calculated based on unifying the three sub-policies Each policy normalized to 0.0 < N < 1.0 before combining using weight factors Allows hierarchy of policies prio = W urg × N urg + W tix × N tix + W psx × N psx N urg = normalized Urgency N tix = normalized Tickets N psx = normalized Posix W = weighting factors

39 Sun Proprietary/Confidential: Internal Use Only 39 SGE 6 Scheduler Next-generation Functionality Look-ahead planning  Resource Reservation / Preemption / Backfilling Can reserve any resource, e.g. memory, CPU, license, disk  Ensure that important jobs get the resources they need, while maximizing overall utilization

40 Sun Proprietary/Confidential: Internal Use Only 40 Resource Reservation: an example

41 Sun Proprietary/Confidential: Internal Use Only 41 SGE 6 Reporting Analysis / Monitoring / Accounting “ARCO”: module for doing analysis, monitoring, accounting reports, etc.  Fine-grained resource recording, stored in RDBMS in well-defined schema Web-based console tool provided for generating reports, queries, etc. Standard SQL access for 3 rd party tools  Customalizable, extensible

42 Sun Proprietary/Confidential: Internal Use Only 42 SGE 6 Interfaces Standards Compliance DRMAA 1.0 XML-based monitoring  Status command 'qstat' outputs full status info in well-defined XML DTD JDBC/ODBC access to usage information And, of course, the CLI

43 Sun Proprietary/Confidential: Internal Use Only 43 Summary SGE has an extensible and flexible resource model that can describe a wide variety of scenarios Enterprises will benefit from the advanced resource selection policies and look-ahead scheduling Detailed utilization tracking allows for usage analysis and utility- computing capabilities

44 Sun Proprietary/Confidential: Internal Use Only 44 Grid Engine Open source Grid Engine open source project established in 2000 Sun's late 90's strategy: “Give away software, sell our high-margin systems” Today: approximately 10,000 Grid Engine production deployments, nearly all with free software

45 Sun Proprietary/Confidential: Internal Use Only 45 Grid vision Grid Engine project still the main repository of core code Sun Grid Engine: Sun's branded version  Consists of core from the open source, plus proprietary modules to provide added value  Sun sells a license and provides support on major Unix platforms Over time: add more differentiators

46 Sun Proprietary/Confidential: Internal Use Only Other 6.1 Features DRMAA 1.0 > C and Java™ language bindings MacOS X on Intel support Linux on Itanium support MySQL support for ARCo Java Admin API > open source only FreeBSD, NetBSD, Linux/SPARC ports > open source only

47 Sun Proprietary/Confidential: Internal Use Only 47 Key Resources Sun Grid Engine software  http://www.sun.com/gridware http://www.sun.com/gridware  http://gridengine.sunsource.net http://gridengine.sunsource.net Technical resources  HOWTO page at the Grid Engine Project site  Sunsolve Infodocs (some available only to contract customers)‏ Technical Training  SunEd WE-1600 (web-based, 8 hr)‏  SunEd ES-337 (instructor-led, 3 days)‏

48 UNC/BASS 2008


Download ppt "Hung-Sheng Tsao, Ph.D UNC/BASS 2008 SGE overview."

Similar presentations


Ads by Google