Presentation is loading. Please wait.

Presentation is loading. Please wait.

Job Scheduling: History and Evolution

Similar presentations


Presentation on theme: "Job Scheduling: History and Evolution"— Presentation transcript:

1 Job Scheduling: History and Evolution
November 17, 2010 Welkin Associates

2 Agenda The Scheduler Study Overview Brief History of Job Scheduling
Job Scheduler Terminology and Components The Scheduler Study Overview Brief History of Job Scheduling Job Scheduler Functions Job Scheduler Architectures Evaluation of Job Schedulers and Vendors Q/A

3 Job Scheduler Terminology and Components
Job: An executable unit. Can be a systems command, a script, an application program, or a collection of executable units. Jobs can have one or more steps, each containing an executable unit. Job-Dependencies: Logical events or conditions that must be satisfied in order for jobs to be scheduled for execution. Predecessor and Successor. Job-Flow: A collection of two or more jobs related by job-dependencies. Schedule: The current active set of scheduled jobs and job-flows ready to be released for execution and monitored by the Master Scheduler. Components Master Scheduler: Central software component that defines, modifies, stores job and job-flow definitions, dependencies, security information, events, and schedules. Provides automatic submission of executables, monitors status of all jobs and job-flows, provides alerts, controls starting, stopping, restarting of jobs and job-flows. Creates Audit trail of all events. Agent: A program that resides on each server that is controlled by the Master. The agent monitors jobs on its server, communicates with the Master about the status of jobs, and executes commands relayed by the Master. GUI Console: Console display for Administrators and Operators. Relational Database: The central repository used by the Master to store all job, job-flow, dependencies, security information, schedules and audit trails.

4 Job Scheduler Terminology and Components (Cont.)
Administrator Operator Agent Job-Flows Administrator GUI console Operator GUI console Agent Job-Flows Master Scheduler Agent Job-Flows Logs RDMS

5 Scheduler Study Overview
OPUS: Legacy Job Scheduler used for Satellite data processing Developed by Space Telescope Science Institute (STScI ) in 1994 for HST Very fast and efficient pipeline processor. Powerful job-flow features and GUI console Adopted and customized for NOAA as a Job Scheduler on AIX p-series for migration from z/OS Mainframe (2005) OPUS requires replacement by a COTS Scheduler: STScI no longer supports OPUS OPUS GUI written in Java has compatibility issues with latest version of Java The Scheduler Study Evaluated using two in-house Schedulers (NDE and STARS) Evaluated COTS Schedulers and vendors.

6 Brief History of Job Scheduling
In the beginning… there were 80 column Hollerith cards The term “batch” was derived from stacks of punched cards The first true Job Schedulers were people and card readers Therefore, originally Job Scheduling known as “batch processing” 1964 IBM OS/360 JCL presented first job dependency functions

7 Brief History of Job Scheduling (Cont.)
Early Mainframe era: Batch processing using card decks Later Mainframe era: JCL on IBM Mainframes. Jobs: Dependencies/Priorities/Queues/TOD-Calendar/Alerts/Reruns Evolution of 3rd party vendor Mainframe Job Schedulers IBM Tivoli Workload Scheduler (TWS) CA 7 Mainframe Scheduler BMC Control-M ASG Zeke Scheduler Unix systems: crons Today: Job Scheduling has greatly matured, with a number of software vendors that have developed and marketed their Job Scheduler products for over 25 years

8 Brief History of Job Scheduling (Cont.)
Workload Automation Maturity Pyramid

9 Brief History of Job Scheduling (Cont.)
Gartner: The future automation model will be “IT Workload Automation Broker” (ITWAB)

10 Job Scheduler Functions
Create and edit jobs, job-flows, users, groups, schedules Calendar scheduling File-Watchers (Trigger file events) Job parameters within job-flows available Job Queues, Classes, Priority Virtual resources to control jobs and files High Availability / Failover capability Reliable and fast Scheduler database with open schema Scheduler commands to start, stop, suspend, resume, and restart jobs Alerts generated for job errors and sent to Operators (GUI console, , cell) Reporting tools GUI console: Windows, Web-based, Unix/Linux Flexible and configurable for Administrators and Operators, easy to use Graphical display of jobs and job-flows. Click and drag highly preferable Detailed display of job-flow and job status, dependencies, errors/alerts

11 Job Scheduler Architectures
Architecture: Master-Agent vs. Peer-to-Peer

12 Job Scheduler Architectures (Cont.)
Architecture: Master – Agent

13 Research/Advisory Firms Job Scheduler Evaluations
EMA Criteria and Methodology for Evaluating Job Schedulers

14 Research/Advisory Firms Job Scheduler Evaluations (Cont.)
EMA Radar Report for Job Schedulers 01/2010

15 Research/Advisory Firms Job Scheduler Evaluations (Cont.)
November 2009 “Market Overview: Workload Automation, Q3 2009” Workload Automation Vendors Forrester List of Evaluated Job Schedulers

16 Research/Advisory Firms Job Scheduler Evaluations (Cont.)
The Gartner Magic Quadrant is copyrighted (04/28/2009) by Gartner, Inc., and is reused with permission. The Magic Quadrant is a graphical representation of a marketplace at and for a specific time period. It depicts Gartner’s analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the Magic Quadrant, and does not advise technology users to select only those vendors placed in the “Leaders” quadrant. The Magic Quadrant is intended solely as a research tool, and is not meant to be a specific guide to action. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. Source: Gartner, Inc. Magic Quadrant for Job Scheduling 28 April 2009, Report Id Number: G Milind Govekar, Gartner Analyst Gartner Job Scheduler Magic Quadrant Report 04/2009

17 Q/A

18 Additional Slides for reference…

19 Workload Automation Example

20 Future Growth of WLA and Job Scheduling
Forrester: Forecast for Growth of WLA and Job Scheduling

21 Evaluation of Job Schedulers and Vendors
Scheduler Product Name Open Source, Freeware, COTS ASCI ActiveBatch COTS Absyss Visual TOM ActivMindz TaskForest Open Source APM Software GmbH APX/PCC Apple XGRID Argent Job Scheduler ASG Zena for Distributed Systems Zeke for z/OS Mainframes Automation Anywhere Ayehu eyeShare BMC Control-M Camellia Software Batch Job Server (BJS) CISCO Tidal Enterprise Scheduler (TES) Cluster Resources Maui Cluster Scheduler Computer Associates AutoSys

22 Evaluation of Job Schedulers and Vendors
Scheduler Product Name Open Source, Freeware, COTS Corporate Practical Solutions Bqplus COTS Cronless.com cronless Open Source David G. Walker php JobScheduler Free Ware Exact JobBOSS Exspans Automan Flux Corp. Flux Grand Logic JobServer Help/Systems Robot/SCHEDULE Enterprise Honico Batchman Hux Software HQ Scheduler IBM Tivoli Workload Automation Suite IndependIT BICsuite Scheduler ISE EnterpriseSCHEDULE JOX IT JOX LXI LXI Scheduler

23 Evaluation of Job Schedulers and Vendors
Scheduler Product Name Open Source, Freeware, COTS MJTNet Macro Scheduler COTS MVP Systems JAMS Neteject.com VisualCron Network Automation Automate7 Nguyen Anthuan SetCronJob Free Ware OAR Team OAR Open Source Open Systems Mgmt Cosbatch OpsWise Automation Center ORSYP Dollar Universe OW2 Consortium ProActive Parallel Suite Platform Computing Load Sharing Facility (LSF) PTC Software PTC Scheduler Redwood Software Cronacle ROC Software Maestro Scheduler Runtime Design Automation Network Computer (NC) SMA OpCon/xps SoftTree Tech 24x7 Scheduler

24 Evaluation of Job Schedulers and Vendors
Scheduler Product Name Open Source, Freeware, COTS Sourceforge Jcrontab Open Source Stonebranch Indesca COTS Sun Microsystems N1 Grid Engine Sypsoft Job2Do Tenth Generation Job Scheduler for Windows Terracotta Quartz Enterprise Job Scheduler The Coens Group Jobstream UC4 Workload Automation Suite Univa UD Grid MP University of Warwick TITAN Grid Scheduler University of Wisconsin Condor Vinzant Global ECS (GECS) Xi Software GNUbatch XuiS Software EnterpriseSchedule


Download ppt "Job Scheduling: History and Evolution"

Similar presentations


Ads by Google