We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byTaylor Bain
Modified over 2 years ago
Fujitsu Laboratories of Europe © 2003 A Broker/Scheduler Architecture for Grid Services Dr. David Snelling Fujitsu Laboratories of Europe Open Issues in Grid Scheduling October 21-22, 2003 How to Build a Schprokerer
Fujitsu Laboratories of Europe © 2003 What is a Schprokerer? The Function of Optimizing Workload on the Grid. It is not Controlling or Monitoring that Workload. Not the Unicore NJS or GRAM. Does Include: Bids, Optimization, SLAs, etc... What is Workload? Resource usage over time Workload Profile Not Necessarily Workflow See sidebar.
Fujitsu Laboratories of Europe © 2003 Workflow Sidebar What is Workflow? A description workload consisting of more than one activity. Activity = task | job | process | function A net to describe the dependencies between activities. DAGs, DFGs, Pert Charts, BPDs,... Categories of Workflow Task or Job (partial) sequences Data/Task/Message Flow Workflow Description Types Static vs. Dynamic (changing) Persistent vs. Dynamic (on demand) Explicit (DAGs etc.) vs. Implicit (message sets) Stateful vs. Stateless
Fujitsu Laboratories of Europe © 2003 Workflow Sidebar Continued Extended Functions Loops and Conditionals Error management Workload Exertion Types Deployment InvocationEntities Jobs Processes or Services Other workflows People Tokens Implicit or explicit on dependency arcs. From BPMN © BPMI
Fujitsu Laboratories of Europe © 2003 Functions - 1 Resource Publication Resource Abstraction Compute # of atoms to processor time on machine X. Resource Reservation Issue of Tickets Generation of Bids for Resource Use Cost + QoS User Incarnation Authorization checkingNegotiation SLAs, WS-Agreement,...
Fujitsu Laboratories of Europe © 2003 Functions - 2 Gather Performance Information Gather Adherence Information Were SLAs Honored? Gather Load Information Locally and Grid wide. Make Commitments on behalf of other Schprokerers Optimize Workload versus Policies A scheduler for example.
Fujitsu Laboratories of Europe © 2003 Functions - 3 Translate Information Models Ontology for resource description Convert Grid Currency between VOs Decide to Migrate Workload Compute/Data workload and network workload Not actually migrate the workload Decide to Preempt Workload Not actually preempt the workload Locate Necessary Resources
Fujitsu Laboratories of Europe © 2003 Functions - 4 Aggregate Workflow Task Bids E.g. Bidding for complete Workflows Predicting Actual Workload of a WL description E.g. Workflow loop, conditionals, etc. Topology checking Data not at the end of a thin pipe. Prediction of Resource Loading
Fujitsu Laboratories of Europe © 2003 OGSA Perspective - I
Fujitsu Laboratories of Europe © 2003 OGSA Perspective - II
Fujitsu Laboratories of Europe © 2003 OGSA Perspective - III
Fujitsu Laboratories of Europe © 2003 Architectural Assumption Use the WS-Agreement Approach Distributed Scheduling with Local Information From the Grid perspective, we can only plan Not Central scheduling based on distributed information The Grid cannot control all the resources therefore we cant schedule.
Fujitsu Laboratories of Europe © 2003 Class Relationships WorkloadSource Demand side Initiate Agreements Consign Work Workload Describes the work WorkloadDescription Describes resource requirements Term in agreement WS-Agreement Manifest the contract WorkloadManagement Supply side Provide load info. InfoService Publish information OntologyService Translation of resource description WorkloadOptimizer Implement workload management policy Broker Supply vs Demand Policy Optimization Scheduler Collective, Temporal Optimization Dynamic Runtime workload optimization
Fujitsu Laboratories of Europe © 2003 CRC Analysis - 1 WorkloadSource: Demand resources Initiate Agreements Consign workload to ResourceManager Create workload Workload: The work Fully describe the work to be done Provide WorkloadDescriptions Subtypes: Workflow, Job, Task, WS-deployment, WS- invocation
Fujitsu Laboratories of Europe © 2003 CRC Analysis - 2 WorkloadDescriptions: Resource request Define the (abstract) resource requirements of a workload Part of an Agreement initiation WS-Agreement: Contract Encapsulate agreed terms of usage contract Define contract between WorkloadSource and ResourceManager ResourceManager: Supplier of resources Access control and management of resources Supply load information etc. to InformationService Notify GridMonitor of workload deltas etc.
Fujitsu Laboratories of Europe © 2003 CRC Analysis - 3 InformationService: Global Information Source Maintain and publish information Gather data from: ResourceManager LoadPredictor GridMonitor GridSpies Filter information via ResourceOntologyMapper ResourceOntologyMapper: Translation Translate representations of resources Provide to InformationService only?
Fujitsu Laboratories of Europe © 2003 CRC Analysis - 4 WorkloadOptimizer: Meet policy requirements Implement workload management policy Create/Negotiate Agreements with WorkloadSources Check with AuthorizationService Subtypes: Broker: Supply vs. Demand policy agreements Check with the GridBank (exchange rates only) Scheduler: Collective, temporal optimization DynamicOptimizer: Runtime workload adaptation
Fujitsu Laboratories of Europe © 2003 Functions - 1 Resource Publication - InformationService Resource Abstraction - Broker Compute # of atoms to processor time on machine X. Resource Reservation - ResourceManager Issue of Tickets - ResourceManager Generation of Bids for Resource Use - ResourceManager Cost + QoS User Incarnation - AuthorizationService Authorization checking Negotiation - WorkLoadOptimizer and WorkloadSource SLAs, WS-Agreement,...
Fujitsu Laboratories of Europe © 2003 Functions - 2 Gather Performance Information - GridMonitor Gather Adherence Information - GridSpies Were SLAs Honored? Gather Load Information - GridMonitor Locally and Grid wide. Make Commitments on behalf of other Schprokerers - Broker Optimize Workload versus Policies - WorkloadOptimizer A scheduler for example.
Fujitsu Laboratories of Europe © 2003 Functions - 3 Translate Information Models - ResourceOntologyMapper Ontology for resource description Convert Grid Currency between VOs - GridBank Decide to Migrate Workload - DynamicWorkloadOptimizer Compute/Data workload and network workload Not actually migrate the workload Decide to Preempt Workload - DynamicWorkloadOptimizer Not actually preempt the workload Locate Necessary Resources - InformationService
Fujitsu Laboratories of Europe © 2003 Functions - 4 Aggregate Workflow Task Bids - Scheduler E.g. Bidding for complete Workflows Predicting Actual Workload of a WL description - Scheduler E.g. Workflow loop, conditionals, etc. Topology checking - Scheduler Data not at the end of a thin pipe. Prediction of Resource Loading - LoadPredictor
RMS and Scheduling for Future Generation Grids Ramin Yahyapour University Dortmund Leader CoreGRID Institute on Resource Management and Scheduling CoreGRID.
Dr. Daniel Sabbah Vice President of Strategy & Technology IBM Software Group Bringing Grid & Web Services Together Globus World San Francisco, CA Tuesday,
December 2009 Data Integration in Grid Environments Alex Poulovassilis, Birkbeck, U. of London.
Workflows and Scheduling in Grids Ramin Yahyapour University Dortmund Leader CoreGRID Institute on Resource Management and Scheduling CoreGRID – Summer.
Tecnologia dei Servizi Grid e cloud computing - Lezione 003a 0 Lezione 3a - 20 ottobre 2009 Il materiale didattico usato in questo corso è stato mutuato.
O.Wäldrich CoreGRID Summer School Bonn, July 25, 2006 Resource Orchestration in Grids Wolfgang Ziegler Department of Bioinformatics Fraunhofer.
TIB JINI. INTRODUCTION Here we look at only one type(web,object,file). Clear separation between computation and coordination exists. A distributed system.
Copyright 2004 Bernd Brügge TUM Software Engineering WS TUM System Design II Bernd Brügge Technische Universität München Applied Software Engineering.
Database Planning, Design, and Administration. Stages of the Database System Development Lifecycle.
OGSI Evolution: WS-Resource Framework and WS-Notification Carl Kesselman Globus USC/ISI
OGF19 -- NC 1 Service Level Agreements and QoS: what do we measure and why? Omer F. Rana School of Computer Science, Cardiff.
Copyright © 2005 SOA Software, Inc. All Rights Reserved. Specifications Subject to Change Without Notice. Overcoming the SOA Network Fallacy Roberto Medrano.
Virtual Playground: from Quality of Service to Quality of Life in the Grids Katarzyna Keahey Argonne National Laboratory.
1 Software Architecture in Practice Architectural Design.
Automated eContract Negotiation in Web Service Environment: Trust Management and Electronic Contract Management Aspects Doctoral student Marius Šaučiūnas.
Chapter 6 Architectural Design Slide 1 Chapter 6 Architectural Design.
16-17 October 2003 Grids and Applied Language Theory: Declarative Grid Service Orchestration with OGSA-DQP (A A A Fernandes) 1 Declarative Grid Service.
©2006 University of Southampton IT Innovation Centre and other members of the SIMDAT consortium A SIMDAT Perspective on Grid Standards and Specifications.
Collaborative Testing of Web Services -- The Service oriented framework and implementation in Semantic WS Hong Zhu Department of Computing and Electronics.
1 A Cloud Reference Framework … for discussion only … Please send comments and suggestions to Bhumip Khasnabish Friday,
Sept. 8, 2008 Seminar: Paderborn University Towards A Service-Oriented Approach to Testing Web Services Hong Zhu Department of Computing Oxford Brookes.
Building Portals to access Grid Middleware National Technical University of Athens Konstantinos Dolkas, On behalf of Andreas Menychtas.
Computer Systems & Architecture Lesson 2 4. Achieving Qualities.
Fujitsu Laboratories of Europe © 2004 What is a (Grid) Resource? Dr. David Snelling Fujitsu Laboratories of Europe W3C TAG - Edinburgh September 20, 2005.
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 10Slide 1 Chapter 10 Architectural Design.
2010 FutureGrid User Advisory Meeting Architecture Roadmap Long term vision 10:00-10:45, Monday, August 2, 2010 Pittsburgh, PA Gregor von Laszewski Representing.
Update on CAISO / PacifiCorp Energy Imbalance Market Presentation to WECC Seams Issues & Market Issues Subcommittees October 10, 2013 Jim Price, Senior.
Page 1 LAITS Laboratory for Advanced Information Technology and Standards Duh 7/10/03 Geospatial Service Workflow Concepts and Tools Liping Di Laboratory.
Service Level Agreement Based Scheduling Heuristics Rizos Sakellariou, Djamila Ouelhadj.
© 2016 SlidePlayer.com Inc. All rights reserved.