1. GRID AND CLOUD COMPUTING
Grid Architecture and Tools
Courtesy:
Dr Gnanasekaran Thangavel

2. Grid Architecture and Tools:
UNIT 2 OVERVIEW
Grid Architecture and Tools:
– The Hourglass model
– Layered Architecture
– Simulation Tools
– Clouds and Grids
– Service Oriented Architecture
11/28/2018

3. Grid Architecture
11/28/2018

4. The Hourglass Model Focus on architecture issues
Propose set of core services as basic infrastructure
Used to construct high-level, domain-specific solutions (diverse)
Design principles
Keep participation cost low
Enable local control
Support for adaptation
“IP hourglass” model
A p p l i c a t i o n s
Diverse global services
Core
services
The term “the Grid” was coined in the mid-1990s to denote a (then) proposed distributed computing infrastructure for advanced science and engineering. A key issue in a grid computing system is that resources from different organizations are brought together to allow the collaboration of a group of people or institutions. Such a collaboration is realized in the form of a virtual organization (VO).
The hour-glass model of grid computing architecture as proposed by Dr. Ian Foster (2001) consists of thin center, wide top and wide bottom with layered combinations.
Local OS
11/28/2018

5. The architecture consists of four layers.
The lowest fabric layer provides interfaces to
local resources at a specific site. These interfaces
are tailored to allow sharing of resources within a
virtual organization.
The connectivity layer consists of small, standard
collection of communication protocols for
supporting grid transactions that span the usage
of multiple resources. In addition, the connectivity
layer will contain security protocols to authenticate
users and resources.
The next layer in the hierarchy is the collective layer. It deals with handling access to multiple resources and typically consists of services for resource discovery, allocation and scheduling of tasks onto multiple resources, data replication, and so on.
The application layer consists of the applications that operate within a virtual organization and which make use of the grid computing environment.
11/28/2018

6. Layered Grid Architecture (By Analogy to Internet Architecture)
Application
Internet
Transport
Application
Link
Internet Protocol Architecture
Collective
“Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services
Resource
“Sharing single resources”: negotiating access, controlling use
Layered Grid architecture defines Grid in terms of a layered collection of protocols:
Fabric layer includes the protocols and interfaces that provide access to the resources that are being shared, including computers, storage systems, datasets, programs, and networks. This layer is a logical view rather then a physical view. For example, the view of a cluster with a local resource manager is defined by the local resource manger, and not the cluster hardware. Likewise, the fabric provided by a storage system is defined by the file system that is available on that system, not the raw disk or tapes.
The connectivity layer defines core protocols required for Grid-specific network transactions. This layer includes the IP protocol stack (system level application protocols [e.g. DNS, RSVP, Routing], transport and internet layers), as well as core Grid security protocols for authentication and authorization.
Resource layer defines protocols to initiate and control sharing of (local) resources. Services defined at this level are gatekeeper, GRIS (Grid Resource Information Service), along with some user oriented application protocols from the Internet protocol suite, such as file-transfer.
Collective layer defines protocols that provide system oriented capabilities that are expected to be wide scale in deployment and generic in function. This includes GIIS (Grid Index Information Service), bandwidth brokers, resource brokers,….
Application layer defines protocols and services that are specialized in nature, targeted towards a specific application domain or class of applications.
Connectivity
“Talking to things”: communication (Internet protocols) & security
Fabric
“Controlling things locally”: Access to, & control of, resources
11/28/2018

7. Example: Data Grid Architecture
App
Discipline-Specific Data Grid Application
Collective
(App)
Coherency control, replica selection, task management, virtual data catalog, virtual data code catalog, …
Replica catalog, replica management, co-allocation, certificate authorities, metadata catalogs,
Collective
(Generic)
Access to data, access to computers, access to network performance data, …
Resource
Communication, service discovery (DNS), authentication, authorization, delegation
Connect
Fabric
Storage systems, clusters, networks, network caches, …
11/28/2018

8. Simulation tools GridSim – Job scheduling
SimGrid – Single client multiserver scheduling
Bricks – Scheduling
GangSim- Ganglia VO
OptoSim – Data Grid Simulations
G3S – Grid Security Services Simulator – security services
11/28/2018

9. Simulation tools
GridSim is a Java-based toolkit for modeling, and simulation of distributed resource management and scheduling for conventional Grid environment.
GridSim is based on SimJava, a general purpose discrete-event simulation package implemented in Java.
All components in GridSim communicate with each other through message passing operations defined by SimJava.
GridSim allows modeling and simulation of entities in parallel and distributed computing systems such as users, applications, resources, and resource brokers/schedulers for design and evaluation of scheduling algorithms.
Discrete event simulation (DES) is the process of codifying the behavior of a complex system as an ordered sequence of well-defined events.
11/28/2018

10. Salient features of the GridSim
It allows modeling of heterogeneous types of resources.
Resources can be modeled operating under space- or time-shared mode.
Resource capability can be defined (in the form of MIPS (Million Instructions Per Second) benchmark.
Resources can be located in any time zone.
Weekends and holidays can be mapped depending on resource’s local time to model non-Grid (local) workload.
Resources can be booked for advance reservation.
Applications with different parallel application models can be simulated.
11/28/2018

11. Salient features of the GridSim
Application tasks can be heterogeneous and they can be CPU or I/O intensive.
There is no limit on the number of application jobs that can be submitted to a resource.
Multiple user entities can submit tasks for execution simultaneously in the same resource, which may be time-shared or space-shared. This feature helps in building schedulers that can use different market-driven economic models for selecting services competitively.
Network speed between resources can be specified.
It supports simulation of both static and dynamic schedulers.
Statistics of all or selected operations can be recorded and they can be analyzed using GridSim statistics analysis methods.
11/28/2018

12. A Modular Architecture for GridSim Platform and Components.
Appn Conf
Res Conf
User Req
Grid Sc
Output
Application, User, Grid Scenario’s input and Results
Grid Resource Brokers or Schedulers …
Appn modeling
Res entity
Info serv
Job mgmt
Res alloc
Statis
GridSim Toolkit
Single CPU
SMPs
Clusters
Load
Netw
Reservation
Resource Modeling and Simulation
SimJava
Distributed SimJava
Basic Discrete Event Simulation Infrastructure
PCs
Workstation
Distributed Resources
Virtual Machine
11/28/2018

13. Cloud: Definition
“A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet.” (According to Foster, Zhao, Raicu and Lu, Cloud Computing and Grid Computing 360-Degree Compared, 2008)

14. CLOUDS

15. Cloud Computing Just a new name for Grid? Yes… …No….
Nevertheless Yes!!!

16. Cloud: just a new name for Grid?
YES:
Reduce the cost of computing
Increase reliability
Increase flexibility (third party)

17. Cloud: just a new name for Grid?
NO:
Great increase demand for computing (clusters, high speed networks)
Billions of dollars being spent by Amazon, Google, Microsoft to create real commercial large-scale systems with hundreds of thousands of computers – shows computers with 100,000+ computers
Analysis of massive data

18. Cloud: just a new name for Grid?
Nevertheless YES:
Problems are the same in clouds and grids
Common need to manage large facilities
Define methods to discover, request and use resources
Implement highly parallel computations

19. Clouds: key points of the definition
Differences related to traditional distributed paradigms:
Massively scalable
Can be encapsulated as an abstract entity that delivers different levels of service
Services can be dynamically configured (via virtualization or other approaches) and delivered on demand

20. Clouds: reasons for interest
Rapid decrease in hardware cost, increase in computing power (multi-cores etc.) and storage capacity
Exponentially growing data size
Widespread adoption of Services Computing and Web 2.0 apps

21. Clouds: yet about definition…
“The interesting thing about Cloud Computing is that we’ve redefined Cloud Computing to include everything that we already do I don’t understand what we would do differently in the light of Cloud Computing other than change the wording of some of our ads.” Larry Ellison (Oracle CEO), quoted in the Wall Street Journal, September 26, 2008

22. Clouds: yet about definition…
“A lot of people are jumping on the [cloud] bandwagon, but I have not heard two people say the same thing about it. There are multiple definitions out there of “the cloud.”” Andy Isherwood (HP VP of sales), quoted in ZDnet News, December 11, 2008

23. Clouds: yet about definition…
“It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign. Somebody is saying this is inevitable — and whenever you hear somebody saying that, it’s very likely to be a set of businesses campaigning to make it true.” Richard Stallman (known for his advocacy of free software), quoted in The Guardian, September 29, 2008

24. Clouds: yet about definition…
From a hardware point of view, three aspects are new in Cloud Computing:
The illusion of infinite computing resources available on demand, thereby eliminating the need for Cloud Computing users to plan far ahead for provisioning;
The elimination of an up-front commitment by Cloud users, thereby allowing companies to start small and increase hardware resources only when there is an increase in their needs; and
The ability to pay for use of computing resources on a short-term basis as needed (e.g., processors by the hour and storage by the day) and release them as needed, thereby rewarding conservation by letting machines and storage go when they are no longer useful.

25. Clouds: relation with other paradigms

26. Web 2.0, Clouds, and Internet of Things
HTC: High - Throughput Computing HPC: High - Performance Computing P2P: Peer to Peer MPP: Massively Parallel Processors
11/28/2018

27. What is a Service Oriented Architecture?

28. What is a Service Oriented Architecture (SOA)?
A method of design, deployment, and management of both applications and the software infrastructure where:
All software is organized into business services that are network accessible and executable.
Service interfaces are based on public standards for interoperability.

29. Key Characteristics of SOA
Quality of service, security and performance are specified.
Software infrastructure is responsible for managing.
Services are cataloged and discoverable.
Data are cataloged and discoverable.
Protocols use only industry standards.

30. What is a “Service”? A Service is a reusable component.
A Service changes business data from one state to another.
A Service is the only way how data is accessed.
If you can describe a component in WSDL, it is a Service.
WSDL (Web Service Definition Language) is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information. The operations and messages are described abstractly, and then bound to a concrete network protocol and message format to define an endpoint.

31. Why Getting SOA Will be Difficult
Managing for Projects:
Software: years
Hardware: years;
Communications: years;
Project Managers: years;
Reliable funding: years;
User turnover: 30%/year;
Managing for SOA:
Data: forever.
Infrastructure: 10+ years.

32. Why Managing Business Systems is Difficult?
40 Million lines of code in Windows XP is unknowable.
Testing application (3 Million lines) requires >1015 tests.
Probability correct data entry for a supply item is <65%..
Output / Office Worker: >30 e-messages /day.

33. How to View Organization for SOAP r i v a t e A p p l i c a t i o n s a n d F i l e s P E R S O N A L L E V E L P r i v a c y a n d I n d i v i d u a l
VARIETY HERE S e c u r i t y B a r r i e r G r a p h i c I n f o W i n d o w , P e r s o n a l T o o l s , I n q u i r y L a n g u a g e s C u s t o m i z e d A p p l i c a t i o n s , P r o t o t y p i n g T o o l s , L o c a l L O C A L L E V E L A p p l i c a t i o n s a n d F i l e s A p p l i c a t i o n s S e c u r i t y B a r r i e r A p p l i c a t i o n s D e v e l o p m e n t
& M a i n t e n a n c e A P P L I C A T I O N L E V E L B u s i n e s s S e c u r i t y B a r r i e r
Service A
Service B
OSD B U S I N E S S L E V E L P r o c e s s S e c u r i t y B a r r i e r F u n c t i o n a l P r o c e s s A
OSD: Object based Storage Device F u n c t i o n a l P r o c e s s B F u n c t i o n a l P r o c e s s C P R O C E S S L E V E L F u n c t i o n a l P r o c e s s D C o r p o r a t e P o l i c y , C o r p o r a t e S t a n d a r d s , R e f e r e n c e M o d e l s , D a t a M a n a g e m e n t a n d T o o l s , I n t e g r a t e d S y s t e m s C o n f i g u r a t i o n D a t a B a s e , S h a r e d C o m p u t i n g a n d E N T E R P R I S E L E V E L T e l e c o m m u n i c a t i o n s
STABILITY HERE I n d u s t r y S t a n d a r d s , C o m m e r c i a l O f f - t h e - S h e l f P r o d u c t s a n d S e r v i c e s G L O B A L L E V E L

34. SOA Must Reflect Timing
Corporate Policy, Corporate Standards, Reference Models,
Data Management and Tools, Integrated Systems
Configuration Data Base, Shared Computing and
Telecommunications, Security and Survivability
Business A
Business B
Infrastructure
Support
Applications Development & Maintenance
ENTERPRISE
PROCESS
BUSINESS
APPLICATION
LOCAL
Graphic InfoWindow, Personal Tools, Inquiry Languages
Customized Applications, Prototyping Tools, Local
Applications and Files
GLOBAL
Industry Standards, Commercial Off-the-Shelf
Products and Services
PERSONAL
Private Applications and Files
Functional Process A
Functional Process B
Functional Process C
Functional Process D
LONG TERM
STABILITY &
TECHNOLOGY
COMPLEXITY
SHORT TERM
ADAPTABILITY &
SIMPLICITY

35. SOA Must Reflect Conflicting Interests
Personal
Local
Organizations
Missions
Enterprise

36. Organization of Infrastructure Services
(Enterprise Information)
Data
Services
Security
Services
Computing
Services
Communication
Services
Application
Services

37. Organization of Data Services
Discovery
Services
Management
Services
Collaboration
Services
Interoperability
Services
Semantic
Services

38. Data Interoperability Policies
Data is an enterprise resource.
Single-point entry of unique data.
Enterprise certification of all data definitions.
Data stewardship defines data custodians.
Zero defects at point of entry.
De-conflict data at source, not at higher levels.
Data aggregations from sources data, not from reports.

39. Organization of Security Services
Transfer
Services
Protection
Services
Certification
Services
Systems
Assurance
Authentication
Services

40. Security Services = Information Assurance
Conduct Attack/Event Response
Ensure timely detection and appropriate response to attacks.
Manage measures required to minimize the network’s vulnerability.
Secure Information Exchanges
Secure information exchanges that occur on the network with a level of protection that is matched to the risk of compromise.
Provide Authorization and Non-Repudiation Services
Identify and confirm a user's authorization to access the network.
Non-Repudiation is the assurance that someone cannot deny something. Typically, nonrepudiation refers to the ability to ensure that a party to a contract or a communication cannot deny the authenticity of their signature on a document or the sending of a message that they originated.

41. Organization of Computing Services
Facilities
Resource
Planning
Control &
Quality
Configuration
Services
Financial
Management

42. Computing Services Provide Adaptable Hosting Environments
Global facilities for hosting to the “edge”.
Virtual environments for data centers.
• Distributed Computing Infrastructure
Data storage, and shared spaces for information sharing.
• Shared Computing Infrastructure Resources
Access shared resources regardless of physical access device.

43. Organization of Communication Services
Interoperability
Services
Spectrum
Management
Connectivity
Arrangements
Continuity of
Services
Resource
Management

44. Communication Services
Provide Information Transport
Transport information, data and services anywhere.
Ensures transport between end-user devices and servers.
Expand the infrastructure for on-demand capacity.

45. Organization of Application Services
Component
Repository
Code Binding
Services
Maintenance
Management
Portals
Experimental
Services

46. Application Services and Tools
• Provide Common End User Interface Tools
Application generators, test suites, error identification, application components and standard utilities.
Common end-user Interface Tools.
, collaboration tools, information dashboards, Intranet portals, etc.

47. A Few Key SOA Protocols
Universal Description, Discovery, and Integration, UDDI defines the publication and discovery of web service implementations.
The Web Services Description Language, WSDL, is an XML-based language that defines Web Services.
SOAP is the Service Oriented Architecture Protocol. It is a key SOA in which a network node (the client) sends a request to another node (the server).
The Lightweight Directory Access Protocol, or LDAP is protocol for querying and modifying directory services.
Extract, Transform, and Load, ETL, is a process of moving data from a legacy system and loading it into a SOA application.

48. References
Kai Hwang, Geoffery C. Fox and Jack J. Dongarra, “Distributed and Cloud Computing: Clusters, Grids, Clouds and the Future of Internet”, First Edition, Morgan Kaufman Publisher, an Imprint of Elsevier, 2012.
Distributed Computing.
Jie Wu, Distributed System Design, CRC Press, 1999.
Distributed Computing, Wikipedia
csnotes.upm.edu.my/kelasmaya/web.nsf/.../$FILE/Distributed%20Computing.ppt
11/28/2018

49. Assignment #1 Visit the links for GridSim information on slide 9 to:
Write a report on GridSim, its main features
Selected any of the related publications about GridSim and summarise in you report.
11/28/2018

50. Questions and Comments?
Thank You
Questions and Comments?
11/28/2018