A.V. Bogdanov Private cloud vs personal supercomputer

t Data evolution Utility computing Local DPC Global DPC Computations on-demand Grid ComputingAdaptive enterprise SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. New concepts of large datasets processing

Scott McNealy is an American business executive. He co-founded Sun Microsystems in 1982 (Sun was acquired by Oracle). Scott McNealy SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

“Network is a computer” Our vision: “Computational center area network is a Grid” SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Scott McNealy What is the Grid?

Greg Papadopoulos Greg Papadopoulos, Ph.D. was Executive Vice President and Chief Technology Officer (CTO) of Sun Microsystems. SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Services-Oriented Architecture Service provider Service broker Service requestor Publish Find Bind

Larry Smarr is a leader in scientific computing, supercomputer applications, and Internet infrastructure. Larry Smarr Smarr was a director of the National Computational Science Alliance SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

The way to organize distributed computations. To boost the usage of the networks (local and global). To promote computational facility of the next generation. SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Metacomputing

Tasks of a large supercomputer center User access and authorization Search for data Preprocessing Processing Postprocessing Data archiving SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Collection of high end resources Connected by high performance networks Scientific and other instruments Providing innovative distributed solutions David Abramson Source: David Abramson, ICCS’2004 SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. What is the Grid?

PC Gateway server archive- server FileNet- archive ? Multi-tier-architecture SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Heterogeneous structure of the Grid: calculation systems with different architectures problems with taking into account specific features of each calculation system jointed into the Grid How to solve? to use standard optimized calculation libraries SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Problems to be solved

Metacomputing Service-oriented architecture (SOA) Virtualization SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Technologies

Various platforms integration Friendly user interface development Consolidation and insurance of distributed data retention Organization of remote collective work opportunity Information security insurance Natural paradigm for parallelization (both for computational and storage) SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. The main problems under development

 reduction due to Linux kernel modification; Inclusion of parallelized elements from other UNIX systems – SCO, AIX… Single image of operation system Linux modification Specialized operating systems SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Many results in the world Not solved problem – Dynamic Unification of Resources DYNAMITE SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Key element - middleware

Basic principle of a Metacomputer SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Basic principle of a Metacomputer Shared Memory Programming in Metacomputing Environment

SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Service Oriented Architecture Realization

SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Service Oriented Architecture Realization

The idea of Cloud Computing is the transmission of the organization of data computing and processing mainly from personal computers to the servers of the World Wide Web SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Introduction

Cloud Computing SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Basics

SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Tasks that cannot be solved in any different way We can use API and web-interface even in everyday tasks, which do not require large resources The idea of a co-usage of the resources on demand and only in the needed amount What tasks make us use a cloud?

Traditional methods of use of computational resources Cloud methods What tasks make us use a cloud? SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Cloud is determined completely by its API Operational environment must be UNIX – like SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Our principles

Cloud uses protocols, compatible with popular public clouds Cloud processes the data on the base of distributed file systems The consolidation of data is achieved by distributed Federal DB Our principles SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia.

Load balancing is achieved by the use of virtual processors with controlled rate Processing of large data sets is done via shared virtual memory Cloud uses complex grid – like security mechanisms SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Our principles

Virtual eco-system Corporate IT grid-architecture SPbSU and Institute for High Performance Computing and Integrated Systems, St.Petersburg, Russia. Convergention of means of virtualizaion, grid and SOA

Differences between having power and using it