Transparency in Distributed Operating Systems Vijay Akkineni

Operating Systems Generations Centralized Operating systems Network Operating Systems Distributed Operating Systems Cooperative Autonomous Systems Cloud Computing

Partitioning of COS ApplicationsFinanceWord processingWeb Application SubsytemsProgramming EnvrionmentDatabase Systems UtilitiesCompilerCommand Interpreter library System ServicesFile SystemMemory ManagerScehduler KernelCPU Multiplexing, Interrupt Handling, Device Drivers, Synchronization primitives, Interprocess communication

Network Operating System Peer to Peer communications. Seven layer OSI architecture. Examples Remote Login, File Transfer, Messaging, Network Browsing, Remote Execution.

Distributed Operating System Loosely coupled systems. Sharing or resources and coordination of resources. Transparency – Key difference between NOS and DOS. Distributed resources and activities are to be managed and controlled.

Distributed Operating Systems

Cooperative Autonomous Systems Characterized by Service Integration. Middleware – Cobra, JMS, RMI.

Transparency Hide irrelevant system dependent details from the users Higher Implementation Complexities Single System Image Minimal Knowledge

Location Transparency User has no awareness of object locations,objects are mapped and referred to by logical names. WebServices UDDI, Federated Services - SOA

Access Transparency Ability to access local and remote system objects in uniform way. The physical separation of system objects is concealed from the user. Examples – Accessing a file from the local file system and from a cloud drive.

Migration Transparency Logical Resources and Physical processes migrated by the system, from one location to another in an attempt to maximize efficiency, reliability, availability or security should be automatically controlled by the system Example – Application Servers using JNDI

Replication Transparency Exhibit consistency of multiple instances of files and data. System elements are copied to remote points in the system in an effort to possibly increase efficiencies through better proximity or provide increased reliability through duplication. Examples – Google's Big Table, HDFS.

Concurrency Transparency Sharing of Objects without interference. Similar to Time sharing. An important challenge when designing distributed systems is how to deal with concurrent accesses. Example – An important design goal for distributed database. Transactional Integrity and ACID properties during multiple transactions happening concurrently.

Failure Transparency Failure Transparency tries to mask failures so that they are not seen or noticed by the users. It is difficult to identify between a resource that has failed and a resource which is performing badly (slowly). Consider opening a webpage - is it dead or painfully slow, how long should the browser wait? Examples - Map Reduce Frameworks, DFS Replication on Data Nodes.

Performance Transparency Attempt to achieve a consistent and predictable performance level even with changes to system structure or load distribution. When parts of the system experience significant delay or load imbalance, the system is responsible for the automatic, rapid, and accurate detection and orchestration of a remedy. Examples - Load balancing, Speculative execution in Map Reduce.

Size/Scale Transparency A system's geographic reach, number of nodes, level of node capability, or any changes therein should exists without any required user knowledge or interaction. Research Area - Currently there is lot of ongoing research on running Map Reduce job across the data centers. Partition compute jobs based on geographical locality.

Revision transparency System occasionally have need for system-software version changes and changes to internal implementation of system infrastructure. Examples - Linux Kernel Upgrades and how it should not effect the existing software applications on the OS.

Revision transparency System occasionally have need for system-software version changes and changes to internal implementation of system infrastructure. Examples - Linux Kernel Upgrades and how it should not effect the existing software applications on the OS.

Parallelism Transparency The most difficult aspect of transparency,”Holy Grail” of distributed system designers. Systems parallel execution of processes throughout the system should occur without any required user knowledge. Examples – Parallel Algorithms on multicore processors and Map Reduce tasks on multiple systems.

Transparency Summarized Access Hide differences in data representation and how a resource is accessed LocationHide where a resource is located MigrationHide that a resource may move to another location Relocation Hide that a resource may be moved to another location while in use Replication Hide that a resource may be shared by several competitive users Concurrency Hide that a resource may be shared by several competitive users FailureHide the failure and recovery of a resource PersistenceHide whether a (software) resource is in memory or on disk

Major Research Areas AreasTransparencies Communication Networks Synchronization Distributed Algorithms Interaction and Control Transparency Process Scheduling Deadlock Handling Load balancing Performance Transparency Resource Scheduling File Sharing Concurrency Control Resource Transparency Failure Handling Configuration Redundancy Failure Transparency

Towards Cloud OS Permit autonomous management of its resources on behalf of its users and applications. Cloud OS operations must continue despite loss of nodes, entire clusters, and network partitioning. The Cloud OS must be operating system and architecture agnostic. The Cloud must support multiple types of applications, including legacy. Cloud OS management system must be decentralized, scalable, have little overhead per user and per machine and be cost effective

Logical Model of Cloud OS

Cloud Middleware Heterogeneous Nature of Cloud hindering adoption of Cloud Technologies. IBM is researching into Altocumulus middleware, which offers a uniform API for using Amazon EC2, Eucalyptus, Google AppEngine, and IBM HiPODS Cloud, aiming to provide an API which is Cloud agnostic.

References “HP performance-optimized datacenter (POD).” Data Sheet, “Amazon EC2.” [Online] Apache hadoop Map Reduce