Advanced Operating Systems Chapter 6.1 – Characteristics of a DFS Jongchan Shin

Table of contents What is DFS? Characteristics of DFS - Transparency - Concurrent file updates - File replication - Fault tolerance - Consistency - Security - Efficiency Current work Future work

What is DFS? (Distributed File System) client / server-based application that allows clients to access and process data stored on the server as if it were on their own computer.

Characteristics of DFS Transparency - Access transparency - Location transparency - Mobility transparency - Performance transparency - Scaling transparency

Access transparency Client programs should be unaware of the distribution of files.

Location transparency Client program should see a uniform namespace. Files should be able to be relocated without changing their path name.

Mobility transparency Neither client programs nor system admin program tables in the client nodes should be changed when files are moved either automatically or by the system admin.

Performance transparency Client programs should continue to perform well on load within a specified range.

Scaling transparency Increase in size of storage and network size should be transparent.

Concurrent file updates Changes to a file by one client should not interfere with the operation of other clients simultaneously accessing or changing the same file.

File replication Several copies of a file’s contents at different locations enable multiple servers to share the load of providing the service

File tolerance It is essential that the file service continue to operate in the face of client and server failures

Consistency File systems have offered on-copy update semantics. One-copy update semantics : a model for concurrent access to files in which the file contents seen by all of the processes accessing or updating a given file are those that they would see if only a single copy of the file contents existed.

Security In addition to access control lists, there is a need to authenticate client requests so that access control at the server is based on correct user identities.

Efficiency A goal is to provide a comparable level of performance of at least the same power and generality as conventional file systems.

Current work - HaDoop HDFS -Hadoop Distributed File System -Provides a distributed file system and a framework for the analysis and transformation of very large data sets using the MapReduce (Distributed computation framework) paradigm. -Stores file system metadata and application data separately.

Current work - HaDoop -NameNode a master server that manages the file system namespace and regulates access to files by clients. -DataNode Manage storage attached to the nodes that they run on.

Future work -Using big data and distributed processing -From source to destination -Which station is the most people => For advertisements or promotions.

References [1] file-systemhttp://searchcio-midmarket.techtarget.com/definition/distributed- file-system [2] [3] [4] [5] Randy chow, 1997, Distributed operating systems & Algorithms [6] Shvachko, K. ; Hairong Kuang ; Radia, S. ; Chansler, R. “The Hadoop Distributed File System”, Mass Storage Systems and Technologies (MSST), 2010 IEEE 26 th Symposium on.

Q & A