NINJA

Project of UC Berkeley Computer Science Division Paper : The Ninja Architecture for Robust Internet-Scale Systems and Services People : Eric Brewer,David Culler, Anthony Joseph, Randy Katz,Steven Gribble, Matt Welsh Randy Katz,Steven Gribble, Matt Welsh The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Points to be covered :Points to be covered : Goal : The end goal of the Ninja project is to enable the development of a menagerie of Internet-based services development of a menagerie of Internet-based services which are interoperable and immediately accessible which are interoperable and immediately accessible across the spectrum of user devices ranging from PCs across the spectrum of user devices ranging from PCs and workstations to cellphones and Personal Digital and workstations to cellphones and Personal Digital Assistants. Assistants. Examples of Applications of Ninja Vision : * Sanctio * Sanctio * Juke Box * Juke Box

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Components of Ninja Architecture : 1)Building Robust services * Scalability * Availability * Scalability * Availability * Fault-Tolerance * Consistency * Fault-Tolerance * Consistency * Persistence * Persistence All this in under the Growing Traffic Load All this in under the Growing Traffic Load * Concept of Base : Cluster of Workstations * Concept of vSpace :Cluster based software Platform

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices 2)Device Diversity : Concept of Units 3)Adaptation by Active proxies : Concept of Distillation * Computational power * Computational power * Network Connectivity * Network Connectivity * Interface Capabilities of devices * Interface Capabilities of devices e.g. Wireless Base stations, Network gateways, Firewalls, Gateways. Firewalls, Gateways. 4) The Composition of Services : Composition of services into a PATH as well as adaptation along that PATH. * SDS (Service Discovery Service ) * SDS (Service Discovery Service )

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Bases : Scalable Platforms for internet Services # Concept of Conditioning the Service: Graceful operation during the bursts during the bursts # A programming Model : * Splitting a Service into Stages * Splitting a Service into Stages e.g. : web server service = reading + retrieving + return e.g. : web server service = reading + retrieving + return * Design Patterns * Design Patterns 1.Wrap 2. Combine 1.Wrap 2. Combine 3. Pipeline 4. Replicate 3. Pipeline 4. Replicate # Java –Based I/O Substrate Implementation # Java –Based I/O Substrate Implementation Jaguar : (Java Access to generic Underlying Architectural Resources ) Jaguar : (Java Access to generic Underlying Architectural Resources ) Extension to the Java Environment Extension to the Java Environment

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices #vSpace: Cluster based execution environment for * Replication * Load balancing * Fault Tolerance Concept of Concept of * Worker : fixed size thread pool + methods * Worker : fixed size thread pool + methods * Service definition : set of workers * Service definition : set of workers * Service Publication :freezing of the code * Service Publication :freezing of the code * Clone :replication of the workers along * Clone :replication of the workers along various other nodes various other nodes * Clone Group : set of worker clones * Clone Group : set of worker clones

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Distributed Data Structures : # Normal D.S. Like : Binary tree, Hash Table etc. # Properties Expected : * High throughput * High throughput * High Concurrency * High Concurrency * Availability * Availability # Strictly Consistent in Nature : # Atomic in Nature

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Active proxies : Execute in an environment local to devices and hence perform context aware optimizations and transformations on behalf of devices 1)Dynamic Service Adaptation : Distillation of service content into a format more suitable for small devices content into a format more suitable for small devices eg. 1) HTML to WML for WAP phones eg. 1) HTML to WML for WAP phones 2)Design of the light weight protocols 2)Design of the light weight protocols 2) Secure Service Access from Diverse Clients: trusted active proxies can perform context aware transformations on data before it arrives at a kiosk active proxies can perform context aware transformations on data before it arrives at a kiosk 3) Multiple device fusion : Combining capabilities of various devices.Eg. Display of local kiosk and entering data thru PDA various devices.Eg. Display of local kiosk and entering data thru PDA

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Paths : Composition of Services Across the Wide-Area 1)Automatic Path Creation : (APC ) * Finding path between the networks * Finding path between the networks * Creating the network connection * Creating the network connection * Fine Tuning the performance of data flow * Fine Tuning the performance of data flow * Error Correction * Error Correction 2) Path = Operators (perform computation on data) + Connectors (protocol translations between + Connectors (protocol translations between operators ) operators ) 3) Operators : Strongly types and know the input they accept and output they produce. accept and output they produce.

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices * Operators have Cost parameters associated with it which helps in route optimization. helps in route optimization. * Types of Operators : * Long lived :- standard Ninja services * Long lived :- standard Ninja services * Dynamically generated : light weight and short * Dynamically generated : light weight and short lived transformation elements created by the lived transformation elements created by the APC facility as and when required APC facility as and when required Both these types of the operators ensure that that the path can be reconstructed when A failure occurs. can be reconstructed when A failure occurs. * Example of Path : Mapquest Service

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Logical path Creation Physical path Creation Path Instantiation and Execution Path Tear Down Path Construction Process

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Service Location Service across the Wide Area (SDS): * Services announce their presence * Services announce their presence * Location of the announced services * Location of the announced services * help clients decide the trustworthiness of the * help clients decide the trustworthiness of the service and vice a versa. service and vice a versa. # Properties of the SDS : * Fully secure * Fully secure * Semantically rich * Semantically rich * Scalable * Scalable * Fault Tolerant * Fault Tolerant

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Design of Service Discovery Service : 1) Elements of the SLS 1) Elements of the SLS * Clients * Services * Clients * Services * SDS servers. * SDS servers. 2) Hierarchical arrangement of servers : Domain concept 3) Uses of XML for encoding both services and queries 4) Certificate Authority : (CA) trusted source which provide the proof of binding between a principal and its public the proof of binding between a principal and its public and Private keys in the form of certificates. and Private keys in the form of certificates. 5) Capability Manager : (CM) Manages individual access control Lists on behalf of each authenticated service. control Lists on behalf of each authenticated service. Hiding the services rather than disallowing the access Hiding the services rather than disallowing the access

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices in SDS Wide Area Operation in SDS * Information is propagated up to parent servers and queries are partitioned among and forwarded to the relevant servers. * To prevent upper level servers in the hierarchy from being loaded by the query load SDS filters the information while it propagates the information forward. * SDS servers dynamically organize themselves into potentially multiple hierarchies for data partitioning and query routing.

The Ninja Project Enabling Internet-scale Services from Arbitrarily Small Devices Ninja and Related Work : Ninja is not Java 1) Jini 2) EJB 1) Jini 2) EJB 3) CORBA 4) Centaurus 3) CORBA 4) Centaurus 5) Active networks 5) Active networks Future Work in Ninja 1)Challenges in automatically composing service components to meet the needs of particular device components to meet the needs of particular device 2) Unsolicited advertising : SPAM 3) Requirement of Services capturing bits rather than eyeballs eyeballs