1. Object-Based Distributed Shared Memory
Arnshea Clayton
CSC 8320 Advanced Operating Systems

2. Overview Objects defined. Object-based memory organization.
Design considerations.
Advantages vs. Disadvantages.
Implementation: Linda
Implementation: Orca
Implementation: Enterprise JavaBeans
Future Directions

3. Objects defined
Objects are abstract data types defined by the programmer.
Objects encapsulate both data and operations on that data.
Data stores object state.
Operations called methods.
Object-Oriented design allows access to state only via methods.
Object-Oriented design is the most popular methodology in modern software engineering.
Objects are the fundamental unit of abstraction in object oriented design.

4. Object-based memory organization
Global address space abstracted as a collection of objects.
Address locations in global address space are not directly accessible.
Objects in collection, which is stored in the global address space, accessible to all processes.
Object methods can be invoked from any process.

5. Design considerations
Non-replicated scenario:
Lower bandwidth requirement in multicomputer scenario.
Single point of failure.
Potential performance bottleneck.
Object migration can reduce the impact of performance bottlenecks by moving object closer to invoking process.

6. Design considerations
Replicated scenario:
Memory consistency during updates.
Update all copies.
High communication overhead during updates.
Highest performance since all method invocations are local.
Invalidate old copies.
Lower communication overhead during update.
Potentially lower performance during access.
Can be mitigated by replicating current copy on demand.

7. Advantages vs. Disadvantages
Advantages to using Object-based distributed shared memory over other distributed shared memory paradigms:
Modularity. Encapsulation of data and operations into a single entity decreases coupling with respect to other modules.
Flexibility. Information hiding (data access through methods) centralizes control over data access. This makes it simpler and less error prone to modify an object. It also presents many opportunities for optimization.
Synchronization and access can be integrated together cleanly.

8. Advantages vs. Disadvantages
Disadvantages to using Object-based distributed shared memory over other distributed shared memory paradigms:
More difficult to incorporate into pre-existing programs that assume direct access to linear globally shared address space.
Performance penalty for data access since data access mediated through methods.

9. Implementation: Linda
Provides processes (possibly on multiple machines) with highly structured distributed shared memory (DSM).
Access to DSM only possible through a small set of primitives that must be incorporated into the programming language.
Includes small syntax extension.
Linda libraries, pre-compiler and runtime environment constitute entire DSM system.
Pre-compiler necessary to convert syntax extension into library calls useable by native language.
Linda’s small number of primitives simplifies incorporation of Linda into any existing programming language.
Reduces learning curve for programmers.

10. Linda: Memory Structure
Tuple: An ordered collection of elements.
(a, b, c, d) is a 4-tuple. (b, c, d, a) is a different 4-tuple composed of the same four elements in a different order.
DSM in Linda is a collection of tuples of varying sizes and types known as the tuple space.
Each tuple is distinguished by its size, the type and order of its elements and, ultimately, the value of its elements.

11. Linda: Memory Structure
By convention the first element in a tuple is typically a string. This string can be considered the “name” of the tuple.
(“abc”, 2, 5)
(“matrix-1”, 1, 6, 3.14)
(“family”, “is-sister”, “Carolyn”, “Elinor”)

12. Linda: Operations
Access to collection of tuples provided by four primitives: in, out, read and eval.
out – Writes the specified tuple to the tuple space.
out(“abc”, 1, 2) writes a 3-tuple to tuple space.
in – Reads (and removes) a tuple from the tuple space.
Content-based addressing. Memory identified by template.
Read and removal are atomic. If two processes compete for the same tuple only one will win; the other will block until a tuple satisfying the template becomes available.
in(“abc”, 1, ? i) searches the tuple space for a 3-tuple with “abc” as the first element and 1 as the second element. If a tuple matching this template is found it is removed from the tuple space and the value of its 3rd element is assigned to variable i.

13. Linda: Operations
read – Reads (but does not remove) a tuple from the DSM.
read(“abc”, 1, ? i) does the same thing as in from the previous example but does not remove the tuple from the tuple space.
eval – Evaluates parameters in parallel and deposits resulting tuples into DSM. This is how parallel processes are created in Linda.

14. Linda: Operations
Linda is not a fully-fledged object-based shared memory.
Small set of primitives do not allow definition of user defined operations on objects (tuples).
Linda’s memory organization is more structured than page-based or shared-variable shared memory and is therefore considered a kind of object-based shared memory.

15. Linda: Subspace Organization
in() and read() operations require search through tuple space to locate matching tuples.
2 optimizations employed to reduce search time:
Partition by signature: all tuples with the same signature (type, number and order of elements) stored in same partition.
e.g., (“abc”, 1, 3.14) stored in same partition as (“bcd”, 2, 2.78) but in different partition from (“abc”, 1, 3.14, 2.78).
Sub-partition by first element.
e.g., (“abc”, 1, 3.14) stored in same sub-partition as (“abc”, 2, 2.78) but in different sub-partition than (“bcd”, 2, 2.78).

16. Linda: Multiprocessor Implementation
Multiprocessor with shared memory (e.g., direct access, bus, ring, etc…) architecture supports simplest Linda implementation.
Entire collection of tuples stored, by subspace partition/sub-partition, in shared memory.
Access to collection synchronized using OS/hardware provided locking.

17. Linda: Multicomputer Implementations
Multicomputer scenario, where no single shared memory is available, requires different implementation.
Implementation 1- Local in()/Global out():
Every machine replicates entire tuple space.
Writes (out(), eval()) done to every replica.
Reads (in(), read()) use local tuple space.
Since in() requires delete a message must be broadcast to all other processes to delete the tuple from all replicas.
Implementation 1 is simple but high communication overhead due to broadcast requirement of write may reduce scalability for large collections with large number of machines.
S/Net Linda system (Carriero and Gelernter, 1986) uses this system.

18. Linda: Multicomputer Implementations
Implementation 2 - Global in()/Local out():
Every machine maintains partial collection.
Writes (out()) performed locally (only local collection modified).
Reads (in(), read()) performed by broadcast.
Processes with matching tuple respond with the matching tuple and, in the case of in(), delete the tuple from their local collection.
Excess responses stored by requester in local collection (effectively treated like a local out()).
Request repeated over increasing interval until response received.
Implementation 2 benefits from a lower memory requirement (each process only maintains a partial collection) than Implementation 1.

19. Linda: Multicomputer Implementations
Implementation 3 – Partial replication:
All processes are logically organized into a rectangular grid-like network.
Writes (out(), eval()) are written to all machines in the sender’s row.
Read (in(), read()) requests are broadcast to all machines in the reader’s column. Since the grid is rectangular there is always exactly 1 process guaranteed to have the tuple if any has it (the intersection of the written row and the read column).
Implementation 3 combines the best attributes of Implementations 1 and 2: Lower communication overhead (all broadcasts are partial) and lower memory requirement (each machine only maintains a partial replica of the tuple space).

20. Linda: Multicomputer Implementations
Implementation 4: No-broadcast solution.
Two part tuple space partitioning: by signature then by first element value.
Tuple server assigns each partition to a processor.
Read and Write requests go through the Tuple server.
Implementation 4 is ideal for communication constrained environments (e.g., widely distributed processors, processors distributed under extreme conditions, etc…).

21. Implementation: Orca
Orca (Bal, et al, 1990, 1992) is a more general object-based shared memory than Linda because it allows objects to contain user defined operations.
Orca is comprised of a language, compiler and runtime system.

22. Orca: Language Orca’s syntax is loosely based on Modula-2.
Each statement is a pair made up of a guard (non-side effect boolean expression) and a block of statements.
An operation is comprised of a set of statements.
The guard for each statement is evaluated in an unspecified order. Once a guard evaluates to true the corresponding block is executed.

23. Orca: Distributed Shared Memory
Objects become distributed in Orca by use of fork() operation.
fork(data) procedure; starts a separate process on the specified processor. The new process executes the specified procedure.
The data parameter is now shared between the invoking process and the newly created process.
Example: distributing an object to all processes
for i in 1 .. n do fork foobar(s) on i; od;
Each newly created process will have access to object s and will begin execution in procedure foobar.

24. Orca: Synchronization
Orca guarantees that each object method call is atomic.
Each method call is effectively a critical region the execution of which can only be undertaken by a single process at a time.
This limits the practical size of Object methods in Orca.
Multiple simultaneous calls to the same method are processed as if each were executed one after another in sequence.
The evaluation of guards in Orca provides condition synchronization
Parallel programs often need to restrict the execution of a set of instructions based on conditions that only apply to a subset of processes.

25. Orca: Memory Consistency
The Orca runtime can maintain a single copy of an object or replicate it across every process that uses it. Objects can be migrated between these 2 states.
Memory accesses (via method calls) can be read (read-only) or write (read/write).
For local single copy objects operations are performed locally (lock then read and/or write).
For remote single copy objects operations are performed via RPC (remote lock then read and/or write).

26. Orca: Memory Consistency
For replicated objects method invocations vary based on whether the object is modified (considered a write).
Read (non-modifying calls) on replicated objects are handled locally.
Writes (modifying calls) on replicated objects require updates to all replicated copies:
If reliable (error corrected), totally ordered broadcasting is available a blocking broadcast is used to update all copies (the sender blocks until all replicas acknowledge the update).
If broadcasting is unreliable, or broadcasting is unavailable, a two-phase commit protocol is used:
A lock on the primary copy is obtained.
The primary copy is updated.
The primary copy locks each remaining copy.
When all remaining copies are locked, the primary copy sends the update.
The primary copy unlocks the remaining copies.
The primary copy acknowledges the update and unlocks the primary copy.
NOTE: These locks are exclusive: reads and writes block until the locks are released.

27. Orca: Memory Consistency
Decision to replicate an object based on compiler estimate of read to write ratio:
Higher read to write ratio increases likelihood of replication.
The estimate is updated by the runtime on each fork() operation.
Object Migration is a side effect of all runtimes using the same algorithm to determine whether or not an object should be replicated

28. Object-based Distributed Shared Memory
Less Error prone than other forms of Distributed Shared Memory since synchronization is implicit.
The tradeoff is in programming complexity.
Programs that expect direct memory access have to be rewritten entirely.
In the case of Orca an entirely new language must be learned.

29. Implementation: Enterprise JavaBeans
Enterprise JavaBeans (EJBs).
Fully object oriented (including inheritance).
Both single-copy and replicated objects supported. Replication is handled by the runtime (called an Application Server). Replication algorithms vary from vendor to vendor.
Detailed Security and Persistence model.
Requires no new syntax in the java language.
Objects distinguished by whether or not they are stateless (Entity beans) or stateful (Session beans).
EJB 2.0 introduced Message oriented beans which persist until delivery and provide simplified model for asynchronous notification.

30. Future Directions Web-services Security
Coalescing around stateless distributed shared objects.
Stateless distributed shared objects allow greater scalability.
Industry standardization on protocols to support interoperability.
Security
Increased granularity, down to the level of individual method calls.

31. References
Andrew S. Tanenbaum, Distributed Operating Systems, Amsterdam, The Netherlands: Pearson Education, 1995, pp. 356 – 371
N Carriero, D Gelernter, “The S/Net’s Linda Kernel”, ACM Transactions on Computer Systems, vol. 4, pp , May 1986
V Krishnaswamy, “A Language Based Architecture for Parallel Computing” PhD Thesis, Yale Univ, 1991
Linda G. DiMichiel, Enterprise JavaBeans Specification v2.1, 2003,