Presentation is loading. Please wait.

Presentation is loading. Please wait.

A DAPT IST-2001-37126 Clustering Implementation in JBoss Jakša Vučković Università di Bologna.

Similar presentations


Presentation on theme: "A DAPT IST-2001-37126 Clustering Implementation in JBoss Jakša Vučković Università di Bologna."— Presentation transcript:

1 A DAPT IST Clustering Implementation in JBoss Jakša Vučković Università di Bologna

2 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna2 Clustering J2EE Replicating ResourcesReplicating Resources CommunicationCommunication Concurrency ControlConcurrency Control Failure HandlingFailure Handling Controlling Access to External ResourcesControlling Access to External Resources

3 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna3 Replicating Resources Stateful Session Bean ReplicationStateful Session Bean Replication Entity Bean ReplicationEntity Bean Replication HTTP Session ReplicationHTTP Session Replication JNDI Tree ReplicationJNDI Tree Replication

4 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna4 Sticky Sessions and Homogeneous Deployment JBoss’ staff recommends the use of sticky sessions and homogeneous deploymentJBoss’ staff recommends the use of sticky sessions and homogeneous deployment –Using sticky sessions means that all requests of a client arrive to the same node during a session –Homogeneous deployment means that each application component is replicated an all the nodes, so requests don’t need to span on different nodes Sticky sessions + homogeneous deployment imply that each request will be entirely processed on one nodeSticky sessions + homogeneous deployment imply that each request will be entirely processed on one node

5 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna5 Stateful Session Beans Each instance of a SFSB is associated to a clientEach instance of a SFSB is associated to a client The lifetime of a SFSB is determined by a timeout from the last client accessThe lifetime of a SFSB is determined by a timeout from the last client access EJBs cannot generate threadsEJBs cannot generate threads All access to a SFSB instance should be serializedAll access to a SFSB instance should be serialized SFSBs are non-reentrantSFSBs are non-reentrant

6 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna6 SFSB Replication After each invocation of a SFSB, if the bean changed, a message is broadcast to the clusterAfter each invocation of a SFSB, if the bean changed, a message is broadcast to the cluster The message contains the serialized state of the session beanThe message contains the serialized state of the session bean Before broadcasting the message a distributed lock is acquiredBefore broadcasting the message a distributed lock is acquired The broadcast operation is synchronous (blocking) with acknowledgements from all the nodes of the clusterThe broadcast operation is synchronous (blocking) with acknowledgements from all the nodes of the cluster

7 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna7 RMI Clients The clients downloads an “intelligent” stub from the server and uses it to access the beanThe clients downloads an “intelligent” stub from the server and uses it to access the bean The stub encapsulates the load balancing policy and the failover mechanismThe stub encapsulates the load balancing policy and the failover mechanism The stubs maintains a list of server addresses which is updated on each requestThe stubs maintains a list of server addresses which is updated on each request

8 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna8 RMI Clients - Load Balancing JBoss provides two basic load balance policies:JBoss provides two basic load balance policies: –RoundRobin –FirstAvailable (Sticky Sessions) These Policies can be applied both to:These Policies can be applied both to: –Home Interface (Instantiating the bean) –Remote Interface (Invoking the bean methods)

9 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna9 RMI Clients - Failover If a call to a node fails the client stub fails over to another server or raises an exception to the client applicationIf a call to a node fails the client stub fails over to another server or raises an exception to the client application The stub fails over to another server when:The stub fails over to another server when: –there is a communication exception –The server does not responds because it crashed or a network partition occurred The stub reports an exception when:The stub reports an exception when: –There are no more servers to fail over to –The server reported a GenericClusteringException with the flag COMPLETED_YES or COMPLETED_MAYBE

10 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna10 RMI Clients - Failover If a client makes a request to a node and does not receive the response four scenarios are possible:If a client makes a request to a node and does not receive the response four scenarios are possible: 1.The node crashed before receiving the request – failover safe 2.The node received the request but crashed before broadcasting the state – failover safe 3.The node received and processed the request, broadcast the state and then crashed before sending the reply – the failover node must recognize a duplicate request The client indicates in the request message if it is a failover or the original requestThe client indicates in the request message if it is a failover or the original request

11 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna11 Partial Replication Problem We have two beans S1 and S2 on two nodes A and BWe have two beans S1 and S2 on two nodes A and B Client S1 S2 node A node B S1S2 S1

12 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna12 Partial Replication Problem The client invokes S1 on node AThe client invokes S1 on node A Client S1 S2 node A node B S1S2 S1

13 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna13 Partial Replication Problem S1 invokes S2S1 invokes S2 Client S1 S2 node A node B S1S2 S1

14 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna14 Partial Replication Problem Invocation of S2 is completed and its state is replicated on node BInvocation of S2 is completed and its state is replicated on node B Client S1 S2 node A node B S1S2 S1

15 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna15 Partial Replication Problem S1 tries to make another call to S2 but node A crashesS1 tries to make another call to S2 but node A crashes Client S1 S2 node A node B S1S2 S1

16 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna16 Partial Replication Problem Now we have an inconsistent state in node BNow we have an inconsistent state in node B Client S1 S2 node B

17 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna17 Partial Replication Problem There are two possible solutions to this problem:There are two possible solutions to this problem: –A replication aware Transaction Manager would need to roll back the state of S2 in B also –The updated state is broadcast to other nodes only when committing the whole transaction

18 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna18 Network Partitions A network failure could divide the cluster in two or more partitions that cannot communicateA network failure could divide the cluster in two or more partitions that cannot communicate Node A Node C Node B Node D No shared data (Entity Beans, DB) is accessed!!!

19 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna19 Network Partitions The client stub maintains a list of available serversThe client stub maintains a list of available servers If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partitionIf a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node C Node B Node D client No shared data (Entity Beans, DB) is accessed!!!

20 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna20 Network Partitions The client stub maintains a list of available serversThe client stub maintains a list of available servers If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partitionIf a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node C Node B Node D client No shared data (Entity Beans, DB) is accessed!!!

21 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna21 Network Partitions The client stub maintains a list of available serversThe client stub maintains a list of available servers If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partitionIf a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node C Node B Node D client No shared data (Entity Beans, DB) is accessed!!!

22 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna22 Network Partitions Each SFSB is associated to a client, so there will be no other clients accessing the instance on another partitionEach SFSB is associated to a client, so there will be no other clients accessing the instance on another partition Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB CD No shared data (Entity Beans, DB) is accessed!!!

23 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna23 Network Partitions Each clients list of available servers will be updated to contain only the nodes on its partitionEach clients list of available servers will be updated to contain only the nodes on its partition Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB CD No shared data (Entity Beans, DB) is accessed!!!

24 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna24 Network Partitions Merging When the cluster merges the clients will still have the old list of nodesWhen the cluster merges the clients will still have the old list of nodes Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB CD No shared data (Entity Beans, DB) is accessed!!!

25 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna25 Network Partitions Merging Only when the client makes the next request, the bean gets replicated on the other nodes and the client’s list is updatedOnly when the client makes the next request, the bean gets replicated on the other nodes and the client’s list is updated Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB CD No shared data (Entity Beans, DB) is accessed!!!

26 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna26 Network Partitions Merging S3 is not updated so the client must maintain a separate list of nodes for each SFSBS3 is not updated so the client must maintain a separate list of nodes for each SFSB Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB CD No shared data (Entity Beans, DB) is accessed!!!

27 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna27 Network Partitions Merging Problem Suppose node C crashes nowSuppose node C crashes now Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB D No shared data (Entity Beans, DB) is accessed!!!

28 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna28 Network Partitions Merging Problem Client 2 wants to invoke s2 and fails over to node AClient 2 wants to invoke s2 and fails over to node A Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB D No shared data (Entity Beans, DB) is accessed!!!

29 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna29 Network Partitions Merging Problem If s2 invokes s3 we find an “out of date” state on this nodeIf s2 invokes s3 we find an “out of date” state on this node Client 2 Client 1 s2 s3 s1 s2 s3 s1 s2 s3 s1 AB D No shared data (Entity Beans, DB) is accessed!!!

30 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna30 Entity Beans Entity Beans are persistentEntity Beans are persistent Unlike Session Beans, Entity Beans are not associated to clientsUnlike Session Beans, Entity Beans are not associated to clients Can be accessed by multiple clients, but not concurrentlyCan be accessed by multiple clients, but not concurrently If an Entity Bean is declared as reentrant it can be accessed concurrently but only from the same transactionIf an Entity Bean is declared as reentrant it can be accessed concurrently but only from the same transaction

31 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna31 Entity Beans JBoss does not replicate Entity Beans, replication is left to the database tierJBoss does not replicate Entity Beans, replication is left to the database tier JBoss controls concurrent access to the databaseJBoss controls concurrent access to the database All JBoss nodes share the same databaseAll JBoss nodes share the same database It is transparent to JBoss if the database is clustered or notIt is transparent to JBoss if the database is clustered or not

32 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna32 Entity Beans Entity Bean state is loaded from the DB before each business method invocation and stored after the invocation (Commit option B and C)Entity Bean state is loaded from the DB before each business method invocation and stored after the invocation (Commit option B and C) To each J2EE transaction is associated a DB transactionTo each J2EE transaction is associated a DB transaction There are to policies to deal with concurrency:There are to policies to deal with concurrency: –Optimistic –Pessimistic

33 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna33 Optimistic Policy Transactions are executed concurrentlyTransactions are executed concurrently Each Transaction has it’s own instance of the Entity BeanEach Transaction has it’s own instance of the Entity Bean Shared row locking is used in the databaseShared row locking is used in the database Before committing the container verifies if the Bean changed in the databaseBefore committing the container verifies if the Bean changed in the database –If yes the transaction is rolled back and re-executed –If not the new value is stored in the DB

34 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna34 Pessimistic Policy Transactions are not executed concurrentlyTransactions are not executed concurrently Exclusive Row locking at the DB is usedExclusive Row locking at the DB is used A transaction blocks if it tries to access a locked row until the row is releasedA transaction blocks if it tries to access a locked row until the row is released A Distributed Deadlock Detection algorithm is used to detect DeadlocksA Distributed Deadlock Detection algorithm is used to detect Deadlocks

35 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna35 Entity Bean Replication Failures If the node owner of a transaction crashes the DB rolls back the transactionIf the node owner of a transaction crashes the DB rolls back the transaction Network partitions must be handled by the DB clustering mechanismNetwork partitions must be handled by the DB clustering mechanism With a non replicated database, the approach of a primary partition is used where the primary partition is defined to be the one containing the databaseWith a non replicated database, the approach of a primary partition is used where the primary partition is defined to be the one containing the database

36 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna36 Entity Bean Replication Relies intensively on DBRelies intensively on DB –Maximizes DB I/O If we don’t want a single point of failure, we must use a replicated databaseIf we don’t want a single point of failure, we must use a replicated database Entity Beans are not used as cacheEntity Beans are not used as cache A distributed cache with distributed locks is under developmentA distributed cache with distributed locks is under development

37 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna37 HTTP Session Replication JBoss is able to work with two Serlvet ContainersJBoss is able to work with two Serlvet Containers –Jetty –Tomcat With both containers we have three replication schemes:With both containers we have three replication schemes: –Instant Snapshotting: HTTP Sessions are replicated after each HTTP request –Interval Snapshotting: HTTP Sessions are replicated on a time based interval –Economic Snapshotting (JBoss 3.2): HTTP Sessions are replicated only if setAttribute() has been called

38 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna38 HTTP Session Replication An HTTP Session is implemented as a serializable objectAn HTTP Session is implemented as a serializable object The HTTP Session state is NOT broadcast to the other nodes but replication is implemented through an Entity BeanThe HTTP Session state is NOT broadcast to the other nodes but replication is implemented through an Entity Bean The HTTP Session is a cmp-field of that beanThe HTTP Session is a cmp-field of that bean The CMP engine serializes the HTTP Session object along with its contentsThe CMP engine serializes the HTTP Session object along with its contents The HTTP Session contains also references (remote objects) of SFSBsThe HTTP Session contains also references (remote objects) of SFSBs SFSBs do not get serialized with the HTTP SessionSFSBs do not get serialized with the HTTP Session

39 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna39 HTTP Session Replication Failures are handled by the EJB Replication mechanismFailures are handled by the EJB Replication mechanism Client fail over behavior depends on the HTTP dispatcherClient fail over behavior depends on the HTTP dispatcher A JBoss HTTP Dispatcher is under developmentA JBoss HTTP Dispatcher is under development

40 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna40 JNDI Tree Replication Each node has a local JNDI TreeEach node has a local JNDI Tree All nodes share a replicated HA-JNDI TreeAll nodes share a replicated HA-JNDI Tree When a client looks up an object on node NWhen a client looks up an object on node N –The replicated HA-JNDI service is invoked first –If it cannot find it there it looks in the local JNDI Tree –If it’s not there it asks all other nodes if they have it in their local JNDI Tree –If it hasn’t been found a NameNotFoundException is thrown

41 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna41 JNDI Tree Replication Unlike clients, EJBs access only the local JNDI TreeUnlike clients, EJBs access only the local JNDI Tree This solution was chosen because ofThis solution was chosen because of –Compatibility with existing applications –Distinction between local and replicated objects –Low network traffic in homogeneous clusters

42 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna42 Distributed Cache Deadline for June (JavaONE)Deadline for June (JavaONE) Based on JavaGroupsBased on JavaGroups Will be a Jboss ServiceWill be a Jboss Service Highly ConfigurableHighly Configurable Will be applied to Entity Bean Clustering and Session Bean ClusteringWill be applied to Entity Bean Clustering and Session Bean Clustering Will enable replication of the DatabaseWill enable replication of the Database

43 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna43 Distributed Cache Semantics Three semantics:Three semantics: –Asynchronous broadcast update and return immediatelybroadcast update and return immediately –Synchronous broadcast update and wait for acknowledgementsbroadcast update and wait for acknowledgements –Serialized Synchronous acquires locks before updating a beanacquires locks before updating a bean ensures ordering but not consistencyensures ordering but not consistency consistency is obtained at another levelconsistency is obtained at another level

44 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna44 Cache Contents The cache will implement the XAResource interfaceThe cache will implement the XAResource interface This will enable the use of the cache in a Two Phase Commit protocolThis will enable the use of the cache in a Two Phase Commit protocol The mapping of the XAResource semantics to the cluster is not a trivial problemThe mapping of the XAResource semantics to the cluster is not a trivial problem This could enable to do the DB update, cluster replication and sending the response to the client an atomic operationThis could enable to do the DB update, cluster replication and sending the response to the client an atomic operation

45 A DAPT Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna45 Conclusions JBoss clustering algorithm does not handle all failure patterns correctlyJBoss clustering algorithm does not handle all failure patterns correctly Simply changing the GC semantics will not solve the problemsSimply changing the GC semantics will not solve the problems To make the existing mechanism work correctlyTo make the existing mechanism work correctly –A replication aware Transaction Manager needs to be implemented –During state merging the “out-of-date” beans must be marked An alternative would be to re-implement the clustering from scratchAn alternative would be to re-implement the clustering from scratch The development of the Distributed Cache by JBoss authors is a good opportunity to redefine the clustering algorithmThe development of the Distributed Cache by JBoss authors is a good opportunity to redefine the clustering algorithm


Download ppt "A DAPT IST-2001-37126 Clustering Implementation in JBoss Jakša Vučković Università di Bologna."

Similar presentations


Ads by Google