Resin and Session Clustering

by Dana H. P'Simer

Wed, Apr 30, 2008

In a world where AJAX is becoming the norm instead of the exception we need technology platforms that allow us to scale these applications. I have been working on an application for a major hotel chain that is used by their call center agents to book reservations at their hotels. The application uses AJAX techniques to run several queries simultaneously and presents the data in a very dynamic interface that closely resembles a traditional desktop application. There are multiple tabs in many areas of the application that load data in the background so that it is available when the user wants to see it.

The application is deployed on Resin Pro v3.1.2 and we are attempting to use Resin’s session clustering technology. It seems that our sessions are too large and that Resin’s session clustering does not take into account the possibility that two requests could be accessing and even modifying the session at the same time. However, this is the nature of AJAX applications.

So we are looking for solutions. The problem with highly concurrent systems that require replication of changing data is that the less granular the replication capabilities of a clustering technology the more chance you have for conflicts. Take, for example a problem from the 90’s of database replication. Some of the first replication technologies worked at the record level. Changes to a record were replicated with the entire record so when the replicated change arrived at its destination, the entire record was either applied or not depending on conflict resolution policies. In a system that I worked on, we chose to use field level replication. In doing so, we were able to avoid most conflicts.

What we need is a finely granulated replication technology that will only replicate the changes to the object graphs rooted at the session. Resin’s session clustering technology serializes the session and writes it to disk and sends it over the network to the backup server. Once the serialized session reaches it’s destination, it replaces the session that may already be there based on conflict resolution policies.

We could put in our own session replication mechanism that increases granularity by replicating at the attribute level. However, since some of our objects contain significant amounts of data this may not be granular enough.

Another option that we are exploring is Terracotta. This is very interesting technology. It instruments the byte code as it is loaded into the JVM altering the behavior of PUTFIELD and GETFIELD opcodes. The Terracotta system uses a back-end server to hold the distributed shared object (DSO) state. On PUTFIELD, changes are replicated back to the back-end server and on GETFIELD, the referenced object is “Faulted In” if it is not yet present in the current JVM.

There are a number of things that Terracotta has to do to get this working.

  1. Each ClassLoader needs to have a unique and predictable name that will match the corresponding ClassLoader in all the nodes of the cluster.
  2. There are certain classes that cannot be replciated, like any of the java.lang.ref Reference classes. There is a list on Terracotta’s website. Most of these are quite obvious, like Thread
  3. For performance reason, we should not be instrumenting every class. Therefore Terracotta gives a way to include/exclude classes from instrumentation
  4. For session clustering, a Servlet Filter (javax.servlet.Filter) must be injected into the beginning of the filter chain so that the session can be watched and new attributes can be added to the cluster and removed attributes can be removed from the cluster.

To accomplish these tasks, Terracotta uses Terracotta Integration Modules ( TIMs ). These are modules that are configured using a subset of the OSGi standard and what they do is implement ASM ClassAdapters that will instrument classes in the container to inject the ClassLoader naming logic and the Filter.

Currently there is no Resin integration module that is fully functional. There is a project on terracotta forge. It is somewhat incomplete since it does not name all the ClassLoaders that need names and it does not inject the session filter.

Since we have not decided to go ahead with Terracotta for our sessions, we have not continued to work on this. I did, however, get a rudimentary application working with Terracotta session clustering. I ran into issues, that I knew I would run into, when I attempted to call our in house SDK to retrieve Hotel information and put that in the session. No go, but I know why that is and it is the subject of another blog article.