Popular @
Forum Controls
Spotlight Features
The Rich Engineering Heritage Behind Dependency Injection
Andrew McVeigh takes us on a tour of the rich heritage behind dependency injection, what it represents, and tells us why its here to stay.
NetBeans 6: Matisse Updates
NetBeans 6 delivers great updates to the Matisse GUI builder. Spend a few minutes with Roman Strobl and get an expert briefing on what's new and what has changed. (sponsored)
Introduction to Groovy Part 3
In this, the third and final installation of Andres' Introduction to Groovy series, you learn about how Groovy handles variable numbers of arguments, named parameters, currying, and more about Groovy operators. Including, some new operators.
Easier Custom Components with Swing Fuse
Swing Fuse (actually just Fuse), is a framework designed to make it easier to create your own custom desktop components. In this article, Daniel Spiewak shows you how to get started and provides sample source code you can download.
Benchmark Analysis: Guice vs Spring
Willam Louth shows how he uses JXInsight Probes to investigate probable performance issues with code bases that he is not familiar with. He also highlights possible pitfalls in creating a benchmark, as well as in the analysis of results.
Replies:
11 -
Pages:
1
Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
URL: kunaalbhasin@gmail.com
At 11:25 PM on Jan 16, 2007, Kunal Bhasin wrote:
Fresh Jobs for Developers Post a job opportunity
My motivation to write this article came from a recent visit I made to one of our customers. They have a distributed JEE middleware layer which uses protocols like JMS and RMI to move data across multiple JVMs running on multiple commodity hardware boxes. The JMS layer is used for sending asynchronous messages whereas the RMI layer is used for sending broadcast messages to multiple consumers. During very high usage, the network pipe gets extremely busy (up to 70%). This is a gigabit network we are talking about. Obviously, there was a concern of the network getting completely clogged at the rate at which the usage was growing. In fact, the network would get so busy at peak usage times, that they had to introduce artificial pauses and they couldn't afford to send real time messages any more. This is the reason I was there to demonstrate how Terracotta could reduce this overhead significantly. We dicided to replace the RMI-boradcast layer with Terracotta DSO as a pilot. Let's look at how RMI and Terracotta would work in this scenario to understand the test.
A normal RMI invocation works as follows: a client application invokes a method on an object on the server. The parameters are marshalled and sent out over the network to the server machine, where the computations are performed, results are marshalled and sent back to the client. The two main areas where delays occur are the marshalling (object serialization and deserialization) and the network overheads. Object serialization overhead is a lot higher if you are sending deep object graphs as parameters since a deep-clone serialization (and deserialization) needs to be performed. In the above mentioned case, each consumer JVM ran a RMI server and the client JVM (the broadcaster) would iterate through a list of consumer IPs or hostnames and send a message (invoke a method) on each of the consumer JVMs.
Terracotta is an open source clustering solution that works as follows: each clustered JVM is policed or managed by a central Terracotta server. Whenever a shared object is mutated (within the boundaries of a lock), the delta level fine grained (field level) changes are maintained, and propagated to other clustered JVMs if and when the same object is accessed. Since there is no serialization and deserialization involved, there is no marshalling and unmarshalling overhead - no matter how deep the objec graph is. Also, since Terracotta only ships fine-grained changes, the difference in the amount of data sent across the network can be huge depending on the case. All this is possible with Terracotta as it maintains physical object identity across the cluster (i.e. foo == bar is preserved, not just foo.equals(bar)).
Ok, all this is in theory. Let's look at the actual test now. To simulate the customer environment, I wrote a simple RMI application with 1 broadcaster node broadcasting to 3 consumer nodes an object graph which is 10 levels deep and is of total size 10k. For this result, the boradcast happened every 10 millisenconds and I used ntop to monitor network usage.
Results:
The RMI broadcast sent a total of 2.6 GigaBytes of data over one hour i.e. ~ 722 KiloBytes per second.
With Terrcotta, the overall network usage over one hour was 491 MegaBytes i.e. ~ 136 KiloBytes per second.
This is a 5.3 times improvement by using the Terracotta solution. Now if you scale these results to 40 consumers and 4 broadcasters (actual real world scenario) - the improvement is ~ 11 times.
Let's analyze these results. In this case, Terracotta is able to achieve such improvements because in the entire 10K object graph which is 10 levels deep, only one String value (100 bytes) is changing and being broadcasted. With the RMI solution, the entire 10K object graph is being broadcasted to every consumer node whereas Terracotta only sends the bytes that have changed.
If we consider a non-broadcast use case where the changes do not need to be sent to every other node in the cluster, Terracotta would do a lot better whereas a serialization based clustering solution would still send the entire object graph to every other node to keep the cluster coherent. (There are some serialization based clustering solutions that implement a buddy replication system or some other mechanism where the object graph is not shipped to every other node, but the drawbacks associated with those is outside the scope here.) Terracotta guarantees cluster coherency whithout having to send the changes to every peer in the cluster because the central server knows which object is being accessed on which node through the lock semantics and by preserving object identity. In fact, due to the object identity being preserved, Terracotta sends fine grained delta-level changes ONLY WHERE they need to be sent i.e. only to the JVM where the same object is accessed.
All of the above improvements are over and above not having to maintain any RMI code and working with natural POJOs only. I have not pointed out the performance gains in the above test seen as a result of no serialization and not having to marshall and unmarshall the 10K, 10 level deep object graph as that is outside the scope of this discussion. I wanted to point out the impact network overhead can have on a typical JEE application.
For those who are interested, in the next part of this article I will post the source code of the test and instructions on how to run it.
Links:
ntop - http://www.ntop.org
RMI - http://en.wikipedia.org/wiki/Java_remote_method_invocation
Terracotta - http://www.terracotta.org
11 replies so far (
Post your own)
Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
Very interesting. I've been looking into Terracotta on the side lately, and it boggles me why I don't see more excitement about it. It is really an amazing piece of work. People tend to think of it as just a distributed cache or clustering solution, but it can be used for so much more. With some love and attention from the Java open source community, it could move into other solution areas where it would be a good fit (such as rich client, etc).Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
I think this ad-article is misguiding in that it bashes RMI for de-/serializing while claiming that Terracotta does no such thing. In my view you cannot send an object (and a representation of a partial object change would still be an object itself) over network without some kind of serializing.The point you should have made was that it's more efficient to only transmit delta changes than the whole new updated object graph.
Suggested clarifications
Kunal -Overall, an interesting article; however there were two details that I think that you should clarify:
1. Terracotta is licensed under a license that is not accepted by OSI (the Open Source Initiative that approves open source licenses), so you should not refer to it as "open source".
2. As an employee of Terracotta, you should point out that you are associated with the company when you write an article about your company's technology. Otherwise, it could be construed as deliberately misleading.
(To avoid pot & kettle analogies, my employer can be found in my sig below.)
Peace.
Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
> I think this ad-article is misguiding in that it> bashes RMI for de-/serializing while claiming that
> Terracotta does no such thing. In my view you cannot
> send an object (and a representation of a partial
> object change would still be an object itself) over
> network without some kind of serializing.
> The point you should have made was that it's more
> efficient to only transmit delta changes than the
> whole new updated object graph.
While you are technically correct, it is conceptually possible for them to do more efficient serialization of objects using the metainformation that they have, primarily because Java's serialization is relatively inefficient.
The same metainformation is also useful for doing partial object transfers and updates.
However, the big potential difference comes when an object is composed of many objects (and so on), and only one little bit out of all of that gets modified. I would characterize this as being a "sparse serialization", which should be efficient.
In most of the applications that I see, it would not make a performance difference because the individual object graphs (e.g. application "data objects") tend to be small. However, if a customer were managing a gigabyte-sized object and just tweaking parts of it, then this approach would obviously be preferable to the RMI "serialize the entire graph" approach.
Peace.
Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
Not using Java serialization isn't just about efficiency. There are more important reasons not to use Java serialization. I just wrote a blog entry about why.Re: Suggested clarifications
Cameron -[1. Terracotta is licensed under a license that is not accepted by OSI (the Open Source Initiative that approves open source licenses), so you should not refer to it as "open source".]
When talking open source, what matters is end-user use and vendor redistribution. As for end users, We say we are open source because anyone may take Terracotta source code and use it in current or modified form for any internal or customer facing applications without restriction. For example Tangosol may use our code in any
manner they wish and do so freely. We do request that people who redistribute our code give TC credit. We think that is quite liberal and in fact far less restrictive than some copy left viral licenses approved by the OSI.
[2. As an employee of Terracotta, you should point out that you are associated with the company when you write an article about your company's technology. Otherwise, it could be construed as deliberately misleading.]
I have stated several times on my blog that I am a Terracotta employee; I apologize but honestly have no control over my blog being syndicated in this manner. I hear the intent behind your comment, however, and will make sure to put such a disclaimer in my blog entries in the future. I did mention in this particular blog entry that I was visiting a customer with the intent of solving their problem with Terracotta
Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
Cameron -[In most of the applications that I see, it would not make a performance difference because the individual object graphs (e.g. application "data objects") tend to be small. However, if a customer were managing a gigabyte-sized object and just tweaking parts of it, then this approach would obviously be preferable to the RMI "serialize the entire graph" approach.]
Not serializing has benefits at all scales, not only at the Gigabyte level. We have seen significant benefits for even small data sets. Read our whitepapers at http://www.terracotta.org/confluence/display/docs/Articles to learn more about our relative performance gains. I also have a third party study I will put online ASAP that shows the relative benefit for small deltas to small object graphs, large deltas to large graphs, and everything in between. The study is conclusive that TC is very much faster than Java Serialization in all cases. And, most importantly, the lack of Java serialization gives us the following: http://techdem.blogspot.com/2007/01/dont-sacrifice-object-identity-to.html which is a benefit separate of performance.
Also, as I have discussed this particular article, our customer saw significant improvements with Terracotta with an object graph 10 levels deep and 10 KiloBytes in size.
Regards,
Kunal.
Re: Reducing bottlenecks in the JEE stack - Network Overhead: Part 1.
Jonas,[I think this ad-article is misguiding in that it bashes RMI for de-/serializing while claiming that Terracotta does no such thing. In my view you cannot send an object (and a representation of a partial object change would still be an object itself) over network without some kind of serializing.]
Thanks for reading and for commenting! First off, my intent to write this article wasn't to *bash* RMI. I have been a long time user of Java based technologies and frameworks including RMI and JMS and they have helped me to solve many problems. In many cases though, they have also left me frustrated. I was just sharing my recent experiences (of working with the Terracotta technology) and how it has helped me get over some of those frustrations. I thought this might be useful to many other folks like me and would help to make their programming lives easier. Here is an article by William Grosso which drills down on RMI/Serialization - http://www.oreilly.com/catalog/javarmi/chapter/ch10.html - and the first part of the article discusses some of the drawbacks I talked about (multiple object copies, network overhead due to deep clone serialization etc.).
I was comparing Terracotta to Java Serialization within the context of remote RMI calls. This blog entry by Hung throws some light on how Terracotta is able to achieve delta-level fine-grained changes over the network without serialization (in the true sense of the word) - http://unserializableone.blogspot.com/2006/12/using-java-bytecode-in-clustering.html
and this blog entry by Dem describes the advantage of preserving object identity of Java objects across multiple JVMs - http://techdem.blogspot.com/2007/01/dont-sacrifice-object-identity-to.html. Besides these, there is a lot more literature here - http://www.terracotta.org
In short, Terracotta internally maintains the state of shared objects and hence is able to do considerable optimizations with regards to sending the state changes of these objects - without relying on Java Serialization - and you don't have to maintain all the RMI code (multiple copies of objects all over the place).
Regards,
Kunal.
Re: Suggested clarifications
> For example Tangosol may use our code in any> manner they wish and do so freely.
I have considered the possibility, and spoke at length with Jonas Boner about ways we could integrate it with our Coherence software.
> We think that is quite liberal and in fact far less
> restrictive than some copy left viral licenses
> approved by the OSI.
Agreed. I would characterize it as a viral license whose only symptom is a logo
Peace.
Re: Suggested clarifications
[I would characterize it as a viral license whose only symptom is a logo]Cameron, just to be very clear, attribution is ONLY a requirement when someone wants to redistribute the Terracotta Software. This is something the vast majority of our users and prospects do not want to do. There is an enterprise license available that does not require attribution for those interested in re-distribution without attribution.
Regards,
Kunal.
Re: Suggested clarifications
> Cameron, just to be very clear, attribution is ONLY a> requirement when someone wants to redistribute the
> Terracotta Software. This is something the vast
> majority of our users and prospects do not want to
> do. There is an enterprise license available that
> does not require attribution for those interested in
> re-distribution without attribution.
As corporate lawyers point out, the terms "distribution" and "redistribution" are fairly flexible when it comes to software, so companies will often view almost any use as triggering the clause.
Unfortunately, we have seen this block adoption of other similarly licensed software (e.g. Mule).
Peace.