Popular @
Forum Controls
Spotlight Features
The Rich Engineering Heritage Behind Dependency Injection
Andrew McVeigh takes us on a tour of the rich heritage behind dependency injection, what it represents, and tells us why its here to stay.
NetBeans 6: Matisse Updates
NetBeans 6 delivers great updates to the Matisse GUI builder. Spend a few minutes with Roman Strobl and get an expert briefing on what's new and what has changed. (sponsored)
Introduction to Groovy Part 3
In this, the third and final installation of Andres' Introduction to Groovy series, you learn about how Groovy handles variable numbers of arguments, named parameters, currying, and more about Groovy operators. Including, some new operators.
Easier Custom Components with Swing Fuse
Swing Fuse (actually just Fuse), is a framework designed to make it easier to create your own custom desktop components. In this article, Daniel Spiewak shows you how to get started and provides sample source code you can download.
Benchmark Analysis: Guice vs Spring
Willam Louth shows how he uses JXInsight Probes to investigate probable performance issues with code bases that he is not familiar with. He also highlights possible pitfalls in creating a benchmark, as well as in the analysis of results.
Using JMS For Distributed Software Development
At 12:28 PM on Feb 13, 2006, Matthew Schmidt wrote:
Fresh Jobs for Developers Post a job opportunity
Read on to hear Roger's experiences.
24 replies so far (
Post your own)
Re: Using JMS For Distributed Software Development
This is a fine article on the features of a JMS server, but I would be more interested in the techniques and design choices involved in the developing of applications that work against such servers.Re: Using JMS For Distributed Software Development
gregor hohpe's book and website aren't bad for a starting point. There are lots of async messaging architecture patterns included.http://www.eaipatterns.com/index.html
http://www.amazon.com/gp/product/0321200683
I haven't seen any good recent JMS books. I think the last decent one was in 2002.
Re: Using JMS For Distributed Software Development
Dear Will,I am just working with JMS quite intense in a WebSphere/MQ Server environment. Where the project itself allows it (both time, and what general MQ topics to extract out of it) I might share some tips and techniques right from a practical case here.
If you need anything more "desparate", please just leave me a message here with my profile...
Re: Using JMS For Distributed Software Development
And most of the JMS books I've seen are basically focused on the actual plumbing details rather than the higher level design issues of using MOM.That's a helpful website, but it's focused more on the integration issue of disparate parts of the system.
Roger is (or at least seems to be from prior postings) an advocate of messaging not just on the edges of a system for integration and interfacing, but as a more fundamental design element of the system, including using it instead of where one might typically use an RPC call or, better said, simply using Messaging as an RPC medium.
Cameron is also a similar advocate, but neither of them has as of yet been able to provide any more detail about the design and decision making process of using Messaging over RPC, or any patterns or idioms that come up during the development of such a system.
I read this article hoping that Roger would address those issues, but was rather disappointed to find it more an emueration of the features of a current MOM server.
I'm not suggesting their wrong or their advocacy is misplaced, just that they have yet to back it up with more detailed examples.
books on JMS programming
JMS Programming BooksEnterprise JMS Programming
by Shaun Terry
Paperback: 408 pages
Publisher: Wiley; 1st edition (February 1, 2002)
Java Message Service
by Richard Monson-Haefel, David Chappell
Paperback: 238 pages
Publisher: O'Reilly Media, Inc.; 1 edition (December 1, 2000)
Java Message Service API Tutorial and Reference: Messaging for the J2EE Platform
by Mark Hapner, Rich Burridge, Rahul Sharma, Joseph Fialli, Kim Haase
Paperback: 544 pages
Publisher: Addison-Wesley Professional; 1st edition (February 26, 2002)
I have read the first book in this list and can heartily recommend it for those first learning to program JMS. It has basically two sections - the first being about JMS and the APIs with some guidance of when and how to use them, with the latter being devoted to more extensive examples. There is also coverage of the Message Driven Bean EJB.
Messaging Pattern Book
Enterprise Integration Patterns : Designing, Building, and Deploying Messaging Solutions
by Gregor Hohpe, Bobby Woolf
Hardcover: 686 pages
Publisher: Addison-Wesley Professional (October 10, 2003)
I picked up this book after I had already been doing JMS programming for some time (I had also had experience with MSMQ). This book attempts to be somewhat messaging technology neutral and strives to present fundamental patterns. That's probably a good thing but after such heavy exposure to JMS I found much of the material rather redundant and the examples that were couched in non-JMS messaging settings rather soured me overall.
Well, it's just that JMS is the strongest general purpose enterprise messaging API around and all these other messaging technologies are quite inferior. Given that most JMS implementations have heterogeneous support for other languages and platforms, it's not as though JMS is limiting to Java only. Indeed, I find JMS much more universal, than say Microsoft's MSMQ. Also, JMS has quite a number of useful features and idioms that other systems don't have. When you use JMS you really develop a style for doing things that depends on those capabilities. Consequently a lot of the presented material, where JMS was absent, seemed rather retrograde relative to where I was at.
This is one of those patterns books that may be good to have on the shelf, but if I'd only bought a JMS book and learned that, I would have been just as well off in my overall skill set and knowledge of messaging.
JMS Newbie
Like many JEE developers, my experience with JMS is pretty limited (using a queue for async logging).If I have a rich client, would it be feasible to do all communication to/from the app server using async JMS? How do I 'get stuff back'? Can I send a query to the server, and the server send something back on another queue to the client? Can clients be message consumers w/o embedding an EJB container?
Thanks for listening.
Re: JMS Newbie
> If I have a rich client, would it be feasible to do> all communication to/from the app server using async
> JMS? How do I 'get stuff back'? Can I send a query
> to the server, and the server send something back on
> another queue to the client? Can clients be message
> consumers w/o embedding an EJB container?
You can certainly use JMS for your rich client. Some might argue that you SHOULD use JMS (or some other asynchronous protocol) as GUI applications are particularly suited, even within themselves, to asynchronous programming.
The J2EE tutorial has a nice little JMS section which you can play with for examples.
Regarding your specific question, one of the properties of a JMS message can be a "Reply To" queue that is passed along with the message. There is also the concept of "anonymous" queues, basically queues created on the fly by a program rather than preconfigured and named within the JMS server.
Combining these techniques, your program can create an anonymous queue on start up, and attach it to any message being sent out using that Reply To property. The handler of the message can then send back data on that queue. This call back is not automatic, however, you need to write the code to send the results back.
Meanwhile, your program can either simply block on that queue waiting for a message using the recieve() method, or register a MessageListener which has an onMessage method as an asynchronous message handler that is called whenever any message on your queue is received. Your handler would then take the retrieved message and inject its results in to your global event dispatch system just like, say, a mouse click or anything else.
Re: JMS Newbie
One thing to keep in mind is that with JMS there is quite a bit of flexibility of how to do things. One might tend to develop a style or approach that suits oneself but that wouldn't necessarily be the only way. With that being said, I'll try to respond:> Like many JEE developers, my experience with JMS is
> pretty limited (using a queue for async logging).
>
> If I have a rich client, would it be feasible to do
> all communication to/from the app server using async
> JMS?
Yes. All my distributed clients, which are coded as .NET Winform rich-client apps, communicate with the middle-tier strictly via JMS message passing. (Also, there are some embedded computer end-points that communicate via JMS as well - they control or interact with hardware and in turn reflect or respond to the rest of the world via JMS messaging. At my company we successfully use JMS messaging to do quite a bit of soft real-time programming which spans the network.)
[Refer to my article for the section on bridging and the Saleforce.com SOA data center example. By only using messaging and no RPC it becomes possible to devise such arrangements. I also use a similar technique to test applications that are under development by copying and bridging messages from actual live production systems - using a bridge rule and the shadow queue technique that I describe in the article. I can not overstate how useful it is to be able to develop a soft real-time applications using live, real-world data feeds. Again, doing such is not as feasible with synchronous point-to-point protocols.]
A benefit of using JMS is that communication can be initiated bi-directionally. Your clients could become topic subscribers and business logic in the middle-tier (or even the database back-end) could publish notifications events to that topic which your clients respond to. JMS typically makes use of persistent socket connections so that your clients see a notification event as soon as it gets published into the topic. Even still, the message is held safely queued until delivered to all subscribers.
In the jargon this is referred to as server-side push of notification events and can be a very handy feature to design software to utilize if it's available. In the world of HTTP and the Internet, RSS - a polling solution - is used to accomplish similar objective. (Polling, if not fine grained, is not adequate for soft real-time control, though, and if is made fine grained, then tends to become too much overhead. It helps to use queuing in conjunction to either polling or event notification and JMS certainly has intrinsic queuing.)
In JMS a topic is a specialization of a queue and is intended for a loosely coupled, one-to-many fan-out of message dissemination. RPC, in contrast, is point-to-point so an RPC connection would have to be established from the publisher to each intended recipient. This could be a considerable configuration burden and in some cases of firewalls, NAT translation, etc., it may not even be possible for the middle-tier to initiate RPC connections to client end-points. Also, rather surprisingly, not all messaging systems implement a one-to-many pub/sub capability. Some messaging systems offer nothing more than queued point-to-point communication. (With AJAX, you don't even get any queuing - just asynchronous invocation.)
> How do I 'get stuff back'?
So a topic could be used to push messages to clients at any arbitrary time. Thus this could be a means of returning results back to clients. If the result needs to be processed by a very specific client, then just attach a property to that message that is unique to the targeted client. That client can filter for messages it receives from the topic based on that property. A given distributed application could use a single well-known topic for dispensing messages to all interested client end-points - whether the messages be notification events or returned results. It can be a nice simplification if all in-coming messages are processed out of a single JMS destination (JMS jargon for queue or topic) - but it is merely just a kind of a style preference.
> Can I send a query to the server, and the server send
> something back on another queue to the client?
As said at the outset, JMS is rather flexible and there can be multiple ways to do things based on situation or style preference. We just noted that topics can be used to return results asynchronously to a client. However, one can also associate a queue or topic with the out-going messages that the client publishes. The consumer of these messages can then publish results to that specified queue or topic where the client will then receive and process the result message. There is a standard message property called ReplyTo that can be set for this purpose. (Just keep in mind that if the result message is published to a topic then a property needs to be associated to it that will uniquely address the message for the specific intended client.)
Now it is also possible for the client to create a private temporary queue. This queue can be set in the ReplyTo property of out-going messages. Then only that specific client will receive the result message. This approach then becomes very similar to doing a synchronous RPC call: issue a message and block while waiting for the response message. This can be avoided, though, by generally using the private temporary queue to handle all responses for all out-going messages and do so with a generic asynchronous message listener callback. The tmp queue gets created early on and is used through out the run of the application. Out-going messages are published and their ReplyTo property is set to this tmp queue - but one no longer blocks waiting for the response in a synchronous fashion. Instead just rely on the installed message listener callback to handle any result message when it happens to arrive. In this way the application can be designed as an asynchronously driven state machine. Over the years I've found this the most effective way to organize the design of a distributed application end-point.
Personally I tend to use the first approach that I outlined above. I have all notifications and result messages pouring into a single topic that all clients subscribe to. A property on the message will denote it for a specific client when that is necessary. My applications tend to be built where there is one topic for in-coming messages and one queue for out-going messages. I establish a single JMS connection to the JMS server and create multiple JMS sessions on that single connection (a given JMS session is safe to use in only one thread context). A session for publishing or a session for consuming messages can be created to operate concurrently on different threads. These sessions can be left in operation for the duration of the application run where they operate in a completely asynchronous manner. (If your messaging threads move data to or from the GUI display objects, just remember to marshal those interactions onto the app's main GUI thread.)
> Can clients be message consumers w/o embedding an EJB container?
Yes. The EJB container for Message Driven Beans (MDB) is just a particular JMS client that has been written to use the JMS APIs. It provides for a convenient way to code a bean component to consume messages within the context of a JEE app server. But otherwise it is nothing special - well, it also makes it possible for JMS to participate in JEE distributed transactions. (BTW, MDBs can also publish messages - such if a result message needs to be returned to the other messaging end-point.)
When writing, say, a GUI desktop client, one need only insure the JMS client libraries are in the classpath and then code to the JMS spec. Even for such clients you can do such things as publish multiple messages to a JMS queue or topic under the atomicity of a transaction (i.e., either all messages will get published - or if rolled back due to an error, no messages get published). Occasionally this can be a handy feature when you need to send messages to several respective end-points due to some event, but you wouldn't want some end-points to get a message and not so others in the case a failure occurs at the time of message publishing.
>
> Thanks for listening.
I'm sorry for the verbosity of the responses to your questions - but the richness of the JMS messaging system lends a fertile realm of possibility for how to do things. It seems a bit of a disservice to give just a few, brief, off-the-cuff answers that would paint an overly narrow picture.
Re: JMS Newbie
There are a lot of ways to coordinate the return path. Roger goes into a lot of them in his reply. ;o) I've used both a preconfigured response path, and temporary destinations.If your consuming on the client, Spring has some decent utility classes (be sure to use connection pools), that most implementations should be able to plug into (example: http://activemq.org/Spring+Support). I believe they're working on publishing template code for Spring 2.0, as well.
Re: JMS Newbie
Thank you SO much for the detailed response.My work is considering moving away from a browser-centric JEE architecture and using rich clients. The asynchronous schemes you describe seem much more robust and flexible than RMI/IIOP.
Again, thank you.
Re: JMS Newbie
Thank you.Yours and Roger's responses have been most helpful.
vignette #1: JMSType and client end-point message filtering
(I'm going to try and pop in here on a regular basis and write up what I'll call a "vignette" of some JMS related subject or technique.)Standard JMS Property - JMSType
Today we'll look at the standard JMS message property, JMSType, and its application in JMS client end-point message filtering.
A JMS message has several standard properties. Some will always be set by the messaging system while others are for optional use. One of the optional-use properties is JMSType, which can be set to some arbitrary string value.
Most IT organizations that engage in JMS messaging will likely want to establish a convention for setting JMSType on all published messages. It's a good policy that every distinct kind of message should have a JMSType that is unique to it.
(Perhaps if the payload of the message is going to be processed by a tool such as JAXB, and result in a Java class object when de-serialized, then using the fully qualified package name of the Java class might be a reasonable convention.)
The fundamental idea here is that a client end-point that is consuming the message will be able to look at the JMSType property to decide how to process it. Also, the client might establish a message filter for its JMS session that will select for messages out of a topic that correspond to a specific JMSType.
I will illustrate with an example:
I have a message where JMSType is set as "Tideworks.TrafficCheckpoint.VehicleEvent". My client application selects for this message with this expression:
JMSType='Tideworks.TrafficCheckpoint.VehicleEvent'
If this is the only kind of message that my client end-point consumes, then it could just assume that any message it receives (or that its message listener callback is invoked for) is of that type and proceed to process accordingly.
If, however, the client consumes several kinds of messages that may be delivered on a single JMS session, then the client might use internal programmatic logic that examines the JMSType property of a received message to then dispatch it to the appropriate processing action. The filter expression for the session might look like so:
JMSType LIKE 'Tideworks.TrafficCheckpoint.VehicleEvent%'
My client will now receive VehicleEvent messages as well as its sub types. Let us say the sub types are the following:
"Tideworks.TrafficCheckpoint.VehicleEvent.OCR"
"Tideworks.TrafficCheckpoint.VehicleEvent.RFID"
Now it turns out that the OCR variant is actually the same message format as VehicleEvent but is being published by a different mechanism (which will be an important distinction in itself). The RFID message is the same as VehicleEvent format but with some additional fields of information.
My client application has been written to know how to process all three types of VehicleEvent message - but only in certain combinations (or else its state machine logic will get muddled). What complicates the picture is that at different customer sites the client application may need to utilize a specific VehicleEvent or else a specific pairing of VehicleEvent. Fortunately we can control that externally in the deployment of the client application by placing an appropriate message filter expression into its configuration file.
So at one customer site, just the normal VehicleEvent message (which is being generated by something called a bobtail detector) should be processed, i.e.:
JMSType = 'Tideworks.TrafficCheckpoint.VehicleEvent'
Even though this site is also producing the OCR variant, we don't want the client application to see it and the above filter enforces that restriction.
Another customer site, however, is instead using the OCR originated version of this event:
JMSType = 'Tideworks.TrafficCheckpoint.VehicleEvent.OCR'
And yet another customer site will be configured to use the OCR and RFID variants in tandem. The client processes the first one of this pair that it receives and ignores the latter. However, if one hardware system (OCR vs RFID) goes down, the other can be used as backup with respect to providing this event to the client application and thus insure continuity of operation. Thus this variation of the filter expression insures that both these message types can be seen by the client application:
JMSType LIKE 'Tideworks.TrafficCheckpoint.VehicleEvent.%'
Standard JMS Property - JMSExpiration
So what would happen to event messages that are perhaps being published but are not actually being consumed by any client end-point? For instance, in our first example customer site above, the OCR variant of VehicleEvent is being published but the filter expression omits it:
JMSType = 'Tideworks.TrafficCheckpoint.VehicleEvent'
This filter expression insures that the client sees and processes the standard bobtail detector variant of VehicleEvent but the OCR variant now remains in the JMS topic unconsumed by any client end-point.
Well another useful policy - in addition to the policy of always setting JMSType - is to always explicitly set the expiration or time-to-live of published messages. Consequently all VehicleEvent message publishers could be setting the message expiration to, say, 15 seconds. (By JMS convention this value is actually expressed in milliseconds so 15000 would be the actual number used.) When a message remains in a queue or topic and its time-to-live expires, the JMS provider will expunge the message. (In soft real-time messaging, this is usually what one wants to happen as an event that becomes too historically old is no longer relevant. A financial distributed application will likely take the opposite point of view, though, and tend to use message persistence coupled with infinite time-to-live.)
The time-to-live can be set via various JMS APIs when messages are published. Internally in the JMS provider (the JMS server implementation), the current GMT of the server and the expiration are added to come up with a time for the message to expire by. A consumer of messages can see what the expiration was set to by looking at the standard JMS property, JMSExpiration (not that it's particularly useful to do so - other than diagnostic when trace dumping messages).
In some JMS implementations, such as Tibco EMS that I use, it is possible to specify an expiration policy on a queue or topic in server configuration. Doing such can be useful if, say, an external vendor is publishing messages but not bothering to set a useful time-to-live (JMS will default to infinite time-to-live if none is set explicitly - which often times in practice message immortality is not desired).
Well, that concludes this vignette which has touched on JMSType, filter expressions, and JMSExpiration.
Re: vignette #1: JMSType and client end-point message filtering
How would you conflate the combination of JMSTypes and actual MessageQueues to something like methods within a SessionBean?Simply, since RPC follows most procedural or even OO based design paradigms, dividing up methods in to blocks of functionality (e.g. Session Beans) is a reasonably "natural" and regular thing during the design on an application.
We simply refactor organizationally based on the model.
How do you approach this using queues?
If you were using MDBs, would you have a single one per queue and then have it dispatch to the actual functionality based on JMSType? Would you have multiple MDBs each filtering on a different JMSType?
What rules of thumb do you use? What are the trade offs?
For example, on one extreme, why not have a single Message Queue, and then simply have all of the clients filtering on some criteria (like JMSType) vs the other extreme of having one Message Queue for each JMSType? How do you populate the middle groud?
Re: vignette #1: JMSType and client end-point message filtering
[Will, the bulk of my response is aimed at new-comers to this subject matter - to you a lot of my comments will be repetitious in view of what I've said already elsewhere.]> How would you conflate the combination of JMSTypes
> and actual MessageQueues to something like methods
> within a SessionBean?
From '94 to '99 I worked at Microsoft where I first developed my own solution for remoting a COM interface over RPC and then later started using DCOM which was Microsoft's official solution at that time for remoting an object interface. After half a decade of developing several different distributed systems using remoted object interfaces I came to the conclusion that it is an anti-pattern to devise distributed software in this manner. I now prefer to exclusively use asynchronous message passing and I completely abstain from using the various forms of RPC. Given that the network is neither reliable or deterministic in performance it makes more sense to me to operate asynchronously in programming the interactions of distributed nodes.
I have not found it particularly helpful to try and model interactions between such nodes in an OOP fashion and instead prefer data-transfer-objects and an event notification idiom. Messaging is a means to build event driven software that spans the network. The applications I build all have soft real-time aspects to their operation due to real world events that feed into them. JMS messaging has worked out quite well for this.
As in a Swing app, you want to take a generated event and just pop it into a queue and then let the various interested parties that care about the event receive it and proceed to respond to it. That is publisher/subscriber event notification and is used heavily in my distributed applications.
Also, I want my end-points to be loosely coupled. The intermediary of a queue or topic, around which interested parties rendezvous for interaction, works out well in this regard. Doing something like RMI into a session bean method is too tightly coupled. A lot of the cool things I do in organizing, recombining components, message transformation, and redirecting message flows would be exceedingly tedious to replicate with a tightly coupled RPC approach. I have found that in practice the numerous advantages of loose coupling flexibility vastly outweigh any presumed conceptual advantage of a remoted object interface.
Queuing means that the message producer(s) and the message consumer(s) do not have to work in exact lock step (a queue is a buffering mechanism - and also a serialization mechanism, of course). This leads to overall greater efficiency and throughput. Synchronous RPC is intrinsically flawed by its very nature as tightly coupled end-points must work in rigid lock step.
Also, RPC code tends to be overly encombered with error handling at the point of use. JMS makes it more feasible to keep error handling out of band
>
> Simply, since RPC follows most procedural or even OO
> based design paradigms, dividing up methods in to
> blocks of functionality (e.g. Session Beans) is a
> reasonably "natural" and regular thing during the
> design on an application.
>
> We simply refactor organizationally based on the
> model.
>
> How do you approach this using queues?
I drop in new MDBs or new handlers on the client side. Or I write a message transformation bean that adapts some new message (that, say, is being published by some external vender) into one that is currently recognized. A lot of times writing a simple adapter can be a way to avoid modifying the internals of some existing application that is currently working very robustly. (Why risk disturbing a stable system if it can be easily avoided?)
Now I do have MDBs that process multiple message types. So refactoring such an MDB might consist of merely adding new message handler code to it and suitably modify its message filter expression.
>
> If you were using MDBs, would you have a single one
> per queue and then have it dispatch to the actual
> functionality based on JMSType? Would you have
> multiple MDBs each filtering on a different JMSType?
>
> What rules of thumb do you use? What are the trade
> offs?
I have some of each.
Sometimes a given MDB needs to respond to several different semantic messages that are related. For instance I have a config file repository where the MDB responds to message verbs for manipulating the storage of files - putting, getting, listing, deleting.
>
> For example, on one extreme, why not have a single
> Message Queue, and then simply have all of the
> clients filtering on some criteria (like JMSType) vs
> the other extreme of having one Message Queue for
> each JMSType? How do you populate the middle groud?
My basic convention (but not strictly adhered to) is:
A given distributed application will tend to use one queue (MDBs in a JEE cluster do round-robin load balanced processing off of the queue). The client end-points will publish to the queue in order to post messages to the middle-tier.
And then there will be one topic. Server-side push notification and result messages are published to the topic. (Sometimes a private temp queue may be used for reply messages.)
Bridge rules in the server can be used to further stitch multiple applications together.
Refer to this link where I give more in-depth discourse on this:
http://www.javalobby.org/java/forums/m91991515.html
I also would recommend reading my postings to this thread:
Why are Java developers ignorant of JMS and messaging in general?