JMS first impressions

I’m currently working on a project using JMS. It’s my first experience with messaging, though I’ve read a bit about it; back in the boom, you could get the O’Reilly JMS book just for giving up demographic data (thanks Sonic!). However, this project is using JBossMQ, sending TextMessages containing XML into a number of Queues. I’ve integrated message sending into existing client applications. These applications come under heavy load periodically, so we wanted to make producing messages as simple as possible. The XML documents that these applications create are simply well-formed, and typically small. The consumers of these messages, on the other hand, undertake some fairly slow activities: they do some data massaging, and update or insert into multiple databases. Therefore, the consumers process messages asynchronously.

Such behavior demonstrates the strength of messaging (outlined most eloquently here): because of the decoupling between the producer of the message and the consumer, some objects can do “stuff” at a different rate of speed than the other objects which are needed to finish handling the “stuff”. The downside, of course, is the difficulty of receiving any form of confirmation (unlike typical synchronous systems, where, since the calling object blocks until return, passing back values is simple). Enterprise messaging systems, for which JMS simply provides a uniform (and somewhat limited) API, provide some guarantees–at least the producer can rest assured that the message did get to some consumer. This contract means that the producer can throw many many messages into a queue, confident that even if it takes a long time to parse and handle each message, every one will be passed off to a consumer. Of course, if the rates of message production and consumption are vastly different, there can be problems.

(When a component of an typical, synchronous system fails, then the caller is left to handle the wreckage. Since messaging systems interpose, when a consumer fails, the producer never finds out or has to deal with it.)

It seems like messaging systems would be great for integrating disparate systems as well–after all, message formats can be entirely arbitrary and don’t have to be understood by the messaging server at all. For example, this project has a perl program that was generating quite a bit of interesting data; it would have been nice to put this into a queue for a java program to consume. Unfortunately, the options for having a non-java producer participate in a JMS system are limited:

1. Some implementations provide client APIs for other languages (I saw a posting about SonicMQ and C++). JBossMQ has none that I could find.

2. Perl can call methods on java objects. A bit scary for a production system, and not a solution for producers/consumers written in other languages.

3. You could set up a java service listening on a port that would just take what’s given and send a JMS message containing that to a queue. Now you lose much of the robustness of a messaging solution, since you’re dependent on this service to make sure your messages get through.

4. Cut out the java service above, and decode the format that JBossMQ is using–since it’s listening on a port, and you have access to the JBossMQ source you could probably hack up a client to send a message directly. This would be a maintenance hassle and isn’t portable between JMS implementations.

The perl client problem ended up going away because we used a scalable, asynchronous message delivery system–Sendmail. (I wonder whether anyone has ever slapped JMS on top of Sendmail. A quick Google search showed nothing, but it seems like a natural pairing. I’m a bit worried about the reliability of delivery, but I’ve a sysadmin friend who says that if you control both the beginning and endpoints of a mail system, you can guarantee delivery.) All in all, JMS seems like a clean standard manner in which to enforce separation of concerns and gain a fair amount of robustness.

Internet Bookmobile

I have to say that the Internet Bookmobile is way cool. (I remember the bookmobile of my youth, and it didn’t carry anywhere near 20,000 books!) I think that the Internet Bookmobile shows two things:

1. The enduring power of the book. I’m definitely not the first person to say this, but the portability, durability, cost and readability of bound books is hard to beat with any electronic format. They’re going to be around for a long time, despite the efforts of e-book software purveyors.

2. Digital, public domain texts are a good thing. (But copyright keeps getting extended.)

(Thanks to Brian D Foy for the link.)

struts module ClassCastException

If you get this exceptions like this:

2004-07-21 15:35:06 action: null
        at org.apache.struts.action.ActionServlet.initModulePlugIns(ActionServle
        at org.apache.struts.action.ActionServlet.init(
        at javax.servlet.GenericServlet.init(
        at org.apache.catalina.core.StandardWrapper.loadServlet(
        at org.apache.catalina.core.StandardWrapper.load(
        at org.apache.catalina.core.StandardContext.loadOnStartup(
        at org.apache.catalina.core.StandardContext.start(
        at org.apache.catalina.core.ContainerBase.start(
        at org.apache.catalina.core.StandardHost.start(
        at org.apache.catalina.core.ContainerBase.start(
        at org.apache.catalina.core.StandardEngine.start(
        at org.apache.catalina.core.StandardService.start(
        at org.apache.catalina.core.StandardServer.start(
        at org.apache.catalina.startup.Catalina.start(
        at org.apache.catalina.startup.Catalina.execute(
        at org.apache.catalina.startup.Catalina.process(
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
        at java.lang.reflect.Method.invoke(
        at org.apache.catalina.startup.Bootstrap.main(

2004-07-21 15:35:06 StandardWrapper[/yourmodule:action]: Marking servlet action as unavailable
2004-07-21 15:35:06 StandardContext[/yourmodule]: Servlet /yourmodule threw load() exception

and you’re using struts with modules, make sure that all of the classes referenced in all of the module-level struts-config.xml files are in the classpath of Tomcat.

OJB and object caching

I’m working on a project with ObJectRelationalBridge 1.0RC4. This release is a year old, but has suited our needs up to now, but now I’ve a couple of gripes.

1. The caching online documentation doesn’t apply to my version. Hey, RC4 isn’t the latest and greatest, so that’s fair enough, but it would be nice if it was clear to which version the documentation applied. Once I realized (through a xerces exception) that this was the case, I was able to download the correct version and view that documentation locally. But what if I’d been using rc1, which doesn’t appear to be downloadable anymore? Perhaps I could get documentation from CVS, but this just shows that you really should keep virgin downloads of your external dependencies (code, documentation, whatever) for future reference. Would it be overkill to spider the software’s website when you make the decision to go with a particular version, and store that off somewhere?

2. In OJB 1.0RC4, object caching doesn’t seem to work (using the ObjectCacheDefaultImpl class). Not too much useful on the mailing list but I did a bit of sleuthing on my own. P6Spy is a slick java application that decorates any JDBC driver, writing all the statements that clients are making to a log file, then passing those statements on through to the driver. Installation on tomcat was very easy, and it was enlightening to see what OJB had been up to under the covers: going back to the database to recreate a user object, even though ObjectCacheDefaultImpl has no timeout for cached objects.

I’ll probably update the project to the newly released OJB 1.0.0 (don’t those numbers strike fear into your heart? I suppose after 7 release candidates over more than a year, 1.0.0 should be pretty solid) and see if object caching works.

Book Review: How to Lie with Statistics

How to Lie with Statistics, by Darrel Huff, should be required reading for everyone. The cachet of numbers are used all the time in modern society. Usually to end arguments–after all, who can argue with “facts”? Huff shows how the same set of numbers can be tweaked to show three different outcomes, depending on where you start and what you use. The fundamental lesson I learned from this book is that mathematical calculation involves a whole set of conditions, and any number derived from such a calculation is meaningless without understanding those conditions.

He also mentions that colleagues have told him that the flurry of meaningless statistics is due to incompetence–he dispatches this argument with a simple query: “Why, then, do the numbers almost always favor the person quoting them?” Huff also provides five questions (not unlike the five d’s of dodgeball) for readers to ask, when confronted with a statistic:

1. Who says so?

2. How does he know?

3. What’s missing?

4. Did somebody change the subject?

5. Does it make sense?

All this is wrapped up in a book with simple examples (no math beyond arithmetic, really) and quaint 1950s prose. In addition humor runs from the beginning (the dedication is “To my wife with good reason”) to the end (on page 135, Huff says “Almost anybody can claim to be first in something if he is not too particular what it is”). This book is well worth a couple hours of your time.

“How To Lie With Statistics” at Amazon.

More on the difficulty of tuning the JVM

Here’s a great overview of the JVM memory model (for all of Sun’s JVMs, including the latest changes). I find it intensely interesting that he brushes over what, for me, is the most complicated part of any web application tuning–testing. He outlines a process for initially sizing the various compartments of the JVM heap (for versions 1.4 and below), and then says: ‘The [resizing] process continues by “testing and tweaking” until things look good.’

Wow. Talk about waving your hands. I did some testing of a web application a few months ago, but when I presented the results, I was very clear that they were guidelines only. I didn’t have the resources or ingenuity to replicate the behavior of real users on real clients. A few years ago, I was part of a project that ran aground on this same rock, costing the company I was working for plenty of money. Using software to imitate user behavior is hard. A short list of the differences between software and users:

1. Users get distracted–by popups, their kids, etc. Software, not so much.
2. Users are connecting via a variety of methods, with a wide range of quality levels–modems, broadband.
3. Users don’t use applications in the way developers intended. The testing software, on the other hand, is programmed by developers, who naturally have it use the application in the way they intended

The adaptive memory model for the next JDK (wow, now it’s J2SE 5–Sun pulled another Solaris reversioning trick) that the author outlines might make the “tweaking” portion of his hand waving, err, I mean tuning, easier but leaves the “testing” as difficult as ever.

An open letter to Climbing magazine

Here’s a letter to Climbing magazine. I’m posting it here because I think that the lessons Climbing is learning, especially regarding the Internet, are relevant to every print magazine.

I just wanted to address some of the issues raised in the Climbing July 2004 Editorial, where you mention that you’ve cut back on advertising as well as touching on the threat to Climbing from website forums. First off, I wanted to congratulate you on adding more content. If you’re in the business of delivering readers to advertisers you want to make sure that the readers are there. It doesn’t matter how pretty the ads are–Climbing is read for the content. I’m sure it’s a delicate balance between (expensive) content that readers love and (paid) advertisements which readers don’t love; I wish you the best in finding that balance.

I also wanted to address forums, and the Internet in general. I believe that websites and email lists are fantastic resources for finding beta, discussing local issues, and distributing breaking news. Perhaps climbing magazines fulfilled that need years ago, but the cost efficiencies of the Internet, especially when amateurs provide free content, can be hard to beat. But, guess what? I don’t read Climbing for beta, local issues, or breaking news. I read Climbing for the deliberate, beautiful articles and images. This level of reporting, in-depth and up-close, is difficult to find on the web. Climbing should continue to play to the strengths of a printed magazine–quality, thoughtful, deliberate articles and images; don’t ignore breaking news, but realize that’s not the primary reason subscribers read it. I don’t see how any magazine can compete with the interactivity of the Internet, so if Climbing wants to foster community, perhaps it should run a mailing list, or monitor rec.climbing (and perhaps print some of the choice comments). I see you do run a message board on–there doesn’t look to be much activity–perhaps you should promote it in the magazine?

Now for some concrete suggestions for improvement. One of my favorite sections in Climbing is ‘Tech Tips.’ I’ve noticed this section on the website–that’s great. But, since this information is timeless, and I’ve only been a subscriber for 3 years, I was wondering if you could reprint older Tech Tips, to add cheap, useful content to Climbing. Also, I understand the heavy emphasis on the modern top climbers–these are folks that have interesting, compelling stories to tell, which are interesting around the world. Still, it’d be nice to see ‘normal’ climbers profiled as well, since most of us will never make a living climbing nor establish 5.15 routes, but all climbers have stories to share. And a final suggestion: target content based on who reads your magazine. Don’t use just a web survey, as that will be heavily tilted in favor of the folks who visit your website (sometimes no data is better than skewed data). Instead find out what kind of climbers read your magazine in a number of ways: a web survey, a small survey on subscription cards, paper surveys at events where Climbing has presence, etc. This demographic data will let you know if you should focus on the latest sick highball problem, the latest sick gritstone headpoint or the latest sick alpine ascent.

Finally, thanks for printing a magazine worth caring about.

© Moore Consulting, 2003-2010 +