There’s a a fascinating chapter of Java Programming with Oracle JDBC available online. What I find most interesting is that (as of 2001), the thin driver is good enough to use everywhere–in fact, in most cases outlined above, it outperforms the OCI (type 2) driver. In addition, typically it takes 10s of repeated calls to a given PreparedStatement to make using the PreparedStatement faster than a regular Statement.
Expresso authentication and authorization
I’ve only briefly worked with Expresso. But I’ve heard good things about it. However, one ‘feature’ is really chapping my hide at the moment. Apparently, the only way to authenticate someone is to call the attemptLogin method on a ‘Controller’ object (a subclass of a Struts Action), which is protected and takes, among other things, the http request and response. There’s no way I can find to just pass in a username/password and authenticate. In addition, the authorization system is not broken out either. In OO fashion, you ask an object if a user can access it, and the object knows enough to reply.
I’m not trying to rag on the Expresso developers. After all, they are giving away a fine, full featured java web framework for free. But this just drove home to me how important it is in web applications to have the classes that talk http be nothing more than a thin translating layer around business classes. For instance, all a struts action should do is convert http forms to domain specific value objects, and then call business methods on business objects.
If this was the case in Expresso, it’d be trivial for me to leverage Expresso’s existing authentication model–I’d just have to fall the methods on the business object, perhaps after creating a domain specific value object. Now, however, I’ll probably have to monkey around with the http request and response, and decode exactly what parameters it wants, and fake those up.
Open source portal search
I’ve been looking at some open source portals. My client has an existing java application, written in Expresso that has some reasonably complex logic embedded in it. Additionally, it’s massively internationalized, with dynamic international content coming from a database, and static content coming from a set of resource bundles. There’s an existing process around updating both of these sets of data. And when we’re talking internationalization, we’re talking Asian character sets as well as the European character sets.
So, the criteria for the portal were:
1. Support for multi-byte character sets and easy localization.
2. Ability to integrate with Expresso’s authentication and authorization systems.
3. Support for normal portal features–adding/moving/removing portlets, minimize/maximize portlets.
4. Documentation.
I looked at a fair number of portals, including jcorporate’s own ePortal, eXo, Liferay, Jetspeed 1, Jetspeed 2, and Pluto (a last alternative, to be avoided if possible, is to roll our own portal-like application). First, I looked at ePortal, but that’s a dead project, with no releases. Then, I installed pluto, which seemed like it would be a perfect fit to be integrated into Expresso. However, integrating pluto looked complex, and after installing it (fantastic instructions for installing pluto here), I realized that pluto did not have a layout manager that would allow for the addition, rearranging or moving of portlets.
I then battled with Jetspeed 2, which involved installing a subversion client and building from source. This looked to be pretty cool, but the sheer lack of documentation, and the fact that there have been no releases, caused me to shy off. This is no failure of Jetspeed 2–this is what projects in development are like; I think it will be a fine project when done but my client just doesn’t need to be on the bleeding edge. I also took a quick look at Liferay, which seems to be a much more full featured portal application than we needed. After reading this blog on portals I decided to take a closer look at eXo. However, the documentation wasn’t fantastic, and it wasn’t readily apparent how to plug in authentication.
I also downloaded and installed Jetspeed 1; if you download the src distribution, you get the helpful tutorial. While Jetspeed 1 is not a standards based solution (I expect most of the portlets will be custom developed anyway), the user community is fairly active, as indicated by the mailing list, and I’ve found the documentation to be extensive. In addition, it meets the localization requirements and the pluggable authentication and authorization systems.
I’m less than thrilled about having to use maven for builds. Others have said it better than I, but it’s just too much for my needs. However, I was able to get an independent directory tree for my project by copying over the maven.xml, project.properties, and project.xml from the tutorial directory to an empty directory. Then I tweaked the project.* files, ran maven jetspeed:genapp, tweaked a few settings in TubineResources.properties to make sure the localization settings were correct, and, voila, I have a working project tree, that, using the Jetspeed maven plugin, is one command away from a deployable war file.
OJB and object caching, pt II
Well, I was wrong, when I posted that OJB rc4 didn’t support caching. Because of the way the application is architected, there are two places where we grab data from the database. I know, I know, don’t repeat yourself. But when you’re using free JAAS modules and free O/R mapping tools, you can’t be too picky.
The upshot is, when I actually look at SQL statements for a typical two user session, I see 21 of a certain select statement for when caching using the org.apache.ojb.broker.cache.ObjectCacheEmptyImpl class, and only 6 when performing exactly the same user actions with the org.apache.ojb.broker.cache.ObjectCacheDefaultImpl class. Don’t ask me why it’s not a 2 to 1 ratio; I’m looking into it. (Deep are the ways of object caching.)
Book Review: Java Transaction Processing
Since many financial institutions have standardized on it, I hear Java is the new COBOL. Whether or not this is true, if Java is to become the business language of choice, transaction support is crucial. (By ‘transaction,’ I mean ‘allowing two or more decisions to me made under ACID constraints: atomically, consistently, (as) in isolation and durably’.) Over the last five years, the Java platform has grown by leaps and bounds, not least in this area.
Java Transaction Processing by Mark Little, Jon Maron and Greg Pavlik, explores transactions and their relationship with the Java language and libraries. Starting with basic concepts of transactions, both local and distributed, including the roles of participant and coordinator, and the idea of transaction context, the book covers much old but useful ground. Then, by covering the Java Transaction API (JTA) as well as OTS, the OMG’s transaction API which is JTA’s foundation, this book provides a solid understanding of the complexities of transactions for Java programmers who haven’t dealt with anything more complex than a single RDBMS. I’d say these complexities could be summed up simply: failures happen; how can you deal with them reliably and quickly?
The book then goes on to examine transactions and the part they play in major J2EE APIs: Java Database Connectivity (JDBC), Java Message Service (JMS), Enterprise Java Beans (EJB) and J2EE Connector Architecture (JCA). These chapters were interesting overviews of these technologies, and would be sufficient to begin programming in them. However, they are complex, and a single chapter certainly can’t do justice to any of the APIs. If you’re new to them, expect to buy another book.
In the last section, the authors discuss the future of transactions, especially long running activities (the Java Activity Service) and web services. This was the most interesting section to me, but also is the most likely to age poorly. These technologies are all still under development; the basic concepts, however, seem likely to remain useful for some time. And, if you need to decide on a web service transaction API yesterday, don’t build your own, read chapter 10.
There were some things I didn’t like about Java Transaction Processing. Some of the editing was sloppyperiods or words missing. This wasn’t too big a problem for me, since the publisher provided me a free copy for review, but if I were paying list price ($50) I’d be a bit miffed. A larger annoyance was incorrect UML and Java code snippets. Again, the meaning can be figured out from the text, but it’s a bit frustrating. Finally, while the authors raise some very valid points about trusting, or not, the transaction system software provider, I felt the constant trumpeting of HP and Arjuna technologies was a bit tedious. Perhaps these companies are on the forefront of Java transactions (possible); perhaps the authors are most familiar with the products of these companies (looking at the biographies, this is likely). The warningsfind out who is writing the transaction software, which is probably at the heart of your business, and how often they’ve written such software beforewere useful, if a bit repetitive.
That said, this book was still a good read, if a bit long (~360 pages). I think that Java Transaction Processing would be especially useful for an enterprise architect looking to leverage existing (expensive) transactional systems with more modern technology, and trying to see how Java and its myriad APIs fit into the mix. (This is what I imagine, because I’m not an enterprise architect.) I also think this book would be useful to DBAs; knowing about the Java APIs and how they deal with transactions would definitely help a DBA discuss software issues with a typical Java developer.
To me, an average Java developer, the first section of the book was the most useful. While transactions are fairly simple to explain (consider the canonical bank account example), this section illuminated complexities I’d not even thought ofoptimizations, heuristic outcomes, failure recovery. These issues occur even in fairly simple setupsI’m working at a client who wants to update two databases with different views of the same information, but make sure that both are updated or neither; this seems to be a typical distributed transaction. The easiest way to deal with this is to pretend that such updates will always be successful, and then accept small discrepancies. That’s fine with click-throughsmoney is a different matter.
However, if you are a typical web developer, I’m not sure this book is worth the price. I would borrow it from your company’s enterprise architect, as reading it will make you a better programmer (as well as giving you a sense of historytransactions have been around for a long time). But, after digesting fundamental distributed transaction concepts, I won’t be referencing this book anytime soon, since the scenarios simply don’t happen that often (and when they do, they’re often ignored, as outlined above).
Decreasing the size of a midlet jar
The J2ME application I have been working on has been ready for testing for quite some time, but I didn’t want to get a new AT&T phone. For J2ME, you really need a GSM phone–I don’t think any of the older TDMA models support it. But the GSM network coverage doesn’t match the coverage of the TDMA network–especially out west (aside: isn’t that magnifying glass pretty cool?). So I put off buying a phone until my summer road tripping was done.
I’ve had a Nokia 6160 for almost 4 years. Even though friends mocked the size of it, it was a great phone–durable, good talk time. I thought I’d try another Nokia, and got one of the lower end GSM phones, the 6200. This supported J2ME, and weighed maybe half as much. I was all stoked to try the application on my brand new phone.
I started download the jad file, and was getting ‘File Too Large’ errors. A couple of searches later, I found Nokia’s developer device matrix which is much more useful than the User Guide or the customer facing description of phones. Whoops. Most of the Series 40 (read: affordable) Nokia devices only supported J2ME applications which were, when jarred up, less than 64K in size.
Our application, however, was about 78K. This highlights one of the differences between J2ME and J2SE/J2EE. When coding in the latter world, I was never concerned about code size–getting the job done quickly was paramount, and if I needed to use 13 libraries which bloated the final size of my application, I did. On a cell phone, however, there’s no appeal to adding memory or changing the JVM configuration to optimize memory use. If the Nokia phone only accepts jars of 64K or less, I had three options:
1. Write off the Nokia Series 40 platform. Ugh–I like Nokias, and other folks do too.
2. Do some kind of magic URL classloading. This seemed complicated and I wasn’t sure how to do it.
3. Decrease the size of the jar file.
Now, the 78K jar had already been run through an obfuscator. I wasn’t going to get any quick and easy gains from automated software. I posted a question on the JavaRanch J2ME forum and received some useful replies. Here’s the sequence I went through:
1. Original size of the application: 79884 bytes.
2. Removal of extra, unused classes: 79881. You can see that the obfuscator did a good job of winnowing out unused classes without my help.
3. Changed all the data objects (5-6 classes), which had been written in classic J2SE style with getters and setters for their properties, to have public variables instead: 79465
4. Combined 3 of the data object classes into one generic class: 78868
5. Combined 5 networking classes into 2: 74543
6. Removed all the logging statements: 66044. (Perl to the rescue–$ perl -p -i -e 's!Log\.!//Log.!' `find . -name "*.java" -print |xargs grep -l 'Log\.'`)
7. Next, I played around with the jode obfuscator which Michael Yuan recommended. I was able to radically decrease the size of the jar file, but, unfortunately, that jar file didn’t work on the phone. I also got a ton of exceptions:
java.util.NoSuchElementException
at jode.bytecode.BytecodeInfo$1.next(BytecodeInfo.java:123)
at jode.obfuscator.modules.LocalOptimizer.calcLocalInfo(LocalOptimizer.java:370)
at jode.obfuscator.modules.LocalOptimizer.transformCode(LocalOptimizer.java:916)
at jode.obfuscator.MethodIdentifier.doTransformations(MethodIdentifier.java:175)
at jode.obfuscator.ClassIdentifier.doTransformations(ClassIdentifier.java:659)
at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:320)
at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
at jode.obfuscator.ClassBundle.doTransformations(ClassBundle.java:421)
at jode.obfuscator.ClassBundle.run(ClassBundle.java:526)
at jode.obfuscator.Main.main(Main.java:189)
I'm sure I just didn't use it right, but the jar file size was so close to the limit that I abandoned jode.
8. Instead, I put all the classes in one file (perl to the rescue, again) and compiled that: 64057 bytes. The jar now downloads and works on my Nokia 6200 phone.
When I have to do this again, I'll definitely focus on condensing classes, basically replacing polymorphism with if statements. After removing extraneous Strings and concatenating all your classes into one .java file (both of which are one time shots), condensing classes is the biggest bang for your buck.
JMS first impressions
I’m currently working on a project using JMS. It’s my first experience with messaging, though I’ve read a bit about it; back in the dot.com boom, you could get the O’Reilly JMS book just for giving up demographic data (thanks Sonic!). However, this project is using JBossMQ, sending TextMessages containing XML into a number of Queues. I’ve integrated message sending into existing client applications. These applications come under heavy load periodically, so we wanted to make producing messages as simple as possible. The XML documents that these applications create are simply well-formed, and typically small. The consumers of these messages, on the other hand, undertake some fairly slow activities: they do some data massaging, and update or insert into multiple databases. Therefore, the consumers process messages asynchronously.
Such behavior demonstrates the strength of messaging (outlined most eloquently here): because of the decoupling between the producer of the message and the consumer, some objects can do “stuff” at a different rate of speed than the other objects which are needed to finish handling the “stuff”. The downside, of course, is the difficulty of receiving any form of confirmation (unlike typical synchronous systems, where, since the calling object blocks until return, passing back values is simple). Enterprise messaging systems, for which JMS simply provides a uniform (and somewhat limited) API, provide some guarantees–at least the producer can rest assured that the message did get to some consumer. This contract means that the producer can throw many many messages into a queue, confident that even if it takes a long time to parse and handle each message, every one will be passed off to a consumer. Of course, if the rates of message production and consumption are vastly different, there can be problems.
(When a component of an typical, synchronous system fails, then the caller is left to handle the wreckage. Since messaging systems interpose, when a consumer fails, the producer never finds out or has to deal with it.)
It seems like messaging systems would be great for integrating disparate systems as well–after all, message formats can be entirely arbitrary and don’t have to be understood by the messaging server at all. For example, this project has a perl program that was generating quite a bit of interesting data; it would have been nice to put this into a queue for a java program to consume. Unfortunately, the options for having a non-java producer participate in a JMS system are limited:
1. Some implementations provide client APIs for other languages (I saw a posting about SonicMQ and C++). JBossMQ has none that I could find.
2. Perl can call methods on java objects. A bit scary for a production system, and not a solution for producers/consumers written in other languages.
3. You could set up a java service listening on a port that would just take what’s given and send a JMS message containing that to a queue. Now you lose much of the robustness of a messaging solution, since you’re dependent on this service to make sure your messages get through.
4. Cut out the java service above, and decode the format that JBossMQ is using–since it’s listening on a port, and you have access to the JBossMQ source you could probably hack up a client to send a message directly. This would be a maintenance hassle and isn’t portable between JMS implementations.
The perl client problem ended up going away because we used a scalable, asynchronous message delivery system–Sendmail. (I wonder whether anyone has ever slapped JMS on top of Sendmail. A quick Google search showed nothing, but it seems like a natural pairing. I’m a bit worried about the reliability of delivery, but I’ve a sysadmin friend who says that if you control both the beginning and endpoints of a mail system, you can guarantee delivery.) All in all, JMS seems like a clean standard manner in which to enforce separation of concerns and gain a fair amount of robustness.
struts module ClassCastException
If you get this exceptions like this:
2004-07-21 15:35:06 action: null
java.lang.ClassCastException
at org.apache.struts.action.ActionServlet.initModulePlugIns(ActionServle
t.java:1142)
at org.apache.struts.action.ActionServlet.init(ActionServlet.java:486)
at javax.servlet.GenericServlet.init(GenericServlet.java:256)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:918)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:810)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3279)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:3421)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1123)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:638)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1123)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:343
)
at org.apache.catalina.core.StandardService.start(StandardService.java:388)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:506)
at org.apache.catalina.startup.Catalina.start(Catalina.java:781)
at org.apache.catalina.startup.Catalina.execute(Catalina.java:681)
at org.apache.catalina.startup.Catalina.process(Catalina.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:243)
2004-07-21 15:35:06 StandardWrapper[/yourmodule:action]: Marking servlet action as unavailable
2004-07-21 15:35:06 StandardContext[/yourmodule]: Servlet /yourmodule threw load() exception
and you’re using struts with modules, make sure that all of the classes referenced in all of the module-level struts-config.xml files are in the classpath of Tomcat.
OJB and object caching
I’m working on a project with ObJectRelationalBridge 1.0RC4. This release is a year old, but has suited our needs up to now, but now I’ve a couple of gripes.
1. The caching online documentation doesn’t apply to my version. Hey, RC4 isn’t the latest and greatest, so that’s fair enough, but it would be nice if it was clear to which version the documentation applied. Once I realized (through a xerces exception) that this was the case, I was able to download the correct version and view that documentation locally. But what if I’d been using rc1, which doesn’t appear to be downloadable anymore? Perhaps I could get documentation from CVS, but this just shows that you really should keep virgin downloads of your external dependencies (code, documentation, whatever) for future reference. Would it be overkill to spider the software’s website when you make the decision to go with a particular version, and store that off somewhere?
2. In OJB 1.0RC4, object caching doesn’t seem to work (using the ObjectCacheDefaultImpl class). Not too much useful on the mailing list but I did a bit of sleuthing on my own. P6Spy is a slick java application that decorates any JDBC driver, writing all the statements that clients are making to a log file, then passing those statements on through to the driver. Installation on tomcat was very easy, and it was enlightening to see what OJB had been up to under the covers: going back to the database to recreate a user object, even though ObjectCacheDefaultImpl has no timeout for cached objects.
I’ll probably update the project to the newly released OJB 1.0.0 (don’t those numbers strike fear into your heart? I suppose after 7 release candidates over more than a year, 1.0.0 should be pretty solid) and see if object caching works.
More on the difficulty of tuning the JVM
Here’s a great overview of the JVM memory model (for all of Sun’s JVMs, including the latest changes). I find it intensely interesting that he brushes over what, for me, is the most complicated part of any web application tuning–testing. He outlines a process for initially sizing the various compartments of the JVM heap (for versions 1.4 and below), and then says: ‘The [resizing] process continues by “testing and tweaking” until things look good.’
Wow. Talk about waving your hands. I did some testing of a web application a few months ago, but when I presented the results, I was very clear that they were guidelines only. I didn’t have the resources or ingenuity to replicate the behavior of real users on real clients. A few years ago, I was part of a project that ran aground on this same rock, costing the company I was working for plenty of money. Using software to imitate user behavior is hard. A short list of the differences between software and users:
1. Users get distracted–by popups, their kids, etc. Software, not so much.
2. Users are connecting via a variety of methods, with a wide range of quality levels–modems, broadband.
3. Users don’t use applications in the way developers intended. The testing software, on the other hand, is programmed by developers, who naturally have it use the application in the way they intended
The adaptive memory model for the next JDK (wow, now it’s J2SE 5–Sun pulled another Solaris reversioning trick) that the author outlines might make the “tweaking” portion of his hand waving, err, I mean tuning, easier but leaves the “testing” as difficult as ever.
Follow me on Twitter
Follow me on Blue Sky