Right tool, right job, part II

Here’s a nice explication of ‘choose the right tool for the job,’ by a pundit. The entire J2EE stack is often overkill, and I’ve seen shops that would have been much better served by a series of PHP applications, perhaps with java on the backend for heavier duty processing and legacy code reuse.

My only quarrel with his analysis is that he assumes we know how long an application is going to live. How many applications that started as prototypes get rolled to production and are the cause of endless headaches five years down the road?

As in any complex situation, when designing an application, and choosing how complicated to make its architecture, you need to balance between two principles:

1. It’s easier to do things right the first time. You’re in the code, you have an understanding of the requirements, there’s a green field, you have a budget–therefore making sure the application works is easier when you’re first writing it.

2. You aren’t gonna need it. This slogan, from the Agile folks, says that you have no way of knowing what you’re going to need five years in the future, so why try to design for that?

Taken too far, each principle is disastrous. If you follow principle number one fully, you’ll end up in analysis paralysis. If you take point two to heart, you’ll miss out on big chunks of functionality that have to be retro-fitted on later, to the detriment of a schedule and possibly the design.

That’s why they (architects, not pundits) get paid the big bucks.


How to evaluate an open source project

There are a fantastic number of open source projects out there, on SourceForge, apache, and elsewhere. This is fantastic because you can often leverage other work that folks have done, as well as knowledge and mistakes they’ve made. However, it can be extremely difficult to evaluate accurately how much such projects can help you, especially if you’ve not used them before, or if they are obscure. In addition, you probably don’t have a lot of time to choose a solution–clients that go with open source solutions tend to have budget constraints. I present here a quick checklist that I use to evaluate projects that I’m going to depend upon:

1. Know your drop dead features. These are features that the software package must have in order to be considered. Be careful not to make this too long. The primary purpose of this list is to allow you to quickly exclude packages that you know won’t work for you, and if you make it too long, you might be left with no options.

2. Look at the documentation attached to the project. This is the first place to start ruling a project out–if it doesn’t promise the features you need, move on. Also, look at a demo or screen shots, if possible. This lets you see how the package works. Compare behavior with the list of needed features.

3. Install the software. If you have difficulty installing it, that’s not an insurmountable issue–often open source projects aren’t the smoothest installations. However, installing it and spending a few hours playing around with this software that is going to be a significant part of your project can let you know if the impressions you received from the demo and documentation are correct–is it going to be easy enough to tweak/deploy/extend this software package?

4. In the world of open source support, the mailing list is king. Does this project have a mailing list? Is it archived? Is it googled? How active is it? If there’s no mailing list, is there a set of forums? The mailing list (or forum) is where you’re going to go when you have a smart question to ask, and you will, so you want this resource to be as strong as possible.

5. Look at the documentation again. The first time you did so, you were just looking to exclude a project based on feature set. This time, you want to see how much the documentation can help you. Is there a tutorial? Are the advanced topics that concern you covered? For java projects, is there javadoc? Is it more than just the methods and arguments that are automatically generated? What version of the software is the documentation for?

Of course, the more I depend on a piece of software, the more time I want to spend on evaluation. On the other hand, the process laid out above is entirely technical in nature, and, as we know, there may be other reasons for choosing, or not choosing, software. Installed base, existing team experience and knowledge, project timeline, or the fact that the CEO’s brother is on the board of a company with a rival offering all can influence software package choice. There are many factors to be considered, but this list is how I start off.


Expresso authentication and authorization

I’ve only briefly worked with Expresso. But I’ve heard good things about it. However, one ‘feature’ is really chapping my hide at the moment. Apparently, the only way to authenticate someone is to call the attemptLogin method on a ‘Controller’ object (a subclass of a Struts Action), which is protected and takes, among other things, the http request and response. There’s no way I can find to just pass in a username/password and authenticate. In addition, the authorization system is not broken out either. In OO fashion, you ask an object if a user can access it, and the object knows enough to reply.

I’m not trying to rag on the Expresso developers. After all, they are giving away a fine, full featured java web framework for free. But this just drove home to me how important it is in web applications to have the classes that talk http be nothing more than a thin translating layer around business classes. For instance, all a struts action should do is convert http forms to domain specific value objects, and then call business methods on business objects.

If this was the case in Expresso, it’d be trivial for me to leverage Expresso’s existing authentication model–I’d just have to fall the methods on the business object, perhaps after creating a domain specific value object. Now, however, I’ll probably have to monkey around with the http request and response, and decode exactly what parameters it wants, and fake those up.


Open source portal search

I’ve been looking at some open source portals. My client has an existing java application, written in Expresso that has some reasonably complex logic embedded in it. Additionally, it’s massively internationalized, with dynamic international content coming from a database, and static content coming from a set of resource bundles. There’s an existing process around updating both of these sets of data. And when we’re talking internationalization, we’re talking Asian character sets as well as the European character sets.

So, the criteria for the portal were:

1. Support for multi-byte character sets and easy localization.

2. Ability to integrate with Expresso’s authentication and authorization systems.

3. Support for normal portal features–adding/moving/removing portlets, minimize/maximize portlets.

4. Documentation.

I looked at a fair number of portals, including jcorporate’s own ePortal, eXo, Liferay, Jetspeed 1, Jetspeed 2, and Pluto (a last alternative, to be avoided if possible, is to roll our own portal-like application). First, I looked at ePortal, but that’s a dead project, with no releases. Then, I installed pluto, which seemed like it would be a perfect fit to be integrated into Expresso. However, integrating pluto looked complex, and after installing it (fantastic instructions for installing pluto here), I realized that pluto did not have a layout manager that would allow for the addition, rearranging or moving of portlets.

I then battled with Jetspeed 2, which involved installing a subversion client and building from source. This looked to be pretty cool, but the sheer lack of documentation, and the fact that there have been no releases, caused me to shy off. This is no failure of Jetspeed 2–this is what projects in development are like; I think it will be a fine project when done but my client just doesn’t need to be on the bleeding edge. I also took a quick look at Liferay, which seems to be a much more full featured portal application than we needed. After reading this blog on portals I decided to take a closer look at eXo. However, the documentation wasn’t fantastic, and it wasn’t readily apparent how to plug in authentication.

I also downloaded and installed Jetspeed 1; if you download the src distribution, you get the helpful tutorial. While Jetspeed 1 is not a standards based solution (I expect most of the portlets will be custom developed anyway), the user community is fairly active, as indicated by the mailing list, and I’ve found the documentation to be extensive. In addition, it meets the localization requirements and the pluggable authentication and authorization systems.

I’m less than thrilled about having to use maven for builds. Others have said it better than I, but it’s just too much for my needs. However, I was able to get an independent directory tree for my project by copying over the maven.xml, project.properties, and project.xml from the tutorial directory to an empty directory. Then I tweaked the project.* files, ran maven jetspeed:genapp, tweaked a few settings in TubineResources.properties to make sure the localization settings were correct, and, voila, I have a working project tree, that, using the Jetspeed maven plugin, is one command away from a deployable war file.


Book Review: Java Transaction Processing

Since many financial institutions have standardized on it, I hear Java is the new COBOL. Whether or not this is true, if Java is to become the business language of choice, transaction support is crucial. (By ‘transaction,’ I mean ‘allowing two or more decisions to me made under ACID constraints: atomically, consistently, (as) in isolation and durably’.) Over the last five years, the Java platform has grown by leaps and bounds, not least in this area.

Java Transaction Processing by Mark Little, Jon Maron and Greg Pavlik, explores transactions and their relationship with the Java language and libraries. Starting with basic concepts of transactions, both local and distributed, including the roles of participant and coordinator, and the idea of transaction context, the book covers much old but useful ground. Then, by covering the Java Transaction API (JTA) as well as OTS, the OMG’s transaction API which is JTA’s foundation, this book provides a solid understanding of the complexities of transactions for Java programmers who haven’t dealt with anything more complex than a single RDBMS. I’d say these complexities could be summed up simply: failures happen; how can you deal with them reliably and quickly?

The book then goes on to examine transactions and the part they play in major J2EE APIs: Java Database Connectivity (JDBC), Java Message Service (JMS), Enterprise Java Beans (EJB) and J2EE Connector Architecture (JCA). These chapters were interesting overviews of these technologies, and would be sufficient to begin programming in them. However, they are complex, and a single chapter certainly can’t do justice to any of the APIs. If you’re new to them, expect to buy another book.

In the last section, the authors discuss the future of transactions, especially long running activities (the Java Activity Service) and web services. This was the most interesting section to me, but also is the most likely to age poorly. These technologies are all still under development; the basic concepts, however, seem likely to remain useful for some time. And, if you need to decide on a web service transaction API yesterday, don’t build your own, read chapter 10.

There were some things I didn’t like about Java Transaction Processing. Some of the editing was sloppy—periods or words missing. This wasn’t too big a problem for me, since the publisher provided me a free copy for review, but if I were paying list price ($50) I’d be a bit miffed. A larger annoyance was incorrect UML and Java code snippets. Again, the meaning can be figured out from the text, but it’s a bit frustrating. Finally, while the authors raise some very valid points about trusting, or not, the transaction system software provider, I felt the constant trumpeting of HP and Arjuna technologies was a bit tedious. Perhaps these companies are on the forefront of Java transactions (possible); perhaps the authors are most familiar with the products of these companies (looking at the biographies, this is likely). The warnings—find out who is writing the transaction software, which is probably at the heart of your business, and how often they’ve written such software before—were useful, if a bit repetitive.

That said, this book was still a good read, if a bit long (~360 pages). I think that Java Transaction Processing would be especially useful for an enterprise architect looking to leverage existing (expensive) transactional systems with more modern technology, and trying to see how Java and its myriad APIs fit into the mix. (This is what I imagine, because I’m not an enterprise architect.) I also think this book would be useful to DBAs; knowing about the Java APIs and how they deal with transactions would definitely help a DBA discuss software issues with a typical Java developer.

To me, an average Java developer, the first section of the book was the most useful. While transactions are fairly simple to explain (consider the canonical bank account example), this section illuminated complexities I’d not even thought of—optimizations, heuristic outcomes, failure recovery. These issues occur even in fairly simple setups—I’m working at a client who wants to update two databases with different views of the same information, but make sure that both are updated or neither; this seems to be a typical distributed transaction. The easiest way to deal with this is to pretend that such updates will always be successful, and then accept small discrepancies. That’s fine with click-throughs—money is a different matter.

However, if you are a typical web developer, I’m not sure this book is worth the price. I would borrow it from your company’s enterprise architect, as reading it will make you a better programmer (as well as giving you a sense of history—transactions have been around for a long time). But, after digesting fundamental distributed transaction concepts, I won’t be referencing this book anytime soon, since the scenarios simply don’t happen that often (and when they do, they’re often ignored, as outlined above).


Decreasing the size of a midlet jar

The J2ME application I have been working on has been ready for testing for quite some time, but I didn’t want to get a new AT&T phone. For J2ME, you really need a GSM phone–I don’t think any of the older TDMA models support it. But the GSM network coverage doesn’t match the coverage of the TDMA network–especially out west (aside: isn’t that magnifying glass pretty cool?). So I put off buying a phone until my summer road tripping was done.

I’ve had a Nokia 6160 for almost 4 years. Even though friends mocked the size of it, it was a great phone–durable, good talk time. I thought I’d try another Nokia, and got one of the lower end GSM phones, the 6200. This supported J2ME, and weighed maybe half as much. I was all stoked to try the application on my brand new phone.

I started download the jad file, and was getting ‘File Too Large’ errors. A couple of searches later, I found Nokia’s developer device matrix which is much more useful than the User Guide or the customer facing description of phones. Whoops. Most of the Series 40 (read: affordable) Nokia devices only supported J2ME applications which were, when jarred up, less than 64K in size.

Our application, however, was about 78K. This highlights one of the differences between J2ME and J2SE/J2EE. When coding in the latter world, I was never concerned about code size–getting the job done quickly was paramount, and if I needed to use 13 libraries which bloated the final size of my application, I did. On a cell phone, however, there’s no appeal to adding memory or changing the JVM configuration to optimize memory use. If the Nokia phone only accepts jars of 64K or less, I had three options:

1. Write off the Nokia Series 40 platform. Ugh–I like Nokias, and other folks do too.

2. Do some kind of magic URL classloading. This seemed complicated and I wasn’t sure how to do it.

3. Decrease the size of the jar file.

Now, the 78K jar had already been run through an obfuscator. I wasn’t going to get any quick and easy gains from automated software. I posted a question on the JavaRanch J2ME forum and received some useful replies. Here’s the sequence I went through:

1. Original size of the application: 79884 bytes.

2. Removal of extra, unused classes: 79881. You can see that the obfuscator did a good job of winnowing out unused classes without my help.

3. Changed all the data objects (5-6 classes), which had been written in classic J2SE style with getters and setters for their properties, to have public variables instead: 79465

4. Combined 3 of the data object classes into one generic class: 78868

5. Combined 5 networking classes into 2: 74543

6. Removed all the logging statements: 66044. (Perl to the rescue–$ perl -p -i -e 's!Log\.!//Log.!' `find . -name "*.java" -print |xargs grep -l 'Log\.'`)

7. Next, I played around with the jode obfuscator which Michael Yuan recommended. I was able to radically decrease the size of the jar file, but, unfortunately, that jar file didn’t work on the phone. I also got a ton of exceptions:

java.util.NoSuchElementException
        at jode.bytecode.BytecodeInfo$1.next(BytecodeInfo.java:123)
        at jode.obfuscator.modules.LocalOptimizer.calcLocalInfo(LocalOptimizer.java:370)
        at jode.obfuscator.modules.LocalOptimizer.transformCode(LocalOptimizer.java:916)
        at jode.obfuscator.MethodIdentifier.doTransformations(MethodIdentifier.java:175)
        at jode.obfuscator.ClassIdentifier.doTransformations(ClassIdentifier.java:659)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:320)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.ClassBundle.doTransformations(ClassBundle.java:421)
        at jode.obfuscator.ClassBundle.run(ClassBundle.java:526)
        at jode.obfuscator.Main.main(Main.java:189)

I'm sure I just didn't use it right, but the jar file size was so close to the limit that I abandoned jode.

8. Instead, I put all the classes in one file (perl to the rescue, again) and compiled that: 64057 bytes. The jar now downloads and works on my Nokia 6200 phone.

When I have to do this again, I'll definitely focus on condensing classes, basically replacing polymorphism with if statements. After removing extraneous Strings and concatenating all your classes into one .java file (both of which are one time shots), condensing classes is the biggest bang for your buck.


XML for data transmission

I was using XML as a file format for a recent project. Not a standardized dialect, just an ad hoc, custom flavor whipped up to hold the data of particular interest. Now, I’ve used many file formats and always thought XML was a bit hyped up. After all, it’s bulky and angle brackets can be a bit tedious to wade through. In addition, parsing it is hard (creating it is difficult too, if you want to do so by creating a DOM tree). Not too hard, you say. Well, compare the joy of a StringTokenizer parsing a pipe delimited line to the pain of traversing around a DOM tree (to say nothing of the “if” hell of a SAX handler).

However, XML does have strong points. Storing hierarchical data in XML is easier. And, as far as I’m concerned, XML’s killer feature as a file transport format is its self-documenting nature. Sure, you can put headers at the top of a pipe delimited file, but it’s easy enough to forget them. Omitting the XML tags isn’t really possible–you can choose obscure names for the tags that cloud the meaning of the data, but that’s about the worst you can do. Using a custom flavor of XML as a data transmission format can be a really good idea; it just means you’ll have a helper class to do the node traversing contortions everytime you want to read it. (I’m purposefully ignoring the technologies like JAX because I haven’t used them.)



java memory management, oh my!

How much do you understand basic java? Every day I find some part of this language that I’m not aware of, or don’t understand. Some days it’s cool APIS (like JAI) but today it’s concurrency. Now, language managed memory is a feature that’s been present in the languages in which I’ve been programming since I started. I’ve looked at C and C++, but taking a job coding in those seems to me it’d be like a job with a long commute–both have obstacles keeping you from getting real work done. (I’m not alone in feeling this way.) But this thread of comments on Cameron Purdy’s blog drove home my ignorance. However, the commenters do point out several interesting articles (in particular, this article about double checked locking was useful and made my head hurt at the same time) to alleviate that. I took a class with Tom Cargill a few years back, which included his threading module, that helped a bit.

However, all these complexities are why servlets (and EJBs) are so powerful. As long as you’re careful to only use local variables, why, you shouldn’t have to worry about threading at all. That’s what you use the container for, right? And we all know that containers are bug free, right? And you’d never have to go back and find some isolated thread related defect that affected your code a maddeningly miniscule amount of time, right?


PL/SQL

I recently wrote a basic data transformation program using Java and PL/SQL. I hadn’t used PL/SQL (which is an Oracle-specific procedural language for stored procedures) since writing a basic data layer for my first professional project (a Yahoo! like application written in PL/SQL, perl and Story Server–don’t ask). Anyway, revisiting PL/SQL reminded me of some of the things I liked and disliked about that language.

I like:

Invalidation of dependencies. In PL/SQL, if package A (packages are simply arbitrary, hopefully logical, groups of procedures and functions) calls package B, A depends on B. If the signatures of B are recompiled (you can separate the signatures from the implementations) package A simply won’t run until you recompile it. This is something I really wish other languages would pick up, because it at least lets you know when something you depend on has changed out from under you.

I dislike:

The BEGIN and END blocks, which indicate boundaries for loops and if statements, are semantically no different than the { and } which I’ve grown to love in perl and Java. But for some reason, it takes me back to my pascal days and leaves a bit of a bad taste in my mouth.

I’m unsure of:

The idea of putting business logic in a database. Of course, schemas are intimately tied to the business layer (ask anyone trying to move to a different one) and anyone who pretends that switching databases in a java applications is a simple matter of changing a configuration file is smoking crack, but the putting chunks of business logic in the data layer introduces a few problems. Every different language that you use increases the complexity of a project–and to debug problems with the interface between them, you need to have someone who knows both. Also, stored procedures don’t fit very well into any of the object relational mapping tools and pretty much force you to use jdbc.



© Moore Consulting, 2003-2017 +