New vs old technologies

Compare the truths outlined here: “…for many businesses, sticking with what they have is the cheapest choice and best ROI” with Rands’ comments on tool cruft.

Of course, engineers aren’t businesses. But they operate under some of the same constraints–deadlines, limited money, etc. Despite what Rands says, there’s a balance to be struck between the new and the old. Of course, most folks, including myself, tend to lean towards the old and the known because it feels safer. But the known is (often) safer. Dion talks about it here and likewise doesn’t come to any conclusions.

I don’t want to sound like an old fogey, but I’ve been burned before in the past by short deadlines, new technologies and inexperienced users (of which I was one). I’m looking at Spring, having heard it praised to the sky, and want to use it on my next project. (Spring, incidentally, reminds me of a supercharged version of ATG’s Nucleus; what’s old is new again.) New tech is great, but not because it’s new. And old tech is safe, but not because it’s old. Each of these is appropriate when it’s the right tool for the job, but it’s hard to divorce that choice from my kneed jerk reactions and emotions–that’s what methods like ROI and research are designed to do.

Precision and Accuracy in Software

Back in college, when I took first year physics lab, there was a section of the course that focused on teaching the difference between precision and accuracy in measurement. This distinction was crucial in experimental physics, since measurement is the bedrock of such experimentation. Basically, precision is how many digits of a measurement actually mean something. If I’m measuring the length of a room with my stride (and found it to be 30 feet long), the precision is less than if I were to measure the length of the room with a tape measure (and found it to be 33 feet, 6 and ¾ inches long). However, it’s possible that the stride measurement is more accurate than the length found with the tape measure, that is, it reflects how long the room actually is. (Perhaps there’s clothing on the floor which adds tape measurement, but which I stride over.)

These concepts aren’t just valid in physics; I think they’re also useful in software. When building a piece of software, I am precise if I build what I say I am going to build, and I am accurate if what I build actually meets the client’s business needs, that is, it solves the business problem. Almost every development tool either makes development more precise or more accurate.

The concept of precision lends itself easily to automation. For example, unit testing is rapidly gaining credence as a useful software technique. With unit testing, a developer writes test cases for each part of their code (often at the method level). The running of these tests ensures that code is actually doing what the developer thinks it is doing. I like writing unit tests; it gives me comfort to know that corner cases are taken care of and that changes to code can be fairly easily regression tested. Other techniques besides unit testing that help ensure precision include:

Round tripping: using a tool like TogetherJ, I can ensure that the model (often described in UML) and the code are in sync. This makes it easier for me to verify my mental model against the code.

Specification writing: The more precise a spec is, the easier it is to translate into code.

Compilers: the checking that occurs at compilation time can be very helpful in ensuring that the code is doing what I think it is doing–at a very low level. Obviously, this technique depends on the language used.

Now, precision is needed, because if I am not confident that I understand what the code is doing, then I’m in real trouble. However, accuracy is much more important. Having a customer onsite is a great example of a technique to ensure accuracy: you have a business domain expert available all the time for developers’ questions. In this situation, when a developer stumbles across a part of the business problem that they don’t quite understand, the don’t do what developers normally do (in order of decreasing accuracy):

1. Ask another developer, which works great if the target audience is developers, but not so well otherwise.
2. Best approximation (read: guess at the correct answer).
3. Ignore the issue. (‘I’ve got a lot more code to write before I can go home today, and we’re shipping in two weeks. We’ll just let the customer discover it and deal with it as a bug.’)

Instead, they have a real live business person, to whom this software really matters (hopefully), who they can ask. Doing this makes it much more likely that the final solution will actually solve the business problem. Other techniques to help improve accuracy include:

Issue tracking software (I use Bugzilla): Having a place where questions and conversations are recorded is truly helpful in making sure the mental model of the business user and the programmer are in sync. Using a web based tool means that non-technical users can participate and contribute.

Specification writing: A well written spec allows both the business user and developer to have a sense of what is being built, which means that the business user can correct invalid notions at an early stage. However, if a spec is too detailed, it can be used to justify precision at the cost of accuracy (‘hey, the code does exactly what’s specified’ is the excuse you’ll hear).

Spring and other dependency injection tools, as well as IDEs: These tools help accuracy by decreasing the costs of changing code.

Precision and accuracy are both important in software engineering. Perhaps the best way to characterize the two concepts is that precision is the mapping of the programmer’s model of the problem to the computer’s model, whereas accuracy is the mapping of the business’ needs to the programmer’s model. However, though both are needed, accuracy is much harder to obtain. Knowing that I’m building precisely what I think I’m building is beneficial only insofar as what I think I’m building is actually what the customer needs.

Useful tools: javap

javap lets you examine java class files and jar files in a number of ways. See this web page for more information. For me, it’s an API reference. I use it in two ways:

1. When I’m coding, and I need to know the exact syntax of a method, I shell out: javap java.util.StringTokenizer. (Yes, I know that any modern IDE will do this for you without shelling out, but javap will work anywhere java is installed and with any editing tool. You trade portability for convenience.) One large catch is that inherited methods are not shown:

$ javap
Compiled from ""
public class extends{
public int read();
throws java/io/IOException
static {};
public void close();
throws java/io/IOException
public void reset();
throws java/io/IOException
public boolean markSupported();
public boolean ready();
throws java/io/IOException
public void mark(int);
throws java/io/IOException
public long skip(long);
throws java/io/IOException
public int read(char[],int,int);
throws java/io/IOException
public java.lang.String readLine();
throws java/io/IOException
java.lang.String readLine(boolean);
throws java/io/IOException

Running javap on does not show the method read(char[]), inherited from (This example is from the J2SE 1.4 libraries.)

2. Sometimes, the javadoc is too up-to-date (or your jar files are too old) to answer questions about an API. For example, I’m working on a project with Jetspeed which depends on Turbine version 2.2. Unfortunately, this is an extremely old version of Turbine (release 16-Aug-2003), and the javadoc doesn’t appear to be available. (Updated Dec 11: It looks like the Turbine 2.2 javadoc is indeed online. Whoops.) Generating the javadoc with ant is certainly an possibility, and if I found myself going time and again to verify the API of Turbine 2.2, I’d do that. But for a quick one- or two-off question about an API that no web search turns up, javap can be very handy.

In short, if you have a quick question about an API, javap can help you out.

Wireless application deployment solutions

It looks like there’s another third party deployment solution being touted. Nokia is offering deployment services for distribution of wireless applications called Preminet Solution (found via Tom Yager). After viewing the hideous Flash presentation and browsing around the site (why oh why is the FAQ a PDF?) this solution appears to be very much like BREW, with perhaps a few more platforms supported (java and symbian are two that I’ve found so far). Apparently the service isn’t launched yet, because when I click on the registration link, I see this:

“Please note that Preminet Solution has been announced, but it is not yet commercially launched. The Master Catalog registration opens once the commercial launch has been made.”

For mass market applications, it may make sense to use this kind of service because the revenue lost by due to paying Nokia and the operators is offset by more buyers. However, if you have a targeted application, I’m not sure it’s worthwhile. (It’ll depend on the costs, which I wasn’t able to find much out about.)

In addition, it looks like there’s a purchasing application that needs to be downloaded or can be installed on new phones. I can’t imagine users wanting to download another application just so they can buy a game, so widespread acceptance will probably have to wait until the client is distributed with new phones.

It’ll be interesting to see how many operators pick up on this. It’s another case of network effects (at least for application vendors); we’ll see if Nokia can deliver the service needed to make Preminet the first choice for operators.

Anyway, wonder if this competitor is why I got an email from Qualcomm touting cheaper something or other? (Didn’t really look at it, as I’ve written brew off until a J2ME bridge is available.)

Instant messaging and Yahoo!’s client

Like many folks, I’ve grown to depend on instant messaging (IM–it’s also a verb, to IM is to ‘instant message’), in the workplace. It’s a fantastic technology, but two years ago, I turned up my nose at it. (Of course, 5 years ago, in a similar manner, I turned up my nose at a cell phone, and now I wouldn’t be caught dead without it, so perhaps I’m not the best prognosticator.) What can it possibly offer that email can’t? I’m going to examine it from the perspective of a software developer, since that’s what I know; from that perspective, there are two main benefits IM offers that email doesn’t, timeliness and presence:

I didn’t think it was possible, but email can be too formal at times. When you have a question that needs to be answered right away or it becomes superfluous, IM is perfect. If it’s a question about consistency of API or an area that you know the recipient knows much better than you, sometimes 30 seconds of their time can be worth 15 minutes of yours. Of course, there’s a judgment call to be made; if you’re constantly IMing questions about the API of java.lang.String, you risk breaking up the answerer’s flow. However, used in moderation, it can greatly increase the communication between team members, especially when it’s a distributed team.

Presence is also a huge benefit of most IM software. This means that you have a list of ‘buddies’ that the IM software monitors for you. When each signs on or signs off, you’re made aware of that fact. This means that you can tell whether it’s worthwhile calling someone with a deeper question, or if you should just compose an email. The technical details of presence are being codified at the IETF and I foresee this becoming more and more useful, because it’s a non intrusive way for folks to manage their availability. It fulfills some of the same functions as a ‘door closed/door opened’ policy in an office, extending worldwide.

I use Yahoo IM because it fits my needs. Russell Beattie has recently written an overview of the main competitors and their clients, but technical geegaws like integration with music really don’t matter all that much to me. Much more important are:

1. Does everyone I need to talk to have an account? How easy is it for them to get an account?

2. Does it have message archiving? How searchable are such archives?

3. How stable is the client?

That’s about all I considered. I guess I let my contrarian streak speak too–I’m not a big fan of Microsoft, so I shyed away from Windows Messenger. There are some nice additional features, however. The ability to have a chat session, so that you can IM to more than one person at once, a la IRC, is nice. Grouping your buddies is great–each company I’ve consulted/contracted for has their own group in my IM client. I just discovered these instructions to put presence information on a web page. Combined with Maven and its intranet, or just put on any intranet page, this could be a useful tool for developers.

(I just read the Terms of Service for Yahoo, and I didn’t see any prohibitions on commercial use of Yahoo Messenger; however, there are a couple of interesting clauses that anyone using it should be aware of. In section 3, I found out that Yahoo can terminate your account if your information is not kept up to date (not really enforceable, eh?). And in section 16, “[y]ou agree not to access the Service by any means other than through the interface that is provided by Yahoo! for use in accessing the Service.” I wonder if that prohibits Trillian?)

One issue I have with the Yahoo client is the way status works. Presence is not a binary concept (there/not there); rather, it is broken down into various statuses–(not there/available/busy/out to lunch…). What I find myself doing is being very conscientious about changing my status from available to unavailable. However, I rarely remember to change back, which degrades the usefulness of the presence information. (If you have to ping someone over IM to see if they’re actually there, it means you might as well not have status information at all.) I spent some time browsing the preferences of Yahoo’s client, as well as googling, but didn’t find any way to have the client pop up a message the first time I IM someone when my status is not available.

IM is very useful, and I can’t imagine working without it now. I don’t know what I’m going to do when Yahoo starts charging for it.

NextBus: a mobile poster child

I think that NextBus is a fantastic example of a mobile application. This website, which you can access via your mobile phone, tells you when the next bus, on a particular line, is coming. So, if you’re out and about and have had a bit much to drink, or if you’ve just forgotten your bus schedule, you can visit their site and find out when the next bus will be at your stop. It’s very useful.

This is almost a perfect application for a mobile phone. The information needed is very time sensitive and yet is easy to display on a mobile phone (no graphics or sophisticated data entry needed). NextBus has a great WAP interface, which probably displays well on almost every modern phone. The information is freely available (at least, information on when the next bus is supposed to arrive is freely available–and this is a good substitute for real time data).

And yet, there are profound flaws in this service. For one, it abandons a huge advantage by not knowing (or at least remembering) where I am. When I view the site to find out when the 203 is coming by next, I have to tell the site that I’m in Colorado, and then in Boulder. The website is a bit better, remembering that I am an RTD customer, but the website is a secondary feature for me–I’m much more interested in information delivered to my phone.

Also, as far as I can tell, the business model is lacking (and, no, I haven’t examined their balance sheets). I don’t know how NextBus is going to make money, other than extracting it from those wealthy organizations, the public transportation districts. (Yes, I’m trying to be sarcastic here.) They don’t require me to sign in or pay anything for the use of their information, and I see no advertising.

So, a service that is almost perfect for the mobile web because of the nature of the information it conveys (textual and time sensitive) is flawed because it’s not as useful as it could be and the business model is up in the air. I can’t imagine a better poster child for the mobile Internet.

Syndication and blogs

I’ve tried to avoid self-referential blogging, if only becuase I’m not huge into navel staring. But, I just ran across an interesting blog: Wendyopolis, which is apparently associated with a Canadian magazine. Now, according to google, blogs are defined as:

“A blog is basically a journal that is available on the web. The activity of updating a blog is “blogging” and someone who keeps a blog is a “blogger.” Blogs are typically updated daily using software that allows people with little or no technical background to update and maintain the blog.

Postings on a blog are almost always arranged in chronological order with the most recent additions featured most prominently.”

(From the Glossary of Internet Terms)

However, I’d argue that there are several fundamental characteristics of a blog:

1. Date oriented format–“most recent additions featured most prominently.”
2. Informal, or less formal, writing style.
3. Personal voice–a reader can associate a blog with a person or persons.
4. Syndicatability–the author(s) provide RSS or Atom feeds. The feeds may be crippled in some way, but they are available.
5. Permalinks–postings are always available via an unchanging URL.

I can’t really think of any other salient characteristics. But the reason for this post is that Wendyopolis, which looks to be a very interesting weblog, doesn’t have #4. In some ways, that’s the most important feature, because it allows me to pull content of interest to one location, rather than visit sites.

I’ve written about this before, so I won’t beat a dead horse. Suffice it to say that, while Wendyopolis may speak to me right now, the chances of me ever visiting that blog ever again are nil, because of the lack of syndication. Sad, really.

Jetspeed 1 and 2

Here’s an article about Jetspeed 2. I’m actually very excited about this, though I feel that compliance to the spec is overrated. Perhaps I’m not really an enterprise developer, but I haven’t seen very many situations where I wanted a portlet (or an EJB) that had been developed on one portal server and deployed to another. But some of the Spring oriented features and ability to deploy struts application war files as portlets seem pretty neat and useful.

I’m currently working on a portal application, and we’re using Jetspeed 1–version 1.5, which has extensive documentation and has actually been released. JS1 is built on the Turbine framework. I hadn’t done much looking at this (other than a glance when I was looking at Torque for an OR layer). But so far, I’ve been really really impressed with Jetspeed and Turbine.

What has most impressed me is the configurability of these frameworks. There are a set of properties files that specify services, things like logging, persistence, localization, session validation, and authentication. The developers have done a great job of breaking these up into well defined chunks and letting me subclass services and plug in my own implementations. One example: our users do not login to the portal. Rather, they are authenticated by another application, which sets a cookie. I was able to disable Jetspeed’s own authentication system (which looks to a database) and plug in mine without making any modifications to source code. With properties overriding, I didn’t even have to modify the default properties files. Fantastic.

Jetspeed supports both JSP and Velocity as templating languages for the view. Velocity is used throughout the default portal, and I decided to use it as well. It’s an interesting language which has a lot of the benefits I’ve read about Groovy–there’s no types, and methods and properties look the same. It does look a bit perlish, I’ll admit–lots of $ and #, but it’s been fun to learn a different view language.

JS1 also provides an easy framework for developing portlets–an MVC framework, in fact. I don’t want to repeat what the relevant section of the portlet tutorial says, so I’ll just mention that developing dynamic content is a breeze.

I don’t want to say that Jetspeed has been a pure joy, however. There were a few days a couple of weeks ago that I wondered whether it had been a good choice at all. We have an existing base of users (hence the alternate authentication) and were noticing that the portal was loading slow when running against a user table of around 100K users. Then I ran some tests with The Grinder and noticed it was running really slow. Like 30 seconds to render a page with 5 portlets. Luckily, some sleuthing around indicated that there was much extraneous database access going on, and when that was eliminated, performance became acceptable.

In addition, the default build process is mavenized. I had an extremely bad experience and ended up writing a simple ant build script to do what I want. It’d be nice if both were options–since lots of people come to JS1 looking to slap a portal together for cheap, rather than thrash around with a complex build process. (I’d say at least forty percent of the messages to the mailing list are build related–not a good sign.) I just went to a BJUG talk about Maven, and Thompson was persuasive, so I might give maven another chance.

All in all, though, I’ve been very happy with Jetspeed. I am looking forward to JS2; I wish the timing had been better so I could have used it on the current project.

Useful tools: p6spy

This entry kicks off a series of entries where I’ll examine some of my favorite tools for development. Some of them will be long, some short, but all of them will highlight software I use to make my life a bit easier.

A large, large chunk of the development I do is taking data from a relational database to a an HTML screen, and back again. Often there are business rules for transforming the data, or validation rules, but making sure the data is stored safely and consistently is a high priority, and that means a relational database.

However, I do much of my work in java, which means that the relational-OO impedance mismatch is a common problem. One common way to deal with it is to use an OR tool–something like OJB or JDO. These tools provide object models of your database tables, usually with some help from you. You then have the freedom to pretend like your database doesn’t exist, and use these objects in your application. The OR framework takes care of the dirty work like SQL updates and caching.

Every convenience has its price, however, and OR mapping tools are no exception. The same abstraction that lets you pretend that you’re simply dealing with objects means that you cannot easily examine the SQL that is generated. In addition, the way that you’re using the objects may cause performance issues, because you’re treating the data as objects, rather than rows.

It’s much the same issue as calling methods over the network via RMI or accesing files via NFS: the abstraction is great and means that programmers don’t have to think about the consequences of remote access. But the failure of the abstraction can be catastrophic, all the more so because the programmer was not expecting to have to deal with the grotty details under the abstraction (that’s the whole point, right?).

OR tools do not fail often, or have many catastrophic failure modes, but they sure can be slow. With open source software, you can dig around and see how SQL is being generated, but that’s tedious and time consuming. With commercial products, you don’t even have that option. (Some OR tools may have their own ‘Show me the SQL’ switch–I haven’t run into them.)

Enter p6spy. p6spy can be used in place of any JDBC driver. You point it to the the real driver and it passes on any configuration or SQL calls to that driver. But p6spy logs every SQL statement passed to it and every result set passed back. (A fine non object oriented example of the Decorator pattern.)

It took me about 15 minutes to figure out how to use p6spy, the software is open source with decent documentation, the latest version has data source support, and it scratches an itch that most, if not all, java developers will have at some time. With p6spy, you can find out what that OR tool is doing under the covers–it’s an easy way to peel back some of the abstraction if needed.

© Moore Consulting, 2003-2017 +