Skip to content

The ant jar task and duplicate files can cause bizarre behavior and missing/incorrect files when unzipping

I just ran into some bizarre behavior. I’m building a web application on Windows XPSP2, using Ant 1.6.1. The application worked fine. Then, I added one more copy instruction, to move a GWT component I’d just added to the appropriate place in the war file. Suddenly, the application ceased to work.

After a few hours of tracking down the issue, I found that it didn’t matter whether it was the new GWT component or an old one; it didn’t matter whether the copy went to the same directory or a new one–simply adding the new set of files caused the issue. Then I noticed that the unzipped version of the application differed. In particular, the configuration files differed. That explained why the application wasn’t working–it wasn’t configured correctly.

But, why were the configuration files different?

I examined the generated jar file. When I unjarred it, the configuration files were correct. I was using the jar command, whereas the ant script was using unzip. I made sure the jar file was copied correctly. I made sure the old directory was deleted, and that the ant unzip task would overwrite existing files. Still, no fix–I was seeing the incorrect configuration files.

Then, this part of the jar task documentation jumped out at me:

Please note that the zip format allows multiple files of the same fully-qualified name to exist within a single archive. This has been documented as causing various problems for unsuspecting users. If you wish to avoid this behavior you must set the duplicate attribute to a value other than its default, “add”.

The other possible values for the duplicate attribute to the jar task listed are “fail” and “preserve”. It doesn’t explain what the other options actually do; “fail” causes the jar task to fail when duplicate files are encountered. This seems to be sane default behavior, and I’m not sure why it’s not the case. “preserve” seems to preserve the first file added, and doesn’t add duplicates, but doesn’t tell you that duplicates exist.

Update, 2:09:   “preserve” does tell you that duplicates exist, in this form: WEB-INF/web.xml already added, skipping

I had, for a variety of reasons, a jar task that was adding two sets of the configuration files, with the same names and paths, to the war file. Something about adding a few extra files seemed to flip a switch, and change existing behavior. Whereas before, the unzip task picked the correct configuration file, now the unzip task picked the incorrect file. I don’t know more than that, because I didn’t dig down into the source.

The answer is to move the correct files to the top of the jar task, and change the “duplicate” attribute to be “preserve”.

I hope this post saves someone a few hours of banging their head.

[tags]ant, jar files, duplicate files, headbanging[/tags]

Java Programming | moore |

InvocationException using GWT RPC and custom objects

If you are using GWT 1.4 and sending custom objects over the wire, and see an exception like this in your log files:


2007-11-09 11:06:13 Exception while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Type 'com.foo.common.data.User' was not assignable to 'com.google.gwt.user.client.rpc.IsSerializable'
and did not have a custom field serializer.  For security purposes, this type will not be serialized.
at com.google.gwt.user.server.rpc.impl.LegacySerializationPolicy.validateSerialize(LegacySerializationPolicy.java:136)
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize(ServerSerializationStreamWriter.java:331)
at com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamWriter.writeObject(AbstractSerializationStreamWriter.java:81)
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serializeValue(ServerSerializationStreamWriter.java:259)
at com.google.gwt.user.server.rpc.RPC.encodeResponse(RPC.java:574)
at com.google.gwt.user.server.rpc.RPC.encodeResponseForSuccess(RPC.java:442)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:530)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:265)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost(RemoteServiceServlet.java:187)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

You may be very interested in this release note from the 1.4.59 release:

RPC now generates a serialization policy file during compilation. The serialization policy file contains a whitelist of allowed types which may be serialized. Its name is a strong hash name followed by .gwt.rpc. This file must be deployed to your web server as a public resource, accessible from a RemoteServiceServlet via ServletContext.getResource(). If it is not deployed properly, RPC will run in 1.3.3 compatibility mode and refuse to serialize types implementing Serializable. Updated 3/6/2011 to correct link location.

This exception caused me a world of grief, first because I wasn’t seeing anything in standard out (it logged to a different file), then because I couldn’t find the source of LegacySerializationPolicy, then because I have the gwt source in one directory, but because of rewrites, it appears to be served from another directory. It manifests on the client side as an InvocationException with a fantasticly unhelpful error message.

The fix is simple: copy the serialization policy file to wherever the GWT files appear to be served from.

GWT | moore |

FogBugz world tour, Boulder edition

I went and saw Joel Spolsky’s talk about FogBugz6 tonight. It seems to be quite the powerful software development tool. But I’m afraid that it seems to suffer like every tool–it forces you into certain methods of development. For example, there’s no way to ensure that every bug entered is viewed by QA. Now, that isn’t a problem for the teams I currently work on, but I can see it being a problem for teams I have worked on. Joel mentioned very valid reasons for doing this, but they only seem valid for the subset of development teams that FogBugz targets.

In fact, as I left, almost every conversation I heard was about the product, and how people could fit it into their process, rather than use the process it gives you. Because FogBugz really is more than a bug tracking system–it now goes from documentation/requirements gathering all the way to estimation to bug tracking to customer support. FogBugz appears to be a tool that is used in almost the entire software development life cycle–hey look, it’s RUP lite.

But I’ve used never version 6, and I’m sure there are significant wins. My other concerns are that the software estimation parts sound like they’re 1.0 features (just from the words he used–at best 1.1 since they used FogBugz6 to develop FogBugz6); I’d rather wait until the features are more settled. I’m sure you could use just the bug tracking system, and they’ve certainly taken the ‘Web 2.0’/instant response/make it feel like a desktop application ideas to heart. The cost is another concern; while minimal, it is greater than $0. On many projects I’m on, just using any bug tracker, let alone an entire software development tool, is difficult, and you can’t beat stealth bug tracker installs. (I’m on record as saying “I have to say that I think the open source solutions (Bugzilla and PHPBT) are going to eat the commercial solutions’ lunch for small projects, because they are a cheaper substitute with all the required attributes”, just as an FYI.)

One thing that really surprised me at the talk is how many folks were there evaluating FogBugz as opposed to seeing Joel speak. Around one third of the audienced had used or was using FogBugz. Joel opened up the floor to questions, and every single one except one (of mine) was about features or flaws in FogBugz. I mean, this is the guy who wrote the Joel Test and no one took the opportunity to ask him general development questions, even though he said he’d field them. I don’t know what the deal was.

Will I give FogBugz a try? Not right now. But I’ll keep an eye on what they’re doing.
[tags]software development tools, bug tracking, fogbugz, joelonsoftware[/tags]

Programming Web Applications | moore |

Interesting posts about web application performance

The good folks over at the YUI blog posted this: What the 80/20 Rule Tells Us about Reducing HTTP Requests a while ago. I bookmarked it, but wanted to point it out to other folks–it’s a nice bit of research, with numbers and graphs and all that good stuff. It opened my eyes to various non intuitive aspects of web application performance. The whole series is a nice read; part 1 is linked above and here’s part 2, part 3 and part 4.

Web Applications YUI | moore |

GWT impressions

After about a year of working with GWT, it seems to me like there are two places where it is really useful as a technology:

  1. Quickly building relatively sophisticated user interfaces for entire web based applications. An intranet ‘client-server’ type application, like a timesheet, would be a perfect fit. If you use Java on the server side, domain objects can even be shared.
  2. Building small widgets that have anything beyond the simplest logic. This is the best way to integrate GWT into an existing application–add small bits of functionality that improve the user experience. You can use GWT to manage, reuse and package this logic.

However, what GWT is best for is not the limiting factor for GWT; rather, if you aren’t a Java developer, GWT just doesn’t make sense. (I’m ignoring the fact that if a user doesn’t have JavaScript enabled, GWT doesn’t make sense, since this is a failing of almost all the Web 2.0 rich user interface toolkits).

For me, being a Java developer and a fan of Eclipse, GWT is a natural fit for a number of reasons. The Java object serialization support, the use of an IDE to code Javascript, the JRE emulation, and the event driven user interface model all make it extremely comfortable to develop in the language. If you’re already coding the server side in Java, GWT is one less language to learn (until you need to do something that isn’t provided for in the emulation libraries, or you need to use a Java 1.5 feature, or a bug leaks up through the abstraction; of course, these problems will never happen).

While I don’t have deep knowledge with other toolkits (I’ve worked slightly with the Yahoo! User Inteface Library and have toyed with Dojo), it seems to me that many many folks can get by using them; there’s no tie to Java.

If someone was going to ask me whether or not they should use GWT, I’d boil it down to the following questions:

  1. Are your developers familiar with Java? (If ‘no’, don’t use GWT.)
  2. Are your developers familiar with JavaScript? (If ‘yes’, consider not using GWT.)
  3. Are you integrating with an existing app? (If ‘yes’, GWT might be a good fit.)
  4. If so, are you planning to ‘web 2.0’-ify the existing application, or add widgets to enhance existing functionality? (If planning to ‘web 2.0’-ify existing functionality, don’t use GWT.)

On a final note, I don’t want to bag on GWT too much. GWT has improved tremendously over the past year or so, and I’m very glad to have used it. I think it’s quite cool tech, and I think it has really improved the user experience on my client’s site.

Thank you, Google, for releasing GWT and making it available for me to use.

Dynamic Languages GWT Java YUI | moore |

Four year blogaversary

It’s been four years and a few days since I started writing this blog. It’s been a great experience. I’ve had good months (15 posts) and bad months (1 post), but I’ve written 352 posts over the last four years. I’ve learned a lot from this blog–how to shoot my mouth off, how to acknowledge mistakes in public, and how to write a post that gets hijacked by others for their own purposes. I thought I’d take the opportunity to highlight four of my favorite posts.

  • Business Process Crystallization: This is a nice little post about how software is both shaped by the business process it will assist and shapes that same process. It’s one of my original thoughts–at least I haven’t seen too much literature taking this view of software.
  • Installing the median user defined function on MySQL: I always enjoy step by step tutorials, and this is one of the better ones I have written. I also enjoyed the challenge of making a mysql user defined function work when I’d never done any significant C development.
  • The Social Life of Information: This book was a slow read, but explains a fascinating concept–that much of the information we receive is contextual and connotative. This was a revelation to me, since I’m more comfortable with denotative, explicit information.
  • Step By Step: A Mortgage Calculator using GWT: I enjoyed this blogging experience, the entire series. My client and I worked together to release a site repeatedly over a week, as well as releasing the source code and documenting the entire experience of learning, developing and deploying a new technology.
Blog | moore |

Choosing new technology, or tail chasing

Robert Hanson, who built the very useful GWT Widget Library, has an interesting post where he asks:

Let’s say that you are a developer, and you have been spending the past year or so really getting to know a given technology. Now you are being told that the technology you are using is inferior to this “other” technology. You take a look and realize that it might be best to switch. A year later you finally have a good understanding of the tool, and use it with great skill. Then someone tells you about this “other” technology.

How many of us built our own MVC frameworks only to move to Struts, then maybe on to Spring MVC. Sure, there are some improvements made in each technological step, but since you are spending most of your time really getting to know a product you often spend little time getting the most out of it. This is compounded by the fact that you often use several of these products at the same time, adding to what you need to learn.

So what is a dog to do? Although you are moving forward, you never quite catch the tail. Should you just stop moving forward, or run faster or slower?

Personally, I think that there is a middle ground. As a developer, you need to keep up on broad trends and tools, because they can make you so much more productive. The problem arises when you don’t know how much more productive you will be, until you use the technology or tool for a while….

However, just because there is a new tool around, that doesn’t mean you have to use it. In fact, if you have an existing technology that does the job, you should not abandon it just to move to the new technology. There’s always a cost analysis, because learning a new technology is not free. Your time is worth something.

This cost analysis is something that developers should learn to do and appreciate because that process is exactly what most companies need to do before they decide to implement or build new software. Just like a developer, most companies think that a new technology, or system, will help them, but are unsure how much it will help them, and how much it will cost them. Just as for a company, a developer deciding to learn and use a new technology is not solely a technology decision.

There are many ways to minimize the risk of learning a new technology–prototype, read documentation, be conservative and consult someone who’s an expert in the new technology (which means they’ve already made some of the mistakes). Each of these have benefits and detriments. Prototyping takes more time than the others. Reading documentation is great if there is documentation, and if the documentation is accurate, but might teach one as many lessons as using the technology. Being conservative means that you’ll probably miss out on some productivity improvements, just as you’ll miss out on some time sinks. Consulting an expert is great, if you have access and know what questions to ask.

I think the answer to Robert’s final question is intensely context sensitive. It depends on the following five considerations, among others:

  • how crucial a new technology is to your productivity (ie, if you are a java business developer, learning GWT might be lower on the list than learning Spring)
  • how easy you think it will be to learn
  • whether you can be paid to learn it
  • how much spare time you have
  • whether you have a project to use the new technology on

[tags]tail chasing,technology[/tags]

Programming Technology | moore |

IE 7 does not have a built in javascript debugger (update)

I recently was debugging the dreated “Operation Aborted” error, which was appearing in both IE6 and IE7. (Incidentally, Joel has written a great delineation of the three parts of software: design, development and debugging.) I usually do development in FF and then test in IE. But this bug is specific to IE, which made debugging harder. I was using IE6, looking at the error message and trying to find the line numbers it referenced. Pretty miserable. The fact that I was dealing with external code (Google Analytics, Google Website Optimizer) made it all the more fun!

I was testing at work, and feeling a bit frustrated. I found a reference to the Microsoft Script Debugger, but wasn’t able to find it separate from Microsoft Office.

I went home and tested on IE7, and saw the same error. I thought I’d try to find the debugger for IE7. It was easier than I thought–all you have to do is go to ‘Tools’, then ‘Internet Options…’, click the Advanced tab and uncheck ‘Disable Script Debugging (Internet Explorer)’. Then, when you visit a page with a javascript error, it asks you if you want to debug it. Fantastic!

As far as I know, that was a vanilla install of IE7. I’m thrilled they’re shipping a debugger, as it will make developing for them just a little bit easier.

Update: I was incorrect, I think that you need to install the script debugger separately.  Link below.

[tags]ie7,javascript debugger[/tags]

Dynamic Languages Web Applications | moore |

Beware collections with GWT 1.4 Final

I just upgraded our application to GWT 1.4.60, aka 1.4 Final. The upgrade was for the most part, pretty smooth. Only one issue reared its head. I have a number of objects that are Serializable that contain other Serializable objects. I had not marked them with the @gwt.typeArgs javadoc metadata, and things had worked just fine in GWT 1.4.10 (RC1). However, with 1.4 Final, I kept getting an InvocationException.

These tend to be real pains, because I’ve never seen them raised in code I write. Instead, they seem to arise from misconfiguration. Last time I saw one, I was compiling against the wrong version of gwt-servlet.jar.

This time, I hadn’t marked the contents of various Sets and Maps with the contents (using the aforementioned @gwt.typeArgs markup). I received warnings at compile time about that, but had received warnings with RC1 as well. I had, in typical developer fashion, noticed that the code still worked and thus ignored the warnings.Putting that markup in my code seemed to solve the problem.

There’s one thread on the groups about something similar to this. Here’s the tracking issue.

GWT | moore |