Skip to content

The ant jar task and duplicate files can cause bizarre behavior and missing/incorrect files when unzipping

I just ran into some bizarre behavior. I’m building a web application on Windows XPSP2, using Ant 1.6.1. The application worked fine. Then, I added one more copy instruction, to move a GWT component I’d just added to the appropriate place in the war file. Suddenly, the application ceased to work.

After a few hours of tracking down the issue, I found that it didn’t matter whether it was the new GWT component or an old one; it didn’t matter whether the copy went to the same directory or a new one–simply adding the new set of files caused the issue. Then I noticed that the unzipped version of the application differed. In particular, the configuration files differed. That explained why the application wasn’t working–it wasn’t configured correctly.

But, why were the configuration files different?

I examined the generated jar file. When I unjarred it, the configuration files were correct. I was using the jar command, whereas the ant script was using unzip. I made sure the jar file was copied correctly. I made sure the old directory was deleted, and that the ant unzip task would overwrite existing files. Still, no fix–I was seeing the incorrect configuration files.

Then, this part of the jar task documentation jumped out at me:

Please note that the zip format allows multiple files of the same fully-qualified name to exist within a single archive. This has been documented as causing various problems for unsuspecting users. If you wish to avoid this behavior you must set the duplicate attribute to a value other than its default, “add”.

The other possible values for the duplicate attribute to the jar task listed are “fail” and “preserve”. It doesn’t explain what the other options actually do; “fail” causes the jar task to fail when duplicate files are encountered. This seems to be sane default behavior, and I’m not sure why it’s not the case. “preserve” seems to preserve the first file added, and doesn’t add duplicates, but doesn’t tell you that duplicates exist.

Update, 2:09:   “preserve” does tell you that duplicates exist, in this form: WEB-INF/web.xml already added, skipping

I had, for a variety of reasons, a jar task that was adding two sets of the configuration files, with the same names and paths, to the war file. Something about adding a few extra files seemed to flip a switch, and change existing behavior. Whereas before, the unzip task picked the correct configuration file, now the unzip task picked the incorrect file. I don’t know more than that, because I didn’t dig down into the source.

The answer is to move the correct files to the top of the jar task, and change the “duplicate” attribute to be “preserve”.

I hope this post saves someone a few hours of banging their head.

[tags]ant, jar files, duplicate files, headbanging[/tags]

12 thoughts on “The ant jar task and duplicate files can cause bizarre behavior and missing/incorrect files when unzipping

  1. Chii says:

    thanks, that was helpful! The default for jar should have been preserve – that is the one with the least surprise.

  2. Phil says:

    Yeah, this bit us too. We were building a WAR file using Ant, and didn’t notice that duplicate files had been added until we tried deploying to WebSphere, which threw some weird “invalid archive” exception.

  3. Marc says:

    thanks, for sharing this info. We also spend some hours searching for some a solution due to a ZIP double entry (SUN Java Applet Runtime did not download a JAR file due to a “verification error” because of a dublicate entry)

    IMHO duplicate=”preserve” should be the default, not “add”

  4. Alex says:

    Hi Dan,

    I had the exact same problem duplicate files in my jar archive. Since I started java programing one month ago I’m not the best with ant. But I discovered I had a big mistake in my jar task that leads to duplicate files without a warning.

    in my jar task i used basedir as attribute and also provided a fileset inside of the jar-task which looked like this:

    …..

    but the correct solution for me is this notation:

    …..

  5. moore says:

    Thanks Paul!

    Alex, I think you may have had some code removed by the WP comment system….

  6. rdixit says:

    I think that there should be a “overwrite” option for this. In many cases, adding does not work, nor does preserving the first option. I had a similar problem to yours and now what I do is remove the previous incorrect file and add it again from a different source.

  7. Vivek Gautam says:

    thanks a lot! its working, i spent a lot of time to get a solution bfore i came across ur suggestion , the code i added is as follows , may be somebody else will also benefit frm it :

  8. Kumar says:

    thanks, i have one doubt regarding this. I am creating EAR file . We are working on huge project. There mey occurance of duplicate files , in the sence file name may be same in different packages. If i want to findout that duplicate file which is sitting in different packages what i have to mention in ant script. Can any one suggest me plz…………….

  9. moore says:

    Hi Kumar,

    This issue (with zip) is only for files with the same ‘fully qualified name’. That means that the issues arises if you jar up (from the ‘src’ and ‘extra-src’ directories, respectively):
    top/src/com/foo/Main.java
    top/extra-ssrc/com/foo/Main.java

    But if you start from the ‘top’ directoy, both Main.java files have different fully qualified names, so it’s not an issue.

    Hope this helps.

  10. sam says:

    Thanks, saved me another 2 hours of debugging. +

  11. Alan jader says:

    I was facing the same problem, thank you for the post, its very useful… could you tell me how do you make tests to know if its working really correct ?

    ps.:in my case, I verify if there’s no duplicate in the folder with an automatic tool (www.dublicatefilesremover.com)

Comments are closed.