Skip to content

All posts by moore - 97. page

Decreasing the size of a midlet jar

The J2ME application I have been working on has been ready for testing for quite some time, but I didn’t want to get a new AT&T phone. For J2ME, you really need a GSM phone–I don’t think any of the older TDMA models support it. But the GSM network coverage doesn’t match the coverage of the TDMA network–especially out west (aside: isn’t that magnifying glass pretty cool?). So I put off buying a phone until my summer road tripping was done.

I’ve had a Nokia 6160 for almost 4 years. Even though friends mocked the size of it, it was a great phone–durable, good talk time. I thought I’d try another Nokia, and got one of the lower end GSM phones, the 6200. This supported J2ME, and weighed maybe half as much. I was all stoked to try the application on my brand new phone.

I started download the jad file, and was getting ‘File Too Large’ errors. A couple of searches later, I found Nokia’s developer device matrix which is much more useful than the User Guide or the customer facing description of phones. Whoops. Most of the Series 40 (read: affordable) Nokia devices only supported J2ME applications which were, when jarred up, less than 64K in size.

Our application, however, was about 78K. This highlights one of the differences between J2ME and J2SE/J2EE. When coding in the latter world, I was never concerned about code size–getting the job done quickly was paramount, and if I needed to use 13 libraries which bloated the final size of my application, I did. On a cell phone, however, there’s no appeal to adding memory or changing the JVM configuration to optimize memory use. If the Nokia phone only accepts jars of 64K or less, I had three options:

1. Write off the Nokia Series 40 platform. Ugh–I like Nokias, and other folks do too.

2. Do some kind of magic URL classloading. This seemed complicated and I wasn’t sure how to do it.

3. Decrease the size of the jar file.

Now, the 78K jar had already been run through an obfuscator. I wasn’t going to get any quick and easy gains from automated software. I posted a question on the JavaRanch J2ME forum and received some useful replies. Here’s the sequence I went through:

1. Original size of the application: 79884 bytes.

2. Removal of extra, unused classes: 79881. You can see that the obfuscator did a good job of winnowing out unused classes without my help.

3. Changed all the data objects (5-6 classes), which had been written in classic J2SE style with getters and setters for their properties, to have public variables instead: 79465

4. Combined 3 of the data object classes into one generic class: 78868

5. Combined 5 networking classes into 2: 74543

6. Removed all the logging statements: 66044. (Perl to the rescue–$ perl -p -i -e 's!Log\.!//Log.!' `find . -name "*.java" -print |xargs grep -l 'Log\.'`)

7. Next, I played around with the jode obfuscator which Michael Yuan recommended. I was able to radically decrease the size of the jar file, but, unfortunately, that jar file didn’t work on the phone. I also got a ton of exceptions:

java.util.NoSuchElementException
        at jode.bytecode.BytecodeInfo$1.next(BytecodeInfo.java:123)
        at jode.obfuscator.modules.LocalOptimizer.calcLocalInfo(LocalOptimizer.java:370)
        at jode.obfuscator.modules.LocalOptimizer.transformCode(LocalOptimizer.java:916)
        at jode.obfuscator.MethodIdentifier.doTransformations(MethodIdentifier.java:175)
        at jode.obfuscator.ClassIdentifier.doTransformations(ClassIdentifier.java:659)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:320)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.PackageIdentifier.doTransformations(PackageIdentifier.java:322)
        at jode.obfuscator.ClassBundle.doTransformations(ClassBundle.java:421)
        at jode.obfuscator.ClassBundle.run(ClassBundle.java:526)
        at jode.obfuscator.Main.main(Main.java:189)

I'm sure I just didn't use it right, but the jar file size was so close to the limit that I abandoned jode.

8. Instead, I put all the classes in one file (perl to the rescue, again) and compiled that: 64057 bytes. The jar now downloads and works on my Nokia 6200 phone.

When I have to do this again, I'll definitely focus on condensing classes, basically replacing polymorphism with if statements. After removing extraneous Strings and concatenating all your classes into one .java file (both of which are one time shots), condensing classes is the biggest bang for your buck.

Java Programming | moore |

Book Review: Divorce Your Car

Divorce Your Car, by Katie Alvord, is thought provoking. In the United States of America, an automobile is many things to many people: transportation, status symbol, hobby, money pit. Alvord takes apart the place of the car in modern society (the focus of the book is on North America, though she does refer to Europe and the Third World in places) and roundly condemns our dependence.

Her book is split into three parts–the first covers the history of the automobile and other forms of transport. She legitimizes what I’d often heard and dismissed as a myth–the car industry bought up the transit systems of cities in the US early in the 20th century and replaced them with buses. The second is a laundry list of the negative effects of the car (which, I must confess, I didn’t finish–too depressed after the first thirty pages). The final section covers alternatives, including walking, biking, mass transit, non-gasoline cars, and telecommuting.

I found the book to be quite good in outlining the problem and highlighting solutions. The dependence of modern life on the car is a dependence on convenience. But, to some extent, it’s a matter of inertia. Automobiles are so prevalent and easy that many of us never try the alternatives, let alone use them in preference to our car. A strong point is that she realizes that car-free living isn’t for anyone, and makes a point that going car-lite can have a positive effect as well. She also touches on the far reaching implications that technology decisions have had on our society, our cities and our lives–from subsidies to the development of advertising. It would have been interesting to read more about that, but what she did say was definitely thought provoking.

However, I do have three quibbles. Alvord cites sources extensively, but her arguments would be more compelling were the sources less biased (as you can tell by titles like Asphalt Nation) and more first hand. She ignores two factors that would affect my divorce. Giving up your car, or at the very least being aware of alternatives, makes driving after drinking less likely–a good thing! On the other hand, if you don’t have a car, you suddenly have a dearth of available camping and hiking activities. But these concerns aren’t everyone’s, to be sure.

Overall, a book well worth reading, especially if you commute a lot. Too bad they don’t sell it as a book on tape!

Books Technology and Society | moore |

XML for data transmission

I was using XML as a file format for a recent project. Not a standardized dialect, just an ad hoc, custom flavor whipped up to hold the data of particular interest. Now, I’ve used many file formats and always thought XML was a bit hyped up. After all, it’s bulky and angle brackets can be a bit tedious to wade through. In addition, parsing it is hard (creating it is difficult too, if you want to do so by creating a DOM tree). Not too hard, you say. Well, compare the joy of a StringTokenizer parsing a pipe delimited line to the pain of traversing around a DOM tree (to say nothing of the “if” hell of a SAX handler).

However, XML does have strong points. Storing hierarchical data in XML is easier. And, as far as I’m concerned, XML’s killer feature as a file transport format is its self-documenting nature. Sure, you can put headers at the top of a pipe delimited file, but it’s easy enough to forget them. Omitting the XML tags isn’t really possible–you can choose obscure names for the tags that cloud the meaning of the data, but that’s about the worst you can do. Using a custom flavor of XML as a data transmission format can be a really good idea; it just means you’ll have a helper class to do the node traversing contortions everytime you want to read it. (I’m purposefully ignoring the technologies like JAX because I haven’t used them.)

Programming | moore |

JMS first impressions

I’m currently working on a project using JMS. It’s my first experience with messaging, though I’ve read a bit about it; back in the dot.com boom, you could get the O’Reilly JMS book just for giving up demographic data (thanks Sonic!). However, this project is using JBossMQ, sending TextMessages containing XML into a number of Queues. I’ve integrated message sending into existing client applications. These applications come under heavy load periodically, so we wanted to make producing messages as simple as possible. The XML documents that these applications create are simply well-formed, and typically small. The consumers of these messages, on the other hand, undertake some fairly slow activities: they do some data massaging, and update or insert into multiple databases. Therefore, the consumers process messages asynchronously.

Such behavior demonstrates the strength of messaging (outlined most eloquently here): because of the decoupling between the producer of the message and the consumer, some objects can do “stuff” at a different rate of speed than the other objects which are needed to finish handling the “stuff”. The downside, of course, is the difficulty of receiving any form of confirmation (unlike typical synchronous systems, where, since the calling object blocks until return, passing back values is simple). Enterprise messaging systems, for which JMS simply provides a uniform (and somewhat limited) API, provide some guarantees–at least the producer can rest assured that the message did get to some consumer. This contract means that the producer can throw many many messages into a queue, confident that even if it takes a long time to parse and handle each message, every one will be passed off to a consumer. Of course, if the rates of message production and consumption are vastly different, there can be problems.

(When a component of an typical, synchronous system fails, then the caller is left to handle the wreckage. Since messaging systems interpose, when a consumer fails, the producer never finds out or has to deal with it.)

It seems like messaging systems would be great for integrating disparate systems as well–after all, message formats can be entirely arbitrary and don’t have to be understood by the messaging server at all. For example, this project has a perl program that was generating quite a bit of interesting data; it would have been nice to put this into a queue for a java program to consume. Unfortunately, the options for having a non-java producer participate in a JMS system are limited:

1. Some implementations provide client APIs for other languages (I saw a posting about SonicMQ and C++). JBossMQ has none that I could find.

2. Perl can call methods on java objects. A bit scary for a production system, and not a solution for producers/consumers written in other languages.

3. You could set up a java service listening on a port that would just take what’s given and send a JMS message containing that to a queue. Now you lose much of the robustness of a messaging solution, since you’re dependent on this service to make sure your messages get through.

4. Cut out the java service above, and decode the format that JBossMQ is using–since it’s listening on a port, and you have access to the JBossMQ source you could probably hack up a client to send a message directly. This would be a maintenance hassle and isn’t portable between JMS implementations.

The perl client problem ended up going away because we used a scalable, asynchronous message delivery system–Sendmail. (I wonder whether anyone has ever slapped JMS on top of Sendmail. A quick Google search showed nothing, but it seems like a natural pairing. I’m a bit worried about the reliability of delivery, but I’ve a sysadmin friend who says that if you control both the beginning and endpoints of a mail system, you can guarantee delivery.) All in all, JMS seems like a clean standard manner in which to enforce separation of concerns and gain a fair amount of robustness.

Java | moore |

Internet Bookmobile

I have to say that the Internet Bookmobile is way cool. (I remember the bookmobile of my youth, and it didn’t carry anywhere near 20,000 books!) I think that the Internet Bookmobile shows two things:

1. The enduring power of the book. I’m definitely not the first person to say this, but the portability, durability, cost and readability of bound books is hard to beat with any electronic format. They’re going to be around for a long time, despite the efforts of e-book software purveyors.

2. Digital, public domain texts are a good thing. (But copyright keeps getting extended.)

(Thanks to Brian D Foy for the link.)

Technology and Society | moore |

struts module ClassCastException

If you get this exceptions like this: 

2004-07-21 15:35:06 action: null
java.lang.ClassCastException
        at org.apache.struts.action.ActionServlet.initModulePlugIns(ActionServle
t.java:1142)
        at org.apache.struts.action.ActionServlet.init(ActionServlet.java:486)
        at javax.servlet.GenericServlet.init(GenericServlet.java:256)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:918)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:810)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3279)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:3421)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1123)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:638)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1123)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:343
)
        at org.apache.catalina.core.StandardService.start(StandardService.java:388)
        at org.apache.catalina.core.StandardServer.start(StandardServer.java:506)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:781)
        at org.apache.catalina.startup.Catalina.execute(Catalina.java:681)
        at org.apache.catalina.startup.Catalina.process(Catalina.java:179)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:324)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:243)

2004-07-21 15:35:06 StandardWrapper[/yourmodule:action]: Marking servlet action as unavailable
2004-07-21 15:35:06 StandardContext[/yourmodule]: Servlet /yourmodule threw load() exception

and you’re using struts with modules, make sure that all of the classes referenced in all of the module-level struts-config.xml files are in the classpath of Tomcat.

Java | moore |

OJB and object caching

I’m working on a project with ObJectRelationalBridge 1.0RC4. This release is a year old, but has suited our needs up to now, but now I’ve a couple of gripes.

1. The caching online documentation doesn’t apply to my version. Hey, RC4 isn’t the latest and greatest, so that’s fair enough, but it would be nice if it was clear to which version the documentation applied. Once I realized (through a xerces exception) that this was the case, I was able to download the correct version and view that documentation locally. But what if I’d been using rc1, which doesn’t appear to be downloadable anymore? Perhaps I could get documentation from CVS, but this just shows that you really should keep virgin downloads of your external dependencies (code, documentation, whatever) for future reference. Would it be overkill to spider the software’s website when you make the decision to go with a particular version, and store that off somewhere?

2. In OJB 1.0RC4, object caching doesn’t seem to work (using the ObjectCacheDefaultImpl class). Not too much useful on the mailing list but I did a bit of sleuthing on my own. P6Spy is a slick java application that decorates any JDBC driver, writing all the statements that clients are making to a log file, then passing those statements on through to the driver. Installation on tomcat was very easy, and it was enlightening to see what OJB had been up to under the covers: going back to the database to recreate a user object, even though ObjectCacheDefaultImpl has no timeout for cached objects.

I’ll probably update the project to the newly released OJB 1.0.0 (don’t those numbers strike fear into your heart? I suppose after 7 release candidates over more than a year, 1.0.0 should be pretty solid) and see if object caching works.

Java | moore |

Book Review: How to Lie with Statistics

How to Lie with Statistics, by Darrel Huff, should be required reading for everyone. The cachet of numbers are used all the time in modern society. Usually to end arguments–after all, who can argue with “facts”? Huff shows how the same set of numbers can be tweaked to show three different outcomes, depending on where you start and what you use. The fundamental lesson I learned from this book is that mathematical calculation involves a whole set of conditions, and any number derived from such a calculation is meaningless without understanding those conditions.

He also mentions that colleagues have told him that the flurry of meaningless statistics is due to incompetence–he dispatches this argument with a simple query: “Why, then, do the numbers almost always favor the person quoting them?” Huff also provides five questions (not unlike the five d’s of dodgeball) for readers to ask, when confronted with a statistic:

1. Who says so?

2. How does he know?

3. What’s missing?

4. Did somebody change the subject?

5. Does it make sense?

All this is wrapped up in a book with simple examples (no math beyond arithmetic, really) and quaint 1950s prose. In addition humor runs from the beginning (the dedication is “To my wife with good reason”) to the end (on page 135, Huff says “Almost anybody can claim to be first in something if he is not too particular what it is”). This book is well worth a couple hours of your time.

“How To Lie With Statistics” at Amazon.

Books Technology and Society | moore |

More on the difficulty of tuning the JVM

Here’s a great overview of the JVM memory model (for all of Sun’s JVMs, including the latest changes). I find it intensely interesting that he brushes over what, for me, is the most complicated part of any web application tuning–testing. He outlines a process for initially sizing the various compartments of the JVM heap (for versions 1.4 and below), and then says: ‘The [resizing] process continues by “testing and tweaking” until things look good.’

Wow. Talk about waving your hands. I did some testing of a web application a few months ago, but when I presented the results, I was very clear that they were guidelines only. I didn’t have the resources or ingenuity to replicate the behavior of real users on real clients. A few years ago, I was part of a project that ran aground on this same rock, costing the company I was working for plenty of money. Using software to imitate user behavior is hard. A short list of the differences between software and users:

1. Users get distracted–by popups, their kids, etc. Software, not so much.
2. Users are connecting via a variety of methods, with a wide range of quality levels–modems, broadband.
3. Users don’t use applications in the way developers intended. The testing software, on the other hand, is programmed by developers, who naturally have it use the application in the way they intended

The adaptive memory model for the next JDK (wow, now it’s J2SE 5–Sun pulled another Solaris reversioning trick) that the author outlines might make the “tweaking” portion of his hand waving, err, I mean tuning, easier but leaves the “testing” as difficult as ever.

Java | moore |