CU Talk: Supporting the Software Revolution

Last Thursday, I went to a talk (one of the CU CS Colloquia) about software and the problems it faces today at the University of Colorado called “Supporting the Software Revolution”. Amer Diwan gave a talk about some of his research and how it deals with modern software programs, which are becoming larger and larger, with the following ramifications:

they are
1. harder to write–more collaborative.
2. harder to understand
3. more resource intensive

He talked about some of his research in the educational sphere, where he was working against the traditional engineering bias against collaboration by training students to work in teams. Amer also mentioned his research into tools to help discover documentation to increase understanding of large programs. But the meat of his research, as well as the focus of this talk, was on technologies to improve performance, including hardware aware software and visualization.

Amer primarily discussed vertical profiling, which stated that because of the multilayered nature of todays applications (application on top of framework on top of virtual machine on top of hardware) it is not enough to simply profile the application and the hardware, since each level can interact with each other level in non intuitive ways.

The answer is vertical profiling, where you instrument each layer appropriately. (Some layers, such as the jikes JVM, are pre-instrumented.) Find a target metric, like instructions per cycle. Instrument different all the different metrics (for example, new object allocations is one thing could be instrumented for the virtual machine level). Then, align all these metrics with one common metric to combat nondeterministic behavior.

(This is where I get a bit fuzzy–I believe that they used some kind of algorithm to match up the instructions per cycle with other interesting metrics. He mentioned some kind of algorithm that had previously been used for speech recognition. Not sure how to align three variables [time, instructions per cycle, and one other interesting metric] on one chart.)

Then, after all the metrics have been aligned in time, look for interesting and disturbing patterns–this is where the humans come in and rank the graphs by similarity. Then see if one metric depends on another–you can then discard the dependent metric, since you’re looking for the root issue. After you think you have found the root issue, make a change to that, and profile the stack again. If the pattern of interest is gone, you are validated and have addressed the root issue–otherwise back to the drawing board.

This was interesting because of the alignment in time of different metrics (which is, unfortunately, the piece I understand and remember the least). Other than that, it was pretty much a well thought out and methodical explication of common knowledge of profiling. Change one thing at a time, look for dependencies, validate your suppositions, be aware of interactions between the layers of your application. It is nice to see someone trying to turn the black art of performance tuning a bit more into a science.

So, if you’re ever on a project that has the time and budget to deal with performance issues in a methodical manner, vertical profiling is worth a look. More here:

Performance explorer: understanding java application behavior

One of Diwan’s student’s research pages

InstallAnywhere Impressions

I helped write a java program a few months ago, a product designed to run on mail servers. Now, we had the program packaged up as a zip file (for windows) and a tarball (for unix). That wasn’t my decision, but it makes sense to me–if you are deploying a program on a mail server, you should know how to unzip files and edit configuration files.

But, that’s not what the client wanted. They came back recently with a few changes, and a desire to install via InstallAnywhere. I am no expert at InstallAnywhere, but the client didn’t have the engineering cycles to repackage the program, so they paid me to do it. What follows is my overall impression of InstallAnywhere, and a few tips and tricks.

Overall, I like InstallAnywhere. This program makes it easy to build java program installers for a variety of different platforms (most of the unices, Macs and Windows), execute sundry actions pre and post install, and grab user input while installing. It supports both GUI and console installation procedures. In particular, the Windows installer was a snap, and I didn’t have to learn the first thing about the registry–InstallAnywhere took care of that, even to the point of having the program show up on the ‘Add/Remove Programs’ Control Panel.

On the downside, there are a bevy of options and the help file wasn’t exactly the best. They have a free trial version, but it complains every time you install from a file built with the trial version; such installers stop working around 10 days after you build with the trial version as well–but the trial version doesn’t tell you about the future failure.

A few tips:

* It’s possible to keep the install configuration file in CVS, except for the fact that it hardcodes paths to resources that it includes in the install file. I was able to work around that by using ant’s replace task.

* When you start up the program (on unix), you can’t kill it normally, via either cntrl-c or backgrounding it and running the kill command on the process. I believe this is because the default behavior of a launcher is to listen to the console for stdin. You can change this easily enough, but the default really should be correct.

* The installer builder doesn’t default to generating a log, even though many of the default log messages point you to the install log file. You have to click a checkbox on the Project Info Pane in the Advance Installer.

* The console installer insisted that there were errors in the installation process even though the program, post install, worked fine and there were no errors written in the installer log. I think this is due to the fact that I’m using the trial version, but am not sure.

* There doesn’t seem to be any way in the InstallAnywhere GUI to specify that if the DISPLAY variable is set (on unix systems), the GUI installer should run, otherwise the console installer should run. If you want, you can edit the generated install.bin installer script–search for ‘FAILSAFE’ and use a modern editor capable of long lines–but I couldn’t figure out a way to automate this change. This is my biggest gripe, since this is a very typical demand. If you don’t run install.bin -i console to specify a console installation, you get a lovely exception:

Stack Trace:
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
        at java.awt.GraphicsEnvironment.checkHeadless(
        at java.awt.Window.(
        at java.awt.Frame.(
        at java.awt.Frame.(
        at javax.swing.JFrame.(
        at com.zerog.ia.installer.LifeCycleManager.f(DashoA8113)
        at com.zerog.ia.installer.LifeCycleManager.g(DashoA8113)
        at com.zerog.ia.installer.LifeCycleManager.a(DashoA8113)
        at com.zerog.ia.installer.Main.main(DashoA8113)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
        at java.lang.reflect.Method.invoke(
        at com.zerog.lax.LAX.launch(DashoA8113)
        at com.zerog.lax.LAX.main(DashoA8113)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

Overall, I’d say if you’re delivering a java based product to typical users, InstallAnywhere is a good choice. I don’t know how much it costs, but the experience for the users is as seamless as you want it to be, the width and breadth of installer customization is impressive, and it didn’t take too long to get up to speed.

Yahoo IM and VOIP

I just installed the latest version of Yahoo Messenger. That and a 20 dollar headset lets me make free phone calls to anyone else on my messenger list (though they have to have the correct version of the software and a headset as well–and Yahoo reserves the right to start charging for the service.)

I may be a bit behind the wagon, since eBay just bought Skype and I have a friend who has been using vonage for over two years, but I was quite impressed witht he ease of install and the sound quality. I’ll be interested to see where Yahoo takes this. They already allow you to make calls to external phone numbers, though you do have to pay 2 cents a minute.

Why is this any different than Skype or Vonage? Because:
1) it leverages the investment most folks have in their IM address books
2) when you’re on IM, you sometimes want to have phone conversations to supplement the quick questions that IM is so good at conveying
3) yahoo is a fairly well known name (as is google, who is also getting into the business of VOIP). I don’t have any numbers on relative bases, but from personal experience, the number of folks who IM is greater than the number of folks who use VOIP.

Remember the days when there was such a business as carrying long distance voice traffic? There are rapidly coming to a close, driven by VOIP and cellular phone usage, I think.

Update: The Economist has an article on Skype and VOIP in general.

© Moore Consulting, 2003-2017 +