Last Thursday, I went to a talk (one of the CU CS Colloquia) about software and the problems it faces today at the University of Colorado called “Supporting the Software Revolution”. Amer Diwan gave a talk about some of his research and how it deals with modern software programs, which are becoming larger and larger, with the following ramifications:
1. harder to write–more collaborative.
2. harder to understand
3. more resource intensive
He talked about some of his research in the educational sphere, where he was working against the traditional engineering bias against collaboration by training students to work in teams. Amer also mentioned his research into tools to help discover documentation to increase understanding of large programs. But the meat of his research, as well as the focus of this talk, was on technologies to improve performance, including hardware aware software and visualization.
Amer primarily discussed vertical profiling, which stated that because of the multilayered nature of todays applications (application on top of framework on top of virtual machine on top of hardware) it is not enough to simply profile the application and the hardware, since each level can interact with each other level in non intuitive ways.
The answer is vertical profiling, where you instrument each layer appropriately. (Some layers, such as the jikes JVM, are pre-instrumented.) Find a target metric, like instructions per cycle. Instrument different all the different metrics (for example, new object allocations is one thing could be instrumented for the virtual machine level). Then, align all these metrics with one common metric to combat nondeterministic behavior.
(This is where I get a bit fuzzy–I believe that they used some kind of algorithm to match up the instructions per cycle with other interesting metrics. He mentioned some kind of algorithm that had previously been used for speech recognition. Not sure how to align three variables [time, instructions per cycle, and one other interesting metric] on one chart.)
Then, after all the metrics have been aligned in time, look for interesting and disturbing patterns–this is where the humans come in and rank the graphs by similarity. Then see if one metric depends on another–you can then discard the dependent metric, since you’re looking for the root issue. After you think you have found the root issue, make a change to that, and profile the stack again. If the pattern of interest is gone, you are validated and have addressed the root issue–otherwise back to the drawing board.
This was interesting because of the alignment in time of different metrics (which is, unfortunately, the piece I understand and remember the least). Other than that, it was pretty much a well thought out and methodical explication of common knowledge of profiling. Change one thing at a time, look for dependencies, validate your suppositions, be aware of interactions between the layers of your application. It is nice to see someone trying to turn the black art of performance tuning a bit more into a science.
So, if you’re ever on a project that has the time and budget to deal with performance issues in a methodical manner, vertical profiling is worth a look. More here: