Joel is soliciting suggestions for best software essays of 2004. Very worthwhile browsing.
All posts by moore - 95. page
Problem solving via email composition
I’m currently struggling with an open source project, working to make the UI look like the one the designer showed the client. It’s a bit frustrating, because it’s sleuth work, and the clues are all in the code. Code I’ve not worked with, but with which I am rapidly gaining familiarity.
I cannot imagine how frustrating this would be if it were a closed source tool. Check that, I can. I worked on a project a few years ago with a closed source portal product, and it simply wouldn’t do what we asked it to do. Well, when you’re a small company, you do what you need to do in order to get the job done. This included some decompilation I’m sure voided our warranty, but delivered the client what was promised. It’s awful when you’re put in that position.
Regardless, I’ve been able to tweak this open source project to make it do what I need. When I find myself facing an issue that is not covered in the on line documentation, I’ve found that the best way to solve the problem is to start writing an email to the user list.
Now, because it’s written communication to a group of peers, some of whom are quite knowledgeable about the subject, I’m fairly careful about the content. I outline the problem carefully, explain the searches of the mailing list and web that I’ve already undertaken, and detail any avenues of solution that I’ve already explored. This rather rigorous analysis of the problem often leads to other glimmers of solutions, log files I have forgotten to check, and more hypotheses to prove or disprove. As I build the email, the problem becomes more and more manageable.
In my efforts to not appear a fool to peers, I reckon I have not sent about seventy percent of the emails I’ve composed–either I found another solution, I missed some crucial bit when before I started the email, or after doing due diligence on the problem I discovered the answer on the mailing list. It’s a bit weird, but crafting an excellent ‘can you help me’ email to a user list helps me even if the email isn’t sent.
Coding Standards
We’ve all been caught up in the religious war of coding standards. Whether it’s ‘tab vs spaces’ or ‘3 spaces vs 4 spaces’ or ‘curly brace on same line as if clause or curly brace on line below if clause’, these arguments can take up a significant amount of time on a project. Ken Arnold has an interesting post where he recommends that coding standards be integrated into the language specification, and enforced by the compiler. His reasons:
1. There’s no real productivity difference between one coding style and another.
2. There’s real productivity lost in setting up pretty printers, re-formatting code manually, and arguing about the better style.
3. If programmers have latitude, they’ll use it. (See perl.)
So, take away the freedom of programmers to format their code in a non standard manner. Make coding style part of the compiler–not as a warning though (as Bruce Eckel says “when you discover you can ignore them [warnings], you do”)–actually have the compiler error out if the formatting is incorrect.
Sure, I’d wail along with the rest of the coding community, for a while. But then I’d adjust, especially if the tools were there. And I’d probably be more productive. Talk about tough love.
Right tool, right job, part II
Here’s a nice explication of ‘choose the right tool for the job,’ by a pundit. The entire J2EE stack is often overkill, and I’ve seen shops that would have been much better served by a series of PHP applications, perhaps with java on the backend for heavier duty processing and legacy code reuse.
My only quarrel with his analysis is that he assumes we know how long an application is going to live. How many applications that started as prototypes get rolled to production and are the cause of endless headaches five years down the road?
As in any complex situation, when designing an application, and choosing how complicated to make its architecture, you need to balance between two principles:
1. It’s easier to do things right the first time. You’re in the code, you have an understanding of the requirements, there’s a green field, you have a budget–therefore making sure the application works is easier when you’re first writing it.
2. You aren’t gonna need it. This slogan, from the Agile folks, says that you have no way of knowing what you’re going to need five years in the future, so why try to design for that?
Taken too far, each principle is disastrous. If you follow principle number one fully, you’ll end up in analysis paralysis. If you take point two to heart, you’ll miss out on big chunks of functionality that have to be retro-fitted on later, to the detriment of a schedule and possibly the design.
That’s why they (architects, not pundits) get paid the big bucks.
How to evaluate an open source project
There are a fantastic number of open source projects out there, on SourceForge, apache, and elsewhere. This is fantastic because you can often leverage other work that folks have done, as well as knowledge and mistakes they’ve made. However, it can be extremely difficult to evaluate accurately how much such projects can help you, especially if you’ve not used them before, or if they are obscure. In addition, you probably don’t have a lot of time to choose a solution–clients that go with open source solutions tend to have budget constraints. I present here a quick checklist that I use to evaluate projects that I’m going to depend upon:
1. Know your drop dead features. These are features that the software package must have in order to be considered. Be careful not to make this too long. The primary purpose of this list is to allow you to quickly exclude packages that you know won’t work for you, and if you make it too long, you might be left with no options.
2. Look at the documentation attached to the project. This is the first place to start ruling a project out–if it doesn’t promise the features you need, move on. Also, look at a demo or screen shots, if possible. This lets you see how the package works. Compare behavior with the list of needed features.
3. Install the software. If you have difficulty installing it, that’s not an insurmountable issue–often open source projects aren’t the smoothest installations. However, installing it and spending a few hours playing around with this software that is going to be a significant part of your project can let you know if the impressions you received from the demo and documentation are correct–is it going to be easy enough to tweak/deploy/extend this software package?
4. In the world of open source support, the mailing list is king. Does this project have a mailing list? Is it archived? Is it googled? How active is it? If there’s no mailing list, is there a set of forums? The mailing list (or forum) is where you’re going to go when you have a smart question to ask, and you will, so you want this resource to be as strong as possible.
5. Look at the documentation again. The first time you did so, you were just looking to exclude a project based on feature set. This time, you want to see how much the documentation can help you. Is there a tutorial? Are the advanced topics that concern you covered? For java projects, is there javadoc? Is it more than just the methods and arguments that are automatically generated? What version of the software is the documentation for?
Of course, the more I depend on a piece of software, the more time I want to spend on evaluation. On the other hand, the process laid out above is entirely technical in nature, and, as we know, there may be other reasons for choosing, or not choosing, software. Installed base, existing team experience and knowledge, project timeline, or the fact that the CEO’s brother is on the board of a company with a rival offering all can influence software package choice. There are many factors to be considered, but this list is how I start off.
Expresso authentication and authorization
I’ve only briefly worked with Expresso. But I’ve heard good things about it. However, one ‘feature’ is really chapping my hide at the moment. Apparently, the only way to authenticate someone is to call the attemptLogin method on a ‘Controller’ object (a subclass of a Struts Action), which is protected and takes, among other things, the http request and response. There’s no way I can find to just pass in a username/password and authenticate. In addition, the authorization system is not broken out either. In OO fashion, you ask an object if a user can access it, and the object knows enough to reply.
I’m not trying to rag on the Expresso developers. After all, they are giving away a fine, full featured java web framework for free. But this just drove home to me how important it is in web applications to have the classes that talk http be nothing more than a thin translating layer around business classes. For instance, all a struts action should do is convert http forms to domain specific value objects, and then call business methods on business objects.
If this was the case in Expresso, it’d be trivial for me to leverage Expresso’s existing authentication model–I’d just have to fall the methods on the business object, perhaps after creating a domain specific value object. Now, however, I’ll probably have to monkey around with the http request and response, and decode exactly what parameters it wants, and fake those up.
Open source portal search
I’ve been looking at some open source portals. My client has an existing java application, written in Expresso that has some reasonably complex logic embedded in it. Additionally, it’s massively internationalized, with dynamic international content coming from a database, and static content coming from a set of resource bundles. There’s an existing process around updating both of these sets of data. And when we’re talking internationalization, we’re talking Asian character sets as well as the European character sets.
So, the criteria for the portal were:
1. Support for multi-byte character sets and easy localization.
2. Ability to integrate with Expresso’s authentication and authorization systems.
3. Support for normal portal features–adding/moving/removing portlets, minimize/maximize portlets.
4. Documentation.
I looked at a fair number of portals, including jcorporate’s own ePortal, eXo, Liferay, Jetspeed 1, Jetspeed 2, and Pluto (a last alternative, to be avoided if possible, is to roll our own portal-like application). First, I looked at ePortal, but that’s a dead project, with no releases. Then, I installed pluto, which seemed like it would be a perfect fit to be integrated into Expresso. However, integrating pluto looked complex, and after installing it (fantastic instructions for installing pluto here), I realized that pluto did not have a layout manager that would allow for the addition, rearranging or moving of portlets.
I then battled with Jetspeed 2, which involved installing a subversion client and building from source. This looked to be pretty cool, but the sheer lack of documentation, and the fact that there have been no releases, caused me to shy off. This is no failure of Jetspeed 2–this is what projects in development are like; I think it will be a fine project when done but my client just doesn’t need to be on the bleeding edge. I also took a quick look at Liferay, which seems to be a much more full featured portal application than we needed. After reading this blog on portals I decided to take a closer look at eXo. However, the documentation wasn’t fantastic, and it wasn’t readily apparent how to plug in authentication.
I also downloaded and installed Jetspeed 1; if you download the src distribution, you get the helpful tutorial. While Jetspeed 1 is not a standards based solution (I expect most of the portlets will be custom developed anyway), the user community is fairly active, as indicated by the mailing list, and I’ve found the documentation to be extensive. In addition, it meets the localization requirements and the pluggable authentication and authorization systems.
I’m less than thrilled about having to use maven for builds. Others have said it better than I, but it’s just too much for my needs. However, I was able to get an independent directory tree for my project by copying over the maven.xml, project.properties, and project.xml from the tutorial directory to an empty directory. Then I tweaked the project.* files, ran maven jetspeed:genapp, tweaked a few settings in TubineResources.properties to make sure the localization settings were correct, and, voila, I have a working project tree, that, using the Jetspeed maven plugin, is one command away from a deployable war file.
Open Books at O’Reilly
I’m always on the lookout for interesting content on the internet. I just stumbled across Free as in Freedom, an account of Richard Stallman, which is published under the umbrella of the Open Books Project.
Relearning the joys of DocBook
I remember the first time I looked at Simple DocBook. I have always enjoyed compiling my writing–I wrote my senior thesis using LaTeX. When I found DocBook, I was hooked–it was easier to use and understand than any of the TeX derivatives, and the Simplified grammar had just what I needed for technical documentation. I used it to write my JAAS article.
But, I remember it being a huge hassle to set up. You had to download openjade, compile it on some systems, set up some environment variables, point to certain configuration files and in general do quite a bit of fiddling. I grew so exasperated that I didn’t even setup the XML to PDF conversion, just the XML to HTML.
Well, I went back a few weeks ago, and found things had improved greatly. With the help of this document explaining how to set DocBook up on Windows (updated 12/2/2006 to fix a broken link) I was able to generate PDF and HTML files quickly. In fact, with the DocBook XSL transformations and the power of FOP, turning a Simplified DocBook article into a snazzy looking PDF file is as simple as this (stolen from here):
java -cp "C:\Programs\java\fop.jar; \
C:\Programs\java\batik.jar;C:\Programs\java\jimi-1.0.jar; \
C:\Programs\java\xalan.jar; C:\Programs\java\xerces.jar; \
C:\Programs\java\logkit-1.0b4.jar;C:\Programs\java\avalon-framework-4.0.jar" \org.apache.fop.apps.Fop -xsl \ "C:\user\default\xml\stylesheets\docbook-xsl-1.45\fo\docbook.xsl" \ -xml test.xml -pdf test.pdf
Wrap that up in a shell script, and you have a javac for dcuments.
Abstractions, Climbing and Coding
I vividly remember a conversation I had in the late 1990s with a friend in college. He was an old school traditional rock climber; he was born and raised in Grand Teton National Park. We were discussing technology and the changes it wreaks on activities, particularly climbing. He was talking about sport climbing. (For those of you not in the know, there are several different types of outdoor rock climbing. The two I’ll be referring to today are sport climbing and traditional, or trad, climbing. Sport climbers clip existing protection to ensure their safety; traditional climbers insert their own protection gear into cracks.) He was not bagging on sport climbing, but was explaining to me how it opened up the sport of climbing. A rock climber did not need to spend as much money acquiring equipment nor as much time learning to use protection safely. Instead, with sport climbing, one could focus on the act of climbing.
At that moment it struck me that what he was saying was applicable to HTML generation tools (among many, many other things). During that period, I was just becoming aware of some of the WYSIWYG tools available for generating HTML (remember, in the late 1990s, the web was still gaining momentum; I’m not even sure MS Word had ‘Save As HTML’ until Word 97). Just like trad versus sport, there was an obvious trade off to be made between hand coding HTML and using a tool to generate it. The tool saved the user time, but acted as an abstraction layer, clouding the user’s understanding of what was actually happening. In other words, when I coded HTML from hand, I understood everything that was going on. On the other hand, when I used a tool, I was able to make snazzier pages, but didn’t understand what was happening. Let’s just repeat thatI was able to do something and have it work, all without understanding why it worked! How powerful is that?
This trend, towards making complicated things easier happens all the time. After all, the first cars were difficult to start, requiring hand cranking, but now I just get in the car and turn the key. This abstraction process is well and good, as long as we realize it is happening and are willing to accept the costs. For there are costs, in climbing, but also in software. Joel has something to say on this topic. I saw an example of this cost myself a few months ago, when Tomcat was not behaving as I expected, and I had to work around an abstraction that had failed. I also saw a benefit to this process of abstraction when I was right out of school. In 1999, there was not the body of frameworks and best practices that currently exist. There was a lot of invention from scratch. I saw a shopping cart get built, and wrote a user authentication and authorization system myself. These were good experiences, and it was much easier to support this software, since it was understood from the ground up by the authors. But, it was hugely expensive as well.
In climbing terms, I saw this trade off recently when I took a friend (a much better climber than I) trad climbing. She led a pitch far below her climbing level, and yet was twigged out by the need to place her own protection. I imagine that’s exactly how I would feel were I required to fix my brakes or debug a compiler. Dropping down to a lower abstraction takes energy, time, and sometimes money. Since you only have a finite amount of time, you need to decide at what abstraction level you want to sit. Of course, this varies depending the context; when you’re working, the abstraction level of Visual Basic may be just fine, because you just need to get this small application written (though you shouldn’t expect such an application to scale to multiple users). When you’re climbing, you may decide that you need to dig down to the trad level of abstraction in order to go the places you want to go.
I recently read an interview with Richard Rossiter, who has written some of the canonical guidebooks for front range area climbing. When asked where he thought “climbing was going” Rossiter replied: “My guess is that rock climbing will go toward safety and predictability as more and more people get involved. In other words, sport climbing is here to stay and will only get bigger….” A wise prediction; analogous to my prediction that sometimes understanding the nuts and bolts of an application simply isn’t necessary. I sympathize. I wouldn’t have wanted to go climbing with hobnail boots and manila ropes, as they did in the old days; nor would I have wanted to have to write my own compiler, as many did in the 1960s. And, as my college friend pointed out, sport climbing does make climbing in general safer and more accessible; you don’t have to invest a ton of time learning how to fiddle with equipment that will save your life. At the same time, unless you are one of the few who places bolts, you are trusting someone else’s ability to place equipment that will save your life. Just like I’ve trusted DreamWeaver to create HTML that’s readable by browsersif it does not, and I don’t know HTML, I have few options.
Note, though, that it is silly for folks who sit at one level of abstraction to denigrate folks at another. After all, what is the real difference between someone using a compiler and someone using DreamWeaver? They’re both trying to get something done, using something that they probably don’t understand. (And if you understand compilers, do you understand chip design? How about photo-lithography? Quantum mechanics? Everyone uses things they don’t understand at some level.)
It is important, however, to realize that even if you are using a higher abstraction level, there’s a certain richness and joy that can’t be achieved unless you’re at the lower level. (The opposite is true as wellI’d hate to deal with strings instead of classes all the time; sport climbing frees me to enjoy movement on the rock.) Lower levels tend to be more complicated (that’s what abstraction doeshides complex ‘stuff’ behind a veneer of simplicity), so fewer folks enjoy the benefits of, say, trad climbing or compiler design. Again, depending on context, it may be well worth your while to dip down and see whether an activity like climbing or coding can be made more fulfilling by attacking it at a lower level. You’ll possibly learn a new skill, which, in the computer world can be a career helper, and in the climbing world may save your life at some time. You’ll also probably appreciate the higher level activities if and when you head back to that level, because you’ll have an understanding of the mental and temporal savings that the abstraction provides.
Follow me on Twitter
Follow me on Blue Sky