All posts by moore - 92. page

The Free Lunch Is Over, via Random Thoughts, is a fascinating look at where CPUs are headed, and what effect that has on software development. The subtitle: “The biggest sea change in software development since the OO revolution is knocking at the door, and its name is Concurrency” drives home the fact that the author believes that concurrency will be the next big thing in software development.

I was struggling to write a relevant post about this topic, becuase I feel like, at least in the companies I’ve been with, there just wasn’t that much object oriented software being written. I’m working on a project right now that has a minimum of object orientation, even though it is written in java. I’m definitely more familiar with small scale projects and web applications, but I know there are plenty of applications out there that are written and working well without the benefits of objects.

Or, should I say, that are written and working well without the benefits of objects directly. Servers, operating systems and general purpose platforms are a different beast and require a different skill set. And by building on top of such platforms, normal programmers don’t have to understand the intricacies of object oriented development–they can benefit without that investment. Of course, they’d probably benefit more if they understood things and there may come a time in their development that they’ll have to. However, the short term gain of being able to continue on their productive plateau may be worth postponing the learning process (which will take them to a higher plateau at a short term cost).

In the same way, I think that multi-threading won’t be required of normal busines developers. I was struggling with this until the latest NTK came out, with this to say:

CPUs aren’t getting faster! Multi-core is the future! Which means we’ll all need to learn concurrent, multi-threaded programming, or else our software is never going to get faster again! That’s what Herb Sutter’s future shock article in Dr. Dobbs says (below). But before you start re-learning APL, here’s a daring thought: maybe programmers are just too *stupid* to write multi-threaded software (not you of course: that guy behind you). And maybe instead we’ll see more *background* processes springing up – filling our spare CPUs with their own weird, low i/o calculations. Guessing wildly, we think background – or remote – processes are going to be the new foreground.

From the Jan 21 edition, which should be online in a day or so. Those Brits certainly have a way with words.

If you’re a typical programmer, let the brilliant programmers who are responsible for operating systems, virtual machines and application servers figure out how to best use the new speed of concurrent processor execution, and focus on process and understanding business needsand making sure they’re met by your software. Or, if you have a need for speed, look at precalculation rather than multi threading.

If you’re ever pulling a url from a database via an Expresso dbobject (Expresso’s O-R layer) and you find yourself with mysterious & characters being inserted, you may want to visit this thread and the FilterManager javadoc. Long story short, add this line:

setStringFilter("fieldname", FilterManager.RAW_FILTER);

to any fields of the dbobject that you don’t want ‘made safe’ by the default filter (which screens out dangerous HTML characters). Tested on Expresso 5.5.

(I’m omitting the rant about changing data pulled from the database without making it loud and clear that default behavior is to filter certain characters. But it’s a Bad Idea.)

I’ve written about PL/SQL before but recently have spent a significant amount of time writing stored procedures. Unlike some of my previous experiences, this time PL/SQL seemed like a great fit for the problem set, which was two fold.

In the first case, some of the stored procedures push data from stage tables, which are loaded via ODBC or SQL*Loader, into tables which the application accesses. PL/SQL is great for this type of task because cursors, especially when used with parameters, make row driven data transformations a pleasure, and fast as well. Handling deltas via updates instead of inserts was alright, and the fact is that PL/SQL code that manipulates data can be positively terse when compared to JDBC PreparedStatements and at least as fast. In addition, these stored procedures can be easily called over an ODBC connection, giving the client the capability to load new data to the stage tables and then call the stored procedure to update or insert the data as needed. (You could definitely do the same thing with a servlet and have the client hit a URL, but that’s a bit less self-contained.)

PL/SQL was also used to implement complex logic that was likely to change. Why do that in PL/SQL in the database rather than in java in the application server? Well, changes to PL/SQL programs don’t require a server restart, which can be quite an issue when a server needs high levels of uptime. Instead, you just recompile the PL/SQL. Sure, you can use the reloadable attribute of the context to achieve the same thing (if you’re using Tomcat) but recompiling PL/SQL doesn’t have the same performance hit as monitoring class files for changes.

Use the right tool for the job. Even if PL/SQL ties your application to Oracle, a judicious use of this language can have significant benefits.

In almost every software project of any length that I’ve participated in, the last few weeks before a release are tense and pressure filled. (Please note that I write custom business software; that’s what these conclusions are drawn from.) Being in the middle of a project release myself, I thought I’d muse on the causes of this pressure. Why are the last few weeks before the deadline so tense? Because software is, above all else, about the details. Joel puts it well in his interview with Salon.com:

The fundamental problem that you’re trying to solve here is that humans think of things in vague, mushy terms. In order to visualize something, they don’t have to actually visualize every part of it. Whereas the programmer, in order to actually implement that thing, to create it, needs to have every part specified.

What happens on projects over a certain level of complexity is this specification is pushed off, often until a decision must be made, or even past that point. This occurs for a number of reasons: programmers want to start coding, the client doesn’t have the information at the moment the issue is raised and it is never revisited, the answers to certain questions (or the questions themselves) are dependent on answers to other questions. In the beginning of a project, big questions are decided, but the small niggling details, which the compiler most certainly needs to know about, are, perhaps noted, but not dealt with.

Why not specify how the system will work before building any of it, to every exacting detail? Some software processes try to do this, but in general, unless the problem is very well understood (in which case the client will almost always be better served by off-the-shelf software), the requirements will change as the project progresses. (Incidentally, if they don’t, the project is a great candidate for offshoring.) The client will better understand the problem and technology and the software team will likewise better understand the problem and domain space. So specifying the entire system up front will likely leave the customers unhappy or the system unused.

Because business software is actually business process crystallization, it matters very much that things are correct. Because business software is implemented by a group of people with specialized skills and a different focus from the users, at best, or no understanding of the business, at worst, software delivery is unlike other deadline driven industries in that changes are expensive and mysterious. I think every software engineer has an example of a simple change request that turned out to have massive implications throughout the system, and this effect is mysterious to normal users.

What matters is not why the details crop up, but that they do. So, the last few weeks of every project consists of mentally running around and nailing down every detail. I expect this is true of every job with fixed deadlines (ever been around a retail store the day before Thanksgiving?). Every issue should be resolved or acknowledged when the software is released, and while some facets are less important than others, no detail is unimportant.