Skip to content

PL/SQL

I recently wrote a basic data transformation program using Java and PL/SQL. I hadn’t used PL/SQL (which is an Oracle-specific procedural language for stored procedures) since writing a basic data layer for my first professional project (a Yahoo! like application written in PL/SQL, perl and Story Server–don’t ask). Anyway, revisiting PL/SQL reminded me of some of the things I liked and disliked about that language.

I like:

Invalidation of dependencies. In PL/SQL, if package A (packages are simply arbitrary, hopefully logical, groups of procedures and functions) calls package B, A depends on B. If the signatures of B are recompiled (you can separate the signatures from the implementations) package A simply won’t run until you recompile it. This is something I really wish other languages would pick up, because it at least lets you know when something you depend on has changed out from under you.

I dislike:

The BEGIN and END blocks, which indicate boundaries for loops and if statements, are semantically no different than the { and } which I’ve grown to love in perl and Java. But for some reason, it takes me back to my pascal days and leaves a bit of a bad taste in my mouth.

I’m unsure of:

The idea of putting business logic in a database. Of course, schemas are intimately tied to the business layer (ask anyone trying to move to a different one) and anyone who pretends that switching databases in a java applications is a simple matter of changing a configuration file is smoking crack, but the putting chunks of business logic in the data layer introduces a few problems. Every different language that you use increases the complexity of a project–and to debug problems with the interface between them, you need to have someone who knows both. Also, stored procedures don’t fit very well into any of the object relational mapping tools and pretty much force you to use jdbc.

Oracle Programming | moore |

Death marchs and Don Quixote

I just finished watching ‘Lost In La Mancha’ which chronicles Terry Gilliam’s attempt to film his version of the story of Don Quixote, ‘The Man Who Killed Don Quixote’. (More reviews here.) The attempt failed, though there was a postscript that indicated that Gilliam was trying again. (An aside: not the best date movie.)

It was interesting to watch the perspective of the team start upbeat and slowly descend into despair. There were many reasons the filming failed, but what was most fascinating is that it was a death march project that just happened to take place in the sphere of film.

Of course there were certain ‘acts of God’ that contributed to the failure, but there always are difficulties beyond control. What’s more interesting to me is the disasters that could have been planned for. Read through some of the aspects of ‘Lost In La Mancha’ and see if you recognize any (plenty of spoilers, so don’t read if you want to watch the movie):

1. Gilliam tried to create a $60 million film on a $32.1 million dollar budget. He actually smiles while saying words to this effect!

2. Not all key players present during planning. In pre-production, none of the actors are able to schedule time to rehearse, partly because they all took pay cuts to make this movie (see point 1), partly because they were all busy.

3. Tight timelines. Due to money and scheduling, every day of filming was very carefully planned out; any problems on early days required changes to the entire schedule.

4. A visionary architect wasn’t willing to compromise. Gilliam is well known for his mind-blowing films (Twelve Monkeys, Brazil) and had been working on this movie in his mind for decades. This led to perfectionism, which, given the tight timelines and lack of money, wasn’t always the right use of resources. Addtitionally, Gilliam had a lackadaisical methodology: he mentions several times that his philosophy is ‘just shoot film and it will work out.’ That sounds freakishly similar to ‘just start coding and everything will be fine.’

5. Project history worked against success. This is one of the most interesting points–there were really two kinds of project history present. Film versions of ‘Don Quixote’ have a checkered past–Orson Welles tried for years to make a version, even continuing to film beyond his Don Quixote dying. And Gilliam has had at least one bomb–The Adventures of Baron Munchausen, a box office failure which haunted him for years. In both of these cases, there past actions cast a shadow over the present, affecting morale of the team.

6. When problems arose, the producers didn’t trust the technical staff (the directors). In particular, when weather struck, the directors wanted to allow the team to regroup, whereas the producers, because of points 1 and 3, wanted to film. Strife at the top never helps a project.

7. The equipment and setting was not optimal. Due to, I’m guessing, point 1, the outside scenes are set in a location next to a NATO air base, where jets will be flying overhead (‘only for an hour a day’ according to the first assistant director). The last sound stage in Madrid is reserved–it turns out to be a simple warehouse with awful acoustics.

And then there were some factors that simply were out of the blue. These included some bad weather and the illness of the actor playing Don Quixote. These were what pushed the film over the edge–but it wouldn’t have been on the edge if not for the other factors above. And you can also see that factors snowball on each other–timelines are tight because actors aren’t around; trust between team members is lost because of money and time issues.

It was like watching a train wreck in slow motion, but it was also illuminating to see that the lessons of project management not only are ignored in the software development, but also in film. Misery loves company.

Technology | moore |

Lessons from a data migration

I’ve been working on a data migration project for the last couple of months. There are two schemas, each used by a number of client applications implemented in a number of technologies, and I just wanted to share some of the lessons I’ve learned. Most of the clients are doing simple CRUD but there is some business logic going on as well. I’m sure most of these points will elicit ‘no-duhs’ from many of you.

1. Domain knowledge is crucial. There were many times where I made dumb mistakes because I didn’t understand how one column mapped to another, or how two tables were being consolidated. This would have been easier if I’d had an understanding of the problem space (networking at level 3 and below of the OSI burrito).

2. Parallel efforts end up wasting a lot of time, and doing things in the correct order is important. For instance, much of the client code was refactored before the data layer had settled down. Result? We had to revisit the client layer again. It was hard to split up the data layer work in any meaningful manner, because of the interdependencies of the various tables (thought doing this made more sense than updating client code). Multiple users working on DDL and DML in the same database leads to my next conclusion:

3. Multiple databases are required for effective parallel efforts. Seems like a no-brainer, but the maintenance nightmare of managing multiple developer databases often leads to developers sharing one database. This is workable on a project where most of the development is happening on top of a stable database schema, but when the schema and data are what is being changed, issues arise. Toes are stepped on.

4. Rippling changes through to clients presents you with a design choice. For large changes, like tables losing columns or being consolidated, you really don’t have a choice–you need to reflect those changes all the way through your application. But when it’s a small change, like the renaming of a column, you can either reflect that change in your value objects, or you can hide the changes, either in the DAO (misnamed properties) or database layer (views). The latter choice will lead to confusion down the line, but is less work. However, point #5 is an alternative to both:

5. Code generation a good idea in this case. Rather than having static objects that are maintained in version control, if the value objects and DAOs had some degree of flexibility in terms of looking at the database to determine their properties, adding, deleting and renaming columns would have been much much easier–freeing up more time to fix the GUI and business layer problems that such changes would cause.

Programming | moore |

Understanding the nuts and bolts

I remember back when EJBs first came out and there were all these tools bundled with the application server to build the XML deployment descriptors. Yet, the team I was on built a (perl) application which could generate those same descriptors. Why? Was it a case of ‘Not Invented Here’ syndrome? Someone with more time than sense? Well, perhaps, but it also ensured the team had a portable way of developing deployment descriptors and made sure that someone had a deep knowledge of said files.

Now, I feel the same way about web applications in general and JSF in particular. If you want to really understand the applications you create, you want to build them from the ground up. But, rather than regurgitate the arguments expounded so clearly in The Law of Leaky Abstractions and Beware Evil Wizards, I’d like to talk about where tools are good. This touches on some of the articles I’ve written before, including ease of programming.

Tools tend to be a fundamental parts of large systems that have a lot of people involved. Specialized knowledge (or lack of same) can lead to tools being built to help or insulate the users from certain grungy parts of a system–hence the EJB roles which split the deployer and programmer roles (among others) apart. That works fine with a large team.

But another interesting aspect of tools is the abstraction. Joel posulates that eventually the abstraction breaks down, and I’ve seen it happen. But, then again, I don’t expect to understand all of the socket handling that Tomcat does, or the TCP stack of the operating system on which Tomcat runs. I might have to delve into it if there are issues and it’s a high performance site, but in the normal course of events, that is simply not required. To link to another pundit, situations arise where such scalability just isn’t in the nature of the application. I’d also submit the tons and tons of VBA apps built on top of Outlook and the large complex spreadsheets build on Excel as examples of applications where software design, let alone a deep understanding of the fundamental building blocks of the language, is not required.

Sometimes, you just want to get the job done, and understanding the nuts and bolts isn’t necessary. In fact, it can be an inhibition. I was talking to an acquaintance today who used to code. When asked why he didn’t anymore, he pointed back to one factor–he wanted to be able to service the customer more quickly. At a higher level of abstraction, you can do that. You give up control, because the implementation of the service is usually in other hands (allowing you to go on to service another customer), because in the end, it all needs to be coded somehow. Tools, like Rave and Visual Studio.NET, make that trade off as well.

Programming | moore |

Denver No Fluff Just Stuff

Well, I just got done with two days of the Denver No Fluff Just Stuff conference. First off, unlike the previous NFJS conferences, this one wasn’t held in the DTC. You forget how far that is from Boulder, until you drive there and back twice on a weekend.

Anyway, I thought I’d share a few choice thoughts and tidbits regarding some of the sessions I attended. These are by no means full summaries of the talks.

Mock objects–Dave Thomas

Mock objects are objects that emulate behavior of external entities that make testing difficult. (I’ve worked with a few Englishmen in my life, and Dave Thomas had the same acerbic sense of humor.) Dave illustrated how to choose when to implement a mock object, as opposed to using the real object. He also touched on the true difficulty of mock objects, which is figuring out how to choose which object to use in your class (factory, pass the correct object into the constructor, AOP, class loaders).

JSF (both beginning and advanced)–David Geary

JSF is the new standard for web frameworks. David compared it to Swing and Struts meeting in a particle accelerator. Thompson’s fussed about tools for JSF, but I don’t think they’ll be needed for all JSF development, just like tools for Struts help, but aren’t required. I think that the most important bit about JSF is that it really tries to treat HTML widgets as full featured GUI components, which is something that is a bit of an adjustment for me. I’m really really used to thinking of HTML interfaces as generated strings, but this higher level approach (which has been used in the rich client world for a long time) is interesting.

There was an expert panel, consisting of several of the speakers. One hot topic was whether EJB 3.0 had been hijacked by Gavin King; everyone seemed to have an opinion on that. However, the choicest statement to emerge was Bruce Tate saying Java’s “type safety is an illusion” because everyone who uses a collection casts whenever they take anything out.

Herding Racehorses, Racing Sheep–Dave Thomas

This was a non-technical talk discussing how to improve programming as a profession. He referenced the Dreyfus Model of Skill Acquisition (novices learn differently from experts), and referenced Patricia Benner and her study of nurses in the 1970s, and how it was analgous to the current situation of developers. A great quote was “Training is how you give people knowledge; time is how you give people experience.” He also talked about how to move up the skill ladder, and how that will make it more difficult to outsource. However, he didn’t talk about how the relative dearth of novices would create a future shortage of experts, other than to acknowledge that everyone, anywhere, can move up the skill ladder and we need to prepare for that. Prepare by having a plan; this makes sense, as what you’re really doing is choosing where to invest your most precious commodity–your time.

TDD in the web tier–Rick Hightower

Rick covered the basics of Test Driven Development, and seemed a bit surprised that everyone wasn’t practicing it; he said it’s helped his code quite a bit. He went over a few tools that make testing (not just unit testing) easier today. A key point seemed to be the differentiation between TDD and Continuous Integration; tests that run for TDD need to be fast, since you’re running them multiple times a day, whereas CI tests can be slower. He also made the offhand comment that you could have JMeter proxy requests from a QA tester (in a web browser) and use Canoo (a JSP testing tool) to automate those tests. Wouldn’t that be cool?–cheaper than LoadRunner, that’s for sure.

Another expert panel. Someone asked “what are you folks going to be looking at in the next 6 months” and I was struck by the lack of diversity in the responses. Groovy, Hibernate, Tapestry came up again and again. Where do the new ideas come from? And where does deep knowledge come from, if everyone is running to a new cool tool every 6-12 months?

An offhand comment that someone made when we were talking abouty why so many apps had extraneous EJBs: “Yup, that was design by resume.”

Appfuse–Rick Hightower

Appfuse is a way to kick start your Struts applications. It provides a large chunk of best practices all in one place, along with a few pages that everyone needs (user creation, user authentication). Its license is liberal enough that you can use the code in your own project. I was struck by how many times Rick mentioned ripping stuff out, but I’m sure that I would learn quite a bit by poking around it. It was also clear to me that AppFuse is great for staring new applications, but I’m not sure it’s a good thing (other than a learning tool) for retrofitting best practices to existing applications. Also, Rick mentioned multiple times that he wouldn’t use Struts for a new application; given that AppFuse is primarily a Struts starter kit, I was a bit confused by this statement.

GIS–Scott Davis

This was a 1,000 foot overview of (primarily java) GIS applications. There are quite a few tools out there for displaying GIS data, which has several standardized formats (both those formally blessed by a standards organization, and those informal standards that grow out of network effects). There aren’t a collection of open source data sets, but you can get a ton of GIS data from government websites. The satellite that Scott’s company owns takes photos that are 15GB of data, and takes 500 such photos a day. Talk about storage needs. Also, anyone who wants to find out a bit more about satellite imaging would do well to read “Private eyes in the sky”, an article from the May 4th 2000 edition of the Economist, which is a good overview of the business.

Again, apologies for the jerky nature of my comments above. (Hey, at least I’m not talking about tugging any unmentionables.) Hangovers are not conducive to good note taking, but even if I had been rested, I still couldn’t do justice to 90 minutes of expert dialog in a paragraph on my blog. But it’s well worth going to one of these conferences, especially if you’re doing java web development.

Java | moore |

Social issues of online gaming

Via the Mobile Community Design weblog comes an interesting presentation on some of the social issues for online gaming (unfortunately, the slide show is IE only). There’s a basic overview of some of graph theory, and heavy emphasis on human social networks as graphs, and how you can exploit and support said networks for your game.

Some fascinating slides in the presentation, chock full of information: “In 1974 GranovetterÂ’s ‘Getting a Job’ found that you get most jobs from weak ties, not strong ones, because weak ties inhabit other clusters and therefore have different information”, the relative size of US cities have been constant since the 1900s, and the actual degrees of separation, from a 1967 experiment, is 5.5, not 6.

I wish I could have gone to the presentation, since I agree with Mike Clark: “bulletware isn’t the content of [the] presentation”, and I’m sure the speaker had plenty of interesting explication of his slides. If nothing else, though, the five pages of bibliography should provide plenty of future reading material.

(Also, check out the Internet timeline for images of the Internet’s growth and more neat graphs.)

Technology and Society | moore |

vi keybindings for Word

Well, someone’s finally done it. William Tan has put together a set of vi key bindings for Microsoft Word. (Thanks for the pointer, NTK!) I just downloaded and installed it, and thought I’d mention a few things.

1. The author mentions the instability (“alpha” nature) of the code. I haven’t run it long, but I get quite a few “Error 5346” and “Error 4198” messages. I’m no VB expert (nor even a newbie) so I have no idea what those mean. It didn’t seem to affect the document I was editing.

2. Installing the .dot file exposed some weirdness. The default location where you’re supposed to put these files (on WinXP, with Word 2003) is c:\Documents And Settings\Username\Application Data\Microsoft\Word\Startup\. Both the Application Data and Microsoft directories in the above path were hidden from Windows Explorer and the dir command in the shell, but you can cd to them.

The easiest way to install the .dot file is to open up Word, navigate via menus: Tools / Options / File Locations / Startup. Click the modify button, which brings up a file dialog box. Then drag the .dot file to that dialog box.

All in all, I’m glad someone has done this. Now, if only they’d do it for an IDE editor. Errm, I mean a free IDE–I know Visual Slickedit has a killer vi emulation mode. Yes, I know about Vimulator for jEdit, but the author’s language (“This plugin is in the early stages of implementation and does not yet provide a consistent or reliable VI-style interface.”), along with the fact it was last released in 2002, scared me away. Actually, it looks like there is one available for Eclipse: viPlugin.

Regardless, a very cool hack. Thanks, William.

Technology | moore |

Book Review: The Social Life of Information

I just finished reading The Social Life of Information, by John Seeley Brown and Paul Duguid. This was not the quickest read; it’s a business book with the obtuseness of vocabulary that implies. However, if you’re a computer person with any desire to see your work in a larger context, this is a book you should read. In it, they examine eight separate areas in which computers, and the internet in particular, have supposedly changed our lives (this is typically called ‘hype’, though the authors don’t use the word) in the latter years of the 20th century. (This book is copyright 2000.) You probably remember some of these claims: the death of the corporation, of the university, of paper documents, of the corporate office. In each chapter, they review one claim, show how the claim’s proponents over-simplify the issue, and look at the (new and old) responses of people and institutions to the problem that the claim was trying to solve. They also examine, in detail, the ways in which humans process information, and how the software that is often touted as a replacement simply isn’t.

I really enjoy ‘ah-ha’ moments; these are times where I look back at my experiences in a new light, thanks to a theory that justifies or explains something that I didn’t understand. For example, I remember when I started my first professional job, right out of college, I thought the whole point of work was to, well, work. So I sat in my cube and worked 8 solid hours a day. After a few months, when I still didn’t know anyone at the office, but had to ask someone how to modify a script I was working on, I learned the value of social interaction at the office. (Actually, I was so clueless, I had to ask someone to find the appropriate someone to ask.) While examining the concept of the home office, the authors state “[t]he office social system plays a major part in keeping tools (and people) up and running.” It’s not just work that happens at the office–there’s collaboration and informal learning.

I’ve worked remotely in the past year for the first time, and anyone who’s worked remotely has experienced a moment of frustration when trying to explain something and wished they were just “there,” to show rather than tell–the authors refer to this process as ‘huddling.’ When someone is changing a software configuration that I’m not intimately familiar, it’s much easier to judge correct options and settings if I’m there. The authors explain that “[huddling] is often a way of getting things done through collaboration. At home with frail and fickle technologies and unlimited configurations, people paradoxically may need to huddle even more, but can’t.” This collaboration is even more important between peers.

Reading about the home office and its lack of informal networks (which do occur around the corporate office) really drove home the social nature of work. After a few years at my company, I had cross-departmental relationships (often struck up over beer Friday) that truly eased some of my pain. Often, knowing who to ask a question is more important than knowing the answer to the question. It’s not impossible to build those relationships when you’re working remotely, but it’s much more difficult.

Another enjoyable moment of clarity arose when the authors discussed the nature of documents. I think of a document as a Word file, or perhaps a set of printed out pages. The explicit information (words, diagrams, etc) that I can get from the document is the focus (and this is certainly the case in document management systems sales pitches). But there’s a lot more to a document. How do I know how much to trust the information? Well, if it’s on a website somewhere, that’s a fair bit sketchier than if it’s in the newspaper, which is in turn less trustworthy than if I’ve experienced the information myself. Documents validate information–we’ve all picked up a book, hefted it, examined it, and judged it based on its cover. The authors say “readers look beyond the information in documents. … The investment evident in a document’s material content is often a good indicator of the investment in its informational content.” Just as if someone says “trust me” you should probably run the other way, information alone can’t attest to its own veracity. The authors also look at aspects to documents (like history, like feel, like layout) that simply aren’t captured when you treat them as streams of bits.

And there are many other examples of ‘hype’ that are deflated in this book, and a few other ‘ah-ha’ moments as well. As I stated above, this is a great read for anyone who thinks there is a technical answer to any problem (or even most problems). By taking apart various claims, and examining the truth and untruth of those claims in a real world context, these two authors give technology credit where it’s due, while at the same time explaining why some of the older institutions and important factors in our lives will remain around. Reading this book was hard work, but understanding what the authors say gives me yet another way to relate to non-technical people, as well as fend off the zealots who claim, in a knee-jerk fashion, that more software solves problems. I majored in physics, in college, but minored in politics. It always seemed that the people problems, though more squishy, were more interesting. This book is confirmation of that fact.

Books Technology and Society | moore |

People and automation

I read this article with interest. I’ve noticed the creep of automated services in the last ten years. Who goes into gas stations any more, unless you need a candy bar. Given the fact that these machines are a fixed cost investment (as opposed to an ongoing expense, like labor), I expect to see more and more of these. When reading this article, what every employee has to ask themselves is ‘Am I an elevator attendant or a bank teller?’.

I remember reading a story in Analog, years ago, about a general purpose robot and the societal dysfunction it caused. These robots could do everything a human being could, but 24 hours a day, rather than 8. Of course, this caused riots among workers, afraid of their jobs being turned over to the robots. Luckily, workers’ organizations and employers were able to come to an compromise–businesses couldn’t own these robots, only people could. Businesses would rent them from individuals, who would thus be able to earn a living.

That’s science fiction for you: both the problems and solutions are outlined in black and white. What we see nowadays is greyer–more and more ATMs are installed, yet tellers are being hired. Robots aren’t general purpose (and humanoid)–they’re slipping into the mainstream industry by industry. People aren’t rioting in protest of robots–they’re traveling extra distance to use them.

But the issues raised are still the same. Every machine that replaces a person (or two and one half people) causes a very real impact on the bottom line of the employee. At the same time, if a business can cut its labor costs, it will need to do so (especially if its competitors are also heading down the automation path). These differences revisit the old labor vs. capital divide (wouldn’t Marx and Engels be proud?), and the answers aren’t simple (or completely known, for that matter).

(The same issues arise in offshoring, and Bob Lewis comments here (sorry, you have to register to read the article). He states that the labor and capital national economies have been coupled for a long time, but now are being decoupled. He doesn’t have any answers, either.)

Technology has been automating away jobs since the Industrial Revolution, if not before. Things have worked out fine in the past, but it hasn’t always been pleasant to live through.

I don’t see any substantive debate on the nature of labor disempowerment. Perhaps this is because “we’ve seen this before, and it has always worked out” or because it’s an uncomfortable issue (especially in an election year) or because “we don’t have any real leaders anymore” or because we’re all vegetated by the modern opiate of the masses? I don’t know whether labor will riot, but brushing the issue under the rug certainly isn’t going to help.

Technology and Society | moore |

What the heck is Flash good for?

Flash is a fairly pervasive rich client framework for web applications. Some folks have issues with it. I’ve seen plenty of examples of that; the Bonnaroo site is an example of how Flash . Some folks think it’s the future of the internet. I like it, when it’s used for good purpose, and I thought I’d share a few of my favorite flash applications:

1. Ishkur’s guide to electronic music has an annoying intro, but after that, it’s pure gold. Mapping the transitions and transformations of electronic music, complete with commentary and sample tracks, I can’t imagine a better way to get familiar with musical genres and while away some time.

2. They Rule is an application that explores the web of relationships among directors on boards of public companies. Using images, it’s much easier to see the interconnectedness of the boards.

3. A couple of short animated pieces: Teen Girl Squad follows the (amateurly drawn) exploits of, well, a set of four teenage girls, and a cute movie about love (originally from http://students.washington.edu/k1/bin/Ddautta_01_masK.swf).

Of course, these all beg the question: what is a rich client good for (other than cool movies)? When is it appropriate to use Flash (or ActiveX, or XUL) rather than plain old (D)HTML? I wish I knew the answer, but it seems to me that there are a couple of guidelines.

1. How complicated is the data? And how complicated is the representation of that data? The more complicated, the more you should lean towards rich clients. I can’t imagine the electronic guide to music being half as effective if it was done in html.

2. How savvy are your users? This cuts both ways–if the users aren’t savvy, then the browser may be a comfortable, familiar experience. However, sometimes rich clients can ‘act smarter’ and make for a better user experience.

3. How large is your userbase? The larger, the more you should tend towards a thin, pervasive client like the browser, since that will ease deployment issues.

I used to think Flash was unabatedly evil, but I’m now convinced that, in some cases, it really makes a lot of sense.

Technology | moore |