Skip to content

java memory management, oh my!

How much do you understand basic java? Every day I find some part of this language that I’m not aware of, or don’t understand. Some days it’s cool APIS (like JAI) but today it’s concurrency. Now, language managed memory is a feature that’s been present in the languages in which I’ve been programming since I started. I’ve looked at C and C++, but taking a job coding in those seems to me it’d be like a job with a long commute–both have obstacles keeping you from getting real work done. (I’m not alone in feeling this way.) But this thread of comments on Cameron Purdy’s blog drove home my ignorance. However, the commenters do point out several interesting articles (in particular, this article about double checked locking was useful and made my head hurt at the same time) to alleviate that. I took a class with Tom Cargill a few years back, which included his threading module, that helped a bit.

However, all these complexities are why servlets (and EJBs) are so powerful. As long as you’re careful to only use local variables, why, you shouldn’t have to worry about threading at all. That’s what you use the container for, right? And we all know that containers are bug free, right? And you’d never have to go back and find some isolated thread related defect that affected your code a maddeningly miniscule amount of time, right?

PL/SQL

I recently wrote a basic data transformation program using Java and PL/SQL. I hadn’t used PL/SQL (which is an Oracle-specific procedural language for stored procedures) since writing a basic data layer for my first professional project (a Yahoo! like application written in PL/SQL, perl and Story Server–don’t ask). Anyway, revisiting PL/SQL reminded me of some of the things I liked and disliked about that language.

I like:

Invalidation of dependencies. In PL/SQL, if package A (packages are simply arbitrary, hopefully logical, groups of procedures and functions) calls package B, A depends on B. If the signatures of B are recompiled (you can separate the signatures from the implementations) package A simply won’t run until you recompile it. This is something I really wish other languages would pick up, because it at least lets you know when something you depend on has changed out from under you.

I dislike:

The BEGIN and END blocks, which indicate boundaries for loops and if statements, are semantically no different than the { and } which I’ve grown to love in perl and Java. But for some reason, it takes me back to my pascal days and leaves a bit of a bad taste in my mouth.

I’m unsure of:

The idea of putting business logic in a database. Of course, schemas are intimately tied to the business layer (ask anyone trying to move to a different one) and anyone who pretends that switching databases in a java applications is a simple matter of changing a configuration file is smoking crack, but the putting chunks of business logic in the data layer introduces a few problems. Every different language that you use increases the complexity of a project–and to debug problems with the interface between them, you need to have someone who knows both. Also, stored procedures don’t fit very well into any of the object relational mapping tools and pretty much force you to use jdbc.

Lessons from a data migration

I’ve been working on a data migration project for the last couple of months. There are two schemas, each used by a number of client applications implemented in a number of technologies, and I just wanted to share some of the lessons I’ve learned. Most of the clients are doing simple CRUD but there is some business logic going on as well. I’m sure most of these points will elicit ‘no-duhs’ from many of you.

1. Domain knowledge is crucial. There were many times where I made dumb mistakes because I didn’t understand how one column mapped to another, or how two tables were being consolidated. This would have been easier if I’d had an understanding of the problem space (networking at level 3 and below of the OSI burrito).

2. Parallel efforts end up wasting a lot of time, and doing things in the correct order is important. For instance, much of the client code was refactored before the data layer had settled down. Result? We had to revisit the client layer again. It was hard to split up the data layer work in any meaningful manner, because of the interdependencies of the various tables (thought doing this made more sense than updating client code). Multiple users working on DDL and DML in the same database leads to my next conclusion:

3. Multiple databases are required for effective parallel efforts. Seems like a no-brainer, but the maintenance nightmare of managing multiple developer databases often leads to developers sharing one database. This is workable on a project where most of the development is happening on top of a stable database schema, but when the schema and data are what is being changed, issues arise. Toes are stepped on.

4. Rippling changes through to clients presents you with a design choice. For large changes, like tables losing columns or being consolidated, you really don’t have a choice–you need to reflect those changes all the way through your application. But when it’s a small change, like the renaming of a column, you can either reflect that change in your value objects, or you can hide the changes, either in the DAO (misnamed properties) or database layer (views). The latter choice will lead to confusion down the line, but is less work. However, point #5 is an alternative to both:

5. Code generation a good idea in this case. Rather than having static objects that are maintained in version control, if the value objects and DAOs had some degree of flexibility in terms of looking at the database to determine their properties, adding, deleting and renaming columns would have been much much easier–freeing up more time to fix the GUI and business layer problems that such changes would cause.

Understanding the nuts and bolts

I remember back when EJBs first came out and there were all these tools bundled with the application server to build the XML deployment descriptors. Yet, the team I was on built a (perl) application which could generate those same descriptors. Why? Was it a case of ‘Not Invented Here’ syndrome? Someone with more time than sense? Well, perhaps, but it also ensured the team had a portable way of developing deployment descriptors and made sure that someone had a deep knowledge of said files.

Now, I feel the same way about web applications in general and JSF in particular. If you want to really understand the applications you create, you want to build them from the ground up. But, rather than regurgitate the arguments expounded so clearly in The Law of Leaky Abstractions and Beware Evil Wizards, I’d like to talk about where tools are good. This touches on some of the articles I’ve written before, including ease of programming.

Tools tend to be a fundamental parts of large systems that have a lot of people involved. Specialized knowledge (or lack of same) can lead to tools being built to help or insulate the users from certain grungy parts of a system–hence the EJB roles which split the deployer and programmer roles (among others) apart. That works fine with a large team.

But another interesting aspect of tools is the abstraction. Joel posulates that eventually the abstraction breaks down, and I’ve seen it happen. But, then again, I don’t expect to understand all of the socket handling that Tomcat does, or the TCP stack of the operating system on which Tomcat runs. I might have to delve into it if there are issues and it’s a high performance site, but in the normal course of events, that is simply not required. To link to another pundit, situations arise where such scalability just isn’t in the nature of the application. I’d also submit the tons and tons of VBA apps built on top of Outlook and the large complex spreadsheets build on Excel as examples of applications where software design, let alone a deep understanding of the fundamental building blocks of the language, is not required.

Sometimes, you just want to get the job done, and understanding the nuts and bolts isn’t necessary. In fact, it can be an inhibition. I was talking to an acquaintance today who used to code. When asked why he didn’t anymore, he pointed back to one factor–he wanted to be able to service the customer more quickly. At a higher level of abstraction, you can do that. You give up control, because the implementation of the service is usually in other hands (allowing you to go on to service another customer), because in the end, it all needs to be coded somehow. Tools, like Rave and Visual Studio.NET, make that trade off as well.

Arrogance

Ah, the arrogance of software developers. (I’m a software developer myself, so I figure I have carte blanche to take aim at the foibles of my profession.) Why, just the other day, I reviewed a legal document, and pointed out several places where I thought it could be improved (wording, some incorrect references, and whatnot). Now, why do I think that I have any business looking over a legal document (a real lawyer will check it over too)? Well, why shouldn’t I? I think that most developers have a couple of the characteristics/behaviors listed below, and that these can lead to such arrogance.

1. Asking questions

Many developers have no fear, even take pride, in asking good, difficult questions about technical topics. Asking such questions can become a habit. A developer may ask a question, and feel comfortable about it, when he/she is entirely out of his/her depth.

2. Attention to detail

Developers tend to be capable of focusing on one thing to the exclusion of all else. This often means that, whatever the idea that comes along, a developer will focus on it exclusively. Such focus may turn up issues that were missed by the less attentive, or it may just be nit picking. (Finding small issues isn’t nitpicking when one is developing–it’s pre-emptive bug fixing.)

3. Curiosity and the desire to learn

Most developers are curious. In part because computers are so new, and in part because software technologies change so rapidly, hackers have to be curious, or they’re left behind, coding Cobol (not that there’s anything wrong with that!). This sometimes spills out into other portions of their lives, tweaking their bodies or the mechanics of an IPO.

4. Know something about something difficult

Yeah, yeah, most developers are not on the bleeding edge of software. But telling most people what they do usually causes some kind of ‘ooh’ or raised eyebrows conveying some level of expectation of the difficulty of software development. (Though this reaction is a lot less universal than it was during the dotcom boom–nowadays, it could just as easily be an ‘ooh’ of sympathy to an out of work .) Because developers are aware that what they do often isn’t that difficult (it’s just being curious, asking questions, and being attentive), it’s easy to assume that other professions usually thought difficult are similarly overblown.

Now, this arrogance surfaces in other realms; for example, business plans. I am realizing just how far I fall short in that arena. I’ve had a few business plans, but they often fall prey to the problem that the gnomes had in South Park: no way to get from action to profit. I’m certainly not alone in this either.

In the same vein of arrogance, I used to make fun of marketing people, because everything they do is so vague and ill-defined. I always want things nailed down. But, guess what, the real world is vague and ill-defined. (Just try finding out something simple, like how many people are driving Fords, how women use the internet, or how many people truly, truly love Richie Valens. You appear to be reduced to interviewing segments of the population and extrapolating.) And if you ask people what they want, they’ll lie to you. Not because they want to lie, but because they don’t really know what they want.

I guess this is a confession of arrogance on the part of one software developer and an apology to all the marketroids I’ve snickered at over the years (oops, I just did it again :). (I promise to keep myself on a shorter leash in the future.) Thanks for going out into the real world and delivering back desires, which I can try to refine into something I can really build. It’s harder than it looks.

WAP vs J2ME

When I gave my talk about J2ME to BJUG a few weeks ago, one of the points I tried to address was ‘Why use J2ME rather than WAP.’ This is a crucial point, because WAP is more widely distributed. I believe the user interface is better, there is less network traffic, and there are possibilities for application extension that just don’t exist in WAP. (Though, to be fair, Michael Yuan makes a good point regarding issues with the optional packages standards process.)

I defended the choice of using MIDP 1.0 because we needed wide coverage and don’t do many complicated things with the data, but WAP is much more widely support than J2ME, by almost any measure. If you don’t have an archaic phone like my Nokia 6160, chances are you have a web browser. And WAP 2.0 supports images and XHTML, giving the application almost everything it needs without learning an entirely new markup language like WML.

So, we’ve decided to support XHTML and thus the vast majority of existing clients (one reason being that Verizon doesn’t support J2ME–at all.) So I’ve gotten a quick education in WAP development recently, and I just found a quote that just sums it up:

“As you can see, this is what Web programmers were doing back in 1994. The form renders effectively the same on the Openwave Browser as it does on a traditional web browser, albeit with more scrolling.”

This quote is from Openwave, a company that specializes in mobile development, so I reckon they know what they’re talking about. A couple of comments:

1. WAP browsers are where the web was in 1994. (I was going to put in a link from 1994, courtesy of the Way Back Machine, but it only goes back to 1996.) I don’t know about you, but I don’t really want to go back! I like Flash, DHTML and onClick, even though they can be used for some truly annoying purposes.

2. “…albeit with more scrolling” reinforces, to me, the idea that presenting information on a screen of 100×100 pixels is a fundamentally different proposition than a screen where you can expect, at a minimum, 640×480. (And who codes for that anymore?) On the desktop, you have roughly 30 times as much screen real estate (plus a relatively rich language for manipulating the interface on the client). It’s no surprise that I’m frustrated with I browse with WAP, since I’m used to browsing in far superior environments.

3. Just like traditional browsers, every time you want to do something complicated, you have to go to the server. You have to do this with XHTML (but not with WML, I believe. WML has its own issues, like supporting only bitmap pictures). That’s not bad when you’re dealing with fat pipes, but mobile networks are slow.

4. Fitting in with the carrier is an issue with WAP. Since the browser is provided, you have no control over some important issues. For example, one carrier we’re investigating requires you to navigate through pages and pages of carrier imposed links before you can get to your own bookmarks. It’s the whole gated community mindset; since the UI sucks, it’s harder to get around than it would be with Firefox.

In short, use WAP 2.0 if you must, but think seriously about richer clients (J2ME, BREW, or even the .Net compact framework). Even though they’ll be harder to implement and roll out, such clients will be easier to use, and thus more likely to become a part of your customers’ lives.

What use is certification?

What good are certifications like the Sun Certified Java Programmer (SCJP) and the Microsoft Certified Systems Engineer programs? Unlike the Cisco certifications, you don’t have to renew these every couple of years (at least the Java certifications–in fact, everything I mention below applies only to the Java certifications, as those are the only ones of which I have more than a passing knowledge). I am a SCJP for Java2, and I have an acquaintance who is a certified programmer for Java1.1; a Java1.1 cert isn’t very useful unless you’re targeting .Net, or writing applets that need to run on most every browser. Yet my colleague and myself can continue to call ourselves ‘Java Certified Programmers.’ I realize that there’s an upgrade exam, but I’ve never met a soul who’s taken it; and I don’t believe I’m prohibited from heading down the Java Certification path and handing Sun more money because I am not an SCJP for the most recent version of Java. In fact, I’m studying right now for the Sun Certified Web Component Developer (SCWCD) and plan to take the exam sometime this summer. Even though these certifications may be slightly diluted by not requiring renewal, I think there are a number of reasons why they are a good thing:

1. Proof for employers.

Especially when you deal with technologies that are moving fast (granted, changes to Java have slowed down in the past few years, but it’s still moving faster than, for example, C++ or SQL), employers may not have the skill set to judge your competence. Oh, in any sane environment you will probably interview with folks who are up to date on technology, but who hasn’t been screened out by HR because of a lack of appropriate keywords. Having a certification is certainly no substitute for proper experience, but it serves as a baseline that employers can trust. In addition, a certification is also a concrete example of professional development: always a good thing.

2. Breadth of understanding.

I’ve been doing server side Java development for web environments for 3 years now, in a variety of business domains and application servers. Now, that’s not a long time in programming years, but in web years, that’s a fair stint. But, studying for the SCWCD, I’m learning about some aspects of web application development that I hadn’t had a chance to examine before. For example, I’m learning about writing tag libraries. (Can you believe that the latest documentation I could find on sun.com about tag libraries was written in 2000?) I was aware of tag libraries, and I’d certainly used them, the struts tags among others, but learning how to implement one has really given me an appreciation for the technology. Ditto for container managed security. Studying for a certification definitely helps increase the breadth of my Java knowledge.

3. Depth of understanding.

Another aspect is an increased depth of understanding; actually reading the JSP specification or finding out what the difference is between overriding and overloading (and how one of them cares about the type of the object, whereas the other cares only about the type of the reference) or in what order static blocks get initialized. (My all time favorite bit of know-how picked up from the SCJP was how to create anonymous arrays.) The knowledge you gain from certification isn’t likely to be used all the time, but it may save you when you’ve got a weird bug in your code. In addition, knowing some of the methods on the core classes saves you from running to the API every time (though, whenever I’m coding, the javadoc is inevitably open). Yeah, yeah, tools can help, but knowing core methods can be quicker (and your brain will always be there, unlike your IDE).

4. A goal can be an incentive.

Personally, I’m goal oriented, and having a certification to achieve gives me a useful framework for expenditure of effort. I know what I’m aiming for and I’m aware of the concrete series of steps to achieve that goal. I can learn quite a bit just browsing around, but for serious understanding, you can’t beat a defined end point. I’d prefer it to be a real-world project, but a certification can be a useful stand in. (Yes, open source projects are good options too–but they may not cover as much ground and certainly, except for a few, are not as widely known as certifications.)

I’ve met plenty of fine programmers who weren’t certified (just as I’ve met plenty of fine programmers who weren’t CS majors). However, I think that certifications can be a useful complement to real world experience, giving job seekers some legitimacy while also increasing the depth and breadth of their understanding of a language or technology.

Software archeology

I presented this evening on J2ME for the kickstart meeting at BJUG, where Grady Booch was the featured speaker. After unknowingly knocking UML in his presence, I enjoyed a fine talk on software archeology. This discipline involves looking at larger, historical patterns of software development. Essentially, when we build software, we are building artifacts. And, just as the plans and meetings of the the slave foremen who built the pyramids used are not recorded, so there are aspects of present day software development that are simply lost when the project ends or the programmers die. One of Booch’s projects is to capture as much of that data as possible, because these architectures are full of valuable knowledge that many folks have sweated for. It needs to happen soon, because, in his words, “time is not on our side” when it comes to collecting this kind of data. Man, I could handle that kind of job.

Speaking of architecture, I stumbled on “Effective Enterprise Java” which looks to be a set of rules for enterprise java development. I really enjoy “Effective Java”, by Joshua Bloch, so I hope that Ted Neward’s book lives up to its name. And I certainly hope this project doesn’t get stranded like “Interface Design” apparently did.

Is transparent access control worth unintelligible error messages?

Partly egged on by Rob and Brian, I just took a long overdue look at container managed security for web applications.

My conclusion: it’s nice, but there is one major flaw that dooms the whole premise. Users expect informative error messages when they ‘sign in’ and there’s no way to do that with container managed security.

I was using Tomcat 4.1, which is to say, I was examining the servlet 2.3 specification. (I just looked at the 2.4 specification and can see no amelioration of the above issue.) I also focused on the FORM method of authentication, as that’s the most customizable. (I imagine, for an intranet app obsessed with security, client certificates would be an worthwhile avenue of investigation.) I found the servlet specs to be very helpful in this process.

With the FORM method of authentication, you can customize the appearance of your login and error pages, to some extent. This is a huge win.

I really liked the automatic access control–no checking at the beginning of every ActionForm or JSP for any specific attribute. Additionally, you can protect different URL patterns easily, and for most of the applications I write, this is enough. If you need to protect buttons on a page, you can always resort to isUserInRole.

Also, you can protect the login and error pages, which should never be accessed directly in a separate /safe directory, to which you can prohibit all access.

For the times when the user is denied access to a resource, you you can create a custom 403 error page, using the error-page directive in web.xml. Unfortunately, you only seem to get three attributes: javax.servlet.error.message, javax.servlet.error.request_uri and javax.servlet.error.status_code, which limits the nature of your response. These were what Tomcat gave me–I don’t think it’s part of the spec. Regardless, IE, with default settings, doesn’t display any custom error messages, which makes this a rather moot point for general webapps.

Creating a logout page is fairly easy, just call session.invalidate() (though there seem to be some non standard methods of doing it as well).

However, as mentioned above, I just don’t think that users will accept the generic login error messages that you are forced to give. For instance, you can’t tell whether a user didn’t enter a password, or entered an incorrect password. You can’t redirect them back to a login page with helpful error messages around the incorrect box. These are fundamental issues with authentication–no serious webapp simply throws up its hands when a user doesn’t login correctly the *first* time.

Separate from user experience, but still related to authentication behavior, you can’t ‘lock out’ users who’ve attempted to login too many times. Sure, you can keep track of how many times they’ve tried to login, but the authentication process is out of your hands.

Additionally, the fact that you’re tied to a particular implementation for user/role definition means that writing custom authentication code that just accesses a RDMBS is actually more portable.

The answer, to the question posed in the title of this post: “is transparent access control worth unintelligible error messages?”, is almost always “no.” And folks accuse developers of not having any sense of user interface!