Skip to content

java memory management, oh my!

How much do you understand basic java? Every day I find some part of this language that I’m not aware of, or don’t understand. Some days it’s cool APIS (like JAI) but today it’s concurrency. Now, language managed memory is a feature that’s been present in the languages in which I’ve been programming since I started. I’ve looked at C and C++, but taking a job coding in those seems to me it’d be like a job with a long commute–both have obstacles keeping you from getting real work done. (I’m not alone in feeling this way.) But this thread of comments on Cameron Purdy’s blog drove home my ignorance. However, the commenters do point out several interesting articles (in particular, this article about double checked locking was useful and made my head hurt at the same time) to alleviate that. I took a class with Tom Cargill a few years back, which included his threading module, that helped a bit.

However, all these complexities are why servlets (and EJBs) are so powerful. As long as you’re careful to only use local variables, why, you shouldn’t have to worry about threading at all. That’s what you use the container for, right? And we all know that containers are bug free, right? And you’d never have to go back and find some isolated thread related defect that affected your code a maddeningly miniscule amount of time, right?

Denver No Fluff Just Stuff

Well, I just got done with two days of the Denver No Fluff Just Stuff conference. First off, unlike the previous NFJS conferences, this one wasn’t held in the DTC. You forget how far that is from Boulder, until you drive there and back twice on a weekend.

Anyway, I thought I’d share a few choice thoughts and tidbits regarding some of the sessions I attended. These are by no means full summaries of the talks.

Mock objects–Dave Thomas

Mock objects are objects that emulate behavior of external entities that make testing difficult. (I’ve worked with a few Englishmen in my life, and Dave Thomas had the same acerbic sense of humor.) Dave illustrated how to choose when to implement a mock object, as opposed to using the real object. He also touched on the true difficulty of mock objects, which is figuring out how to choose which object to use in your class (factory, pass the correct object into the constructor, AOP, class loaders).

JSF (both beginning and advanced)–David Geary

JSF is the new standard for web frameworks. David compared it to Swing and Struts meeting in a particle accelerator. Thompson’s fussed about tools for JSF, but I don’t think they’ll be needed for all JSF development, just like tools for Struts help, but aren’t required. I think that the most important bit about JSF is that it really tries to treat HTML widgets as full featured GUI components, which is something that is a bit of an adjustment for me. I’m really really used to thinking of HTML interfaces as generated strings, but this higher level approach (which has been used in the rich client world for a long time) is interesting.

There was an expert panel, consisting of several of the speakers. One hot topic was whether EJB 3.0 had been hijacked by Gavin King; everyone seemed to have an opinion on that. However, the choicest statement to emerge was Bruce Tate saying Java’s “type safety is an illusion” because everyone who uses a collection casts whenever they take anything out.

Herding Racehorses, Racing Sheep–Dave Thomas

This was a non-technical talk discussing how to improve programming as a profession. He referenced the Dreyfus Model of Skill Acquisition (novices learn differently from experts), and referenced Patricia Benner and her study of nurses in the 1970s, and how it was analgous to the current situation of developers. A great quote was “Training is how you give people knowledge; time is how you give people experience.” He also talked about how to move up the skill ladder, and how that will make it more difficult to outsource. However, he didn’t talk about how the relative dearth of novices would create a future shortage of experts, other than to acknowledge that everyone, anywhere, can move up the skill ladder and we need to prepare for that. Prepare by having a plan; this makes sense, as what you’re really doing is choosing where to invest your most precious commodity–your time.

TDD in the web tier–Rick Hightower

Rick covered the basics of Test Driven Development, and seemed a bit surprised that everyone wasn’t practicing it; he said it’s helped his code quite a bit. He went over a few tools that make testing (not just unit testing) easier today. A key point seemed to be the differentiation between TDD and Continuous Integration; tests that run for TDD need to be fast, since you’re running them multiple times a day, whereas CI tests can be slower. He also made the offhand comment that you could have JMeter proxy requests from a QA tester (in a web browser) and use Canoo (a JSP testing tool) to automate those tests. Wouldn’t that be cool?–cheaper than LoadRunner, that’s for sure.

Another expert panel. Someone asked “what are you folks going to be looking at in the next 6 months” and I was struck by the lack of diversity in the responses. Groovy, Hibernate, Tapestry came up again and again. Where do the new ideas come from? And where does deep knowledge come from, if everyone is running to a new cool tool every 6-12 months?

An offhand comment that someone made when we were talking abouty why so many apps had extraneous EJBs: “Yup, that was design by resume.”

Appfuse–Rick Hightower

Appfuse is a way to kick start your Struts applications. It provides a large chunk of best practices all in one place, along with a few pages that everyone needs (user creation, user authentication). Its license is liberal enough that you can use the code in your own project. I was struck by how many times Rick mentioned ripping stuff out, but I’m sure that I would learn quite a bit by poking around it. It was also clear to me that AppFuse is great for staring new applications, but I’m not sure it’s a good thing (other than a learning tool) for retrofitting best practices to existing applications. Also, Rick mentioned multiple times that he wouldn’t use Struts for a new application; given that AppFuse is primarily a Struts starter kit, I was a bit confused by this statement.

GIS–Scott Davis

This was a 1,000 foot overview of (primarily java) GIS applications. There are quite a few tools out there for displaying GIS data, which has several standardized formats (both those formally blessed by a standards organization, and those informal standards that grow out of network effects). There aren’t a collection of open source data sets, but you can get a ton of GIS data from government websites. The satellite that Scott’s company owns takes photos that are 15GB of data, and takes 500 such photos a day. Talk about storage needs. Also, anyone who wants to find out a bit more about satellite imaging would do well to read “Private eyes in the sky”, an article from the May 4th 2000 edition of the Economist, which is a good overview of the business.

Again, apologies for the jerky nature of my comments above. (Hey, at least I’m not talking about tugging any unmentionables.) Hangovers are not conducive to good note taking, but even if I had been rested, I still couldn’t do justice to 90 minutes of expert dialog in a paragraph on my blog. But it’s well worth going to one of these conferences, especially if you’re doing java web development.

What use is certification?

What good are certifications like the Sun Certified Java Programmer (SCJP) and the Microsoft Certified Systems Engineer programs? Unlike the Cisco certifications, you don’t have to renew these every couple of years (at least the Java certifications–in fact, everything I mention below applies only to the Java certifications, as those are the only ones of which I have more than a passing knowledge). I am a SCJP for Java2, and I have an acquaintance who is a certified programmer for Java1.1; a Java1.1 cert isn’t very useful unless you’re targeting .Net, or writing applets that need to run on most every browser. Yet my colleague and myself can continue to call ourselves ‘Java Certified Programmers.’ I realize that there’s an upgrade exam, but I’ve never met a soul who’s taken it; and I don’t believe I’m prohibited from heading down the Java Certification path and handing Sun more money because I am not an SCJP for the most recent version of Java. In fact, I’m studying right now for the Sun Certified Web Component Developer (SCWCD) and plan to take the exam sometime this summer. Even though these certifications may be slightly diluted by not requiring renewal, I think there are a number of reasons why they are a good thing:

1. Proof for employers.

Especially when you deal with technologies that are moving fast (granted, changes to Java have slowed down in the past few years, but it’s still moving faster than, for example, C++ or SQL), employers may not have the skill set to judge your competence. Oh, in any sane environment you will probably interview with folks who are up to date on technology, but who hasn’t been screened out by HR because of a lack of appropriate keywords. Having a certification is certainly no substitute for proper experience, but it serves as a baseline that employers can trust. In addition, a certification is also a concrete example of professional development: always a good thing.

2. Breadth of understanding.

I’ve been doing server side Java development for web environments for 3 years now, in a variety of business domains and application servers. Now, that’s not a long time in programming years, but in web years, that’s a fair stint. But, studying for the SCWCD, I’m learning about some aspects of web application development that I hadn’t had a chance to examine before. For example, I’m learning about writing tag libraries. (Can you believe that the latest documentation I could find on sun.com about tag libraries was written in 2000?) I was aware of tag libraries, and I’d certainly used them, the struts tags among others, but learning how to implement one has really given me an appreciation for the technology. Ditto for container managed security. Studying for a certification definitely helps increase the breadth of my Java knowledge.

3. Depth of understanding.

Another aspect is an increased depth of understanding; actually reading the JSP specification or finding out what the difference is between overriding and overloading (and how one of them cares about the type of the object, whereas the other cares only about the type of the reference) or in what order static blocks get initialized. (My all time favorite bit of know-how picked up from the SCJP was how to create anonymous arrays.) The knowledge you gain from certification isn’t likely to be used all the time, but it may save you when you’ve got a weird bug in your code. In addition, knowing some of the methods on the core classes saves you from running to the API every time (though, whenever I’m coding, the javadoc is inevitably open). Yeah, yeah, tools can help, but knowing core methods can be quicker (and your brain will always be there, unlike your IDE).

4. A goal can be an incentive.

Personally, I’m goal oriented, and having a certification to achieve gives me a useful framework for expenditure of effort. I know what I’m aiming for and I’m aware of the concrete series of steps to achieve that goal. I can learn quite a bit just browsing around, but for serious understanding, you can’t beat a defined end point. I’d prefer it to be a real-world project, but a certification can be a useful stand in. (Yes, open source projects are good options too–but they may not cover as much ground and certainly, except for a few, are not as widely known as certifications.)

I’ve met plenty of fine programmers who weren’t certified (just as I’ve met plenty of fine programmers who weren’t CS majors). However, I think that certifications can be a useful complement to real world experience, giving job seekers some legitimacy while also increasing the depth and breadth of their understanding of a language or technology.

Inlining of final variables and recompilation

This problem that has bitten me in the ass a few times, and I’d love to hear any bright ideas on how y’all avoid it.

Suppose you have an interface that defines some useful constants:

public interface foo {
 int five = 6;
}

and a class that uses those constants:

public class bar {
 public static void main(String[]args) {
  System.out.println("five: "+foo.five);
 }
}

All well and good, until you realize that five isn’t really 6, it’s 5. Whoops, change the foo java file and rebuild, right? Well, if you use javac *.java to do this (as you might, if you only have the foo and bar files), then you’ll be alright.

But, if you’re like the other 99% of the java development world, and you use a build tool, like ant, smart enough to look at timestamps, you’ll still get 6 for the output of java bar. Ant is smart enough to look at the timestamps of .class and .java files to determine which .java files have changed since it last did a compilation. But it is too dumb to realize that the bar class has a dependency on foo, and should thus be recompiled even though bar.java is older than bar.class. (I haven’t looked at the byte code, but I expect that the value of five is just inlined into the bar class because it’s a final variable.) If you’re using a make based build system, I believe you can use javadeps to build out the correct dependency list, but I haven’t seen anything similar for ant. Another options is to just remember to blow away your build directory anytime you change your ‘constants’.

I guess this is why properties files might be a better choice for this type of configuration information, because they’re always read in anew at startup, and thus cannot be inlined (since they’re a runtime thing). Of course, then you lose the benefits of type checking. Not sure what the correct answer is.

Kris Thompson’s review of my talk

Kris Thompson attended my BJUG kick start talk on J2ME development. I wanted to comment on his post.

1. I wouldn’t say that J2ME development has scarred me. But J2ME is definitely a technology (well, a set of technologies, really) that is still being defined. This leads to pain points and opportunities, just like any other new technology. Lots of ground to be broken.

2. Caching–you can do it, but just like in any other situation, caching in J2ME introduces additional complexities. Can it be worth it, if it saves the user time and effort? Yes. Is it worth it for the application I was working on? Not yet.

3. PNG–it’s easy to convert images from GIF/JPEG format to PNG. Check out the extremely obtuse JAI.create() method, and make sure you check out the archives of the jai-interest mailing list as well.

4. Re: Shared actions between MIDP and web portions of the application, I guess I wasn’t very clear on this–the prime reason that we don’t have shared action classes between these two portions was because, other than in one place (authentication) they don’t have any feature overlap. What you can do on the web is entirely separate from what you can do with the phone (though they can influence each other, to be sure).

Anyway, thanks Kris for the kind words.

As a last note, I would reiterate what Kris mentions: “Find out which phones have the features you want/need” and perhaps add “and make sure your service provider supports those features as well.” Unlike in the server side world, where everyone pretty much targets IE, J2ME clients really do have different capabilities and scouting those out is a fundamental part of J2ME development.

Software archeology

I presented this evening on J2ME for the kickstart meeting at BJUG, where Grady Booch was the featured speaker. After unknowingly knocking UML in his presence, I enjoyed a fine talk on software archeology. This discipline involves looking at larger, historical patterns of software development. Essentially, when we build software, we are building artifacts. And, just as the plans and meetings of the the slave foremen who built the pyramids used are not recorded, so there are aspects of present day software development that are simply lost when the project ends or the programmers die. One of Booch’s projects is to capture as much of that data as possible, because these architectures are full of valuable knowledge that many folks have sweated for. It needs to happen soon, because, in his words, “time is not on our side” when it comes to collecting this kind of data. Man, I could handle that kind of job.

Speaking of architecture, I stumbled on “Effective Enterprise Java” which looks to be a set of rules for enterprise java development. I really enjoy “Effective Java”, by Joshua Bloch, so I hope that Ted Neward’s book lives up to its name. And I certainly hope this project doesn’t get stranded like “Interface Design” apparently did.

Is transparent access control worth unintelligible error messages?

Partly egged on by Rob and Brian, I just took a long overdue look at container managed security for web applications.

My conclusion: it’s nice, but there is one major flaw that dooms the whole premise. Users expect informative error messages when they ‘sign in’ and there’s no way to do that with container managed security.

I was using Tomcat 4.1, which is to say, I was examining the servlet 2.3 specification. (I just looked at the 2.4 specification and can see no amelioration of the above issue.) I also focused on the FORM method of authentication, as that’s the most customizable. (I imagine, for an intranet app obsessed with security, client certificates would be an worthwhile avenue of investigation.) I found the servlet specs to be very helpful in this process.

With the FORM method of authentication, you can customize the appearance of your login and error pages, to some extent. This is a huge win.

I really liked the automatic access control–no checking at the beginning of every ActionForm or JSP for any specific attribute. Additionally, you can protect different URL patterns easily, and for most of the applications I write, this is enough. If you need to protect buttons on a page, you can always resort to isUserInRole.

Also, you can protect the login and error pages, which should never be accessed directly in a separate /safe directory, to which you can prohibit all access.

For the times when the user is denied access to a resource, you you can create a custom 403 error page, using the error-page directive in web.xml. Unfortunately, you only seem to get three attributes: javax.servlet.error.message, javax.servlet.error.request_uri and javax.servlet.error.status_code, which limits the nature of your response. These were what Tomcat gave me–I don’t think it’s part of the spec. Regardless, IE, with default settings, doesn’t display any custom error messages, which makes this a rather moot point for general webapps.

Creating a logout page is fairly easy, just call session.invalidate() (though there seem to be some non standard methods of doing it as well).

However, as mentioned above, I just don’t think that users will accept the generic login error messages that you are forced to give. For instance, you can’t tell whether a user didn’t enter a password, or entered an incorrect password. You can’t redirect them back to a login page with helpful error messages around the incorrect box. These are fundamental issues with authentication–no serious webapp simply throws up its hands when a user doesn’t login correctly the *first* time.

Separate from user experience, but still related to authentication behavior, you can’t ‘lock out’ users who’ve attempted to login too many times. Sure, you can keep track of how many times they’ve tried to login, but the authentication process is out of your hands.

Additionally, the fact that you’re tied to a particular implementation for user/role definition means that writing custom authentication code that just accesses a RDMBS is actually more portable.

The answer, to the question posed in the title of this post: “is transparent access control worth unintelligible error messages?”, is almost always “no.” And folks accuse developers of not having any sense of user interface!

SimpleDateFormat and the 13th month

Wow. I just learned something about SimpleDateFormat, a class that I always resort to when I have to convert a String to a Date in java. Check out this bit of code:

import java.text.*;
import java.util.*;

public class foo {
public static void main (String[] args) throws Exception {
SimpleDateFormat sdf = new SimpleDateFormat("MMddyyyy");
System.out.println("12012000 "+ sdf.parse("12012000"));
System.out.println("13012000 "+ sdf.parse("13012000"));
System.out.println("12322000 "+ sdf.parse("12322000"));
}
}

and the output from that code:

$ java -classpath . foo
12012000 Fri Dec 01 00:00:00 MST 2000
13012000 Mon Jan 01 00:00:00 MST 2001
12322000 Mon Jan 01 00:00:00 MST 2001

Any overflow gets rolled into the the next higher, well, in addition, I’d call this a place. The 32nd day of December is the 1st of Jan, and the 13th month of any year is Jan. This is an implementation detail, as I found no mention of it in the SimpleDateFormat javadoc, nor the DateFormat javadoc, but others have noticed this too.

jalopy now closed source

Jalopy, which I wrote about here, is now closed source. It’s about $40 for a single user license. For more info, see the corporate website. I see that the open source version is still around, though there hasn’t been a release since 1.02, about 18 months (about the same as xdoclet actually).

I totally respect Hunsicker Marco (who is the developer, I think and certainly the owner of the corporate domain) and his right to earn a living. $40 certainly isn’t that much (in fact, he even has a link to the old, free version on his purchase page!), but I hope that he eventually rolls the improvements into the free version, a la ESR’s “Free The Future, Sell the Present” business model.

Scripting languages and productivity

Bruce Eckel has some things to say about different languages and productivity. One quote in particular stood out:

“I didn’t have to look that up, or to even think about it [reading the contents of a file using python], because it’s so natural. I always have to look up the way to open files and read lines in Java. I suppose you could argue that Java wasn’t intended to do text processing and I’d agree with you, but unfortunately it seems like Java is mostly used on servers where a very common task is to process text.”

I agree entirely. I come from a perl background (it’s the language I cut my teeth on, which, I suppose, dates me), and unlike some, I’m unabashedly in favor of it. I’ve looked at python briefly, and it does seem to have perl’s flexibility and agility with less ambiguity. When you have to grab a file from the filesystem (or parse a file and stuff it into a database) there’s simply no comparison, and anyone who reaches for Java to solve such problems simply hasn’t experienced the joy of the freedom of scripting languages.

The problem with such free form languages arises when you start doing large scale systems. Java, for all its faults and complexity, forces choices about implementation to be done at a high level–which framework do we want to use, how do we architect this solution. Perl (note that I’m not talking about python, since I’m a python newbie), on the other hand, is more flexible, and hence allows more latitude. It requires more discipline to code OO perl, or, for that matter, readable perl, than it does to code readable java. (There are different ways to implement objects in perl–see Object Oriented Perl for more information.) By limiting some of the latitude of the developer, you gain some maintainability.

I was trying to think of trivial examples that illustrate this point, but I couldn’t. Perhaps it’s because I’ve been out of touch with perl’s evolving core libraries for so long, or perhaps it’s because all the perl I’ve ever had to maintain has been intensely idiomatic, where all the java I’ve had to maintain has been, though at times obtuse, fairly easy to read, but I just feel that perl is a harder language to maintain than java.

Now, how does this apply to Eckel’s statements? Well, he uses python as his example–stating that you just plain can get more done with python than you can with java. It’s hard to argue with that…. But the majority of code expense and lifecycle is not in the creation but the maintenance. How do the scripting languages stack up for large scale systems? My experience (which, granted, is primarily applicable to small to medium size systems) indicates that the very flexibility which allows Bruce such amazing productivity hampers further enhancements and bug fixing on the code he writes.