One of the hardest decisions every developer faces is build vs buy. In general, build takes more money and time, but can deliver a program closer to the users’ needs with greater flexibility. Buy, on the other hand, limits extension of the software–only in ways that the creators have intended can you typically extend bought software–but delivers it quickly and for a known cost (unless you’re buying Oracle in which case, I hear, the price is negotiable :).
One of the harder components of deciding to buy is comparing features. This usually involves rummaging around websites, downloading evaluation copies and installing them. I’ve done a few of these (for open source portals, open source CMSes and bug tracking tools) and it’s interesting as well as daunting. There’s just a lot out there, and with limited time, I end up making decisions based on less than full fledged implementation. You can’t afford to entirely implement the solution using the proposed software, and every solution will cause you pain (including, for that matter, custom built solutions).
Regardless, a friend sent me a website that takes some of the tediousness out of evaluating CMSes. Sure, it’s not a replacement for downloading the software and trying it out, but it does give you a central starting point and makes it easier to quickly rule out possible solutions. I was also impressed by their inclusion of blogging tools and ease of use as well as the breadth of features compared.