Skip to content

How to rotate catalina.out without restarting tomcat

There seems to be some confusion regarding rotating tomcat logs, in particular rotating catalina.out without restarting Tomcat. With Tomcat4, catalina.out contains everything the application developers logged using System.out.println. Because of the way that Tomcat holds onto the file descriptor on Unix, simply moving the file doesn’t work: Tomcat merrily writes to a file that you can no longer access.

This is important to do for the same reasons you should rotate any other system log; disk space is cheap, but logs can fill it quickly, and the information value of logs decreases rapidly with time. Rotating logs typically means you remove them from the filesystem, but you can also archive them off line with tape, or to a SAN, etc. I’ve run into issues with JSVC and large log files too.

There’s an entry in the FAQ and a pointer to a maillist discussion. But you have to dig a bit and I thought I’d just document what I did. (This works for Tomcat4–Tomcat 5.5 did away with the Logger element and I’m guessing you’d just use some of the log4j configuration and RollingFileAppender.)

First off, you want to make sure that you’re rotating your other logs. This is typically done with a Logger component, configured either at the host or context level. Set the timestamp attribute to be ‘true’.

Now Tomcat is handily rotating all your host and context level logs. You may want to gzip or delete old ones via find:
/usr/bin/find ./logs -mtime +60 -name "mylog_log*"|xargs rm -f

The next step is to set the swallowOutput attribute to ‘true’ on all contexts. According to the docs, this attribute causes

the bytes output to System.out and System.err by the web application [to] be redirected to the web application logger.

Now you have to restart tomcat. But from here on out, Tomcat will rotate all configured logs, and catalina.out will only contain start and stop messages.

Oh yeah, and just as that email thread does, I heartily condemn using System.out.println because log4j is so easy to setup and use. But I’ve written code like that (admittedly, usually with a sheepish grin on my face), so I can’t cast the first stone.

[tags]tomcat4,logfile rotation[/tags]

Tomcat | moore |

My software internationalization article is now up at Ccaps

A few months back, I was approached to write an article for the web newsletter that the folks at Ccaps publish. With the help of Mike and Jeff from Zia Consulting, and Brian Pontarelli, I was able to write what I think is a fairly decent introduction into internationalization from a software engineering standpoint, based on the project I’ve presented about in the past. I hope you’ll enjoy it. (I’ll be archiving it permanently on my website soon.)

Update 4/19/07: I corrected the format of the name Ccaps, and linked to their website.
[tags]i18n[/tags]

Programming Web Applications | moore |

MySQL tuning

If you’re trying to tune MySQL, make sure you measure before and after. mysqlreport is a great way to measure a number of aspects of mysql (make sure you check out the guide). If you want graphing and long term tracking for mysqlreport and just about any other measure you care to track, I recommend cacti, which works with rrdtool.

[tags]mysql performance, monitoring[/tags]

Databases MySQL | moore |

Boulder BarCamp notes

I just went to BarCampBoulder, and had a blast. Basically, it’s a self organized conference. Free, no set schedule (there’s a white board with a tenative schedule) and anyone can give a talk about anything. Very cool. Here’s a random link fest covering topics I chose to make note of.

  • Microformats, especially hCal. Feasibility of machine aggregation of Boulder free events, and complexities inherent in such an effort.
  • An understanding of how slippery the concept of “The Semantic Web” is.
  • An excellent post about handling credit card transactions.
  • To clean up user generated content, and avoid XSS attacks, use UBB or some other markup language. Another option is to validate any user generated content against an XHTML Mobile DTD, which should screen out any nastiness.
  • CA-Cert, a way to generate SSL certificates for free, that is slightly better than self signed certs.
  • CardSpace and OpenID,and the differences and benefits of using each.
  • A post about the Laws of Identity
  • ColourLovers, for all you palette hounds out there.
  • A fascinating discussion of the amount of data running through Digital Globe’s systems.
  • An overview of CMSes worth looking at: Typo3, TextPattern, Radiant
  • A discussion of how open communities work (and don’t), including Wikipedia.

It was a fascinating day. There seemed to be a business track and a technical track, and there ended up being about 30ish folks there. Well worth it, if for no other reason than the intellectual ferment.

BarCamp | moore |

Comments on Yahoo! Mail storage and APIs

Check it out, I’m quoted in a piece about Yahoo! Mail in Infoworld, saying:

As far as I can tell, I’ll never need to delete another e-mail message, but if I did run into a limit, it’d be very easy for me to pull down the messages via the POP interface and store them on a personal hard drive

I stand by my words. I haven’t deleted a message for years, and doubt I ever will again. I’m not really sure what that means in terms of access–I’d be very interested to look at how many messages older than, say, a year, have been accessed. It does mean that I no longer need to decide whether or not to spend time filing/deleting/organizing mail.

I have been a premium user of Yahoo! mail for quite some time. I like the ‘beta’ JS intensive interface, though before I bought a new computer, it taxed my PC.

Personally, I think the bigger announcement is the opening of the Yahoo! mail APIs. Granted, only premium users get the full API, but I can imagine all sorts of neat applications built on that API. (The standard API doesn’t look that useful.) For example, you could build an external program to live out of your Yahoo Mail, like this fellow, but with offline capacity, or a simple blogging client where you leverage Yahoo’s superb rich client interface and platform to generate content, and then push it elsewhere.

Technology Web Applications | moore |

CleanPrint in action

I wanted to congratulate FormatDynamics on a fine piece of technology: CleanPrint. Disclaimer: I have worked for them in the past. I know how hard they’ve worked on this, so I wanted to congratulate them.

To see it in action, visit this page, using IE, and do a print preview. See how the previewed page is formatted for print, including different graphics and advertisements? The printed page is dynamicly formatted from the content on the HTML page, with a minimum of intrusion on the page.

How is this different than a print stylesheet? The dynamic reformatting gives much greater control over the printed page’s appearance than a stylesheet, for one. There’s also an advertising component.

It’s a fine technology that will hopefully save paper and improve the user experience for all those people (many!) who print web pages. Nice job, guys!
[tags]web page printing, cleanprint, print advertising[/tags]

Technology | moore |

Out of the box complete open source WAMP stack

If you’re looking for an out of the box WAMP stack, I had good luck with Apache2Triad. A big download, but you’re getting Apache, PostGreSQL, MySQL, a mail server and an FTP server, plus various admin tools. I had things running in about 15 minutes–very cool, given that I’d spent a fair bit of time just trying to get PostGreSQL installed alone. The only hiccup was that my spyware tool thought SlimFTPd was a virus.

Beware, though, if you have an existing MySQL installation. Apache2Triad won’t blow away the data, but it will usurp the Services entry. And then, if you remove Apache2Triad, your existing MySQL instance is orphaned. I was able to get mine up and running again, from the command line. But to get it running again as a service required a complete uninstall/reinstall. No fun.

So, if you have no dev environment and need a quick start, take a look at this package. If you, on the other hand, have existing development tools installed, be more careful than I was.

[tags]WAMP[/tags]

Databases Web Applications | moore |

Review of MODx

Update 7/2010: Here’s a more recent review of MODx.  I’m bummed, but I have not yet had another opportunity to use MODx.

A couple of weeks ago, I mentioned I’d be reviewing MODx in the near future. I recently used it and was quite impressed by this project. It is a web CMS written in PHP with mysql as the datastore. I say ‘web CMS’ because MODx is designed to manage web content, as opposed to a more enterprisey CMS like StoryServer, which can manage all different types of content with complex workflows. You could use MODx to manage, say, printed brochures, but that would take some finagling. Web content is the sweet spot of this framework. Currently at version 0.9.5, it is fairly mature and ready for use.

I wanted to review MODx because I believe it deserves more attention, and as an example of how I’d evaluate an open source project, on the back end, rather than on the front end, of a project.

The good:

full featured

I was able to find everything I needed in MODx, or in the extensions repository. This included a thumbnail generator, email forms, integrated rich text editor, SEO friendly links, database updating forms, and a content redistributor (that syndicated content internally within the site).

admin interface

I think the administrative interface is excellent for non technical users. It’s responsive, and intuitive.

user access/authentication system

There’s a very well thought out access system. You can assign users to roles, which lets them access certain
functions in the admin interface, and groups, which define groups of documents that a user may modify. Here is more documentation on this feature.

defined development architecture

When you start working with MODx and you want to do something more than a cookie cutter website, you start hearing jargon, like snippets, plugins, chunks and template variables. All of these are MODx specific concepts, and it takes a while to wrap your head around them. But when you do, you appreciate the thoughtfulness of the architecture. In particular, you rarely have to modify existing source–there are hooks and easy ways to tie in custom code. (Most of these hooks are for the user side–to modify the admin interface, I had to hack some existing PHP.)

community

The MODx forums are the heart of the community. There are quite a few active members. I found the community to be very responsive and friendly to any questions I had, no matter how dumb.

a growing set of extensions

The MODx repository has a number of useful extensions. I especially liked that each entry in the repository is labeled with the version of MODx it supports.

active development

The project has gone from start up to 0.9.5 in less than two years.

open source

The license is GPL version 2.

caching

You can turn on a simple form of caching, which will serialize a generated page to disk. Unfortunately, there’s no way to expire that cache. You can delete it, on a site wide or page by page basis, but you can’t say ‘expire the cached version of this page in one month’. Still, for many pages, this is an appropriate form of caching and can noticeably speed up the site.

The bad:

documentation

There is a ton of documentation for MODx, even a wiki. But I always felt like I was missing something–either it was hard to find what you wanted, or when you did, it wasn’t enough. An example is the API documentation. Here’s a sample function call that you’d make on the $modx object. No explanation of the returned data structure is available, and no actual example of how to call this function. I became very friendly with var_export($var,TRUE); and print statements to navigate these returned structures.

dependency on the database

MODx is very tightly bound to mysql. No problem there–mysql is a great database. But I mean, it’s really tied to mysql. By default, all code you write (see ‘defined development architecture’ above) is stored in the database. That’s not the place for code! Luckily, you can avoid that by using an include: include($modx->config['filemanager_path'].'/assets/libs/thumbs/thumbplugin.php');

This way, the code is on the file system, and can be versioned, etc. Also, since MODx depends on the database for so much functionality, make doubly sure you backup the database.

error messages in development

PHP syntax errors can be hard to track down. I ended up using a lot of command line debugging: php -l -f foo.php and looking in error log to see messages.

5000 document limit

This is a big one. Because of the caching mechanism, you can’t have more than 5000 documents in a MODx website.  However, this is acknowledged as a lack, and the team is working on it.

no search in specific forum

The forums are great, and are divided up into various sections. However, there was no way (that I could find), to search within a particular forum, or even within just the forums. This meant that when you were searching, you ended up with a lot of extraneous results.

Sure, MODx isn’t right for every site. But if you have a PHP savvy developer, a non technical userbase, requirements more complex than brochureware, and want to get a site up and running quickly, MODx is worth a look. As I’ve said before, use the right tool for the job.

Much thanks to the developers of MODx for putting together a great generic web CMS development platform!

Update 11/2009: HostColor offers MODx hosting for a reasonable price.  If you’re looking, check ’em out (click the CMS Hosting link).  Disclaimer: I make a bit of money if you visit them and/or sign up.

[tags]modx,php, web cms[/tags]

PHP Web Applications | moore |