April 19, 2006

apachebench drops hits when the concurrency switch is used

I've used apachebench (or ab), a free load testing tool written in C and often distributed with the Apache Web Server, to load test a few sites. It's simple to configure and gives tremendous throughput. (I was seeing about 4 million hits an hour over 1 gigabit ethernet. I saw about 10% of that from jmeter on the same machine; however, the tests that jmeter was running were definitely more compex.)

Apachebench is not perfect, though. The downsides are that you can only hit one url at a time (per ab process). And if you're trying to load test the path through a system ("can we have folks login, do a search, view a product and logout"), you need to map that out in your shell script carefully. Apachebench has no way to create more complicated tests (like jmeter can). Of course, apachebench doesn't pretend to be a system test tool--it just hits a set of urls as fast as it can, as hard as it can, just like a load tool should.

However, it would be nice to be able to compare hits recieved on the server side and the log file generated by apachebench; the numbers should reconcile, perhaps with some fudge factor for network errors. I have found that these numbers reconcile as long as you only have one client (-c 1, or the default). Once you start adding clients, the server records more hits than apachebench. This seems to be deterministic (that is, repeatable), and worked out to around 4500 extra requests for 80 million requests. As the number of clients approached 1, the discrepancy between the server and apachebench numbers decreased as well.

This offset happened with Tomcat 5 and Apache 2, so I don't think that the issues is with the server--I think apachebench is at fault. I searched the httpd bug database but wasn't able to find anything related. Just be aware that apachebench is very useful for generating large HTTP request loads, but if you need to reconcile for accuracy, skip the concurrency offered.

Posted by moore at April 19, 2006 10:13 PM
© Moore Consulting, 2003-2006