I have a client who uses a Google Mini (a search appliance created by Google) as a key part of their business. The appliance worked fine, gave great search results, and was happily humming along until very recently. At some point over the last couple of months, the number of 504 errors visible to the users using the mini increased drastically.
At one point, no matter what term you searched on, you would see this error message:
Error: Unknown XML result type.
View page source to see the offending XML.
When I took a look at this to try to troubleshoot, the results were very arbitrary--some search terms would show this message, some would not. However, when a search term did show the error message, it would do so consistently, at least for 5-10 minutes. Perhaps that had to do with caching?
The benefit of the mini is that you get great google like search results. The detriment is that it is a black box--you don't even get a login to the box, etc. And when your support contract runs out, you're left with very few options.
So, we troubleshot with one hand tied behind our back. We:
- tried rebooting the box a couple of times (via the web interface)
- considered rolling back the index, but we only had about 36 hours of index history
- generated and analyzed the search logs
- searched the Google Groups for mini support. All that turned up was this post which was not exactly helpful, since we no longer had support
Finally, the client suggested resetting the index, as it was conceivable that the mini index had been somehow corrupted. This was an option for the client because their documents are re-crawled every day. If that wasn't the case, it would have been a harder decision.
Resetting the index solved the problem. Some numbers:
- a few months ago, 0.5% of search requests were returning either 500 or 504 errors.
- just before the index was reset, 40.2% of the requests were either 500 or 504 errors.
- now, 0% of requests are either 500 or 504 errors.
FYI, this post is applicable to the Google Mini version 4.6.4--I'm not sure how more current versions of software work/break.
Technorati Tags: google mini, black box troubleshooting, 504 error