Simple REST database solutions

I’ve been looking for a simple solution to generate a REST api for a mysql database, with minimal to no coding.  I need to be able to do simple inserts and updates, and some simple querying (including for number ranges and or clauses).

The options are few and none seems to entirely fit the bill.

  • RestSQL is a java layer that has a simple XML configuration.  It supports some advanced features (triggers for biz logic, table composition) but doesn’t yet have numeric comparisons for gets.
  • DBSlayer is a C program that doesn’t do REST, but does JSON results for SQL queries over HTTP
  • PHPRest is a RESTful interface.  From the docs, it is not clear if it supports JSON or any queries more complex than by primary key.

Real estate public records data sources

If you are looking for public records data sources (deeds, sales transactions, assessor records) in an electronic format, you have four options.  Normally this data is kept at the county level, so there haven’t been too many companies that have rolled it up into a nationwide database.  Even for the companies that have, there are timing and data integrity issues that you should be aware of before you commit.

I listed options here because I had to do some digging to find them, and wanted to spare you that.

First, here are some possible solutions that don’t work.

So, what are your four options?

  1. If you are focused on a narrow geographic area, or want to take on a huge project, you can contact each county and see if they have an offering.  I contacted one local county and they wanted $700 for a CD of assessor data.  This has some obvious downsides, but may be enough for your needs.  (I expect some counties won’t offer electronic versions, but that is speculation.)
  2. Core Logic
  3. Data Quick
  4. LPS SiteX

In the latter three cases, you want to call and ask to talk to someone about ‘public record licensing’ if you want to do anything interesting with the data (remix it, aggregate it, display it to your customers, etc).  Public record licensing is not as straighforward as MLS IDX data licensing (and that is saying something!).  It looks like each deal is semi-custom.

You’ll want to speak to a sales representative for full details like pricing, update timing and geographic coverage.  Hope this helps!


Image Recognition Automation Fail

I had a friend who asked me to take a quick look at a business problem he was having.  He had a set of photos (of vinyl record albums) that he was looking at to identify the artist.  After finding the artist, he’d do some additional categorization work and then push the image and metadata to a ecommerce site he is running.

He wanted a way to more quickly identify the artist and album, preferably without his intervention.

The first thing I suggested was Mechanical Turk, as this seemed like a great example of a Human Interaction Task.  However, my friend tried this and found it to be more work (mostly proofing, I think) than it was help.

He also pointed out that Google Image Search does exactly what he wants.  You can post an image or URL to it (you have to use the camera in the search box), and it will give you back like images, best guess for the image matching, and links related to the image.  Pretty cool!

However, there is no API for the new Google Image Search.  Sure, you have the old, deprecated Images API, but it doesn’t have access to post an image or a URL, just keywords.  A bit of looking around revealed this StackOverflow post, which deconstructs the new Image Search parameters.  However, the deconstructed URL gives back javascript which needs to be executed by a browser.  I looked briefly at Selenium and webdriver to do that, but couldn’t figure it out.

I also looked at Kooaba’s API, but they didn’t get back to me when I signed up for a free developer account, and their API only covers books, CDs and DVDs.  Also from a StackOverflow post, I looked at MacroGlossa and IQEngines.  Neither of them seemed to work–MacroGlossa wanted a category (and, shocker, vinyl record albums was not a category) and IQEngines let me submit an image wasn’t successful in identifying it.

I had to admit I was defeated.


Source Code Escrow

Have you ever considered asking a software vendor to put their source code in escrow? I recently broached the topic with a vendor I was evaluating. They didn’t seem too happy about the topic (at first, they were a bit sarcastic [“does Oracle do software escrow?”] and then they deleted tech support forum post). I did a bit of research, and there doesn’t seem to be a clear consensus on software escrow. Here are two interesting articles: Are you just following the herd? and Source Code Escrow.

Just having started thinking about the topic, my thoughts are still up in the air, but here’s my first reaction. Source code escrow makes sense when the following conditions are met:

  • The software is not open source (duh)
  • The software is or will be crucial to your business
  • The company is small or the future of the company is up in the air
  • You (the purchaser) have the technical capabilities to support the software should you receive it, or you’re willing to invest in those who can
  • You are willing to pay more for source that will be escrowed

I think of source code escrow like a warrenty on a car, or a professional license of open source software (like RedHat or Alfresco). It’s not for everyone (a car mechanic isn’t going to buy the warrenty) but it’s a nice option to have. And, to stretch a metaphor, if you’re going to soup up your car and build a livelihood on it, it’s nice to have a warrenty.


DualPhone 3088 Skype Phone: first impressions

AKA, the internet eats the phone system, installment one.

We recently bought a skype phone (having been inspired by this post).  We wanted to use our cell phones less (having been inspired by this book), and Skype seemed like an alternative to the local telco (Qwest).  Of course, it just increases our dependence on our local ISP (Comcast).  Gotta love broadband duopolies.

There was one requirement that made the phone relatively easy to choose.

  • It had to work without a computer being on.  I didn’t want to have to tie up a laptop or netbook for this phone.

This criteria limits you to about three phone models.  There are tons of phones that work as long as they are plugged into a computer (and they are cheaper too).  After spending some time wandering through Amazon.com’s catalog (a local store was also called, and they didn’t have anything), we selected the DualPhone 3088.  Here’s the Amazon page.

Impressions from using this phone for about a month.

  • It was easy to set up–I just created another skype account and set it up for some subscriptions.
  • The phone service is pretty good, though it has cut out a couple of times in use.  Of course, having AT&T as a cell phone provider, this is something we’re used to.
  • The phone feels cheap.  The handset is light and not quite full sized.

The economics are powerful, though.  A full subscription to skype (Skype In, so that others can call us, and Skype Out, so that we can call regular phone numbers) is around 9 bucks a month.  As we cut back our cell phone minutes, we’ll see whether a Skype phone is a full blown solution.


What to do when you get 504 and 500 errors from your Google Mini

I have a client who uses a Google Mini (a search appliance created by Google) as a key part of their business.  The appliance worked fine, gave great search results, and was happily humming along until very recently.  At some point over the last couple of months, the number of 504 errors visible to the users using the mini increased drastically.

At one point, no matter what term you searched on, you would see this error message:

Error: Unknown XML result type.

View page source to see the offending XML.

504

When I took a look at this to try to troubleshoot, the results were very arbitrary--some search terms would show this message, some would not.  However, when a search term did show the error message, it would do so consistently, at least for 5-10 minutes.  Perhaps that had to do with caching?

The benefit of the mini is that you get great google like search results.  The detriment is that it is a black box--you don't even get a login to the box, etc.  And when your support contract runs out, you're left with very few options.

So, we troubleshot with one hand tied behind our back.  We:

  • tried rebooting the box a couple of times (via the web interface)
  • considered rolling back the index, but we only had about 36 hours of index history
  • generated and analyzed the search logs
  • searched the Google Groups for mini support.  All that turned up was this post which was not exactly helpful, since we no longer had support

Finally, the client suggested resetting the index, as it was conceivable that the mini index had been somehow corrupted.  This was an option for the client because their documents are re-crawled every day.  If that wasn't the case, it would have been a harder decision.

Resetting the index solved the problem.  Some numbers:

  • a few months ago, 0.5% of search requests were returning either 500 or 504 errors.
  • just before the index was reset, 40.2% of the requests were either 500 or 504 errors.
  • now, 0% of requests are either 500 or 504 errors.

FYI, this post is applicable to the Google Mini version 4.6.4--I'm not sure how more current versions of software work/break.

Technorati Tags: , ,


Hadoop logging case study

Here’s a fascinating case study of the evolution of a (large!) log analysis toolset. It follows Rackspace as they strive to allow searching logfiles from an ever growing number of servers, starting from manual ssh/grep, through various incarnations of a MySQL database, and ending up using Hadoop/Lucene/Solr.  There’s (some) more info on the rackspace blog.  Cool stuff.

Via George Fairbanks.

Technorati Tags: , , ,


Solix website launch

I was able to help launch a revamped website on Monday.  Solix Biofuels is an algal oil producer moving from R&D into production. I’m not really a biology person, but I did enjoy reading about their technology.  To my layperson’s eye, algal energy sources seem much more sustainable than crop fuel sources.

It’s been a while since I’ve been the prime mover behind a deployment–there’s always a bit of nail biting when you finally reveal work to the world–but this one was fairly smooth.

Technorati Tags: , , ,


CleanPrint Installed on my blog

About six months ago, I met with some friends who I’d worked with in the past, at Format Dynamics.  They gave me a bit of software to install on my blog, and I installed it and forgot about it. But it is worth mentioning.
It is a wordpress plugin that interfaces with CleanPrint, which is a pretty cool piece of software (I’ve written about it in the past.)  Installation is as easy as typical WP plugin installation.  You can try it out by going to any individual wordpress page and looking for the ‘Print Blog’ button:

I printed my notes from the BDNT meetup, just as an example; you can download the PDF here: June 2009 New Tech Meetup Notes.pdf.  (PDF created using the excellent PDF995 adware.)

The other option for pretty printing, of course, is to create your own print.css.  I believe that the plan is to eventually be able to display ads on the printouts, with some revenue sharing agreement.

Technorati Tags: , ,




© Moore Consulting, 2003-2019