Dion Almaer points to an interesting article about search engine indices, proposing they be shared across search engines. The article is worth a read, but the points are:
- bots use a tremendous amount of bandwidth across the internet
- they all get the same pages; the value is in their algorithms
- therefore, the capacity of the internet would be increased if a company were to make their index available for all search engines to use
- Google has the largest index, therefore it might be best if they did the sharing
I think there are a couple of issues with this scenario, but the largest is that having the biggest index is a competitive advantage for Google–why would they give that up merely to free up internet bandwidth that isn’t costing Google a penny? Stephen O’Grady succinctly sums up this issue.
In addition, I’m sure that any search engine company that was basing their results on an index would want to have a hand in controlling the index; Dion later suggests a third party index, but I’m not sure where the impetus comes for that.
I think this idea falls under the “wouldn’t it be nice” category, but don’t see any way to get from here to there. Good intellectual exercise, though.