Tell me if you find this ironic: do a search on Google for natural search and you find nothing but listings for SEO firms and tips and tricks for improving your natural search rank (sidebar - would you really want to hire a firm that doesn't make the first page of this search?). Ok, ok, I recognize that these results likely match the searcher's interests quite well, but the fact that natural search is increasingly becoming anything but natural has been a topic frequently on my mind as of late.
Over the last couple of years we have experienced a wide-spread awakening to the reality that, with a little work, one can significantly influence how search engines rank our sites. I don't think the impact of this knowledge on our experience as search consumers is yet clear.
On the good side, one could argue that efforts to manipulate natural search gave birth to what we now call web 2.o - we discovered that search engines love content, but content development is time consuming and therefore expensive, so we all seem to have decided to see if we could get our community of users to develop content for us - turns out the answer is a strong yes (at least for now). Hard to argue that the recent explosion of content on the web is anything but a great thing (well, those of you reading this may of course be presently reconsidering that opinion).
However, is content developed purely for the purpose of influencing search results truly valuable? Consider the results of this search. While eBay does of course have a great marketplace for Cellular Test Equipment, I wrote the text that causes this link to show up first on Google entirely for the purpose of improving our natural search rank. As brilliant as my prose obviously is, it is hard to argue that this content will be valuable to search consumers.
One argument says that those with the greatest economic incentives will invest the most resources in developing the best content for a particular search query and that therefore the free market will prevail in pushing the most useful results to the top. I consider myself as much of a free market believer as anyone, but I am not completely convinced that this is true. One of the primary elements of Google search algorithm, PageRank, can throw this off in a big way. If Joe's Electronics Gear was the best place on the web to buy Cellular Test Equipment, they could write content all day long and with a little work on my end, I could leverage eBay's PageRank to make sure that I was always listed ahead of him. That strikes me as a problem.
As members of the web's "high PageRank" establishment continue to invest in mastering natural search manipulation, it is hard to imagine that search consumers will be able to enjoy a search experience that yields the most desirable results. While I typically assume that when people say that Google results seem to be getting worse that they are just picking on the big guy - it seems likely that as natural search becomes less and less natural, the experience of search consumers is bound to suffer.
Interestingly, while the obvious answer may be to refine the ranking algorithm without revealing information about how to adjust to the refinements, I presume that this would be a particularly difficult decision for the search incumbents to make. Why? Well, largely speaking, the folks with the highest PageRank also happen to be the biggest search advertisers. Yes, pushing big advertisers down the page in natural search results might increase the need for them to spend on advertising, but a decision of this nature may also be met with quite a negative reaction from big customers that see the search engines as "partners". Funny how this starts to sound like exactly what Clayton Christensen would predict. Incumbents can't help but listen closely to their big customers. (From what Ro says, it sounds like this is already happening with the Google/AOL deal). This opens the door for disruptors to sneak in with a better product - perhaps a product that is not initially the most attractive to the incumbent's big customers, but is attractive to a segment of users.
Wouldn't surprise me at all.
Yet another reason why I really think it is much too early in all of this to take the current search leaders as a given for the future.