Concentration in Web searching raises worries about fairness

During the last five years, the internet search market has consolidated into a duopoly, with very few alternatives to the two dominating services. That -- and not Microsofts current proposal to buy Yahoo -- is the real big issue for the average internet user.

Eric Schmidt, CEO Google CEO, recently said that a merger between Yahoo and Microsoft might harm the openness of the internet and the free flow of information.

But in reality, it is hard to see how the proposed merger would change the present situation significantly. Since Microsoft launched its proposal to acquire Yahoo in February, Yahoo’s management have tried to fend off the unwanted suitor. Their main argument, as repeated in a business presentation on Tuesday, is simple:

Google is big, so are we and Microsoft is far behind, and that will make us extremely profitable in the future. (Which of course is just the reason why Microsoft’s management wants to buy the company.)

If Jerry Yang’s estimates of market share are correct, the two leadings search engines already have a nearly total dominance. Which means that we have a duopoly in place, Microsoft merger or not.

That could be bad news, some say, for new approaches to navigating the web.

“I think it is worrisome, primarily because it might stifle innovation," says James Hendler, professor at Rensselaer Polytechnic in New York. "We have seen that happening before in operating systems, web browsers. I don’t think it will change our view of the world, but it means that a lot of new approaches in search that won’t get tried.”

In his own research, Mr. Hendler aims at making information on the web searchable in new ways.

“People who are trying to launch new search engines are going to have a very hard time," he says, given the concentration in the industry.

So, why would anyone want a new search engine? Well, you can regard the databases and the search algorithms used as a representation of the web, in the same manner as the photo is a representation of a piece of the physical reality. Change the angle, the lightning or the focus and the character of the picture changes. In the same way, there are differences in how different search engines find and rank pages, in their perspective of the web so to say.

One example of a different approach to search is Mahalo, an attempt to create a service based on input from users, in a manner reminding of Wikipedia. At least in theory, human-powered search could give quite different results from computerized search.

Another issue is "trust." The big search engines use their own proprietary technology to find and rank web pages. So how do you know that the results are fair?

Well, you don’t, some say.

“In particular, since there are no regulatory agencies watching how search engines determine which result should be number one and which should be number 393,551, what happens if that's not actually fair?” asks Dave Taylor, a Colorado-based blogger.

Mr. Taylor's concerns are shared by Mr. Hendler who opines, “It is easy to manipulate searches. Google has chosen to differentiate between sponsord search results from non-sponsored. But we only have their word for it.”

“You have to trust Google or Microsoft," he adds. "But in the end there is the question, why am I trusting them? And if I don’t trust them, where will I go if there is only one or two of them?”