Earlier this month, a Microsoft contest called BingItOn encouraged people to try out Bing vs Google in the same window — and decide which was better. Microsoft's claim was that the overall Bing:Google win ratio is 2:1. Our claim is that a comparison is pointless.

People quite enjoy comparing search results in an "objective" way, then pronouncing "The Best" across parameters. Less objective people simply root for their personal favourite in the same way they root for sports teams — which the folks at Bing noticed, and thought up the BingItOn contest. The event has brought in some PR for Microsoft — good and bad — and it's invited quite a bit of commentary (objective as well as just-for-fun). But here's why (IMHO) there's either no question of, or no need for, a comparison:

#1. "Relevant results" is a subjective matter. You can't have an objective ranking based on subjective judgments. This point is somewhat obvious, but think about it.

#2. The relevance of the phrase "Relevant results" is questionable: Very few of us are looking for one (or two) pages that we want to get to. The search term leads into an area of exploration, so to speak. To elaborate: We place considerable objective emphasis on relevance, but how often do we read the most relevant result page(s) — and stop the search there?

#3. On a results page (on most pages, actually!), we're used to scrolling through. We're close to having perfected the art of spotting what we want to see. Suppose the "best" search engine brings up "the most relevant" result at #5, and the other one puts it at #8. How much time do we typically lose when we scroll down by 3 results? 430 milliseconds?

#4. What we end up reading is guided, to a good extent, by what the search engines show us. This modern-day cyclicality pretty much throws Relevance out the window.

Come to think of it, "relevance in search results" — and results quality — is a hangover from 2001, when search engines sometimes went off track. "Google versus Bing" is, now, more a matter of whose results pages we enjoy.

Aside: Does anyone have an idea why BingItOn.com simply redirects to Bing.com in India?

Just-for-fun update: Here's a proper blooper on Bing's part. Type "bingiton" in Google, and the first result is bingiton.com. Type "bingiton" in Bing, and you're suggested: "Including results for binghamton. Do you want results only for bingiton?" The first page has only "binghamton", and no bingiton — dot com or otherwise. Probably because the contest isn't available in India, and the first result would redirect back to Bing. That amounts to surreptitious, self-promoting self-censorship. Hence:

Google: 1  |  Bing: Minus Infinity

Tags : TechTree Blog, Internet, Google, Microsoft, RMR