Archives For Internet search

Former TOTM blog symposium participant Joshua Gans (visiting Microsoft Research) has a post at TAP on l’affair hiybbprqag, about which I blogged previously here.

Gans notes, as I did, that Microsoft is not engaged in wholesale copying of Google’s search results, even though doing so would be technologically feasible.  But Gans goes on to draw a normative conclusion:

Let’s start with “imitation,” “copying” and its stronger variants of “plagiarism” and “cheating.” Had Bing wanted to do this and directly map Google’s search results onto its own, it could have done it. It could have set up programs to enter terms in Google and skimmed off the results and then used them directly. And I think we can all agree that that is wrong. Why? Two reasons. First, if Google has invested to produce those results, if others can just hang off them and copy it, Google’s may not earn the return on its efforts it should do. Second, if Bing were doing this and representing itself as a different kind of search, then that misrepresentation would be misleading. Thus, imitation reduces Google’s reward for innovation while adding no value in terms of diversity.

His first reason why this would be wrong is . . . silly.  I mean, I don’t want to get into a moral debate, but since when is it wrong to engage in activity that “may” hamper another firm’s ability to earn the return on its effort that it “should” (whatever “should” means here)?  I always thought that was called “competition” and we encouraged it.  As I noted the other day, competition via imitation is an important part of Schumpeterian capitalism.  To claim that reducing another company’s profits via imitation is wrong, but doing so via innovation is good and noble, is to hang one’s hat on a distinction that does not really exist.

The second argument, that doing so would amount to misrepresentation, is possible, but I’m sure if Microsoft were actually just copying Google’s results their representations would look different than they do now and the problem would probably not exist, so this claim is speculative, at best.

Now, regardless, I doubt it would be profitable for Microsoft to copy Google wholesale, and this is basically just a red herring (as Gans understands–he goes on to discuss the more “innocuous” imitation at issue).  While I think Gans’ claims that it would be “wrong” are just hand waiving, I am confident it would be “wrong” from the point of view of Microsoft’s bottom line–or else they would already be doing it.  In this context, that would seem to be the only standard that matters, unless there were a legal basis for the claim.

On this score, Gans points us to Shane Greenstein (Kellogg).  Greenstein writes:

Let’s start with a weak standard, the law. Legally speaking, imitation is allowed so long as a firm does not violate laws governing patents, copyright, or trade secrets. Patents obviously do not apply to this situation, and neither does copyright  because Google does not get a copyright on a search result. It also does not appear as if Googles trade secrets were violated. So, generally speaking, it does not appear as if any law has been broken.

This is all well and good, but Greenstein goes on to engage in his own casual moralizing, and his comments are worth reproducing (imitating?) at some length:

The norms of rivalry

There is nothing wrong with one retailer walking through a rival’s shop and getting ideas for what to do. There is really nothing wrong with a designer of a piece of electronic equipment buying a rival’s product and studying it in order to get new ideas for a  better design. 

In the modern Internet, however, there is no longer any privacy for users. Providers want to know as much as they can, and generally the rich suppliers can learn quite a lot about user conduct and preferences.

That means that rivals can learn a great deal about how users conduct their business, even when they are at a rival’s site. It is as if one retailer had a camera in a rival’s store, or one designer could learn the names of the buyer’s of their rival’s products, and interview them right away.

In the offline world, such intimate familiarity with a rival’s users and their transactions would be uncomfortable. It would seem like an intrusion on the transaction between user and supplier. Why is it permissible in the online world? Why is there any confusion about this being an intrusion in the online world? Why isn’t Microsoft’s behavior seen — cut and dry — as an intrusion?

In other words, the transaction between supplier and user is between supplier and user, and nobody else should be able to observe it without permission of both supplier and user. The user alone does not have the right or ability to invite another party to observe all aspects of the transaction.

That is what bothers me about Bing’s behavior. There is nothing wrong with them observing users, but they are doing more than just that. They are observing their rival’s transaction with users. And learning from it. In other contexts that would not be allowed without explicit permission of both parties — both user and supplier.

Moreover, one party does not like it in this case, as they claim the transaction with users as something they have a right to govern and keep to themselves. There is some merit in that claim.

In most contexts it seems like the supplier’s wishes should be respected. Why not online? (emphasis mine)

Where on Earth do these moral standards come from?  In what way is it not “allowed” (whatever that means here) for a firm to observe and learn from a rival’s transactions with users?  I can see why the rival would prefer it to be otherwise, of course, but so what?  They would also prefer to eradicate their meddlesome rival entirely, if possible (hence Microsoft’s considerable engagement with antitrust authorities concerning Google’s business), but we hardly elevate such desires to the realm of the moral.

What I find most troublesome is the controlling, regulatory mindset implicit in these analyses.  Here’s Gans again:

Outright imitation of this type should be prohibited but what do we call some more innocuous types? Just look at how the look and feel of the iPhone has been adopted by some mobile software developers just as the consumer success of graphic based interfaces did in an earlier time. This certainly reduces Apple’s reward for its innovations but the hit on diversity is murkier because while some features are common, competitors have tried to differentiate themselves. So this is not imitation but it is something more common, leveraging without compensation and how you feel about it depends on just how much reward you think pioneers should receive.

It is usually politicians and not economists (other than politico-economists like Krugman) who think they have a handle on–and an obligation to do something about–things like “how much reward . . .pioneers should receive.”  I would have thought the obvious answer to the question would be either “the optimal amount, but good luck knowing what that is or expecting to find it in the real world,” or else, for the Second Best, “whatever the market gives them.”  The implication that there is some moral standard appreciable by human mortals, or even human economists, is a recipe for disaster.

One of my favorite stories in the ongoing saga over the regulation (and thus the future) of Internet search emerged earlier this week with claims by Google that Microsoft has been copying its answers–using Google search results to bolster the relevance of its own results for certain search terms.  The full story from Internet search journalist extraordinaire, Danny Sullivan, is here, with a follow up discussing Microsoft’s response here.  The New York Times is also on the case with some interesting comments from a former Googler that feed nicely into the Schumpeterian competition angle (discussed below).  And Microsoft consultant (“though on matters unrelated to issues discussed here”)  and Harvard Business prof Ben Edelman coincidentally echoes precisely Microsoft’s response in a blog post here.

What I find so great about this story is how it seems to resolve one of the most significant strands of the ongoing debate–although it does so, from Microsoft’s point of view, unintentionally, to be sure.

Here’s what I mean.  Back when Microsoft first started being publicly identified as a significant instigator of regulatory and antitrust attention paid to Google, the company, via its chief competition counsel, Dave Heiner, defended its stance in large part on the following ground:

All of this is quite important because search is so central to how people navigate the Internet, and because advertising is the main monetization mechanism for a wide range of Web sites and Web services. Both search and online advertising are increasingly controlled by a single firm, Google. That can be a problem because Google’s business is helped along by significant network effects (just like the PC operating system business). Search engine algorithms “learn” by observing how users interact with search results. Google’s algorithms learn less common search terms better than others because many more people are conducting searches on these terms on Google.

These and other network effects make it hard for competing search engines to catch up. Microsoft’s well-received Bing search engine is addressing this challenge by offering innovations in areas that are less dependent on volume. But Bing needs to gain volume too, in order to increase the relevance of search results for less common search terms. That is why Microsoft and Yahoo! are combining their search volumes. And that is why we are concerned about Google business practices that tend to lock in publishers and advertisers and make it harder for Microsoft to gain search volume. (emphasis added).

Claims of “network effects” “increasing returns to scale” and the absence of “minimum viable scale” for competitors run rampant (and unsupported) in the various cases against Google.  The TradeComet complaint, for example, claims that

[t]he primary barrier to entry facing vertical search websites is the inability to draw enough search traffic to reach the critical mass necessary to become independently sustainable.

But now we discover (what we should have known all along) that “learning by doing” is not the only way to obtain the data necessary to generate relevant search results: “Learning by copying” works, as well.  And there’s nothing wrong with it–in fact, the very process of Schumpeterian creative destruction assumes imitation.

As Armen Alchian notes in describing his evolutionary process of competition,

Neither perfect knowledge of the past nor complete awareness of the current state of the arts gives sufficient foresight to indicate profitable action . . . [and] the pervasive effects of uncertainty prevent the ascertainment of actions which are supposed to be optimal in achieving profits.  Now the consequence of this is that modes of behavior replace optimum equilibrium conditions as guiding rules of action. First, wherever successful enterprises are observed, the elements common to these observable successes will be associated with success and copied by others in their pursuit of profits or success. “Nothing succeeds like success.”

So on the one hand, I find the hand wringing about Microsoft’s “copying” Google’s results to be completely misplaced–just as the pejorative connotations of “embrace and extend” deployed against Microsoft itself when it was the target of this sort of scrutiny were bogus.  But, at the same time, I see this dynamic essentially decimating Microsoft’s (and others’) claims that Google has an unassailable position because no competitor can ever hope to match its size, and thus its access to information essential to the quality of search results, particularly when it comes to so-called “long-tail” search terms.

Long-tail search terms are queries that are extremely rare and, thus, for which there is little user history (information about which results searchers found relevant and clicked on) to guide future search results.  As Ben Edelman writes in his blog post (linked above) on this issue (trotting out, even while implicitly undercutting, the “minimum viable scale” canard):

Of course the reality is that Google’s high market share means Google gets far more searches than any other search engine. And Google’s popularity gives it a real advantage: For an obscure search term that gets 100 searches per month at Google, Bing might get just five or 10. Also, for more popular terms, Google can slice its data into smaller groups — which results are most useful to people from Boston versus New York, which results are best during the day versus at night, and so forth. So Google is far better equipped to figure out what results users favor and to tailor its listings accordingly. Meanwhile, Microsoft needs additional data, such as Toolbar and Related Sites data, to attempt to improve its results in a similar way.

But of course the “additional data” that Microsoft has access to here is, to a large extent, the same data that Google has.  Although Danny Sullivan’s follow up story (also linked above) suggests that Bing doesn’t do all it could to make use of Google’s data (for example, Bing does not, it seems, copy Google search results wholesale, nor does it use user behavior as extensively as it could (by, for example, seeing searches in Google and then logging the next page visited, which would give Bing a pretty good idea which sites in Google’s results users found most relevant)), it doesn’t change the fundamental fact that Microsoft and other search engines can overcome a significant amount of the so-called barrier to entry afforded by Google’s impressive scale by simply imitating much of what Google does (and, one hopes, also innovating enough to offer something better).

Perhaps Google is “better equipped to figure out what users favor.”  But it seems to me that only a trivial amount of this advantage is plausibly attributable to Google’s scale instead of its engineering and innovation.  The fact that Microsoft can (because of its own impressive scale in various markets) and does take advantage of accessible data to benefit indirectly from Google’s own prowess in search is a testament to the irrelevance of these unfortunately-pervasive scale and network effect arguments.