VoxEU Column Frontiers of economic research Productivity and Innovation

Taking advantage of the vast amount of data generated on the internet

The internet has reduced dramatically the cost of varying prices, displays, and information provided to consumers. This column discusses how this change enables both passive and active experimentation by retailers, and how this experimentation can be used by economic researchers in a way that takes advantage of the scale and heterogeneity of online markets.

Since the early days of the internet, it has been clear that the vast and detailed data being collected in online markets would provide opportunities to study consumer behaviour, to test theories of competition and market structure, and to analyse the effects of changes in search costs, product variety, and market organisation, all in relatively structured environments. While in theory the scale and diversity of many internet markets should be ideal for this purpose, in practice it has not always been easy to leverage these advantages. The difficulty has been to find empirical approaches that yield plausible identification of parameters of interest across a wide range of settings.

The two main approaches to studying online commerce have been observational studies that relate sales outcomes such as the auction price on eBay to differences in sale parameters such as reserve prices, shipping fees, or seller reputation (eg Bajari and Hortacsu 2004), and field experiments in which a researcher sells identical items under different conditions (eg Doleac and Stein 2010 on this site and Lucking-Reiley 1999). Both types of studies typically focus on a narrow product category – a particular type of coin or trading card, or a specific laptop or electronics component. Even then, observational studies can have a difficult time controlling for confounding differences between listings, such as the identity of the seller or the exact quality of the item. Field experiments help address these confounding issues, but often with small sample size. More importantly, focusing on narrowly defined products can lead to results that may be specific to a particular item or time window. An empirical approach that strips away the scale and diversity of online markets such as eBay, Amazon, or Google AdWords, exactly the features that make these markets remarkable, is somewhat disappointing.

In recent research (Einav et al 2011) we propose instead to leverage the experimentation of market participants. The idea is to search through the data from online markets for episodes in which firms have made targeted changes in their sales strategies, and then track the results of these changes. Our paper illustrates this idea using data from eBay’s large-scale online marketplace, which is a primary sales channel for tens of thousands of retailers. We define a “seller experiment” on eBay to be a case where a given seller lists a given item multiple times while making targeted changes to pricing or auction parameters. The scale of the data allows us to identify millions of these episodes, and to focus on cases where different offers were made sequentially or simultaneously, or by particular types of sellers or for particular types of items.

As an illustration, Figure 1 shows the eBay listings displayed following a search for “taylormade driver” (a type of golf club) on 12 September 2010. Within this narrow product category, it is already apparent that listings vary widely. The products themselves are differentiated, as are the sellers and the sales mechanisms, and shipping arrangements and fees. Even a quick perusal of the figure should make it clear that attributing patterns in the data to specific strategies or choices of sales mechanisms, even for a narrowly defined set of products, is a challenge.

Figure 1.                                                       

Figure 2 illustrates how a “seller experiment” can circumvent this concern. The figure shows listings from a seller who on the same day, 12 September 2010, had 31 listings for a specific TaylorMade driver. Of these, 20 were auctions that were scheduled to end in the next week, and 11 offered the driver for a fixed price of $124.99. Although it may or may not have been his intent, we can think of the seller as running a field experiment to test the relative performance of auctions and posted prices. Similar experiments can be found to investigate other aspects of pricing and sales strategy. For instance, in Figure 2, the displayed listings have two different shipping fees (either $7.99 or $9.99), offering an opportunity to see if shipping fees are internalised into auction prices, or whether buyers manage to choose the cheaper of two identical fixed price items.

Figure 2.

Of course, the amount that can be learned from a single seller and item is limited. The power of the approach comes from combining the evidence from many similar experiments. In our paper, and in ongoing work, we assemble data from several hundred thousand experiments similar to the one shown in Figure 2, and use the pooled data to investigate a range of old and new questions about consumer behaviour and auction design – in particular, to quantify the price dispersion across auctions for identical listings, to evaluate the hypothesis that consumers bid “excessively” in auctions to the point that they pay more than the item’s posted price, to measure the effect of auction reserve prices, to analyse the impact of “buy now” options in consumer auctions, and to assess whether consumers systematically underweight shipping fees.

Our findings sharpen, enrich, and in some cases overturn earlier results. For instance, we find substantial price variation across auctions for identical items conducted by the same seller, even when the auctions are held concurrently. But we observe relatively few instances of obvious overbidding, that is cases where a bidder pays more at auction than a concurrent posted price for the same item offered by the same seller. We find clear and consistent relationships between auction reserve prices, sale probabilities and closing prices, and trace out non-parametric “auction demand curves” that turn out to be at odds with the usual “regularity” assumption made in textbook auction models. We also confirm earlier findings that certain prices, such as shipping fees, are not fully internalised by buyers.

While the implementation of our approach is specific to the eBay data we have been using, we believe the general idea can be powerful well beyond one specific marketplace, however large it is. Advertisers in internet advertising markets such as the Google AdWords or AdSense markets, or Yahoo’s RightMedia exchange, regularly vary their bidding strategies or advertising content. Employers who regularly use online job matching sites, or traders in online exchanges, also may rely on forms of experimentation, whether conscious or otherwise. The key point is the low transaction costs of making targeted changes, and the scale and scope of available data in online settings. The amount of data being collected about economic activity is growing rapidly; learning from the experiments of others may be one way to make use of it.


Bajari, Patrick and Ali Hortacsu (2004), “Economic Insights from Internet Auctions”, Journal of Economic Literature, 42, 457-486.

Einav, Liran, Theresa Kuchler, Jonathan Levin, and Neel Sundaresan (2011), “Learning from Seller Experiments in Online Markets”, NBER Working Paper No. 17385.

Doleac, Jennifer and Luke CD Stein. (2010) “Race has a hand in determining market outcomes” VoxEU.org, 29 June.

Lucking-Reiley, David (1999), “Using Field Experiments to Test Equivalence Between Auction Formats: Magic on the Internet”, American Economic Review,89(5):1063-1080.

630 Reads