Parallel Experimentation on Advertising Platforms

This paper studies the measurement of advertising effects on online platforms when parallel experimentation occurs, that is, when multiple advertisers experiment concurrently. It provides a framework that makes precise how parallel experimentation affects this measurement problem: while ignoring parallel experimentation yields an estimate of the average effect of advertising in-place, this estimate has limited value in decision-making in an environment with advertising competition; and, account for parallel experimentation provides a richer set of advertising effects that capture the true uncertainty advertisers face due to competition. It then provides an experimental design that yields data that allow advertisers to estimate these effects and implements this design on JD.com, a large e-commerce platform that is also a publisher of digital ads. Using traditional and kernel-based estimators, it obtains results that empirically illustrate how these effects can crucially affect advertisers' decisions. Finally, it shows how competitive interference can be summarized via simple metrics that can assist decision-making.