Scalable Estimation of Multinomial Response Models with Uncertain Consideration Sets

A standard assumption in the fitting of unordered multinomial response models for J mutually exclusive nominal categories, on cross-sectional or longitudinal data, is that the responses arise from the same set of J categories between subjects. However, when responses measure a choice made by the subject, it is more appropriate to assume that the distribution of multinomial responses is conditioned on a subject-specific consideration set, where this consideration set is drawn from the power set of {1,2,...,J}. Because the cardinality of this power set is exponential in J, estimation is infeasible in general. In this paper, we provide an approach to overcoming this problem. A key step in the approach is a probability model over consideration sets, based on a general representation of probability distributions on contingency tables. Although the support of this distribution is exponentially large, the posterior distribution over consideration sets given parameters is typically sparse, and is easily sampled as part of an MCMC scheme that iterates sampling of subject-specific consideration sets given parameters, followed by parameters given consideration sets. The effectiveness of the procedure is documented in simulated longitudinal data sets with J=100 categories and real data from the cereal market with J=73 brands.