Machine learning for demand estimation in long tail markets
File(s)long_tail_revision2.pdf (947.12 KB)
Accepted version
Author(s)
Adam, Hammaad
He, Pu
Zheng, Fanyin
Type
Journal Article
Abstract
Random coefficient multinomial logit models are widely used to estimate customer preferences from sales data. However, these estimation models can only allow for products with positive sales; this selection leads to highly biased estimates in long tail markets, that is, markets where many products have zero or low sales. Such markets are increasingly common in areas such as online retail and other online marketplaces. In this paper, we propose a two-stage estimator that uses machine learning to correct for this bias. Our method first uses deep learning to predict the market shares of all products, where the neural network’s structure mirrors the random coefficient multinomial logit model’s data generating process. In the second stage, we use the predictions of the first stage to reweight the observed shares in a way that corrects for the induced bias and maintains the causal interpretation of the structural model. We show that the estimated parameters are consistent in the number of markets. Our method performs well on simulated and real long tail data, producing accurate estimates of customer behavior. These improved estimates can subsequently be used to provide prescriptive policy recommendations on important managerial decisions such as pricing, assortment, and so on.
Date Issued
2024-08-01
Date Acceptance
2022-12-27
Citation
Management Science, 2024, 70 (8), pp.5040-5065
ISSN
0025-1909
Publisher
Institute for Operations Research and Management Sciences
Start Page
5040
End Page
5065
Journal / Book Title
Management Science
Volume
70
Issue
8
Copyright Statement
© 2023, INFORMS.
Identifier
https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:001085188900001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=a2bf6146997ec60c407a63945d4e92bb
Subjects
Business & Economics
COMPETITION
economics: econometrics
Management
MODELS
Operations Research & Management Science
PRICES
Science & Technology
Social Sciences
Technology
utility-preference: choice functions
utility-preference: estimation
Publication Status
Published
Date Publish Online
2023-10-10