Over the past two decades, validation of choice models has focused on predictive validity rather than parameter bias. In real-world validation of choice models, true parameter values are unknown, so examination of parameter bias is not possible. In contrast, the main focus of this study is parameter bias in simulated scanner-panel choice data with known parameter values. Study of parameter bias enables the assessment of a fundamental issue not addressed in the choice modeling literature—the extent to which the logit choice model is capable of distinguishing unobserved effects that give rise to persistence in observed choices (e.g., heterogeneity and state dependence). Although econometric theory provides some information about the causes of bias, the extent of such bias in typical scanner data applications remains unclear. The authors present an extensive simulation study that provides information on the extent of bias resulting from the misspecification of four unobserved effects that receive frequent attention in the literature—choice set effects, heterogeneity in preferences and market response, state dependence, and serial correlation. The authors outline implications for model builders and managers. In general, the potential for parameter bias in choice model applications appears to be high. Overall, a logit model with choice set effects and the Guadagni–Little loyalty variable produces the most valid parameter estimates.

Random coefficients choice models are seeing widespread adoption in marketing research, partly because of their ability to generate household-level parameter estimates with limited data. However, the power of such models may tempt researchers to trust that they continue to produce reasonable estimates, when in fact either model misspecification or insufficient data limits the models' ability to recover household-level parameters successfully. If household-level choice behaviors are not recovered successfully, managerial decisions such as marketing-mix planning and targeting, direct marketing, segmentation, and forecasting may not produce the desired results. This study addresses the following questions. First, can random coefficients choice models correctly identify markets characterized by preference and response heterogeneity, state dependence, the use of alternative decision heuristics that result in reduced choice sets, and combinations of these effects? If so, how much data is required, and is this realistic given the size of data sets typically used in marketing analyses? Which model selection criteria should be used to identify these markets? When there is spurious market identification, which parameters contribute to the spurious result? An extensive simulation experiment is conducted wherein random coefficients logit models with varying specifications of parameter heterogeneity, state dependence effects, and choice set heterogeneity are applied to 128 experimental conditions. The results show which types of markets can be identified reliably and which cannot. Based on the results of the simulation, the authors develop a model selection heuristic that identifies the correct market in 81% of the experimental conditions. In contrast, strict application of the best model selection criterion alone results in correct market identification in at most 34% of experimental conditions. Interestingly, we find that the amount of data (number of households or number of purchases per household) does not affect our ability to identify the correct market type with this heuristic, so there is a good chance of identifying the correct market type even when little data is available.