This paper introduces random versions of successive approximations and multigrid algorithms for computing approximate solutions to a class of finite and infinite horizon Markovian decision problems (MDPs). We prove that these algorithms succeed in breaking the curse of dimensionality for a subclass of MDPs known as discrete decision processes (DDPs).

This paper formulates a simple, regenerative, optimal-stopping model of bus-eng ine replacement to describe the behavior of Harold Zurcher, superinte ndent of maintenance at the Madison (Wisconsin) Metropolitan Bus Comp any. Admittedly, few people are likely to take particular interest in Harold Zurcher and bus engine replacement per se. The author focuses on a specific individual and capital good because it provides a simp le, concrete framework to illustrate two ideas: (1) a "bottom-up" a pproach for modeling replacement investment and (2) a "nested fixed point" algorithm for estimating dynamic programming models of discre te choice. Copyright 1987 by The Econometric Society.

[It is commonly believed that textbook publishers attempt to "kill off" competition from used textbooks through yearly edition changes. In the context of Wicksell's model of durable goods, Peter Swan has shown that such "planned obsolescence" is never optimal: a monopolist seller of durable goods maximizes profits by setting product durability equal to the competitive or socially optimal level, and efficiently extracts consumer surplus through sales price alone. This paper formulates a monopolist seller's choice of price and durability as the solution to a Stackelberg game between the monopolist and consumers. We employ a new equilibrium model of a durable goods market which, unlike Wicksell's model, recognizes that scrappage of durables is endogenously determined. We show that with endogenous scrappage, consumers have a substitution possibility which constrains the profits of a monopolist seller. This constraint on profits causes the monopolist to distort durability from the socially optimal level. We derive conditions under which this distortion takes its most extreme form: the monopolist kills off competition from used durables by producing new assets of zero durability.]

[This paper presents a dynamic model of consumer trading on the primary, secondary, and scrap markets for a stochastically deteriorating durable good in a stationary economy with perfect information and no transaction costs. We explicitly model the trading process by tracking each durable from its "birth" in the primary market, through its sequence of owners in the secondary market, until its "death" in the scrap market. We prove that a stationary equilibrium tests, characterize the distribution of consumer holdings of durables, and show that equilibrium asset prices are shadow prices to a particular regenerative optimal stopping problem. We show that each heterogeneous agent equilibrium is observationally equivalent to a homogeneous agent equilibrium. We derive a differential equation for equilibrium rental rates, and a functional equation which links rental rates to asset prices. These equations show precisely how the structure of durable prices and rental rates embody the functional form and population distribution of preferences and the technological characteristics of durable goods.]

This paper provides an empirical analysis of how the U.S. Social Security and Medicare insurance system affect the labor supply of older males in the presence of incomplete markets for loans, annuities, and insurance. We estimate a detailed dynamic programming (DP) model of the joint labor supply and Social Security acceptance decision, focusing on a sample of males in the low to middle income brackets whose only pension is Social Security. The DP model delivers a rich set of predictions about the dynamics of retirement behavior, and comparisons of actual vs. predicted behavior show that the DP model is able to account for wide variety of phenomena observed in the data, including the pronounced peaks in the distribution of retirement ages at 62 and 65 (the ages of early and normal eligibility for Social Security benefits, respectively). We identify a significant fraction of health insurance constrained individuals who have no form of retiree insurance other than Medicare, and who can only obtain fairly priced private insurance via their employer's group plan. The combination of significant individual risk aversion and a long tailed (Pareto) distribution of care expenditures implies that there is a significant security value for these individuals to remain employed until they are eligible for Medicare coverage at age 65. Overall, our model suggests that a number of heretofore puzzling aspects of retirement behavior can be viewed as artifacts of particular details of the Social Security rules, whose incentive effects are especially strong for lower income individuals and those who do not have access to fairly priced loans, annuities, and insurance.

We revisit the comparison of mathematical programming with equilibrium constraints (MPEC) and nested fixed point (NFXP) algorithms for estimating structural dynamic models by Su and Judd (SJ, 2012). Their implementation of the nested fixed point algorithm used successive approximations to solve the inner fixed point problem (NFXP-SA). We re-do their comparison using the more efficient version of NFXP proposed by Rust (1987), which combines successive approximations and Newton-Kantorovich iterations to solve the fixed point problem (NFXP- NK). We show that MPEC and NFXP are similar in speed and numerical performance when the more efficient NFXP-NK variant is used.