A Solution to Optimal Control of Linear Systems with Unknown Parameters - Chow, Gregory C. (The Review of Economics and Statistics, 1975)
N the study of optimal economic policy using a linear econometric model and a quadratic welfare function the parameters of the model are often assumed to be known for certain. Under this assumption the solution in the form of an optimal feedback equation can be obtained easily. Although one recognizes that in a realistic situation the parameters of an econometric model are never known for certain, he might still apply the above solution, using a set of estimates of the parameters as if they were the true values, if he believes that it is a good approximation to optimal policy. Such a procedure is well known to be a certainty equivalence solution. In a recent paper (Chow, 1973b), I have presented a method of obtaining the optimal feedback equations and the associated welfare costs by allowing for uncertainty in the parameters as expressed in the posterior density function computed from data available at the time of the current decision but not for possible future revision of this posterior density in the derivation of the current policy. Because future learning about the model is not explicitly taken into account in the design of the current policy, the above method is not truly optimal. However if the sample period is long as compared with the planning period, this method will probably be close to being optimal. The purposes of this paper are to present an approximate solution to optimal when learning is taken into account and to contrast this solution with the first two solutions. In the literature, the term control is used for a problem having the dual purpose of improving the system performance and of learning more about the system for the sake of future control. Numerous approximate solutions to this problem have been suggested.' The solution of this paper appears to be the simplest in conception, and yet it incorporates all theoretical elements in the calculations. It contains a logical structure which brings out clearly the effect of learning on the optimization process and enables the effect to be measured numerically. It provides useful contrasts to the certainty equivalence solution and the solution for unknown parameters without learning, being a natural generalization of these two solutions. We will set up the problem and describe the method of solution in section II. This method will be compared in section III with the two other methods just mentioned, both in conceptual terms and in terms of computations. Two simpler, modified versions of the method will also be briefly described. They are simpler to compute but they still take learning partially into account. Some numerical results using a simple one-equation model will be presented in section IV to bring out the effects of learning on the optimal solution. This paper is confined mainly to presenting the method and providing some illustrative calculations. A comprehensive study of the effect of learning on optimal policies using the method of this paper remains to be undertaken. From the viewpoint of economics in general, other than the study of quantitative economic policy using econometric models, the content of this paper may also be relevant. Maximization is in the heart of economics. Most of economic theory assumes maximization to take place in Received for publication December 26, 1973. Revision accented for publication July 8, 1974. * I am much indebted to Andrew Abel for extremely able research assistance, to Edison Tse, Ray C. Fair and several members of the Econometric Research Program seminar at Princeton for valuable suggestions and discussions, to a referee for comments on an early draft, and to the National Science Foundation for financial support through Grant GS32003X. 1 The references in the literature are too numerous to cite. In the economics literature, Prescott (1972) deals with the problem of learning using a very simple model but provides no new method of solution; its results were computed by complete enumeration. MacRae (1972) and Tse (1974) provide interesting approximations to the optimal solution and are highly recommended to the reader.