Switching Costs and the Gittins Index
The Theorem of Gittins and Jones (1974) is, perhaps, the single most powerful result in the literature on Bandit problems. This result establishes that in independent-armed Bandit problems with geometric discounting over an infinite horizon, all optimal strategies may be obtained by solving a family of simple optimal stopping problems that associate with each arm an index known as the dynamic allocation index or, more popularly, as the Gittins index. Importantly, the Gittins index of an arm depends solely on the characteristics of that arm and the rate of discounting, and is otherwise completely independent of the problem under consideration. These features simplify significantly the task of characterizing optimal strategies in this class of problems.