A taste for variety

A decision maker repeatedly chooses one of a finite set of actions. In each period, the decision maker's payoff depends on a fixed basic payoff of the chosen action and the frequency with which the action has been chosen in the past. We analyze optimal strategies associated with three types of evaluations of infinite payoffs: discounted present value, the limit inferior, and the limit superior of the partial averages. We show that when the first two are the evaluation schemes (and the discount factor is sufficiently high), a stationary strategy can achieve the best possible outcome. However, for the latter evaluation scheme, a stationary strategy can achieve the best outcome only if all actions that are chosen with strictly positive frequency by an optimal stationary strategy have the same basic payoff.