Discounted stochastic games, the 3M property and stationary Markov perfect equilibria

We show that all discounted stochastic games DSGs satisfying the usual assumptions have Nash payoff selection correspondences having fixed points. Our fixed point result is surprising because it is well known that Nash payoff selection correspondences are badly behaved, being in general neither convex valued nor closed valued in the appropriate topologies (in this case the weak star topologies). Here we show that because all DSGs satisfying the usual assumptions have upper Caratheodory (uC) Nash (equilibrium) correspondences containing uC Nash sub-correspondences having the 3M property (defined here), these uC Nash sub-correspondences are continuum valued and therefore induce interval-valued uC player payoff sub-correspondences - and therefore, Caratheodory approximable uC player payoff sub-correspondences. Finally, because these uC player payoff sub-correspondences are Caratheodory approximable, their induced Nash payoff selection sub-correspondences have fixed points - implying that the DSGs to which they belong have stationary Markov perfect equilibria.