In this paper we study distributionally robust stochastic programming in a setting where there is a specified reference probability measure and the uncertainty set of probability measures consists of measures in some sense close to the reference measure. We discuss law invariance of the associated worst case functional and consider two basic constructions of such uncertainty sets. Finally we illustrate some implications of the property of law invariance.