This paper studies dynamic stochastic optimization problems parametrized by a random variable. Such problems arise in many applications in operations research and mathematical finance. We give sufficient conditions for the existence of solutions and the absence of a duality gap. Our proof uses extended dynamic programming equations, whose validity is established under new relaxed conditions that generalize certain no-arbitrage conditions from mathematical finance.