We propose the policy graph as a way of formulating multistage stochastic optimization problems. We also propose an extension to the stochastic dual dynamic programming algorithm to solve a class of problems formulated as a policy graph. This class includes discrete-time, infinite horizon, multistage stochastic optimization problems with continuous state and control variables.