sddp

Policy with guaranteed risk-adjusted performance for multistage stochastic linear problems

  • Bernardo Freitas Paulo da Costa
  • Vincent Leclère
    • Categories Convex Optimization, Dynamic Programming, Stochastic Programming Tags , , ,

    Risk-averse multi-stage problems and their applications are gaining interest in various fields of applications. Under convexity assumptions, the resolution of these problems can be done with trajectory following dynamic programming algorithms like Stochastic Dual Dynamic Programming (SDDP) to access a deterministic lower bound, and dual SDDP for deterministic upper bounds. In this paper, we leverage … Read more

    A Multicut Approach to Compute Upper Bounds for Risk-Averse SDDP

  • Joaquim Dias Garcia
  • Mario Pereira
    • Categories Dynamic Programming, Energy, Stochastic Programming Tags , , , ,

    Stochastic Dual Dynamic Programming (SDDP) is a widely used and fundamental algorithm for solving multistage stochastic optimization problems. Although SDDP has been frequently applied to solve risk-averse models with the Conditional Value-at-Risk (CVaR), it is known that the estimation of upper bounds is a methodological challenge, and many methods are computationally intensive. In practice, this … Read more

    Sampling-based Decomposition Algorithms for Multistage Stochastic Programming

  • Harsha Gangammanavar
    • Categories Stochastic Programming Tags , , ,

    Sampling-based algorithms provide a practical approach to solving large-scale multistage stochastic programs. This chapter presents two alternative approaches to incorporating sampling within multistage stochastic linear programming algorithms. In the first approach, sampling is used to construct a sample average approximation (SAA) of the true multistage program. Subsequently, an optimization step is undertaken using deterministic decomposition … Read more

    Convergence of Trajectory Following Dynamic Programming algorithms for multistage stochastic problems without finite support assumptions

  • Maël Forcier
  • Vincent Leclère
    • Categories Dynamic Programming, Stochastic Programming Tags , ,

    We introduce a class of algorithms, called Trajectory Following Dynamic Programming (TFDP) algorithms, that iteratively refines approximation of cost-to-go functions of multistage stochastic problems with independent random variables. This framework encompasses most variants of the Stochastic Dual Dynamic Programming algorithm. Leveraging a Lipschitz assumption on the expected cost-to-go functions, we provide a new convergence and … Read more

    Risk-Averse Stochastic Optimal Control: an efficiently computable statistical upper bound

  • Vincent Guigues
  • Alexander Shapiro
  • Yi Cheng
    • Categories Stochastic Programming Tags , , , , ,

    In this paper, we discuss an application of the SDDP type algorithm to nested risk-averse formulations of Stochastic Optimal Control (SOC) problems. We propose a construction of a statistical upper bound for the optimal value of risk-averse SOC problems. This outlines an approach to a solution of a long standing problem in that area of … Read more

    Dual SDDP for risk-averse multistage stochastic programs

  • Bernardo Freitas Paulo da Costa
  • Vincent Leclère
    • Categories Dynamic Programming, Stochastic Programming Tags , , , ,

    Risk-averse multistage stochastic programs appear in multiple areas and are challenging to solve. Stochastic Dual Dynamic Programming (SDDP) is a well-known tool to address such problems under time-independence assumptions. We show how to derive a dual formulation for these problems and apply an SDDP algorithm, leading to converging and deterministic upper bounds for risk-averse problems. … Read more

    Batch Learning in Stochastic Dual Dynamic Programming

  • Daniel Ávila
  • Nils Löhndorf
  • Anthony Papavasiliou
    • Categories Dynamic Programming, Stochastic Programming Tags , , , ,

    We consider the stochastic dual dynamic programming (SDDP) algorithm, which is a widely employed algorithm applied to multistage stochastic programming, and propose a variant using batch learning, a technique used with success in the reinforcement learning framework. We cast SDDP as a type of Q-learning algorithm and describe its application in both risk neutral and … Read more

    Stochastic Dual Dynamic Programming for Multistage Stochastic Mixed-Integer Nonlinear Optimization

  • Shixuan Zhang
  • Xu Andy Sun
    • Categories (Mixed) Integer Nonlinear Programming, Global Optimization, Stochastic Programming Tags , , ,

    In this paper, we study multistage stochastic mixed-integer nonlinear programs (MS-MINLP). This general class of problems encompasses, as important special cases, multistage stochastic convex optimization with \emph{non-Lipschitz-continuous} value functions and multistage stochastic mixed-integer linear optimization. We develop stochastic dual dynamic programming (SDDP) type algorithms with nested decomposition, deterministic sampling, and stochastic sampling. The key ingredient … Read more

    Inexact cuts in Stochastic Dual Dynamic Programming

  • Vincent Guigues
    • Categories Stochastic Programming Tags , , , ,

    We introduce an extension of Stochastic Dual Dynamic Programming (SDDP) to solve stochastic convex dynamic programming equations. This extension applies when some or all primal and dual subproblems to be solved along the forward and backward passes of the method are solved with bounded errors (inexactly). This inexact variant of SDDP is described both for … Read more

    Stochastic dual dynamic programming with stagewise dependent objective uncertainty

  • Regan Baucke
  • Anthony Downward
  • Oscar Dowson
    • Categories Stochastic Programming Tags , , , ,

    We present a new algorithm for solving linear multistage stochastic programming problems with objective function coefficients modeled as a stochastic process. This algorithm overcomes the difficulties of existing methods which require discretization. Using an argument based on the finiteness of the set of possible cuts, we prove that the algorithm converges almost surely. Finally, we … Read more