Stability of Markovian Stochastic Programming

Multi-stage stochastic programming is notoriously hard, since solution methods suffer from the curse of dimensionality. Recently, stochastic dual dynamic programming has shown promising results for Markovian problems with many stages and a moderately large state space. In order to numerically solve these problems simple discrete representations of Markov processes are required but a convincing theoretical … Read more

Water resources management: A bibliometric analysis and future research directions

The stochastic dual dynamic programming (SDDP) algorithm introduced by Pereira and Pinto in 1991 has sparked essential research in the context of water resources management, mainly due to its ability to address large-scale multistage stochastic problems. This paper aims to provide a tutorial-type review of 32 years of research since the publication of the SDDP … Read more

Numerical Methods for Convex Multistage Stochastic Optimization

\(\) Optimization problems involving sequential decisions in  a  stochastic environment    were studied  in  Stochastic Programming (SP), Stochastic Optimal Control  (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and  SOC modelling   approaches. In these frameworks there are natural situations  when the considered problems are  convex. Classical approach to sequential … Read more

A Quasi-Newton Algorithm for Optimal Discretization of Markov Processes

In stochastic programming and stochastic-dynamic programming discretization of random model parameters is often unavoidable. We propose a quasi-Newton learning algorithm to discretize multi-dimensional, continuous discrete-time Markov processes to scenario lattices by minimizing the Wasserstein distance between the unconditional distributions of process and lattice. Scenario lattices enable accurate discretization of the conditional distributions of Markov processes … Read more

Data-Driven Stochastic Dual Dynamic Programming: Performance Guarantees and Regularization Schemes

We propose a data-driven scheme for multistage stochastic linear programming with Markovian random parameters by extending the stochastic dual dynamic programming (SDDP) algorithm. In our data-driven setting, only a finite number of historical trajectories are available. The proposed SDDP scheme evaluates the cost-to-go functions only at the observed sample points, where the conditional expectations are … Read more

Markov Chain-based Policies for Multi-stage Stochastic Integer Linear Programming with an Application to Disaster Relief Logistics

We introduce an aggregation framework to address multi-stage stochastic programs with mixed-integer state variables and continuous local variables (MSILPs). Our aggregation framework imposes additional structure to the integer state variables by leveraging the information of the underlying stochastic process, which is modeled as a Markov chain (MC). We demonstrate that the aggregated MSILP can be … Read more

Modeling uncertainty processes for multi-stage optimization of strategic energy planning: An auto-regressive and Markov chain formulation

This paper deals with the modeling of stochastic processes in long-term multistage energy planning problems when little information is available on the degree of uncertainty of such processes. Starting from simple estimates of variation intervals for uncertain parameters, such as energy demands and costs, we model the temporal correlation of these parameters through autoregressive (AR) … Read more

Stochastic Dual Dynamic Programming for Optimal Power Flow Problems under Uncertainty

We propose the first computationally tractable framework to solve multi-stage stochastic optimal power flow (OPF) problems in alternating current (AC) power systems. To this end, we use recent results on dual convex semi-definite programming (SDP) relaxations of OPF problems in order to adapt the stochastic dual dynamic programming (SDDP) algorithm for problems with a Markovian … Read more

Stochastic Dual Dynamic Programming And Its Variants – A Review

We provide a tutorial-type review on stochastic dual dynamic programming (SDDP), as one of the state-of-the-art solution methods for large-scale multistage stochastic programs. Since introduced about 30 years ago for solving large-scale multistage stochastic linear programming problems in energy planning, SDDP has been applied to practical problems from several fields and is enriched by various … Read more

Distributionally Robust Stochastic Dual Dynamic Programming

We consider a multi-stage stochastic linear program that lends itself to solution by stochastic dual dynamic programming (SDDP). In this context, we consider a distributionally robust variant of the model with a finite number of realizations at each stage. Distributional robustness is with respect to the probability mass function governing these realizations. We describe a … Read more