Dynamic Programming

Multi-Echelon Inventory Management for a Non-Stationary Capacitated Distribution Network

  • Alvaro Maggiar
  • Irene Song
  • Alp Muharremoglu
    • Categories Applications - OR and Management Sciences, Dynamic Programming, Supply Chain Management Tags , ,

    We present an inventory management solution for a non-stationary capacitated multi-echelon distribution network involving thousands of products. Assuming backlogged sales, we revisit and leverage the seminal multi-echelon inventory management results in the literature to establish the structural properties of the problem, and derive an efficient and practical solution method. In particular, we describe how the … Read more

    Convergence of Trajectory Following Dynamic Programming algorithms for multistage stochastic problems without finite support assumptions

  • Maël Forcier
  • Vincent Leclère
    • Categories Dynamic Programming, Stochastic Programming Tags , ,

    We introduce a class of algorithms, called Trajectory Following Dynamic Programming (TFDP) algorithms, that iteratively refines approximation of cost-to-go functions of multistage stochastic problems with independent random variables. This framework encompasses most variants of the Stochastic Dual Dynamic Programming algorithm. Leveraging a Lipschitz assumption on the expected cost-to-go functions, we provide a new convergence and … Read more

    Solving large-scale unit-commitment problems using dual dynamic programming and open-source solvers

  • Bruno Colonetti
  • Samuel Brito
  • Erlon Finardi
  • Victor M. Zavala
  • Lucas Picarelli
    • Categories 0-1 Programming, Applications - Science and Engineering, Dynamic Programming Tags , , ,

    The astonishing dimensions and complexity of power systems render them impossible to be managed without the help of cutting-edge software. Due to a lack of scalable, reliable and well documented free and open-source solutions, system operators, regulators, and government agencies often rely on proprietary software to provide them information that ultimately will be used to … Read more

    Robust Phi-Divergence MDPs

  • Chin Pang Ho
  • Marek Petrik
  • Wolfram Wiesemann
    • Categories Dynamic Programming, Robust Optimization Tags , ,

    In recent years, robust Markov decision processes (MDPs) have emerged as a prominent modeling framework for dynamic decision problems affected by uncertainty. In contrast to classical MDPs, which only account for stochasticity by modeling the dynamics through a stochastic process with a known transition kernel, robust MDPs additionally account for ambiguity by optimizing in view … Read more

    Randomized Policy Optimization for Optimal Stopping

  • Xinyi Guan
  • Velibor Mišić
    • Categories Dynamic Programming, Finance and Economics, Unconstrained Optimization Tags , , , ,

    Optimal stopping is the problem of determining when to stop a stochastic system in order to maximize reward, which is of practical importance in domains such as finance, operations management and healthcare. Existing methods for high-dimensional optimal stopping that are popular in practice produce deterministic linear policies — policies that deterministically stop based on the … Read more

    Parallel Dual Dynamic Integer Programming for Large-Scale Hydrothermal Unit-Commitment

  • Bruno Colonetti
  • Erlon Finardi
  • Victor M. Zavala
  • Samuel Brito
    • Categories Dynamic Programming, Parallel Algorithms, Scheduling Tags , , ,

    Unit commitment has been at the center of power system operation for well over 50 years. Yet, this problem cannot be considered solved due to its size and complexity. Today, operators rely on off-the-shelf optimization solvers to tackle this challenging problem, and often resort to simplifications to make the problem more tractable and solvable in … Read more

    Discrete Optimal Transport with Independent Marginals is #P-Hard

  • Bahar Tașkesen
  • Soroosh Shafieezadeh-Abadeh
  • Daniel Kuhn
  • Karthik Natarajan
    • Categories Dynamic Programming, Linear Programming, Stochastic Programming

    We study the computational complexity of the optimal transport problem that evaluates the Wasserstein distance between the distributions of two K-dimensional discrete random vectors. The best known algorithms for this problem run in polynomial time in the maximum of the number of atoms of the two distributions. However, if the components of either random vector … Read more

    Distributionally Robust Modeling of Optimal Control

  • Alexander Shapiro
    • Categories Control Applications, Dynamic Programming, Stochastic Programming Tags , , , , , , ,

    The aim of this paper is to formulate several questions related to distributionally robust Stochastic Optimal Control modeling. As an example, the distributionally robust counterpart of the classical inventory model is discussed in details. Finite and infinite horizon stationary settings are considered. ArticleDownload View PDF

    ADMM-based Unit and Time Decomposition for Price Arbitrage by Cooperative Price-Maker Electricity Storage Units

  • Miguel F. Anjos
  • James Cruise
  • Albert Solà Vilalta
    • Categories Convex Optimization, Dynamic Programming, Energy Tags , , , , , , ,

    Decarbonization via the integration of renewables poses significant challenges for electric power systems, but also creates new market opportunities. Electric energy storage can take advantage of these opportunities while providing flexibility to power systems that can help address these challenges. We propose a solution method for the optimal control of multiple price-maker electric energy storage … Read more

    Data-Driven Ranges of Near-Optimal Actions for Finite Markov Decision Processes

  • Wesley Marrero
  • Mariel S. Lavieri
  • Ambuj Tewari
  • Jeremy Sussman
  • Rodney A. Hayward
    • Categories Applications - OR and Management Sciences, Applications - Science and Engineering, Dynamic Programming Tags , , , , ,

    Markov decision process (MDP) models have been used to obtain non-stationary optimal decision rules in various applications, such as treatment planning in medical decision making. However, in practice, decision makers may prefer other strategies that are not statistically different from the optimal decision rules. To benefit from the decision makers’ expertise and provide flexibility in … Read more