approximate dynamic programming

A Survey on the Applications of Stochastic Dual Dynamic Programming and its Variants

  • Bonn Kleiford Seranilla
  • Nils Löhndorf
    • Categories Dynamic Programming, Integer Programming, Stochastic Programming Tags , , ,

    Stochastic Dual Dynamic Programming (SDDP) is widely recognized as the predominant methodology for solving large-scale multistage stochastic linear programming (MSLP) problems. This paper aims to contribute to the extant literature by conducting a comprehensive survey of the literature on SDDP within the realm of practical applications. We systematically identify and analyze the various domains where … Read more

    Guaranteed bounds for optimal stopping problems using kernel-based non-asymptotic uniform confidence bands

  • Martin Glanzer
  • Sebastian Maier
  • Georg Ch. Pflug
    • Categories Dynamic Programming, Optimization in Data Science, Stochastic Programming Tags , , , ,

    In this paper, we introduce an approach for obtaining probabilistically guaranteed upper and lower bounds on the true optimal value of stopping problems. Bounds of existing simulation-and-regression approaches, such as those based on least squares Monte Carlo and information relaxation, are stochastic in nature and therefore do not come with a finite sample guarantee. Our … Read more

    Multistage Stochastic Facility Location under Facility Disruption Uncertainty

  • Bonn Kleiford Seranilla
  • Nils Löhndorf
    • Categories Production and Logistics, Stochastic Programming, Supply Chain Management Tags , , ,

    We consider a multistage variant of the classical stochastic capacitated facility location problem under facility disruption uncertainty. Two solution algorithms for this problem class are presented: (1) stochastic dual dynamic integer programming (SDDiP), the state-of-the-art algorithm for solving multistage stochastic integer programs, and (2) shadow price approximation (SPA), an algorithm utilizing trained parameters of the … Read more

    Optimizing Vaccine Distribution in Developing Countries under Natural Disaster Risk

  • Bonn Kleiford Seranilla
  • Nils Löhndorf
    • Categories (Mixed) Integer Linear Programming, Dynamic Programming, Stochastic Programming Tags , , ,

    For many developing countries, COVID-19 vaccination roll-out programs are not only slow but vaccination centers are also exposed to the risk of natural disaster, like flooding, which may slow down vaccination progress even further. Policy-makers in developing countries therefore seek to implement strategies that hedge against distribution risk in order for vaccination campaigns to run … Read more

    Randomized Policy Optimization for Optimal Stopping

  • Xinyi Guan
  • Velibor Mišić
    • Categories Dynamic Programming, Finance and Economics, Unconstrained Optimization Tags , , , ,

    Optimal stopping is the problem of determining when to stop a stochastic system in order to maximize reward, which is of practical importance in domains such as finance, operations management and healthcare. Existing methods for high-dimensional optimal stopping that are popular in practice produce deterministic linear policies — policies that deterministically stop based on the … Read more

    Approximate Dynamic Programming for Crowd-shipping with In-store Customers

  • Kianoush Mousavi
  • Merve Bodur
  • Mucahit Cevik
  • Matthew J. Roorda
    • Categories Civil and Environmental Engineering, Dynamic Programming, Transportation Tags , , , ,

    Crowd-shipping has gained significant attention as a last-mile delivery option over the recent years. In this study, we propose a variant of dynamic crowd-shipping model with in-store customers as crowd-shippers to deliver online orders within few hours. We formulate the problem as a Markov decision process and develop an approximate dynamic programming (ADP) policy using … Read more

    Dynamic Repositioning in Free-Floating Bike Sharing Systems Using Approximate Dynamic Programming

  • Xue Luo
  • Li Li
  • Lei Zhao
    • Categories Applications - OR and Management Sciences, Transportation Tags , , , ,

    In bike sharing systems, the spatiotemporal imbalance of bike flows leads to shortages of bikes in some areas and overages in some others, depending on the time of the day, resulting in user dissatisfaction. Repositioning needs to be performed timely to deal with the spatiotemporal imbalance and to meet customer demand in time. In this … Read more

    Random-Sampling Multipath Hypothesis Propagation for Cost Approximation in Long-Horizon Optimal Control

  • Hans D Mittelmann
  • Shankarachary Ragi
    • Categories Applications - Science and Engineering, Control Applications Tags , , ,

    In this paper, we develop a Monte-Carlo based heuristic approach to approximate the objective function in long horizon optimal control problems. In this approach, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the weighted average of the costs along all the trajectories. … Read more

    Dynamic optimization for airline maintenance operations

  • Felipe Delgado
  • Mathias A. Klapp
  • Carlos Lagos
    • Categories Airline Optimization Tags , , ,

    The occurrence of unexpected aircraft maintenance tasks can produce expensive changes in an airline’s operation. When it comes to critical tasks, it might even cancel programmed flights. Despite of it, the challenge of scheduling aircraft maintenance operations under uncertainty has received limited attention in the scientific literature. We study a dynamic airline maintenance scheduling problem, … Read more

    A Dynamic Mobile Production Capacity and Inventory Control Problem

  • Alan L. Erera
  • Satya S. Malladi
  • Chelsea C. White III
    • Categories Applications - OR and Management Sciences, Dynamic Programming, Production and Logistics Tags , , ,

    We analyze a problem of dynamic logistics planning given uncertain demands for a multi-location production-inventory system with transportable modular production capacity. In such systems, production modules provide capacity, and can be moved from one location to another to produce stock and satisfy demand. We formulate a dynamic programming model for a planning problem that considers … Read more