Markov Decision Process – Optimization Online

Approximate solution of infinite-horizon risk-sensitive Markov decision processes

Published: 2026/07/15, Updated: 2026/08/21

Dynamic Programming, Other Topics Markov Decision Process, risk sensitivity, sequential decision making

Infinite-horizon risk-sensitive Markov decision processes (MDPs) under the discounted cost criterion are challenging to solve because the optimal policy may be nonstationary. Existing methods typically reformulate the problem as a continuous-state risk-neutral MDP and rely on state discretization or value-function approximation, often without explicit stopping conditions or error bounds. In this paper, we present approximate … Read more

Numerical Methods for Convex Multistage Stochastic Optimization

Published: 2023/04/18

Convex Optimization, Dynamic Programming, Stochastic Programming cutting planes algorithm, dynamic programming, Markov Decision Process, risk measures, Stochastic Approximation method, stochastic dual dynamic programming, stochastic optimal control, stochastic programming

Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and SOC modelling approaches. In these frameworks there are natural situations when the considered problems are convex. Classical approach to sequential optimization … Read more