Markov Decision Process – Optimization Online

Approximate solution of infinite-horizon risk-sensitive Markov decision processes

Published: 2026/07/15

Dynamic Programming, Other Topics Markov Decision Process, risk sensitivity, sequential decision making

Infinite-horizon risk-sensitive Markov decision processes (MDPs) under the discounted cost criterion are challenging to solve because the optimal policy may be non- stationary. Existing solution methods reformulate the problem as a continuous-state (risk-neutral) MDP and solve it using state-discretization or value function approximation. Such approaches typically lack explicit stopping conditions or error bounds. In this … Read more

Numerical Methods for Convex Multistage Stochastic Optimization

Published: 2023/04/18

Convex Optimization, Dynamic Programming, Stochastic Programming cutting planes algorithm, dynamic programming, Markov Decision Process, risk measures, Stochastic Approximation method, stochastic dual dynamic programming, stochastic optimal control, stochastic programming

Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and SOC modelling approaches. In these frameworks there are natural situations when the considered problems are convex. Classical approach to sequential optimization … Read more