sequential decision making – Optimization Online

Approximate solution of infinite-horizon risk-sensitive Markov decision processes

Published: 2026/07/15

Dynamic Programming, Other Topics Markov Decision Process, risk sensitivity, sequential decision making

Infinite-horizon risk-sensitive Markov decision processes (MDPs) under the discounted cost criterion are challenging to solve because the optimal policy may be non- stationary. Existing solution methods reformulate the problem as a continuous-state (risk-neutral) MDP and solve it using state-discretization or value function approximation. Such approaches typically lack explicit stopping conditions or error bounds. In this … Read more

Deep Learning for Sequential Decision Making under Uncertainty: Foundations, Frameworks, and Frontiers

Published: 2026/04/12

I. Esra Buyuktahtakin Toy

Data Science Algorithms, Optimization in Data Science, Stochastic Programming decision making under uncertainty, Decision-focused learning, deep learning, learning-to-optimize, Machine learning and optimization, markov decision processes, multi-stage stochastic programming, operations research, optimization under uncertainty, reinforcement learning, sequential decision making, stochastic optimization

Artificial intelligence (AI) is moving increasingly beyond prediction to support decisions in complex, uncertain, and dynamic environments. This shift creates a natural intersection with operations research and management sciences (OR/MS), which have long offered conceptual and methodological foundations for sequential decision-making under uncertainty. At the same time, recent advances in deep learning, including feedforward neural … Read more

Adaptive Two-stage Stochastic Programming with an Application to Capacity Expansion Planning

Published: 2019/06/08

Applications - OR and Management Sciences, Stochastic Programming approximation algorithms, capacity expansion planning, energy, mixed-integer programming, sequential decision making, stochastic programming

Multi-stage stochastic programming is a well-established framework for sequential decision making under uncertainty by seeking policies that are fully adapted to the uncertainty. Often, e.g. due to contractual constraints, such flexible and adaptive policies are not desirable, and the decision maker may need to commit to a set of actions for a certain number of … Read more