Reinforcement Learning via Parametric Cost Function Approximation for Multistage Stochastic Programming

The most common approaches for solving stochastic resource allocation problems in the research literature is to either use value functions (“dynamic programming”) or scenario trees (“stochastic programming”) to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier … Read more

Nonstationary Direct Policy Search for Risk-Averse Stochastic Optimization

This paper presents an approach to non-stationary policy search for finite-horizon, discrete-time Markovian decision problems with large state spaces, constrained action sets, and a risk-sensitive optimality criterion. The methodology relies on modeling time variant policy parameters by a non-parametric response surface model for an indirect parametrized policy motivated by the Bellman equation. Through the interpolating … Read more

A Non-Parametric Structural Hybrid Modeling Approach for Electricity Prices

We develop a stochastic model of zonal/regional electricity prices, designed to reflect information in fuel forward curves and aggregated capacity and load as well as zonal or regional price spreads. We use a nonparametric model of the supply stack that captures heat rates and fuel prices for all generators in the market operator territory, combined … Read more