Multi-model Partially Observable Markov Decision Processes

We propose a new multi-model partially observable Markov decision process (MPOMDP) model to address the issue of model ambiguity in partially observable Markov decision process. Here, model ambiguity is defined as the case where there are multiple credible optimization models with the same structure but different model parameters. The proposed MPOMDP model aims to learn … Read more

Optimizing Active Surveillance for Prostate Cancer Using Partially Observable Markov Decision Processes

We describe a finite-horizon partially observable Markov decision process (POMDP) approach to optimize decisions about whether and when to perform biopsies for patients on active surveillance for prostate cancer. The objective is to minimize a weighted combination of two criteria, the number of biopsies to conduct over a patient’s lifetime and the delay in detecting … Read more

Decomposition Methods for Solving Markov Decision Processes with Multiple Models of the Parameters

We consider the problem of decision-making in Markov decision processes (MDPs) when the reward or transition probability parameters are not known with certainty. We consider an approach in which the decision-maker (DM) considers multiple models of the parameters for an MDP and wishes to find a policy that optimizes an objective function that considers the … Read more

Multi-model Markov Decision Processes

Markov decision processes (MDPs) have found success in many application areas that involve sequential decision making under uncertainty, including the evaluation and design of treatment and screening protocols for medical decision making. However, the usefulness of these models is only as good as the data used to parameterize them, and multiple competing data sources are … Read more

Branch and Price for Chance Constrained Bin Packing

This article considers two versions of the stochastic bin packing problem with chance constraints. In the first version, we formulate the problem as a two-stage stochastic integer program that considers item-to-bin allocation decisions in the context of chance constraints on total item size within the bins. Next, we describe a distributionally robust formulation of the … Read more

Robust Markov Decision Processes for Medical Treatment Decisions

Medical treatment decisions involve complex tradeoffs between the risks and benefits of various treatment options. The diversity of treatment options that patients can choose over time and uncertainties in future health outcomes, result in a difficult sequential decision making problem. Markov decision processes (MDPs) are commonly used to study medical treatment decisions; however, optimal policies … Read more

Sequential Bounding Methods for Two-Stage Stochastic Programs

Citation Alexander H. Gose Graduate Program of Operations Research, North Carolina State University, Raleigh, NC 27695, Brian T. Denton Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI 48109, October 17, 2014 (Accepted for publication to INFORMS Journal on Computing)

Fast Approximations for Online Scheduling of Outpatient Procedure Centers

This paper presents a new model for online decision making. Motivated by the health care delivery application of dynamically allocating patients to procedure rooms in outpatient procedure centers, the online stochastic extensible bin packing problem is described. The objective is to minimize the combined costs of opening procedure rooms and utilizing overtime to complete a … Read more