stochastic optimization

An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization

  • Arda Aytekin
  • Hamid Reza Feyzmahdavian
  • Mikael Johansson
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities or different communication delays), synchronous and cyclic operations are inefficient since they will leave workers idle waiting … Read more

    Stochastic Optimization using a Trust-Region Method and Random Models

  • Ruobing Chen
  • Matt Menickelly
  • Katya Scheinberg
    • Categories Stochastic Programming, Unconstrained Optimization Tags , , ,

    In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function $f(x)$, obtained from stochastic observations of the function or its gradient. Our method also utilizes estimates of function values to gauge progress that is being made. The convergence analysis … Read more

    From Predictive to Prescriptive Analytics

  • Dimitris Bertsimas
  • Nathan Kallus
    • Categories Applications - OR and Management Sciences Tags , , , , ,

    In this paper, we combine ideas from machine learning (ML) and operations research and management science (OR/MS) in developing a framework, along with specific methods, for using data to prescribe optimal decisions in OR/MS problems. In a departure from other work on data-driven optimization and reflecting our practical experience with the data available in applications … Read more

    Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization

  • Wei Liu
  • Shiqian Ma
  • Xiao Wang
    • Categories Nonlinear Optimization, Stochastic Programming Tags , , , , , ,

    In this paper we study stochastic quasi-Newton methods for nonconvex stochastic optimization, where we assume that only stochastic information of the gradients of the objective function is available via a stochastic first-order oracle (SFO). Firstly, we propose a general framework of stochastic quasi-Newton methods for solving nonconvex stochastic optimization. The proposed framework extends the classic … Read more

    Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functions

  • Ethan Fang
  • Mengdi Wang
  • Han Liu
    • Categories Convex Optimization, Statistics, Stochastic Programming Tags , , , ,

    Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \E_v\[f_v\big(\E_w [g_w(x)]\big) \]$. In order to solve this stochastic composition problem, we propose a class … Read more

    Randomized First-order Methods for Saddle Point Optimization

  • Cong D. Dang
  • Guanghui Lan
    • Categories Convex and Nonsmooth Optimization Tags , , , ,

    In this paper, we present novel randomized algorithms for solving saddle point problems whose dual feasible region is a direct product of many convex sets. Our algorithms can achieve ${\cal O}(1/N)$ rate of convergence by solving only one dual subproblem at each iteration. Our algorithms can also achieve ${\cal O}(1/N^2)$ rate of convergence if a … Read more

    Robust Unit Commitment with Dispatchable Wind: An LP Reformulation of the Second Stage

  • Germán Morales-España
    • Categories Applications - Science and Engineering, Robust Optimization Tags , , ,

    Abstract— The increasing penetration of uncertain generation such as wind and solar in power systems imposes new challenges to the Unit Commitment (UC) problem, one of the most critical tasks in power systems operations. The two most common approaches to address these challenges — stochastic and robust optimization — have drawbacks that prevent or restrict their … Read more

    Dynamic Generation of Scenario Trees

  • Georg Ch. Pflug
  • Alois Pichler
    • Categories Stochastic Programming Tags , ,

    We present new algorithms for the dynamic generation of scenario trees for multistage stochastic optimization. The different methods described are based on random vectors, which are drawn from conditional distributions given the past and on sample trajectories. The structure of the tree is not determined beforehand, but dynamically adapted to meet a distance criterion, which … Read more

    Multiperiod Multiproduct Advertising Budgeting: Stochastic Optimization Modeling

  • Cesar Beltran-Royo
  • Laureano F. Escudero
  • Huizhen Zhang
    • Categories Marketing Tags , , , ,

    We propose a stochastic optimization model for the Multiperiod Multiproduct Advertising Budgeting problem, so that the expected profit of the advertising investment is maximized. The model is a convex optimization problem that can readily be solved by plain use of standard optimization software. It has been tested for planning a realistic advertising campaign. In our … Read more

    Symmetric confidence regions and confidence intervals for normal map formulations of stochastic variational inequalities

  • Shu Lu
    • Categories Complementarity and Variational Inequalities, Stochastic Programming Tags , , , , ,

    Stochastic variational inequalities (SVI) model a large class of equilibrium problems subject to data uncertainty, and are closely related to stochastic optimization problems. The SVI solution is usually estimated by a solution to a sample average approximation (SAA) problem. This paper considers the normal map formulation of an SVI, and proposes a method to build … Read more