Stochastic Approaches

An Algorithm for Stochastic Convex-Concave Fractional Programs with Applications to Production Efficiency and Equitable Resource Allocation

  • Cheolmin Kim
  • Sanjay Mehrotra
    • Categories Global Optimization, Global Optimization Applications, Stochastic Approaches Tags , , , ,

    We propose an algorithm to solve convex and concave fractional programs and their stochastic counterparts in a common framework. Our approach is based on a novel reformulation that involves differences of square terms in the constraints, and subsequent employment of piecewise-linear approximations of the concave terms. Using the branch-and-bound (B&B) framework, our algorithm adaptively refines … Read more

    Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates

  • Kwassi Joseph Dzahini
  • Michael Kokkolaras
  • Sébastien Le Digabel
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Stochastic Approaches Tags , , ,

    This work introduces the StoMADS-PB algorithm for constrained stochastic blackbox optimization, which is an extension of the mesh adaptive direct-search (MADS) method originally developed for deterministic blackbox optimization under general constraints. The values of the objective and constraint functions are provided by a noisy blackbox, i.e., they can only be computed with random noise whose … Read more

    An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias

  • Krishnakumar Balasubramanian
  • Stanislav Volgushev
  • Murat A. Erdogdu
  • Lu Yu
    • Categories Statistics, Stochastic Approaches, Stochastic Programming Tags , ,

    Structured non-convex learning problems, for which critical points have favorable statistical properties, arise frequently in statistical machine learning. Algorithmic convergence and statistical estimation rates are well-understood for such problems. However, quantifying the uncertainty associated with the underlying training algorithm is not well-studied in the non-convex setting. In order to address this short-coming, in this work, … Read more

    K-Adaptability in stochastic optimization

  • Enrico Malaguti
  • Michele Monaci
  • Jonas Prünte
    • Categories Branch and Cut Algorithms, Stochastic Approaches, Stochastic Programming

    We consider stochastic problems in which both the objective function and the feasible set are affected by uncertainty. We address these problems using a K-adaptability approach, in which K solutions for the underlying problem are computed before the uncertainty dissolves and afterwards the best of them can be chosen for the realised scenario. This paradigm … Read more

    Expected complexity analysis of stochastic direct-search

  • Kwassi Joseph Dzahini
    • Categories Nonlinear Optimization, Stochastic Approaches, Unconstrained Optimization Tags , , , , ,

    This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It introduces an algorithm designed to optimize differentiable objective functions $f$ whose values can only be computed through a stochastically noisy blackbox. The proposed stochastic directional direct-search (SDDS) algorithm accepts new iterates by imposing a … Read more

    A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality

  • Coralia Cartis
  • Adilet Otemissov
    • Categories Global Optimization, Stochastic Approaches Tags , , ,

    We investigate the unconstrained global optimization of functions with low effective dimensionality, that are constant along certain (unknown) linear subspaces. Extending the technique of random subspace embeddings in [Wang et al., Bayesian optimization in a billion dimensions via random embeddings. JAIR, 55(1): 361–387, 2016], we study a generic Random Embeddings for Global Optimization (REGO) framework … Read more

    A Class of Stochastic Variance Reduced Methods with an Adaptive Stepsize

  • Tiande Guo
  • Congying Han
  • Yan Liu
    • Categories Convex Optimization, Stochastic Approaches, Unconstrained Optimization Tags

    Stochastic variance reduced methods have recently surged into prominence for solving large scale optimization problems in the context of machine learning. Tan, Ma and Dai et al. first proposed the new stochastic variance reduced gradient (SVRG) method with the Barzilai-Borwein (BB) method to compute step sizes automatically, which performs well in practice. On this basis, … Read more

    Efficient global unconstrained black box optimization

  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories Stochastic Approaches, Unconstrained Optimization Tags , , , ,

    For the unconstrained optimization of black box functions, this paper introduces a new randomized algorithm called VRBBO. In practice, VRBBO matches the quality of other state-of-the-art algorithms for finding, in small and large dimensions, a local minimizer with reasonable accuracy. Although our theory guarantees only local minimizers our heuristic techniques turn VRBBO into an efficient … Read more

    Optimization of Stochastic Problems with Probability Functions via Differential Evolution

  • Bayrammyrat Myradov
    • Categories Basic Sciences Applications, Stochastic Approaches, Stochastic Programming

    Chance constrained programming, quantile/Value-at-Risk (VaR) optimization and integral quantile / Conditional Value-at-Risk (CVaR) optimization problems as Stochastic Programming Problems with Probability Functions (SPP-PF) are one of the most widely studied optimization problems in recent years. As a rule real-life SPP-PF is nonsmooth nonconvex optimization problem with complex geometry of objective function. Moreover, often it cannot … Read more

    Computational Aspects of Bayesian Solution Estimators in Stochastic Optimization

  • Gérard Cornuéjols
  • Danial Davarnia
  • Burak Kocuk
    • Categories Statistics, Stochastic Approaches, Stochastic Programming Tags , , , ,

    We study a class of stochastic programs where some of the elements in the objective function are random, and their probability distribution has unknown parameters. The goal is to find a good estimate for the optimal solution of the stochastic program using data sampled from the distribution of the random elements. We investigate two common … Read more