Expected complexity analysis of stochastic direct-search

This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It introduces an algorithm designed to optimize differentiable objective functions $f$ whose values can only be computed through a stochastically noisy blackbox. The proposed stochastic directional direct-search (SDDS) algorithm accepts new iterates by imposing a … Read more

A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality

We investigate the unconstrained global optimization of functions with low effective dimensionality, that are constant along certain (unknown) linear subspaces. Extending the technique of random subspace embeddings in [Wang et al., Bayesian optimization in a billion dimensions via random embeddings. JAIR, 55(1): 361–387, 2016], we study a generic Random Embeddings for Global Optimization (REGO) framework … Read more

A Class of Stochastic Variance Reduced Methods with an Adaptive Stepsize

Stochastic variance reduced methods have recently surged into prominence for solving large scale optimization problems in the context of machine learning. Tan, Ma and Dai et al. first proposed the new stochastic variance reduced gradient (SVRG) method with the Barzilai-Borwein (BB) method to compute step sizes automatically, which performs well in practice. On this basis, … Read more

Efficient global unconstrained black box optimization

For the unconstrained optimization of black box functions, this paper introduces a new randomized algorithm called VRBBO. In practice, VRBBO matches the quality of other state-of-the-art algorithms for finding, in small and large dimensions, a local minimizer with reasonable accuracy. Although our theory guarantees only local minimizers our heuristic techniques turn VRBBO into an efficient … Read more

Optimization of Stochastic Problems with Probability Functions via Differential Evolution

Chance constrained programming, quantile/Value-at-Risk (VaR) optimization and integral quantile / Conditional Value-at-Risk (CVaR) optimization problems as Stochastic Programming Problems with Probability Functions (SPP-PF) are one of the most widely studied optimization problems in recent years. As a rule real-life SPP-PF is nonsmooth nonconvex optimization problem with complex geometry of objective function. Moreover, often it cannot … Read more

Computational Aspects of Bayesian Solution Estimators in Stochastic Optimization

We study a class of stochastic programs where some of the elements in the objective function are random, and their probability distribution has unknown parameters. The goal is to find a good estimate for the optimal solution of the stochastic program using data sampled from the distribution of the random elements. We investigate two common … Read more

Barzilai-Borwein Step Size for Stochastic Gradient Descent

One of the major issues in stochastic gradient descent (SGD) methods is how to choose an appropriate step size while running the algorithm. Since the traditional line search technique does not apply for stochastic optimization algorithms, the common practice in SGD is either to use a diminishing step size, or to tune a fixed step … Read more

Cutting Box Strategy: an algorithmic framework for improving metaheuristics for continuous global optimization

In this work, we present a new framework to increase effectiveness of metaheuristics in seeking good solutions for the general nonlinear optimization problem, called Cutting Box Strategy (CBS). CBS is based on progressive reduction of the search space through the use of intelligent multi-starts, where solutions already obtained cannot be revisited by the adopted metaheuristic. … Read more

A mean-risk MINLP for transportation network protection

This paper focuses on transportation network protection to hedge against extreme events such as earthquakes. Traditional two-stage stochastic programming has been widely adopted to obtain solutions under a risk-neutral preference through the use of expectations in the recourse function. In reality, decision makers hold different risk preferences. We develop a mean-risk two-stage stochastic programming model … Read more

Computational investigation of simple memetic approaches for continuous global optimization

In [Locatelli et al., 2014] a memetic approach, called MDE, for the solution of continuous global optimization problems, has been introduced and proved to be quite efficient in spite of its simplicity. In this paper we computationally investigate some variants of MDE. The investigation reveals that the best variant of MDE usually outperforms MDE itself, … Read more