Morteza Kimiaei

Heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization

  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories (Mixed) Integer Nonlinear Programming, Bound-constrained Optimization, Global Optimization Tags

    This paper discusses MATRS, a new matrix adaptation trust region strategy for solving noisy derivative-free mixed-integer optimization problems with simple bounds.  MATRS repeatedly cycles through five phases, mutation, selection, recombination, trust-region, and mixed-integer in this order. But if in the mutation phase a new best point (point with lowest inexact function value among all evaluated … Read more

    An active set method for bound-constrained optimization

  • Arnold Neumaier
  • Behzad Azmi
  • Morteza Kimiaei
    • Categories Bound-constrained Optimization, Nonlinear Optimization Tags

    In this paper, a class of algorithms is developed for bound-constrained optimization. The new scheme uses the gradient-free line search along bent search paths. Unlike traditional algorithms for bound-constrained optimization, our algorithm ensures that the reduced gradient becomes arbitrarily small. It is also proved that all strongly active variables are found and fixed after finitely … Read more

    A subspace inertial method for derivative-free nonlinear monotone equations

  • Morteza Kimiaei
  • Abdulkarim Hassan Ibrahim
  • Susan Ghaderi
    • Categories Nonlinear Systems and Least-Squares Tags

    We introduce SILSA, a subspace inertial line search algorithm, for finding solutions of nonlinear monotone equations (NME). At each iteration, a new point is generated in a subspace generated by the previous points. Of all finite points forming the subspace, a point with the largest residual norm is replaced by the new point to update … Read more

    Effective matrix adaptation strategy for noisy derivative-free optimization

  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories Global Optimization Applications, Nonlinear Optimization, Optimization Software Benchmark Tags

    In this paper, we introduce a new effective matrix adaptation evolution strategy (MADFO) for noisy derivative-free optimization problems. Like every MAES solver, MADFO consists of three phases: mutation, selection and recombination. MADFO improves the mutation phase by generating good step sizes, neither too small nor too large, that increase the probability of selecting mutation points … Read more

    New subspace method for unconstrained derivative-free optimization

  • Morteza Kimiaei
  • Arnold Neumaier
  • Parvaneh Faramarzi
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags

    This paper defines an efficient subspace method, called SSDFO, for unconstrained derivative-free optimization problems where the gradients of the objective function are Lipschitz continuous but only exact function values are available. SSDFO employs line searches along directions constructed on the basis of quadratic models. These approximate the objective function in a subspace spanned by some … Read more

    Globally linearly convergent nonlinear conjugate gradients without Wolfe line search

  • Arnold Neumaier
  • Morteza Kimiaei
  • Behzad Azmi
    • Categories Nonlinear Optimization Tags

    This paper introduces a new nonlinear conjugate gradient (CG) method using an efficient gradient-free line search method. Unless function values diverge to $-\infty$, global convergence to a stationary point is proved for continuously differentiable objective functions with Lipschitz continuous gradient, and global linear convergence if this stationary point is a strong local minimizer. The $n$-iterations … Read more

    An improvement of the Goldstein line search

  • Arnold Neumaier
  • Morteza Kimiaei
    • Categories Nonlinear Optimization Tags , ,

    This paper introduces CLS, a new line search along an arbitrary smooth search path, that starts at the current iterate tangentially to a descent direction. Like the Goldstein line search and unlike the Wolfe line search, the new line search uses, beyond the gradient at the current iterate, only function values. Using this line search … Read more

    Efficient composite heuristics for integer bound constrained noisy optimization

  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories Bound-constrained Optimization, Integer Programming, Optimization Software Benchmark Tags , , , ,

    This paper discusses a composite algorithm for bound constrained noisy derivative-free optimization problems with integer variables. This algorithm is an integer variant of the matrix adaptation evolution strategy. An integer derivative-free line search strategy along affine scaling matrix directions is used to generate candidate points. Each affine scaling matrix direction is a product of the … Read more

    LMBOPT — a limited memory method for bound-constrained optimization

  • Behzad Azmi
  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories Nonlinear Optimization Tags , , ,

    Recently, Neumaier and Azmi gave a comprehensive convergence theory for a generic algorithm for bound constrained optimization problems with a continuously differentiable objective function. The algorithm combines an active set strategy with a gradient-free line search CLS along a piecewise linear search path defined by directions chosen to reduce zigzagging. This paper describes LMBOPT, an … Read more

    An improved randomized algorithm with noise level tuning for large-scale noisy unconstrained DFO problems

  • Morteza Kimiaei
    • Categories Optimization Software Benchmark, Stochastic Programming, Unconstrained Optimization Tags , , , ,

    In this paper, a new randomized solver (called VRDFON) for noisy unconstrained derivative-free optimization (DFO) problems is discussed. Complexity result in the presence of noise for nonconvex functions is studied. Two effective ingredients of VRDFON are an improved derivative-free line search algorithm with many heuristic enhancements and quadratic models in adaptively determined subspaces. Numerical results … Read more