Morteza Kimiaei – Optimization Online

Applications - OR and Management Sciences, Integer Programming, Optimization in Data Science backward induction, combinatorial optimization, disjunctive programming, global optimization, machine learning, neural networks, probabilistic programming, reinforcement learning, submodular optimization

Combinatorial decision problems lie at the intersection of Operations Research (OR) and Artificial Intelligence (AI), encompassing structured optimization tasks such as submodular selection, dynamic programming, planning, and scheduling. These problems exhibit exponential growth in decision complexity, driven by interdependent choices coupled through logical, temporal, and resource constraints. Classical optimization frameworks—including integer programming, submodular optimization, and … Read more

Machine Learning Algorithms for Improving Black Box Optimization Solvers

Published: 2025/09/29

Global Optimization, Nonlinear Optimization, Optimization in Data Science black-box optimization, machine learning, Meta-Black-Box Optimization, reinforcement learning, robust optimization, surrogate models

Black-box optimization (BBO) addresses problems where objectives are accessible only through costly queries without gradients or explicit structure. Classical derivative-free methods—line search, direct search, and model-based solvers such as Bayesian optimization—form the backbone of BBO, yet often struggle in high-dimensional, noisy, or mixed-integer settings. Recent advances use machine learning (ML) and reinforcement learning (RL) to … Read more

Restarting nonlinear conjugate gradient methods

Published: 2025/01/14

Saman Babaie-Kafaki

Nonlinear Optimization global convergence, local convergence, nonlinear conjugate gradient method, nonlinear optimization, restart procedure

Mahsa Yousefi

In unconstrained optimization, due to the nonlinearity of the objective function or rounding errors in finite precision arithmetic, it can happen that NaN or infinite step sizes appear in the nonlinear conjugate gradient (NCG) method, or otherwise the step violates the sufficient descent condition (SDC). In this case the conjugate gradient (CG) direction must often … Read more

A class of diagonal quasi-Newton penalty decomposition algorithms for sparse bound-constrained nonconvex optimization

Published: 2025/01/14

Ahmad Mousavi

Saman Babaie-Kafaki

Bound-constrained Optimization, Constrained Nonlinear Optimization, Quadratic Programming Cardinality bound-constrained optimization, diagonal quasi-Newton method, global convergence, penalty decomposition method

This paper discusses an improved quasi-Newton penalty decomposition algorithm for the cardinality bound-constrained optimization problems whose simple bounds on the variables are assumed to be finite. Until an approximate stationary point is found, this algorithm approximates the solutions of a sequence of penalty subproblems by a two-block decomposition scheme. This scheme finds an approximate solution … Read more

Heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization

Published: 2023/04/20, Updated: 2025/03/24

(Mixed) Integer Nonlinear Programming, Bound-constrained Optimization, Global Optimization Mixed-integer; derivative-free noisy optimization; heuristic optimization; randomized optimization; evolution strategy; trust region; line search

This paper introduces MATRS, a novel matrix adaptation trust-region strategy designed to solve noisy derivative-free mixed-integer optimization problems with simple bounds in low dimensions. MATRS operates through a repeated cycle of five phases: mutation, selection, recombination, trust-region, and mixed-integer, executed in this sequence. But if in the mutation phase a new best point (the point … Read more

An active set method for bound-constrained optimization

Published: 2023/04/14

Behzad Azmi

Bound-constrained Optimization, Nonlinear Optimization Bound constrained optimization; active set strategy; line search method; global convergence

In this paper, a class of algorithms is developed for bound-constrained optimization. The new scheme uses the gradient-free line search along bent search paths. Unlike traditional algorithms for bound-constrained optimization, our algorithm ensures that the reduced gradient becomes arbitrarily small. It is also proved that all strongly active variables are found and fixed after finitely … Read more

A subspace inertial method for derivative-free nonlinear monotone equations

Published: 2023/04/04, Updated: 2023/09/11

Abdulkarim Hassan Ibrahim

Susan Ghaderi

Nonlinear Systems and Least-Squares Monotone equations; derivative-free optimization; inertial technique; global convergence; complexity result

We introduce SILSA, a subspace inertial line search algorithm, for finding solutions of nonlinear monotone equations (NME). At each iteration, a new point is generated in a subspace generated by the previous points. Of all finite points forming the subspace, a point with the largest residual norm is replaced by the new point to update … Read more

Effective matrix adaptation strategy for noisy derivative-free optimization

Published: 2023/03/18, Updated: 2024/03/16

Global Optimization Applications, Nonlinear Optimization, Optimization Software Benchmark Noisy derivative-free optimization; evolution strategy; heuristic optimization; stochastic optimization

In this paper, we introduce a new effective matrix adaptation evolution strategy (MADFO) for noisy derivative-free optimization problems. Like every MAES solver, MADFO consists of three phases: mutation, selection and recombination. MADFO improves the mutation phase by generating good step sizes, neither too small nor too large, that increase the probability of selecting mutation points … Read more

New subspace method for unconstrained derivative-free optimization

Published: 2022/12/24